It was interesting to see that they tested some of the same systems we did, often having similar experiences with the systems although they only tested a few cases using computing-related texts. They did, however, have a very fine-grained points system looking at things such as legal issues and the technical basis for the server systems and the presence of licenses for using search machines such as Google or Yahoo.
Not surprisingly, Turnitin comes out on top. Why do I say "not surprisingly"? Well, JISC seems tightly entwined with the NorthumbriaLearning and the latter are the European re-sellers for Turnitin as well as a resource center for teachers. I am not quite clear on how close these two organizations are.
JISC did give this survey to an outside person to conduct, and had an academic advisory board look at the evaluation questions and suggest products to test. But the appendix entry on turnitin is a glowing sales document that avoids all of the issues with Turnitin (such as being overeager to store copies of papers in their database), whereas the others are more apt to have problems noted - problems that we, too, had in many cases.
The survey is still a very valuable collection of data - all the more so because they used questionnaires to elicit more data (or more refusals to give information) from the various companies. I am just curious as to how independent the study really is.
Update October 5, 2007: William Murray from NorthumbriaLearning has sent me this clarification of the relationship JISC/NL. Thanks, William, glad to post it!
"The relationship between JISC (the government funded Joint Information Systems Committee in the UK) and NL needs explaining. The confusion occurs because all JISC services are branded with JISC in front of them. We run JISC-PAS not JISC!
Turnitin won a national tender in 2002 put out by JISC to run a national detection service in the UK and Northumbria University (our original parent company) won a second national tender for the advisory service JISC-PAS (Plagiarism Advisory Service) that supports it.
We (Northumbria Learning) have been managing JISC-PAS and reselling Turnitin ever since with JISC’s endorsement. JISC wanted an independent survey to reaffirm (or otherwise) their support for their original choice of detection solution in 2002. NCC Group Ltd were chosen because they are independent of NL and JISC-PAS.
Within JISC-PAS our primary aim is to encourage holistic change within institutions through better information literacy, better course design, better research practice and better teaching of core skills. We happen to think that solutions like Turnitin provide the ‘ah-ha’ moment (Jude Caroll’s term not mine) that focuses the minds of all concerned. In my view detection is a change agent for better practice (I taught informatics at Northumbria University for ten years so I think this is a good thing. I would have loved to be able to use Turnitin, our class sizes were huge 300+ in some cases which made consistency in marking a nightmare). But specifically to address your points:
JISC are not entwined with Northumbria Learning, we run the JISC-PAS and Turnitin service on behalf of JISC.
NCC group ran an independent survey
NCC group allowed *ALL* providers to vet *ALL* the information in their report and agree it as factually correct before publication.
All providers were given the opportunity to improve their scores prior to publication
The extent to which they contributed ‘sales’ information was entirely up to the companies concerned.
Its aim was to identify which system could be deployed enterprise wide, with high volumes of through put and used on a national scale *in the
Having a central database was on of the reasons Turnitin was selected by JISC. This is why (in this context) it was not a flaw."
Posts
No comments:
Post a Comment
Please note that I moderate comments. Any comments that I consider unscientific will not be published.