Test of Plagiarism Detection Software

I am currently redoing my E-Learning-Unit on plagiarism, "Fremde Federn Finden" (in German). As part of the work I am repeating the test that I conducted on plagiarism detection software in 2004. Then I used 10 papers that I wrote myself with a known amount of plagiarism / originality to see how well the software measured up. It was not a pretty sight - often flipping a coin was just as effective.

For the repeat of the experiment I have 10 more papers and will be conducting tests over the summer of the following products from various countries:

• CopyCatch Investigator - UK - http://www.copycatchgold.com/index.html
• Doc Cop - Australian - http://www.doccop.com/
• Docoloc - German - http://www.docoloc.de/
• Ephorus - Dutch - http://www.ephorus.nl/
  
• Eve2 - Unknown, no address given - http://www.canexus.com/
• MyDropBox/ SafeAssignment - English - http://www.mydropbox.com/
• Plagiarism Finder - German - http://www.plagiarism-finder.de
• StrikePlagiarism - Polish - http://strikeplagiarism.com/
• TextGuard - German - http://www.TextGuard.de
• turnitin - USA - http://turnitin.com/static/index.html (iThenticate seems to be the same software but targeted to businesses and not schools - http://www.ithenticate.com/static/home.html)
• Urkund - Swedish - http://www.urkund.com/

If any of my readers know of any other software, please let me know! If you are a software producer of plagiarism detection software, please contact me so that I can include you in the test.

The results will be published online in September 2007.

Stolen from the Wikipedia

Many pupils and students think that the Wikipedia is there to let them hand in term papers very easily. The German Wikipedian Avatar has started a collection of stories about sad happenings to people getting caught in a plagiarism.

They include a student trying to cover his tracks by changing the Wikipedia article and some links to discovered plagiarisms by journalists and publishers of articles in the Wikipedia. There are some from both the English and the German WP, more are welcome.

Whistleblowing hurting your career

Janet Stemwedel's blog article "The price of calling out misconduct" about an article in Science describing the situation of some graduate students who blew the whistle on their advisor is very depressing.

Here we have courageous grad students working to better science by letting the community know that their advisor is making up data, and they end up paying the bill. Janet quotes a fascinating paper from Science and Engineering Ethics by C. K. Gunsalus's excellent paper "How to Blow the Whistle and Still Have a Career Afterwards." I think I will print out a copy for my administration, as the next article is on how to avoid whistleblowing problems in your institution.

Very important is finding both sides to an issue, as many people often misunderstand what is actually happening and start threatening lawsuits without being sure. On the other hand, if it really is misconduct, then you have to do something.

Plagiarism increased four-fold in Sweden

Svenska Dagbladet reports that the number of reported plagiarisms at the University of Södertorn, near Stockholm, has increased four-fold during the last five years - and that with the total number of students decreasing.

They cite a report put out by the Hogskoleverket, the government university agency, which will be published next month. The report finds more plagiarism in term paper writing than in cheating on exams. Under the Swedish system, students who are caught cheating or plagiarizing are brought before a board, the disciplinnämnden, which decides if punishment should be meted out. Punishment is suspension from school for a period of up to 6 months - usually pronounced just before exam time, so that the deliquient cannot take some exams.

Taking exams in Sweden is vital - if you pass 75% of the credits of your first year at college, you can get funding for the next year, and so on. So there is quite an incentive to get those 75% credits.

The university uses a so-called plagiarism detection software for checking term papers, the article does not mention which one. Out of 15 400 submitted papers last year there were 36 suspensions meted out. 2003 there were only 10 suspensions pronounced. That is a quota of 0.23 % - and far, far below what teachers report when they hand-check term papers. There are reports accumulating pointing to figures more in the 10-30% range.

The report continues that 3 of the suspended students took the university to court - and won their suspension rescinded. That looks to me like an 8% false positive rate in the software. Perhaps they need to look hard at their software, or find other methods - like using search machines - for assessing this problem.

You can't solve social problems with software - and most certainly not with software that is this bad.