Test of Plagiarism Detection Software

It's finished, it's published. We worked feverishly right up to the wire. On Sept. 26 we sent copies of the preliminary reports (they were still in line for some language polishing) to the companies tested, so that they could prepare a statement, if they so chose.

We held a press conference this afternoon, cutting over to the new version of the plagiarism portal and the E-Learning unit on plagiarism detection ("Fremde Federn Finden", in German) at the start of the conference. We had 5 reporters in attendance and many who requested virtual press materials. The online magazine "Spiegel Online" had requested that we write a summary article for them, so we just cut out sleep for a few days in order to get it done.

We have had a lot of interest from reports and of course the companies tested. If we learn of other systems, we will be glad to test them as we have time (which will be spare time, as the financing for this project runs out tomorrow), although the results might not be comparable, as the Internet is constantly changing.

Here is a copy of the ranking page:

Ranking

Excellent Systems

No system was ranked as excellent - but there have been many people who attended plagiarism detection seminars who scored 100% on the same tests!

Good Systems

Nr. 1 : Ephorus

Acceptable Systems

Nr. 2 : Docoloc
Nr. 3 : Urkund, Copyscape (premium), PlagAware
Nr. 6 : Copyscape (free)
Nr. 7 : TextGuard
Nr. 8 : turnitin, ArticleChecker
Nr. 10 : picapica

Unacceptable Systems

Nr. 11 : DocCop
Nr. 12 : iPlagiarismCheck, StrikePlagiarism
Nr. 14 : CatchItFirst

We hope that our work can help these companies to produce better results. But our summary for 2007 is the same as for 2004: It is better to use a search machine yourself, the software just costs money and is not necessarily very good at finding all plagiarisms.

Blog post on plagiarism detection

I am busy round the clock tidying up and getting my report about plagiarism detection finished for Thursday, and one major online publication has requested that we also write an article for them, so sleep is down to a minimum at the moment.

But I must post this link from "Hardcore Ambiguity" entitled "A few more stabs at plagiarism". The author is spittin' mad about services such as www-dot- academicintegrity -dot-com. We had found a different one during our tests, but this is just outrageous. I won't link to them either (but you can easily find the page). They write about academic integrity on the same page that they offer you their paper-writing services. Just disgusting.

Now back to work

Coming soon: plagiarism detection software test

I have been super busy lately, as we are testing as many plagiarism detection software systems as we can get our fingers on. The final results will be announced on September 27, 2007 at 12.00 UTC+1, so if you are considering obtaining such a system, you might want to wait a few weeks.

We looked at 15 plagiarism detection systems, one collusion detection system and two code sharing detection systems. We used 20 short papers of known plagiarism degree, two files for collusion detection and two 400-line program code files specifically prepared with 14 different types of "masking" behavior.

Stay tuned for the results - in German first, I'm afraid, but we will get around to putting it in English soon, I hope!

Stealing from a Plagiarism Site

We are currently testing plagiarism detection software and were evaluating the results from one system when we discovered a strange link. Some blog at Windows Live had plagiarized one of our plagiarism test cases about the history of Döner, a Turkish fast food popular in Germany.

The site did give a source - plagiat.fhtw-berlin.de is listed at the bottom as being a source. Of course, this is our portal, not the E-Learning unit and most certainly not the exact source. A large chunk and a smaller one are taken verbatim from our page without using those pesky "..." signs that just mean extra typing for the author, I suppose.

I wanted to send the "author" a take-down notice, as my texts may only be quoted or used as per copyleft, that is, that the derivative work also be under the same license and that a link to the license and the authors remain intact. But I have to sign up for Microsoft's "Windows Live" in order to contact the author!

I was just about to sign up in order to contact the author when I found a little link at the bottom of the page labeled "Legal". Following this I find a large page with a paragraph entitled Notices and Procedure for Making Claims of Copyright Infringement. This is interesting, there is even an E-Mail address given and a procedure for issuing take-down notices.

Okay, so this is a test for Microsoft. I wrote an email, detailing the amount of the plagiarism and the problem: I have permission from the original authors of this plagiarism to use this content - my plagiarizer doesn't. I will report on whether Microsoft bothers to answer, and if they do indeed get the page taken down.

I find it very troublesome that someone just uses the text they found without bothering to understand the context in which it was written. It seems we must not only educate our students on avoiding plagiarism, but also the general public. I am happy when people write - that's what blogs are for. But I do not understand that people find it okay to just copy other people's texts - and publish it on their blogs!

This makes it clear that the schools must act and teach about the proper use of content and good research practices, something I thought was self-evident, but apparently is not.

Plagiarism Decrease on use of Detection Software?

The German online SZ-online (Süddeutsche) reports that a dean of a business school has said that the amount of plagiarism has sharply decreased since they started using plagiarism detection software.

Now, I know of no method for actually measuring the amount of plagiarism, as we can only measure the plagiarisms that we actually find. And the intermediate results of my plagiarism detection software test are not encouraging.

It may be that this works as a psychological deterrent - but as soon as students wise up to the fact that the software doesn't catch everything, things will probably return to normal.

Or is anyone aware of any research that a) measures the actual amount of plagiarism or b) that detection software works as a deterrent?

In lieu of scientific findings, this is just wishful thinking.

German "Researcher" discovered to be a plagiarist

A German "researcher" has been discovered to be a plagiarist. Hans-Werner G. (Articles in Spiegel, Süddeutsche, Die Welt) has been discovered not only as having plagiarized some of his publications, he also made up an institute at the University of Maastricht in Holland that he was supposedly working for.

The journal "Research Policy" has just published a retraction of a 14-year-old article by G. In an editorial the journal states that "the article from 1993 is a clear and serious case of plagiarism", according to Spiegel.
During the course of the investigation other plagiarisms were discovered, included a plagiarism from a researcher in Zimbabwe of a plagiarism that G. published.

What shocks me more than this - I have many such cases and accusations that I hear about - is the reaction of the readers in the online forums attached to the articles. Many state things like:

• You can't keep writing things different
  Sure you can - each writer has his or her own voice and style when writing. There are so many different words to choose from, it is nearly impossible to select the exact same wording and structure, and that over many sentences, as someone else.
• Who can remember all the things they ever read?
  Well, if you are not writing down what you read and taking notes, then you are doing shoddy research. You don't just read a bunch of stuff and then write. You have to carefully note down who said the stuff you are building on - or refuting.
• There is so much published, no one can read it all.
  Of course - but no one insists that you have all of the possible sources in what you write, just that you give credit to those who inspired you.
• 90% of all dissertations are plagiarisms
  This may be the case - but then they are not dissertations. We need to be taking more care to teach students and young scientists what scientific work is all about, how you give credit, how you write down your own ideas. And we have to set a good example, and not plagiarize ourselves!
• Poor bloke, it's publish or perish out there!
  Well, we have to get away from this basing all decisions on who gets jobs and research money on impact factors and citations indexes and number of publications. There are so many publications and so many illegitimate ones out there. You can found your own "International Institute for Whatever" by just setting up a web page and printing cards - like G. did, his newest affiliation is the "International Institute for Technology Management and Economics", at the same address as his home telephone number listing.
  

We have to find new ways of judging what good science is - and who the people are who are doing good science. The methods we currently used are broken, beyond repair in my opinion.

A funny aside is that the guy is an economist (Ökonom in German), according to Spiegel, but the not-so-exact Welt has tried to translate this for the "man on the street" and thought that the "Öko" meant "ecological" (which it does in some cases, but not this one) and thus has him as an "ecology scientist". Why on earth an ecology scientist would be writing about SDI (the Strategic Defense Initiative that blew millions (billions?) of dollars attempting to defend against the "Russians" over 20 years ago) is beyond me.

Plagiarism Discussion in German Schools

I am very happy to announce that the German publisher Cornelsen is coming out with a new German school book for 9th graders (to be used in the states Hesse and NRW) that is republishing one of my articles about plagiarism: "Der große Online-Schwindel", which was published in Spiegel-Online a few years ago.

They've chosen just 2 pages out of this long essay that was published in 4 parts. It is good for pupils at the 9th grade level to be discussing plagiarism outright as a topic, and not because the teacher caught a cheater and now has to discuss the topic with the class. An ounce of prevention is worth a pound of cure, as the saying goes.