Plagiarism "finding" software

The German computer mag c't has an article in number 1/2007 (Plagiatfinder: Prüfzwang für Studienarbeiten, S. 78) about software for "finding" plagiarisms. Oh well, at least I am correctly sited: It is useless to try and solve social problems with software.

But the article is still quite euphoric about using software. Sigh. My tests in 2004 were not encouraging - often, you could just flip a coin an be just as right about whether a paper was plagiarized or not. But many companies scream now "We are NEW! We are IMPROVED!", so I am forced to spend my summer term's research allowance (all of 4 hours a week out of 18 off teaching to do research) in order to repeat the tests. Stay tuned for the results after the summer break 2007.

Until then: just use a search machine and your brain. You will get better results.

In Criticism of turnitin

I have been telling all the journalists that call me for some time now that I don't believe in solving social problems - in this case, plagiarism at university - with software. It gives the illusion of a solution without actually doing anything about solving the basic problem, namely, that many students do not have any idea how to do research or how to write up something.

I found this blast against turnitin, the multi-million-dollar plagiarism detection service that conviniently keeps copies of all of the material offered to it for testing, so that it has more good stuff for its database. Scroll down to the scenarios - these are not just made up, I had a similar story reported to me just the other day.

A student had turned in a pre-version of his paper to his professor, who checked it against turnitit - no problem, seemed original. When the paper was finished, he turned it in to the office. Official policy was to run a check before giving the paper to the professors to read and grade - and guess what, the paper now registered as a "high-probability of plagiarism" - because it was very similar to the first draft stored in the database.

I don't want to repeat the article here - just get over there and read the article yourself, following all the links to convince yourself that this is not just an angry competitor writing.

Journalism Class Cheaters?

Janet D. Stemwedel's blog had a link to this article in the New York Times by Karen W. Arenson: "Amid Cheating Allegations, Columbia Class Picks Up Pens." NYT December 3, 2006. Unfortunately, it is no longer available free, at least not to me, although I am registered sigh.

Anyway, in a nutshell: The teacher of an ethics in journalism class gave an open-book, take-home, computer-administrated exam. Inside of 30 hours the students had to log on for 90 minutes and take the test. An anonymous source let the teacher know that someone logged on early and was offering info about the test to other students.

The teacher reacted by making everyone write an essay about what they would do if they were editor at a paper and there was an allegation that some not specified author was fabricating his or her stories.

This is a hard problem - you hear of misconduct, but you have no concrete evidence of who did exactly what. This teacher made a learning experience for the entire class out of the situation (and not just collective punishment). I think this is the way we need to go in many cases of plagiarism and unethical conduct: discuss why this is not okay with the students, have them explore the topic themselves.

We need numbers!

I picked up the broschure of the Polish company, strikeplagiarism.com, who is offering plagiarism detection software. The broschure quotes: "Dr. Joan Bleicher from Hamburg signalled in 2004 that the problem concerns 25% of dissertations and in 2005 already 50%".

This pisses me off. Spiegel online quoted her in 2004 as saying "according to my observations, one term paper in seven is a plagiarism" (for the mathematically challenged, this is 14%). I wrote to her then, requesting a link to her published observations or to more information on how she arrived at her data. She did not answer.

So after seeing this, I again wrote to her (her very long literature list has no publications whatsoever on plagiarism). She answered that she was just speaking about her own experiences. Sigh. These journalists - they are always asking me for numbers and if the incidence is increasing, I just explain patiently that we cannot measure something like this. She spoke off-the-cuff and the journalist ran with it and the numbers are now being re-interpreted by other innumerate journalists.

Mike Hart and Tim Friesner have a great paper putting together the numbers from English-language sources: Plagiarism and Poor Academic Practice – A Threat to the Extension of e-Learning in Higher Education? If you need to find some quotable numbers, this is the place to go. And the numbers are so scary, that they ought to encourage everyone to get started on this topic now.

In Germany I am aware of only two investigations, both thesis work on a Magister level:

• Sarah Knoop: Plagiat per Mausklick - Das Plagiieren von Internettexten in wissenschaftlichen Hausarbeiten. Eine explorative Befragung von Studierenden und Dozenten an der WWU Münster.
• Sebastian Sattler: Plagiate in Hausarbeiten – Empirische Prüfung direkter und indirekter `Rational-Choice-Modelle´ anhand einer Leipziger Studierendenbefragung (paper about his thesis done in Leipzig given at the DGS 2006)
  (Spiegel-Online interview in German)

Both determined that there is enough self-reported plagiarism that teachers should be shocked into action.