False Positives

Inside HigherEd reports on a plagiarism detection track at a large writing conference in San Francisco. Two important points from the article

• One paper, given by a team from Texas Tech University, described an experiment in which they ran 400 papers (apparently of unknown plagiarism degree) through turnitin and SafeAssign. They discovered that the systems returned different values for degree of plagiarism (no suprise to me, as we almost never see identical values in our tests) and flagged many innocent bits as being potential plagiarisms. I cannot find a link to their paper, but the team is listed on the conference site as having given the session "Plagiarism Detection Technology: From the roundtable to the grail—deconstructing Turnitin and Safe Assignment" by Susan Lang, Laura Palmer, Monica Norris, and Kathleen Gillis.
• Much more troubling was the discussion in this article about how iParadigms, the company that sells turnitin, paid for the travel of some researchers who were giving (positive) talks about turnitin. Shouldn't a researcher, who perhaps has no travel money, at least tell the audience listening who paid for the travel? I realize this is rather like the company purchasing an ad on the first slide, but isn't it more transparent than a speaker saying nothing?

There are some good discussion posts at the end of the article (and a kind comment on this blog).

Umeå Study of Plagiarism Detection Software

An article in a Lund, Sweden, student newspaper I found complained that the university's choice of plagiarism detection software was bad because it only found 25% of the plagiarisms in a study done in 2006 by the University of Umeå.

After some unsucessful Googling I entered in a few words from the article and quickly found them in the abstract of two reports from 2006:

• Anna Nordström och Susanne Sjöberg: UTVÄRDERING AV URKUND, ETT VERKTYG FÖR PLAGIATKONTROLL. Oktober 2006. Report 16 (yes, the names of the reports seem to be a sick joke of the IT department on this sociology unit)
  

• Anna Nordström: UTVÄRDERING AV GENUINETEXT, ETT VERKTYG FÖR PLAGIATKONTROLL. Oktober 2006. Report 17

The first is a report commissioned from CERUM by an organisational unit of the University of Umeå about a test of the Swedish plagiarism detection software Urkund. CERUM is the Center for Regional Studies at the University of Umeå.

The university purchased a license for Urkund for four departments at the school for a year and had CERUM look at the usability, effectiveness, and moral problems associated with the use of the software. They interviewed teachers and students prior to and after use and describe very clearly the issues found. Among them:

• Teachers found the system easy to use, as students send their papers to an email address at Urkund. They then send on the paper to the teachers and later send a report on anything found to the teachers. This does, however, pose a problem as student's email addresses are used as the subject line and were often difficult to connect up to real names.
• Teachers felt that the system was effective.
• Using the system didn't save any time, but they feel that it is important for them to deal with plagiarism.
  
• Teachers and students alike felt that using the system worked as a deterrent to plagiarism.
• Students were generally happy with the software and had no moral or ethical problems, as they felt that everyone was being handled equally, something that is very important for Swedes.
  
• There was no mention made of the copyright problems. Urkund keeps a copy of all papers unless the students answer their acknowledgement email and request their paper not be put in a database.
• Neither teachers nor students felt that there was a problem in their relationship with each other based on the use of this system.
  

In order to test the effectiveness of the system, CERUM requested the four departments give them some common texts from textbooks, online resources, and magazines from their field, as well as from the Internet. Urkund says that it checks the Internet, many publications (including the Swedish National Encyclopedia), and of course their own database.

The researchers added material on their own in order to have 20 sources for each department. They then constructed test material (it is not clear from the report if they put all into one text or made a few papers, the report states differently in different chapters) and ran it through Urkund. Only 18 of the 80 sources were found by Urkund, a rather sorry result.

While the investigation was being done, it was discovered that there were a handful of teachers at the school using the Genuine Text system. This Swedish system, which according to their web page is used in Sweden, Denmark, Russia, and some parts of Africa, only searches the Internet and its own database. It is a web-based system that has three ways of submitting: students upload a file, teachers upload files, or they copy and paste material into a field. They offer statistics for administrators and plagiarism reports that are suitable for submitting to the appropriate disciplinary bodies. It is not clear from the report if students have a way of opting out of having their material stored in the database.

CERUM was commissioned to have a look at this as well, and they interviewed the teachers and students who had similar responses as the Urkund subjects. Interestingly enough, although Genuine Text does not test publication data bases, they managed to score 22 out of 80 plagiarisisms found. There were only 14 plagiarisms from the Internet, Genuine Text found 7 of them.

Both reports lead me to the following conclusions:

• The software proved just as ineffective in these tests as in mine. They find about half of the Internet sources and very little else.
  
• Teachers are so happy to have something found, that they believe it to be effective.
  
• The major use of plagiarism detection software is in deterrance. I found many forums in which students wondered how good the systems really were and what their chances for being found out were. There were tips being given on changing around words so as to confuse the system. Unfortunately, this works for many software systems, although Google can often deal with changed word order.
  
• No one considers copyright (or patent!) issues in student works, universities and companies keep copies without explicit permission, as would be necessary by EU copyright law.
• If the students are informed properly (orally and in writing) about the use of the software, they are happy about it being used.
  

I am attempting to contact the authors, who are no longer with the department, to see if I can find further information about the study.

German student thrown out of college for plagiarism

A student of business at the University of Münster submitted his final thesis in 2007. The professors - using software - found numerous passages that were not quoted at all or were improperly quoted. They decided to fail the student.

He went to court, and Germany being Germany, it has taken almost 2 years for his day in court to come. As the WDR reports, the student lost his case. The judge decided that the university was correct in failing the student. The university system has threatened to levy fines (a dubious proposition) in the past, but in this case is not planning on trying that, a wise decision.

This is the first case in Germany of which I am aware in which a university has won a plagiarism case against a student. Usually the universities are ordered to give the student another chance instead of exmatriculating them.

Three cheers for the University of Münster with the courage to go to court on this one!

Plagiarius Awards

The Plagiarius Awards 2009 have been announced! The Aktion Plagiarius e.V. in Germany awards yearly prizes in the form of a black dwarf with a golden nose (inspired by a play on words of the German saying, sich eine goldene Nase verdienen, to earn a golden nose) to companies that instead of coming up with innovative products choose to imitate other products.

This year's winners are: The electronic hand dryer “HTE”, the knee support „GenuTrain®“, and the trolley “Carrycruiser”. Two of the plagiarisms come from China, but first prize goes to a German plagiarist.

But I don't want to be a scientist!

I had an interesting conversation with a student teacher last night. She's new on my sports team and I was explaining to her what I do including my work fighting plagiarism.

She exclaimed: Oh no, you can't do that, how else can I pass except by copying?

I thought at first she was joking, but she was dead serious. She only wants to be a sports and maths teacher, not a scientist. And she feels that things have been said so well by others, why should she try and say it again. I pointed out that this is how she learns things, by putting things in her own words. "I don't have time for that!" she exclaimed.

She was, of course, shocked to learn that her change-a-word-here-and-there process doesn't help: Three words suffice.

I think we have an even larger problem that student plagiarism rolling towards us like a tsunami wave. If even the young teachers - cut & pasters themselves - don't see a problem in copying, how will we teach the next generation independent thought and how to write?

German Degree Mill shut down

Spiegel online reports that a German Degree Mill that sold doctorates has been shut down.

The Institut für Wissenschaftsberatung in Bergisch Gladbach had been in business for the past 20 years. Because of a very high fine in a case of bribery, the company is now bankrupt and their home page defunct.

The degree mill boasted proudly of having paired off thousands of people who wanted doctorates and were willing to pay for it with cash-starved universities and/or greedy professors.

Of course, they were always "legal" - only offereing assistance and advice, no ghostwriting, they said.

One of the two owners was put on trial for bribing a university professor in 68 cases. In July 2008 it was revealed that they paid the professor 4200 Euros (sent to the bank account of a relative): 2100 for accepting the "doctoral student" and 2100 Euros after graduation.

The state court in Hildesheim sentenced the law professor from the University of Hanover - who had raked in over 150,000 in fees that he said that he needed to do renovations on his house - to three years in prison and 150,000 in fines. The degree mill owner was sentenced to 3 1/2 years prison and fined 75,000 Euros, although only 8 of 68 people who paid the first fee actually made it through and obtained doctorates.

The second owner of the company, fearing that he could be sentenced for criminal delay in filing, especially as a second case is still in the courts, filed for bankruptcy. The liquidator has had the web page taken down.

The University of Hanover now asks all doctoral students to swear that they have not used an "advice service".

This seems about right

Found on rateyourstudents.blogspot.com who found it on graphjam.com:

That seems about right.