Friday, March 15, 2013

A Journalistic Quicky on Plagiarism

Time and again I get exasperated with journalists who are only out for a quick sensation instead of actually researching a question. Last Friday a journalist wrote me an email at 3 in the morning asking me to help him answer a question from a student. They are starting a new "service", answering real questions. The question was: How can I find out if I have plagiarized in my term paper? He wanted a very short answer, 4-5 lines, and it could be as "perky" (flott) as possible.

I was traveling, giving a workshop on plagiarism detection for professors and leaving for a week of vacation in the morning. Since this is not a question that can be answered in 4-5 lines, I just didn't reply.

So what was the result? I was contacted today by a friend with a question about a software system that I had "recommended" in some article. Checking my alert service I found the article and was aghast -- the software in question was one that came in last in my 2010 test of so-called plagiarism detection software. The entire article turns the results of my tests upside down.

I suppose it is because I am a computer scientist and I test software that journalists expect that I "recommend" the top-rated software, tell people what to buy and where the magic button is for quickly and painlessly finding plagiarism. In fact, there is no recommended software, the category at the top is "partially useful". It says at the top of the page:
Our recommendation: Only use these systems when suspicions of plagiarism arise that cannot be found with 3-5 words in a search machine. The focus should be on teaching students about plagiarism and how to avoid it instead of investing time in using software. Most of the work involved later is in preparing a plagiarism case and dealing with the plagiator, and for this good documentation is needed. Very few systems provide good documentation.
The journalist banged together three statements, the first and the last attributed to me and the second one not directly including my name, but people have read it as being from me. He writes (numbering by me):
(1) Die Berliner Medieninformatikerin Debora Weber-Wulff hat vor Kurzem verschiedene Softwares getestet. Sieger war das kostenpflichtige Programm turnitin, das mittlerweile an über 80 deutschen Universitäten und Forschungseinrichtungen genutzt wird. Sie können es möglicherweise über Ihre Hochschule kostenlos nutzen.
(2) Eine brauchbare kostenfreie Alternative ist Viper.
(3) Kurze Texte können Sie außerdem gratis bei PlagAware kontrollieren lassen, das in Weber-Wulffs Analyse ebenfalls solide abgeschnitten hat.
 ad (1): "vor Kurzem" means recently. Um, my last test was 2010, published in 2011. We are currently retesting the software, hoping to publish the results in September. These tests take a lot of time to conduct. "Sieger" means the winner. There were no winners. There were 5 systems that were given the grade of C-, as they did not even manage to reach 70 % of the points in the test. None of these can be considered winners, but they can be used as tools if one has a suspicion of plagiarism. They have far too many false negatives and false positives for general use, and all five are on about the same level. And the idea of letting students use them (for free even!) to run their papers through until they come out clean gives me shudders down the spine. The students will resort to swapping word orders and using unfitting synonyms until the numbers (which will differ for the same text from system to system and even in some cases from test to test) reach an acceptable level. There must be someone with experience in reading and interpreting the reports to go over this together with the students if it is to be a meaningful learning experience.

ad (2): Oh dear. I suppose the journalist was looking for a free system, as opposed to the expensive ones. I have no idea what terms he googled to find this system, but he didn't read this on my results page. Technically, he doesn't say that I find the system useful ("brauchbar"), this is his own judgement. But sandwiched in between two references to my work, people misread it as me recommending it. Nothing could be further from the truth. The street address is the same for the company offering the plagiarism detection service and for a paper mill / essay writing service. Their respective telephone numbers differ only in the last digit. And actually reading the "Terms and conditions" should scare the daylights out of anyone: If you submit a paper to Viper, you give the company "All Answers Limited" the irrevocable right to keep a copy of the paper forever and to use it for marketing purposes, either for Viper or any "associated website".  The email account I used for the test (we use a different email address for every test to see what kind of spam shows up later) now gets fun letters with subject lines such as "Enjoy the sun and get your dissertation done on time". Oh, and Viper has the distinction of being the worst system for detecting plagiarism in the test, coming in last with just 24% of the effectiveness points on all of our test cases in 2010.

ad (3): Another system is mentioned as being "solide", respectable, in my analysis. Again, "partially useful" is not even close to respectable.

I wonder why the journalist is in effect advertising the services of three companies, one of which is extremely questionable, instead of really trying to answer the question. At least the comments are good: Most of the comments note that one does not actually plagiarize by mistake. If you are taking careful notes and making the difference clear between what is your own words and ideas and what is from someone else, you are fine.

Now, how do I get journalists who just breeze through writing something like this without actually reading and understanding what hat been written on the topic to start doing real research themselves instead of just blowing off an email to me? I suppose I need to answer all journalists, as soon as possible, with short and snappy sayings. I do wish, however, that they would read at least the summaries before they ask.



11 comments:

  1. The "journalist" is, in fact, a grad student, prouding himself in his admission to an 'elite university': http://www.zeit.de/studium/hochschule/2012-05/Politikwissenschaft-Kritik and http://www.schekker.de/content/einer-eliteuniversit%C3%A4t-england-%E2%80%93-die-kaderschmiede-teil-2.

    ReplyDelete
    Replies
    1. Oh, my:

      http://juliankirchherr.wordpress.com/about/

      Delete
    2. It's good, though, to have FDP members that take plagiarism as a serious issue.

      Delete
    3. Yeah, if he would take this serious, he would have cited Debora Weber-Wulff correctly. If he would take this serious, he would have updated his bad article after it was pointed out to him.

      So I come to the conclusion that he does not care about the proper quotation of the arguments and conclusions of other people.






      Delete
  2. Some people put a 'dear journalists before your write me' FAQ on their webpage -- I think your text here is pretty close to it, if you rewrite it a bit. Slip in something so you know that they read at least your FAQ before the wrote you ...

    Some ZEIT Online articles have a very low quality -- they 'fix' this by blocking the accounts/removing the comments of people pointing this out -- even if they do it polity. I couldn't believe myself, but it happened to me, too.

    ReplyDelete
  3. Woppa ... never thought about it this way, but it seems that journalism can have the same problems as science when it comes to (bad) research ...

    I hope that ZEIT Online reacts to the criticism ... I also wonder if ZEIT really wants to answer questions this way. I mean, really, checking with software? Or "Laut Software ist alles okay; der Dozent sieht das aber womöglich anders." -- ähm ... really, just a matter of perspective?

    The whole ZEIT Online article is superficial and seems to give tips to make a text appear to be free from plagiarism in the end, instead of avoiding plagiarism by using better work methods ... I would have expected better from ZEIT.

    ReplyDelete
    Replies
    1. If one thinks about it, the 'whole ZEIT Online article' as you said, shows the problem with improper citing -- if he had cited Debora Weber-Wulff correctly, he would have written a different article ... Because he didn't took the time needed to do this -- he create /false/ knowledge. This is bad for the people and bad for Weber-Wulff -- and bad for him. ZEIT Online should update the article with a short excuse and be done with it.


      Delete
  4. Just for fun, a few (bad) sources, about the prices ZEIT ONLINE pays for an article ...

    http://www.journalismus.com/job/honorare/online.php?s=z
    http://wasjournalistenverdienen.tumblr.com/page/2

    ReplyDelete
  5. Liebe Frau Weber-Wulff,

    vielen Dank für Ihren Blog-Beitrag - und bitte entschuldigen Sie, dass ich mich erst jetzt wieder bei Ihnen melde.

    Wir haben den Artikel nun Ihrem Kommentar folgend angepasst und um eine Anmerkung ergänzt. Das Programm "Viper" wird nicht länger empfohlen. Zudem erwähnen wir nun im Kern-Text auch nicht mehr Ihre Studie, so dass nicht länger der Eindruck entstehen kann, Sie würden Studenten zur Nutzung dieser Programm raten. Genau wie Sie betonen wir weiterhin im Text, dass Plagiatssoftwares leicht an ihre Grenzen stoßen.

    Die Ursprungsversion des Artikels findet sich weiterhin hier: http://juliankirchherr.wordpress.com/2013/03/15/studentenfragen-habe-ich-versehentlich-plagiiert/. Zudem habe ich dort auch auf diesen Blog-Beitrag verlinkt.

    Viele Grüße
    Julian Kirchherr

    ReplyDelete
    Replies
    1. I don't know if you exchanged mails with Weber-Wulff, but sadly, what you/ZEIT Online writes, is wrong again.

      "Anmerkung: In einer früheren Version des Textes wurde eine Untersuchung zur Wirksamkeit verschiedener Plagiatssoftwares zitiert. Die Autorin dieser Untersuchung, Debora Weber-Wulff, rät Studenten nicht zur Nutzung dieser Programme."

      You never quoted her correctly. Ever. That's the whole point. Your/ZEIT Online's update is just weasel wording around this issue. Learn to summarizes the idea of others correctly, learn to quote, learn from your mistakes and learn to admit them.



      Delete
  6. Liebe Frau Weber-Wulff,

    vielen Dank für Ihren Blog-Beitrag - und bitte entschuldigen Sie, dass ich mich erst jetzt wieder bei Ihnen melde.

    Wir haben den Artikel nun Ihrem Kommentar folgend angepasst und um eine Anmerkung ergänzt. Das Programm "Viper" wird nicht länger empfohlen. Zudem erwähnen wir nun im Kern-Text auch nicht mehr Ihre Studie, so dass nicht länger der Eindruck entstehen kann, Sie würden Studenten zur Nutzung dieser Programm raten. Genau wie Sie betonen wir weiterhin im Text, dass Plagiatssoftwares leicht an ihre Grenzen stoßen.

    Der Ursprungsartikel findet sich weiterhin hier: http://juliankirchherr.wordpress.com/2013/03/15/studentenfragen-habe-ich-versehentlich-plagiiert/. Dort verlinke ich auch auf diesen Blog-Beitrag.

    Viele Grüße
    Julian Kirchherr

    ReplyDelete

Please note that I moderate comments. Any comments that I consider unscientific will not be published.