Test of the Picapedia System

Stefan Weber noted recently that the plagiarism research group in Weimar has tweaked their experimental system picapica. We tested the system back in 2007 while it was in a beta stage. We have looked at it occasionally in recent years, but were not able to test it for various reasons. The system back in 2007 took quite a long time to return very little in the way of finding sources.

They have now focused the system on comparing a text with the current version of the Wikipedia in 10 different languages,  English, German, Spanish, Catalan, Basque, French, Italian, Dutch, Portuguese, and Swedish. That would make it a useful tool for teachers looking for Wikipedia copies in student papers, as students will tend to take the current version of an article and not a historic one. 

I decided to test the system using the 2013 test cases that were based on the Wikipedia, as well as a translation from a Wikipedia, an original work, and a plagiarism with no Wikipedia sources. Both .pdf and .doc files were used. The results:

• 21-Tibet (3 sources including WP:DE): WP source found, and another Wikipedia article with an identical phrase
• 33-Eyjafjallajoekull (1 source, WP:EN): WP source found
• 36-Champagne (translation from WP:FR): nothing found
• 43-Brüder-Grimm (2 sources, one WP:DE): WP source found
• 45-Strelitzia (highly disguised plagiarism from WP:EN): WP source found
• 46-Thermoskanne (25% automatically disguised plagiarism from WP:EN): WP source found
• 47-Tessellation (4 sources, none WP): no sources found
• 50-Union-Jack (original paper with correct references to WP:DE): It notes a similarity to the Union Jack lemma, but does not flag any reused text.
• 51-London-Blitz (4 sources, one from WP:EN): WP source found
• 52-Boxer-Rebellion (disguised plagiarism from WP:DE): WP source found
• 57-Fallingwater (1 source, WP:DE): 2 possible WP sources named, one is correct
• 58-Phillip-K-Dick (disguised plagiarism from WP:DE): WP source found
• 60-Rolltreppe (3 sources, 1 WP:EN properly referenced): It notes a similarity to the Escalator lemma, but does not flag any reused text.
• 63-Hebrew-Plag (1 source, WP:HE): nothing found, this language is not given as a possible one

The system did not always find the total amount of plagiarism, but it pointed to the correct source in all cases except the impossible ones (#36, #47, #63). It also did not report plagiarism for correctly quoted Wikipedia, something many systems do not get right.

The text is first uploaded to their server (in Germany) and deleted after examination, according to their privacy policy. However, they do keep search results and thus may have portions of the uploaded text stored in some form. I repeated the test a day later and saw no trace of the results from the previous day influencing the repeat tests. They do record IP addresses and use Google Analytics, but offer the service free of charge (even for commercial use) as long as it is not abused and as long as the user does not pretend that they developed the system themselves.

So the system does appear to be quite useful for a small subset of plagiarism detection problems, namely identifying text that has been taken from a current Wikipedia. If it is necessary to look for text in older versions of Wikipedia articles, the tool WikiBlame can be quite useful for identifying and dating text taken from the Wikipedia many years prior.

Stormy Waters

It all began with a Facebook posting on April 22, 2014: Arne Janning posted a longish article to his friends asking for help. He had found a recent book by two prominent historians (Karsten, A. & Rader, O. B. (2013) Grosse Seeschlachten -- Wendepunkte der Weltgeschichte von Salamis bis Skagerrak. München: C.H. Beck) to contain plagiarism from the Wikipedia. He exaggerated by saying that "every page contained plagiarism", and wondered what he should do.

The first thing Janning should have done was perhaps to check his privacy settings, as his post was public and the case quickly caught fire and was widely reported on. Maritime puns seem to be the norm for the titles of the articles, as I have also chosen: Spiegel Online ["Abschreiben bei Wikipedia: Zwei Historiker geraten in Plagiatssturm"], Neue Zürcher Zeitung, ["Seeschlacht mit unzulässigen Beibooten"], Süddeutsche ["Wendepunkte der Weltgeschichte aus Wikipedia kopiert"], FAZ ["Unter der Flagge Wikipedias"]. The authors and the publisher promptly threatened Jennings with legal action. According to Spiegel Online, one of the authors, Radar, noted that he did not actually steal intellectual property, as he only used "technical details" from the Wikipedia. "In earlier days we used the Brockhaus [encyclopedia], today we use the Wikipedia," he is quoted as stating [translation dww].

The blog Erbloggtes noted that there were at least two pictures used from the Wikipedia as well as some text, and the pictures were printed without attribution. That is a definite copyright infringement, although one of the pictures was indeed in the public domain, the other was not. Many other blogs joined the discussion: Archivalia, Schmalenstroer, hellojed, plagiatsgutachter. The Wikipedia-Kurier discussion was, as so often, extensive.

The publisher soon decided to withdraw the book, as reported by BuchMarkt, Meedia, and others. Beck ran the book through plagiarism detection software (iThenticate) and declared the parts written by Arne Kasten to be "free from unmarked quotations", despite the fact that it is impossible to prove the absence of plagiarism. One can only demonstrate the presence of plagiarism by a synoptic documentation showing the plagiarism and the source together. The other author, however, had not only plagiarized from the Wikipedia, but from an article published online in 2003. The publisher notes in a pseudo-scientific manner the "exact" word counts and percentages found, although I have repeated shown in my work (for example, my 2013 test) that such numbers are meaningless. Additionally, a reader cannot tell which parts of this book were written by which author, so they both are responsible for the entire book, in my opinion.

Beck also couldn't resist bashing Janning, still threatening legal action, perhaps to deflect criticism from itself for not having properly edited the book. A good comment by Jörg Hopfgarten in the Boersenblatt notes the publisher would be better off to understand that this was just an angry customer blowing off steam, ranting. Customers have a right to do just that without consulting a lawyer, especially when it can easily be seen that they are at least partially right. Amazon is full of similar reactions, this was just the media picking up on the keyword "plagiarism" and running with it, without having independently verified the accusations. Indeed, none of the Seeschlachten books in the Berlin libraries were out on loan when I obtained a copy, although perhaps they all purchased the Kindle version.

Beck closes their press notice with a condescending offer to "participate in a discussion about the use of the Wikipedia in academics." Jan Englemann notes on the Wikimedia blog that the discussion is for all practical purposes already over, as there are numerous court rulings on the legality of the Creative Commons license that the Wikipedia articles are under, CC-BY-SA. It is, perhaps, time for publishers to understand how a legal use of Wikipedia texts works: Link to the license and authors, and put the material that uses a Wikipedia text under at least this license. Open licenses do not mean that the material is free to be misappropriated.

Many of the blogs discussing the topic have started documenting the plagiarized portions, in particular using a German system Picapica (also called Picapedia), that compares text to the current version of the Wikipedia. I will be bringing a short test of this system in my next blog entry.

Schavan court decision published

The German court in Düsseldorf that heard former German education minister Annette Schavan's arguments against the University of Düsseldorf rescinding her doctorate has published the written form of the judgement, just a few days after Schavan announced in a personal note that she will not be pursuing the case in the upper courts. Schavan's note makes it clear that she still does not understand what the problem really is about. She speaks of academic ethics and bemoans that so many upstanding academics had advised her that she was doing things right. They were all wrong.

Schavan's lawyers had loaded their guns with pretty much every halfwitted idea they could cobble together that was supposed to show that the university was wrong. The judge writing the judgement neatly and clearly knocks down every one, choosing examples from Schavan's own dissertation to illustrate point after point. The judgement is so thorough in its analysis, that it should convince anyone considering taking a university to court for rescinding a doctorate of the hopelessness of that undertaking. There are quite a number of nuggets in the decision that should force many other universities to have a hard look at their current practice of accepting problematic doctorates (¶129):

Dass einzelne Autoren (zum Beispiel Stadter und Fend) sich durch das Nichtzitieren ihrer Werke in der Dissertation der Klägerin nicht nachteilig betroffen fühlen, ist für die hier allein entscheidende Frage, nämlich ob die Klägerin getäuscht hat, offensichtlich ebenfalls irrelevant. (It is obviously irrelevant for the question at hand, which is whether the petitioner has deceived [the faculty], that some authors (for example Stadter and Fend) do not feel themselves to be detrimentally affected by their works not being quoted in the dissertation of the petitioner. 

Exactly. It has nothing to do with the feelings of this or that person, but whether any reader of the work could be unclear as to whether it is the author or some third person speaking at some point. This needs to be understood in universities such as Cottbus and Heidelberg, although the latter case also involves members of a working group "sharing" text.

The judge closes with the clear statement (¶238):

Lediglich vorsorglich ist anzumerken, dass auch im Übrigen keine Anhaltspunkte für mildere Maßnahmen ersichtlich sind. (As a precautionary measure it is observed that there is no evident basis for milder measures.)

The blog Causa Schavan has pointed discussion of the judgement, including making clear who the persons are who are alluded to "anonymously" in the text. The blog Schavanplag, which originally published the plagiarism documentation publicly under Schavan's name, also includes a discussion of the financial and legal problems the government will have if they try and put Schavan in place as the Ambassador to the Holy See.

I still have not found a comment from one of the Allianz organizations that came out so loudly, denouncing the University of Düsseldorf and defending Schavan. One would expect them as academics to state: Okay, we understand now, we were wrong. If a reader does find such a statement, please post it in the comments section here!

Plagiarism or best practice in medicine?

VroniPlag Wiki [where the author is a participant] has documented extensive text parallels in four medical dissertations published on eDoc servers in the past weeks: Tz - Aho - Da - Feb. Tz and Feb are doctorates in dentistry, Aho and Da in medicine. Tz sports three "Dr." titles in front of his name, Feb uses this impressive collection of titles on his home page in his native country:

Dr. Dr. <Feb> D.D.S., Dipl. Funct. Dent., Dipl. Orth., M.S.D., Ph.D., Ph.D.

The doctorates were granted from different universities:

• Feb was granted two doctorates in dentistry, one in 2009 from the University of Tübingen and one in 2010 from the Charité in Berlin.  The Charité thesis is the one documented at VroniPlag Wiki, and it builds strongly on a thesis submitted to the University of Gießen in 2005, and the University of Würzburg in 2004. The thesis was given the Alex-Motsch-Preis 2009, awarded by the Deutschen Gesellschaft für Funktionsdiagnostik und -therapie – see [1], [2]. The thesis is a heavyweight for a medical thesis with 150 pages in the main part. However, almost 30 % of these pages are plagiarized.
• Tz has one doctorate in medicine, one in dentistry, and one in natural science. 56 % of the thesis in dentistry, which consists of just 30 pages and was submitted to the University of Mainz, is taken from a thesis submitted to the University of Gießen in 2003.
• Aho has 18 pages of his 58 page medical thesis that was submitted to the University of Hamburg that are completely taken from other works. In all, 51 % of the pages are found to contain plagiarism. One interesting copy & paste problem can be seen in this fragment, the formula becomes completely senseless: Aho/Fragment_040_01. This thesis was published in 2012, well after the zu Guttenberg plagiarism discussions were going on in Germany, so one wonders why the author would put a copied thesis online.
• The medical thesis of Da, with 45 pages, was submitted to the University of Freiburg/Br. in 2007. 51 % of the thesis is taken from other sources, in particular two dissertations from the Humboldt University Berlin [3], [4].

The three cases documented before these on VroniPlag Wiki are not from medicine directly, but from natural sciences close to medicine. Iam (Göttingen) has 35 % of the 81 pages containing text parallels, Mag (TU Braunschweig) 28 % of 110 pages, and Arc (FU Berlin) has a whopping 62 % of the 51 page thesis taken from earlier sources.

One can see by the size of the theses that that they are more similar to Master's theses than doctoral dissertations. The universities are often not interested in rescinding the doctorates, see for example the University of Gießen ([5, dentistry], [6, medicine], [7, medicine], [8, dentistry], three of the four not rescinded, one still under consideration, one other dissertation in law not rescinded). And as one of the authors of a medical dissertation documented on the VroniPlag Wiki pages stated when we spoke on the phone, they were told to use particular texts in their theses by their advisors. There was a folder next to the machine they used for their experiments, and everyone just copied the text from there. The real value of the thesis, he explained, was in the experiments conducted. Did they not realize that when publishing their theses online, the whole world can see the text copied?

So is this not plagiarism, but best practice in medicine and dentistry? Even if it is the same experiment, is it not possible to give the reference to the paper that first described the experiment, explaining changes that one has made to the setup? Why is it necessary to take so much text when the theses themselves are quite small? Two theses from the LMU Munich have been found that discuss experiments done in different years, but for which the text is practically identical ([9, 2006], [10, 2007]), without the later thesis mentioning the earlier one.

If it is best practice, then I think German universities should stop awarding doctorates in medicine. This is not science, but going through the motions in order to have those letters "Dr." (or multiples of the letters) to put on one's shingle. Let's not confuse the two.

Short links

Thanks to a correspondent for combing Google News for these links:

• The International News (30 March 2014) reports that a professor in India Pakistan is being forced to retire on account of plagiarism in research articles he published:
  

  The Punjab University (PU) syndicate on Saturday, confirming plagiarism charges against the varsity’s Institute of Chemistry Prof Dr Zaid Mahmood, penalised him with forced retirement under the PEEDA Act. [...] Dr Zaid had been claiming that his research papers were published before 2007 and therefore they could not be made a subject of the inquiry as per the HEC’s plagiarism policy.

  As if just waiting a number of years somehow changes a plagiarism into a non-plagiarism. The article does not state if the papers are being retracted.
  
  
• The Guardian has a piece (21 March 2014) on how easy it is to plagiarize using the Internet, but also how easy it is to find people out:
  

  The act of uncovering and investigating acts of plagiarism is becoming easier by the day. Search engines, online plagiarism checkers (of varying quality) and the viral publicity opportunities afforded by social media all play their part. Plagiarism searches can be compelling, like addictive puzzles where positive results elicit mental fist-pumps of delight.

• The Times Higher Education notes (3 April 2014) that a senior sociologist, caught by a young PhD in plagiarizing from the Wikipedia, of all places, finds that rules about referencing don't apply to his scholarship:
  

  An eminent sociologist has claimed that high-quality scholarship does not depend on “obedience” to “technical” rules on referencing after a PhD student accused him of plagiarising from websites, including Wikipedia, in his latest book.
  
  Zygmunt Bauman, emeritus professor of sociology at the University of Leeds, was responding to claims that he fails to clearly indicate that several passages in his 2013 book Does the Richness of the Few Benefit Us All? are exact or near-exact quotations from the online encyclopedia and other web sources.

  Bauman tries to put the PhD student down by snorting that ideas aren't owned by anyone. But really, shouldn't every academic be able to clearly state what is from others? It does then rather save face when it turns out that something one is using from other people without attribution is just plain wrong...

Stem cell controversy in Japan

The Japanese stem cell researcher Haruko Obokata has been found guilt of misconduct in her work, according to the Guardian, the Tagesspiegel, and the Japan Times. Her work proposed a simple method of generating so-called STAP-cells in a petri dish. She published two articles in Nature on January 30, 2014 ([1], [2]).

But when researchers could not replicate the results and accusations of photo manipulation and plagiarism arose, an investigation in her lab was started. The report on the investigation determined that she was guilty of scientific misconduct, just as Chinese scientists have announced that they were able to replicate the results with a slightly different set-up. Two days later, that group announced that maybe it wasn't really a replication. Some felt that this was an April Fool's joke, the Chinese researcher insists not.

This entire issue does raise questions about how to deal with accusations of misconduct. Obokata insists that the picture manipulations had nothing to do with the results. There are many other authors listed on the Nature papers, did they convince themselves that everything was okay before publishing? Or was there a rush to cry "First!" with a revolutionary development? It remains to be seen how this story will continue.

Russian Plagiarism

A reader sent in two interesting links on plagiarism in Russia:

• The Washington Post (18 March 2014):
  Russia's plagiarism problem: Even Putin has done it!
  

  Russia has a really big plagiarism problem. So many businessmen, academics and high-ranking government officials — President Vladimir Putin included — have been found to have plagiarized their college and doctoral theses that Russia’s education minister just denounced the revelations, saying they were hurting Russia’s reputation.
  “People not versed in this topic will get the idea that all academics are cheats and liars,” Education and Science Minister Dmitry Livanov just told the Kommersant newspaper, according to a Russian news agency. ”It’s a severe reputational problem for Russian science.”
  
  [...] Olga Khvostunova of the Institute of Modern Russia, a nonpartisan think tank in Washington, wrote about the plagiarism scandal in Russia in this piece, detailing some of the history of the uncovering of the plagiarism scandals in Russia. She wrote about Putin's plagiarism: "The scandal over Putin's dissertation led nowhere. But because the head of state's deed had no repercussions whatsoever, a new trend emerged in the country: plagiarism in the writing and defense of dissertation works began on an unprecedented scale."

  I would say that the "reputational problem" is not in the reporting about the academic misconduct, but in the misconduct itself.
  
  
• University World News (24 February 2014):
  Government to combat plagiarism and illegal degrees
  

  The biggest scandal over fake dissertations occurred in the summer of 2013 at Moscow State Pedagogical University, one of the country’s top institutions. Due to serious violations in the preparation of dissertations, several people lost their degrees, the local dissertation council was closed and the head of the university was fired.

  It seems that the Russian software company Anti-Plagiat,  looked at 14, 500 history theses and found plagiarism in every 10th one, according to the Moscow News.
  
• Update: Just found this in Ria Novosti (from 10 February 2014):
  

  Russia’s Education and Science Minister has denounced a grassroots campaign to expose alleged academic plagiarism among high-ranking state officials.
  
  Whistleblowers are harming the public image of Russian academics, Dmitry Livanov said in an interview with Kommersant daily published Monday.

  Strange, I thought it was the plagiarists who were harming the public image of Russian academics.
  

Previous articles on Copy-Shake-Paste about Russia can be found here.