Sunday, February 22, 2015

The Phantom Degree

The French Minister of Higher Education and Research, Geneviève Fioraso, turns out only to have an English degree and not one in English and Business, as Mediapart (paywall), Le Monde, and  Le Figaro report.

According to Fioraso, somehow "Who's Who in France" mixed up her degrees, and made a double degree in English and in Business out of an English degree with an "option" on Business (screenshot of the entry). She is taking steps to correct this information, she says. Who's Who in France advertises that they verify the degrees from the grand écoles, which does make sense as the Wikipedia entries are free to read, so it would be quite interesting to see how this information came to be in their databases. One must pay 6 € in order to view the entry, however, so I'll stick with the screenshot above.

The government was quick to assure the general public that the minister was chosen for all the great things she has done and not on the basis of a specific degree. Yes, we've heard that before in Germany, in connection with plagiarism cases.

Friday, February 13, 2015

Austrian term papers clog plagiarism detection system

The Austrian online newspaper derStandard.at reports on a bit of problem with their new high school term paper submission system for the school leaving certificates matura. Pupils in Austria are now expected to submit a 40,000 to 60,000 character long term paper (vorwissenschaftliche Arbeit) by the middle of their last year of school. The paper will be graded by teachers and the students must give a presentation on their work.

Of course, since Austria is well aware that there is a plagiarism problem, at least as far as pupils and students are concerned [not so much for doctoral dissertations, but that is another blog post], the term papers must be checked for plagiarism by a so-called plagiarism detection system.

The due date 2015 is Friday, February 13. Surprise, many students have waited until the last minute, and the system is throwing errors that appear to point to the system being swamped. Apparently, they did not also reckon with such large files as are being uploaded. The server operator noted that they were expecting the files to be around 1 MB, instead they were getting 60 MB large files.

Not to fear - there is a Plan B in action: the pupils can submit a printed version at their schools in order to keep the deadline. Or, as one teacher noted in a comment, submit at 5 a.m. The server runs well at that time of the night.

2013 there were almost 44,000 pupils granted their diplomas in Austria. Teachers will now, in addition to grading these papers, have to wade through the results of the plagiarism-detection software, although they also generate false positives as well as false negatives, thus not determining plagiarism but giving some ideas as to where perhaps there could be some plagiarism. Even assuming that a teacher only spends an average of 10 minutes per paper interpreting the results (and this is generous, as the reports are not easy to read and the numbers reported can be quite misleading), this means a minimum of 7-8000 extra hours of work nationwide, but probably tenfold that.

If the pupils are anything like the ones I see in the first semester, they love to take pictures they found on the Internet to spice up their texts - they are much more visually oriented than the older generations. The software will certainly not be able to identify pictures that are not used according to license, so the teachers will also need to use Google's image search or a system such as TinEye to look for the potential sources, increasing the amount of time needed for grading.

Maybe the idea of a term paper submitted centrally needs to be rethought? Of course, they have to learn how to do research and to write about a topic. But we need to be thinking about how to develop methods of assessment that are plagiarism-proof, instead of adding more broken software to a broken system.

Saturday, February 7, 2015

The disappearing links

I've noticed a change in what Google returns when you search on the names of persons who have been documented on the VroniPlag Wiki plagiarism documentation site as having extensive text parallels that could constitute plagiarism in their dissertations and habilitations. It used to be that when one searched for their names, the link to VroniPlag Wiki came at the top of the search.

Now there is a notice that some links have been removed because of personality rights questions, and links show up -- if at all -- past the first page of results. This is perhaps due to a recent court ruling.  The European High Court (EuGH, 13.05.2014 - C-131/12) ruled that people can have links about them personally "forgotten" by search engines. The pages naming them do not have to be removed, but they can insist that the search engines not point to such pages.

So it is not enough to just google a name to see if there are any problems associated with scientific publications about a person. One would now need to know where to look in order to find out if, for example, plagiarism in a doctorate has been documented or a paper withdrawn or issues with a publication documented.

This is unfortunate for scientific purposes, as it is vital that other scientists are made aware of dissertations, papers, and books that have been withdrawn for plagiarism or other academic misconduct. Otherwise they will try and replicate experiments that were forged, or build on top of wrong material. I have heard the excuse that a paper is plagiarized, but the contents are true. That is not always the case, sometimes in plagiarizing, something gets taken out of context and the meaning is changed.

Privacy is important, but scientific papers and dissertations are not part of one's private life. They are contributions to the body of science, and are thus public and open to criticism. That's what keeps us honest as scientists: if we goof up, our names are forever associated with our misdeeds.

Update: There has been some discussion about which cases are affected. The following cases (there may be more) show me the VroniPlag Wiki link either on the second page or not at all: Alm - Bm - Cl - Nig - Rh - Tt

  

Friday, January 30, 2015

A Patchwork Thesis

In 2008, a graduate of a German Fachhochschule (University of Applied Sciences, abbreviated FH) submitted a dissertation to the Tomáš-Baťa-Universität in Zlín, in the Czech Republic, a good 800 km from his place of residence. At that time he was in his mid-40s and had been working as a public official for the past 18 years.

One may wonder why a mid-career public official would go so far afield to obtain a dissertation, when there are excellent universities near his hometown. In Germany at that time, a Diplom from a Fachhochschule was not sufficient to be admitted for doctoral work. Often extra coursework would be required, or even a Diplom or Master's at a university had to be completed before beginning work on a doctorate.

Today, the Diplom is no longer offered, but instead Master's degrees from both universities and FHs are acceptable for beginning work on a doctorate at a university. Then as now, however, doctorates earned in other EU countries can be used back home, so there is quite some interest in obtaining degrees outside of Germany.

Zlín offers a four-year doctoral program in Management and Economics that charges 1,600 €/year in tuition that requires submission of a written dissertation that is generally published online in the Zlín Digital Library.  One of these dissertations, the one submitted by the German public servant, is a 100-page dissertation that is now documented as case #140 on the VroniPlag Wiki site.

The "barcode" representation for the manual VroniPlag Wiki documentation for the case looks quite like a bit like a patchwork quilt. This barcode is often misunderstood as being the result of a software-based plagiarism investigation. Nothing could be further from the truth: All discovery and documentation is done manually with the help of small software tools for various tasks, and all documentation is reviewed by a second researcher.

The bar code uses five different colors:
  • white is for pages that have not yet been investigated, or for which nothing has yet been found;
  • bright red is for pages that contain text parallels on over 75 % of the lines of text on a page. The line counting is not automatic, but must be done and reviewed by two researchers;
  • dark red is for pages that have text parallels on between 50 and 75 % of the lines;
  • black is used for pages that have text parallels, but that make up 50 % or less of the page;
  • blue is used for pages that are excluded from consideration. These are normally the title pages, the table of contents, the literature list and any appendices.
For this dissertation, there are some additional blue bands: Towards the end of the thesis there is a list of abbreviations and one of figures and tables that are sandwiched in between pages of content.

http://de.vroniplag.wikia.com/wiki/Msc
[Msc 2008]
The two blue bands after the first few pages are quite interesting. The first one, extending from page 13 to page 21, is taken verbatim from the Catechism of the Catholic Church. The second one, running from page 26 to page 33, is verbatim from a European commission document. Discounting these pages, as each one has a brief reference given when the copy begins, there is only a total of 65 pages of content in this dissertation.

http://de.vroniplag.wikia.com/wiki/Msc/Fragment_024_04
The patchwork continues when looking at the individual pages, as there are also problems on many of those pages. For example, extensive swatches of text are taken verbatim from seven Wikipedia articles without reference. The pages 2325, which deal with some topics in German history, are lifted entirely from the Wikipedia with only minor adjustments.








http://de.vroniplag.wikia.com/wiki/Msc/Fragment_058_01Ten fragments are taken from a journal article that appeared in the Academy of Management Review. There are occasional references to the article given in the text, but it is not made clear that the pages 5663 are almost entirely from this article, and taken verbatim. Page 58 includes an interesting copy & paste error: the printed version of the article has a footnote from the previous page continued at the bottom of the left-hand column. In the dissertation, this text can be found sandwiched-in  between the text from the left-hand column and the text from the right-hand one. The sentences thus make no sense whatsoever.



On page 72, the Daimler-Benz sustainability report is copied with the "we" pronouns changed to "they" or "their" or "Daimler".

http://de.vroniplag.wikia.com/wiki/Msc/072

Pages 7779 are taken verbatim from a discussion paper for the European Sustainable Development Network Conference 2008. A copy & paste error on page 79 caused quotations marks from the original to be reproduced as | or —.

http://de.vroniplag.wikia.com/wiki/Msc/Fragment_079_01

Whenever the writing shifts from proper English sentences to word-for-word literal translations from German, the thesis becomes quite unreadable. I quote from page 50:
Germany applies in 2001 above a surface of 357,020 sq. kms, the population around catches 82,330,000 million people (2000: 82,260,000). Of it 40,326,000 persons (49.1%) were gainfully employed in 2000. In 2001 there were 2.4% of the employed persons in the agriculture, forestry and fishery, 22.0 % in the producing trade without the building trade, 6.7 % in the building trade, 25.4 % in trade, guest's trade and traffic, 15.2% in the area of financing, renting and services for companies and 28.3% in the sector of public and private service providers (cf. "Germany in figures" in 2002). While trying to explain how many of these persons have been employed in small and middle companies or to define the boundary between them and large companies in Germany one pushes fast to his borders, because there are not enough actual statistical facts offered from the Statistical Federal Office16. Merely on data delivered by the Institute Of Middle Class Research17
[Note: The institution referred to in the last sentence is the Institut für Mittelstandsforschung in the original. It actually researches small and medium-sized enterprises, not the middle class.]

The University of Zlín publishes the reviews by the thesis examiners online, a commendable gesture. Two excerpts are documented in the Findings section on the VroniPlag Wiki:
  • Review 1 (17.11.2008): "Author considers that CSR [Corporate Social Responsibility] is suitable way how to change current managerial thinking which describe from different point of views, e. g. historical progress in religious aspect."
  • Review 2 (17.11.2008): "The dissertation is written very cultivate, digest at the high academic level."
I beg to differ. I don't find this patchwork of other people's words to be either at a high academic level or acceptable scholarship. Above all, it is a mystery to me that people are not aware that when they publish their works in a digital library that they are available world-wide for discussion. Does no one read the theses critically before publication?

The University of Zlín has been informed of the situation and has been sent this report containing all the documentation produced manually by VroniPlag Wiki about the dissertation. The university promptly acknowledged the receipt of the documentation by email. Many other universities, sadly, don't manage to do even that.

Saturday, December 20, 2014

Christmas Links

I seem to be getting more and more links I can't adequately deal with, but which I don't want to withhold from readers. So here is some Christmas reading:
  • The "Neurosceptic" blog of Discover Magazine has a piece about The Strange Case of “Publication Integrity and Ethics” which details a number of integrity and ethics questions around the supposed new journal.
  • The Times Higher Education has a piece on post-publication peer-review that describes more of the chilling consequences that occur when lawyers meddle with scientific inquiry. Physics professor Philip Moriarty is quoted with: “If you are publicly funded and you put your research into the public domain but no one can criticise you for it without facing legal proceedings, that seems to me to be a very badly damaged system.” Exactly.
  • Retraction Watch obtained a $400.000 grant to set up a retractions database! This is great news, I hope that the database can be used to calculate a Retraction Index, that is, how many retractions per article published a journal has, and perhaps how long did it take for the retractions to take place after the initial information of the journal.
  • Bernd Kramer recently published a book in German about obtaining a doctorate in Germany without doing the work ("Der schnellste Weg zum Doktortitel. Warum selbst recherchieren, warum selbst schreiben, wenn's auch anders geht?"). The cover is a horrible stock photo, but the book makes quite interesting reading. Kramer gave an interview in Deutschlandradio in November 2014 about it.
  • Reports of fake peer reviews are increasing. Vox has an article about 110 papers retracted in the past two years on account of faking peer reviews. Retraction Watch reported on SAGE publishers retracting 60 papers from just one journal for this reason. The Minister of Education in Taiwan, Wei-ling Chiang, had been added to some of these papers as a co-author (he says without his knowledge). He stepped down because of the scandal in July 2014, according to IEEE Spectrum
  • Taipeh Times reported in August of 2013 that Andrew Yang, the former Taiwanese Minister of National Defense was forced to resign in a plagiarism scandal a few days after taking office. He had published a book in 2007 that friends had ghostwritten for him. They had, however, plagiarized large parts of the book.
  • The University of Nevada in Las Vegas fired an English professor for "serial plagiarism." The student newspaper, The Rebel Yell, also reports on the case.
  • End of November 2014 the Vice Chancellor of Delhi University in India was jailed and released on account of plagiarism.
  • There is a nasty case of plagiarism reported from early 2014 at the Chicago State University. The dissertation of the Senior Vice President and Provost of the university was being investigated, and the university confirmed to press that they were doing so. She sued the university for violating privacy laws, stating that she did not plagiarize [1]. There exist documentations of plagiarism in her dissertation in a blog ([2] - [3] - [4] - [5] - [6]). Despite the documentation, the University of Illinois, Chicago has ruled that her dissertation is not a plagiarism ([7]). The Chicago Tribune had three plagiarism experts (Tricia Bertram Gallant, Teddi Fishman, and Daniel Wueste look at the thesis ([8]). All three find the thesis problematic. The question is, are the students to be held to a different standard than the person who is enforcing that academic standard? A thorny question.

Monday, December 1, 2014

Diverse links

Here are some links that need documenting:
There will be more, I'm afraid, to come. 

A visit to the Academy

The Berlin-Brandenburg Academy of Sciences and Humanities invited me to speak this past week at a non-public meeting about plagiarism detection software for the working group Zitat und Paraphrase (Quotation and Paraphrase). I was a bit leery of speaking there, as some of the members of the group have publicly demonstrated a quite problematic interpretation of plagiarism as far as it concerned the dissertation of one particular person (see [1] - [2] - [3] for detailed online articles in German about this particular case, and a recent German essay [4] that compares this plagiarism case with one from the early 90s).

Since I do enjoy a good discussion, I agreed to speak. Unfortunately, the meeting was not open to the public, so I am only able to repeat the points of my presentation here, not the ensuing discussion. As it turned out, there were not many members of the group there, and none of the vociferous members I had been expecting.

I first made it exceedingly clear that VroniPlag Wiki is not a machine or software of any sort, but an academic community in which I take part. After discussing Teddi Fishman's definition of plagiarism, which I would extend to include "without properly attributing the work" in point 3 and removing point 5 entirely, I gave a few examples of some of the different forms of plagiarism. These were followed by screenshots of a few plagiarism detection systems that have complicated reports or report essentially meaningless numbers.

One important point that is often overlooked when using such systems is that they all suffer from both false positives as well as false negatives: This is an inherent problem with attempting to determine plagiarism using software. Quotations are difficult to detect reliably, especially if they are only indented; literature references should of course be similar to references used in other papers; and some systems begin to mark anything longer than 6 or 7 words as text similarity. All of these can be the source of a false positive, in addition to simple programming errors, which I have also seen. The other side of the coin is the false negatives, and they are quite simple to understand: If the software does not have access to a source, it will not be able to determine that it is indeed a source. Translated text, for example, is next to impossible to identify with software, as well as non-digitized content.

I then discussed the small, general tools that can be used to manually detect and document plagiarism. After a few examples of documented plagiarism from historic cases and from current cases at VroniPlag Wiki, I closed by asking some ethical questions that I include in my forthcoming chapter on plagiarism detection software for the "Handbook of Academic Integrity":
  • Is it necessary to find all the plagiarism in a text?
  • Is it ethical for a university to use plagiarism detection software?
  • Is it ethical for a university to use plagiarism detection software as a formative device?
  • Is it ethical for a university to offer plagiarism detection software for teachers to use?
  • Is it ethical for a university to offer plagiarism detection software for researchers to use?
We had a good discussion afterwards unfortunately, there was no time to linger on and talk further over a cup of coffee. I do hope that those who were present can serve as multiplicators, explaining to their peers that there is no magic silver bullet software for finding plagiarism, just a number of useful tools, large and small, that all incur a cost of time and effort to use.

[1] Causa Schavan (n.d.) Articles about "Zitat und Paraphrase." [Blog]. Retrieved December 1, 2014, from https://causaschavan.wordpress.com/?s=Zitat+und+Paraphrase
[2] Erbloggtes. (n.d.) Articles about "Zitat und Paraphrase." [Blog]. Retrieved December 1, 2014, from https://erbloggtes.wordpress.com/?s=Zitat+und+Paraphrase
[3] Dannemann, G. (2013, March 3). Die Ex-Ministerin und ihre Unterstützer: Schavanzentrisches Weltbild. Retrieved December 1, 2014, from http://www.tagesspiegel.de/wissen/die-ex-ministerin-und-ihre-unterstuetzer-schavanzentrisches-weltbild/7863836.html
[4] Ebert, T. (2014). Sag mir, wie hältst Du es mit dem Plagiat? Von Elisabeth Ströker zu Annette Schavan. Merkur, 68(12), 1070–1080.