Wednesday, August 15, 2012

The Case of Faheed Zakaria

The news from the States is full of stories about India-born Fareed Zakaria, an American journalist with Time and CNN who has been suspended in a plagiarism scandal. He has admitted to plagiarizing a number of paragraphs from a New Yorker article on gun control. His article, "The Case for Gun Control", is still online, as is of course the article by Harvard University history professor Jill Lepore, "Battleground America". CNN appears to have suspended him for similar plagiarism in a blog that has been taken offline.

I feel that this is a problem, to take stuff offline. I want to be able to examine the evidence for myself. My news feeder (Shameless plug for Highbeam) from the States led me to the Seattle Times' article "CNN's Zakaria sorry for plagiarism". There an example of the "plagiarism" is given, and I looked up, puzzled. This was statement of fact. The only text similarity was the name of the person quoted and the book quoted from.

I dug around for the articles and threw them at the comparison tool, SIM_TEXT, that we use on the VroniPlag Wiki.

Fareed Zakaria


Jill Lepore
Hmm. Wow. We have cases of plagiarism in Germany that have massively more copy and pasted that were found by their universities (BTU Cottbus and University of Heidelberg) to be only "technical weaknesses": Dd with 44 % of the pages affected: 28 -  35 - 48 - 103; Nk with 75 % of the pages affected (many from her doctoral advisor): 17 - 66 - 81 - 90.

I am, of course, of the opinion that both universities listed above are in error and that these theses are grave plagiarisms, the doctoral titles should be rescinded.Fareed's sin is the selection of facts and quotes, that were lifted from the New Yorker. It is right to suspend him, give him some time to think about what he did here. I don't know about other texts that might contain similar material. But this is far, far less than the Jayson Blair case. But should he be fired? What do my readers think?

Update 2012-08-19: The Chicago Tribune reported on Aug. 16. that Zakaria has been reinstated both at Time and at CNN. I'm quite glad, as it would have been very difficult to explain large-scale copy and paste in Germany being okay, and a paraphrasing error in the US costing one their job.

Saturday, August 11, 2012

Inquiry results published

The Ludwigshafener hospital has published a press release about the results of the inquiry into the research of Joachim Boldt. He was the former "retraction king", having had to retract 88 of his publications. He has since been "de-throned" by Yoshitaka Fujii (as reported by Retraction Watch).

Boldt already left the hospital in November of 2010, after criticism of his research grew too loud to ignore. The board examining his papers needed a good 18 months to go through everything and determine that "in a large number of the studies investigated, the conduct of research failed to meet required standards. False data were published in at least 10 of the 91 articles examined, including, for instance, data on patient numbers/ study groups as well as data on the timing of measurements."

They try and play it down as being mostly a procedural thing, and are relieved that no patients came to harm. They promise that they have fixed procedures.

But I still have a few questions:
  • Where did the money for this research come from? Was this government money? Was it from a pharmaceutical company?
  • Has anyone used the since-withdrawn studies? That is, did anyone else quote his papers or try and replicate the experiments?
  • Is Boldt still permitted to practice medicine?
  • The hospital states that they will be monitoring future clinical studies - how will they be encouraging people to speak up about falsification of data? That has nothing to do with monitoring, which brings up notions of even MORE paperwork. How are they going to foster an environment in which people can question the research being done without fear of retaliation.
  • Why did this take 18 months?

Wednesday, August 8, 2012

Plagiarism Vocabulary

I was digging around looking for papers on how exactly plagiarism detection software works, when I was directed to "Classifications of Plagiarism Detection Engines", published in 2005 by Fintan Culwin and Thomas Lancaster in the online journal ITALICS 4(2). As a software engineer I was quite enjoying digging out the ancient papers quoted there about detecting plagiarism in programming exercises in FORTRAN. Ahh, those were the days, my first programming language.... and what a great use of Halstead's Software Science!

Then I realized that Thomas Lancaster had submitted his dissertation "Effective and Efficient Plagiarism Detection" in 2003 to the London South Bank University, London, UK. He has an excellent, detailed classification of the plagiarism detection systems available at that time, and a good overview of a lot of the technical papers that are to be found on the topic. The glossary alone is a joy to read, and I have asked for and received permission to repeat portions here. There are also a number of papers that Lancaster has published or prepared on the topic included in the appendix. Lancaster focuses in the thesis on a four-step process for determining plagiarism:
  • Collection stage - The first stage of the four-stage plagiarism detection process. This
    is where students submit their work to an electronic system so it can later be analysed for similarity.
  • Analysis stage - The second stage of the four-stage plagiarism detection process. Here all submissions are compared with each other (for intra-corpal plagiarism detection) or the external sources such as the Web (for extra-corpal plagiarism detection) to find submissions that are similar to each other or the Web sources.
  • Confirmation stage - The third stage of the four-stage plagiarism detection process. Here a tutor checks the pairs of student submissions that have been judged to be similar to see if they represent plagiarism or they represent legitimate shared citations or false hits. The tutor decides which pairs will go on to be investigated further.
  • Investigation stage - The fourth and final stage of the four-stage plagiarism detection
    process. This is where pairs of similar submissions have been found and they have
    been confirmed by human inspection to be similar and possible cases of plagiarism. In this case further evidence is collected, such as student interviews and marked up
    copies of the submissions and penalties are given.
My selection from the glossary (my favorite definition is in blue):
  • Academic plagiarism - Plagiarism carried out by academics, for instance copying journal articles and submitting them as their own work for possible career development.
  • Attribute counting metrics - A count of some property of a single document which
    might involve tokenisation. This has been redefined to remove the inconsistencies
    from the literature but is not considered a sensible classification.
  • Authorship attribution - The branch of linguistics that aims to calculate the author of
    a work based on knowledge of works by other known authors. This is not appropriate for plagiarism detection since there is no corpus of known work by a given student.
  • Characters Metric - A simple metric that measures the number of sequences of
    characters of a chosen length two documents have in common. 
  • Cheating - Unauthorised behaviour that is going against student etiquette when trying for an academic award or to gain an advantage over other students. Examples include plagiarism, use of cribs in exams and paying someone to complete an assignment specification on your behalf. 
  • Closeness Calculation - A computationally part of automated plagiarism detection
    where a single number is generated from a number of different metrics to decide how similar two submissions are.
  • Contractive plagiarism - Plagiarism where the source is larger than the copy and
    hence the source has been reduced in some way to create the student submission.
  • Corpal Metrics - A multi-dimensional metric that is a measure of a property of an
    entire corpus, for instance the proportion of submissions using a given keyword.
  • Collusion - Where two students discuss and work on an assignment specification
    together and complete elements of their final submissions together. This might be
    judged to be intra-cor[p]al plagiarism.
  • Direct copy - Two student submissions that are identical to one another with no
    attempt at disguise. One is a direct copy of the other. 
  • Disguise - Where a student has attempted to change a source and hand it in as their
    own submission so that the use of the original source won't be noticed.
  • Expansive plagiarism - Plagiarism where the source has been extended, either by
    adding new thoughts or adding filler words and phrases to make a student submission. 
  • Extra-corpal plagiarism - Plagiarism where the plagiarism source is outside the
    corpus of student submissions, for instance a Web site or material from a book.
  • False hits - Pairs of submissions that are ranked high enough for a tutor to investigate them but are judged to be dissimilar, thus being a waste of tutor time.
  • Free text plagiarism - Plagiarism that has been done in natural language, for instance, altering the words of another writer and presenting it as your own work.
  • Hybrid metric systems - A system that a combination of both attribute counts and
    structure metrics to find similar submissions. This has been defined to remove the
    inconsistencies from the literature but is not considered a sensible method of
    classification. 
  • Intra-corpal plagiarism - Plagiarism entirely within a corpus, primarily meaning two
    students who have copied from one another.
  • Missed pairs - A pair of submissions that contains plagiarism but is not automatically ranked in the upper portion of an ordered list of similar pairs and hence not investigated further by a tutor. 
  • Mosaic plagiarism - Plagiarism where chunks from different sources are used and rearranged in a way that could be considered like a mosaic is created from combining and arranging different pictures.
  • Multiply sourced - A student submission or external source that has been used in
    multiple student submissions.
  • Ostrich plagiarism policy - Where an academic institution states that plagiarism does not exist in their institution and has no formal way of dealing with it.
  • Paraphrasing - Using the ideas of another but rewriting them in your own words
    without suitable and continual acknowledgement. 
  • Plagiarism - Taking the words or ideas of another and presenting them as your own
    without suitable acknowledgement.
  • Proactive plagiarism policy - A policy of an academic institution where plagiarism is actively sought out on a regular basis, perhaps by using automated detection methods and cases are followed up when they are found.
  • Professional plagiarism - Plagiarism in a professional setting, for instance copying an internal report or company Web page from another source or using a service that
    writes standard CVs or job applications. 
  • Reactive plagiarism policy - The academic policy where plagiarism is not actively
    sought out but is taken seriously and followed up when it is identified during the
    course of marking.
  • Similarity - Where two submissions have words or ideas in common they are said to
    be similar. When they have been looked at by a tutor they may also be judged to be
    plagiarised.
  • Singularly sourced - A plagiarism source that has been copied from once only.
  • Source code plagiarism - Plagiarism of source code submissions, where two students
    have handed in programs where one has been derived from the other in some way.
    Detecting this is a well understood area since the constrained language reduces the
    number of possibilities that must be checked.
  • Structural Metrics - A metric that measures a property of one or more submissions
    where knowledge of the structure of the documents is needed.
  • Synthetic corpus - A corpus of documents that have been generated using synthetic
    means by taking sequences of words or characters in a known and defined order.
  • Thesaurising - A technique for plagiarism where words in a source are replaced by
    synonyms or changed in such a way that the submission makes the same points but
    the intention is that the plagiarism will not be discovered.
  • Visual metrics - A metric which is a based on some property of the similarity
    visualisation that would be generated for a given pair of student submissions.
  • Words Pair Metric - A simple metric that measures the number of sequences of word
    pairs in common between two documents. Identified as the most effective simple
    metric.
I find it immensely helpful to have terms that are generally understood when we are speaking about plagiarism. I would personally use "Synonomizing" instead of "Thesaurising" (which I can't pronounce). I also like the focus on the process of determining plagiarism and not the products - the software that is used in the process. Lancaster's focus in the thesis is on intra-corpal plagiarism and the visualization of similarity. It is well worth a read, if you are working in this area.

Friday, August 3, 2012

Scientific ghostwriting

The online journal Laborjournal has an interesting editorial on scientific ghostwriting (in German) that includes a number of legal cases in Germany and some interviews with ghostwriters. The author also contacted a company that says that they do not write doctoral dissertations. Writing as a medical student, they inquired as to the costs for a medical dissertation on a particular topic. They were given an offer of 5900 € for such a thesis, much less than the 7-10.000 € normally taken for a Diploma- or Masters-Thesis of about 100 pages.

The also include an interview with the authors of PlagScan, a so-called plagiarism detection system. They do concede that software only finds about 60-70 % of plagiarism, as we have demonstrated a number of times over the years. But the authors are confident that they can soon write a system that can even detect ghostwriting.

I'm not of that opinion. I know that I write differently when I am blogging than when I am writing a scientific paper. I have different styles. Trying to detect ghostwriting with software is just not possible. However, we as professors do have a chance to detect this if we have our students regularly submit drafts so that we can see progress. This, however, takes quite a lot of time, giving feedback on drafts. We have to meet with the students and ask hard questions like: where did you find this paper? We don't have that in our library....

And time is the problem. I don't know about you, but my day only has 24 hours. The more students I have, the less time I have to work with each one individually. If we are not reading the theses, why should the students write them? This creates a culture in which people can get away with submitting ghostwritten papers. Of course, what are they going to do when they have to write after their studies? They've never learned to do research, to structure, to write. So that will either ensure an expensive market for ghostwriters, or they will end up getting fired - if there is someone with the guts to call them out on their lack of skills.

The university system needs reforming, and it needs it yesterday. And not just in Germany, it would seem.

Friday, July 27, 2012

VroniPlag Wiki - Case 28, Theology

I can barely keep up here ... Earlier this week, VroniPlag Wiki published case 28,  a dissertation in theology, approved by the University of Tübingen. In this rather bizarre case (but in a way, they are all bizarre in their own ways) the submitter already has a dissertation in medicine. And since he is working on spiritual healing, he just includes that dissertation here, without reference.

VroniPlag Wiki disregarded this in its page count, as there is a slight chance that the advisors accepted this. The medical doctorate is not included on the CV for Tübingen, but the Dr. Dr. is proudly displayed on web pages where the submitter is active.  There are about a quarter of the rest of the pages with plagiarism. A third of these pages include plagiarism on over 75 % of the pages.

Nature wins libel trial

Nature won a long-running libel suit in July brought against them in England by Mohamed El Naschie on account of the following article (which is now back online) by Quirin Schiermeier : Self-publishing editor set to retire: Criticism grows over high number of self-penned articles in physics journal. (Published online 26 November 2008 in Nature 456, 432 (2008) | doi:10.1038/456432a)

Nature has both published a report on the outcome of the suit and an article by the author of the original article describing the process. The main focus of the 2008 article had been about how El Naschie had been self-publishing in an Elsevier journal (Chaos, Solitons and Fractals), of which he was editor-in-chief, at a very unusual rate, and that he was retiring from this position. Schiermeier was looking into how exactly this enormous publishing output was happening.

Schiermeier was also curious to know more about this person, as there were a number of blogs that were criticizing him, for example the El-Naschie Watch, but that were being threatened with litigation. Threats of legal action in science are often in inverse proportion to the scientific value of the contested results.  Schiermeier notes:
Still, unlike other science journalists, such as Simon Singh, who had previously had to go through the libel ordeal on their own, I was in the comparatively comfortable situation that my employers had the resources, the stamina and the willpower to take the case on.
Síle Lane, a campaigns manager at Sense About Science in London, is reported by Nature as noting that "[i]ndividuals who lack the financial support of an institution capable of defending against a libel action will probably look at the amount of time and money it took to defend this case and decide that they shouldn’t speak out."

The judgement ([2012] EWHC 1809 (QB)) is available online at Nature and its 91 pages make a fascinating read. I think it should be required reading for anyone dealing with the scientific publishing process, peer review, impact factors, or junk journals and vanity publishing.

The Honorable Mrs. Justice Sharp, DBE, the judge for this suit, should have been made DBE for her thorough analysis of the situation, but she already is. She looked at the norms of scientific publishing, the peer review system, an the problem of journal self-citation by calling on expert witnesses.

In paragraph 90 we see that El Naschie published 58 papers in 2008 (the year he was being forced to step down as editor). That would be more than one paper a week, and the next most frequent published author at the journal that year had only 9 publications. The next paragraphs contain some fascinating statistics comparing the publication record of other editors-in-chief of top-rate journals. 

In paragraph 114 in response to El Naschie's statement that no peer group existed for assessing his theories, so his only recourse was to self-publish, Sharp states: "We are not living in the age of Galileo. Scientists or those with a keen interest in science as informed amateurs are free to publish and thus disseminate their ideas to the outside world: and in the age of the internet, there are many platforms by which they can do so."

Paragraph 120 details the expert witness of Professor Neil Turok, professor for applied mathematics and theoretical physics. His gruesome task was to actually read and analyze all 58 papers published by El Naschie in 2008 in the journal in question. What an enormous amount of effort!
120.
Through Professor Turok, it is said by the Defendants that on analysis, the 58 papers or articles contained the following defects:
i) A failure to define terminology and concepts, including in particular a failure to present the principles and equations of “E-infinity theory” and the predictions which are said to be deduced from it;
ii) Strongly expressed conclusions, unsupported by any, or any intelligible process of logical reasoning; in particular, the repeated unexplained reliance on numerical coincidences in support of the assertion that the Claimant’s “E-infinity theory” is correct;
iii) Statements which are meaningless or obscure, even to a readers with expertise in the field of theoretical physics;
iv) Statements which are simply wrong;
v) Elementary errors of spelling and grammar;
vi) A lack of any, or any substantial, contribution of new knowledge to the field;
vii) An excessive degree of citation of other articles written or co-written by the Claimant, in particular in order to justify assertions which should have been
supported by self-contained argument or references to the work of independent authors (the articles published by the Claimant in CSF in 2008 contained approximately 301 citations of his own articles in CSF, including citations of “in press” articles: i.e. those articles which were due to be, but which had not at the material time, been formally published);
viii) The use of those articles to advertise other articles by the Claimant.
The judgement then goes on to examine each and every one of these defects. In paragraph 122 already Sharp notes: "But to paraphrase Samuel Johnson, you do not have to be a carpenter to see when the legs of a table are uneven, or even to see that it has no legs at all." On questioning about peer review, Turok states in paragraph 169: "Peer review is essentially what separates theoretical physics from chaos."

It goes on, paragraph after paragraph detailing the bizarreness of the case. This includes the witness Anke Boehm who worked for El Naschie insisting that it was not the policy of the journal to keep reviews and papers once they had been published (paragraph 182) and then speaking of "deleting" the reviews when El Naschie insisted that everything was only done on paper and the he does not use email.  Paragraph 290 notes that El Naschie admitted that some of the people who were speaking for him or working at the journal just did not exist.

The judgment also includes material from Elsevier that makes it clear that even they found the journal problematic (paragraph 228). An internal review of the journal is quoted as stating “Although the journal has a growing impact factor, we are concerned that this is too dependent upon self-citation […] The journal appears to have a high acceptance rate with many accepted papers coming from China." I don't normally praise Elsevier, but I am glad to see that they took the initiative here in terminating the editorship of El Naschie.

I wish to thank Nature for standing up for the name of science reporting and against junk journals! It cost them much time and money, I do hope that they can get the money for the lawyers back from El Naschie, which will probably not be an easy task.

Thursday, July 26, 2012

Danish Neuroscientist found guilty of misconduct

It has been a long and complicated story. The newspaper for the University of Copenhagen published "Penkowa for Dummies" in the spring of 2011, which gives a good background to the skirmishes of the past year.

Milena Penkowa, since resigned from her position as professor at the University of Copenhagen, had been the shooting star in research in Denmark. Sure, there had been some dark mutterings about her dissertation, as Nature reported in January 2011 on the basis of a report in Weekendavisen by Poul Pilgaard Johnsen. The thesis was first rejected, then accepted on a second review.

Then some of her research had to be retracted, as it was not replicable. Retraction Watch lists 2 papers out of over 100 that were retracted and two letters of concern that have been published in journals. And some research money had been used to pay for lawyers and restaurant visits. The university promised to start a full-fledged investigation.

The investigation was conducted by a group of foreign experts, Weekendavisen reports in its issue from July 20 (they are, unfortunately, not online). Hans Lassmann from the Medical University in Vienna chaired the committee that examined the 102 publications by Penkowa. 23 of the papers were deemed to be unnecessary to examine more closely. In 26 of the remaining 79 papers the committee determined scientific irregularities. Out of these 26 papers, 16 were determined to be scientific misconduct.

The misconduct has involved, among other things, mismatches between the number of lab animals in the papers and in the animal registers, problems with quantitative data, and problems with pictures (for example just turning the pictures as evidence of new work). The data archives were chaotic and filled with errors.

Berlingske, another Danish paper, reported July 15 by Claes Lautrup on the results of the investigation. Not only Penkowa, but also the University of Copenhagen were found to be guilty of misconduct. Penkowa still claims innocence, but notes that her career is ruined already, there is nothing left for her to give up. The university has refused to comment. By law, the report must be made public and is scheduled for August 7.

The university has started a new program for PhD students that includes mandatory courses in research ethics, good scientific practice, record keeping, and documentation, according to Weekendavisen.

The university must now decide what to do with the 16 articles, whether to contact the journals for retraction. They have already taken Penkowa to court on a case of defrauding university funds, which she blamed on a student. She lost the case and has been fined. She currently is running a company for advising patients with neurological problems and writing a book about dog psychology, Weekendavisen writes. 

There is also a political aspect to the drama, involving the rector of the university and the former federal minister of science, but that is just an added layer of complication to an already tangled web.

(If there are any translation errors in my summary, please let me know! -dww)