The results are, as always, not exciting. Software just cannot find translation plagiarisms or plagiarisms that are taken from books. Teacher, however, are often very good at spotting "fishy" texts like this and can even find sources using search machines. Even removing all the test cases with this kind of plagiarism from our evaluation, we still had no software that was given the grade of "very good". The systems have gotten slightly better, however.
We also evaluated the usability of the systems - many have lots of problems in this area. A particular problem is the numbers reported - it is often not clear, what the numbers or percent values given mean. And the reports are sometimes not very useful or the system is difficult to use in a university setting. Having to submit papers one at a time and having to wait a longish period between each test is not acceptable.
One system - one of the worst - has forbidden us from giving their name. We just refer to them as XXXX. The ranking is:
Good Systems (Grade "B")
- 1: Copyscape Premium
- 2: Plagiarism-Detector
- 3: Copyscape Free
- 4: Urkund
- 5: Docoloc - PlagAware (new version)
- 7: Ephorus