The Cloze procedure - and plagiarism

Readability

The Cloze procedure was originated for testing readability of text in order to measure "the effectiveness of communication." The name "cloze" was given to the measurement.
It is pronounced like the verb "close" and is derived from "closure." The last term is one gestalt psychology applies to the human tendency to complete a familiar but not-quite-finished pattern ... (Taylor, 1953, p 415)
Taylor, W.L. (1953). Cloze procedure: A new tool for measuring readability. Journalism Quarterly, 30. 415-433 (D.H. Hill Libraries Call No. PN4700 .J7)

Brown, J.D. 1989 Cloze item difficulty. JALT Journal, 11, 46-67 The Japan Association for Language Teaching

To this day it is used by teachers to measure readability of text.

Plagiarism Detection

When I wrote the Fifth Word test (and made it freely available) for plagiarism detection, I wasn't aware of the Cloze procedure - but was aware of the information theory ideas of redundancy in context, and its use in predicting missing words. I took advantage of this to set up a challenge that would be much easier for an author than for a plagiarizer. Later I was told by a faculty member that she remembered something in reading theory along these lines - but I couldn't find more. Recently I was told about this procedure (thanks, Donna) and its use.

The earliest reference I can find about its application in detecting plagiarism: Glatt, B.S., & Haertel, E.H. (1982) The use of the cloze testing procedure for detecting plagiarism. Journal of Experimental Education, 50, 127-136.

From the Conclusion: "Our experimental results indicate that, in fact, the cloze procedure can be of substantial value in distinguishing plagiarized from non-plagiarized papers. ... due to the seriousness of the offense, accusations of plagiarism should not be made unless there is considerable cause for suspicion. ... The only definitive evidence of plagiarism remains a direct comparison of the the submitted work to the plagiarized source."

Note that the surname of the senior author is the name of one of the plagiarism detection services Glatt Plagiarism Services, Inc. That's because it's the same person!
This company offers a free "Self-Detection Test" which appears to use the Cloze procedure.

Another paper on this same topic is: An Evaluation of the Cloze Procedure as a Test for Plagiarism Standing, Lionel & Donald Gorassini 1986 Teaching of Psychology. Vol. 13, No. 3. 130-132. (found in ACADEMIC SEARCH FULLTEXT ELITE database AN 6384895)
Abstract: Two experiments that evaluate the effectiveness of the cloze test as a method for detecting plagiarism are reported. The first experiment employed student essays on the same topic, and required subjects to fill in missing words from either their own or another student's work. The second study repleated this design, but using a "plagiarized" essay on a new topic that had not been studied by the subjects. In both studies, cloze performance was significantly better in "nonplagiarizing" than "plagiarizing" situations. Used with due caution, the cloze test appears to be potentially helpful in detecting plagiarism."

Interestingly, this testing procedure has also been perverted by a paper mill to help prepare students for challenges! (This source says, "Essays are to be purchased for information purposes only and can be thought of as examples. They are meant to be used as reference, to get ideas for sources, thoughts on structure, and inspire creativity. They are not intended for plagiarism." It is left to the reader to interpret the effect of this disclaimer. :-)

The 5th Word Test for Plagiarism

This is a "challenge" test and, as originally conceived, would require the student to read the "holeized" material to the instructor who would follow using the original material or the original material with the 5th words in bold. This is a confrontational situation. It can be somewhat ameliorated by having the student write down words to fill in the blanks, as in this version. While this lessens the confrontation, it still must be done in the presence of the instructor, or in some other proctored environment. (Otherwise the student can simply fill in the blanks perfectly while looking at what was submitted.)

Lessening the confrontational aspect of this is good, but this increases the burden of grading. An improved version, suggested by Michael Censlive, who is on the faculty of Middlesex University, London, eases this burden by numbering the blanks, and then giving the instructor a numbered list of the words which were blanked out. This allows the student to write a numbered list of words to fill in the numbered blanks, and then the instructor can more easily compare the answers with the words in the original by comparing the two numbered lists.

Another suggestion for decreasing the confrontational effect via a different approach was made by Thomas Hardiman, of NC State University. He suggested that this test could be given to all of the students in the class. In that way, no student would feel singled out. This likely would be done during a class and so there would, in effect, be proctoring simply because the instructor would be present. The down side would be the necessity of preparing a paper/test for each student individually. This is eased by simply cut/paste from each student work into the window of the 5th Word Test and then cut/paste into a word processor page. If the tested portion of the work is only a portion of the page, then it can have the numbering at the bottom of the page, making it easier to take and to grade.

Note that while the 5th Word Test and the Originality Check (from tunitin.com) both try to detect plagiarism, they differ substantially in their basis.

The 5th Word Test measures the student's familiarity with the submitted written material, specifically the word choices of the author (also known as "style".) A student can fake this by becoming sufficiently familiar with the submitted work - even when there has been plagiarism.

The Originality Check compares the submitted work with a very large content database (including web pages, journal articles and student papers) and reports similarities. A student can evade detection by by using material which isn't in the database, e.g. by hiring someone to write new material.

So it appears that neither method is foolproof. It remains for the instructor to stay alert, notice uneveness in a student's writing and work with the students in a positive manner so the students see the importance of the assignments and are motivated to do their own work.


Copyright 2004, 2007, 2011, 2013, 2017, 2022, 2023 by Henry E. Schaffer     Comments and suggestions are welcome, and should go to hes@ncsu.edu
Licensed under the Creative Commons Attribution-NonCommercial 4.0 International Public License
https://creativecommons.org/licenses/by-nc/4.0/legalcode
Last modified 10/22/2023
Disclaimer - Information is provided for your use. No endorsement is implied.