Comments and Corrigenda in Scientific Literature
How self-correcting is the written record of scientific and engineering endeavors?
Errors Are Inevitable
It comes as no surprise that the scientific literature contains mistakes, because an integral aspect of scientific research is to cope with errors. The types of possible mistakes vary with the purpose of the work. Thomas Kuhn identified three activities in science per se: establishing facts, reconciling facts with theory and articulating theory. A similar spectrum of work occurs in research into engineering processes or medical treatments. All these activities involve choices that are susceptible to error. For example, measurements are subject to fluctuations, and phenomena generally must be observed indirectly, so the quality of the selected data may be poor, or the chosen transformation of the data to the quantity of interest may be inappropriate.
Scholarly journals exhibit a variety of responses to errors in research articles (see the table at right). The peer review process undoubtedly detects some errors but journals supply little information about rejection rates and reasons. (Acceptance for publication depends on correctness, relevance and significance. Reviewers offer opinions about the correctness of articles, but they cannot be expected to duplicate the work to be certain.) Authors themselves report the majority of known mistakes and correct them in corrigenda and errata (see the top graph below right). Errors found by readers may appear in formal “comment on” and “reply to” exchanges. Identifying errors in complicated reasoning is itself a type of research that may result in separate articles. For example, the present article disputes the sometimes-pessimistic assessment of scientific error rates by supplying more data and a broader context. Finally, evaluating the merit of data and theories is intrinsic to scientific discourse. In recent decades the annual incidence of all three types of corrections (corrigenda and errata, comments and replies, and other disputatious material) has been nearly constant at about 1.6 percent of full-length journal articles, which at present amounts to almost 25,000 corrective articles per year.
It is misleading to impugn the scientific literature by focusing on fraudulent research as the Wall Street Journal did in 2011. Forced retractions number only a few hundred annually by the newspapers’s own estimate, which is vanishingly small compared to the tens of thousands of corrections. Some retractions do involve fraud that can be difficult to prove. However, the journal did not explain that many retractions are prompted by simpler chicanery such as plagiarism or unapproved authorship. These transgressions are easier to notice now that documents are available through electronic media, which may account for the recent increase in the still very small numbers of retractions.