Do We Really Need the S-word?
The use of “significance” in reporting statistical results is fraught with problems—but they could be solved with a simple change in practice
What Does It Really Mean?
What exactly does the word significant mean in statistical contexts, if it does not mean “important” or “meaningful”? When someone labels a result as statistically significant, does it merely mean that the p-value (a value calculated to quantify evidence against a hypothesis) is less than 0.05, or that the 95-percent confidence interval does not include 0? If so, perhaps it is time to ask whether we really need to use a word that carries substantial meaning in our day-to-day language to describe something so simple.
Did the people who introduced the word’s use in statistics intend for it to be interpreted according to its current everyday meaning? The answer is not simple. In his 2001 book The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century, David Salsburg contends the word carried much less weight in the late 19th century, when it meant only that the result showed, or signified, something. Then, in the 20th century, significance began to gather the connotation it carries today, of not only signifying something but signifying something of importance. The coinciding of this change in meaning with a steady increase in its use by more scientists with less statistical training has had a big impact on the interpretation of scientific results. My sentiments echo Salsburg’s: “Unfortunately,” he writes, “those who use statistical analysis often treat a significant test statistic as implying something much closer to the modern meaning of the word.”