Do We Really Need the S-word?
The use of “significance” in reporting statistical results is fraught with problems—but they could be solved with a simple change in practice
The Significance of Significant
Perhaps you are thinking this is a trivial suggestion, a mere matter of semantics. Can such a small change manifest real improvements? There are not many easy ways to improve scientific inference, but I believe this is one of them. The significance of the word significant should not be easily dismissed. The word carries strong connotations from its everyday usage that are difficult, if not impossible, to let go of simply because we find ourselves interpreting statistical results. In practice, it is all too easy to slide from the constant vigilance required to maintain these real semantic differences amid the multitude of assumptions and procedural details involved in statistical analysis.
The s-word is, of course, not limited to discourse among scientists. It finds its way into media reports on research, which are read by the public, most of whom have far too little statistical background to understand the different meaning of the word in the context of statistical results. The rest of a sentence may contain incomprehensible statistical jargon, but the s-word is recognizable and easily digestible—and the meaning attached to it is, of course, its everyday sense: important and meaningful. Thus, “important and meaningful” is the message sent to an audience without the background to understand what led to the printing of that weighty word.
The journal Science suggests that potential authors “use significant only when discussing statistical significance,” acknowledging the subtleties attached to its use and meaning in the context of scientific research. I suggest we take a further step and omit its use in statistical contexts as well. If most of us are not capable of separating the statistical meaning from the everyday meaning, and if, as I argue, the word really is not needed to explain statistical results, why maintain our dependence on it? Let’s free ourselves to justify our statements more adequately and describe our results more wisely.
For readers with the background necessary to successfully critique results, we should provide the information they need to make their own informed opinions, based on sound reasoning and justification. Rather than giving in to the false dichotomy evoked by “significant” or “not significant”—a dichotomy most often based on arbitrary and hidden criteria—we should focus on why we believe results are (or are not) meaningful and important. Replacing the s-word in our writing and speech allows us the space to do just that.