Subscribe
Subscribe
MY AMERICAN SCIENTIST
LOG IN! REGISTER!
SEARCH
 
Logo IMG

MACROSCOPE

Do We Really Need the S-word?

The use of “significance” in reporting statistical results is fraught with problems—but they could be solved with a simple change in practice

Megan D. Higgs

New Twist, Old Debate

The dangers of implementing arbitrary p-value cutoffs and the importance of distinguishing between statistical significance and practical significance are now well recognized by scientists. Andrew Gelman and Hal Stern address the common error of making comparisons based on statistical significance in a 2006 paper for The American Statistician, titled “The difference between ‘significant’ and ‘not significant’ is not itself statistically significant.” I contend that in addition, our tendency to repeatedly use the same wording to explain statistical results masks potential improvements in inference corresponding to recognition of these issues. My goal here is not to revisit ideas that have been described eloquently and repeatedly over the past decades (by James O. Berger and Donald A. Berry in a 1988 article for American Scientist, and by Jacob Cohen in a 1994 article for American Psychologist, among others) but instead to propose a simple change in wording. I suggest we replace the word significant with a concise and defensible description of what we actually mean to evoke by using it.

For example, if we mean that the two-sided p-value is less than 0.05, then let us say just that. Let the reader judge whether that result is to be deemed significant by our modern scale and by their knowledge of the science. If, instead, we mean the result is in fact practically important, let’s say as much and clearly communicate our justification for doing so. If we believe our research is important, let us convince our critics through our choice of other words. We can use the p-value to back up our arguments, without appealing to the s-word. If we do not feel forced to label results associated with larger p-values “nonsignificant” or “insignificant,” it may even help curb the long-standing publication bias. By replacing the s-word with defensible statements, we can easily and quickly clean up the often sloppy dissemination of scientific results.




comments powered by Disqus
 

EMAIL TO A FRIEND :

Of Possible Interest

Engineering: The Story of Two Houses

Perspective: The Tensions of Scientific Storytelling

Letters to the Editors: The Truth about Models

Subscribe to American Scientist