NAVIGATING P-VALUES AND NULL HYPOTHESIS TESTS: INSIGHTS AND RECOMMENDATIONS
Abstract
The American Statistical Association (ASA) has addressed the long-standing concerns surrounding conventional P-value hypothesis testing by formulating a set of six principles, outlined in a 2016 publication by Wasserstein and Lazar. These principles aim to clarify the proper definitions and applications of P-values in hypothesis testing, offering significant benefits to the scientific community. They emphasize that P-values indicate the extent of data incompatibility with a specified statistical model but do not measure the probability of the hypothesis being true or the data arising solely from chance. Furthermore, these principles stress that scientific decisions should not solely rely on specific P-value thresholds, underscoring the importance of complete reporting and transparency in statistical inference. Additionally, P-values do not measure effect size or result significance, nor do they independently provide substantial evidence for a model or hypothesis.
These logically and mathematically sound principles are expected to play a pivotal role in resolving debates about the utility of P-value hypothesis tests. Their adoption promises to rectify misconceptions in textbook explanations, classroom instruction, and scientific paper interpretations regarding hypothesis testing. This transformative information has the potential to eliminate flawed thinking and language associated with P-value null hypothesis tests, assuming their continued use in research and practice.
Keywords:
P-values, hypothesis testing, statistical inference, scientific decision-making, effect sizeDownloads
Published
Issue
Section
How to Cite
License
Copyright (c) 2023 Sarah Elizabeth Reynolds, Michael Jonathan Nguyen
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
ASA. (2016), www.amstat.org/asa/files/pdfs/P-ValueStatement
Cohen, J. (1988), Statistical Power Analysis for the Behavioral Sciences (2nd Ed.). Hillsdale, N.J.: L. Erlbaum Associates.
Cohen, J. (1990), “Things I Have Learned (so far).” American Psychologist, 45(12), 1304-1312.
Erdfelder, E., Faul, F., & Lang, A. (2009). G*Power (Version 3.1.2)[Computer program], http://www.psycho.uniduesseldorf.de/aap/projects/gpower/
Glass, G. V., and Stanley, J. C. (1970), Statistical Methods in Education and Psychology, Englewood Cliffs, N. J.: Prentice-Hall, Inc. Hogg, R. V., Tanis, E. A., & Zimmerman, D. L. (2015), Probability and Statistical Inference (9th Ed.). Boston, MA: Pearson.
Kirk, R. E. (1996), “Practical significance:A Concept Whose Time Has Come.”Educational and Psychological Measurement, 56(5), 746-759.
Marshall, D. D., Falley, B. N., & Hamner, M. S. (2015), “Hypothesis Testing and Statistical Confidence: An
OverdueObservation on the Efficacy of a Hypothesis Test.” American Reviewof MathematicsandStatistics, 3(1), 22-26. DOI: 10.15640/arms.v3n1a3.
Moore, D. S., McCabe, G. P., & Craig, B. A. (2009), Introduction to the Practice of Statistics (6th Ed.), New York: W. H. Freeman and Company.
Rossman, A. J., & Chance, B. L. (2012), Workshop Statistics: Discovery With Data (4th Ed.). Hoboken, NJ: John Wiley and Sons. Schervish, M. J. (1996), “P Values: What They Are and What They Are Not.” The American Statistician,50(3),203-206.
Schmidt, F. L. (1996), “APA Board of Scientific Affairs to Study Issue of Significance Testing, Make Recommendations.” Score, 19, 6.
Schmidt, F. L., & Hunter, J. E. (1997), “Eight common but false objections to the discontinuation of significance testing in the analysis of research data.” In L. L. Harlow,
S. A. Mulaik, & J. H. Steiger (Eds.), What If There Were No Significance Tests? (37-64). Mahwah, N. J.: Lawrence Erlbaum.
Starnes,D.S.,Yates,D.,&Moore,D.S.(2011),StatisticsThroughApplications. NewYork,NY:W.H.Freeman andCompany.
Wasserstein, R. L., & Lazar, N. A. (2016), “ASA Statement on Statistical Significance And P-Values.” The American Statistician, DOI: 10.1080/00031305.2016.1154108.
Wilkinson, L., and the APA Task Force on Statistical Inference, APA Board of Scientific Affairs. (1999), “Statistical Methods in Psychology Journals: Guidelines and Explanations.”American Psychologist, 54: 594-604.
