P-hacking is a term used to describe various techniques that researchers can use to increase the chances of finding statistically significant results in their study, even if the results are not actually meaningful. This is a form of data manipulation that can lead to the publication of false positive results.
In a study that compares the effectiveness of two different physiotherapy interventions for rotator cuff-related shoulder pain, a researcher may conduct multiple analyses of the data, only reporting the ones that show a statistically significant difference between the two interventions. This is p-hacking because the researcher is selectively reporting only the results that support their hypothesis while ignoring the ones that do not.
Another example of p-hacking in physiotherapy research is when a researcher conducts multiple comparisons between different interventions but does not adjust for these multiple comparisons. This can lead to an increased chance of finding statistically significant results by chance, even if the interventions are not truly different.
By selectively removing some of the data, the researcher is effectively “p-hacking” the results, as it increases the probability of finding a statistically significant difference between the two interventions, even if there isn’t any real difference. The researcher might then present the data as if it were a fair comparison, which can lead to inaccurate conclusions and mislead future research and treatment. This is just one of the ways to do this.
It’s important to note that P-hacking can occur unintentionally and can be the result of a researcher’s lack of understanding of statistics or the pressure to publish positive results. But it can also be a deliberate act, driven by a desire to achieve a specific outcome. To avoid p-hacking, researchers should pre-register their study design and analysis plan, report all the results, and use appropriate statistical methods to adjust for multiple comparisons. Also, the use of replication studies and independent replication of findings can help to increase confidence in the results.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking”. Journal of Experimental Psychology: General, 143(2), 534-547.
Daniele Fanelli (2012). Negative results are disappearing from most disciplines and countries. , 90(3), 891–904. doi:10.1007/s11192-011-0494-7
Lakens, D. (2019). The value of preregistration for psychological science: A conceptual analysis. 心理学評論, 62(3), 221-230.
Storey, J. D. (2003). The positive false discovery rate: a Bayesian interpretation and the q-value. The annals of statistics, 31(6), 2013-2035.
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., & van Aert, R. C. (2016). M., & van Assen, MALM (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7.
Like what you're learning?
Use the assessment app
Over 300 orthopedic physical assessment tests
Statistics, basic assessments, and screening tests included
What customers have to say about the Assessment E-Book
The Assessment E-Book This book helped me in my studying for my exam and in assessing my first patients. Awesome! Also for beginners!
The Assessment E-Book It’s an amazing Compilation! Congrats to all the work you have put in there. You’ll propably find all the test’s you’ve been looking for with propper explaination and source to doublecheck for you self. definetly a must have for every student, but it will also help an experienced practioner. Im looking forward to the lifelong updates on the topics.
Great work, guys
The Assessment E-Book A must-have for all physiotherapists, osteopaths and manual therapists. The authors conducted an extensive research on assessment tests in manual therapy. I find it very easy to read. The more I read the more I learn. Thank you!
The Assessment E-Book This book is great! It is very structured and detailed. It works extremely well on my Macbook and iPad.
The Assessment E-Book The best way to spend 80euros. Totally worth it. The amount of work you put behind this must have been absolutely huge. Every physical or physiotherapist should own it.
Congrats guys you’ve done an incredible job.
I’ve learnd a lot of new things and my approach to therapy in general have totally changed.
In one word: amazing. Keep going guys ! Best wishes from france.
This content is for members
Create your free account to gain access to this exclusive content and more!
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.