What's a P-Value? | Statistics

What’s a P-Value? | Statistics
In simple terms the p-value expresses how surprised you are with the data, assuming there is no effect. The lower the p-value, the more incompatible the data seems with your model (i.e. the assumption that there is no effect).
Eg.
Treatment A is compared to treatment B, you assume there is no effect or no difference; you expect the null hypothesis to be correct. You perform the test and get a p-value of 0.02. That means that the data you gathered is pretty surprising, considering that you assumed the groups would not differ.
The p-value exists to protect yourself from randomness. If you perform a study, chances are that the effects you see are just random— or data noise, as we call it. That’s why you might see noticeable differences in the mean values between groups, but no statistically significant effect. It can go the other way around as well. A study might show a non-significant result, but there might be a true effect; perhaps because the sample size is just too small.
P-values are influenced by a few different factors: sample size, effect size, and the type of test with its assumptions.
Sample size: the bigger the group, the faster you’ll get statistically significant results with small differences— and vice versaEffect size: the bigger the effect size, the faster you’ll get statistically significant results, even with smaller groups— and vice versaType of test: a test gets more sensitive to differences with certain assumptions about for example the data distribution, independence of measures, homoscedasticity, one-sided vs two-sided, between-group vs within-group, etc.
Eg.Â
A huge study can find statistically significant results with even the smallest of effects. These effects might not mean a thing. This is where clinical significance comes into play.The original penicillin study used a tiny sample to make the data show that there are huge effects on eliminating bacteria.
The threshold for statistical significance most researchers use (i.e. p < 0.05) is just arbitrary. All things considered, it should change based on your study setup. If you really do not want false positive results (eg. a decision to undergo a life-threatening operation), you need a low threshold number. If you really don’t want false negatives (eg. diagnosing aggressive tumors), you need a high-powered study with subsequently a higher p-value threshold number. This illustrates the give-and-take relation between type 1 (α) and type 2 (ß) errors.
Do note that the p-value is derived from the data, not the theory. You cannot ‘prove’ your theory with a statistically significant effect. The only thing you can do is try to refute your theory with different studies, if it holds, your theory stands. This is falsification.
References
ALL ORTHOPEDIC TESTS IN ONE PLACE
What customers have to say about the Assessment E-Book
- Vince199225/04/20The Assessment E-Book This book helped me in my studying for my exam and in assessing my first patients. Awesome! Also for beginners!Simon Pagitz06/04/20The Assessment E-Book It’s an amazing Compilation! Congrats to all the work you have put in there. You’ll propably find all the test’s you’ve been looking for with propper explaination and source to doublecheck for you self. definetly a must have for every student, but it will also help an experienced practioner. Im looking forward to the lifelong updates on the topics.
Great work, guys - Jordi Burrut27/10/19The Assessment E-Book A must-have for all physiotherapists, osteopaths and manual therapists. The authors conducted an extensive research on assessment tests in manual therapy. I find it very easy to read. The more I read the more I learn. Thank you!Josh07/07/19The Assessment E-Book This book is great! It is very structured and detailed. It works extremely well on my Macbook and iPad.
- Polo_soa15/02/19The Assessment E-Book The best way to spend 80euros. Totally worth it. The amount of work you put behind this must have been absolutely huge. Every physical or physiotherapist should own it.
Congrats guys you’ve done an incredible job.
I’ve learnd a lot of new things and my approach to therapy in general have totally changed.
In one word: amazing. Keep going guys ! Best wishes from france.