Stats & Methodology

What's Statistical Power? | Statistics

Check our shop
P value wiki

What’s Statistical Power? | Statistics

The power is the long-term probability of a series of identical studies to detect a statistically significant effect (eg. p<0.05) if there is any. The probability of a type 2 error in a series of identical studies is one minus the power (1-ß, often 20%).

Eg.

One hundred studies are conducted within the same population with the same treatment A vs treatment B structure. The true treatment difference in real life between A and B is a 30% higher chance of full recovery in treatment A. When the stats are performed on these one hundred studies (same population, same variance, same standard deviation), on average about 20 studies will not show a statistically significant effect. This is the type 2 error rate, or false negatives— directly related to the statistical power (1-ß).

So to put it simply, an inadequately powered study will less often show a statistically significant effect, while there actually is a difference.

Power is influenced by a few factors, just like with p-values.

Sample size: bigger sample = more power (clearer differences between groups, fewer data noise)Variance: smaller variance = more powerEffect sizes: bigger effect sizes = more power (easier to spot by a test)Type of statistical test: some tests yield more power in exchange for more assumptions (there are no free lunches in stats)

It is crucial to understand, though, that the statistical power (eg. 80%) is there for one measurement tool, for one point in time, for one effect size.

So an underpowered study increases the risk of type 2 errors (false negatives), but, it increases the risk of type 1 errors as well (false positives), with inflated effects. This is called ‘the winner’s curse’. This is why you simply cannot throw multiple outcome measures at a sample size and measure at multiple points in time without letting your statistical power crash. Good researchers and clinicians know that secondary outcome measures are merely suggestive because the study is not powered for that amount of measures. You need new studies to confirm those suggestions.

I can imagine this sounds a bit counterintuitive. Let’s look at an example.

Eg.

You are lecturing a group of 200 students and decide to split them up into two groups. The aim of your study is to see if there are gender differences like more females in one group compared to the other. There’s no difference. You then look at eye color, hair color, length of their index finger, benchpress PR, QOL, age, amount of siblings, etc. Chances are you will encounter a statistically significant result somewhere. This is the multiple comparison problem.

References

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European journal of epidemiology, 31(4), 337–350. https://doi.org/10.1007/s10654-016-0149-3

Ingre M. (2013). Why small low-powered studies are worse than large high-powered studies and how to protect against “trivial” findings in research: comment on Friston (2012). NeuroImage, 81, 496–498. https://doi.org/10.1016/j.neuroimage.2013.03.030

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European journal of epidemiology, 31(4), 337–350. https://doi.org/10.1007/s10654-016-0149-3

Like what you're learning?

Use the assessment app

  • Over 300 orthopedic physical assessment tests
  • Statistics, basic assessments, and screening tests included
  • Direct links to PubMed references
  • Concise test descriptions
  • Video demonstration
  • Easy search & favorites function
E-Book

ALL ORTHOPEDIC TESTS IN ONE PLACE

SEE ALL PRODUCTS
Assessment app banner
Assessment E-book
Reviews

What customers have to say about the Assessment E-Book

Wait before you go!GET ACCESS TO A FREE HEADACHE REHAB PROGRAM

Subscribe now and download a free home exercise program for headaches designed by our headache expert René Castien