What's the Alpha Level?

Go to

What’s the Alpha Level? | Statistics

In frequentist statistics, the alpha level (also known as the significance level) is the probability of rejecting the null hypothesis when it is true. In the context of physiotherapy research, the null hypothesis might be that there is no difference in pain reduction between two different physiotherapy interventions. The alpha level is typically set at 0.05, which means that there is a 5% chance of incorrectly rejecting the null hypothesis (i.e., concluding that there is a difference in pain reduction when there actually isn’t) in the long term.

It is especially important to consider this as a long-term result. If 100 similar studies are conducted, 5 of them, on average, will show a false positive result if there is no effect.

Let’s say a study compares two physiotherapy interventions for lower back pain, and the results show that the mean pain reduction for Intervention A is 6 points on a pain scale, and the mean pain reduction for Intervention B is 8 points on a pain scale. With an alpha level of 0.05, the researchers would reject the null hypothesis and conclude that there is a statistically significant difference in pain reduction between the two interventions because the difference in means is greater than what would be expected by chance.

It’s important to note that setting an alpha level of 0.05 is a convention and not a rule. The choice of alpha level depends on the context of the study and the potential consequences of a false positive or false negative result. For example, if the consequences of a false positive result (i.e., concluding that a treatment is effective when it is not) are more severe, researchers might choose to use a lower alpha level (e.g. 0.01) to decrease the probability of a false positive result.

We want to stress again the importance of a long-term view. You cannot simply say that there’s a 5% chance that the paper has become a false positive result. When the research is conducted, it simply is a false positive, or it is not. The 5% speaks of long-term results. Doing this test in multiple studies with similar conditions will result in about 5% of the papers having a false positive result.

A physiotherapy intervention may appear to be very effective for reducing lower back pain, with a small p-value (indicating a statistically significant difference) and large effect size. However, if this single study is not replicated in other studies, it’s difficult to determine if the results are due to chance or a real effect.

A long-term view takes into account the results of multiple studies over time to provide a more comprehensive understanding of the effectiveness of an intervention. This approach is particularly important in physiotherapy research, where the results of a single study may not generalize to other populations or settings.