What's a Confidence Interval?

Go to

What’s a Confidence Interval? | Statistics

A confidence interval is a range of values that is used to estimate an unknown population parameter with a certain level of confidence. It is a way to express the uncertainty around an estimate of a population parameter, such as the mean.

For example, in a study investigating the effectiveness of a new physiotherapy treatment for lower back pain, the researchers might collect data from a sample of patients and calculate the mean pain reduction for the treatment group. A 95% confidence interval for the mean pain reduction would be a range of values within which the true population mean pain reduction is expected to fall with a 95% probability. This means that if the same study is conducted 100 times, about 95 studies will have the true population mean within their 95% confidence interval.

A typical example would be:

“The results show a mean VAS reduction in pain at four weeks of 2.3 (95% CI 1.8 – 2.8).”

The confidence interval is calculated based on the sample statistics and the level of confidence desired (commonly 95% or 99%). It is important to note that a confidence interval does not indicate whether the null hypothesis is true or false, but it provides an interval of values that’s likely to include the true population parameter with a certain level of confidence.

What about the width?

A narrow confidence interval indicates that the sample mean is a more precise estimate of the population mean, while a wide confidence interval indicates that the sample mean is less precise. In general, larger sample sizes tend to result in narrower confidence intervals, and thus more precise estimates of population parameters.

Misconceptions

It is important to note that one cannot say that there is a 95% chance that the true population mean lies within a given interval of a certain paper. It will simply be the case, or not. However, if this study is repeated an infinite number of times, the true mean will be found within the generated intervals 95% of the time.

CI’s vs P-values

A confidence interval provides a range of values for a population parameter (such as the mean difference between two groups) that is estimated from a sample. The interval is calculated such that, if we were to repeat the study multiple times, a certain percentage of the intervals (determined by the level of confidence, usually 95%) would contain the true population value.

In contrast, a p-value is a probability that the null hypothesis (e.g. no difference between groups) is true given the sample data and the assumption that the null hypothesis is correct. A small p-value (typically less than 0.05) is often used to reject the null hypothesis and suggest that there is evidence of a difference between groups.

However, the p-value only provides a binary answer to the question of whether the null hypothesis can be rejected or not. It does not give any information about the magnitude or precision of the effect, or how likely it is to occur in a larger population. This is where the confidence interval can be more useful. By providing a range of values for the effect, the confidence interval gives a clearer picture of how much the treatment is expected to change the outcome and how much uncertainty there is in that estimate. In addition, the confidence interval can help to avoid over-interpretation of a significant p-value, as a statistically significant result does not necessarily mean that the effect is practically or clinically significant. Effect sizes can help you make this judgment.

In summary, a confidence interval provides a more complete and nuanced picture of the results of a study, while a p-value only provides a binary answer to the question of statistical significance.