Introduction

This article aimed to investigate the prognostic validity of a previously developed clinical prediction tool (PRIS) to identify patients at risk for recurrent instability after shoulder dislocation within 1 year after their first-time traumatic anterior shoulder dislocation (FTASD). The authors hypothesized that their tool would have high specificity, sensitivity and predictive validity.

Methods

Eligible patients were between 16-40 years old, had a FTASD recorded by the government corporation responsible for administering no-fault injuries, and were living in New Zealand. An RX was necessary to exclude other shoulder pathologies and to confirm an anterior shoulder dislocation was or had been present and reduced (by applying external force).

Predictive ability, discriminative ability, calibration, and overall accuracy were examined.

• Predictive ability was evaluated using ROC curve analysis to find a cut point of maximal sensitivity and specificity (Youden’s index).
• Discriminative ability was measured by area under the curve (AUC): a larger AUC indicates higher accuracy and validity of the prediction tool.
• Calibration of the tool was measured by the Hosmer-Lemeshow test, indicating how well the observations follow the predictions
• Accuracy of the predictive cut point was calculated by the sum of the true positives and true negatives divided by the total number of tests
• Calculation of sensitivity (Sn), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV) with their 95% confidence intervals

Results

Results indicated that 76% of the validation population did not have recurrent instability events within 1 year after their FTASD. The PRIS-tool achieved following values:

• Overall accuracy: Sn=39%, Sp=95%, PPV=70%, NPV=83%, -LR=0.65, +LR=7.39.
• Discriminative ability: AUC=0.69, indicating a limited discriminative validity and thus a limited predictive value.
• Calibration revealed a poor goodness of fit between the predicted and observed data.

Talk nerdy to me

Some aspects endanger the internal validity of the tool. While outcomes are explicitly described, they are not objectively measured (outcomes were obtained through telephone calls). Some prognostic factors (such as the period of immobilization or self-reported hypermobility) may be subject to recall bias or interpretation. Furthermore, prognostic factors were measured within 12 weeks following the FTASD, making it likely that not all patients were measured at a similar time point. No information was provided on how missing data were handled and similarly, no information on the chosen procedure for logistic regression analysis was given.

Take home messages

Despite an essential step in the development of the predictive model was made, the PRIS tool failed in its intent to identify people at risk for recurrent shoulder instability. While the authors state that PRIS can predict who will not have recurrent instability within one-year post-FTASD, this was not the primary goal of this study and thus this conclusion should be taken with caution. However, this tool, in combination with a thorough clinical examination, may provide physiotherapists with an indication of those not at risk for recurrent instability in the short term who can be treated conservatively.

Reference

Olds, M., Ellis, R., & Kersten, P. (2020). Predicting Recurrent Instability of the Shoulder (PRIS): a valid tool to predict which patients will not have repeat shoulder instability after first-time traumatic anterior dislocation. Journal of Orthopaedic & Sports Physical Therapy, 50(8), 431-437.