Let's say we wanted to compare re-tear rates between rotator cuff repairs with and without graft augmentation. Our research fellow found these results showing a statistically significant benefit of the graft.
However, she rechecked her data and found that one patient in the no-graft group originally thought to have had a re-tear actually had an intact repair. When the data were re-analyzed, the significance of the benefit of grafting disappeared.
As can be determined from the Fragility index calculator, our study had a fragility index of one: only one result needed to change for the result to loose statistical significance.
The authors of The Statistical Fragility of Studies on Rotator Cuff Repair with Graft Augmentation sought to determine the fragility index (FI) and fragility quotient (FQ) for studies investigating the value of augmenting rotator cuff repair (RCR) with a graft. A lost to follow-up (LTF) value greater than the FI indicates statistical instability for the reported outcomes and conclusions.
They performed a systematic review to identify studies of RCR with graft augmentation. They included 17 studies (1,098 patients) that compared least one statistically analyzed dichotomous outcome variable for graft and no graft repairs (eg cuff integrity at followup). The fragility index (FI) was determined by changing each reported outcome event within 2 x 2 contingency tables until statistical significance or non-significance was reversed. The associated fragility quotient (FQ) was determined by dividing the FI by the sample size. Lost to follow-up (LTF) values were also extracted from each included study.
The median FI was 4 (IQR 2-5), indicating that the reversal of 4 patients’ outcomes would have reversed the finding of significant difference from significant to non-significant or vice-versa. The median FQ was 0.08 (IQR 0.04-0.15), indicating that in a sample of 100 patients, reversal of 8 patients’ outcomes would reverse statistical significance. 56% of reported outcomes were statistically fragile, having LTF greater than their respective FI.
They concluded that studies of RCR with graft augmentation lack statistical stability: a few altered outcome events would reverse statistical significance. They recommend that future studies of graft augmentation include FI and FQ along with traditional statistical significance analyses to provide better context on the strength of conclusions.
Comment: Assessment of fragility should be a part of all comparative studies. See:
PRP for Cuff Disease: Data Fragility of a Level I Study
You can support cutting edge shoulder research that is leading to better care for patients with shoulder problems, click on this link.
Follow on twitter: https://twitter.com/shoulderarth
Follow on facebook: click on this link
Follow on facebook: https://www.facebook.com/frederick.matsen
Follow on LinkedIn: https://www.linkedin.com/in/rick-matsen-88b1a8133/
Here are some videos that are of shoulder interest
Shoulder arthritis - what you need to know (see this link).
How to x-ray the shoulder (see this link).
The ream and run procedure (see this link).
The total shoulder arthroplasty (see this link).
The cuff tear arthropathy arthroplasty (see this link).
The reverse total shoulder arthroplasty (see this link).
The smooth and move procedure for irreparable rotator cuff tears (see this link).
Shoulder rehabilitation exercises (see this link).
