We tend to have highest confidence in Level I randomized clinical trials because they control for variables that may be overlooked in comparative case series - even with devices such as propensity matching.
Level I randomized clinical trials been reported that explore important aspects of reverse total shoulder arthroplasty. Some recent examples are listed below:
Inferior tilt of the glenoid component does not decrease scapular notching in reverse shoulder arthroplasty: results of a prospective randomized study.
Results after primary reverse shoulder arthroplasty with and without subscapularis repair: a prospective randomized trial.
Reverse Shoulder Arthroplasty Is Superior to Plate Fixation at 2 Years for Displaced Proximal Humeral Fractures in the Elderly: A Multicenter Randomized Controlled Trial.
Can a functional difference be detected in reverse arthroplasty with 135° versus 155° prosthesis for the treatment of rotator cuff arthropathy: a prospective randomized study.
Clinical performance of lateralized versus non-lateralized reverse shoulder arthroplasty: a prospective randomized study
A randomized single-blinded trial of early rehabilitation versus immobilization after reverse total shoulder arthroplasty
Reverse total shoulder arthroplasty provides better shoulder function than hemiarthroplasty for displaced 3- and 4-part proximal humeral fractures in patients aged 70 years or older: a multicenter randomized controlled trial.
Reverse or Hemi Shoulder Arthroplasty in Proximal Humerus Fractures: A Single-Blinded Prospective Multicenter Randomized Clinical Trial.
Reverse shoulder arthroplasty versus nonoperative treatment for 3- or 4-part proximal humeral fractures in elderly patients: a prospective randomized controlled trial.
A comparison of concentric and eccentric glenospheres in reverse shoulder arthroplasty: a randomized controlled trial
Reverse shoulder arthroplasty versus hemiarthroplasty for acute proximal humeral fractures.A blinded, randomized, controlled, prospective study.
Positioning of the metaglene in reverse shoulder arthroplasty: deltopectoral versus anterosuperior
approach: a prospective randomized trial
Influence of glenosphere size on the development of scapular notching: a prospective randomized study.
Can small glenospheres with eccentricity reduce scapular notching as effectively as large glenospheres without eccentricity?
Bony increased-offset reverse shoulder arthroplasty vs. metal augments in reverse shoulder arthroplasty: a prospective, randomized clinical trial with 2-year follow-up.
Intravenous tranexamic acid reduces total blood loss in reverse total shoulder arthroplasty: a prospective, double-blinded, randomized, controlled trial.
Reverse shoulder arthroplasty with and without latissimus and teres major transfer for patients with
combined loss of elevation and external rotation: a prospective, randomized investigation.
The fragility quotient (FQ) is the fragility index divided by the number of patients in the study. In this example it would be 1/60 or 0.01667.
This post received and interesting reaction from Mark Cote, Director of Outcomes Research for Sports Medicine & Orthopaedic Surgery. Massachusetts General Hospital. He gave permission to quote it here:
"I think it’s great to go beyond p-values; however, this type of approach does little to advance interpretation. We’ve discussed this from time to time in the past. The fragility index is trying to move away from “statistical significance” yet it’s rooted in statistical significance, i.e. how many patients would have to change their outcome in order to flip the p-value to above 0.05. In a randomized clinical trial (RCT) of bio-augmented RCR versus standard RCR, if the bio-aug group had less failures than standard and the result was statistically significant (p<0.05), then the fragility index would be the number of patients in the bio-aug group that would have needed to fail before the difference is no longer p<0.05.
The problem is the fragility index can be misleading. For one, RCTs are designed with the MINIMUM number of patients needed to reach statistical significance, yet we act surprised when these studies are “fragile” because the fragility index equals 2 or 3 patients. Of course it does, it was deliberately designed to barely reach statistical significance! When's the last time you, or a medical student, resident, fellow, or faculty came into my office and asked for a sample size that went well beyond p<0.05? The findings in the blog should not be surprising, nor should the growing number of systematic reviews of RCTs involving the elbow, knee, etc., all of which conclude the same thing, i.e. the available evidence is fragile. If we did this in RCR or instability, it’s likely to be fragile as well.
Finally, a large fragility index doesn’t equal robust findings. In fact, it can be quite the opposite. If you recall, we were looking at the fragility index for RCTs on proximal humerus fractures a few months back. The one study that had a large index seemed to be celebrated yet when you looked closer at all of the studies in the review, it had the biggest loss to follow-up as well as other biases . Here, the large fragility index was a product of a bias study. This is also not surprising. You would expect well planned RCTs that by tradition enroll the MINIMUM number of patients needed to reach statistical significance to have small fragility indexes. Conversely, a RCT that despite planning (or alleged planning) that produced results that are wildly inconsistent with their power analysis can indicate a number of things, almost all of which indicate pronounced bias. Dr. Matsen does a good job noting the importance of losses to follow up but the general premise of the fragility index is flawed. Specifically, there IS or IS NOT a difference in revision rates between arthroplasty groups. It would be nice if nature was so straightforward, but unfortunately it isn’t. All null hypotheses, i.e. no difference between groups, are wrong. The question is how well a study estimated a difference and whether it’s of a size that we care about.
Again, I think it’s great to see folks trying to move away from statistical significance. It’s like acknowledging that you need to stop smoking 3 packs a day. But to turn to the fragility index as some sort of panacea or surrogate for robustness is about the same as thinking that switching to light cigarettes will mitigate the health risk of your 3-pack a day habit.
That said, a paper on these problems may be worth pursuing…
Inferior tilt of the glenoid component does not decrease scapular notching in reverse shoulder arthroplasty: results of a prospective randomized study.
Results after primary reverse shoulder arthroplasty with and without subscapularis repair: a prospective randomized trial.
Reverse Shoulder Arthroplasty Is Superior to Plate Fixation at 2 Years for Displaced Proximal Humeral Fractures in the Elderly: A Multicenter Randomized Controlled Trial.
Can a functional difference be detected in reverse arthroplasty with 135° versus 155° prosthesis for the treatment of rotator cuff arthropathy: a prospective randomized study.
Clinical performance of lateralized versus non-lateralized reverse shoulder arthroplasty: a prospective randomized study
A randomized single-blinded trial of early rehabilitation versus immobilization after reverse total shoulder arthroplasty
Reverse total shoulder arthroplasty provides better shoulder function than hemiarthroplasty for displaced 3- and 4-part proximal humeral fractures in patients aged 70 years or older: a multicenter randomized controlled trial.
Reverse or Hemi Shoulder Arthroplasty in Proximal Humerus Fractures: A Single-Blinded Prospective Multicenter Randomized Clinical Trial.
Reverse shoulder arthroplasty versus nonoperative treatment for 3- or 4-part proximal humeral fractures in elderly patients: a prospective randomized controlled trial.
A comparison of concentric and eccentric glenospheres in reverse shoulder arthroplasty: a randomized controlled trial
Reverse shoulder arthroplasty versus hemiarthroplasty for acute proximal humeral fractures.A blinded, randomized, controlled, prospective study.
Positioning of the metaglene in reverse shoulder arthroplasty: deltopectoral versus anterosuperior
approach: a prospective randomized trial
Influence of glenosphere size on the development of scapular notching: a prospective randomized study.
Can small glenospheres with eccentricity reduce scapular notching as effectively as large glenospheres without eccentricity?
Bony increased-offset reverse shoulder arthroplasty vs. metal augments in reverse shoulder arthroplasty: a prospective, randomized clinical trial with 2-year follow-up.
Intravenous tranexamic acid reduces total blood loss in reverse total shoulder arthroplasty: a prospective, double-blinded, randomized, controlled trial.
Reverse shoulder arthroplasty with and without latissimus and teres major transfer for patients with
combined loss of elevation and external rotation: a prospective, randomized investigation.
While these Level I studies assess the statistical significance (i.e. the p values) for their findings, the p value does not indicate how robust or fragile the findings are. In some cases a change in outcome for just a few patients could flip the p value from significant (<.05) to insignificant (>.05) or, conversely, from insignificant (p>.05) to significant (p<.05).
A useful indicator of the robustness of the result of an RCT is the fragility index (FI), which is the minimum number of results which must be reversed to change the statistical significance of a given outcome. The Fisher exact test provides convenient way of measuring the FI (readers are encouraged to experiment with it using this link).
Here is a hypothetical example comparing the incidence of of acromial fractures in a randomized study of hemiarthroplasty versus reverse total shoulder. In the first scenario, none of the 30 patients having hemiathroplasty experienced acromial fractures, whereas 6 of the 30 patients having reverse total shoulders experienced acromial fractures. The result was statistically significant at p < .05.
However, if one of the 30 patients having hemiarthroplasty was discovered to have an acromial fracture, the result was no longer statistically significant at p < .05.
The fragility quotient (FQ) is the fragility index divided by the number of patients in the study. In this example it would be 1/60 or 0.01667.
Studies with lower fragility indices and lower fragility quotients are less robust, even if p<.05.
The fragility index can also be compared to the number of patients lost to followup (LTFU), i.e. for whom the outcome is unknown. If the number of patients lost to followup is greater than the fragility index, the observed difference is not robust, even if p<.05
The authors of The Statistical Fragility of The Management Options for Reverse Shoulder Arthroplasty: A Systematic Review of Randomized Control Trial with Fragility Analysis examined the statistical fragility of Randomized Control Trials (RCTs) reporting outcomes from reverse total shoulder arthroplasty (RSA). They included 19 RCTS representing 1,146 patients, of which 41.2% were male, with a mean age of 74.2 and mean follow-up of 22.1 months.
The median number of patients in the studies was 59, with a median of 9 patients Lost To Follow-up (LTFU). The median FI was 4.5, indicating more patients were lost from the trial than the number of outcomes which would have to change to reverse the finding of significance. The median FQ was 0.083.
Comment: Among these Level I RSA studies, the clinical evidence was vulnerable to statistical fragility, with a median FI of 4.5 indicating that the reversal of just a handful of outcomes was sufficient to change the finding of statistical significance. This should be viewed in the context of the median number of patients LTFU being equal to 9. Thus the median trial lost more patients to follow-up than the number of outcomes which would have to be changed to reverse a finding of significance. Approximately 2/3s of included events may have had reversed significance findings had there been a more complete follow-up.
When interpreting the results of randomized control clinical trials, it is important to consider both the fragility index in comparison to the number o patients lost to followup.
"I think it’s great to go beyond p-values; however, this type of approach does little to advance interpretation. We’ve discussed this from time to time in the past. The fragility index is trying to move away from “statistical significance” yet it’s rooted in statistical significance, i.e. how many patients would have to change their outcome in order to flip the p-value to above 0.05. In a randomized clinical trial (RCT) of bio-augmented RCR versus standard RCR, if the bio-aug group had less failures than standard and the result was statistically significant (p<0.05), then the fragility index would be the number of patients in the bio-aug group that would have needed to fail before the difference is no longer p<0.05.
The problem is the fragility index can be misleading. For one, RCTs are designed with the MINIMUM number of patients needed to reach statistical significance, yet we act surprised when these studies are “fragile” because the fragility index equals 2 or 3 patients. Of course it does, it was deliberately designed to barely reach statistical significance! When's the last time you, or a medical student, resident, fellow, or faculty came into my office and asked for a sample size that went well beyond p<0.05? The findings in the blog should not be surprising, nor should the growing number of systematic reviews of RCTs involving the elbow, knee, etc., all of which conclude the same thing, i.e. the available evidence is fragile. If we did this in RCR or instability, it’s likely to be fragile as well.
Finally, a large fragility index doesn’t equal robust findings. In fact, it can be quite the opposite. If you recall, we were looking at the fragility index for RCTs on proximal humerus fractures a few months back. The one study that had a large index seemed to be celebrated yet when you looked closer at all of the studies in the review, it had the biggest loss to follow-up as well as other biases . Here, the large fragility index was a product of a bias study. This is also not surprising. You would expect well planned RCTs that by tradition enroll the MINIMUM number of patients needed to reach statistical significance to have small fragility indexes. Conversely, a RCT that despite planning (or alleged planning) that produced results that are wildly inconsistent with their power analysis can indicate a number of things, almost all of which indicate pronounced bias. Dr. Matsen does a good job noting the importance of losses to follow up but the general premise of the fragility index is flawed. Specifically, there IS or IS NOT a difference in revision rates between arthroplasty groups. It would be nice if nature was so straightforward, but unfortunately it isn’t. All null hypotheses, i.e. no difference between groups, are wrong. The question is how well a study estimated a difference and whether it’s of a size that we care about.
Again, I think it’s great to see folks trying to move away from statistical significance. It’s like acknowledging that you need to stop smoking 3 packs a day. But to turn to the fragility index as some sort of panacea or surrogate for robustness is about the same as thinking that switching to light cigarettes will mitigate the health risk of your 3-pack a day habit.
That said, a paper on these problems may be worth pursuing…
Mark"
You can support cutting edge shoulder research that is leading to better care for patients with shoulder problems, click on this link.
Follow on twitter: https://twitter.com/shoulderarth
Follow on facebook: click on this link
Follow on facebook: https://www.facebook.com/frederick.matsen
Follow on LinkedIn: https://www.linkedin.com/in/rick-matsen-88b1a8133/
Here are some videos that are of shoulder interest
Shoulder arthritis - what you need to know (see this link).
How to x-ray the shoulder (see this link).
The ream and run procedure (see this link).
The total shoulder arthroplasty (see this link).
The cuff tear arthropathy arthroplasty (see this link).
The reverse total shoulder arthroplasty (see this link).
The smooth and move procedure for irreparable rotator cuff tears (see this link).
Shoulder rehabilitation exercises (see this link).
Follow on twitter: https://twitter.com/shoulderarth
Follow on facebook: click on this link
Follow on facebook: https://www.facebook.com/frederick.matsen
Follow on LinkedIn: https://www.linkedin.com/in/rick-matsen-88b1a8133/
Here are some videos that are of shoulder interest
Shoulder arthritis - what you need to know (see this link).
How to x-ray the shoulder (see this link).
The ream and run procedure (see this link).
The total shoulder arthroplasty (see this link).
The cuff tear arthropathy arthroplasty (see this link).
The reverse total shoulder arthroplasty (see this link).
The smooth and move procedure for irreparable rotator cuff tears (see this link).
Shoulder rehabilitation exercises (see this link).
