Thursday, April 6, 2023

The challenge of measuring shoulder arthroplasty outcomes: bias, ceiling effects, and practicality.

Each surgeon has the opportunity - indeed the responsibility - to keep track of her or his surgical outcomes for the purpose of knowing what is and what is not working in the practice. This point is discussed in detail in this link. In that many failures of arthroplasty occur more than five years after surgery, long-term followup is critical.




Three of the key elements of an effective/informative/practical outcome system are (1) capturing the highest possible percentage of patients treated, (2) being able to present the results to patients in terms that patients and surgeons understand, and (3) having a system that is validated and universally applicable so that data can be compared among centers.

#1 requires minimizing exclusion bias. Many scales, such as the Constant Score, the UCLA score, and the Shoulder Arthroplasty Smart score require the patient to return to the office for the measurement of ranges of motion (and, in some cases, strength). In addition to risking observer bias and inter-observer variability, the requirement of returning to the office risks selectively excluding those patients living at a distance from their provider, those unwilling or unable to return, and those of limited economic means. Computer-based scoring systems, such as the PROMIS and Computer Adaptive Testing, risk selectively excluding patients without access to computers, those who are not computer literate and those not proficient in English. The ideal system makes it easy for all patients to be included in long-term followup: inexpensive, quick to complete, accessible independent of the location of the patient and independent of the patient's computer literacy and access.

#2 requires that the outcome data are presented in a way that is meaningful to the patient and surgeon. Most patients will have difficulty understanding the significance of a "score of 72" on PROMIS, Constant, UCLA or SAS, but many would understand the significance of the improvement in specific shoulder functions achieved by their surgeon for a specific condition presented as shown below (showing results obtained using the Simple Shoulder Test results for extended head hemiarthroplasty in the treatment of patients having cuff tear arthropathy with retained active elevation).





#3 Most of the commonly used outcome measures have been carefully validated, for example see Is the Simple Shoulder Test a valid outcome instrument for shoulder arthroplasty? which shows, in spite of the fact that 15% percent of the patients achieved the maximal SST score, there was a near-perfect correlation between satisfaction and the final SST score, suggesting that the "ceiling effect" is likely to have little clinical significance.





What this means is that a shoulder that can perform each of the 12 SST functions (below) is an excellent and highly satisfactory shoulder.





If the ceiling effect was a concern, one could add a thirteenth question: "Can you throw a football 100 yards with the affected arm?". Very few shoulders, normal or post-arthroplasty, would hit the ceiling of 13/13 "yes" responses.


In the same vein, the authors of Validation of a machine learning–derived clinicalmetric to quantify outcomes after total shoulderarthroplasty and Exactech Equinoxe anatomic versus reverse total shoulder arthroplasty for primary osteoarthritis: case controlled comparisons using the machine learning–derived Shoulder Arthroplasty Smart score correctly point out that the Shoulder Arthroplasty Smart score (you can experiment with it on this link) does not have a ceiling effect. In order to achieve the ceiling of the SAS score, the shoulder needs to be measured as having 180 degrees of active forward elevation, internal rotation to T7, and 90 degrees of active external rotation with the arm at the side.


These values will be difficult to attain because they are substantially greater than those found in the general population (see Shoulder range of movement in the general population: age and gender stratified normative data using a community-based cohort): average active shoulder flexion was 160° and average active external rotation was 59°.

Another approach for those concerned about the "ceiling effect" is put forth by the authors of Quantifying success after anatomic total shoulder arthroplasty: the minimal clinically important percentage of maximal possible improvement. They expressed the amount of improvement as the percentage of maximum possible improvement (%MPI) (based on a prior study: The prognosis for improvement in comfort and function after the ream-and-run arthroplasty for glenohumeral arthritis: an analysis of 176 consecutive cases). The %MPI is calculated as (postoperative score - preoperative score)/(perfect score - preoperative score). The "ceiling" in the %MPI would only be reached if the score improved from the worst possible score to the best possible score - a rare event.
Then they determined the minimal clinically important difference (MCID) for the %MPI using the anchor method. Interestingly their calculated MCID-%MPI values are similar for many of the commonly used scores: 33% for the SST, 32% for the ASES score, 38% for the UCLA score, 30% for the Shoulder Pain and Disability Index score, and 33% for the Shoulder Arthroplasty Smart score.


Comment: A surgeon's choice of the optimal patient followup system needs to be made in consideration of the above factors as well as the required staff time and cost of implementation. The goal is to capture long-term data on the highest percentage of patients treated using a method that is affordable and practical for the office.

You can support cutting edge shoulder research that is leading to better care for patients with shoulder problems, click on this link.

Follow on twitter: https://twitter.com/shoulderarth
Follow on facebook: click on this link
Follow on facebook: https://www.facebook.com/frederick.matsen
Follow on LinkedIn: https://www.linkedin.com/in/rick-matsen-88b1a8133/

Here are some videos that are of shoulder interest
Shoulder arthritis - what you need to know (see this link).
How to x-ray the shoulder (see this link).
The ream and run procedure (see this link).
The total shoulder arthroplasty (see this link).
The cuff tear arthropathy arthroplasty (see this link).
The reverse total shoulder arthroplasty (see this link).
The smooth and move procedure for irreparable rotator cuff tears (see this link).
Shoulder rehabilitation exercises (see this link).