These authors sought to answer two questions: (1) What proportion of patients completed patient-reported outcome questionnaires at baseline and at 1 year and 2 years of follow-up after shoulder arthroplasty, and what methods did they use to complete these questionnaires (either automated or manual data collection)? (2) What factors were associated with questionnaire completion?
Their shoulder arthroplasty registry uses an electronic platform to collect patient-reported outcomes.
Patients without an email address on file were excluded; 90% (1907 of 2128) of patients were included in the study. The population was 50% women with a mean age of 67 ± 9 years.
The proportion of questionnaire completion at baseline, 1 year, and 2 years were 72% (1369 of 1907),
47% (456 of 972), and 33% (128 of 393), respectively.
At baseline, 63% (868 of 1369) of patients completed their data through the commercial proprietary Outcomes Based Electronic Research Database platform (OBERD, Universal Research Solutions, Columbia, MO, USA), while 37% (501 of 1369) responded to telephone and email reminders from the research assistant. At 1 year, 84% (381 of 456) of patients who completed their data did so automatically and 16% (75 of 456) required manual assistance. At 2 years, the proportions were 81% (103 of 128) and 19% (25 of 128) for automated and manual data collection, respectively.
Of the patients who completed their questionnaires, 63% (868 of 1369) did so through automated emails at baseline, 84% (381 of 456) did so at 1 year, and 81% (103 of 128) did so at 2 years.
The remainder completed their questionnaires through manual data collection with a research assistant:
37% (501 of 1369) at baseline, 16% (75 of 456) at 1 year, and 19% (25 of 128) at 2 years.
After controlling for potentially confounding variables like patient demographics, college education, and living arrangements, women were less likely to complete baseline questionnaires than men (odds ratio 0.78 [95% confidence interval 0.62 to 0.99]; p = 0.04), and white patients (OR 1.6 [95% CI 1.05 to 2.44]; p = 0.03) were more likely than nonwhite patients to have complete baseline questionnaires.
At 2 years of follow-up, patients with a college education (OR 2.06 [95% CI 1.14 to 3.71]; p = 0.02), those who lived alone (OR 2.11 [95% CI 1.13 to 3.94]; p = 0.02), and those who had higher baseline Shoulder Activity Scale scores (OR 1.05 [95% CI 1.00 to 1.11]; p = 0.04) were more likely to have complete questionnaires than those without a college education, those who lived with other people, and those with lower SAS scores, respectively.
The authors concluded that the challenges of adopting an online platform include low follow-up proportions and the need for manual assistance by a research assistant to increase patient
completion of questionnaires.
Comment: The goal of clinical outcomes research is to obtain followup on the highest percentage of patients using validated outcome measures. Barriers to achieving this goal include (1) requiring the patient to return to the office (as is required for the Constant score), (2) requiring the patient to have an email, (3) requiring the patient have access to a computer, and (4) requiring the patient to learn to use a computer adaptive system.
It is generally recognized that if the percentage of patients not included in a study (i.e. lost to followup) exceeds the suspected complication rate, the study is not useful in determining the actual complication rate. As an example if the re-revision rate after revision of an anatomic shoulder arthroplasty to a reverse shoulder arthroplasty is suspected to be 11% based on a preliminary sample (see previous post) but 20% of the patients in the study were lost to followup, we don't know if the re-revision rate is 11% or 31%.
In the current study of an outcomes based electronic research database platform 10% of patients had to be excluded because they did not have emails. Of the remaining 90%, only 63% of patients completed baseline questionnaires data through the commercial proprietary OBERD system, while 37% responded to telephone and email reminders from the research assistant.
The percentages of patients lacking completed questionnaires was 28% at baseline, 53% at one year, and 67% at two years.
The "selling point" of computer adaptive testing is thought to be the reduction in the "question burden" to the patient (see below).
However, if in reducing this burden other obstacles are inserted to a high percentage of followup, the increased cost of such a system may not be justified.
F'inally, as this study demonstrated, the use of computer adaptive testing can selectively exclude women, non-white patients and patients without a college education. Such a participation bias can dramatically affect the generalizability of data achieved with such as system.
For our longitudinally maintained arthroplasty database, currently taking >1000 patients, we ask participants for their preferred method of followup at the time of consent. About 1/3rd request mail followup and the other 2/3rds request email. None use an automated system. Our two year followup ranges between 80 and 85%.
