To establish minimum clinically important difference (MCID) for measurements in an orthopaedic patient population with joint disorders.

Adult patients aged 18 years and older seeking care for joint conditions at an orthopaedic clinic took the Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS^® PF) computerized adaptive test (CAT), hip disability and osteoarthritis outcome score for joint reconstruction (HOOS JR), and the knee injury and osteoarthritis outcome score for joint reconstruction (KOOS JR) from February 2014 to April 2017. MCIDs were calculated using anchor-based and distribution-based methods. Patient reports of meaningful change in function since their first clinic encounter were used as an anchor.

There were 2226 patients who participated with a mean age of 61.16 (SD = 12.84) years, 41.6% male, and 89.7% Caucasian. Mean change ranged from 7.29 to 8.41 for the PROMIS^® PF CAT, from 14.81 to 19.68 for the HOOS JR, and from 14.51 to 18.85 for the KOOS JR. ROC cut-offs ranged from 1.97-8.18 for the PF CAT, 6.33-43.36 for the HOOS JR, and 2.21-8.16 for the KOOS JR. Distribution-based methods estimated MCID values ranging from 2.45 to 21.55 for the PROMIS^® PF CAT; from 3.90 to 43.61 for the HOOS JR, and from 3.98 to 40.67 for the KOOS JR. The median MCID value in the range was similar to the mean change score for each measure and was 7.9 for the PF CAT, 18.0 for the HOOS JR, and 15.1 for the KOOS JR.

This is the first comprehensive study providing a wide range of MCIDs for the PROMIS^® PF, HOOS JR, and KOOS JR in orthopaedic patients with joint ailments.

The National Institutes of Health sponsored an initiative to develop a Patient-Reported Outcomes Measurement Information System (PROMIS^®)[1] using item response theory (IRT) methods. These methods have been recommended for use in the evaluation of arthroplasty outcomes because of their unique measurement properties[2]. Computerized adaptive testing (CAT) using IRT minimizes respondent burden without sacrificing instrument precision[3,4]. The PROMIS^® Physical Function (PF) CAT assesses physical function in five domains but is not directly targeted at joint function. Yet the useful measurement and administration qualities of the PROMIS^® PF CAT make it a valuable addition to patient-reported outcomes (PRO) assessments in joint reconstruction[5,6]. Aside from minimizing burden, top quality PRO instruments also offer reliable and valid scores that are easy to interpret[7].

Recent attention in PRO development has focused in on the interpretability of scores, particularly in terms of how meaningful the outcomes are to patients[8]. Change in function is an important clinical outcome, thus PROs should be able to detect change in patient function. In interpreting change, the minimum clinically important difference (MCID) reflects the smallest amount of meaningful change[9]. “MCIDs are patient derived scores that reflect changes in a clinical intervention that are meaningful for the patient”[10]. Meaningful change is important, as it can serve as a benchmark of treatment effect, and it is critical in decision making. Whether or not a treatment produces a statistically significant outcome is less informative than whether it produces meaningful change, as an inflated sample size can yield statistical significance without clinical relevance[11,12].

Multiple methods have been developed for determining MCID values and there is little agreement on the best standard to apply[10]. Distribution-based approaches rely on statistical methods and probability sampling. They describe how much change falls beyond random levels of variation. They rely on distributions of scores and how much the scores vary between patients in reaching a magnitude of change that is beyond chance fluctuation[13]. But distribution methods cannot tell us whether the amount of change is meaningful from the clinician’s or the patient’s perspective[8]. An alternative approach is to use anchor-based methods which relates the change in patient scores to some other measure of health outcomes[13].

Determining the MCID of the PROMIS^® PF is an important step to understand the meaning of the scores. Collecting the longitudinal data necessary to analyze meaningful change takes time. Because the PROMIS^® development began quite recently in 2004[1], there have been very few studies estimating MCIDs for PROMIS^® measures and little is known about MCID values in the orthopaedic adult reconstruction population[14]. Initial MCID development for PROMIS^® instruments has begun in specific patient populations such as pediatrics[15] or cancer patients[16] but studies are lacking in orthopaedic patients.

Other PROs have been developed that are specific to the domain of joint function and it is helpful to consider the measurement properties of these newer instruments side by side. The knee injury and osteoarthritis outcome score for joint reconstruction (KOOS JR) and the hip disability and osteoarthritis outcome score for joint reconstruction (HOOS JR) are two joint specific instruments recently introduced in the arthroplasty arena[17-20]. Both instruments have been approved by the Centers for Medicaid and Medicare Services for use in joint replacement registries[21-24]. The HOOS and KOOS were selected given their use in joint replacement registries in the United States, though other valid measures such as the Oxford hip or knee score are more common in European registries and the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) has a longer-term use for more general assessments of osteoarthritis[21]. Prior research has demonstrated that the HOOS and KOOS measures are sensitive to change[25]; however, we are unaware of any studies that develop MCIDs for these instruments[26]. The purpose of this study was to determine the MCIDs of the PROMIS^® PF CAT, the HOOS JR, and the KOOS JR in an orthopaedic population with lower-extremity joint conditions.

Determining the values of MCID for PRO measures enhances the interpretability and utility of the instruments. Determining score changes which reflect meaningful change requires understanding the patient perception of improvement or worsening. Anchor-based methods of determining an MCID tie the change scores to the patient reports of change and supplement the distribution-based methods which focus on precision in scores from a statistical standpoint. The current comprehensive assessment of MCID values utilized multiple methods of anchor-based and distribution-based estimation.

The PROMIS^® PF CAT mean change scores in this population of orthopaedic joint patients ranged from 7.29 to 8.41 depending on follow-up points. The ROC area under the curve ranged from 1.97 to 8.18. Distribution-based methods ranged using the SD method from a low of 2.45 (1/3 SD) to 4.35 (½ SD). The values using the MDC approach produced much larger values from 11.64 (MDC_90%) to 21.55 (MDC_99%). MCID values vary depending on the type of patient population (e.g., orthopaedic, cancer, pediatric) being evaluated, but overall the MCID values identified with the MDC approach in this study are much higher than the 2-3 point MCID of the PROMIS^® mobility subscale identified in a pediatric population[15] or the 4-6 point PROMIS^® PF CAT MCID values identified in a cancer population[16].

The larger MCID values in this study could have been obtained for a number of reasons. The values could be population specific, reflecting an underlying difference in functional gain between the orthopaedic adult patient population and the pediatric and cancer patient populations previously studied. However, the most likely explanation is that the MCID value obtained is a reflection of the method used for calculation. Unlike other studies that commonly use just one approach to estimate MCID values, the current study used multiple approaches and multiple time points to arrive at twenty-eight MCID values per instrument, allowing better triangulation of results. This study found a great deal of consistency across the follow-up time-points for any one particular method of calculating MCIDs (Tables 2 and 3), suggesting that the methods were responsible for the variation of the MCID values and that the length of follow-ups was mostly irrelevant. It is important to recognize that this study incorporated a wide array of cut-off standards, ranging from lenient to extremely strict. Since MDC_95% and MDC_99% are extremely strict cut-off standards, it is not surprising that these MCID values are high. None of the research reviewed as background for this study calculated MCIDs using such strict criteria[15,16]. The lowest MCID values derived from this study may be appropriate for screening purposes, but the median value (Figure 1) may be a better estimate of true and meaningful change when applying a conservative standard for evaluating treatment effects or for respondent analyses[50]. The more stringent cut-off standard derived MCID values reported in this study provide more definite assurance that important change has occurred.

The mean change for the HOOS JR ranged from 14.81 to 19.68. The ROC area under the curve ranged from 6.33 to 43.36. The 43.36 value appeared to be an outlier and is not considered a reliable estimate. The SD values ranged from 3.90 (1/3 SD) to 10.96 (½ SD). The MDC method of detecting change yielded a range of MCIDs of 17.54 (MDC_90%) to 43.61 (MDC_99%). These SD values are consistent with previous research identifying an MCID of 9.1 (1/2 SD method) for the HOOS in an arthroplasty population[51]. The median of the HOOS JR MCIDs was 18.0 and may be an appropriate conservative estimate for evaluating treatment effects.

For the KOOS JR, the mean change ranged from 14.51 to 18.85. The ROC maximized MCID values ranged from 2.21 to 8.16. The SD values ranged from 3.98 (1/3 SD) to 8.02 (½ SD). The MDC method of detecting change yielded a range of MCIDs from 20.84 (MDC_90%) to 40.67 (MDC_99%). The MDC_90% values reported here are actually lower than previous research on the KOOS JR which produced a range for improved individuals from 28.3 to 35.5[52]. The median KOOS JR MCID was 15.1 in this patient population.

The findings demonstrate that the method used to estimate MCIDs has a large impact on MCID value determination. The KOOS JR, for example, had a 2 point to 40 point difference in MCID depending on the method used. There is not yet consensus on a standardized approach for establishing MCID[10]. The lack of agreement between MCID values reported in this study, depending on the method used, is consistent with findings in the literature[53]. There have been recommendations that MCID determination should be standardized, but the best methods have not been agreed upon[38]. Until a more standardized method is established, the comprehensive range of MCIDs presented in this study provided much deeper insights than many existing studies. This comprehensive presentation enables patients, clinicians, care-givers and decision makers to be well-positioned in making their judgment call as to which MCID value(s) they should select based on how lenient or strict a standard they would like to set for their patients. It is tempting for clinicians to oversimplify and search for a single fixed MCID value, yet a single MCID value is often unstable[8]. A range of MCID values such as those presented in this study should indeed be considered by clinicians and health care practitioners.

The study may be limited by the type of patients who self-selected to return for follow-up visits, which is a common phenomenon across all orthopaedic and other clinics. Patients returning for longer-term follow-up were generally those experiencing more severe conditions and thus may not reflect the full range of improvement in condition. However, this should not be of much concern, as these patients returning for follow-ups were representative of the ones treated regularly in clinics, rendering the results of this study even more practical for standard orthopaedic practice. MCID value determination is generally not dependent on the severity of condition, thus the shorter-term or longer-term follow-up periods would not have impacted the MCID values, as evidenced from the empirical results of this study. In addition, not every patient in this study completed the outcome measures pre-treatment, thus some change scores may reflect a baseline time-point that was post-intervention. Yet these evaluations still produced meaningful change since longitudinal change over time was the main focus, not necessarily change from a specific time point to another specific time point.

One additional limitation comes from the use of anchor-based questions to determine clinically relevant change. An anchor-based question with a global rating of change measure is subject to recall bias, with some research indicating that reports of change may be more related to the current health status than real change from baseline[10]. Different anchor questions may produce different results as well. Distribution methods also have limitations as they derive MCIDs based on the variance of the data, which might be difficult to interpret. Since all methods have strengths and limitations, it was thus our intention for this study to be comprehensive in nature to cover a variety of approaches, allowing readers to make informed decisions in selecting MCID values based on their personal value judgments as to which target MCID is worthwhile to pursue. Lastly, the demographic characteristics of this sample may not be representative of those in the United States and may affect MCID scores. Future research should investigate MCIDs in a more diverse demographic sample. It should also establish MCIDs linking baseline scores using Rasch methodology to provide deeper insights.

Overall, this study utilized rigorous methodologies to develop a wide range of MCID values for the PROMIS^® PF, HOOS JR, and KOOS JR in an orthopaedic sample of patients with joint disorders. As there is no such concept as a correct or incorrect MCID, individual value judgments are necessary to apply MCIDs to treatment planning and in guiding patient expectations of treatment change.