Minimally important difference of the Child Oral Health Impact Profile for children with orofacial anomalies

Children born with cleft lip/palate typically require multiple surgeries and ongoing evaluations that extend well into adolescence and young adulthood [1]. Surgical interventions can include secondary palatal surgeries for improved speech, lip and nose revisions for improved facial performance, and alveolar bone graft surgery for improved functional well-being (e.g., tooth and bone development) [2]. Most children with cleft have multiple surgeries before they complete treatment and/or reach adulthood, yet little is known about the long-term effects of these interventions on patient-reported outcomes such as quality of life. While it is often assumed by surgeons, caregivers, and patients that cleft-related surgeries have a positive impact on patients’ lives, this assumption may be unfounded. Patients may experience treatment burnout, a phenomenon studied in other chronic conditions such as diabetes [3] and general orthodontic treatment [4], or fail to derive substantive benefit from treatment beyond measured clinical outcomes.

Oral health-related quality of life (OHRQoL), as a “multidimensional construct that includes a subjective evaluation of an individual’s oral health, functional well-being, emotional well-being, expectations and satisfaction with care, and sense of self” [5], may be particularly salient to children with orofacial anomalies. While there are multiple OHRQoL measures available for patient assessment in oral health, the Child Oral Health Impact Profile (COHIP) was specifically designed for children aged 8–15 years with applicability to a broad range of oral conditions [6]. Although the COHIP has been used to measure significant change in OHRQoL for children receiving surgery for cleft lip/palate, statistically significant change may not adequately assess whether a clinical intervention has a qualitative impact on the patient. The minimally important difference (MID), defined as “the smallest difference in score in the domain of interest which participants perceive as beneficial” [5, 7], can be used as a complementary, subjective tool for clinical assessment of meaningful improvement in patients [8‐11].

Previously, children with orofacial anomalies reporting large clinical change were shown to have higher scores for individual COHIP domains (e.g., oral health, functional and emotional well-being) [5, 12]. However, the COHIP MID has not been reported. Further, MID methods in dental research are generally underutilized [8]. Thus, the primary objective of this study was to estimate the minimally important difference for the COHIP in children with craniofacial conditions using both the anchor and distributional methods.

Data for analysis were derived from a 6-year, multi-center, prospective longitudinal study of youth with cleft conducted from 2009 to 2015. Youths and their caregivers participating in this study were followed at one of six major cleft treatment centers from the United States, including New York University Langone Medical Center, Children’s Hospital of Philadelphia, Lancaster Cleft Palate Center, Children’s Healthcare of Atlanta, University of Illinois-Chicago, and University of North Carolina-Chapel Hill. Participants included any child having a cleft lip and palate or cleft palate only between 7.5 and 18.5 years of age who spoke English or Spanish. Children who had a diagnosis of either an incomplete cleft lip without cleft of the alveolus, craniofacial syndrome or other complex medical conditions were excluded from the study. Participants were assessed at baseline and observed over two or three subsequent follow-up visits. The average length of time observed in the study for participants was 414 days, and the length of time between follow-ups ranged from 6 months to two years. During the course of the study, some patients received a surgical intervention and some did not. The primary objective of the parent study was to evaluate the effects of surgery for cleft lip/palate on psychosocial functioning, including depression, anxiety and resiliency. The secondary objective was to assess change in oral health related quality of life using the COHIP. Details of the study design, including study sample and surgical procedure descriptions, are available in a separate publication [13]. Analyses from this study do not evaluate the effects of surgical interventions for cleft lip/palate.

The analytic sample was approximately 57 % male and 53 % aged 12 years or older (Table 3). The sample was predominantly white (62.2 %) and had cleft lip and palate (80.8 %) as compared to cleft palate only (19.2 %). The average prior surgery history was 4.5 surgeries prior to the start of the study (baseline), with a standard deviation of 2.6. Compared to study participants who only presented at baseline, the analytic sample was not significantly different across gender or age, but was significantly different with respect to race/ethnicity and whether participants had cleft lip and palate or cleft palate only (data not shown). The average COHIP oral health domain score (Table 4) was 35.6 (SD = 6.5), followed by functional well-being (24.4, SD = 4.4), social-emotional well-being (31.6, SD = 7.4), school-environment (17.4, SD = 2.9), and self-image (22.7, SD = 4.6). Overall COHIP scores had a mean of 131.7 with a standard deviation of 18.9.

MID estimate results for the anchor-based approach using the Global Assessment of Change scale (Table 5) indicate that the minimally important difference ranged from −0.32 to 2.95. The mean change from baseline for each COHIP domain stratified by GAC change categories is also shown. MID estimates were 0.15 for oral health, −0.32 for functional well-being, 0.12 for social-emotional well-being, 0.84 for school environment and 0.60 for self-image. The MID for the overall COHIP was 2.95. Effect size statistic estimates for these COHIP domains were 0.16, 0.12, 0.22, 0.21 and 0.19, respectively. Effect size estimates for the overall COHIP was 0.25. Results for standardized response means of each domain were similar to effect sizes.

Stratified Global Assessment of Change scores (Table 6) indicate that overall anchor-based MIDs for the COHIP score were 3.29 and −10.17 for positive and negative change, respectively. Negative MIDs for individual COHIP domains included −0.93 (oral health), −1.86 (functional well-being), −0.29 (social-emotional well-being), −0.18 (school environment), and 1.07 (self-image). For positive MIDs, scores included 0.24 (oral health), 0.06 (functional well-being), 0.19 (social-emotional well-being), 1.01 (school environment), and 0.54 (self-image).

Most quantitative research, including quality of life research, use tests of statistical significance to interpret findings and study results. However, statistical significance in OHRQoL measures does not identify whether changes achieved have a qualitative impact on the patient [15], and large sample sizes can reveal statistically significant differences that may not be clinically meaningful or relevant to the patient [16]. Since a statistically significant change might not indicate real effects on patients’ lives, there has been a growing trend in quality of life outcomes research to measure clinically meaningful change using minimally important differences [8]. Instead of defining change on the basis of a statistical test of mean scores, MID uses the subjective perspective of the patient to determine what kind and how much change is meaningful [10, 17]. Despite MID’s utility as a clinically meaningful and sensitive assessment of change over time, it is rarely utilized in oral health and cleft research [18].

This study is the first to identify the MID of the Child Oral Health Impact Profile, a validated measure of oral health-related quality of life, in youth with orofacial anomalies. The sample included youth followed for ongoing assessments at cleft treatment centers and purposefully included only those seen for annual evaluations and not receiving a surgical intervention during the initial study period. COHIP MIDs provide a valuable tool for interpreting clinically meaningful change in OHRQoL in youth with cleft over time. However, it is also important to compare minimally important differences with other samples of youth with cleft, as well as youth with other oral conditions or who receive alternate treatment. Further, it is recommended that MID estimates are compared with qualitative data to obtain a more comprehensive understanding of patients’ subjective experiences with cleft treatment over time [19]. Finally, comparing youth perceptions of quality of life change with proxy ratings by caregivers can identify the level of agreement with external subjective evaluations [20].

The study findings may have important clinical and treatment implications. MID estimates can be used pre and post cleft-related surgery to determine the impact of particular surgery types, as well as the optimal timing of surgical interventions on youth OHRQoL. While all cleft centers follow Parameters of Care established by the American Cleft Palate-Craniofacial Association ²¹, there is wide variation across individual centers regarding the amount and timing of surgery recommended and completed with patients ²². Further, preliminary unpublished results from the parent study of children with cleft indicate that there may be diminishing returns for those patients undergoing more surgery than others with the same condition. Therefore, determining the type and timing of surgical interventions (e.g., orthognathic versus lip/nose revisions; childhood versus adolescence) that culminate in the most positive clinically meaningful change for patients could have substantial ramifications for cleft care.

Despite their usefulness, there are some disadvantages to using anchor-based methods to determine the MID. Anchor-based methods fail to consider instrument precision, their reliability is unknown, and they are influenced by a specific rating scale and anchors ²³. Measurement error due to recall bias and confounding by response shift are additional concerns. Finally, the validity and reliability of global change measures is suspect, as is valid self-judgment of change over time ^8,24. For these reasons and following established recommendations, we provided estimates of the Minimally Important Difference through both anchor-based and distribution methods, including the effect size statistic and standardized response means. However, there are alternative distribution-based measures of MIDs, including the standard error of measurement (SEM), paired t-statistic, and half standard deviation. Thus, different distribution methods of MIDs, as well as the choice of anchor used for estimates of global assessments of change, may yield varying results [8]. While the SEM measure incorporates instrument reliability in its calculation, and is therefore not sample dependent, it does not have a simple interpretation like that of standard effect sizes [11]. Due to its popularity and robustness to homogeneity and heterogeneity in sample data [8, 11], the effect size statistic is appropriate for this patient population.

Further limitations stem from the observational design of the parent study. There were no specific inclusion criteria regarding where participants were in the treatment process, the number of prior surgeries received, or appearance, speech proxy, or professional ratings across sites. Further, while the majority of youth with cleft in the US are followed by registered teams with the American Cleft Palate-Craniofacial Association that have experienced surgeons, team philosophies regarding treatment activism is not controlled. Finally, as the analytic dataset used in this study compared COHIP change from baseline to 1st follow-up, any participants who were present at baseline but did not return for follow-up evaluations were not included in analysis. For this study, children who were lost to follow-up after their baseline observation were significantly different from the analytic sample with respect to race and whether children had cleft lip and palate or cleft palate only. Thus, the generalizability of MID results may be further limited.

In conclusion, this research provides an important contribution to the study of OHRQoL and cleft care by identifying MIDs for the overall COHIP and its domains. These estimates can be used to assess clinically meaningful change in OHRQoL among youth with cleft over time. Future research can benefit from comparing MIDs between children with cleft and/or palate who received surgery and continued throughout post-operative follow-ups to those who did not receive a surgery recommendation. Additionally, continued follow-up with participants for which no surgery is recommended or rendered into adulthood may provide additional insight into meaningful change over time.