Comparison of the ECOHIS and short-form P-CPQ and FIS scales

The last decade or so has seen considerable progress in the development, testing and validation of child measures of oral-health-related quality of life (OHRQoL). A number of competing measures now exist [1‐3], with most of those intended for use with children who are old enough to give valid and reliable responses. Measuring oral health in children younger than six years of age has necessitated the use of scales which use adult informants (such as parents) who know the child well. The Parental-Caregiver Perceptions Questionnaire (P-CPQ) and the Family Impact Scale (FIS) are part of the Child Oral-Health-Related Quality of Life (COHQoL) suite of instruments developed over a decade ago [4], and are intended for use with those younger children and their families. The 33-item P-CPQ has four subscales (oral symptoms, functional limitation, emotional well-being and social well-being), and the 14-item FIS has three (parental emotions, parental/family activity, and family conflict). The validity and responsiveness of the P-CPQ and the FIS have been demonstrated recently with children undergoing dental treatment under general anaesthesia in New Zealand [5, 6].

OHRQoL measures tend to be long, and short-form versions are more desirable because respondent burden is minimised and the chance of the measures being used routinely in day-to-day practice is greater. Consistent with the usual pattern with such measures, the development of short-form versions has taken two different paths and resulted in two closely related sets of measures: the 13-item Early Childhood Oral Health Impact Scale, or ECOHIS [7], and the short-form P-CPQ (both 8- and 16-item versions) and FIS (with 8 items) [8]. The ECOHIS was developed using the original 45-item pool used by Jokovic and Locker in developing the P-CPQ scales. The ECOHIS team obtained ratings of those items from health professionals (and associated staff and researchers) who were experienced in dealing with young children. The 36 items remaining from that process then underwent item reduction with a convenience sample of 30 parents of 3-5-year-old children with a range of dental care needs. This resulted in 13 items (9 child-related and 4 on family impact) which were then field-tested with a convenience sample of parents/caregivers of 5-year-olds selected for participation in a larger epidemiological study. Those children had not been selected specifically because they had early childhood caries (ECC). By contrast, the short-form P-CPQ and FIS measures [8] were developed and tested by item impact analysis of data collected in two New Zealand studies (in Wellington and Auckland) of changes in OHRQoL in ECC-affected children undergoing dental treatment under general anaesthesia [5, 6]. The item impact analyses were undertaken with the Wellington study data-set, and the examination of validity and responsiveness was done with the Auckland one.

Cross-sectional construct validity, reliability and responsiveness have been shown to be acceptable for both the ECOHIS [7, 9, 10] and the short-form P-CPQ and FIS measures [8], but there has been no direct comparison of their properties to date. It could be argued that the similarity in item content of the ECOHIS’s child section and the 8-item short-form P-CPQ (whereby the latter contains 5 of the 9 items in the former) means that there would be little difference between them in performance. By contrast, the family section of the ECOHIS and the 8-item FIS differ considerably; the former comprises the two domains of parent distress and family function, and the latter includes the three domains of parental emotions, parental/family activity and family conflict, all of which were in the original FIS. Only 3 items are common to both scales, and the ECOHIS omits the item pertaining to disrupted sleep, an impact which most parents of ECC-affected children would rate as being important. Its omission most likely results from the use of an epidemiological sample (rather than a clinical one) for field-testing the ECOHIS. It might therefore be expected that the real interest in making a direct comparison of the two instruments lies in the relative performance of the components which measure the impact of a child’s condition upon the family and household. Accordingly, the aim of this study was to compare the psychometric properties and responsiveness of the ECOHIS and short-form P-CPQ and FIS scales among New Zealand children with early childhood caries who underwent treatment under general anaesthesia (GA).

This secondary data analysis used data from pretest/post-test clinical studies of consecutive clinical convenience samples undertaken in Wellington in 2005 [5] and Auckland in 2010/11 [6]. Each study obtained prior ethical approval, and written informed consent was obtained from participants before data collection. Parents of children with ECC were asked to complete questionnaires before and after dental care provided to the child under general anaesthetic (GA). Those contained the full P-CPQ and FIS instruments (and therefore also the ECOHIS) and the global rating question “How much is your child’s overall well-being affected by the condition of his/her teeth, lips, jaws or mouth?”, scored on a 5-point scale ranging from ‘Excellent’ to ‘Poor’ (and asked before the other scales). The reference period for the baseline questionnaire was 3 months. Follow-up evaluations took place 1-3 weeks after the child’s procedure, and the follow-up questionnaire asked about the period since the child’s operation. Full details of the methods used in each study are in those earlier reports [5, 6]. The IRBs were the Central Regional Ethics Committee for the Wellington sample, and the Northern X Regional Ethics Committee for the Auckland sample.

Descriptive data on the participants at baseline and follow-up are presented by sample in Table 1. There were fewer Europeans and more Pacific Island children in the Auckland sample, and it was slightly younger, on average. The follow-up rate in the Auckland sample was higher than that in the Wellington one.

This study set out to compare the properties and responsiveness of the ECOHIS and short-form P-CPQ and FIS scales, using data obtained from the parents of New Zealand children with early childhood caries who underwent treatment under GA. It has found that the ECOHIS-Child and the P-CPQ scales are very similar in their internal consistency reliability, cross-sectional construct validity and responsiveness, but that the ECOHIS-Family and the FIS-8 differ in some important ways, despite being similar in their responsiveness.

Before discussing the findings, the limitations of the study must be considered. First, it was a secondary analysis of data collected using the long-form (original) versions of the instruments. The extracted data were used not only in the current comparison, but also in the development of the short-form P-CPQ and FIS measures. Second, ethical concerns meant that we did not investigate test-retest reliability: it would have been an imposition on parents who had already been through a stressful time. Third, we did not investigate family structure and functioning, perhaps compromising our investigation of family impact and how it changed.

Turning to the findings, it appears that, at least for determining changes in OHRQoL associated with treatment for early childhood caries, the ECOHIS-Child and the short-form P-CPQ scales are comparable. This means that investigators planning to use a parent-reported measure of child OHRQoL in monitoring the outcomes of treatment could use either measure, whether the ECOHIS or the 8- or 16-item version of the P-CPQ. However, it could be argued that, other factors being equal, it is preferable to use a measure which adequately covers all four domains. In this respect, the ECOHIS-Child’s relative oversampling of the functional limitations domain and under-sampling of the oral symptoms one may be problematic. As discussed earlier, that most likely reflects the development process for that measure, where field-testing used an epidemiological sample (rather than a clinical one), and there may not have been the same prevalence or impact of symptoms in that sample. Thus, either of the two short-form P-CPQ scales would be preferable for work in clinical samples where disease levels are high, and the ECOHIS-Child might be better deployed in epidemiological survey work.

Turning to the family impact measures, the ECOHIS-Child fell short of the FIS-8 on all aspects except responsiveness (where it was equivalent). Its internal consistency reliability was short of the 0.80 which is deemed to be acceptable [11] (although that is partly a consequence of the lower number of items). The data presented in Figure 1 indicate that its cross-sectional construct validity was inferior (if, indeed, the gradient in mean scores across the response categories for the global question can be taken to be an adequate representation of that). However, its face validity is arguably its greatest weakness, with one of the three family impact domains not sampled at all, and only one item representing another. To be fair, it does use the two parental emotions items which had the greatest impact, but the under-sampling of the parental/family activity and family conflict domains is problematic, and the omission of the disrupted sleep item is particularly so, given its high impact in the New Zealand data. This difference in item content clearly reflects the scales’ different provenance: that of the FIS-8 arises from its testing in children with severe caries, whereas the epidemiological origins of the ECOHIS-Family mean that the more severe effects on families were not sufficiently prevalent to make an impact.

A noteworthy inclusion in the ECOHIS-Family is the item pertaining to the child’s condition causing financial difficulty for the household. That particular item is not included in the FIS-8 because of its low impact. It placed 11^th out of the 14 original items, meaning that it would not have been included in the FIS-8 even if it had been seriously considered (its rank in the Auckland sample was no different from that in the Wellington sample, either). It is also worth considering that the relevance of such an item would differ according to the health system in which the instrument was used, and that would unnecessarily complicate international comparisons.

This investigation of the properties and responsiveness of the ECOHIS and short-form P-CPQ and FIS scales in New Zealand children undergoing dental treatment under GA for early childhood caries has found that the ECOHIS scales have some weaknesses which undermine their suitability for use with children with that condition. They may be better deployed in epidemiological survey work rather than in health services research with high-caries-experience samples. By contrast, the P-CPQ-8, P-CPQ-16 and the FIS-8 seem to be well suited for the latter, but their epidemiological utility remains to be demonstrated.