Patient-focused measures of functional health status and health-related quality of life in pediatric orthopedics: A case study in measurement selection

Study design factors are specified in detailed protocols that clearly define the study objectives and conditions. Well-defined study objectives identify the study subjects, and type and number of measurements required. Study conditions describe practical issues including data collection sites, and budgets for time and resources.

A highly focused, single objective study may require relatively few instruments while a broad-based, multi-faceted study will require numerous measures. For example, a RCT of efficacy for an experimental technique to achieve union of bone compared to a conventional technique might specify patients' reports from a single well-established walking ability scale as the primary outcome measure. However, an economic evaluation of the cost-effectiveness and cost-utility of the experimental technique would require at least two instruments: the single well-established walking ability scale to estimate effectiveness for the numerator in the cost-effectiveness ratio; and a utility-based scale of overall HRQL to estimate quality-adjusted life years for the numerator in the cost-utility ratio.

The age of study subjects affects the choice of measurement instruments and the method of collecting data. Each instrument is valid for a specific age range and the age range is quite small for many pediatric instruments. For example, developmental changes during childhood make it difficult for single instruments to be valid from infancy through adolescence. Many instruments are valid for one or more of the following age categories: infants, pre-school children, primary-school children, adolescents, young / middle-age adults, seniors and elderly. Inability to read well (e.g., young children or populations with high illiteracy rates), or see well (e.g., many elderly populations), or concentrate well (e.g., people on certain strong medications) inhibits use of self-complete questions. Well-designed interviewer-administered questionnaires pose clear, short, well-focused questions with readily understood and easily remembered response options (e.g., yes or no response options).

It is becoming widely accepted in many disciplines that studies of health care programs and technologies should include measures of patients' perceptions of their FHS and HRQL [12]. Measurements of these types have been shown to vary by assessment viewpoint and, although many different viewpoints are valid, the patient perspective is considered to be one of the most important. It has been shown that children as young as 7 years can reliably complete interviewer-administered disease-specific and generic questionnaires about their own health [13].

The locations and cultural characteristics of the study population will determine the language and sometimes the choice of assessors. Relatively few instruments have been carefully translated and culturally-adapted to facilitate use in a large variety of communities. Some cultures accept a variety of assessment viewpoints (e.g., patient, family, nurse, physician, and other allied health professionals) while other cultures recognize only a physician viewpoint for health assessments.

The literature suggests that there may be important effects due to mode of data collection [14] and, therefore, mode of administration should be standardized across subjects, assessors, and assessment points. In general, self-complete questionnaires tend to elicit reports of greater morbidity than interviewer-administered questionnaires. It is hypothesized that subjects may be less inhibited to report disabilities on self-complete questionnaires than to report disabilities to interviewers. The mode of data collection for a study may be determined by numerous factors other than the characteristics of study subjects mentioned above. A low budget study may rely on a mail survey, or distribution of questionnaires in a busy clinic setting, and use self-complete questionnaires. Alternatively, a study involving serial assessments may require follow-up data to be collected by telephone and therefore require use of questionnaires designed for interviewer-administration.

FHS questionnaires should have well-specified assessment periods to help ensure that the subjects and researchers know the period of time covered by the responses. Assessment recall period refers to the period of time the assessor is asked to consider when answering the questionnaire. The period should match the objectives of the study. For instance, if a new surgical technique is thought to reduce the peri-operative burden of morbidity then frequent assessments with a brief recall period (24 hours) might be suitable. Standard assessment periods include the past 24 hours, the past 1 week, the past 2 weeks and the past 4 weeks. Short assessment periods should be used in studies of patients whose FHS varies over time and in studies involving serial assessments to ensure that there is no overlap in assessment times. Relatively long recall assessment durations may be used when it can be assumed patients' health status is fairly stable. For example, a four-week recall assessment period was used to measure the HRQL of patients in a randomized clinical trial and economic evaluation of two alternative treatment strategies for patients with knee osteoarthritis [15, 16].

There are limits to the number and type of measures that respondents can be expected to complete without getting tired or frustrated, and/or that the study budget can afford in regards to both data collection and analysis. The evidence about the limits of respondent burden is sparse. All else being equal, studies involving serial assessments should expect to collect fewer measurements per assessment than single-assessment cross-sectional surveys. To maximize efficiency, instruments should be selected to provide complementary rather than overlapping information.

A valid measure is "sound and sufficient" [31]. There are many ways to assess validity of measures. Assessments of FHS and HRQL measures should consider at least six types of validity: face validity, content validity, construct validity, convergent validity, discriminative validity, and predictive validity [32]. Face validity requires that a measure appear on the surface to make sense in regards to being relevant and useful. Content validity requires that the measure include all important and relevant domains or dimensions of health status. Construct validity describes the extent to which a measure corresponds to theoretical concepts and convergent validity describes the association between related variables. Discriminant validity is a lack of correlation between dissimilar variables or groups. Predictive validity, one type of criterion validity, describes the relationship between current and future measurements.

A measure is reliable if it is sound and dependable [31]. Reliability is assessed by tests of repeatability or reproducibility. Reliability is often assessed in terms of agreement between intra-subject test-retest measurements and inter-assessor measurements [33].

Responsiveness is also referred to as sensitivity to change. It is an important feature for determining a measure's ability to detect effects of treatments or natural changes over time (e.g., due to the aging process). Husted and colleagues reviewed the literature and defined two major types of responsiveness: internal and external responsiveness [34]. Internal responsiveness describes the ability of a measure (instrument) to change and has been assessed using a variety of techniques including the magnitude of statistical significance tests (e.g., p < 0.05 versus p < 0.001), the mean change score divided by the standard deviation of scores at baseline (effect size), and a sensitivity coefficient calculated as the proportion of the variance in change scores due to treatment [32]. It has also been assessed as the ratio of the mean change in patients' scores and the pooled standard deviation of the mean change scores [35], and as the mean change score among those who changed divided by the standard deviation of change scores among stable patients [33]. External responsiveness is concerned with the relationship between change in a measurement and change in a reference measurement of health status. External responsiveness has been assessed using the receiver operating characteristic method, correlations (e.g., Pearson product moment correlation), and regression models. The minimum important difference (MID) is the smallest size of difference that is important from patients' or clinicians' perspectives. The MID between two measurements is a concept closely related to responsiveness when assessing change over time [36].

Ceiling and floor effects are undesirable properties that reduce the validity, reliability, and responsiveness of measures. A ceiling effect may occur when a large proportion of measurement observations are close to the upper bound of the measurement scale. A ceiling effect results in a positively skewed distribution of measurements, limited ability of the measure to discriminate among subjects at the upper end of the scale, and attenuated responsiveness to improvements in health in longitudinal studies. A floor effect may occur when a large proportion of measurement observations are close to the lower bound of the measurement scale. Floor effects create a negatively skewed distribution of measurements, limited ability of the measure to discriminate among subjects at the lower end of the scale, and decreased responsiveness to decrements in health in longitudinal studies. Many generic and specific measures of HRQL may be subject to ceiling effect problems in that they may not be able to describe patients or subjects with above average (supra-normal) health. Some measures are subject to floor effect problems. Some are subject to both. Typically floor effect problems are more serious in clinical studies (which often involve patients with disabilities) and ceiling effect problems may be more problematic in general population studies.

The limits of respondent burden depend upon many factors including the number of questions presented, how the questions are presented, the complexity of questions, the sophistication of respondents, and the respondents' interest in the questions. In general, the allowable length of questionnaire is shorter for mail and phone administration than for face-to-face interviewer administration [37]. One set of guidelines specifies the following maximum lengths: 20 questions for phone surveys; 60 questions for mailed surveys; and 80 questions for face-to-face interview surveys [38]. Another guideline recommends that telephone interviews not exceed 5 to 10 minutes [39]. These guidelines are in general agreement with maximum recommended number of pages for self-administered questionnaires: 2 to 4 page upper limit for topics not especially salient [40]; 12 page upper limit for self-administered questionnaires [41]; and 4 to 6 page upper limit for mailed surveys [42]. For mail-out surveys, the evidence suggests no effect of length on response rates for questionnaires varying from 3 to 9 pages [43, 44] but reduced response rates with questionnaires greater than 12 pages [41].

Applications of FHS and HRQL measures are greatly facilitated by expert advice, detailed instructions and other services designed to support users of a measure. Supporting documentation is usually protected by copyright and should not be used without written permission of the original developers. Documentation obtained from third-party sources should be considered suspect because it is frequently invalid. Licensing fees are used to fund high quality, readily accessible service centers. Permission to use copyright materials is typically granted one study at a time. Support services may also include consultation about the most appropriate versions of questionnaires for use in a specific study. Application packages may include data collection instruments such as questionnaires, procedure manuals, coding algorithms and scoring systems, as well as background information about the conceptual and measurement properties of the instrument.

Recently there has been interest in using measures of FHS and HRQL to evaluate treatment alternatives for NF1-CTD. NF1 is one of the most common genetic disorders in childhood [45]. It is estimated that at least 1 million people throughout the world have NF1 [46]. NF1 has a wide range of clinical manifestations including abnormalities of the skin, nervous system, bones and soft tissues [46]. Other conditions experienced by children with NF1 include short stature and neurologic problems such as learning disabilities or unspecified school performance problems (36%), frequent headaches (28%), mental retardation (6%), and reduced reproductive potential [46‐49].

CTD is rare in the general population, approximately 1 per 140,000 [46]. It has been estimated that approximately 1% of people with NF1 have CTD [46]. CTD is diagnosed usually during the first year of life and fractures often occur before 3 years of age. Frequently, initial presentation is tibial bowing followed by subsequent fracture and pseudoarthrosis [45]. There is no generally accepted standard for management of CTD although most surgeons would suggest initial treatment of either intramedullary fixation with bone grafting or resection and bone transplant. Surgical procedures for the treatment of CTD are fraught with complications and failure of union. For the treatment of CTD, pre-fracture bracing until skeletal maturity may be a better alternative than surgery. CTD is associated with severe complications due to nonunion or pseudoarthrosis after osteotomy and amputation may be required.

Conventional clinical measures of CTD include the Crawford classification system [46]. These measures provide clinicians with important information used in diagnosis and management of well-established symptoms. A list of important concerns could be prepared by interviewing patients and members of their families. Standardized comprehensive tools that integrate multi-dimensional effects would also be useful in quantifying the number and extent of problems experienced by NF1-CTD patients, and other pediatric orthopedic patients with complex issues. The published literature on NF1 and NF1-CTD contains virtually no information based on FHS or HRQL measurements. The exception is a recent paper by Wolkenstein and colleagues [50] who reported results from 128 adult patients in France using the generic health profile SF-36 and a skin-disease-specific measure, Skindex-France.

Surveys of the published literature, experts in the fields, web sites and other sources of information were conducted to determine the dimensions of health that are affected by NF1-CTD, the types of FHS and HRQL measures that have been used, which measures should be considered as potentially useful for studies of NF1-CTD, the measurement properties of potentially useful measures, and the relative merits of various measures. A review of the on-line Quality of Life Instruments Database (QOLID) developed by Dr. Marcello Tamburini and the MAPI Research Institute [51], and correspondence with instrument developers, identified a short list of potentially useful measurement tools in each of the following categories: generic preference-based HRQL systems, major pediatric and other generic health profiles, and disease or function specific measures. Selected measures should have demonstrated properties in accordance with currently accepted criteria [12, 52, 53] and should provide commensurate measurements for patients across a wide age range. Problems with mobility, cognition, pain, emotion (including impacts of problems with self-image), self-care, vision, and fertility are aspects of health reported in the published literature to be compromised in NF1 patients.

There are five important research objectives of an NF1-CTD study that provide a context for applying the framework described in the Methods section:

These detailed objectives require the identification and assessment of leading FHS and HRQL measures for use in both cross-sectional and prospective longitudinal surveys.

The prevalence of NF1-CTD is relatively low. Patients will need to be recruited from numerous clinical centers in North America to generate precise estimates of FHS and HRQL. Questionnaires should be available in at least 3 major languages: English, French and Spanish. The survey population ranges in age from newborn into adulthood and linking results across the study objectives requires that at least some of the assessment tools be in common across the age range of study patients. To avoid potential confounding effects, data collection techniques should be consistent across subjects and measures.

The patient-focus will be represented by collecting measurements from all patients old enough to provide self-assessments, and from parents acting as proxy assessors for all children and adolescents. Self-complete questionnaires requiring minimal supervision should be used to eliminate the need for interviewers at each clinical center, to facilitate use of mail-out surveys, and to avoid potential "interviewer" effects. The number and type of measures per assessment, and the number of serial assessments per patient, should be sufficient to address all the study objectives within the limits of study resources and assessor burden. Measures of morbidity associated with NF1-CTD should be comparable with data on norms from surveys of general populations and other patient groups, and be useful for assessing the effectiveness and efficiency of health care services.

The HRQL measure should be comprehensive and preference-based, to facilitate a broad variety of comparisons. A pediatric health profile measure and other specific measures will be selected to complement the selected preference-based HRQL measure. FHS measures may be focused on one or more of the following: the population of interest (e.g., pediatrics); the major underlying disease (e.g., NF1); the major human function of most interest (e.g., walking ability); the medically-defined health problem of most interest (e.g., tibial dysplasia); the medical speciality most involved with treatment of the health problem (e.g., pediatric orthopedics).

There are six major generic preference-based HRQL utility systems [21], presented here in chronological order of development: QWB [25], 15D [54], HUI [23], EQ-5D [55], AQOL [56] and SF-6D [57, 58]. HRQL scores from these systems represent mean community preference scores. The 15D and AQOL have not been widely used outside of Finland and Australia respectively and, therefore, will not be described further in this paper. The SF-6D has been developed only recently so there is as yet little evidence to report. The major features of QWB, HUI, EQ-5D, and SF-6D systems are summarized in Table 1[21, 25, 57, 59, 60]. The major characteristics vary greatly among the systems. For example, linear additive scoring models do not include effects of preference interactions among attributes or domains but multiplicative scoring functions include these effects. The QWB is available in both self-complete and interviewer-administered formats [61]. The symptoms attribute is a dominant feature of the QWB health status classification system. This emphasis is reflected in the population-derived preference weights. HUI health status classification systems cover more than 10 attributes. There is evidence that HUI scores agree well with mean directly measured standard gamble utility scores from a representative sample of the general population [59, 60, 62, 63]. Numerous versions of HUI questionnaires are available and HUI has a service center [64‐66]. It is available in numerous languages. A closely-related comprehensive health status classification system for pre-school children (CHSCS-PS) has been developed recently [67‐69] for children age 2 through 5 years of age. EQ-5D is very simple and concise. It consists of 5 attributes with 3 levels per attribute, assesses "current" health status, has been used in a large number of studies, and is available in numerous languages. Information, including a long list of references, about EQ-5D is available on the EuroQol Group web site [70]. SF-6D is a multi-attribute health status classification system based on the SF-36 [19, 71, 72]. The SF-36 was not designed to be commensurate with the fitting of a multi-attribute utility function. The SF-6D health status classification system is a sub-set of the attributes defined in the SF-36 health status classification system [57]. SF-6D utility scores may be useful in retrospective studies analyzing previously collected SF-36 data.

The major population-specific health profiles include the Child Health Questionnaire (CHQ), Pediatric Inventory of Quality of Life (PedsQL), Pediatric Evaluation of Disability Inventory (PEI) and TNO-AZL Pre-School Children Quality of Life questionnaire (TAPQOL). The PEI is limited to children age 0.5 – 7 years of age and requires a structured parent interview or clinician observation [73]. TAPQOL [74, 51] is limited to children 0.5 to 5 years of age. Therefore, PEI and TAPQOL will not be discussed further.

The major pediatric disease-/function-/specialty-specific instruments include the PODCI (also referred to as the POSNA or Pediatric Orthopedic Society of North America instrument), ASK, Gillette Functional Assessment Questionnaire Walking Scale (FAQ walking scale), and Wee-FIM. Wee-FIM [75], a popular measure of functional independence, is not being considered because it involves clinician assessments rather than assessments from a patient or parent perspective. In general, disease-specific scales in orthopedics focus on pain and physical function because these factors are major areas of concern for orthopedic patients and no generic health measures have been developed specifically for orthopedic application [73]. No relevant disease-specific measures or disease-specific preference-based tools were identified.

A summary of the major pediatric generic health profiles appears in Tables 2 and 3. The CHQ [76] covers relevant physical domains and provides detail on emotion/psychological health. The PedsQL [77] assesses physical, emotional, social and school functioning. It has demonstrated a return to health 3 months after acute limb fractures [78] and has been used in large general population surveys [79].

Table 4 summarizes the major characteristics of four orthopedic-specific measures. The PODCI [80] was designed specifically as a very comprehensive measure of musclo-skeletal outcomes associated with pediatric orthopedic problems. ASK was designed to measure children's activities in terms of both capacity and performance [81], and it assesses domains not covered in detail by other instruments [82]. The FAQ walking scale provides the most complete measure of walking abilities.

The choice of existing measures is based on a process of elimination considering the relative strengths of each instrument and the complementarities among measures. Neither the SF-36 nor the EQ-5D is valid for use in adolescent patients with orthopedic problems [83]. A review of measurement of HRQL in children by Eiser & Morse [[84]; see also [85, 86]] identified HUI and CHQ and PedsQL as the only 3 generic measures that fulfill all specified review criteria: established reliability and validity; suitable for self- and proxy-report; and brief (<30 items). PODCI outperformed CHQ physical functioning scale for orthopedic patients [87]. However, PODCI has considerable problems with missing data, especially in upper extremity function and physical function and sports scales for children ages 2 to 5 years, associated with the use of "too young" response options [88]. ASK is reported to be more sensitive to change in disability levels than HUI [Young N, personal email communication to W Furlong 2002-02-18]. The FAQ walking scale provides the most complete assessment of functional walking abilities, especially at the upper end of the scale [89].

In summary, there are few measures available for assessing subjects less than 5 years of age and even fewer for subjects less than 2 years of age. Most relevant measures are available in self-complete format only. Preliminary recommendations for the NF1-CTD study were that PedsQL be used as the generic health profile, HUI be the multi-attribute preference-based measure of HRQL utility scores for children age 5 years and older and that CHSCS-PS be the measure for children age 2 through 4 years, ASK be the measure of activity limitation, FAQ walking scale be the measure of walking ability, and that a small feasibility study of these instruments be completed with a convenience sample.

A pilot feasibility study surveyed 8 NF1 patients using HUI and FAQ walking scale measures. Questionnaires were completed by 6 NF1 patients and 3 parents. The combined HUI and FAQ walking scale questions took respondents an average of 13 minutes (range, 9–20 minutes) to complete. The patients were 11 to 50+ years old and had health problems ranging from mild to severe. One patient with tibial dysplasia and 2 patients with scoliosis were included.

HUI data were collected from 5 patients, 2 parents and both the patient and parents in one case. Health problems were reported in 7 of the 8 HUI3 attributes (vision, speech, ambulation, dexterity, emotion, cognition, and pain; no problems with hearing were reported). The attributes associated with the most morbidity, as assessed using HUI3 single-attribute utility scores [60], were pain (mean score = 0.81), speech (0.94), cognition (0.94), and emotion (0.94). For the 7 patients having complete data, 5 had two or more HUI2 and HUI3 attributes at less than full function. On the conventional utility scale in which being dead = 0.00 and in perfect health = 1.00, the HUI3 scores ranged from 0.45 to 1.00. The mean HUI3 score, 0.73, is similar to the mean score of 0.77 for adults with arthritis [90].

FAQ walking scale data were collected from 4 patients (1 of the 5 survey patients did not answer the question), 2 parents and both the patient and parents in one case. Five patients were reported to be at Level 10 (walks, runs, and climbs on level and uneven terrain and does stairs without difficulty or assistance), one patient to be at Level 8 (walks outside the home for community distances, is able to get around on curbs and uneven terrain in addition to level surfaces, but usually requires minimal assistance or supervision for safety), and one patient at Level 6 (walks more than 15–50 ft. outside the home but usually uses a wheelchair or stroller for community distances or in congested areas).

In summary, the feasibility study showed that the HUI and FAQ walking scale questions were acceptable to patients' families and that results, especially for HUI, reflected the large variability in HRQL of the sample of patients.

No single measure will provide sufficient data to address all the important study objectives. A set of measures is required. The set of measures should provide complementary data of health status and preference-based scores of HRQL. Redundancy in measurement is reduced, and efficiency of measurement is increased, by selecting the most comprehensive generic measures and then supplementing these measures with the most appropriate set of specific measures.

It is recommended that HUI be selected as the comprehensive generic measure for ten reasons:

The HUI will provide a broad set of measures for comparisons with other populations and for estimating HRQL on a general scale such that dead = 0.00 and perfect health = 1.00. As a generic measure, HUI also has the ability to capture side effects and the effects of co-morbidities. However, these broad measures may not be responsive to small but important changes in health status. Therefore, HUI should be complemented by a set of instruments focused on pediatric, orthopedic and walking issues.

The PedsQL 4.0, a pediatric generic health profile, should also be included in the set of measures because:

Two specific measures should also be part of the set of instrumentation: the ASK and the FAQ walking scale. ASK is an orthopedic-specific instrument which has been shown to cover the most important domains in the context of musculoskeletal disorders, including the impact of limb lengthening surgery experienced by many children with tibial dysplasia. ASK is also attractive because it provides overall summary scores for both performance and capability measures, and is only moderately burdensome to complete. Walking ability is one of the most important aspects of health that is frequently compromised in NF1-CTD patients, and the FAQ walking scale is the most complete scale of functional walking ability currently available and it only requires asking one question.

CHQ is not recommended because it does not add much to the set of recommended measures and it is burdensome to complete. PODCI is not recommended because it is very burdensome to complete, it has been reported to have major problems with "missing data", and the system for collapsing questionnaire responses into summary scores is not well validated.

Children ages 2 to 5 years of age should be assessed by their parents using three questionnaires: the CHSCS-PS (12 questions), the PedsQL (23 questions); and the FAQ walking scale (1 question). It is expected that all three of these questionnaires can be completed in an average of 15 minutes.

Children and adolescents ages 5 to 17 years of age should be assessed by their parents using four questionnaires: the HUI (15 questions), the PedsQL (23 questions), the FAQ walking scale (1 question), and the ASK (30 questions). These four questionnaires are expected to be completed in an average of 20 to 30 minutes.

Children and adolescents older than 11 years should provide self-assessments using four questionnaires: the HUI (15 questions), the PedsQL (23 questions), the FAQ walking scale (1 question), and the ASK (30 questions). On average, it is expected that respondents will complete all four questionnaires in 20 to 30 minutes.