Assessing the quality of life of health-referred children and adolescents with short stature: development and psychometric testing of the QoLISSY instrument

Patient inclusion was conducted through endocrinology centers in the respective countries. Centers identified potential participant families based on clinic records and solicited their participation by mail which included information about the study along with a consent form. The composition of focus groups was determined by age (8–12, 13–18 years).

The focus group discussion was based on a semi-structured interview developed in prior international paediatric HrQoL research [45]. It was moderated by a trained interviewer and an assistant, who asked questions beginning with life in general and then focused on well-being and functioning with dSS. All discussions were audio-taped and transcribed into the local language.

The transcripts were screened by the researchers and relevant verbatim statements were abstracted, taking care to retain the wording used by the focus group subjects. Each statement was annotated with an identification of age, gender, diagnosis, treatment status and country. Analysis of the statements was carried out in a consensus group meeting of the project investigators. All statements were copied onto individual cards in both their original language and in a translation to English conducted by bilingual (local and English) study center personnel. The use of English as a core language reflects the need to have a mechanism by which all investigators could evaluate and sort the statements into concepts. The content analysis was conducted via a card sorting procedure. The first stage was to separate the generic HrQoL statements from specific growth related HrQoL statements, the specific statements were sorted into conceptually similar areas and thematically labeled thus providing collections of statements reflecting the same theme in different languages. This process then was cross-checked by other groups so that the representativeness of the categories found was validated within the process. The statements were sorted into the dimensional models and sub-domains were used as guidelines for the inclusion of items that were relevant in terms of content. Each statement sub-domain was discussed in the expert group to determine the core statements which uniquely reflected the concepts. A decision was then made about the wording and structure of the response categories so that a self-reported, pen and paper questionnaire for children 8 to 18 years of age was available for pilot testing. In addition, the conceptual model developed as a result of the focus group categorization process led to the differentiation between HrQoL domains and their potential determinants.

The pilot questionnaire was assembled in English, and items as well as the response scales were back-translated into the respective languages by native speakers bilingual in English. The translated version was reconciled with the phrasing of the original focus group items generated in each country. Back-translations were conducted according to international standard translation procedures [46].

All previous focus group participants and subjects previously contacted but unable to attend the focus groups were invited to the centers to participate in the initial pilot test and cognitive debriefing exercise. Each country included at least 24 children or adolescents but was free to include additional subjects to ensure that a large enough sample was available. The pilot test questionnaire data were complemented by clinical, socio-demographic and psychosocial data of the young patients and their parents. A cognitive debriefing exercise followed the completion of the instrument by the participant families. In the cognitive debriefing, children and adolescents were asked to rate a subset of items regarding clarity and relevance for young persons with dSS, importance to the individual situation and if rewording could improve the item. A five-point, Likert-type response scale was used with response options of: not at all/never, slightly/seldom, moderately/quite often, very/very often, extremely/always. The understandability of these response scale categories was also assessed in the cognitive debriefing.

The results from the pilot test were analyzed with regard to preliminary operating characteristics and psychometric properties to guide further selection of items. Preliminary analysis results were reviewed together with the feedback from the cognitive debriefing, allowing the researchers to make decisions regarding item retention for the final field test version of the questionnaire.

In the field test the QoLISSY questionnaire was administered to newly identified study participants meeting inclusion criteria. Additional measures were included in the field test to identify potential determinants and components of HrQoL as well as for construct validation purposes. Among these was the KIDSCREEN-52, a generic HrQoL measure to estimate convergent validity [45]. Subjects approached for and agreeing to the re-test were sent a second mailing about two weeks later with the aim to receive data from 50% of the sample to examine test- retest reliability.

The operating characteristics and psychometric properties of the field test instrument were analyzed at the item and scale levels according to classical test theory [47]. Specifically, items were individually inspected for missing values, distribution characteristics (mean, SD, skewness and kurtosis) and item difficulty. Principal component factor analyses was conducted to explore the dimensionality of the core HrQoL domains and the additional determinant scales, as specified in the conceptual model derived from focus group results. On the scale level, analyses were conducted to evaluate the score distribution and the reliability of the scores (internal consistency, retest and split-half reliability in terms of intraclass correlation coefficients). Criterion validity of the QoLISSY instrument was tested via correlations with the generic KIDSCREEN instrument and via analysis of known group differences regarding diagnosis, treatment status and height.

Confirmatory factor analysis (CFA) was used to test the dimensional structure of the QoLISSY questionnaire. The models’ goodness of fit was assessed using the main fit indexes: maximum-likelihood χ2 p-value and χ2/degrees of freedom, comparative fit index (CFI) and root mean square error of approximation (RMSEA). The reference values of χ2/df ≤ 2, CFI ≥ .95 and RMSEA ≤ .06 were considered indicators of model´s good fit to the data; and the reference values of χ2/df ≤ 5, CFI ≥ .80 and RMSEA ≤ .10 were considered the threshold for assessing the model’s fit as acceptable. All statistical analysis analyses were performed using SPSS (IBM SPSS Statistics V 18) as well as with specific programmes which had been used in previous international HrQoL projects [48].

Results below are presented to provide information on the psychometric performance of the instrument and not to identify differences in populations or evaluate the impact of clinical characteristics on HrQoL, which would require a different study design.

A total of 84 patients, 38 children (8–12 years) and 46 adolescents (13–18 years), with ISS and GHD were recruited into 28 focus groups with an average 4 participants, specifically, the majority having been treated with GH (Table 1).

From the focus group discussions a total of over 4000 statements were identified and condensed. The statements were organized into domains and sub-domains (Figure 1). The statement list was reduced in an investigators consensus meeting by removing repetitious and ambiguous items. The most frequently expressed ideas consistent across countries were retained as well as frequently mentioned concepts within a country to ensure the greatest degree of content validity.

The conceptual model specifies the three core domains of HrQoL, (physical, emotional and social) as well as potential mediators or determinants in terms of coping, beliefs, treatment and concerns regarding the future. In addition, moderator variables such as age, gender, family situation and socio-economic status were included. This model conceptualizes HrQoL as an outcome or dependent variable, which is related to the child’s stature as an independent variable on the background moderators and describes how growth related HrQoL is affected by clinical and psychosocial mediators regulating the intensity of impact of dSS on HrQoL (Figure 2).

Following the statement reduction process a total of 124 items were formulated and arranged into 7 scales for child report. Individual items were constructed to include both positive and negative wording though the predominant direction describes impairment. Scale level scores were constructed based on the mean item score for the particular scale and standardized on a scale of 0 to 100 such that higher scores reflected better quality of life.

The pilot test was conducted on a convenience sample of 94 children (48 children and 46 adolescents across the European centers) and reflected the intended range of age, gender and diagnosis (refer to Table 1).

Some items were discarded at this stage because participants judged them negatively during the cognitive debriefing, e.g. the items were not applicable for the participants or were not easily understood. This information was combined with the results of the item analyses and preliminary psychometric testing. Using descriptive and psychometric indicators, the behaviour of items and domains or scales was assessed and poorly performing items were removed. This process was conducted in a second consensus discussion, resulting in a refined list of 53 candidate items across six of the originally seven scales. The Future scale was removed as a component of the children’s questionnaire and included as a domain in the parent version (see Table 2).

A previously identified convenience sample of parents with children meeting study inclusion criteria were mailed the instrument. A total of 544 questionnaires were sent out to families and 337 were returned to the growth clinics. The return rate across countries ranged from 48% to 92%. Subtracting empty or incomplete questionnaires as well as accounting for the fact that only parents but not their children had responded, 268 returned patient questionnaires were included in the analysis. (n=129: 8–12 years; n=139: 12–18 years, Table 3). In addition, 124 retest responses were received, representing about 50% of the sample as planned. Inspection of item performance resulted in discarding three items from the child version, so that the field test version included 50 items. The results were evaluated for reliability and validity, including criterion and construct validity.

Exploratory factor analysis was used both to identify the scale structure and to support the scoring system of the questionnaire. In line with the conceptual model, items representing the core domains (physical, social and emotional) were subject to a principal component factor analysis, with three factors extracted. The first factor with an Eigenvalue of 10.62 explained 48.3% of the variance, the second added 6.0% to the variance with an Eigenvalue of 1.33 and the third another 4.6% with an Eigenvalue of 1.03, all three contributed to a cumulative 59% of variance explained (full analysis for all items not shown here). Conceptually, the three core scales formed the QoLISSY-QoL total score, with the three domains (Coping, height-related Beliefs, Treatment) serving as additional determinant modules.

The scale distribution characteristics from the overall dataset suggest a relatively high level of HrQoL in the respondents with mean scores in the low 70’s for the core domains and total QoLISSY-QoL. The magnitude of the variance (SD) indicates good variability across the scales, and the low floor and ceiling effects as well as acceptable skewness and kurtosis suggest appropriate operating characteristics (Table 4).

In terms of internal consistency, Cronbach’s α reliability was 0.8 and above, with less than 10 items per scale. Split-half reliability was slightly lower but still satisfactory psychometrically (>.70) [47]. The test-retest reliability, calculated from the retest of a sample of 124 children and adolescents in terms of the intraclass correlation coefficient (ICC, last column in Table 4) was satisfactory with the exception of the Coping scale.

Construct validity was assessed by confirmatory factor analysis of the three core QoL scales and was subsequently repeated for the three mediator domains (not reported here).

The factorial model of the patient-reported QoLISSY core instrument, which comprises the physical, social and emotional domain and a total score of HrQoL, had an acceptable fit, with Χ2 (206, n = 263) = 615.35, p <.05, Χ2/df = 2.99, CFI = .88 and RMSEA = .087. All items showed factorial validity, with significant factor loadings (p < .001) and, except for one item of the social domain and of the emotional domain, standardized regression weights were above the threshold of .50 (Figure 3).

The QoLISSY instrument showed high inter-correlations of the three cores QoL scales while the association with the Coping subscale was lower (Table 5).

Criterion validity was examined in terms of convergent validity using a generic measure and in terms of known groups validity using participants clinical data.

The QoLISSY scale scores demonstrated significant, but only moderate, correlations with the physical and psychological well-being dimensions of the KIDSCREEN-52. These correlations support thematic agreement, but also suggest that the two instruments may not be measuring identical constructs (Table 6).

In terms of known group validity, significant differences were found between ISS and GHD patients (Table 7) between untreated vs. GH-treated children (Table 8) and between degrees of SS (very short vs. less short children, Table 9). Extremely short patients (≤ −2.5 SD, n = 53) were characterized by significantly lower HrQoL than those in the mid range (−1.5 to −2.49 SD, n = 116) or the taller patients in the upper range (0 to −1.49 SD, n = 77).

The operating characteristics and psychometric properties of the child version of the questionnaire in the field test thus fulfilled internal consistency, split half and retest reliability as well as content, construct and criterion validity criteria.