Can the health related quality of life measure QOLIBRI- overall scale (OS) be of use after stroke? A validation study

Patients were recruited during hospitalization in the stroke unit or by telephone within 3 months of discharge. Participants were asked to provide written consent. A local coordinator at all participating hospitals distributed the questionnaires by mail. Self-reported data were collected 3 months after stroke. Incomplete questionnaires were completed by filling in all missing items after an additional telephone interview. When up to two responses on any questionnaire were missing, mean imputation was performed. Questionnaires with more than 2 missing data points were excluded. Test-retest analysis of the QOLIBRI-OS was performed at 12-month follow-up due to the expected stability in functioning post-stroke [27] at this time point. The first 40 participants who answered at 12 months were asked to complete the retest in a 7- to 12-day period. Of these, 36 participants completed and returned the QOLIBRI-OS within the timeframe, which provides a response-rate of 90%. We conducted statistical tests (e.g., Student t- and chi-square tests) comparing the retest group (n = 36) with those not retested (n = 89), but no significant differences in any demographic characteristics or stroke severity emerged.

Information about age, gender, living condition and stroke was collected from the Norwegian Stroke Registry. Questions regarding education, marital status and work status were included in the mailed questionnaires, or were collected from the medical records after consent. Function was assessed with the Modified Rankin scale (MRS) [28], a clinician-reported measure of global disability widely used to evaluate post-stroke outcomes [28]. The scale consists of six categories assessing the level of independence, ranging from independent to bedridden or death. There is extensive evidence on the validity of the MRS [28]. In our study, the MRS was registered at 3 months after telephone interviews, as part of the national stroke registry registration.

Sociodemographic and clinical characteristics of the 125 participants are shown in Table 1. The average age was 70.5 years, and 62% were male. Approximately 50% of patients had less than 11 years of education, and three out of four had retired before stroke.

At 3 months after stroke approximately 75% lived at home without personal assistance. Compared to those who did not respond (n = 36), participants were 5 years younger and a larger proportion lived at home at 3 months. The participants and non-responders differed statistically significantly in age, MRS score at 3 months, and proportion living in an institution and in need of assistance. Gender and stroke subtypes were similar in both groups (Table 1).

Comparisons between participants and patients that were eligible for the validation study, but did not participate, were performed only for age and gender for ethical reasons. However, there were no statistically significant differences in these demographic data between the participants and the patients who refused to participate or between those who, due to a administrate failure, were not contacted.

The QOLIBRI-OS comprises six items that assess the degree of overall satisfaction with “Physical Condition”, “Cognition”, “Emotions”, “Ability to Perform Daily Activities”, “Personal and Social Life,” and “Current Situation and Future Prospects”. A Likert scale provides the following five response categories: not at all (score 1), slightly (score 2) moderately (score 3), quite (score 4), very (score 5) for each item [22]. Accordingly, item score range is 1–5 and sum score range 6–30.

Von Steinbuchel et al. [22] arithmetically converted the sum of all items to a percentage scale (0–100). In the present study, both the raw item scores and the overall sum score were used. The QOLIBRI-OS has demonstrated good internal consistency with a Cronbach’s α value of 0.86 in patients after TBI [22] and 0.88 in patients with subarachnoid haemorrhage. [24]. The QOLIBRI full scale (37 items) questionnaire has been examined in a Norwegian study of patients after TBI and showed metric properties supporting the reliability and factor structure. To date, the QOLIBRI-OS (6 items) has not been validated in Norwegian samples. ​ The QOLIBRI-OS was translated into Norwegian in 2008 in accordance with recommended procedures and is used in a longitudinal international observational study (the European Union Study CENTER-TBI-HEALTH. 2013.2.2.1–1). [29, 30] The translation used in our study was slightly modified to improve language fluency, and checked with back translation by a professional translation service. According to a bilingual professional translator the semantic meaning in our Norwegian version express the meaning of the original English version.

The Hospital Anxiety and Depression Scale (HADS), originally published by Zigmond and Snaith in 1983 [31], is a widely used instrument that screens for non-vegetative symptoms of anxiety (seven items) and depression (seven items) [32]. The HADS items are scored from 0 to 3 with higher scores indicating worse symptoms. A cut-off score of 8 indicates a possible diagnosis of anxiety or depression [33]. The total score (HADS-14) can also be used as a global measure of psychological distress [34]. The HADS questionnaire has been applied several times in Norwegian populations [29], also post-stroke [35].

The EuroQol Five Dimensions Questionnaire (EQ-5D) [36] is a three-level generic HRQOL questionnaire comprising 5 items measuring the dimensions of mobility, self-care, ability to perform daily activities, pain/discomfort and anxiety and depression [37]. The levels are rated as 1, 2, or 3, indicating no (1), some (2), and considerable problems (3). Each dimension can be scored separately. The questionnaire includes the EuroQol Visual analogue Scale.

(EQ-VAS), which is a 0–100 visual analogue scale intended to measure actual self-reported health status from worst to best imaginable health [38].

The Stroke Specific Quality of Life (SS-QOL) scale [14] assesses the functional impact of stroke across 12 domains using 49 items and a five-point Likert scale where higher scores indicate better functioning. The SS-QOL measures energy, mood, family roles, language, mobility, self-care, social roles, thinking, personality, and upper extremity function, vision and work/ productivity. A sum score can be extracted from each domain. Separate domain scores are obtained from unweighted average of all items belonging to a particular domain, but the overall SS-QOL score is most often used as the primary outcome. The SS-QOL scale has recently been translated into Norwegian in accordance with recommended procedures [39, 40].

The QOLIBRI-OS items were described in terms of means and distributional properties. The degree of floor and ceiling effects, as defined by more than 15% of responses in the extreme lower or upper categories of the scale, were reported [44].

A confirmatory factor analysis was conducted to examine the fit of the QOLIBRI-OS as a uni-dimensional model. The maximum likelihood with robust standard errors (MLR) was applied, as the item distributions were non-normal. Model fit was evaluated in terms of the root mean square error of approximation (RMSEA), the standardized root mean square residual (SRMR), the comparative fit index (CFI) and the non-normed fit index (NNFI) [45]. West et al. [45] suggest that RMSEA < 0.05, CFI > 0.95, NNFI > 0.90 and SRMR < 0.06 represent a well-fitting model, while CFI > 0.90, NNFI > 0.85, RMSEA < 0.08, and SRMR < 0.10 indicate a tentatively adequate model.

Cronbach’s α was used to investigate the internal consistency. A value larger than 0.70 is generally recommended for research purposes (e.g., group comparisons), whereas values above 0.90 is desirable for individual clinical assessment [46]. Correlations between QOLIBRI-OS items and its total score were examined (values > 0.40 are preferable) [44] to identify items contributing poorly to the reliability or the ranking of the patients. Test-retest reliability was evaluated with intra-class correlation coefficients (ICCs) based on a two-way mixed model (i.e., treating items and subjects as fixed and random components, respectively). Both ICC absolute agreement and ICC consistency estimates were extracted for comparison purposes [47]. ICC consistency values > 0.75 was considered as excellent.

The degree of missing QOLIBRI-OS data was below 0.5% (Table 3). Single items were moderately positively skewed. The QOLIBRI-OS total score did not show floor or ceiling effects according to the COSMIN criterion we used, whereas a modest ceiling effect (defined as > 15%) was observed in all items with the exception of one (“Physical condition”). All items robustly contributed to the overall QOLIBRI score, with all item-total correlations above 0.4 (ranging between 0.73–0.88).

The model fit indicators of the hypothesized one-factor model were not universally good (robust χ²_df = 9 = 21.83, p = 0.009). Although the relative fit indices were good (CFI = 0.972 and NNFI = 0.953), the important non-centrality index (RMSEA = 0.107) was poorer as opposed to the absolute difference in unexplained standardized residuals that were low (SRMR = 0.029). This model thus yielded mixed support. Removing a single item, i.e., item 3 (“Overall, how satisfied are you with your feelings and emotions?”), yielded a model with excellent universal fit (robust χ²_df = 5 = 3.47, p = 0.63; RMSEA = 0; SRMR = 0.015; CFI = 1.0; NNFI = 1.0).

As shown in Table 3, the ICC of the individual QOLIBRI-OS items were high and ranged from 0.75 to 0.91, whereas the overall score had excellent stability, ICC = 0.93.

Internal consistency of the QOLIBRI-OS overall score was excellent (Cronbach’s α = 0.93). We also calculated Cronbach’s α after removing the item “feelings and emotions”.

to observe changes in the internal consistency. In the resulting five-item scale, the Cronbach’s α declined from 0.93 to 0.90.

As the results of the CFA were mixed, and as the authors considered the item in question (item 3) important for evaluations of HRQOL after stroke, additional correlation analyses were performed. First, the correlation between the five-item (after removing item 3) and six-item overall QOLIBRI-OS was 0.99. Second, the correlations between the HADS, EQ-VAS and the SS-QOL, and the five-and six-item QOLIBRI-OS yielded almost identical results.

Analysis of the a priori hypotheses confirmed construct validity. All a priori hypothesis tests, apart from one hypotheses, showed correlations with the selected other measures in the presumed directions and magnitude (Table 2). The correlation between Physical condition and SS-QOL sum mobility was moderate 0.44, though hypothesised to be high.

The COSMIN criteria indicate that construct validity can be supported if the concurrent correlations with other criterion-related variables are in the magnitude and direction hypothesized or predetermined by the authors. The present results uniformly fulfilled the COSMIN criteria [44]. The lowest correlation was observed between the “satisfaction with physical condition” item and the EQ-5D “mobility” question; this finding is not surprising, as the EQ-5D assesses walking ability in isolation, thus overlooking upper arm function and general health [38]. The highest correlation was observed in a negative relationship between item 3 on the QOLIBRI-OS, “satisfaction with feelings and emotions”, and the HADS total score, which assesses psychological distress [31]; this result is in accordance with previous findings [50]. Emotions contribute substantially to HRQOL, and the high correlation between the QOLIBRI-OS emotion item and mental distress supported maintaining this item, even though the CFA indicated that it might be potentially problematic.

The ICC was tested using both consistency and agreement methods. The results were nearly identical, indicating that the subjects provided rather identical responses. The test-retest stability was particularly high for the overall scale (ICC = 0.93), which is higher than in previously published studies (ICC = 0.81) [22]. This may relate to differences in time periods of assessment. In our study, all participants performed test-retest at 12 months, whereas in former studies of QOLIBRI-OS, test-retest was investigated from 3 months to 15 years post stroke.The test-retest stability of all single items were comparable excellent.

Summarized, the psychometric results of the QOLIBRI-OS administered after stroke in this study are comparable or better than the results determined after TBI and subarachnoid haemorrhage [22, 24].

The literature is ambiguous about the use of single items [51] to assess HRQOL, as single items are less reliable and valid than sum scores. Nevertheless, other scholars have reported that the reliability of global questions regarding HRQOL might be adequate [52‐54].

The EQ-5D [36] has scoring options that include the use of single items. In our study, all of the QOLIBRI-OS items appeared to be uniformly consistent. Means, item-total correlations and test-retest stability varied slightly between items and differed slightly from the results of the total QOLIBRI-OS scale. Moreover, the concurrent validity coefficients of the individual items were high, given the high correlations with criterion-related measures, such as the HADS and SS-QOL. A higher ceiling effect for single items compared to the total score can be expected because of more variation within sum scores. More patients after stroke are expected to have optimal function in one specific aspect assessed by the QOLIBRI-OS, than in all aspects.

For clinical and research purposes after stroke there is no single preferred choice of outcome measure yet [4]. We performed a literature search in PubMed from 2014 to 2016 and discovered that the MRS was by far the most commonly used outcome measure in stroke research studies published from 2014 to 2016. However, the MRS does not assess the patients’ subjective perspectives of their health and wellbeing and is unable to differentiate between physical and cognitive sequelae, which is an important argument for including a patient reported outcome measure (PROM).

However, can the QOLIBRI-OS, which is a brief measure, collect substantial information about important HRQOL domains for patients after stroke? In our opinion, the QOLIBRI-OS assesses the major consequences of stroke. Compared to the SIS [55] the QOLIBRI-OS contains one item measuring satisfaction with physical condition but lacks detailed measurements of strength and hand function. The SS-QOL which has 49 items, also includes domains that assess vision and energy [14]. Both the SIS and SS-QOL address communication. The lack of measurement of communication abilities (speaking and understanding) presents, in our opinion, a weakness of the QOLIBRI-OS for use post-stroke. The lack of a specific communication component is likely due to the fact that the instrument was developed only with generalizing overall questions, and the communication aspect was included in the overall item assessing cognition. In addition, motor activity was assumed to be included in the item assessing satisfaction with the physical condition. However, in stroke, communicative and motoric problems are frequent specific problems [56]. Therefore, we suggest that two additional new items should be developed and added to the QOLIBRI-OS- For instance, an item from the QOLIBRI scale regarding satisfaction with language and communicative skills and one item assessing motor function could be included and the scale should then be re-validated in a comprehensive stroke population. For the time being, however, we recommend the use of the QOLIBRI-OS in patients after stroke because it provides a short, reliable and valid index of HRQOL after stroke.

The strengths of this study are that a major proportion of the unselected stroke population admitted to UNN in 2014 is included. Patients were recruited from stroke units and followed through early rehabilitation, in both hospital and community settings. Of the consenting patients, 78% responded to the main questionnaire, despite the broad inclusion criteria and no exclusion of patients with aphasia or cognitive problems. All patients responded during the same time period post-stroke. The data quality was excellent, and the results were consistent.

A significantly higher portion of non-responders was institutionalized. However, the absolute number of patients with considerable functional deficits post-stroke was low in both groups. A total of 14% of non-responders versus 7% of participants had MRS scores of 4 or 5. This finding may limit the validity of the QOLIBRI-OS in the most severely affected patients post-stroke. Due to Norwegian ethical rules, comparisons between consenters and non-participants are possible for the variables age and gender only, which may limit the representativeness of the results. Furthermore, this study did not evaluate responsiveness to change.

The sample size of 125 patients is less than the first original multinational study of the validity of the QOLIBRI-OS [22], which included 795 patients after TBI and thus provided more substantial statistical evidence of the psychometric data quality. Our study is consistent with the sample sizes from other validation studies of HRQOL measures [18, 57].