Vulnerability in elderly patients with gastrointestinal cancer – translation, cultural adaptation and validation of the European Portuguese version of the Vulnerable Elders Survey (VES-13)

The authors followed the European Organization for Research and Treatment of Cancer (EORTC) guidelines - Quality of Life Group Translation Procedure; and Guillermin et al. recommendations [18, 19]. Briefly, the original questionnaire was translated into European Portuguese and culturally adapted by two healthcare professionals with English fluency, knowledgeable of the translation purpose. This draft version was translated back into English, by two English translators, and compared to the original questionnaire by the investigators and the original VES-13 authors, to assess comprehension of the applied concepts and wording. No problems were identified at this stage.

Face validity of the translated questionnaire was assessed by six medical oncologists at our GI Cancer Clinic. They were asked to review the original and translated questionnaires and classify each question, according to comprehension and accuracy of the translation, using a numerical rating scale of 10 points (1 - poorly clear, to 10 - completely clear).

Cancer patients admitted at our Comprehensive Cancer Centre age ≥65 years with histologically confirmed GI Cancer, Portuguese fluency, and no history of previous systemic therapy for cancer were eligible for both the pilot and prospective validation cohort. Patients presenting cognitive impairment, confusional syndrome or who were illiterate or foreign individuals were excluded from the pilot study. The pilot study also excluded patients unable to read.

This work has been approved by the ethical committee of the “Instituto Português de Oncologia do Porto” in Portugal, institution where it was developed and all the subjects gave their informed consent.

The questionnaire was applied by one of the investigators to included consecutive patients (first pilot n = 20, second pilot n = 22) who were asked to rate each question for comprehension using the previously described numerical rating scale of 10 points. Each patient completed the VES-13 questionnaire twice within 1 to 30 days. At this point, a question was to be reviewed if it had a single rating ≤5 (corresponding to reasonably clear), or if any comprehension problem was noted by the interviewer. Concerns regarding question 3f made necessary a second pilot, after questionnaire adaptation.

After completion of the pilot study, the European Portuguese version of VES-13 was prospectively applied to a cohort of 200 patients to assess internal consistency and construct and criterion validity [20‐22]. To assess construct validity we selected EQ-5D-5L as comparator [23]. EQ-5D-5L is a generic health related quality of life questionnaire which includes five dimensions and a visual analogue scale (VAS) assessing general health. Each dimension is recorded in five severity levels (no problems, slight, moderate, severe and extreme problems, graded from 1 to 5, respectively). The VAS records an individual’s rating for their current health-related quality of life (ranging from 0 - worst imaginable health, to 100 – best imaginable health). Predefined hypothesis about relationships among dimensions of EQ-5D-5L and VES-13 were tested to assess construct validity. To assess criterion validity we used the clinical impression of a trained medical oncologist, blinded to the responses on VES-13, regarding patient’s vulnerability and performance status (PS). Each medical oncologist was instructed to consider the Eastern Cooperative Oncology Group classification – ECOG [24] and Karnofsky scale – KPS [25], and to categorize each patient into the following groups: fit, vulnerable or frail. Performance status was estimated according to exact ECOG PS and KPS scales definition (ECOG PS ranging from 0 –able to carry on all pre-disease performance without restriction, to 5 – dead; KPS ranging from 100 – normal, no complaints, to 0 – dead). Correlations among these criteria and VES-13 were evaluated to assess criterion validity.

The test-retest reliability of the Portuguese version of VES-13 was assessed in the pilot study by calculating the Kappa statistic for each item to assess agreement between test and retest scores [26]. This index takes values between −1 and 1; values near 1 show high test-retest reliability. The categorization by Landis and Koch was used for interpretation of κ values (<0.00 – no agreement; 0.01-0.20 – slight; 0.21-0.40 – fair; 0.41-0.60 – moderate; 0.61-0.80 – substantial and 0.81-1.00 – almost perfect agreement) [27]. Additionally, we calculated the test-retest reliability coefficient (correlation coefficient) for VES-13 total scale score.

Internal consistency of translated VES-13 items was explored by analyzing the inter-item and item-total correlation matrices and calculation of Cronbach’s alpha coefficients. This coefficient ranges from 0 to 1, and larger values indicate higher internal consistency. As recommended by Nunnally and Bernstein, alphas ≥0.70 were considered adequate [21]. An estimation of Cronbach’s alpha if an item were to be deleted from the scale was used to identify which items affected the questionnaire’s internal consistency the most.

Construct and criterion validity was assessed by calculating Spearman’s correlation coefficient between VES-13 and each EQ-5D-5L dimension, clinical judgment and performance status. Interpretation of correlation coefficients was based on the quantitative criteria and qualitative descriptors defined by Cohen [28] (low correlations for coefficients with absolute value between 0.10 and 0.29; moderate correlations for coefficients between 0.30 and 0.49 and high correlations for coefficients between 0.50 and 1.00).

Exploratory factor analysis for VES 13 European Portuguese version was performed using principal components analysis for factor extraction. The hypothesis of unidimensionality of VES-13 was assessed. Selection of the number of factors to retain took into account Kaiser’s criterion (eigenvalues larger than one); graphical analysis of the Scree-plot; and the total variance explained. If adequate, to improve interpretation of factors, orthogonal varimax rotations were to be applied. The Kaiser-Meyer-Olkin (KMO) measure and the Bartlett’s test of sphericity were assessed.

Finally, we performed a ROC curve analysis, to assess the best cutoff point for VES-13 total score for discrimination of Frail/Vulnerable elders, assuming the attending physician’s clinical judgment as the gold standard. Best cutoff selection criterion was based on the method of minimization of the distance to the left upper corner of the ROC plot, calculated as √(1-Sn)² + (1-Sp)².

A prospective cohort of 200 consecutively enrolled senior GI cancer patients (≥65 years), would allow an estimation of the prevalence of vulnerability/frailty with a 95 % confidence level margin of error of 0.07. This sample size would also allow an estimation of validity coefficients (correlation coefficients) larger than 0.20, with 95 % confidence level and 90 % power.

Statistical analysis was performed using the Statistical Package for the Social Sciences Version 20.0 for Windows (SPSS®). Whenever statistical hypothesis testing was used, a significance level of α = 5 % was considered.

After translation and cultural adaptation, all questions scored 6 or higher, corresponding to reasonable comprehension during face validation and no changes were deemed necessary.

The pilot study included 20 patients and comprehension difficulties were apparent to the interviewer for male patients answering two questions that included examples of household tasks. These problems were discussed with the original VES-13 authors, and the Portuguese questionnaire was adapted with the inclusion of different domestic tasks examples. A second pilot test was implemented with 22 patients and no difficulties were noted. All questions scored 6 or higher in VAS and test-retest reliability revealed substantial to perfect agreement between test and retest for individual items (test-retest Kappa ranging from 0.612 to 1.000, p < 0.001) and very high correlation between test and retest VES-13 total scale scores, as shown in Table 1 [27].

The VES-13, EQ-5D-5L and medical oncologist’s clinical assessment were applied during 6 months (June to November, 2012). During this period, 296 elderly patients with GI Cancer were admitted to our GI Cancer Clinic and a total of 206 patients were included (Fig. 1 describes the selection process and reasons for exclusion). Demographic and clinical characteristics of the cohort are described in Table 2. The 90 individuals not included in the sample had epidemiological and clinical characteristics similar to those included in the study.

The cohort’s median age was 73 years (29 % ≥80 years) and 65 % of the subjects were male. Colorectal cancer was the most frequent tumor location (63 %) and 3 % of patients had a history of previous neoplasms. The most prevalent co-morbidities were diabetes (n = 62, 30 %) and cardiovascular disorders (n = 36, 17 %). The median CCI was 7 (interquartile range: 6–11). Fifty-eight patients (28 %) were taking five or more daily drugs, and the more frequent therapeutic groups, as defined by the World Health Organization Anatomical Therapeutic Chemical/Defined Daily Dose, were “cardiovascular system” and “alimentary tract and metabolism” drugs.

Compliance with VES-13 and EQ-5D-5L questionnaires was 100 %, except for EQ-5D-5L questions “pain/discomfort”, “anxiety/depression” and VAS scale, which were above 98 %. Summary results for quality of life assessed using EQ-5D-5L are presented in Table 3. A proportion greater than 70 % of patients indicated that they were facing no problems or slight problems in all EQ-5D-5L dimensions; “self-care” presented the highest result with over 80 % of patients experiencing no problems or slight problems. Overall quality of life assessment for the cohort revealed a median EQ-VAS score of 60 percent (interquartile range: 50–75).

VES-13 European Portuguese version of the questionnaire showed high internal consistency, with Cronbach’s alpha if item deleted ranging from 0.826 to 0.880, and a Cronbach’s alpha for the scale score of 0.848, Table 4. When assessing the correlation of VES-13 and EQ-5D-5L dimensions we obtained, as expected, higher correlation scores for “mobility”, “self-care” and “usual activities” (r_s: 0.634, 0.625 and 0.652 respectively). Although not so strong, statistically significant correlations with “pain/discomfort” (r_s: 0.329) and “anxiety/depression” (r_s: 0.178) domains and with VAS scale (r_s: −0,527) were also observed. There were moderate to strong correlations between VES-13 and clinical judgment, ECOG and KPS scales (r_s: −0.499, 0.599, and −0.576, respectively). Table 5 presents the summary statistics for the correlation coefficients between VES-13 and EQ-5D-5L, performance status and clinical impression. In Fig. 2 we can also see and contrast the distribution of VES-13 total score for the fit and vulnerable/fragile elders, as classified by the clinical judgment of the attending physician. In Fig. 3 we present a ROC curve analysis of the VES-13 total score, assuming the clinical judgment of the attending physician as the gold standard; and showing the cutoff value of >3 as the most appropriate for maximizing both sensitivity and specificity. For this cutoff value the sensitivity was 71 % and the specificity was 84 %. The estimate of the area under the ROC curve was C = 0.818 (95 % CI [0.762 – 0.875]).

Exploratory factor analysis, with factor extraction using principal components, was performed for VES-13. The factor solution included a first component with eigenvalue 6.41 and 49.3 % of the variance explained, a second component with eigenvalue 1.08 and 8.3 % of the variance explained, and all other components with eigenvalues lower than 1.00 and smaller percentages of variance explained. Based on the analysis of the scree plot, the eigenvalues of the components and the percentage of the variance explained, the one factor solution was clearly the more appropriate, supporting the hypothesis of unidimensonality of VES-13. Although the strict application of the Kaiser rule would imply the selection of a two factor solution, the fact is that the second component had an eigenvalue marginally above 1.00 and a low percentage of variance explained, thus a one factor solution is clearly a more sensible solution in this case.

Loadings found in the one factor solution and the KMO and Bartlett’s test of sphericity are presented in Tables 6 and 7. It is possible to assess the adequacy of the one factor solution by observing that loadings of most items are above 0.6, with only the first item (Age category) having a loading of 0.375; and taking into consideration the high value of the KMO statistic (KMO = 0.905) and the result of the Bartlett’s test (p < 0.001).