Detection of response shift in health-related quality of life studies: a systematic review

Health-related quality of life (HRQOL) is the affectation of a patient’s physical and mental health, and their correlates, including health risk and conditions, independence level, social support, and socioeconomic status over time [1, 2].

This concept is established as a facet of an individual’s state of life while measuring its well-being as a medical patient. HRQOL is analyzed as a functional health state and identifies effective strategies to improve patient´s conditions as a result of medical interventions [3, 4].

After a medical treatment, patients may perceive and demonstrate different conditions over time, from the procedure’s initial stages up to many years after treatment has finished. The changes in the measurement of individual’s perception or internal standard is known as Response Shift (RS) [5, 6]. From a clinical perspective, RS is generated as the change in the meaning of a subject self-administrated assessment [7] as a valid and sensitive mechanism to evaluate the change in different moments in time [8].

Spranger and Schwartz [9] theoretical model explains how RS may affect HRQOL as a result of health state changes. As a baseline model, it presents five components: (1) a catalysts, corresponding to an individual health states or its changes as a result or not of a treatment; (2) antecedents, referring to individual’s characteristics influencing catalysts or appraisals mechanisms; (3) mechanism, explained by behavioral, cognitive, or affective processes accommodating changes in catalysts; (4) response shift, representing changes in the meaning of an individual self-evaluation of QOL resulting from changes in internal standards, values, or conceptualization; and (5) perceived QOL. Rapkin and Schwartz [10] propose that QOL appraisal processes must consider how individuals perceive their health status and respond questionnaires about their QOL. The model follows these processes: (1) induction of a frame of reference; (2) sampling based on the frame of reference; (3) judge against standards of comparison; and (4) combine algorithms to formulate a response. The proposal allows dynamic feedback to explain how QOL scores can be stabilized accounting inter-individual and temporary differences despite changes in health status [10].

An individual’s self-assessment may demonstrate changes in three contexts: in the internal standards of the measurement scale (recalibration) indicating that the patient has a new scale for measuring its own state of HRQOL; in the scale of values (reprioritization) representing a change in the priority of elements that influence the context of life; and in the definition of the objective construct (reconceptualization) when a patient raises a redefinition of its own concept [9].

The change processes of a patient must be appropriately measured when the effects of a disease or medical treatments is evaluated [11, 12]. The interpretation of HRQOL data represents a challenge because patients self-report their health conditions at a specific time, which can also be influenced by psychological phenomena [13]. This suggests that HRQOL measurements must consider that the individual reports on its status, at least in two or more moments to detect significant changes over time.

Two main approaches are proposed for the detection of change: methods based on specific study designs and secondary data analysis that includes statistical methods developed to test hypotheses that do not require specific designs [14].

The most commonly used methodology is the retrospective Then Test design [15] that allows evaluating the change of patient´s internal standard by comparing the scores with two other moments: pre-test and post-test [5, 7]. However, this method is sensitive to bias and difficult to be used in longitudinal secondary data analysis [15].

Structural Equation Models (SEM) of Schmidt [16] and Oort [17], Item Response Theory (IRT) of Anota et al. [18], and Guilleaux et al. [19], Relative Importance Method of Lix et al. [20], Latent Trajectory Modelling of Ahmed et al. [21], and Classification and Regression Trees of Li & Schwartz [22] are among the most widely used and proven methods for RS detection in both primary and secondary data.

This bibliographic review has permitted to identify the methods traditionally applied in specific clinical studies related to HRQOL or to the analysis of previously elaborated databases, mainly by medical, academic or research institutions. Although other recent publications have carried out exhaustive systematic reviews to address this issue [15, 23], it is necessary to continue exploring if other alternative statistical methods are used as emerging mechanisms in the RS detection. Consequently, the purpose of this investigation is to systematically identify, analyze, and synthesize the existing evidence of new statistical methods used for RS detection in HRQOL studies.

The systematic review is a structured methodology allowing the identification and integration of different specific studies based on inclusion and exclusion criteria and facilitates the eligibility of relevant or interesting publications [24]. Some critical and determining elements to capture the largest number of eligible publications are the number and orientation of search repositories, and the established inclusion and exclusion criteria.

The PRISMA methodology [25] was applied to develop this systematic review. Through pre-specified eligibility criteria, it allows reducing bias in the identification, selection, synthesis, and summary of results from previously published studies [26]. The precision and reliability of this tool provides important benefits in health-related research [27].

The acceptance of this methodology has been used in recent studies on RS applied to different clinical areas: oncology [23, 28], orthopedic rehabilitation [29], pre-injury [30], as well as patient-reported outcomes (PRO) [31].

After the systematic review, the study examined the advances on HRQOL research through different methods for the detection of RS. It describes how the detection of this phenomenon is being evaluated in recent years, which methods are used the most, and the most identified type of RS. For physicians and researchers, systematic reviews are useful sources of evidence and scientific advances [32], they provide elements so clinical policy-makers can evaluate risks, benefits and effects on health care, as well as new research initiatives [26, 27].

The majority of HRQOL studies and RS detection methods were published during 2016 and 2017. One third of the articles included in the review were studying cancer disease [33‐41]. Then Test was the most used method mainly for primary data measuring recalibration [42‐48]. Several studies remarked that Then Test may also identify other RS types: reprioritization y reconceptualization [33, 49, 50]. Similar results were presented in Sajobi et al. [15] study corresponding to a systematic review about RS detecting methods were 54.5% of the articles used Then Test for this purpose. As same, in Ilie et al. [23] a 60.0% of the cancer studies remarked using this methodology, but authors indicate that interpretation must be cautious as it is bias prone.

Then Test has the advantage for easy handling and analysis, but its disadvantage relies in random errors and/or confound and difficult for interpretation. Therefore, this appraisal suggests the utilization of individualized methods with strong statistical rigor [51].

Oort’s SEM [17] differs from Then Test, since it is used for studies based on primary and secondary data. This multivariate method evidences the capability for detecting changes in all type of RS: recalibration, reprioritization, and reconceptualization. Several studies reiterate Oort’s SEM as an effective method to detect changes in patients, despite the different type of RS or data used [52‐54]. This detection capacity might be the reason for a larger interest for researchers of using this method.

Schmidt’s model was used in two studies detecting only recalibration. In comparative studies with Oort’s method [55, 56], only this type of RS is identified, but conclusions indicated that both approaches use different parameters to identify recalibration: Schmidt defines recalibration as the change in factor variances or factor loadings over time, while Oort by the change in intercepts.

Other methods obtained more records for RS detection: Mixed-Effects and Multiple Linear Regression [57‐60], Classification and Regression Trees [22, 61], Item Response Theory [18, 62] and Relative Importance Methods [63‐65].

Despite a major frequency of methods to detect recalibration, it calls the attention that all three studies using Relative Importance Method detected reprioritization [63‐65]. Mixed-Effect Regression method [57, 58], Classification and Regression Trees [22, 61] and Latent Trajectory Analysis [66] neither provided information on reconceptualization.

During this study we observed that the sample size is related to the type of method used for RS detection. Then Test is more flexible and functional in small samples or in specific studies, while methods based on stronger statistic techniques require larger samples. According to Schwartz et al. [67], for using advanced multivariate methods capable to detect RS, data analysis must include sufficient individuals participating in the study, as well as considering certain number of parameters such a clear the model, loading factor, relationship between the items, data distribution, and processes for estimating these parameters.

The need of exploring new multivariate methodologies for analyzing this phenomenon brings an alternative proposal for studying HRQOL and RS detection through three-way data: dual STATIS Method [68, 69]. The proposal presented by Vicente Galindo [70] centers in a procedure integrating the dual STATIS methods and the comparison of Krzanowski [71] subspaces, and examinates factorial structures from multiple data sets focused to identify the existence of change.

The systematic reviews evaluated in this study [15, 23, 28‐31] confirm the continuous use of the same methods for detecting RS and its different types, their advantages, and limitations. The previous proposal is an example of existing opportunities to continue examining other strong statistic methodologies allowing to deepen this line of investigation and reducing bias and analysis ambiguities.

The systematic review has demonstrated to become an adequate and convenient methodology to identify and synthetize advances of specific topics. Results have evidenced a generalized interest for studying HRQOL and RS for different diagnostic groups. RS detection continue attracting researcher’s attention that have consolidated a set of methods for its analysis: Then Test, Oort’s SEM, Multiple Linear and Mixed-Effects Regressions, Item Response Theory and Latent Trajectory Analysis. Oort’s SEM becomes the most versatile method in its capability for detecting RS in all different types.

Results demonstrated that not all the methods achieve RS detection in similar proportions, mostly capable to identify recalibration, in some cases reprioritization, and in few studies reconceptualization. At the same time, previous systematic reviews and the result from this updating research conclude that same methods have been used during the last years and there is no evidence found of alternative statistical techniques proposed for detecting RS. Perhaps, our study states a need for exploring other methods with similar detection capacity, with robust and highly precise methodology either graphical oriented or with simpler methodologies.

Previous systematic reviews and the need to continue investigating about this phenomenon have motivated the update of this search within the last decade (2010–2020). Although these reviews have used a larger number of search terms including different RS types as well as data measurement characteristics, this systematic review has intentionally generalized the terms using RS and HRQOL to focus the results on those studies centered on this topic.

Since RS can be evaluated from different perspectives, and other disciplines such as statistics, psychology, humanities, and other related social sciences can provide significant contributions, it is recommended to continue studying the RS phenomenon, its detection, and types from a multidisciplinary approach.