Comparison of patient-reported outcomes measurement information system and legacy instruments in multiple domains among older veterans with chronic back pain

This was a cross-sectional pilot study in which we recruited Veterans ≥60 years old with nonmalignant, noninfectious chronic (> 3 months duration) back pain with or without associated leg pain, who were referred and scheduled for an epidural steroid injection (ESI) from the Dallas VAMC Physical Medicine & Rehabilitation (PM&R) Spine Clinic (Table 1). Patients scheduled to receive the steroid injection were screened prior to the date of their procedure and if they met the inclusion criteria listed above, were introduced by the PM&R team and approached by the research staff to obtain informed consent and complete the surveys on the day of the ESI. Exclusion criteria were receiving an ESI within 3 months prior to this study, inability to speak English, severe cognitive impairment (failing a Mini-Cog test with total score < 3) [17] and back pain due to infectious or malignant etiology. This study was approved by the Dallas VA Institutional Review Board.

Participants completed the battery of tests (PROMIS CAT item banks and corresponding validated legacy instruments) (Table 2) prior to the ESI. Eligible participants were directed by research staff to an exam room with a dedicated desktop computer that was used to complete both sets of surveys. Research staff also documented field notes as participants completed the instruments (regarding usability or obstacles to completion). PROMIS instruments were administered as CATs using the PROMIS Assessment Center (www.​assessmentcenter​.​net), and all of the legacy instruments were also administered on the computer for ease of use and ability to automatically measure time to completion, with exception of the graphics from Brief Pain Inventory (BPI), and the Numerical Rating Scale for Pain Intensity (NRS-PI). If the participant was not able to complete all sections of the PROMIS and legacy instruments, within each domain, prior to being called for their ESI procedure, these instruments were considered incomplete. We did not analyze responses from participants who had missing data for either PROMIS or legacy instruments. The presentation order of PROMIS and legacy instruments was randomized.

Based on previous research, domains from PROMIS instruments that represent the impact of back pain in older adults were selected to be administered as CAT in this study [10, 18]. Most of the CAT-administered instruments ranged from four to 12 items. These PROMIS (and corresponding legacy) instruments are listed in Table 2 in the following domains: pain intensity, pain interference, pain behavior, functional status, depression, anxiety, fatigue and sleep. With exception of physical function that does not include a time frame and the social health banks that reference “lately”, all PROMIS instruments reference the ‘last 7 days’. All PROMIS instruments, except for pain behavior, use a rating scale with five response options that reflect intensity or frequency. Pain behavior uses six response options with the lowest response “had no pain”. All PROMIS instruments are designed to produce a score where higher values indicate a greater presence of the construct being measured. PROMIS instrument results are reported as T-scores, with a mean of 50 and a standard deviation of 10 normed on the 2000 U.S. census general population [19]. For example, a patient with a score of 72 on the PROMIS Anxiety instrument indicates the patient is reporting levels of anxiety that are more than two standard deviations above the general population.

The legacy instruments, by domain, that were used in this study are described below. Pain intensity (over the last 7 days) was assessed using the NRS-PI measure. NRS-PI is a unidimensional 11 point scale measure of pain intensity in adults where the end points are the extremes of no pain (0) and worst pain (10) [20]. Pain interference was assessed with two legacy measures, the Short Form Health Survey (SF-36, Bodily Pain) and BPI. The SF-36 is a 36-item patient-reported survey to measure health status, and consists of eight scaled scores, which are the weighted sums of the questions in each section. A lower score indicates more disability [21]. The BPI is a measure of pain related functional impairment, and measures pain interference with seven daily activities, including general activity, walking, work, mood, enjoyment of life, relations with others, and sleep. It is scored as the mean of the seven interference items, that can be used if > 50% of the total items have been completed [22]. Pain behavior was assessed with two legacy measures, the Pain Catastrophizing Scale (PCS) and Fear Avoidance Belief Questionnaire (FABQ). The PCS is a self-reported measure, where patients answer questions about how they feel and what they think about when they are in pain. The measure consists of 13 items scored from 0 to 4, resulting in a total possible score of 52. The higher the score, the more catastrophizing thoughts are present [23]. The FABQ measures patients’ fear of pain and consequent avoidance of physical activity because of their fear, and it consists of 16 items in which a patient rates their agreement with each statement on a 7-point Likert scale (0 = completely disagree, 6 = completely agree) with a maximum score of 96. The higher the score, the more strongly fear avoidance beliefs are held [24]. Functional status was assessed using the Roland-Morris Disability Questionnaire (RMDQ). The RMDQ is a measure of disability where patients are asked to read a list of 24 sentences and to place a tick against appropriate questions based on how they feel each sentence describes them today. The RMDQ is scored by adding up the number of items the patient has ticked, with a maximum score of 24, where higher scores reflect greater levels of disability [25]. Depression and anxiety symptom severity was assessed using the ‘Patient Health Questionnaire-4’ (PHQ-4) for depression and anxiety respectively. The PHQ-4 is a 4-item inventory (two questions on anxiety and two questions on depression) rated on a 4-point Likert-type scale ranging from ‘not at all’ with a score of zero to ‘nearly every day’ with a score of three. The total score is the sum of the four items with higher scores indicating more severe anxiety and depression symptoms [26]. Fatigue was assessed using the Functional Assessment of Chronic Illness Therapy (FACIT) Fatigue Subscale which is a short 13-item, measure of an individual’s level of fatigue during their usual daily activities over the past week. Subject responses are on a four-point Likert scale (0 = not at all and 4 = very much). Final scores are the sum of responses to the items and range from 0 to 52, where higher scores represent less fatigue [27]. Sleep was assessed with the Medical Outcomes Study (MOS) Sleep Scale which is a 12 item self-report sleep measure that contains 7 subscales and 2 overall index scores (a 6-item and a 9-item index) to assess important dimensions of sleep including initiation, maintenance, respiratory problems, quantity, perceived adequacy and somnolence. Higher scores indicate more of the concept being measured [28]. Social support was assessed with the MOS Social Support Survey (MOS SSS) which is a 19-item self-administered survey that covers four subscales (emotional/informational support, tangible support, positive social interaction, and affection), and an overall support index score is calculated and transformed to a 0–100 scale. A higher score for an individual scale or for the overall support index indicates more support is available [29]. It should be noted that unlike measures of social support (MOS SSS) that generally seek information about an individual's perception of the availability of support, the PROMIS Social Isolation instrument assesses perceptions of being avoided by or disconnected from others.

In addition to the instruments outlined above, the following data were obtained from the electronic medical record: age; sex; race (White, Black, other, declined); ethnicity (Hispanic vs non-Hispanic); pain duration; body mass index (BMI) [30]; current (at the time of recruitment) mental health conditions (depression, anxiety and/or PTSD) that are actively being treated (by medications or therapy), medications prescribed at the time of recruitment (antidepressants, anxiolytics and/or analgesics including acetaminophen, NSAIDs and opioids); the Charlson comorbidity index [31]; percent service connection due to a musculoskeletal condition (this is a calculated disability rating given to a Veteran based on the severity of their service-connected conditions, and which determines their disability compensation and eligibility for VA benefits); and history of prior epidural steroid injection at least 3 months prior to enrollment.

Descriptive statistics included means and standard deviations (SD) or median and interquartile range [IQR] for continuous variables depending on normality of the data, and n (percentages) for categorical variables [32]. Assumptions for parametric tests were not met; therefore, non-parametric statistical analyses used Spearman’s Rank-Order Correlations (rho) to measure the strength of association between ranked scores for PROMIS and legacy instruments and also provided an estimate of convergent and discriminant validity. Convergent validity refers to the degree to which two measures of constructs that theoretically should be related, are in fact related; it is estimated using correlation coefficients (rho) [33]. Discriminant validity refers to the degree to which two measures that should not theoretically be related indeed are not highly correlated [34]. The strength of correlation was defined as follows: weak (rho = 0.20–0.39); moderate (rho = 0.40–0.59); strong (rho = 0.60–0.79); very strong (rho = 0.80–1.0) [35]. We considered an additional category for very weak (rho = 0.00–0.19). When interpreting the strength of the relationship, we evaluate the absolute value of rho. Using these ranges for strength of association, we expect that the convergent validity would be strong or very strong; whereas the discriminant validity would be very weak, weak, or moderate. Discriminant validity should be less than the convergent validity.

Mann-Whitney U tests compared the administration length (time to completion) between PROMIS and legacy instruments, by domain. Only completed instruments or domains were included in the analyses. P < .05 was considered to indicate statistical significance. All analyses were conducted using Stata 14 [36].

While the number of participants recruited for this study was 71, the number of completed instruments (n) varied by instrument and are listed in Table 2. For PROMIS instruments, the mean number of administered items per domain ranged from four to six except for pain intensity where three items were administered in all cases. Three domains assessed pain: intensity, interference, and behavior. Pain behavior, as assessed using PROMIS, had a mean (SD) of 60.82 (4.22). This domain was correlated with two legacy instruments: PCS and FABQ with medians [IQR] of 1.58 [0.85–2.62] and 70.5 [57–80], respectively. Functional status was measured using PROMIS Physical Function with median [IQR] of 31.9 [27.2–36.5] and its corresponding legacy measure was the RMDQ with a median [IQR] score of 18 [13–21]. The baseline values of the remaining domains (depression, anxiety, fatigue, sleep disturbance and social) for PROMIS and their corresponding legacy instruments are listed in Table 2.

Table 3 provides the Spearman’s rank-order correlations between all PROMIS and legacy instrument scores. The validity diagonal (as highlighted in Table 3) contains the highest correlation coefficients across the row and column for a particular measured domain (PROMIS instruments across rows and legacy instruments in columns) with exception of PROMIS Pain Intensity, PROMIS Pain Behavior, PROMIS Physical Function, PROMIS Depression and PROMIS Social Isolation where the highest correlation was with a legacy measure different than their corresponding domain instrument. We found moderate convergent validity (rho = 0.51–0.59) in the domains for PROMIS Pain Intensity, PROMIS Pain Interference, and PROMIS Pain Behavior (PROMIS Pain Behavior in regards to PCS and not FABQ); strong convergent validity (rho = 0.61–0.76) for domains of PROMIS Physical Function and PROMIS Depression; very strong convergent validity for domains of PROMIS Anxiety, PROMIS Fatigue and PROMIS Sleep Disturbance (rho = 0.80–0.85). There was weak convergent validity (rho = 0.31–0.34) for the PROMIS Social Isolation and for the PROMIS Pain Behavior domain with respect to FABQ.

Table 3 provides the discriminant validity which can be found in the off-diagonal cells. Using the categories for strength of association, we find that 63 of 88 discriminant validities in the off-diagonal range from very weak to moderate correlations. The exceptions that stand out are in the domains for Pain Intensity with SF-36: BP (rho = − 0.77), Pain Behavior with BPI, PHQ-4, and FACIT (rho = 0.63–0.66), Physical Function with FABQ (rho = − 0.66), Depression with BPI, PCS, RMDQ, and PHQ-4 A (rho = 0.60–0.79), Anxiety with BPI, PCS PHQ4-D, FACIT, and MOS Sleep (rho = 0.62–0.74), Fatigue with BPI, PCS, PHQ4-D, PHQ-4 A and MOS Sleep (rho = 0.62–0.71), Sleep Disturbance with BPI and FACIT (rho = 0.65–0.71), and Social Isolation with BPI, PCS, PHQ4-D and PHQ-4 A (rho = 0.60–0.72).

If we define positive evidence for convergent/discriminant validity as the definition that convergent validity has a higher correlation compared to all discriminant validities for that measure, four (44.4%) PROMIS instruments (Pain Interference, Anxiety, Fatigue and Sleep Disturbance) meet this criteria and three (33.3%) others (Pain Intensity, Physical Function, and Depression) miss this criteria by one comparison. Evaluating the Legacy instruments, seven (63.6%) instruments (NRS-PI, RMDQ, PHQ4-D, PHQ-4 A, FACIT, MOS Sleep and MOS SSS) meet this criteria and one (9.1%) additional instrument (SF-36) missed this criteria by one comparison.

The median time to complete each CAT-administered PROMIS instrument ranged from 23 s to 58 s. The time to complete individual legacy instruments ranged from 13 s to 6 min and 7 s. Administration duration for the NRS-PI was unavailable since it was completed using a paper and pen modality and time to complete was not measured. The median total time participants needed to complete all the PROMIS instruments was shorter than that needed to complete legacy items across most domains, with total median time of 8 min 50 s vs 29 min 14 s respectively, p < 0.001 (Table 4). The median time to complete PROMIS Depression and Anxiety screen was longer than legacy Patient Health Questionnaire (PHQ)-4 Depression and Anxiety (35 vs 13 s; p < 0.001 for depression and 39 vs 15 s; p < 0.001 for anxiety, respectively).

The research staff documented in field notes that PROMIS instruments were easier for participants to complete due to consistent formatting and wording of the questions and the uniformity of the Likert-type scale responses available.