Measuring the impact and distress of health problems from the individual's perspective: development of the Perceived Impact of Problem Profile (PIPP)

In this paper, we report on the development and preliminary validation of an instrument designed to overcome some of the weaknesses of patient-generated measures, while still focusing on the impact of health condition from the individual's perspective. The instrument was designed to be generic rather than disease specific, making it suitable for use across different groups. Unlike existing measures of functioning, the instrument was not designed to assess an individual's ability to perform certain tasks or activities, but rather to better understand the impact and distress caused by the health condition. This was intended to supplement more objective measures (eg. SF-36 [9] and the Health Assessment Questionnaire [10]), providing additional information incorporating contextual factors (environment and personal) identified in the ICF. This information can play an important role in optimising the provision of health care services and setting of priorities. For example, two patients with similar health problems (eg. restricted movement in the knee joint) may experience very different levels of impact and distress. Respondent 1, a professional footballer, may experience highly elevated levels of impact and distress concerning his condition, requiring immediate treatment. In comparison, Respondent 2, a 75 year old with a sedentary life style, may report relatively little impact or distress. This understanding of the meaning of the health condition for the individual can be used in clinical settings to guide the development of treatment programs and to assess outcomes and effectiveness.

Our aim in this study was to develop a relatively short, self-report instrument to assess both the impact and the distress of health problems from the individual's perspective. A set of standardized domains was specified to allow comparison of scores across patient groups and in individuals over multiple time points. The selection of domains for inclusion was guided in part by the ICF, a review of existing measures, and a series of qualitative interviews. Items were selected to assess the impact and distress of the health condition on the Activities and Participation domains of the ICF (Chapters 4 to 9) [1]. An additional component was included to assess the impact on the individual's psychological well-being (including independence and autonomy), an area not yet well addressed in the ICF.

A general test plan was developed representing five separate subscales (Self-care, Mobility, Participation, Relationships, Psychological Well-being) (see Table 1 for items and corresponding ICF code). Prior to the generation of specific items, qualitative interviews were conducted to identify aspects of individuals' lives that they felt were affected by their disability and caused them distress or reduced their perceived quality of life, to ensure that all relevant aspects were identified for inclusion in the scale.

A total item pool of 23 items was generated representing the key domains identified (see Table 1). For each item, respondents were asked to rate on a 6-point scale (a) 'how much impact has your current health problems had on [item of function or activity]'; and (b) 'How much distress has been caused by the impact of your health problem on [same item of function or activity]'. The 6-point scale was anchored on either end by 'no impact' and 'extreme impact' for the Impact scale and by 'no distress' and 'extreme distress' for the Distress scale. For each item, an additional response option was provided for the individual to indicate that the activity was not applicable.

The wording of the items was carefully chosen to ensure that the activities were suitable for both males and females, across different age groups, and for use in different cultural contexts. This study was designed as part of a larger cross-cultural project (RESILIENCE project), with the development of items being conducted in collaboration with researchers from Malaysia and Thailand. The initial version of the PIPP was pilot tested in a series of interviews with individuals with mobility impairment in Australia, Malaysia and Thailand.

Musculoskeletal conditions are the most common cause of physical disability and severe, chronic pain, affecting millions of people globally. These include a wide spectrum of conditions, including those of acute onset (e.g. trauma, injury), and short or long-term disorders (e.g. arthritis, multiple sclerosis). Due to its frequency and chronicity, and the degree of disability that can result, it has a pervasive impact on society. The frequency of musculoskeletal conditions increases with age, and with longer life-spans and hence, ageing populations, there will be an increased number of people with chronic disability disorders with increasing requirements to access healthcare services.

A recommended ICF core set for people with musculoskeletal conditions in acute care included items from the Activities and Participation component such as mobility, self-care, interpersonal interactions, relationships, handling stress and psychological demands [11]. In fact, mobility (walking) was found to have reached a consensus of at least 80% as an important and relevant domain among a range of health conditions, including low back pain, osteoporosis, rheumatoid arthritis, osteoarthritis, chronic ischaemic heart disease and diabetes mellitus [12].

The preliminary validation of the PIPP was conducted using a sample of people with mobility impairment. The impact of mobility impairment on life quality is multi-dimensional, affecting individuals by limitations in activity and restrictions in participation. Subsequent consequences can include reduced employment potential and income, lifestyle limitations, decreased ability to carry out self-care activities, increased dependency on others, and poorer quality of life. Hence, mobility impairment can have a profound impact on the psychosocial status of the individual.

Participants were recruited through co-operation with the Central Highlands General Practice Division. Employing a two staged modified cluster sampling [13, 14], 30 general practitioners in the Central Highlands region were selected. Each practitioner was asked to select seven adult patients from his or her practice who, for any reason, experienced impaired mobility or problems with walking. Rather than diagnostic categories, a liberal criterion was used to identify potential participants, and reasons for mobility impairment included limb amputation (e.g. from accident, cancer or vascular disease), spinal cord injury, stroke, degenerative conditions (e.g. arthritis), and diseases affecting the central nervous system (e.g. multiple sclerosis).

While the aim was to recruit 210 across the 30 practices, initially 178 participants were recruited, who gave verbal consent to the general practitioner or practice nurse and were then contacted by a member of the research team. Twenty-one of these participants declined to proceed for various reasons including personal health (e.g. too ill to partake in the interview), and health of family member (e.g. nursing dying husband), while others were unable to keep appointments despite visits by the research team on more than two occasions. Advertisements were placed in a few local newspapers to increase the number of participants and an additional 12 participants were recruited using this approach. In total, 169 participants were recruited and interviewed in person for this study. Ethics approval for the study was granted by the Human Research Ethics committee of The University of Melbourne.

Additional questions and scales were included in the questionnaire battery to allow an exploration of the validity of the PIPP. These included demographic questions, some details concerning their health condition, and the EQ5D. The EQ5D, developed by the EuroQol Group [15] is a brief standardized, non-disease-specific instrument to measure physical, emotional and social functioning and well-being, and has been used extensively worldwide. It assesses five dimensions – mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Within each domain, there are three possible responses: 'no problems,' 'some problems,' and 'unable to perform.' These five dimensions of health are complemented by a 20-cm vertical visual analogue acale (EQ5D VAS) for the respondent to rate his or her health today from the 'best imaginable health state' set at 100 and 'worst imaginable health state' set at 0. The EQ5D was used with the permission of the EuroQoL Group.

The PIPP Impact and PIPP Distress responses were subjected to Rasch analysis using the RUMM2020 software [16] to assess the psychometric properties of each set of items. For each of the individual subscales, a series of analyses was undertaken to assess overall model fit, threshold ordering, item fit, person fit, and differential item functioning. Rasch calibrated subscale scores were then exported into SPSS Version 12 for further statistical analyses to assess the validity of the subscales. The construct validity of the subscales was assessed in relation to the corresponding component of the EQ5D which was included in the questionnaire booklet. Further details of the Rasch procedures undertaken are provided in the section to follow; a detailed description and illustration of the Rasch analysis procedures is provided in Pallant and Tennant [17].

The RUMM2020 software [16] was used to assess how well the observed data fit the expectations of the Rasch measurement model [18]. The Rasch model provides a template that operationalizes the formal axioms that underpin measurement [19]. Initially the appropriateness of the response scale was assessed by inspection of the threshold values for each item. The term threshold refers to the point between two response categories where either response is equally probable. For a good fitting model, we would expect that, for each of the items, respondents with high levels of the attribute being measured would endorse high scoring responses, while individuals with low levels of the attribute would consistently endorse low scoring responses. This would be indicated by an ordered set of response thresholds for each of the items. One of the most common sources of item misfit concerns respondents' inconsistent use of these response options, resulting in what is referred to as disordered thresholds. This suggests that respondents have difficulty consistently discriminating between response options. Collapsing of categories was used to correct disordered thresholds to improve overall fit to the model.

Three overall fit statistics were considered to assess model fit. Two are item-person interaction statistics transformed to approximate a z-score, representing a standardized normal distribution. If the items and persons fit the model, we would expect to see a mean of approximately zero and a standard deviation of 1. The third fit statistic was an item-trait interaction statistic reported as a chi-square, reflecting the property of invariance across the trait. A significant chi-square indicates that the hierarchical ordering of the items varies across the trait, so compromising the required property of invariance.

Individual person-fit and item-fit statistics were also assessed, both as residuals (a summation of individual person and item deviations) and as a chi-square statistic. Residuals ± 2.5 are considered to indicate adequate fit to the model. The summed chi-square within each group provides the overall chi-square for the item, and the overall chi-square for items is summed to give the item trait-interaction statistic. Bonferroni corrections are applied to adjust the chi-square p value to take account of multiple testing [20]. An estimate of the internal consistency reliability is provided as a Person Separation Index (PSI), where the estimates on the logit scale for each person are used to calculate reliability. PSI values can be interpreted in the same way as Cronbach alpha coefficients, with values above .7 considered adequate [21].

Assessment of differential item functioning (DIF) was used to identify possible item bias [22]. This can occur when different groups within the sample (e.g. males and females), despite equal levels of the underlying characteristic being measured, respond in a different manner to an individual item. Two types of DIF were assessed statistically and graphically using the RUMM2020 software. Uniform DIF exists when one group shows a consistent systematic difference in its responses to an item, across the whole range of the attribute being measured. When there is non-uniformity in the group differences (e.g. it varies across levels of the attribute), then this is referred to as non-uniform DIF. Analysis of variance was conducted for each item, comparing scores across each level of the 'person factor' and across different levels of trait (referred to as class intervals). Uniform DIF is indicated by a significant main effect for the person factor (gender), while the presence of non-uniform DIF is indicated by a significant interaction effect (person factor × class interval).

Prior to assessing the model fit of the subscales, the threshold ordering of the items was inspected. All items showed some degree of disordering, except item 9 (sit or stand). To overcome this problem, all items were recoded from the original 6-point scale (scored: 012345) by collapsing scores to form a simpler 3-point response scale (011112). This resulted in ordered thresholds for all items except item 14 (authority), which showed only minor disordering.

Consistent with the procedures adopted in the previous section, the items from each of the individual subscales were subjected to Rasch analysis. Disordered thresholds were corrected by recoding all items as 011112.

To further assess the characteristics of the PIPP subscales, the person estimates generated in RUMM2020 for each of the subscales were exported to an SPSS data file. These were then subjected to further statistical analyses to assess the relationship among the PIPP subscales, between the impact and distress subscales, and between the subscales and the various components of the EQ5D, which were included in the questionnaire booklet. The descriptive statistics for the Impact and Distress subscales are shown in Table 4. Non-parametric correlation coefficients (Spearman rho) were used given the skewed distribution of scores on a number of the subscales.

Table 5 shows the Spearman correlation coefficients (rho) among the PIPP Impact subscales. The strongest correlation was between the impact on Mobility and Psychological Well-being (rho = .61) with the lowest occurring between Self-care and Participation (rho = .206). The correlations among the distress subscales are shown in Table 6. Overall the correlations among the distress subscales were higher than that observed for the Impact items. The highest inter-correlation was between Mobility and Psychological Well-being (rho = .78), with the lowest between Self-care and Relationship (rho = .365).

Correlation coefficients were also calculated between the corresponding PIPP Impact and PIPP Distress subscales to explore the relationship between respondents' perceptions of the impact of their disability, and the level of distress that this causes (see Table 7). All correlations were above .753 (Mobility), with the highest being recorded for the Relationship subscales (rho = .928).

The validity of the PIPP impact and distress subscales was assessed by investigating the relationship with appropriate corresponding EQ5D items which were included in the questionnaire booklet administered to participants.