Article Text

Download PDFPDF

Measuring disability experienced by adults living with HIV: assessing construct validity of the HIV Disability Questionnaire using confirmatory factor analysis
  1. Kelly K O'Brien1,2,3,
  2. Patricia Solomon2,
  3. Ahmed M Bayoumi3,4,5
  1. 1Department of Physical Therapy, University of Toronto, Toronto, Ontario, Canada
  2. 2School of Rehabilitation Science, McMaster University, Hamilton, Ontario, Canada
  3. 3Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
  4. 4Centre for Research on Inner City Health, Keenan Research Centre of the Li Ka Shing Knowledge Institute, St Michael's Hospital, Toronto, Ontario, Canada
  5. 5Department of Medicine, University of Toronto, Toronto, Ontario, Canada
  1. Correspondence to Dr Kelly K O'Brien; kelly.obrien{at}utoronto.ca

Abstract

Objectives To assess the construct validity of the HIV Disability Questionnaire (HDQ), a self-administered questionnaire that describes the presence, severity and episodic nature of disability experienced by people living with HIV.

Design We conducted a confirmatory factor analysis. We hypothesised that domains in the HDQ characterised six dimensions of disability, each represented by HDQ items: physical symptoms and impairments (20 items); cognitive symptoms and impairments (3 items); mental and emotional health symptoms and impairments (11 items); uncertainty (14 items); difficulties with day-to-day activities (9 items) and challenges to social inclusion (12 items). We developed a measurement model to test these hypotheses. We used maximum likelihood methods of estimation to determine model fit. We considered a threshold for the Root Mean Square Error of Approximation (RMSEA) of <0.05 as an indication of overall goodness of model fit. We considered variables with factor loadings of >0.30 as representing a given domain of disability.

Setting We recruited adults with HIV from hospital clinics, AIDS service organisations and a specialty hospital in Ontario.

Participants Of the 361 adults with HIV who completed the HDQ, 80% were men, 36% were 50 or older and 77% reported living with at least two concurrent health conditions in addition to HIV.

Outcome Measures We administered the HDQ followed by a demographic questionnaire.

Results The model achieved good overall fit as indicated by a RMSEA of 0.030 (90% CI 0.028 to 0.033). All HDQ items represented our hypothesised dimensions of disability (factor loadings >0.30). Factor loadings ranged from 0.34 to 0.90. Domains of disability correlated with each other ranging from r=0.47 (between difficulties with day-to-day activities and uncertainty) to r=0.88 (between mental-emotional health challenges and challenges to social inclusion).

Conclusions The six domain structure of the HDQ demonstrated construct validity when administered to adults living with HIV.

  • REHABILITATION MEDICINE
  • Disability

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • We assessed the construct validity of the HIV Disability Questionnaire (HDQ), a 69 item self-reported questionnaire developed to assess the presence, severity and episodic nature of disability experienced by adults living with HIV.

  • We conducted a confirmatory factor analysis hypothesising that domains in the HDQ represented six dimensions of disability, each represented by HDQ items: physical symptoms and impairments (20 items); cognitive symptoms and impairments (3 items); mental and emotional health symptoms and impairments (11 items); uncertainty (14 items); difficulties with day-to-day activities (9 items) and challenges to social inclusion (12 items).

  • The six domain structure of the HDQ demonstrated construct validity when administered to adults living with HIV. The hypothesised model achieved good overall fit as indicated by a RMSEA of 0.030. All HDQ items represented our hypothesised dimensions of disability (factor loadings >0.30).

  • The HDQ is the first known HIV-specific instrument of disability developed from the perspective of adults living with HIV.

  • The HDQ can be used to describe the multidimensional nature of disability experienced by adults living with HIV and lay the foundation for more widespread measurement of disability in HIV clinical practice and research.

Introduction

As people living with HIV infection are living longer and aging, they are experiencing the complex health-related challenges of the infection, associated comorbidities and potential adverse effects of treatment.1–3 Health-related challenges, collectively termed disability, can include symptoms and impairments (eg, fatigue, neurocognitive impairments, weakness, pain), difficulties with day-to-day activities (eg, household chores), challenges to social inclusion (eg, ability to work) and uncertainty or worrying about the future.4–6 Disability may be experienced as episodic in nature, whereby health challenges fluctuate on a daily basis and over the entire course living with HIV.7 Given current treatments for HIV and the long-term survival for successfully treated individuals, measuring disability is critical for determining the impact of the disease, its comorbidities and interventions.

We developed a new self-administered instrument, called the HIV Disability Questionnaire (HDQ) using the Episodic Disability Framework, a conceptual framework that describes disability experienced by adults living with HIV. The Episodic Disability Framework consists of three main components: (1) dimensions of disability that may fluctuate on a daily basis and over the entire course of living with HIV (symptoms and impairments, uncertainty, difficulties with day-to-day activities, and challenges to social inclusion), (2) contextual factors (social support, stigma, living strategies and personal attributes) that may exacerbate or alleviate the dimensions of disability and (3) triggers, defined as life events that mark momentous or major episodes of disability.7 ,8 The purpose of the HDQ is to describe the presence, severity and episodic nature of disability experienced by adults living with HIV.9 Items in the HDQ were derived from the Episodic Disability Framework.7 The HDQ consists of six domains of disability: physical symptoms and impairments; cognitive symptoms and impairments; mental and emotional health symptoms and impairments; uncertainty; difficulties with day-to-day activities and challenges to social inclusion.10 The HDQ possesses sensibility (face and content validity and ease of use).11 Our aim was to assess the construct validity of the HDQ among adults living with HIV.

Methods

We conducted a cross-sectional study to assess the construct validity of the HDQ. We recruited adults (18 years and older) living with HIV by posting recruitment posters and brochures in hospital clinics, AIDS service organisations and a specialty hospital in southern Ontario. Healthcare providers, who were aware of the study also provided eligible individuals with information about the study and invited them to volunteer to participate. For those who agreed to participate, we administered the HDQ, followed by a demographic questionnaire. All aspects of this project were conducted in collaboration with a Community Advisory Committee comprised of four members including adults living with HIV, representatives from AIDS Service Organisations and a representative from the Ontario Ministry of Health and Long-Term Care. This research was approved by Research Ethics Boards at McMaster University, Hamilton, Ontario and St. Michael's Hospital, Toronto, Ontario, Canada.

HIV Disability Questionnaire

The HDQ consists of 69 items that describe the presence, severity and episodic nature of disability experienced by adults living with HIV. Each item consists of a statement about a health-related challenge (eg, ‘I have trouble remembering things like appointments and when to take my medications’) and has both a seven-point ordinal response scale asking the respondent to rate the challenge on the day of administration (from 0 –‘None at all’ to 6 –‘Extreme trouble’) and a nominal response scale asking whether the challenge fluctuated (or changed) over the past week (‘Yes’ or ‘No’).

All data were entered into a database and 20% of cases were independently checked for accuracy. We removed any cases with >10% of item responses missing. We calculated disability presence, severity and episodic scores for each domain and for the total HDQ scale. Disability presence scores were calculated by summing the number of challenges (≥1 response option on the seven-point scale) for a total disability presence score (ranging from 0–69). Disability severity scores were calculated by summing individual item scores (ranging from 0 to 6) for each domain. Disability episodic scores were calculated by summing the number of challenges identified as episodic (‘Yes’ responses; ranging from 0–69). All presence, severity and episodic scores were linearly transformed to a score ranging from 0 to 100. Higher scores on each scale indicated a greater degree or episodic nature of disability.

Analysis

We conducted a confirmatory factor analysis to assess the construct validity of the domains of disability in the HDQ. We hypothesised that domains in the HDQ represented six dimensions of disability, each represented by the following HDQ items: physical symptoms and impairments (represented by 20 items); cognitive symptoms and impairments (3 items); mental and emotional health symptoms and impairments (11 items); uncertainty (14 items); difficulties with day-to-day activities (9 items) and challenges to social inclusion (12 items; figure 1).

Figure 1

A priori measurement model for confirmatory factor analysis of the HIV Disability Questionnaire.

We assessed convergent construct validity by determining the extent to which disability severity scores in the HDQ represented a hypothesised domain of disability with factor loadings >0.30. We assessed divergent construct validity by determining the extent to which domains of disability were distinct constructs that together comprised the larger construct of disability. We considered correlations between latent variables <0.80 as signifying distinct dimensions of disability.12

We used the maximum likelihood methods of estimation, which is preferred for non-normally distributed data.13 Prior to our analysis, we conducted mean imputation for missing data in order to maximise the sample size for analysis.12 We estimated and reported standardised parameter estimates (or factor loadings) for each item. We defined factor loadings >0.30 as indicating a relationship between an HDQ item and a given domain; these items were considered as ‘loading’ on that domain.

We used a combination of approaches to evaluate the overall goodness of fit of the confirmatory factor analysis solution.12 If the solution demonstrated adequate goodness of fit we considered this as constituting evidence in favour of construct validity of the domains of the HDQ. We considered a Root Mean Square Error of Approximation (RMSEA) <0.05, Comparative Fit Index (CFI) >0.95 and Tucker Lewis Index (TLI) >0.95 to indicate good model fit.14 ,15 The RMSEA is a population-based index that assesses the extent to which a model fits reasonably well in the population by evaluating the discrepancy between the hypothesised model, with optimally chosen parameter estimates, and the population covariance matrix. The RMSEA ranges from 0 to 1, with smaller values indicating better model fit.14 ,15 The CFI and TLI assess model fit by examining the discrepancy between the data and the hypothesised model while adjusting for sample size. CFI and TLI values range from 0 to 1 with higher values indicating better fit.14 ,15 We considered the RMSEA as the primary statistic for overall goodness of model fit because it is less sensitive to sample size and is recommended for confirmatory factor analysis.16 We reported the Chi Square statistic (χ2) but did not consider it a determinant of model fit given its sensitivity to large sample sizes, which can overestimate lack of model fit.12

We estimated our sample size using the rule that a minimum of five participants per item are required for factor analysis.17 With 69 disability severity items in the HDQ, we required at least 345 participants. We used SPSS (V.19.0) for computation of HDQ scores and Mplus (V.7.0) for the confirmatory factor analysis.18 ,19

Results

Three-hundred and sixty-one participants were recruited from AIDS service organisations in Toronto (51%), word of mouth (28%), AIDS service organisations in Hamilton, Niagara and Durham regions (7%), a specialty hospital in Toronto (3%) and hospital clinics in Toronto and other areas of southern, Ontario (3%). The majority of participants were men (80%), the median age was 46 years, most were taking antiretroviral medications (83%), and many (77%) were living with two or more health conditions in addition to HIV (table 1). The most common self-reported concurrent health conditions included mental health challenges such as anxiety and depression (40%), arthritis (27%), osteoporosis or osteopenia (16%) and cardiovascular disease (15%).

Table 1

Characteristics of participants (n=361)

HIV Disability Questionnaire

Participants took a median of 14 min to complete the HDQ (1st–3rd quartile: 10–20 min). Almost all HDQ items (n=66) demonstrated a floor effect with >15% of responses rated ‘0’ indicating no disability. Floor effects were most common in items that referred to symptoms and impairments or difficulties with day-to-day activities. Ten HDQ items demonstrated a ceiling effect with >15% of responses rated ‘6’ indicating the highest severity of disability. Ceiling effects were most common in items pertaining to uncertainty or worrying about the future. Missing responses were <5% across all HDQ disability severity and episodic items.

Median HDQ presence scores (ranging from 0 to 100) ranged from 44 in the difficulties with day-to-day activities domain (1st–3rd quartile: 11–78) to 100 in the cognitive health challenges domain (1st–3rd quartile: 67–100). Median HDQ severity scores were highest in the uncertainty domain (45; 1st–3rd quartile: 27–67) followed by challenges to social inclusion (33; 18–51), cognitive (28; 11–50), mental-emotional (27; 14–53) and physical symptoms and impairments (25; 14–38). Median HDQ episodic scores (number of challenges that fluctuated within the week) were greatest in the physical (35; 8–54) and cognitive symptoms and impairments domains (33; 0–67) followed by the mental-emotional symptoms and impairments domain (18; 0–64), difficulties with day-to-day activities (11; 0–33), uncertainty (7; 0–50) and challenges to social inclusion (0; 0–33). The most common health challenges that were reported as episodic spanned the physical and mental-emotional symptoms and impairments domains and included fatigue (50%), feeling sad, down or depressed (48%), aches and pains (46%), headaches (42%), feeling anxious (41%), having trouble sleeping (40%) and feeling weakness in muscles (40%). Eighty-two per cent of participants completed the HDQ on what they considered a ‘good day’ living with HIV.

Confirmatory factor analysis results

Results of the confirmatory factor analysis are presented in table 2. Correlation matrices are available on request. The RMSEA was 0.030 (90% CI 0.028 to 0.033), satisfying our primary goodness of fit criterion. The χ2 value was 3020.981 (p<0.001). CFI (0.812) and TLI (0.805) statistics did not meet our prespecified criteria for goodness of fit. All HDQ items represented our hypothesised domains of disability (factor loadings >0.30). To interpret the first factor loading in table 2, one standard deviation (SD) increase in physical symptoms and impairments is associated with a 0.64 SD increase in loss of energy. Equivalently, the value of the Pearson correlation coefficient between loss of energy and physical symptoms and impairments is 0.64.

Table 2

Confirmatory factor analysis results—domains of disability in the HIV Disability Questionnaire (HDQ; n=361)

Factor loadings ranged from 0.34 (item 2: ‘I have diarrhoea’, loading on the physical symptoms and impairments domain) to 0.90 (item 22: ‘I have trouble thinking clearly’, loading on the cognitive symptoms and impairments domain). Domains of disability correlated with each other ranging from r=0.47 (between difficulties with day-to-day activities and uncertainty) to r=0.88 (between mental-emotional symptoms and impairments and challenges to social inclusion; table 3).

Table 3

Correlations between domains in the HIV Disability Questionnaire (HDQ)

Discussion

This is the first study to assess the construct validity of the HDQ, the only HIV-specific measure of disability. Results of our confirmatory factor analysis demonstrated good overall model fit of items with the domain structure, supporting the validity of the six domains of disability in the HDQ.

Floor and ceiling effects were evident across the HDQ. We believe that the floor effect, primarily seen in items that addressed day-to-day activities, likely reflected the way in which participants were sampled; most were living independently in the community and faced few challenges to mobility or self-care activities. Ceiling or floor effects, with associated severe skewness of item scores, may deflate standard correlation coefficients20 and lead to an underestimation of factor loadings. While all items loaded significantly on their hypothesised domain of disability (factor loadings >0.30), the factor loadings might be higher if item response scales were less skewed.

Correlations between the latent variables ranged from r=0.47 to r=0.88 (table 3). A high correlation between mental-emotional challenges and challenges to social inclusion (r=0.88) suggested that these latent variables may not be empirically distinct.12 However, these domains of disability were represented by items with similar wording, such as ‘I feel….’ (mental-emotional domain) and ‘I find it hard to….’ (social inclusion domain) which could explain the high correlation and obscure the discriminant nature between these two domains. The correlation between mental and emotional health challenges and challenges to social inclusion also may reflect the influence of mental health on aspects of social inclusion such as employment among people living with HIV.21 ,22 Overall, we concluded six domains of the HDQ represent the dimensions that comprise the larger construct of disability. Nevertheless, when administering the HDQ, researchers, clinicians and community members should acknowledge the interrelationships between dimensions of disability and the influence dimensions may have on each other when interpreting HDQ domain scores.

The prevalence of disability including physical impairments, activity limitations and social participation restrictions among people living with HIV have been documented using measures based on the International Classification of Functioning, Disability and Health (ICF).1 ,5 ,6 ,23 The ICF (and the measures derived from the ICF) do not take into account the domain of uncertainty, nor the episodic nature of HIV. The HDQ was developed from the Episodic Disability Framework, a conceptual framework specifically derived and empirically validated with, adults living with HIV.7 ,8 ,24 Results showed the highest median HDQ severity score was in the uncertainty domain, highlighting the importance of uncertainty as a key component of disability for adults living with HIV. The Episodic Disability Framework also has been used to inform qualitative approaches to exploring experiences of older men who self-identify as having HIV-associated neurocognitive challenges and has been considered an approach to conceptualise disability among people living with HIV internationally.25 ,26 However, this is the first known study to use the Episodic Disability Framework to inform the development and validation of a new quantitative measure of disability for adults living with HIV. With episodic health and disability identified as a key research priority in the HIV and rehabilitation field,27 the HDQ will be integral to accurately and consistently describe the health-related consequences of HIV, aging and related comorbidities with adults living with HIV.

The highest median HDQ presence score was in the cognitive symptoms and impairments domain. In other studies, self-reported cognitive symptoms have been associated with depression.28 We confirmed this association in our analysis; the correlation between cognitive and mental-emotional domains was 0.65. While subjectively measured components of mental health correlate with each other, treatment strategies to address mental-emotional and cognitive health symptoms can differ, suggesting these are distinct clinical concepts29 ,30 as represented in the HDQ.

Compared with other HDQ domains, symptoms and impairments tended to fluctuate more on a daily basis with median HDQ episodic scores greatest in the physical (35 challenges that fluctuated within the week) cognitive (33 challenges), and mental-emotional (18 challenges) symptoms and impairments domains, demonstrating the potential episodic nature of disability. This was expected given items related to symptoms and impairments such as fatigue, weakness and trouble concentrating may fluctuate more readily than those associated with social inclusion such as the ability to engage or re-engage in the workforce. Specific symptoms and impairments that fluctuated the most included fatigue, feeling sad, down or depressed, aches and pains, headaches, feeling anxious, having trouble sleeping and feeling weakness in muscles. Despite low episodic domain scores, participants demonstrated a range of episodic health challenges. Our analysis focused on domains of the disability severity scale of the HDQ. Further work is needed to assess the properties of the episodic scale of the HDQ.

Lastly, the majority of participants (82%) reported completing the HDQ on a ‘good day’ living with HIV despite the presence and severity of disability reported in the HDQ. This may be a reflection of resiliency, adaptation and hardiness among people living with HIV.31 Nevertheless, it is unclear how participants in the sample defined a ‘good day’ versus a ‘bad day’ living with HIV. Further work exploring the interpretation of this item as it relates to the HDQ disability scores is needed.

Implications for practice, research and policy

The HDQ is the first known HIV-specific Disability Questionnaire developed to assess the multidimensional nature of disability experienced by adults living with HIV. The HDQ has the potential to be used in clinical research, practice and policy. Patient-reported outcome measures are important for their ability to detect disability, monitor disease progression and facilitate patient-clinician communication.32 Further psychometric assessment including test–retest reliability, interpretability and responsiveness, will enable researchers to use the HDQ to document the presence, severity and episodic nature of disability experienced by adults living with HIV. The HDQ may be considered for use by clinicians and AIDS service organisations to assess the profile of disability experienced by their clients. This may help to identify areas to implement programmes, services and interventions to reduce disability experienced by clients with HIV. Universal measurement of disability with the HDQ may facilitate more broad and ongoing tracking of episodic disability trends and evaluation of interventions to inform resource allocation, as well as income and employment programme and policies to ensure optimal care and social inclusion for people living with HIV.

Strengths of our analytical approach included our adherence to the COSMIN guidelines for assessing validity and hypothesis testing, such as providing a clear description of the a priori hypothesised measurement model and goodness of fit criteria; providing details on the distribution of HDQ scores and missingness sample; and describing the correlations between domains.33 ,34

Our study has limitations. First, the HDQ was developed and validated primarily with men in their 40 s who were taking antiretroviral therapy, living with concurrent health conditions and not currently working. Generalisability of these findings to other people living with HIV has not yet been demonstrated. Similarly, the validity of the HDQ in other cultural and developmental contexts is unknown. Second, our goal was to validate the domain structure of the HDQ (not to measure disability) so HDQ scores should be interpreted cautiously. Third, our a priori goodness of fit criteria only was met for the RMSEA. We considered the RMSEA as the primary statistic for overall goodness of model fit because it is recommended for confirmatory factor analysis.16 Fourth, while our results indicate that dimensions of disability are correlated to comprise the larger construct of disability, the direct relationships between the domains of disability in the HDQ are unknown. Our results provide a measurement model which can serve as the foundation for future structural equation model analyses to determine the relationships between the domains of disability.

Our analysis focused on assessing the construct validity of the domains of the disability scale of the HDQ. Future research will assess the test–retest reliability, interpretability and responsiveness of the HDQ. Additionally, we will consider the measurement properties of the episodic scale. Longitudinally exploring the episodic nature of disability experienced by adults living with HIV is important to consider in future research.

Conclusions

Results from this confirmatory factor analysis support the validity of the domains of disability in the HDQ when administered to adults living with HIV in Ontario. This is the first known HIV-specific instrument of disability developed from the perspective of adults living with HIV. The HDQ can be used to describe the multidimensional nature of disability experienced by adults living with HIV and lay the foundation for more widespread measurement of disability in HIV clinical practice and research.

Acknowledgments

The authors acknowledge Marilyn Swinton for her contributions as Research Coordinator on this phase of the HIV Disability Questionnaire Study. The authors gratefully acknowledge the members of the Community Advisory Committee including Ken King (Canadian Working Group on HIV and Rehabilitation), James Murray (Ontario Ministry of Health and Long-Term Care), Rob Alexander (HIVER Consulting) and Shane Patey (Toronto People with AIDS Foundation) for their contributions to this work. The authors also acknowledge Community Collaborators in this work including Casey House, Toronto People with AIDS Foundation, Canadian Working Group on HIV and Rehabilitation, McMaster Special Immunology Services (SIS) Clinic, the AIDS Network (Hamilton), AIDS Niagara, AIDS Committee of Durham Region and AIDS Committee of Toronto.

References

Footnotes

  • Contributors KKO led the conceptual design of the study, acquisition of funding, conducted the analysis and drafted the manuscript. AMB and PS participated in the conceptual design of the study, acquisition of funding, analytical interpretations and drafting the manuscript. All authors read and approved the final manuscript.

  • Funding This research was supported by an operating grant from the Canadian Institutes of Health Research (CIHR) HIV/AIDS Community Based Research Programme.

  • Competing interests KKO was supported by a Fellowship from the CIHR, HIV/AIDS Research Program and Michael G DeGroote Postdoctoral Fellowship (McMaster University). AMB was supported by a Canadian Institutes of Health Research/Ontario Ministry of Health and Long-Term Care Applied Chair in Health Services and Policy Research. The Centre for Research on Inner City Health was supported in part by a grant from the Ontario Ministry of Health and Long-Term Care. The views expressed in this article are those of the authors, and no official endorsement by supporting agencies is intended or should be inferred.

  • Patient consent Obtained.

  • Ethics approval St. Michael's Hospital, Toronto, Ontario and McMaster University, Hamilton, Ontario.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement No additional data are available.