Introduction

In recent decades, our understanding of how health and disease are best measured has changed very considerably. For example, we now know that physiological measures often correlate poorly with functional capacity and well-being1 and patients with the same clinical criteria often have dramatically different responses to the impact of symptoms on their lives, this highlighting the subjective perception of disease impact. Outcome measures have been developed that reflect the patient perspective, aiming to understand symptom experiences and the impact of illness. Disease-specific, rather than generic, outcome measures aim to provide a focused picture of the day-to-day concerns of patients, and capture changes in health-related quality of life (HRQL) that may occur as a result of clinical treatment and care. These developments in outcome measurement have influenced the health policy agenda worldwide. In the United Kingdom (UK), they underpin the move towards routine use of patient-reported outcome measures (PROMs) in clinical practice.

PROMs are measures of the outcome of treatment that are reported directly by the patient or carer.2 They are typically short, self-completed questionnaires, most commonly used to measure patients’ health status or HRQL before and after an intervention.3 Over 3,000 generic and disease-specific PROMs exist4 and these are now commonly used in research contexts, particularly in clinical trials. The National Health Service in England routinely collects PROM data from patients undergoing certain surgical procedures to assess quality of care from the patient’s perspective. Pilot work has been completed into their use for long-term conditions, including asthma and chronic obstructive pulmonary disease, in primary care.5 PROMs data could potentially contribute to determinants of service quality, so that patient assessments of the quality of their experiences could be compared across services and between providers and have an impact on The National Health Service funding6 and patient choice.

The use of PROMs in clinical settings to demonstrate improved health and support clinical decision-making raises a number of challenges for clinicians, such as how to identify and choose clinically relevant, valid instruments and when and how to administer them. The acceptability of PROMs to patients and clinicians is not well evaluated. Assessing comorbidities is problematic, as patients with more than one condition may need to complete several disease-specific measures and a generic measure. Clinicians may lack knowledge of how to analyse and interpret PROM data, and if/how they can be utilised to assess changes over time in people with long-term conditions.5–13

It is apparent from the literature that a PROM may include the patient’s perception of symptoms, well-being, health/functional status, HRQL, satisfaction with treatment and outcomes, and perceptions of the humanity of care.2,7 We wished to focus on PROMs that measure health status rather than satisfaction with care and treatment; the latter are more accurately termed ‘patient-reported experience measures’.14 Unable to identify a clear, comprehensive definition of a PROM from the literature to guide our selection of PROMs, we proposed the working definition given in Box 1.

Asthma is one of the most common long-term medical conditions in the world, with an estimated 300 million people affected.15 The majority of asthma hospital admissions and deaths are thought to be preventable.16 Asthma can have considerable impact on personal health and well-being across the age spectrum; therefore PROMs have considerable potential in assessing the impact of asthma on HRQL from the perspective of patients and their caregivers.

Our study set out to identify all available disease-specific PROMs for asthma in children and adults (i.e., articles where the PROM was published first and its development described); identify and appraise the relevant methodological work reporting on development and validation of the PROMs; identify which generic PROMs are used in conjunction with the asthma-specific PROMs identified; identify what PROMS might be suitable for clinical use; and identify gaps in the PROMs available for asthma.

Materials and Methods

We conducted a systematic literature review and report here on the asthma PROMs appraised. As the detailed study protocol has previously been reported,17 we provide below an overview of the methods employed.

Search strategy

We searched the MEDLINE, EMBASE and Web of Science databases for relevant studies (see Supplementary Appendix 1 for search strategies). We also searched the PROQOLID (http://www.proqolid.org), PROMIS (http://www.nihpromis.org) and American Thoracic Society QOL resource (http://qol.thoracic.org/) websites for relevant tools that may not have been published.

Searches were limited to the literature from 1990 to 2012, based on the date of first publication of key PROMs such as the Asthma Quality of Life Questionnaire (AQLQ).18 Additional references were sought by searching the references cited by the identified studies, and unpublished work and research in progress was sought through discussion with experts in the field, and by searching the National Institute of Health Research and Agency for Healthcare Research and Quality databases. We invited experts who are active in the field from a range of disciplines and geographical locations to comment on our search strategy and the list of included studies. There was no language restriction, and where possible all literature was translated for initial screening. Titles and abstracts were screened by two reviewers, and the PROMs that had more than one paper reporting psychometric properties and those validated in the English language were selected for full appraisal.

Quality appraisal, and data synthesis and interpretation

We appraised the original paper describing the tool development and validation and subsequent associated papers describing further developments in validation of the tool, including validation in additional languages. The psychometric properties of each selected PROM were assessed in detail by two researchers using a quality appraisal tool developed by Pesudovs et al.19 They then compared their appraisals and resolved any discrepancies via discussion. We also had to take into account that the quality appraisal tool we used reflects contemporary standards of statistical analysis; we therefore adjusted our expectations from papers published in earlier years—for example, before Rasch analysis was established. A narrative synthesis summarising the development and validation of each PROM was also written by the appraisers. On the basis of this quality appraisal, we then considered whether tools were ready for clinical use and which of them needed further developmental work.

Results

Search results

Our searches identified 6,316 papers, from which we identified 593 papers on PROMs’ developmental work. Among these, we identified 68 PROMs for asthma; 13 were selected through screening as being sufficiently well developed and validated to merit full-quality appraisal: 8 for adults, 4 for children and 1 for a child’s caregiver (see Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram, Figure 1).

Figure 1
figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram. PROM, patient-reported outcome measure.

Full size image

Quality appraisal

The characteristics of the selected instruments are detailed in Table 1, and the quality appraisal in Tables 2 and 3. The narrative review of the 13 selected instruments is presented in Supplementary Appendix 2.

Table 1 Characteristics of included instruments
Full size table
Table 2 Summary of development properties of included PROMs
Full size table
Table 3 Summary of psychometric properties of included PROMs
Full size table

Many studies provided inadequate information on the development or validation of the PROM. There is understandably less validation work on the PROMs developed recently, such as Rhinasthma,20 compared with those published 20 years ago, such as the AQLQ.18

Instruments for use in adults with asthma

The PROMs for adult asthma that we found to be sufficiently well validated to offer promise for use in clinical settings were the AQLQ18 and its derivative, the mini-AQLQ.21 The Living with Asthma Questionnaire22 and Marks Asthma Quality of Life Questionnaire23 were both appraised as being adequately developed, but quality appraisal of their performance was considered unsatisfactory. We found insufficient published data to appraise the development of the St George’s Respiratory Questionnaire,24 although it performed well in appraisal of its psychometric properties.25 The short forms, the Airways Questionnaire 20 and 30,26,27 although potentially useful because of their shortened format, require further validation before they can be recommended. Rhinasthma is promising as a combined asthma/rhinitis PROM.20 The Asthma Control Test28 and the Asthma Control Questionnaire29 for adults are promising measures of asthma control, but focus mainly on symptoms and/or functional status rather than how these affect the patient’s personal perceptions of the impact of asthma on their quality of life.

Instruments for use in children with asthma

Fewer PROMs for children than adults were identified, although the value of finding a suitable PROM for children to self-rate their asthma-related quality of life (QOL) is clear, particularly as parent and clinician proxy assessments of a child’s asthma-related QOL may vary greatly from a child’s self-assessment.30 The Pediatric Asthma Quality of Life Questionnaire31 for children, a derivative of the AQLQ, has limited validation work published, but is the only PROM for children with asthma that addresses asthma-related QOL comprehensively and its performance appeared adequate. The Childhood Asthma Control Test also offers promise in assessing asthma control in children, although it requires further validation work as there has been some debate regarding whether it estimates poorly controlled asthma accurately.32,33 The Childhood Asthma Questionnaire was poorly validated and cannot be recommended.34 The PedsQL is a generic tool with disease-specific modules, including one for asthma, a combination that is potentially useful. Quality appraisal indicated that the development of the Generic Core Scales was adequate, but the performance was variable and the development and performance of the asthma module was less well described.35,36

Instruments for use with caregivers of children with asthma

The Paediatric Asthma Caregiver’s Quality of Life Questionnaire, another associated tool of the AQLQ, measures parental QOL related to their child’s asthma, which may be used in parallel with the child’s own rating through the Pediatric Asthma Quality of Life Questionnaire. Quality appraisal of the Paediatric Asthma Caregiver’s Quality of Life Questionnaire was broadly positive, based on a limited amount of validation work.37

Generic tools

We identified 28 generic tools that have been evaluated for use in people with asthma and that may be used in combination with asthma-specific tools. The most commonly used appear to be the Sickness Impact Profile,38 the 36-item Short Form Health Survey39 and SF-12.40 The EQ-5D41 has been adopted as the generic tool of choice for the evaluation of health status pre- and postoperatively in the National Health Service and for Department of Health-funded pilot studies on PROMs for long-term conditions, as it allows comparisons of performance between different conditions, across services and between providers, and also facilitates cost-effectiveness analysis.6 This is despite questions being raised about its fitness for purpose.42 A previous quality appraisal identified the 36-item Short Form Health Survey and EQ-5D as the most suitable generic measures in asthma.43 This, however, requires further testing.

Discussion

Main findings

Our evaluation of existing PROMs for asthma suggests that they need further validation even for research purposes, particularly PROMs for children. The psychometric properties in particular were found to need more robust validation work. Some PROMS, such as the AQLQ and its derivatives and Rhinasthma, show promise as being potentially useful in clinical practice and further validation work should be conducted on these. The mini-AQLQ, as a relatively compact PROM (15 items), may have particular utility in clinical practice. PROMs that focus primarily on asthma control, such as the Asthma Control Test, Asthma Control Questionnaire and Childhood Asthma Control Test, are inadequate as measures of HRQL.

The majority of PROMs we appraised were developed for use in research contexts, such as determining changes in HRQL as part of a randomised controlled trial of an asthma treatment, rather than in clinical settings. They may, therefore, be more suitable for group comparisons, rather than for determining change in an individual’s QOL over time. PROMs are generally validated at group level and, although there have been attempts to validate outcome measures at individual level using qualitative methods,44 this methodology is poorly developed as yet. This results in a gap in our knowledge of how to use PROMs with individual patients in clinical practice and whether they improve asthma management. Importantly, there has been very little attention paid to the net benefit of interventions from the patient’s point of view using the burden of treatment measurements, where patients are asked to weigh the advantages and drawbacks of an intervention.

Authors of the papers reviewed frequently expressed more confidence in the reliability and validity of their instruments than our quality appraisal supported. Many studies provided inadequate information on the development or validation of the PROM and almost none provided information on the minimal clinically important difference, crucial to determining the clinical significance of a change in an individual’s QOL over time, which is a factor highly relevant to PROMs’ use in clinical settings. Also, where PROMs were developed in one format but might be used in different ways, such as face-to-face, online, phone or post, validation was often arbitrary. Variables such as age, culture and socioeconomic status were often poorly addressed. Small sample sizes were common in reported studies.

From the literature and responses from experts we consulted, some additional PROMs appear to be commonly used in patients with asthma, notably the Asthma Bother Profile45 and the Royal College of Physicians Three Questions,46 the latter used particularly in UK primary care. These were too poorly validated to meet our inclusion criteria. The Asthma Bother Profile, however, appears potentially useful in clinical settings; it was designed for clinical contexts and the management questions are potentially useful in exploring factors that influence self-management skills and patient perspectives of care; therefore further testing is recommended. The Royal College of Physicians Three Questions, although recommended for assessment of asthma control in adults in the UK Asthma Guideline,47 and included as an essential component of annual asthma review within the Quality and Outcome Framework,48 has not been developed in the same rigorous manner or subjected to the same standard of evaluation as other measures. It provides a quick method of assessing asthma control and is an indicator of areas for further clinical assessment, but not a comprehensive estimation of the patient’s asthma-related QOL.

Strengths and limitations of this study

Few previous studies have looked specifically at PROMs for asthma, appraised their use in detail using robust criteria or considered their use in clinical contexts. Asthma provides a good model for considering PROMs’ use in patients with long-term conditions and comorbidities.

We may not have identified all available PROMs for asthma, but our search strategy attempted to address this by using a range of methods. Poor reporting and inadequate abstracts in some of the papers we identified may have led to some PROMs being excluded from full-quality appraisal. The PROMs field is rapidly expanding, and current validation work on some tools we excluded in this review, such as the Control of Allergic Rhinitis and Asthma Test49 and RhinAsthma Patient Perspective,50 which specifically address comorbid asthma and allergic rhinitis, may yet prove to be useful additions to the canon.

Interpretation of findings in relation to previously published work

Other studies have focused mainly on evaluating PROMs for research use and have used a variety of different methods of appraising PROMs. Apfelbacher et al.51 reviewed asthma PROMs using their own criteria and concluded that the purpose of instruments differs widely, tools may be chosen for pragmatic reasons, such as cultural/linguistic availability and tools should focus on HRQL, excluding symptom evaluation. Another review of asthma-related quality of life tools for clinical research was unable to recommend any existing tools, due to lack of adequate psychometric data, problems with scoring and a focus on asthma control rather than quality of life.52 The recently published report on the Department of Health pilot of PROMs for long-term conditions in primary care, including asthma, concluded that existing generic and disease-specific PROMs are less successful for people with multiple conditions, therefore new types of PROM which address this growing need will be required.5

Other studies have also reported on the lack of patient perspectives on PROMs. Active collaboration between clinicians and patients in PROM development was generally apparent in the PROMs we appraised. Patient perspectives of completing PROMs are also important, but under-evaluated,51 and the proposed value of completing PROMs should be transparent. Response rates from patients in the recent DH pilot on PROMs use were worryingly low, 38% overall.5 Research by members of our team, however, administering HRQL questionnaires in an outpatient allergy clinic setting, identified very high response rates in children (73–94%) and adults (80–86%) and little questionnaire fatigue.53–55 The intended use of PROMs by patients to inform effective choices in health care services requires further examination,8,13 particularly in relation to informed decision-making and health literacy.56

Implications for future research, policy and practice

It is a matter of concern that so many PROMs for asthma have been developed but are so poorly validated and barely used. This adds to the difficulty for clinicians in identifying an appropriate PROM for practice. It is not always clear why authors decided to develop new tools, but a common reason appeared to be that they decided a particular group or specific area of interest was not covered by existing tools. Given the lack of well-validated PROMs we identified, we can understand the temptation to start again by developing new tools which might perform more effectively. It may be preferable and more pragmatic for research to focus on developing the methodology for further validation of the most promising existing tools, including ways of determining their validity with individual patients. Whether conducting retrospective validation of existing PROMs or developing new ones, the most rigorous statistical methods should be used, such as Rasch analysis, which enables the examination of the hierarchical structure, unidimensionality and additivity of PROMs.

Advocating routine PROMs use in clinical practice is challenging when existing PROMs have been found to be inadequate. Their use in long-term conditions, where outcomes cannot necessarily be linked directly to healthcare interventions, requires further consideration and clarification of purpose. There is no distinct ‘before and after’ in asthma management as there is in a surgical procedure and clinicians need guidance about what an appropriate outcome might be in asthma care, for example reduced symptom impact, maintenance of function, or lack of deterioration, and when to measure it. With such a range of PROMs available, clinicians may have difficulty choosing a tool that is sufficiently well validated and fit-for-purpose. The unresolved question of how to use PROMs in patients with comorbidities may be a dilemma for clinicians in everyday practice.

Conclusions

We identified many PROMs for asthma, but only a small number were of adequate quality for use in research contexts, and even fewer of these were of potential value in clinical settings. Without further validation work to assess their appropriateness in clinical practice with individual patients, it is difficult to recommend these for routine use. That said, using an imperfectly validated PROM may still be of clinical benefit if it genuinely addresses patient-perceived quality of life and its limitations are understood. Identifying and further developing the best available PROMs and testing them in clinical practice will support the development of resources for clinicians to help them use PROMs meaningfully, such as an online toolkit. Looking ahead, there is a pressing need to develop PROMs to give a nuanced picture of HRQL in patients with related multiple clinical conditions.