Background
Asthma is a serious, highly prevalent disease which affects all age groups, with an estimated 1 in 13 people affected in the United States (US) [
1]. Globally, 300 million people suffer from asthma with 250,000 deaths attributable to asthma each year [
2] and a prevalence of 8.2% which has increased annually between 2001 and 2009 at a rate of 1.2% [
3]. With around 12.8 million people (8.7 million adults and 4 million children (aged 0–17)) having an attack within the last year, resulting in 10.5 million missed school days and 14.2 million missed work days in 2008 [
3], asthma has a significant societal impact and is a serious public health problem [
4].
Airflow limitation measured by forced expiratory volume in one second (FEV
1), has been, and still is, an important endpoint used in asthma clinical trials [
5,
6]. However, there is growing recognition that traditional clinical outcomes, such as lung function, are inadequate indicators of how asthma patients function and feel [
7]. The multifactorial and complex nature of asthma control [
8] is supported by evidence indicating poor correlations between lung function and symptoms and between lung function and ß-agonist usage [
9,
10].
The American Thoracic Society/European Respiratory Society (ATS/ERS) Task Force acknowledged the multi-faceted nature of asthma in its recent definition of asthma control as a summary term capturing symptoms, reliever use, frequency/severity of exacerbations, lung function, and future risk [
5] and the Global Initiative for Asthma (GINA) defines the clinical manifestations (well established markers of symptom severity) of asthma to include symptoms, sleep disturbances, limitations of daily activity, impairment of lung function, and use of rescue medications [
4].
Clinicians tend to underestimate asthma symptom severity, making the use of a patient-completed daily diary for the assessment of asthma symptoms and markers of symptom severity in clinical trials essential [
11-
13]. As identified by the National Institutes of Health (NIH) Asthma Outcomes Workshop [
14] and further confirmed by an internally conducted focused literature review, there is no current asthma diary that can be recommended for use due to the lack of published validation data or appropriate recall period.
The Asthma Control Questionnaire (ACQ) is widely regarded as a useful measure that includes some core asthma symptoms and symptom-related clinical markers of asthma. While the ACQ is designated as a core measure in NIH-initiated clinical research [
8], the seven-day recall period represents a potential shortcoming in the context of FDA recommendations for symptom measures in clinical trials [
15]. Given the variable nature of asthma symptoms, a measure with a shorter recall (i.e., daily), is preferable. Also, while self-reported use of rescue medication is important to capture, its incorporation into a symptom scale may introduce some limitation in symptom scale interpretation (i.e., to what extent do changes in the symptom score simply reflect changes in rescue medication use rather than reflecting a true change in symptoms).
Prior to the Food and Drug Administration (FDA) Guidance for PROs, Amgen had developed an electronic patient-reported outcome (ePRO) diary to assess asthma symptoms and rescue medication use on a daily basis. Following the publication of the FDA Guidance, it was recognized that the measure required additional study to evaluate and document its content validity [
16,
17]. Following the recommendations in the FDA guidance, qualitative patient interviews were conducted to elicit relevant concepts. Subsequently, cognitive interviews were conducted to assess patient comprehension of the concepts presented by the diary items. As the measure under development was intended to be a daily diary, the team, comprising clinical and PRO measurement experts, decided that capturing the broader range of asthma impacts was not feasible. However, as reflected in the literature [
18] the effect on sleep and activities are key markers of symptom severity. Therefore a decision was made that, if supported by patient qualitative research, these two items would be retained (at least in the qualitative phase of study) as markers of symptom severity.
This work aimed to evaluate the content validity of the asthma symptom diary (ASD) for use as an efficacy endpoint in asthma clinical trials to support labelling claims. This article describes the qualitative research phase documenting content validity in adult and adolescent asthma patients, and the resulting revisions. The evaluation of the cross-sectional measurement properties of the ASD, conducted subsequent to the qualitative research, will be reported elsewhere [
19].
Methods
Concept elicitation interviews (n = 50) and cognitive interviews (n = 24) were conducted in adults and adolescent (≥12 years) who had a clinical diagnosis of persistent asthma. Four sites, asthma care clinical practices, in different regions of the US were used to recruit patients for the interviews. Institutional Review Board (Essex IRB Inc.) approval was obtained prior to study initiation.
Subjects
Subjects were screened for eligibility, enrolled, and scheduled for one of the two types of qualitative interviews. Included subjects had an asthma diagnosis for at least one year, were on asthma treatment maintenance medication regimen, and were either non-smokers or ex-smokers (<10 pack years and stopped ≥1 year ago). Subjects were not included if they had any major health problems interfering with asthma or were not able to give informed consent. Additionally, the study was designed to ensure that patient clinical and demographic characteristics were similar to those to be included in the planned clinical trial program.
Study measures
As part of the enrollment process, patients completed a self-administered version of the Asthma Control Questionnaire (ACQ-7) and the Asthma Quality of Life Questionnaire (AQLQ).
The ACQ-7 [
20] is a seven-item instrument assessing asthma control during the past week. The ACQ includes six patient-reported items about nighttime awakening, symptom severity upon awakening, activity limitation due to asthma, shortness of breath due to asthma, wheezing, and use of a short-acting bronchodilator. The final item is the forced expiratory volume in one second (FEV
1) % predicted completed by clinicians following spirometry. Items are scored 0 (good control) to 6 (poor control), and the total ACQ score is the mean of the seven items.
AQLQ12+ (12 years and older) [
21] contains 32 items comprising four domains: Activity Limitation, Asthma Symptoms, Emotional Function, and Environmental Stimuli. Items are scored 1 (severe impairment) to 7 (no impairment), with domain scores calculated as the mean of the items within each domain; the overall AQLQ12+ score is the mean of all 32 items.
Clinical and sociodemographic forms
Clinicians provided additional descriptive information from the patient records and subjects completed a sociodemographic form that included patients’ date of birth, gender, race/ethnicity, years of education, employment status, and income level.
Concept elicitation interviews
The first phase of the qualitative work consisted of individual concept elicitation interviews with 34 adults and 16 adolescent subjects having a diagnosis of asthma. Each face-to-face individual interview was conducted privately and lasted approximately 60 minutes. The semi-structured interview guide asked patients to describe their experience with asthma symptoms and symptom related impacts during a typical day. Rating exercises for symptom bothersomeness and degree of difficulty patients experienced in coping with their asthma were conducted at the end of each interview. Interview results were used to identify the most relevant concepts to the patient experience, and the predominant terminology used by patients in identifying their asthma symptoms and impacts.
Following the concept elicitation interviews an item generation meeting was convened to review the qualitative data and revise and/or create new items to the existing items in the earlier diary. This meeting was attended by clinical experts in the treatment of asthma, PRO development experts, and members of the sponsor’s team.
Cognitive interviews
The cognitive interviews were subsequently conducted in 15 adults and 9 adolescents to assess the patients’ comprehension of the items, the clarity of response options, and the overall feasibility of the measure (format, instructions, appropriate attribution, and language) in the revised version of the ASD. The cognitive interviews were conducted in an iterative process using three waves with evaluation and revision of the diary items following each wave of interviews.
Analysis
All interviews were digitally recorded and then transcribed. ATLAS.ti software was used to organize the codes being assigned to the transcript data. Descriptive statistics from screening and enrollment data were used to further describe the study population.
A concept saturation table was used to identify the point at which no new concepts were being identified in the data. The transcripts were ordered chronologically based on interview date, and grouped into quartiles. The codes of each subsequent transcript group were compared with those from the prior group. If new codes appeared in subsequent group, it would suggest that saturation had not been achieved.
Translatability (cross-cultural applicability) and reading comprehension level (lexibility) assessments were conducted. All alterations made to the ASD items throughout the process were recorded in an item tracking matrix.
Discussion
This qualitative research identifies a set of symptoms and markers of symptom severity that are relevant to the experience of patients who have asthma. The most relevant symptoms identified in both the adult and adolescent populations included chest tightness, wheezing, coughing, and shortness of breath. These symptoms, identified in the qualitative research, are consistent with the core asthma symptoms identified in the literature [
5,
18]. Among impacts that are considered to be markers of symptom severity, limitations in physical activities and sleep disruption were identified in this qualitative research study as being most difficult for patients to cope with. The decision to include these two markers of symptom severity in the ASD was supported by both the qualitative research (patient spontaneous self-report) as well as the literature [
8,
18], which indicates that physical activity and sleep problems feature prominently in asthma. Although activity limitations and sleep disturbances might also be considered as assessing impact, the study team, influenced by the weight of the evidence in the literature, theorized that, in the case of asthma, symptom severity may be expressed in terms of their impact. The study team conceptualized that one or more markers of symptom severity could be so interrelated to symptoms that they might scale together with symptoms. A well-established indicator of asthma symptom severity is interference with sleep and activity limitation. Although these could be considered impacts rather than symptoms per se, they were included as well-established markers of symptom severity, and were endorsed during the qualitative phase and confirmed during the quantitative phase.
Fatigue was another symptom raised in the interviews but it was decided by the team of clinician and PRO experts not to include it as it was posited that it may be a symptom that is not be solely attributable to asthma. Because asthma symptoms frequently vary within a 24-hour period, the team decided to include symptoms in both the morning and evening ASD to accurately capture this variation. During the development process, the ASD item wording was revised to ensure that the items were understandable to patients with lower reading levels, as well as to adolescents. Consideration was also given to selecting language usage that was easily translatable.
The ASD is a newly developed PRO intended for use in clinical trials to support labeling claims. The ASD was developed in line with FDA guidance in terms of content validity in a population similar to the clinical trial population, and includes evidences to support concept relevance, saturation of concept, and appropriate understanding of the concepts presented on the part of the patient. The ASD content, structure, and relevance to intended measurement strategy for assessing the severity of asthma symptoms is supported by input from qualitative interviews with patients, consultation and revision by clinical experts, by asthma literature, and by PRO experts.
It is important to note the considerable overlap between the markers of symptom severity of asthma as defined by GINA (symptoms, sleep disturbances, limitations of daily activity, impairment of lung function, and use of rescue medications) and those identified by participants in this qualitative research. As there are established clinical measures for assessing lung function, this aspect was not explored for inclusion in the ASD; however, all other aspects of asthma, as contemplated by GINA, emerged in the qualitative subject interviews and, in addition to the clinical endorsement of these symptoms throughout the development process, add support to the content validity of the ASD.
Traditionally the assessment of asthma has included clinical factors such as lung function and exacerbation rates. Patient-reported and clinical composite measures have also been used to assess the patient experience although none presently meet the rigorous standards reflected by current FDA guidance [
15]. This together with the evidence that lung function [
7] and clinician reported severity [
11-
13] do not necessarily reflect a patient’s experience of asthma and highlights the need for a more accurate assessment of asthma outcomes. Moving forward, in accordance with recent consensus recommendations [
4,
5,
22], if studies are to accurately assess treatment options, then they need to accurately assess multiple asthma outcomes. Symptoms are one of the key elements that require attention and this need is reiterated in the review by Krishnan et al. [
14] which did not find a single measure suitable for recommendation as a “core” symptom measure. Thus, the development of a patient-reported symptom diary, with documented content validity, serves an important need; to obtain the patients’ perspective on treatment outcomes that are not presently accurately captured in other endpoint assessments.
Qualitative research confirmed the overall content of the original clinician developed ASD. Expert clinicians reviewing the qualitative results agreed that there were no additional asthma symptoms that needed to be added to the diary. Refinements to item language and response items were based upon the results of the qualitative research with patients in conjunction with significant input from clinicians and PRO experts. The similar manner in which both adults and adolescents reported and understood their experiences suggested that one single measure for use in both groups was found to be appropriate. As there was a clear distinction between day and night symptoms for both groups, it was determined that the diary should be administered twice in a 24-hour period—once upon awakening in the morning and once prior to retiring to bed in the evening.
The ASD consists of a single domain assessed twice-daily via an electronic device. The ASD has documented content validity evidenced by qualitative research with asthma patients. This qualitative research study was performed in line with the recommendations in the FDA PRO Guidance. Evaluation of cross-sectional measurement properties has been completed, further confirming unidimensionality (by factor analysis) and content validity (by Rasch analysis), and will be reported in detail in a separate publication.
Conclusions
The development of the ASD was in line with recommendations in the FDA PRO Guidance document, with key input from patients, clinicians and the literature. Revisions were made during the development process based on patient responses, clinical advice and expertise in the development of PROs. Patients included in this research reflected a diverse asthma population in terms of disease control, gender, age (both adults and adolescents), education, ethnicity, and socioeconomic status mirroring the general asthma population as well as the intended trial population of adults and adolescents with persistent asthma. In the concept elicitation interviews concept saturation and content validity was confirmed. The outcome of this study suggests that the ASD has demonstrated content validity. The resulting 11-item morning and 10-item evening diary assessing asthma symptoms and markers of symptom severity, instructions, and response options are well understood and relevant as verified by patients with asthma. This study supports the content validity of the revised ASD, showing it to be consistent with patient experiences and ready for further psychometric testing. The results of the quantitative assessment of the psychometric properties of the revised ASD are reported elsewhere.
Competing interests
Dr Martin is employed by RTI Health Solutions (RTI-HS), while Dr Wiklund, Ms von Maltzahn and Ms Mattera are employed by Evidera. Both companies provide consulting and other research services to pharmaceutical, device, government, and non-government organizations. In their salaried positions, they work with a variety of companies and organizations. They receive no payment or honoraria directly from these organizations for services rendered. Dr. Schatz received consultation fees from Amgen Inc. for this PRO development study. Dr Lin was an employee of Amgen when the work was done. He continues to be a shareholder of Amgen and is currently an employee and shareholder of Gilead Sciences. Dr Globe is an employee and shareholder of Amgen, Inc.
Authors’ contributions
MM was involved in the study conceptualisation, conducting the study and reporting the results from the study. GG, JL and MS were involved in study conceptualisation, protocol design and review and interpretation of results. GG was also involved in the drafting of the manuscript. IW, MSM and RvM were involved in the review and interpretation of results and the drafting of the manuscript. All authors read and approved the final manuscript.