Age and African-American race impact the validity and reliability of the asthma control test in persistent asthmatics

Asthma is a common chronic illness with significant morbidity and mortality, despite the availability of evidence-based treatment guidelines. There continues to be noticeable disparity in asthma outcomes among African-Americans, with rates of asthma-related healthcare utilization and death approximately 2 to 3 times the rates seen in Caucasians [1]. Asthma among African-Americans is disproportionally not well controlled [2‐4]. Adolescence may add additional risk, as children are becoming more independent from their parents and engaging in risk-taking behaviors [5]. Teenagers with uncontrolled asthma are more likely to “normalize” their asthma and have a higher threshold for reporting symptoms and initiating treatment [6].

The National Asthma Education and Prevention Program (NAEPP) guidelines recommend assessment of asthma control at each asthma visit, including frequency of asthma symptoms and rescue medication use, activity limitation, unscheduled healthcare visits, and spirometry measurement [7], and have incorporated use of standardized asthma questionnaires including the Asthma Control Test (ACT)™ into this assessment. Asthma questionnaires have taken a leading role in clinical management and are also frequently used in research for subject selection and for measurement of treatment effects. The self-administered questionnaire assessing impairment during the previous four weeks was validated for use in those 12 years and older [8], but concerns have been raised about the performance of the ACT in adolescents and ethnic minority populations [9], who were under-represented in previous validation studies. While a score of 19 or less out of a possible 25 points showed the greatest sensitivity and specificity for uncontrolled asthma [8], these findings were derived from predominantly Caucasian adult study populations, with median ages of 35 to 45 years [8, 10]. Studies examining the use of the ACT in adolescents of European and Mexican ancestry have reported higher optimal cut points to distinguish well controlled from uncontrolled asthma [11‐13]. This divergence may be in part due to cultural and developmental or age-related differences in the way symptoms are perceived and reported, as well as differences in health literacy when compared to Caucasian adults. While questionnaires like the ACT were initially developed to serve as one component of the asthma control assessment, many busy practices have embraced the ACT as essentially a replacement for the recommended multidimensional assessment. This is especially concerning given the questions surrounding the performance of the ACT in teens and minorities who may already be more likely to have their asthma severity underestimated by healthcare providers [14‐17]. To avoid inappropriate medical management, it is imperative that tools like the ACT, which rely solely on patient-reported data, be validated in the populations in which they are frequently used [18].

To address these questions, we conducted a prospective study to evaluate the validity and reliability of the ACT in African-American and non-African-American adolescents with persistent asthma, with emphasis on the clinical utility of the ACT for asthma management.

Standardized patient-reported questionnaires like the ACT allow providers to quickly assess asthma control in busy clinical practices. While previous studies have provided evidence in support of the validity and reliability of the ACT for detecting suboptimal asthma control [8, 10, 25], information on how the test performs in adolescents and in ethnically diverse populations is lacking given concerns that the questionnaire may not be sensitive to how teenagers and minorities perceive asthma-related impairment. Because minorities and teens/young adults are already at higher risk of poor asthma outcomes, it is imperative to understand the value of the tools being used to drive management decisions in these groups, to avoid overestimating level of control and under-treating asthma.

Our results suggest that the currently accepted ACT cut point score of ≤19 showed reduced accuracy compared to previous studies [8, 10, 25] in identifying asthma that is not well controlled in both AA and nAA adolescents. In both groups, use of a higher cut point score of ≤21 achieved the greatest area under the ROC curve and the best balance of sensitivity and specificity for identification of not well controlled asthma. That these results were seen in both AA and nAA groups suggests that a higher ACT cut point score may be necessary in adolescents. It is well established that children and adolescents are less accurate in describing their perceived asthma control [26‐32], impacted by age and developmental level. They may “normalize” their asthma symptoms and therefore not recognize them as being problematic, or they may minimize symptoms to avoid being categorized as different from their peers [33, 34]. Given these well documented differences in symptom perception and reporting, it seems logical to adjust the expectations of self-administered questionnaires that are based primarily on patient-reported data points. It is also important to emphasize that this is not a problem isolated to teenagers. In fact, using the childhood ACT (cACT) for children under 12 years, our group has published evidence that caregiver perception of asthma control may be even more discrepent from physician assessment than teen perception [9].

Analysis of individual ACT questions demonstrates that certain elements of the assessment may be more predictive of uncontrolled asthma than others in AA teens, particularly activity limitation and nighttime symptoms. This subtest requires further study to determine its potential utility as an adjunct measure of asthma control but emphasizes the importance of considering cultural context in the design of standardized questionnaires. For example, African-Americans with asthma may report less nighttime awakening and dyspnea, two symptoms that account for 20% of the ACT score [15, 26, 35]. This leads to under-reporting of symptoms, which reflects a false level of asthma control when queried by the ACT.

Our findings demonstrate acceptable levels of internal consistency reliability for both the AA and nAA teens at the group level; however, the reliability is below what is the recommended threshold for individual level assessment. This finding recognizes that there is a higher level of measurement error associated with the estimate that may caution providers to not solely rely on the ACT for determining asthma control. Further, the test-retest reliability of the ACT within the AA teen group was lower than the nAA teens. There should be some caution in overinterpreting the test-retest reliability as there was a 6–8 week time lapse between the baseline and follow up assessments; thus, patients’ stable asthma control status could fluctuate and participants’ memory of their control at baseline may be less clear.

Few participants experienced a change in level of asthma control, which was not surprising given the lack of an intervention and the brief 6–8 week study period. The responsiveness of the ACT to changes in asthma control was marginal in both AA and nAA adolescents. We found no significant correlation between change in ACT scores and change in asthma control status among nAA teens, with a small yet statistically significant correlation in AA teens. However, we found no difference in mean change in ACT scores between groups whose asthma worsened, improved, or remained stable in the AA group, suggesting that ACT scores do not significantly increase or decrease with improved or worsened asthma control, respectively. Interestingly, change in ACT score was strongly correlated with change in FEV₁ and FEV₁/FVC in nAA teens but very poorly correlated in AA teens, despite similar median % predicted FEV₁ between the two groups.

The ACT did not discriminate well between participants with higher lung function and those with lower lung function, as seen in previous studies [8, 10, 36]. However, we recognize that in children especially, spirometry alone is often a poor measure of asthma control, since children often have normal spirometry despite poorly controlled asthma. Indeed in our study, the median %predicted FEV₁ for both AA and nAA teens was greater than 90%, despite a significant proportion of our subjects having asthma that was not well controlled.

The fair to moderate agreement (κ) between the ACT and NAEPP-based physician assessment of asthma control is consistent with prior validation studies but, in our view, emphasizes the danger of relying too heavily on the ACT score for medical decision making. We argue that while the ACT using an optimized threshold score may be a useful screening tool, it should not replace a comprehensive physician assessment of control in the clinical setting. However, the context in which the questionnaire is being used may impact its utility and the choice of cut point score. For example, when used in the clinical setting, a more sensitive cut point should be chosen to minimize false negatives (persons with asthma that is not well controlled whose ACT score indicates controlled disease). Conversely in the research setting if the goal is to recruit poorly controlled asthmatics, a cut point that maximizes specificity, reducing the rate of false positives, is desired to avoid recruiting well controlled asthmatics to the study.

Accurate assessment of asthma control requires evaluation of both of the essential elements that comprise control: impairment and risk of future exacerbations. Pediatric asthma is often associated with low levels of day-to-day impairment but excessive risk in the form of frequent exacerbations, often occurring with viral respiratory infections. The ACT primarily measures impairment and does not take into account elements that increase risk (such as history of exacerbations in the prior year, unscheduled healthcare visits or hospitalizations for asthma), which may limit its utility in pediatric patients. The ACT also provides no mechanism to estimate asthma severity. The Composite Asthma Severity Index (CASI) takes into account impairment, exacerbations, controller medication requirement and lung function to provide a more comprehensive evaluation of asthma and to measure response to therapies [37]. A similar multipronged evaluation of asthma may be a better guide for stepping up or down treatment in clinical practice or for measuring asthma treatment responses in clinical research.

Our study has several limitations. The relatively small sample size increased the variance seen and did not allow us to explore sub-group differences by age (e.g., 12–15 years vs 16–18 years) to examine whether younger children had more challenges than older children in understanding the questionnaire or relating their asthma control experiences to answer the questions. All participants and their caregivers were required to speak English to participate in the study, which hindered recruitment of Latino children, another group at high risk of asthma-related morbidity. Our physician assessment of asthma control was binary (well controlled vs not well controlled) in contrast to other studies that incorporated terms such as “partly controlled”, “somewhat controlled”, or “not at all controlled,” which may have limited our ability to assess the discriminant validity of the test. Few participants had a change in their asthma control status between the two study visits, which likely limits our ability to make inferences about responsiveness of the ACT to change. While the AA and nAA teen groups were well matched in terms of age, sex, atopic status, BMI, and FEV₁, the AA group contained a larger proportion of participants on step 5 therapy or above compared to the nAA (49% vs 24%), suggesting increased asthma severity within the AA group. To address the possibility that our study populations may have contained a higher number of “poor perceivers” of asthma symptoms relative to the general adolescent population, we recruited from both general pediatrics clinics and subspecialty clinics in an effort to achieve more generalizable results. However, replication of this study in a larger group with a wider range of disease severity is needed before these findings can be applied to asthmatic teenagers in general.

We found some evidence for the validity of the ACT but not ideal levels of reliability for individual level assessment along with reduced sensitivity for detection of not well controlled asthma in adolescents with persistent asthma, independent of race. A higher threshold value to define asthma control improved the predictive properties of the ACT and requires further investigation. Additionally, AA teens’ scores were less responsive to change than nAA teens, suggesting that use of the ACT to detect changes in clinical status over time may be impaired in this population. While the ACT should be used to support the assessment of asthma control, it should not be relied upon as the sole indicator of control and should not replace the NAEPP-guidelines based physician assessment.