Understanding the importance of key risk factors in predicting chronic bronchitic symptoms using a machine learning approach

Pediatric bronchitic symptoms, especially among children diagnosed with asthma, have been underappreciated and understudied [1‐3] and pose a significant clinical and public health burden, with substantial clinical costs, loss of quality of life, and school absences [4‐6]. Reliable prediction of chronic respiratory symptoms and an understanding of the relative importance of determinants would support prevention efforts, particularly amongst children with asthma who are at the greatest risk [6].

Previous studies have identified environmental and clinical risk factors for respiratory symptoms and exacerbations [1, 2, 6‐9]. For example, ambient air pollution and traffic-related air pollution near busy roads (concentrations of traffic pollutants: particulate matter, black carbon, total nitrogen oxides (NOx), and nitrogen dioxide (NO₂)) have been shown to be associated with asthma exacerbations and respiratory symptoms such as bronchitis and wheeze. Other risk factors include previous medical history, obesity, presence of allergens (e.g. cockroaches), and exposure to second hand smoking [6, 7]. The relative importance amongst the risk factors in predicting bronchitic symptoms has yet to be established.

There have been relatively few studies that have taken personalized approaches to predicting exacerbations or symptoms that incorporate demographic, environmental, and clinical risk factors. One such study focused on predicting asthma, wheeze, and eczema using a large heterogeneous set of attributes in a cross-sectional population setting [10]. Longitudinal information on predictors and exacerbations provide stronger causal evidence. In this study, we aimed to use longitudinal data from the Southern California Children’s Health Study (CHS) to predict annual assessments of chronic bronchitis symptoms using indoor exposures, ambient air pollution exposures and other susceptibility factors, and to evaluate the role of traffic/ambient air pollution in predicting the bronchitic symptoms.

The 4548 participants had information available from 2 to 7 visits each (average of 4.9). At baseline, participants were on average 6.5 years old, approximately half male (51.1%), and primarily Hispanic White (55.7%). 13.2% of the participants reported a diagnosis of asthma (Table 1). The baseline prevalence of bronchitic symptoms was 18.1% overall (36.7% in asthmatics and 11.5% in non-asthmatics). Of those children reporting BCP at baseline, 54% also reported BCP at the first follow-up year. Of those children not reporting BCP at baseline, 12.8% reporting BCP at the first follow-up year.

We constructed GBMs with an interaction depth of 1 because there was little evidence that more complex trees increased predictive ability (CV AUC was similar for GBMs with interaction depths from 1 to 4 and highest for interaction depth of 1, as shown in Additional file 1: Table S1). As shown in Table 2 and Additional file 1: Table S2, the set of symptoms/medication use risk factors yielded GBMs with predictive ability in terms of AUCs, accuracies, sensitivities and specificities nearly as high as that of the GBM using the set of all risk factors. The predictive ability of traffic/air pollution exposures was relatively poor. Specifically, the average CV AUCs for all participant models fitted using: all risk factors, symptoms/medication use, or traffic/air pollution exposures were: 0.77, 0.75, and 0.52, respectively. Average AUCs in the models stratified by asthma status (Table 2) were similar to the average AUCs obtained from fitting models using data on all participants and then validating the model by asthma status (Additional file 1: Table S3).

Relative influence analysis showed that having BCP in the previous year was the single most important predictor of current year BCP, overall, and within each asthma group (Fig. 2). Note that GBMs fit using only lag BCP had AUCs that were slightly lower than the GBMs with all predictors, implying that other predictors had modest predictive ability (Table 2). The average AUCs from models fit with the top 10 predictors were very similar to the average AUCs from models fit with all predictors (Table 2 and Additional file 1: Table S2). Partial dependence plots (Figs. 3, 4 and 5) indicated that, regardless of their asthma status, children with BCP in the previous year were more likely to have BCP in the current year. The top ten risk factors in both asthmatics and non-asthmatics also included: BMI percentile, itchy eyes, wheeze symptoms, age, and traffic/air pollution predictors (e.g., CALINE4 non-freeway NOx). The partial dependence plots suggested that children previously reporting wheeze were at increased risk of current BCP. Children (< 8 years old) and older children (> 14 years old) were also at increased risk. Non-freeway NOx had a positive association with BCP in all and non-asthmatic children. The directions of association were less clear with freeway NOx, BMI percentile, parent stress, and PM_2.5. The results from the same analyses applied to the non-asthmatic subgroups showed no difference results from the non-asthmatic group (Additional file 1: Table S4 and Figure S1).

Based on the partial dependence plots for all participants, and by asthma status, all continuous predictors were categorized except freeway NOx and education level for the logistic regression analysis. Specifically, age was categorized as 0–8, 8–14, and 14+ years old; BMI percentiles were categorized as 0–95 and 95+; non-freeway NOx was categorized as 0–7, 7–15, and 15+ ppb; age of asthma onset was categorized at 0–9 and 9+; PM_2.5 was categorized as 0–10, 10–20 and 20+ μg/m³; parental stress was categorized at 0–7 and 7+; NO2 was categorized as 0–10, 10–25, and 25+ ppb. Freeway NOx and education level was included as a continuous variable.

Logistic regression models had similar average test set AUC to GBM models with the same top 10 predictors (Additional file 1: Table S5, Figure S2, and Figure S3). Of the 10 predictors in the logistic regression model for all participants, asthmatics, and non-asthmatics (all of which were continuous), the maximum pairwise Pearson’s correlations were any asthma medication use and wheeze (R = 0.67), age of asthma onset and wheeze (R = -0.39), and itchy eyes and rhinitis (R = 0.68), respectively. Regression coefficients from the logistic regression models using one of the random training sets (Additional file 1: Table S6) should be interpreted with caution, since they are from prediction models that did not focus on adjustment for potential confounders and the covariates included in the models built for all children, and for children with and without asthma were different. That said, BCP in the previous year was significantly associated with the current year BCP (OR: 3.48 and 95% CI: 2.64–4.58 for all participants; OR: 3.01 and 95% CI: 2.03–4.49 for asthmatics; OR: 3.95 and 95% CI: 2.69–5.79 for non-asthmatics, with different adjustments for each model). Wheeze symptoms in the previous year were positively associated with current year BCP (OR: 1.60 and 95% CI: 1.13–2.26 for all participants; OR: 1.78 and 95% CI: 1.18–2.69 for asthmatics; OR: 1.93 and 95% CI: 1.12–3.28 for non-asthmatics, with different adjustments for each model). Itchy eye symptoms in the previous year had positive associations with current year BCP in the non-asthmatic subset (OR: 1.64 and 95% CI: 1.02–2.65). Children living in areas with high levels of non-freeway NOx [> 15 ppb] were at increased risk of BCP as compared to those with low levels [< 7 ppb] for all participants (OR: 2.51 and 95% CI: 1.36–4.47), and in the non-asthmatic subset (OR: 2.26 and 95% CI: 1.04–4.63).

We used gradient boosting models to build a prediction model and identify key risk factors for bronchitic symptoms in a large cohort of southern California schoolchildren. The best models had moderate discriminative performance (0.71 < AUC < 0.77), which might be considered good for our questionnaire-based outcome [20]. In general, previous symptoms—particularly previous year bronchitic symptoms—were the most informative predictors. Age, BMI percentile, itchy eye symptoms, parent stress, and traffic/air pollution risk factors (CALINE4 non-freeway NOx, and PM_2.5) were amongst the top predictors in models fit by asthma status, and contributed modest additional predictive information. No indoor/home exposures were found to be predictive. The predictive performance of our models was similar in the within-subject and across-subject test sets which suggests, somewhat surprisingly, that these prediction models would perform similarly if applied to future observations in these study participants or to data from new participants.

Our findings on the importance of previous symptoms align with several previous studies of risk factors for chronic cough and asthma exacerbations in children. Forno and Celedon reviewed publications predicting asthma exacerbations in children and reported that a history of recent severe asthma exacerbation was a strong risk factor for subsequent exacerbations, regardless of disease severity or use of controller medication [6, 21‐23]. In a 48-week Pediatric Asthma Controller Trial study with children age 6 to 14 years with middle-to-moderate persistent asthma, a history of an asthma exacerbation requiring a systemic corticosteroid in the past year was associated with a subsequent exacerbation (OR = 2.1, p < 0.001) [22]. Another study using administrative claims data from PharMetrics/IMS Health also confirmed the associations between suboptimal asthma control and history of recent asthma exacerbations with subsequent disease exacerbations [24].

Our finding that the top ten predictors included traffic-related pollution (CALINE4 freeway and non-freeway NOx) and regional, ambient PM_2.5 is in line with previous reports that exposure to poor air quality is associated with bronchitic symptoms in children, but put these results in context since air pollution effects are smaller than effects of previous symptoms. Chen et al. reviewed the findings from the CHS and reported that children with physician-diagnosed asthma were at higher risk of developing chronic lower respiratory tract symptoms such as bronchitis and phlegm (BCP) if they lived in more polluted communities [1, 2]. Specifically, yearly questionnaire based bronchitic symptoms assessment from 1996 to 1999 were associated with the yearly variability of particulate matter with aerodynamic diameter less than 2.5 μg (1.09 μg/m³, CI: 1.01–1.17), NO₂ (1.07 ppb^, CI: 1.02–1.13), and ozone (1.06 ppb, CI: 1.00–1.12) among the cohort of children with asthma in 12 Southern California communities. In a school-based, cross-sectional study in the San Francisco Bay Area in 2001, traffic-related pollution was associated with respiratory symptoms in children. Among those living at their current residence for at least 1 year, the adjusted odds ratio for bronchitis in relationship to an interquartile difference in NOx was 1.07 (95% CI, 1.00–1.14) [9]. Another CHS paper also indicates the relationship between new-onset asthma with traffic-related pollutions near homes and schools [25]. For example, asthma risk had a positive association with modeled traffic-related pollution exposure from roadways near homes (HR 1.51, 95% CI 1.25–1.82) and near schools (HR 1.45, 95% CI 1.06–1.98).

A recent study used several Bayesian network classifiers to predict the risk of asthma exacerbation in 65 pediatric asthma patients ages 1–14.5, with 2–4 measurements each [16, 26]. Using Backward Sequential Elimination and Joining algorithm (BSEJ), the authors achieved 93.84% accuracy and 90.9% sensitivity [16, 26]. The predictive performance of this model was better than we observed with the models in our dataset. Key differences include that our study was population-based whereas Spyroglou et al. recruited children from an asthma clinic who had recent cessation of asthma medication use. The prevalence of asthma exacerbation in the Spyroglou et al. study was 14.9%, similar to the prevalence in our study of BCP for all participants and non-asthmatics (18.1 and 11.5%, respectively), and lower than the prevalence of BCP amongst asthmatics (36.7%). Spyroglou et al. used only a within-participant test set to validate their models (held out the last observation for each participant) whereas we created within- and between- test sets to validate our models. Some features of the BSEJ algorithm make it less applicable to our study (e.g., only categorical predictors are permitted and it is challenging to apply to larger datasets or datasets with missing data).

We used gradient boosting model (GBM) to build prediction models for bronchitic symptoms. As GBM is comprised of multiple trees, successively constructed to overweight data that are hard to classify, it overcomes the biggest drawback of single tree models: their relatively poor predictive performance. Advantages of GBM include that it handles different types of predictor variables, it has a reasonable approach for highly correlated predictors (assigns importance to one of them, rather than splitting the importance across the highly correlated variables), is invariant to monotone transformations of individual predictor variables, is less sensitive to outliers, accounts for missing data using surrogate splits, and allows for automated detection of (potentially high-order) interactions, nonlinear relations [12‐14, 27‐29]. For example, for all participants, the association between non-freeway NOx and log odds of the current BCP was positive. According to the partial dependence plots, the association was nonlinear with a drastic increase when non-freeway NOx was greater than 15 ppb (Fig. 3). The logistic regression with the categorized non-freeway NOx also confirmed the nonlinearity – children living in areas with high levels of non-freeway NOx [> 15 ppb] were at higher risk of BCP as compared to those with low levels [< 7 ppb] for all participants (OR: 2.51, 95% CI: 1.36–4.47) and the risks of BCP were lower for children living in areas with medium levels of non-freeway NOx [> 7 ppb and < 15 ppb] (OR: 1.45, 95% CI: 1.10–1.91) (Additional file 1: Table S6).

A disadvantage of GBM (and prediction-driven machine learning modeling approaches in general) is that they are constructed under the goal of prediction. Effect estimates from these models (e.g., our logistic regression based on GBM results) should be interpreted with caution since the models were not constructed to account for potential confounders. We used relative influence and partial dependence plots to attempt to understand associations of key predictors. There are some drawbacks of these approaches. Specifically, one-way partial dependence plots assume no interaction effects [30]. In our case it was reasonable to consider only one-way partial dependence plots since our GBM models only had an interaction depth of 1. To ensure results were robust to sampling variation, we retrained the models using 50 multiple random training sets and evaluated the models using the corresponding test sets. Finally, we followed up on our GBM results by constructing logistic regression models that we found to have similar predictive performance. The logistic regression models can be readily applied to new study populations and clearly quantify the associations of individual predictors with the outcome.

This study applied machine learning to a longitudinal dataset, with up to 7 assessments per participant. Tree-based ensemble methods like random forest and GBM usually do not consider the dependency structure seen in longitudinal data [28]. If the dependency structure is ignored, correlation in the bootstrap samples used to produce each tree will lead to higher than expected correlation between trees and worse predictive performance. Relatively few studies have addressed the issue of how to model repeated measures data using tree-based models. Adler et al. considered data from a glaucoma registry with repeated measurements the left and/or right eyes of subjects and investigated the impact of varying the number of observations per subject in the training data for each individual base classifier (i.e., each tree) [28]. Specifically, Adler et al. compared the training data selection strategies consisting of: (1) one random observation per subject from a bootstrapped sample of subjects, (2) all observations per subject from a bootstrapped sample of subjects, (3) “naïve” bootstrap sample that ignored the correlation structure, (4) bootstrap samples of the subject-specific mean across all repeated measurements on a subject and (5) the newest observation of one selected eye per subject. Their results showed that sampling one observation was better than sampling all observations of each subject for both random forest and bagging classifiers. Our study took a similar approach and sampled one observation per participant for the ensemble-level training data. We further expanded on the Adler et al. method by creating complimentary within and across-participant holdout test datasets focusing on the generalizability of the model to future measurements on the same participants and completely new participants.

There are limitations to our study. In terms of the statistical methodology, to address the issue of correlation in longitudinal data, we used only up to two observations per participant for training and testing the prediction models and did not take full advantage of the up to 7 years of data per participant. Had the model been trained with repeated measures on each participant, we speculate that the within-participant holdout test AUC would have been higher. Future work might consider model-based longitudinal machine learning approaches such as RE-EM tree [31] for continuous outcome or a tree-based method using GEE (generalized estimating equations) for binary outcomes [32]. More generally, another limitation is that our outcome was assessed by annual self-reported questionnaire, so BCP symptoms might be underreported. Underreporting of symptoms would lead error in the outcome variable of our prediction models that is potentially systematic (e.g., different rates of underreporting for children who are asthmatic vs. non-asthmatic). Additional error in the outcome variable has the potential to harm the predictive ability of our models. However, if underreporting is consistent across years (i.e., a participant’s symptoms are underreported each year) the impacts of this underreporting on our prediction model may be attenuated since our models include reported symptoms in the previous year. Also, we developed models stratified by asthma status, which might attenuate any impacts of differential underreporting by asthma status.

In addition, our study did not consider some key individual level predictors such as individual genetic information, GxE exposures, diet, and indoor air. The lack of those predictors may explain why within-participants and between-participant predictions yielded similar results. This study also has a number of strengths. This study contains a large population-based sample of school children across several southern Californian communities with a wide range of risk factors including medical history, traffic and regional air pollutant exposures, and home exposures. Risk factors were investigated simultaneously to examine their relative importance in predicting the bronchitic symptoms. In addition, our study applied prediction modeling with all participants, and by asthma status.

Our study applied gradient boosting models to predict bronchitic symptoms among school-aged children in a longitudinal framework, offering a novel approach to better understand predictive factors of this outcome. We found that children with previous bronchitic symptoms were at the highest risk of developing subsequent symptoms, while several traffic and regional air pollution exposures also contribute to the overall model predictive ability. A similar approach can be used in future panel studies with more highly time resolved data to create personalized prediction models to potentially predict and prevent acute asthma exacerbations or chronic reparatory disease.