Classification and Regression Tree (CART) analysis to predict influenza in primary care patients

Influenza and other respiratory viruses cause an enormous healthcare burden in the U.S. Each year, influenza alone is responsible for 226,000 (54,000–431,000) hospitalizations [1] and 31.4 million outpatient visits [2]. Appropriate diagnosis and cost-effective treatment are dependent upon timely presentation for care, accurate and reasonably priced testing with short turnaround time, when appropriate. The U.S. Centers for Disease Control and Prevention (CDC) recommends antiviral treatment with a neuraminidase inhibitor for all outpatients with suspected or confirmed influenza who are at higher risk for influenza complications because of age or underlying medical conditions, when treatment can be started within 48 h of illness onset. Treatment also can be considered on the basis of clinical judgment, for outpatients with uncomplicated, suspected influenza who are not at increased risk for developing complicated illness, if antiviral treatment can be initiated within 48 h of illness onset.

Use of antiviral medications, specifically neuraminidase inhibitors, within 2 days of illness onset, has been shown to reduce the time to alleviation of symptoms by about half a day to a day, depending on the authority cited [3‐5], and the risk of secondary complications of influenza such as clinically diagnosed pneumonia in some meta analyses [6, 7], but not in others [3, 4]. Prescribing of antiviral medications among primary care clinicians for treatment of outpatient acute respiratory infections (ARI) is infrequent [8, 9], perhaps because of cost and/or side effects associated with neuraminidase inhibitors [3, 4, 6].

Because these medications are effective only against influenza, they should only be prescribed to patients with suspected or confirmed influenza. Without respiratory viral testing, differentiating influenza from other viral respiratory infections can be difficult. Although molecular multi-viral respiratory panel testing can accurately distinguish possible causes of ARI, these tests are relatively new and expensive. Rapid influenza tests, in contrast, are inexpensive but lack sensitivity [10]. Moreover, clinical experience suggests that the use of respiratory viral testing increases as influenza circulation increases in the community, but is ordered less frequently at the beginning and end of the influenza season.

Clinicians use medical clinical decision algorithms, based on a series of decision rules, to determine risk for a range of medical conditions [11]. The availability of a clinical decision algorithm for determining influenza would benefit patients and potentially minimize costs to the healthcare system by providing the opportunity to diagnose influenza with reasonable accuracy and potentially treat patients with antiviral medication.

This study was designed to determine if an algorithm developed using recursive partitioning resulted in reasonable estimates of the likelihood of ARI being due to influenza infection. Classification and regression trees have been used to analyze mortality in persons with influenza A (H5N1) virus [12] and the likelihood of seasonal influenza [13, 14]. In one study of seasonal influenza, the presence of influenza was most accurately predicted by a model that assigned scores to symptoms; it included fever plus cough, myalgia, duration <48 h and chills or sweats [13]. Using the same data set, but grouping specific symptoms into dichotomous variables (present/absent), a model was developed in which the presence of both temperature >37.3 °C and chills or sweats predicted high risk of influenza [14]. Absence of both symptoms predicted low risk, and presence of only temperature >37.3 °C predicted moderate risk. In the present study, the Classification and Regression Trees [15] (CART) methodology was used to estimate the likelihood of influenza among individuals presenting for outpatient care for ARI in 2011–2012 who were enrolled in the U.S. Influenza Vaccine Effectiveness Network (US Flu VE Network) study.

Distributions of demographic variables for all enrollees with PCR-confirmed influenza or no influenza are shown in Table 1 and for the final developmental and validation samples are shown in Appendix 1: Table 4. All variables used in the tested models are shown. Individuals with PCR-confirmed influenza were more likely to report fever, cough and fatigue at enrollment and less likely to report household smoking, asthma diagnosis and were younger than those without influenza. In this cohort, antiviral use was low. Only 185 of 4173 enrollees (4.4 %) were prescribed an antiviral medication (15 % of cases vs. 6 % of non-cases were prescribed antivirals; Chi square P < 0.001). The use of antiviral medication among those ≥5 years old with a positive PCR test was 22 % (n = 40).

Eighty-five percent of influenza positive cases in this study were not prescribed an antiviral medication; whereas, 94 % of influenza negative cases were not prescribed an antiviral medication. While empiric treatment of certain patients is recommended, influenza prediction tools may be a useful adjunctive approach to improving appropriate use of antiviral medication.

A range of techniques, including clinical judgment [21], clinical decision rules [13, 22, 23], and CART [14], has been used to predict the likelihood of seasonal influenza among individuals presenting for outpatient treatment of an ARI. In multivariable regression analyses, Monto et al. found that presence of fever and cough in adults and adolescents in ambulatory settings best predicted influenza confirmed by cultures, 4-fold increase in antibody titer, immunofluorescence, or PCR [21]. These findings were confirmed among children ≥5 years of age [23]. Subsequent work by Stein et al. [22], also using multivariable regression analyses and PCR to confirm influenza, found no improvement over clinical judgment of either rapid testing or a clinical prediction rule in which specific symptoms were assigned scores.

The advantages of CART are that recursive partitioning does not make any distributional assumptions about the modeled variables, and that it accounts for multi-level interactions among variables. We found a similar area under the curve with CART and logistic regression, but CART used fewer variables.

Afonso et al. used CART to develop a model predicting low and high risk of influenza among 459 patients combining data from two studies from different countries [14]. Their model included temperature >37.3 °C and presence of chills or sweating and produced an AUC of .75-.76. In comparison, we report a decision tree using four symptoms (fever, cough, fatigue and shortness of breath) and presence of household smoke with an AUC of .68–.69. We did not ask about chills/sweating. Our model had a PPV of 23 % and NPV of 95 %. Testing the sensitivity of 81 % and specificity of 52 % of the CART model across a range of prevalence values indicated that, in a more typical influenza season with a hypothetical prevalence of 25 %, the PPV increased to 36 % with a 91 % NPV.

In a review of clinical decision rules for diagnosis of influenza, Ebell and Afonso [24] presented suggestions for future studies of this topic, of which the present study addresses several. For example, the authors suggested using PCR as the reference standard for the detection of influenza, an adequate sample size, and a broad range of patients. PCR was used exclusively for confirmation of influenza in this study of 4,173 patients ≥5 years of age from across the U.S., thus addressing some of these conditions. The value of this analysis is the ability to identify, from among a group of patients with ARI and onset of fever or cough within 7 days, those patients with fever, cough, fatigue, as having influenza with 84 % sensitivity. Thus, the algorithm is good at identifying a group who do not need testing or antivirals (i.e., algorithm negative); however, among those “positive” by the algorithm, laboratory testing or treatment based on clinical judgement is indicated. Our additional analyses that limited the population to those whose onset of symptoms was 2 or fewer days before presentation, with a high risk condition or high risk age group, could be used to develop guidance for prescribing antivirals based on symptoms alone. CART analyses support previous studies using multivariable regression to predict influenza from symptoms [21‐23], but CART offers the advantage of requiring fewer variables for input into a clinical decision tool.

Determining a low-risk group in whom neither testing nor antivirals is warranted could save resources and avoid inappropriate antiviral prescribing and concomitant concerns about the development of viral resistance. A threshold approach to clinical decision-making was advocated by Pauker and Kassirer [25], with one threshold for testing and another for treating. For influenza, these testing and treatment thresholds vary by country and are lower for US compared to Swiss physicians [26]. We consider the 95 % NPV sufficiently good to rule-out influenza and avoid testing in this group. In light of the findings that use of antiviral medication among patients with influenza is low [27], and the cost of PCR testing is high [28, 29], a clinical decision aid based on symptoms offers the opportunity to promote appropriate testing during the times when it is most cost-effective, and increase antiviral medication use among those most likely to benefit, while avoiding a large increase in use in those who will not benefit.

Although CDC recommends empiric use of antivirals, their use remains low. Recursive partitioning using CART analyses to establish a clinical decision algorithm for influenza has good sensitivity and NPV, but limited PPV. Thus, it is good at identifying a group who do not need testing or antivirals; however, among those “positive” by the algorithm, laboratory testing or treatment based on clinical judgment is indicated. Further testing during additional influenza seasons may help to determine how this algorithm could be used to optimize decisions on laboratory testing and antiviral use in patients with ARI.