Influenza is an infectious disease of the respiratory system and the outbreak of influenza occurs worldwide in a seasonal manner, usually during the winter season in temperate climates. It is highly contagious and affects people via droplet contact. Influenza increases the morbidity and mortality among sufferers of cardiovascular disease (CVD), diabetes mellitus, asthma, obstructive pulmonary disease, and malignancy, who have higher chances of suffering from serious medical complications [1‐3]. CVD is the leading global cause of death; approximately 17.9 million people died from CVD in 2016, accounting for 31% of all global deaths. Heart attack and stroke together contributed 85% of these deaths [4]. It is very important to recognize that CVD patients suffer from higher morbidity and mortality rates when they are infected with influenza. Several meta-analyses and systematic reviews have revealed a strong association between influenza infection and acute myocardial infarction [3, 5]. These reviews strongly recommend influenza vaccination for CVD patients.

The coronavirus disease 2019 (COVID-19) is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and was declared a pandemic by the World Health Organization (WHO), on March 11, 2020 [6]. In the COVID-19 pandemic era, patients with comorbidities have an increased case fatality ratio, including 6.0% for hypertension, 7.3% for diabetes, and 10.5% for CVD [7]. With currently no proven vaccines or antiviral treatments, standard public health preventive efforts are being applied. To prevent the twindemic, where the COVID-19 pandemic and the 2020–21 influenza epidemic overlap, it is important to enhance influenza vaccination during the upcoming winter seasons.

In our previous study [8], we found that the vaccination coverage rate is low in non-elderly (< 65 years) CVD patients. To promote influenza vaccination coverage in CVD patients, it is necessary to identify a high risk population with low influenza vaccination adherence.

Machine learning algorithms build a model based on sample data, known as "training data", to make predictions or decisions without being explicitly programmed. As a subcategory of artificial intelligence, machine learning analyzes the patterns of the data and performs simultaneous tests on numerous variables to develop prediction models [9]. Therefore, this technique has a relative advantage over traditional statistical methods that can only address a small number of variables and cannot identify complex interactions between these variables.

The objective of this study was to create a model to predict influenza vaccination adherence using four different machine learning techniques: logistic regression (LR), support vector machine (SVM), random forest (RF), and extreme gradient boosting (XGB).

The univariable and multivariable logistic regression analysis results are summarized in Table 3 and Table 4. Among CVD patients aged ≥ 65 years, there are two significant variables associated with influenza vaccination, namely, the sex and national insurance state. Males (odds ratio [OR], 0.34 within a 95% confidence interval [95% CI], 0.14–0.84) and recipients of medical aid (OR, 0.34; 95% CI, 0.15–0.79) were less likely to receive influenza vaccination according to Table 3. In addition, recent health screening (OR, 1.97; 95% CI, 1.15–3.35) is associated with high adherence to influenza vaccination.

In CVD patients aged < 65 years, age (OR, 1.06; 95% CI, 1.01–1.12) and recent health screening (OR, 3.01; 95% CI, 1.56–5.79) are the two most significant factors associated with influenza vaccination (Table 4). Older CVD patients who have undergone recent health screening in this group are more likely to show high adherence to influenza vaccination.

Table 5 and Fig. 2 show the performance of the LR and the machine learning models in predicting influenza vaccination adherence on the test datasets. The AUC of LR is comparable to that of the three machine learning models. RF shows the best performance, with an AUC of 0.643 for age ≥ 65 years and AUC of 0.740 for age < 65 years; however, the difference from the other models is not significant.

Since patients with CVD present higher morbidity and mortality when infected by seasonal influenza, it is strongly recommended for CVD patients to receive influenza vaccination. However, the influenza vaccination adherence for this population is not high, especially among the non-elderly [8]. Recently, the use of machine learning has received increasing attention over time, especially in medical science where a tremendous amount of data has been generated from research and clinical practice [10, 11].

Using data from KHANES V, 815 adult CVD patients with accessible data were selected to develop machine learning model to identify low adherence to influenza vaccination. We developed several classification model using different machine learning techniques. With these model, CVD patients at risk of influenza infection can be identified so that health care and the promotion of influenza vaccination can be enhanced for these high-risk patients. Because a well-known factor associated with influenza vaccination is the age [8], since the vaccination is free for adults above the age of 65 in Korea, separate classification models were developed for the two age groups of age ≥ 65 years and age < 65 years.

Among the elderly CVD patients (age ≥ 65 years), the sex and national insurance type significantly affect influenza vaccination adherence. On the other hand, for the < 65 age group, the age and recent health screening status are significant factors. Using socio-demographic variables, health-related lifestyle factors, and health status, prediction models were generated using four machine learning techniques, namely, LR, SVM, RF, and XGB. Using a tenfold cross validation, the dataset was split into ten equally sized random groups; nine groups were used as the training sets for the prediction models, and the remaining group was used as the test dataset, and to train the four prediction models. The diagnostic performance of LR is comparable to that of the three machine learning models, and the RF prediction model shows the best AUC for predicting the vaccination status among CVD patients in both age groups.

From KNHANES III to VI, influenza vaccination rates have steadily increased from 2005 to 2014 in South Korea and high vaccination coverage was associated with female gender, rural residence, low education level, high income, and increasing number of chronic diseases [12]. Factors associated with influenza vaccination coverage was also analyzed among elderly, patients with diabetes, chronic obstructive pulmonary disease, asthma, CVD, cancer survivors using the traditional multivariable logistic regression analysis on KNHANES dataset [8, 13‐17]. In the present study, we used machine learning techniques to identify factors associated with low adherence to influenza vaccination among Korean adults with CVD. Machine learning has advantage over traditional statistical methods since it can address a lot of variable information and can easily address complex interactions between these variables.

Influenza vaccination plays an important role in protecting high-risk population, which is a group that is particularly vulnerable for COVID-19. A number of papers that highlight the significant benefits of influenza vaccination in the current COVID-19 pandemic [18]. Influenza vaccination reduced COVID-19 infection risk in COVID-19 infection prediction model [19]. The influenza vaccination would enhance the management of respiratory outbreaks coinciding with the peak flu season, thus enabling more efficient use of healthcare resources [20].

There are several limitations to this study. First, this study has a small sample size. A reliable way to validate the performance of a machine learning model is to train the model with available data and assess its classification performance using newly collected data or a separate dataset. Using unseen data to evaluate a machine learning model gives an unbiased estimate of its performance. We developed machine learning models to classify 16 survey-based variables (most of the variables were binary variables) in a total of 778 adults with CVD using data from KHANES. These 778 adults were subdivided into two groups according to their ages: adults aged ≥ 65 years (n = 496) and adults aged < 65 years (n = 282). Because of the small sample size, the receiver operating characteristics (ROC) curve is not a smooth curve but a step graph. When validation with a separate dataset is not feasible because of the small sample size, K-Fold cross-validation is very economical as it uses all the data for training and also reuses all the data for validation. Cross-validation is a common solution when the available datasets are limited; instead of training a fixed model only once as in the train/test split, several models are iteratively developed using different portions of the data on the cross-validation method. Second, the dataset was imbalanced. Imbalanced data is an unequally distributed dataset in which a certain class of data is significantly larger in quantity than the other data classes [21]. Owing to this disproportionate dataset, the prediction model tends to have good accuracy on the majority class data but poor accuracy on the rest of the data. That is, the prediction model can be inaccurate and biased towards the majority class data. In this study, among the 778 CVD patients, 526 received influenza vaccination and the remaining 252 did not. Because the proportion of people with influenza vaccination is quite different between the two age groups, there is a possibility of learning bias in the machine learning models. Due to these limitations, further studies using larger sample sizes and balanced datasets will be needed.

Influenza vaccination is an important public health goal for protecting high-risk cardiovascular patients in the context of the ongoing COVID-19 pandemic. It is very important to identify the low adherence group and implement strategies to increase influenza vaccination coverage.

Machine learning models showed comparable performance in identifying low influenza vaccination adherence among CVD patients. Machine learning model might be used to enhance the health care for high-risk CVD patients by identifying patients at risk of influenza infection and promoting influenza vaccination to them.