Prospective multicentric validation of a novel prediction model for paroxysmal atrial fibrillation

Echocardiographic and additional clinical data of 305 patients without diagnosis of AF or atrial flutter (50.2% males, 49.8% females) who were interested in undergoing a screening investigation for AF were included in this study between May 2016 and November 2019 at the Department of Cardiology and the Department of Neurology of the University Hospital of Heidelberg (Germany), and four cardiology practices in Germany (Heidelberg, Lüneburg, Essen). Patient data were collected in a de-identified manner. Based on the observed prevalence of pAF in our echocardiographic laboratory, we estimated that about 300 subjects were necessary to estimate sensitivity values with confidence intervals of ± 10% [18, 19]. The study protocol was approved by the ethics committee of the University of Heidelberg (Germany, Medical Faculty Heidelberg, S-491/2015). Written informed consent was obtained from all patients, and the study was conducted in accordance with the Declaration of Helsinki.

Collected score parameters consisted of medical history parameters (age, smoker, heart frequency, sleep apnea, hyperlipidemia, type 2 diabetes mellitus, catheter ablation), medication (beta-blocker), and echocardiographic parameters (aortic root diameter, left atrial diameter, left ventricular end-systolic diameter, TDI A’ velocity). Catheter interventions were ablations of accessory pathways or atrioventricular nodal reentry tachycardia. Patients with a history of atrial fibrillation or atrial flutter ablation were excluded.

Transthoracic echocardiography examinations were performed on commercially available ultrasound systems (GE Healthcare, Philips, Sony). Images included parasternal, apical and subxiphoidal views using 1.5–4.0 MHz phase-array transducers. All examinations were performed with 2D echocardiography for anatomic imaging and Doppler echocardiography for assessment of velocities. Left atrial size was determined as the maximal distance between the posterior aortic root wall and the posterior left atrial wall at the end of the systole. Aortic root and left ventricular end-systolic diameters (LV, ESD) were obtained in the parasternal long axis view. The TDI A’ velocity was measured in the apical four-chamber view. Patients carried 3-lead 7-day Holter ECG devices (Mortara Instruments) for up to 3 weeks. A few patients carried the device for 4 weeks. ECG recordings were evaluated by a cardiologist blinded to the score parameters and the calculated score values. AF was diagnosed in case AF episodes longer than 20 s were documented.

Continuous variables between SR and pAF groups were compared using one-way analysis of variance (ANOVA). Standard deviations are indicated by plus–minus signs (Table 1). Categorical variables between groups were compared with two-tailed Fisher exact test. Predictive scores were previously derived from logistic regression models calibrated with 47 parameters recorded in 1000 patients of the Department of Cardiology at Heidelberg University (Heidelberg, Germany) [17]. Of these 47 parameters, the most predictive 12 parameters were identified by sequential feature selection and likelihood-ratio testing. Scores were scaled between 0 and 100. The 12-parameter score is given by Eq. (1), and the 4-parameter score by Eq. (2):

In Eqs. (1) and (2), variables (age; Ao,root, aortic root diameter; LA, LA diameter; LV,ESD, LV end-systolic diameter; TDI,Aʹ, tissue Doppler imaging, late diastolic velocity of mitral annulus; HF, heart frequency) are divided by their units (y, years; mm, millimetres; cm/s, centimetres per second; 1/min, per minute). Categorical variables (sleep apnea; hyperlipidemia; type II diabetes; smoker; ß-blocker, ß-blocker intake; catheter ablation, status after catheter ablation) are set to 1 or 0 in case of their presence or absence. Using the 12-parameter score, presence of pAF was predicted in case of \(L_{12} \ge 58.35\), and using the 4-parameter score, pAF was predicted in case of \(L_{4} \ge 63.32\). An online calculator was created to simplify application of the scores [20].

For comparison, we assessed the predictive performance of logistic regression models containing only subsets of variables that are part of the 12-parameter score. We tested the following reduced models: (1) a model, reduced by all echocardiographic parameters, with parameters age, heart frequency, sleep apnea, hyperlipidemia, type II diabetes, smoker, beta-blocker, catheter ablation, (2) a model containing the parameters age, gender, BMI, and (3) a model containing only the parameter age. Parameters of these reduced models were obtained from calibration of logistic regression models based on the dataset that was previously used to establish the 4-parameter and 12-parameter scores [17]. ROC curves of these models were obtained as previously described using 100-fold cross-validation [17]. We further compared the predictive performance of our scores with two previous prediction scores, the HAVOC and the ACTEL scores [21, 22]. These two scores were developed to predict AF in patients with cryptogenic stroke or transient ischemic attack (TIA) and use non-invasive clinical parameters as well. In these scores, 95% confidence intervals were estimated by bootstrapping with n = 1000 samples.

Patients were included in the study at the Department of Cardiology and the Department of Neurology at Heidelberg University as well as four cardiology practices (age between 18 and 93 years, median age: 63, 50.2% males, 49.8% females; Supplementary Fig. S1). Patients undergoing an ambulatory echocardiographic investigation as well as healthy volunteers were offered participation in the study (Fig. 1). Patients were asked to wear Holter ECG devices for up to 3 weeks. Patients carried Holter ECG devices on average for 12.4 days (newly diagnosed pAF patients: 12.1 ± 7.8 days; SR patients: 12.4 ± 7.0 days; Supplementary Fig. S2). Table 1 gives an overview of the score parameters in SR and pAF patients. In addition, we compared measurements of a selection of biomarkers that were available for our patient collective (GFR, creatinine, Troponin-T hs, NT-proBNP). These parameters were previously associated with an increased risk for AF [11, 23, 24]. In our study sample, values of these biomarkers did, however, not significantly differ between SR and AF patients (Supplementary Table S1).

Patients with newly diagnosed pAF had significantly higher age, larger aortic root and left atrial diameters, smaller end-systolic diameters, were less frequently smokers and had more frequently undergone a catheter ablation (accessory pathway or atrioventricular nodal reentry tachycardia ablation). Based on our previous study, threshold values of the developed 12-parameter (\(L_{12} \ge 58.35\)) and 4-parameter (\(L_{4} \ge 63.32\)) ECHO-AF scores that allowed prediction of pAF with 80% sensitivity were selected [17]. These score threshold values were applied in this study to prospectively evaluate the predictive performance of the developed predictive scores.

In this study, the 12-parameter and the 4-parameter scores showed sensitivities of 100% and 82% (95%-CI 65%, 93%), specificities of 75% (95%-CI 70%, 80%) and 67% (95%-CI: 61%, 73%), and areas under the receiver operating characteristic (ROC) curves of 0.84 (95%-CI: 0.80, 0.88) and 0.81 (95%-CI 0.74, 0.87) (Fig. 2). No confidence interval can be reported for the sensitivity of the 12-parameter score because all 34 pAF patients of a total of 305 patients were detected by its application, resulting a sensitivity of 100%. Values for sensitivity, specificity, and area under the ROC curve of 12-parameter and 4-parameter scores were higher than expected from the results of our previous retrospective study (Table 2) [17].

The precision of the 12-parameter score, defined as the number of true positives divided by the sum of true and false positives, was 34% (95%-CI 25%, 44%). This value is comparable to the precision value of 36% that was previously estimated based on a retrospective dataset [17]. The 4-parameter score showed a precision of 24% (95%-CI 17%, 32%).

Taken together, the capabilities of the developed 12-parameter and 4-parameter ECHO-AF scores for pAF prediction could be validated indicating that the developed model scores represent a simple, highly sensitive and non-invasive tool for detecting pAF.

To put these results into perspective and assess the predictive value of echocardiographic imaging parameters, we retrospectively tested reduced and modified model variants based on the dataset that was originally used to derive the 12- and 4-parameter scores [17]. Excluding echocardiographic imaging parameters clearly reduced the predictive performance in the original dataset, comprising data from 1000 patients, indicated by an AUC value for the ROC curve of 0.70 (95%-CI 0.65, 0.75) compared to an AUC value of 0.80 (95%-CI 0.76, 0.84) for the full 12-parameter score (Fig. 3a). Other previous screening studies pointed out the predictive value of the parameters age, gender and BMI or were only based on the subject age [23, 25]. Accordingly, we tested logistic regression models reduced to these parameters. Whereas the model based on age, gender and BMI showed an AUC value under the ROC curve of 0.65 (95%-CI 0.60, 0.69; Fig. 3b), classification only based on age resulted in an AUC value of 0.64 (95%-CI 0.59, 0.69; Fig. 3c). For comparison, we further applied two previous scores (HAVOC and ACTEL) originally developed for detecting AF in patients after stroke or TIA that also resulted in ROC curves with comparably small AUC values (0.66 and 0.64; Supplementary Fig. S3A and B) [21, 22]. It was concluded that the predictive performance of the 12- and 4-parameter scores relies on echocardiographic parameters and that they should not be replaced by reduced variants.

To evaluate pAF score models, we collected extensive 3-lead Holter ECG recordings that are informative about characteristics of AF episodes in pAF patients. We characterized properties of AF episodes and analysed Holter ECG carrying durations that were necessary to detect pAF. In 25 of a total of 34 patients with AF, up to 3 weeks of time-resolved Holter ECG data were available (Fig. 4). In other pAF patients, pAF was detected by pacemaker devices or 12-lead ECG recorded after including a patient in the study. In several patients, AF was detected in the second or third week. Profiles of AF episodes strongly differed between AF patients. In 9 patients, only episodes of less than one hour were observed. In three patients, AF was persistent. We did not observe circadian rhythmicity of AF episode frequency (Supplementary Fig. S4). Score values were not significantly correlated with AF burden (Supplementary Fig. S5).

We evaluated how long ECG monitoring was necessary to detect AF in patients in case of our study. To this end, we extracted ECG monitoring times before the first AF episode and calculated cumulative fractions of AF diagnoses dependent on ECG monitoring times (Fig. 5a). Half of the AF patients were diagnosed within 1 day of ECG recording, whereas, 75% were diagnosed within 4.5 days ECG recording (dashed red lines in Fig. 5a). All pAF patients were diagnosed within 14.4 days of ECG monitoring. Figure 5b visualizes AF episode durations and heart rates during AF episodes for different patients. Interestingly, for patients with non-permanent AF (n = 22), heart rate in the presence of AF was weakly positively correlated with the AF episode duration (Spearman rank-order correlation ρ = 0.24, p = 0.014).