Skip to main content
main-content

01.12.2016 | Research | Ausgabe 1/2016 Open Access

Thrombosis Journal 1/2016

External validation of prognostic rules for early post-pulmonary embolism mortality: assessment of a claims-based and three clinical-based approaches

Zeitschrift:
Thrombosis Journal > Ausgabe 1/2016
Autoren:
Erin R. Weeda, Christine G. Kohn, Gregory J. Fermann, W. Frank Peacock, Christopher Tanner, Daniel McGrath, Concetta Crivera, Jeff R. Schein, Craig I. Coleman
Wichtige Hinweise

Electronic supplementary material

The online version of this article (doi:10.​1186/​s12959-016-0081-5) contains supplementary material, which is available to authorized users.

Competing interest

The authors declare that they have no competing interests.

Authors’ contributions

Study concept and design: ERW, CGK, GJF, WFP, CC, JRS, CIC. Acquisition of data: ERW, CGK, CT, DM, CIC. Analysis and interpretation of data: ERW, CGK, GJF, WFP, CT, DM, CC, JRS, CIC. Drafting the manuscript: ERW, CGK, CT, DM, CIC. Critical Revision of the manuscript for important intellectual content: ERW, CGK, GJF, WFP, CT, DM, CC, JRS, CIC. Administrative, technical or material support: ERW, CGK, CIC. Study supervision: CIC. CIC had full access to all the study data and take full responsibility for the integrity of the data and the accuracy of the data analysis. All authors read and approved the final manuscript. The authors meet criteria for authorship as recommended by the International Committee of Medical Journal Editors (ICJME) and were fully responsible for all content and editorial decisions, and were involved in all stages of manuscript development. All authors read and approved the final manuscript.
Abbreviation
CI
Confidence intervals
CTPA
Computed tomography pulmonary angiography
EHR
Electronic health record
ICD-9-CM
International classification of diseases, ninth revision, clinical modification
IMPACT
In-hospital mortality for pulmonary embolism using claims daTa
NPV
Negative predictive value
O2
Oxygen
PE
Pulmonary embolism
PESI
Pulmonary embolism severity index
PPV
Positive predictive value
sPESI
Simplified pulmonary embolism severity index
SSDI
Social security death index
TRIPOD
Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis
USA
United States of America
V/Q
Ventilation/perfusion

Background

Guidelines suggest that patients with pulmonary embolism (PE) who are identified to have a low-risk of early post-PE all-cause mortality may be candidates for abbreviated hospital admission or outpatient treatment if appropriate follow-up can be arranged [ 1, 2]. Data from randomized trials and observational studies suggest that early discharge or outpatient treatment of low-risk PE patients is feasible and safe [ 3, 4].
A prior meta-analysis suggested at least one-third of acute PE patients could be classified as low-risk for early mortality according to clinical prediction rules [ 5]. This same meta-analysis identified the Pulmonary Embolism Severity Index (PESI), simplified PESI (sPESI) and Hestia clinical prediction rules as having high sensitivities and negative predictive values (NPVs) for identifying low-risk PE patients. In order to use PESI [ 6], sPESI [ 7] and Hestia [ 8] in the risk stratification of a patient with PE, access to vital signs, laboratory values, comorbid conditions and a cognitive evaluation at presentation is necessary. While PESI, sPESI and Hestia can be helpful in clinical practice, the extensive clinical data required to score these rules are not commonly found in claims databases or easily accessible to individual hospitals/health-systems. As a result, the utility of PESI, sPESI or Hestia for retrospective identification of low-risk patients with PE is limited.
While not originally derived to aid in prognostication in a prospective clinical setting, the In-hospital Mortality for Pulmonary embolism using Claims daTa (IMPACT) multivariable prediction rule utilizes data accessible within claims databases and/or claims from individual hospitals to retrospectively risk stratify patients with PE for early mortality [ 9]. Prior validation studies suggest IMPACT can accurately identify pulmonary embolism (PE) patients at low-risk of early mortality [ 10, 11]. The ability of IMPACT to correctly identify patients at low- and higher-risk of early mortality has not previously been compared to analogous clinical prediction rules. Therefore, using data from a single center, this study sought to externally validate IMPACT, PESI, sPESI and Hestia for predicting in-hospital and 30 day post-PE mortality.

Methods

Preparation of this study report was in accordance with the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement [ 12]. For this retrospective cohort study, we identified consecutive patients diagnosed with acute PE between October 21, 2010 and May 12, 2015 using computerized claims records for admissions to Hartford Hospital (Hartford, Connecticut, USA). The hospital’s computerized claims system contains information on source of admission, International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes, admission and discharge dates and discharge status. To be eligible for inclusion into this study, patients ≥18 years of age presenting to our institution had to have a primary diagnosis of PE (ICD-9-CM code = 415.1x). Consistent with prior studies, we excluded patients lacking objective confirmation of acute PE according to clinical guidelines. The following were considered confirmatory studies for the diagnosis of acute PE: high probability perfusion-ventilation lung scan (V/Q scan), computed tomography pulmonary angiography (CTPA) or pulmonary angiography diagnostic for PE, or a non-diagnostic V/Q scan or CTPA in combination with an abnormal compression ultrasonography of the lower extremities. Consistent with many prior studies of PE clinical prediction rules [ 5], subjects that received thrombolysis and/or pulmonary embolectomy within the 48 h of presentation were excluded as clinical guidelines do not consider such patients low-risk [ 1, 2]. All patients included in this study were managed according to usual clinical practice for our institution.
Risk stratification of patients with acute PE using IMPACT, PESI, sPESI and Hestia was performed according to published methods (Additional file 1) [ 69]. Patients with an IMPACT predicted mortality risk ≤1.5 % [ 9], PESI score ≤85 [ 6] or a sPESI or Hestia scores =0 [ 7, 8] were classified as low-risk for early mortality. Estimated mortality risk according to the claims-based IMPACT prediction rule [estimated % absolute risk = 1/(1 + exp(-x); where x = −5.833 + (0.026*age) + (0.402*myocardial infarction) + (0.368*chronic lung disease) + (0.464*stroke) + (0.638*prior major bleeding) + (0.298*atrial fibrillation) + (1.061*cognitive impairment) + (0.554*heart failure) + (0.364*renal failure) + (0.484*liver disease) + (0.523*coagulopathy) + (1.068*cancer)] was determined using all available hospital claims data (i.e., ICD-9-CM diagnosis and procedural codes) for each patient’s index PE encounter along with their age at time of presentation. ICD-9-CM coding for relevant IMPACT co-morbidities were performed according to the original IMPACT derivation paper [ 9]. Data necessary to classify patients as low- or high-risk of early mortality according to the PESI, sPESI and Hestia clinical prediction rules [ 68] were obtained by linking all included patients identified through hospital claims to the hospital’s electronic health record (EHR). We used vital signs (heart rate, blood pressure, respiratory rate, oxygen [O2] saturation, body temperature), laboratory values (serum creatinine, platelet count, total bilirubin) and mental status assessments obtained as close to the time of presentation for the index PE encounter as possible to score each of the clinical prediction rules [ 6]. For all patients admitted through the emergency department, the first vital sign, laboratory value and/or mental status assessment upon presentation (but within 24 h) was utilized. For patients directly admitted to the hospital we used the first values recorded on the day of hospital admission. Consistent with previous studies of this type, missing vital, laboratory and mental status assessment data were assumed to be normal [ 6]. For PESI, sPESI and Hestia, the presence of cancer, heart failure, chronic lung disease, severe liver disease (defined as a total bilirubin ≥2.5 mg/dL), heparin-induced thrombocytopenia and recent clinical events (gastrointestinal bleeding within 14 days, stroke with 4 weeks, surgery with 2 weeks) were assessed at time of hospital admission for the index PE encounter. All required data was abstracted from the electronic health record (including vital signs, laboratory values and emergency department, admission and consult notes) by trained study personnel blinded to study outcome.
All-cause in-hospital and 30 day post-PE mortality served as a priori endpoints for this study. In-hospital mortality was determined using the discharge status field for the index admission within the hospital billing system and electronic health record. Thirty-day mortality was based upon searches of the Social Security Death Index (SSDI) [ 11] performed >6 months after the last day of eligible inclusion in this analysis. Computerized health-system encounter data from our hospital’s billing records were queried for subsequent emergency visit claims, clinic visits and/or hospital readmissions outside of 30 days. These were used as confirmatory markers of vital status at 30 days. Beginning in March 2014, rules regarding access to data within the SSDI changed; restricting the release of three most recent years of data [ 13, 14]. For this reason, our 30 day mortality endpoint was assessed only in the subset of consecutive patients presenting to the hospital prior to January 31, 2014.
Baseline characteristics are described with means ± SDs for continuous data and counts and proportions for categorical data. Sensitivity, specificity and negative and positive predictive values for predicting early mortality were calculated for IMPACT, PESI, sPESI and Hestia along with 95 % confidence intervals (CIs). C-statistics were computed to evaluate each rule’s overall discriminative power. All database management and statistical analyses were performed using IBM SPSS Statistics version 22.0 (IBM Corp., Armonk, NY, USA). The study was approved by the Hartford Hospital institutional review board.

Results

A total of 861 patients with a primary ICD-9-CM code for acute PE and objective confirmation of diagnosis were identified (Fig.  1). Of these, 54 received thrombolytic therapy/embolectomy within 48 h of presentation, leaving 807 for analysis. Baseline characteristics of the cohort, stratified by low- and high-risk for each rule are shown in Table  1. Two-hundred and thirty-four patients presented with PE after January 2014 and were excluded from the 30 day mortality endpoint analysis, as SSDI data are not available for patients past this time point. The baseline characteristics of the 30 day mortality analysis patient subset were similar to the overall population (Additional file 1: Table S1).
Table 1
Characteristics of pulmonary embolism patients
Characteristic
Total Cohort a
IMPACT Low-Risk,
IMPACT Higher-Risk
PESI Low-Risk,
PESI Higher-Risk,
sPESI Low-Risk,
sPESI Higher-Risk
Hestia Low-Risk,
Hestia Higher-Risk,
N (%)
N (%)
N (%)
N (%)
N (%)
N (%)
N (%)
N (%)
N (%)
N = 807
N = 230
N = 577
N = 309
N = 498
N = 250
N = 557
N = 211
N = 596
Age (years, mean ± SD)
64.1 ± 16.47
46.4 ± 11.36
71.2 ± 12.38
51.9 ± 14.39
71.6 ± 12.76
56.3 ± 14.49
67.6 ± 16.12
59.9 ± 17.10
65.6 ± 16.00
  > 80 years
145 (18.0)
0 (0)
145 (25.1)
9 (2.9)
136 (27.3)
0 (0)
145 (26.0)
27 (12.8)
118 (19.7)
Male gender
372 (46.1)
114 (49.6)
258 (44.7)
135 (43.7)
237 (47.6)
140 (56.0)
232 (41.7)
99 (46.9)
273 (45.8)
Cancer
254 (31.5)
18 (7.8)
236 (40.9)
15 (4.9)
239 (48.0)
0 (0)
254 (45.5)
51 (24.2)
203 (34.1)
Cancer (ICD-9-CM)
154 (19.1)
0 (0)
154 (26.7)
10 (3.2)
144 (28.9)
3 (1.2)
151 (27.1)
25 (11.8)
129 (21.6)
Chronic cardiopulmonary disease
230 (28.5)
29 (12.6)
201 (34.3)
42 (13.6)
188 (37.8)
0 (0)
230 (41.3)
37 (17.5)
193 (32.4)
 Chronic lung disease
198 (24.5)
27 (11.7)
171 (29.6)
40 (12.9)
158 (31.7)
0 (0)
198 (35.5)
34 (16.1)
164 (27.5)
 Chronic lung disease (ICD-9-CM)
224 (27.8)
22 (9.6)
202 (35.0)
47 (15.2)
177 (35.5)
7 (2.8)
217 (39.0)
36 (17.1)
188 (31.5)
 Heart failure
62 (7.7)
2 (0.9)
60 (10.3)
3 (1.0)
59 (11.8)
0 (0)
62 (11.1)
4 (1.9)
58 (9.7)
 Heart failure (ICD-9-CM)
75 (9.3)
1 (0.4)
74 (12.8)
9 (2.9)
66 (13.3)
5 (2.0)
70 (12.6)
8 (3.8)
67 (11.2)
Altered mental status at presentation
42 (5.2)
3 (1.3)
39 (6.8)
0 (0)
42 (8.4)
6 (2.4)
36 (6.5)
4 (1.9)
38 (6.4)
Cognitive impairment (ICD-9-CM)
60 (7.4)
0 (0)
60 (10.4)
7 (2.3)
53 (10.6)
6 (2.4)
54 (9.7)
7 (3.3)
53 (8.9)
Pulse (beats/min, mean ± SD)
93.2 ± 18.92
94.0 ± 18.51
92.9 ± 19.10
91.6 ± 17.35
94.1 ± 19.78
85.6 ± 13.38
96.6 ± 20.04
87.6 ± 17.73
95.2 ± 18.96
 Pulse ≥ 110 beats/min
169 (20.9)
56 (24.3)
113 (19.6)
47 (15.2)
122 (24.5)
0 (0)
169 (30.3)
29 (13.7)
140 (23.5)
Systolic blood pressure (mmHg, mean ± SD)
133.6 ± 22.90
134.4 ± 21.75
133.3 ± 23.35
136.4 ± 21.32
131.9 ± 23.69
136.8 ± 21.60
132.2 ± 23.33
139.8 ± 22.30
131.4 ± 22.70
 Systolic blood pressure <100 mmHg
36 (4.5)
7 (3.0)
29 (5.0)
3 (1.0)
33 (6.6)
0 (0)
36 (6.5)
3 (1.4)
33 (5.5)
O2 saturation (%, mean ± SD)
96.3 ± 3.50
96.8 ± 3.83
96.1 ± 3.34
96.3 ± 2.42
95.9 ± 3.97
97.0 ± 2.35
96.0 ± 3.87
97.1 ± 2.12
96.0 ± 3.83
 O2 saturation <90 %
32 (4.0)
6 (2.6)
26 (4.5)
1 (0.3)
31 (6.2)
0 (0)
32 (5.7)
0 (0)
32 (5.4)
 Oxygen needed to maintain O2 saturation >90 % for >24 h
412 (51.1)
79 (34.3)
333 (57.7)
112 (36.2)
300 (60.2)
90 (36.0)
322 (57.8)
0 (0)
412 (69.1)
Respiratory rate (breaths/min, mean ± SD)
19.1 ± 3.56
18.7 ± 3.31
19.3 ± 3.64
18.5 ± 2.20
19.5 ± 4.14
18.5 ± 2.51
19.4 ± 3.90
18.2 ± 2.37
19.5 ± 3.84
 Respiratory rate ≥30 breaths/min
19 (2.4)
2 (0.8)
17 (2.9)
0 (0)
19 (3.8)
2 (0.8)
17 (3.1)
2 (0.9)
17 (2.9)
Temperature (degrees Celsius, mean ± SD)
97.6 ± 1.38
97.8 ± 1.29
97.6 ± 1.41
97.9 ± 1.38
97.4 ± 1.47
97.6 ± 1.28
97.6 ± 1.40
97.4 ± 1.24
97.7 ± 1.42
 Temperature <36° Celsius
206 (25.5)
47 (20.4)
159 (27.6)
40 (12.9)
166 (33.3)
68 (27.2)
138 (24.8)
60 (28.4)
146 (24.5)
Thrombolysis or embolectomy > 48 h
4 (0.5)
2 (0.9)
2 (0.3)
3 (1.0)
1 (0.2)
3 (1.2)
1 (0.2)
0 (0)
4 (0.7)
High risk of bleeding b
101 (12.5)
37 (16.1)
64 (11.1)
48 (15.5)
53 (10.6)
37 (14.8)
64 (11.5)
0 (0)
101 (16.9)
PE on anticoagulation
62 (7.7)
17 (7.4)
45 (7.8)
21 (6.7)
41 (8.2)
11 (4.4)
51 (9.2)
0 (0)
62 (10.4)
History of heparin-induced thrombocytopenia
5 (0.6)
1 (0.4)
4 (0.7)
2 (0.6)
3 (0.6)
2 (0.8)
3 (0.5)
0 (0)
5 (0.8)
Medical or social reason for admission c
237 (29.3)
43 (18.7)
194 (33.6)
64 (20.7)
173 (34.7)
42 (16.9)
195 (34.9)
0 (0)
237 (39.8)
Need for intravenous pain medication for > 24 h
87 (10.8)
43 (18.7)
44 (7.6)
48 (15.5)
39 (7.8)
32 (12.8)
55 (9.9)
0 (0)
87 (14.6)
Severe liver impairment d
10 (1.2)
3 (1.3)
7 (1.2)
3 (1.0)
7 (1.4)
3 (1.2)
7 (1.3)
0 (0)
10 (1.7)
Liver disease (ICD-9-CM)
6 (0.7)
0 (0)
6 (1.0)
3 (1.0)
3 (0.6)
2 (0.8)
4 (0.7)
0 (0)
6 (1.0)
Creatinine clearance <30 mL/min
31 (3.8)
2 (0.9)
29 (5.0)
11 (3.6)
20 (4.0)
7 (2.8)
24 (4.3)
0 (0)
31 (5.2)
Renal failure (ICD-9-CM)
60 (7.4)
2 (0.9)
58 (10.1)
18 (5.8)
42 (8.4)
16 (6.4)
44 (7.9)
8 (3.8)
52 (8.7)
Hemodynamically unstable e
94 (11.6)
24 (10.4)
70 (12.1)
30 (9.7)
64 (12.9)
17 (6.8)
77 (13.8)
0 (0)
94 (15.8)
Myocardial infarction (ICD-9-CM)
40 (5.0)
4 (1.7)
36 (6.2)
15 (4.9)
25 (5.0)
16 (6.4)
24 (4.3)
2 (0.9)
38 (6.4)
Cerebrovascular disease (ICD-9-CM)
11 (1.4)
0 (0)
11 (1.9)
5 (1.6)
6 (1.2)
4 (1.6)
7 (1.3)
2 (0.9)
9 (1.5)
Prior major bleeding (ICD-9-CM)
28 (3.5)
1 (0.4)
27 (4.7)
8 (2.6)
20 (4.0)
5 (2.0)
23 (4.1)
3 (1.4)
25 (4.2)
Atrial fibrillation (ICD-9-CM)
87 (10.8)
2 (0.9)
85 (14.7)
15 (4.9)
72 (14.5)
11 (4.4)
76 (13.6)
18 (8.5)
69 (11.6)
Coagulopathy (ICD-9-CM)
41 (5.1)
3 (1.3)
38 (6.6)
11 (3.6)
30 (6.0)
12 (4.8)
29 (5.2)
6 (2.8)
35 (5.9)
aOf the 807 patients, 3 (0.4 %) patients had unknown values for respiratory rate; 2 (0.2 %) for pulse, systolic blood pressure, O2 saturation, and temperature; 516 (63.9 %) for bilirubin (component of liver disease); 1 (0.1 %) for platelets (component of bleed risk); and 1 (0.1 %) for glomerular filtration rate (creatinine clearance estimate)
Hrs hours, ICD-9-CM International Classification of Diseases-Ninth Revision-Clinical Modification, min minutes, SD standard deviation, O2 oxygen
bGastrointestinal bleeding in the preceding 14 days, stroke in the preceding 4 weeks, procedure in the preceding 2 weeks, bleeding disorder or thrombocytopenia (platelet count < 75 × 109/L), or uncontrolled hypertension (systolic blood pressure > 180 mmHg or diastolic blood pressure > 110 mmHg)
cMedical or social reason for hospital treatment was determined by trained study personnel using all data available in the electronic health record including vital signs, laboratory values, and emergency department, admission and consult notes
dCirrhosis or bilirubin > 2.5 mg/dL
ePulse ≥ 100 beats/minute and systolic blood pressure ≤ 100 mmHg or condition requiring admission to an intensive care unit
The IMPACT, PESI, sPESI and Hestia scores (mean ± SD) for the complete cohort of 807 PE patients were 3.9 ± 4.3, 96.4 ± 33.3, 1.1 ± 0.9 and 1.3 ± 1.1, respectively. While mean age for patients was 64.1 ± 16.57; when dichotomized into risk groups, high-risk patients were considerably older, regardless of prediction rule used. At time of presentation, mean vital sign values were within normal ranges in the overall analysis population; but more than half of the subjects required O2 supplementation to maintain saturations >90 %. The use of thrombolysis and/or embolectomy after 48 h was infrequent, occurring in only 4 (0.5 %) of patients.
The overall incidence of all-cause in-hospital mortality was 2.6 % (21/807). IMPACT, PESI, sPESI and Hestia classified 26 % (Hestia)-38 % (PESI) of the cohort as low-risk for early post-PE mortality. Fatality among low-risk patients was low (0–0.6 %), corresponding to NPVs of 99.4 (PESI)-100 % (sPESI and Hestia) (Table  2). IMPACT’s sensitivity was 95.2 % (95 % CI = 74.1–99.8 %), and the sensitivities of clinical rules ranged from 91 (PESI)-100 % (sPESI and Hestia). Specificities of all rules ranged between 26.8 (Hestia)-39.1 % (PESI) and C-statistics from 0.76 (sPESI)-0.86 (Hestia). Additional file 1: Table S2 describes the characteristics of patients who died in-hospital and had discordant risk categorization across any of the four prediction rules.
Table 2
Prognostic test characteristics for in-hospital mortality
 
IMPACT
PESI
sPESI
Hestia
Low-Risk Mortality
1/230
2/309
0/250
0/211
n/N (%)
(0.4 %)
(0.6 %)
(0 %)
(0 %)
High-Risk Mortality
20/577
19/498
21/557
21/596
n/N (%)
(3.5 %)
(3.8 %)
(3.8 %)
(3.5 %)
Sensitivity
95.2 %
90.5 %
100 %
100 %
(95 % CI)
(74.1–99.8 %)
(68.2–98.3 %)
(80.8–100 %)
(80.8–100 %)
Specificity
29.1 %
39.1 %
31.8 %
26.8 %
(95 % CI)
(26.0–32.5 %)
(35.6–42.6 %)
(28.6–35.2 %)
(23.8–30.1 %)
PPV
3.5 %
3.8 %
3.8 %
3.5 %
(95 % CI)
(2.2–5.4 %)
(2.4–6.0 %)
(2.4–5.8 %)
(2.2–5.4 %)
NPV
99.6 %
99.4 %
100 %
100 %
(95 % CI)
(97.2–100 %)
(97.4–99.9 %)
(98.1–100 %)
(97.8–100 %)
C-statistic
0.766
0.792
0.762
0.857
(95 % CI)
(0.685–0.848)
(0.696–0.889)
(0.682–0.842)
(0.796–0.918)
CI confidence interval, IMPACT In-hospital Mortality for Pulmonary embolism using Claims data, NPV negative predictive value, PESI Pulmonary Embolism Severity Index, PPV positive predictive value, sPESI simplified Pulmonary Embolism Severity Index
Among the subset of 573 patients accessible within the SSDI, 33 (5.8 %) died of any cause within 30 days of presentation for PE. All rules classified 27.9 (Hestia)-38.0 % (PESI) of PE patients as low-risk, and fatality occurred in 0.0 % (Hestia)-1.4 % (PESI) of low-risk patients (NPVs = 98.6–100 %) (Table  3). IMPACT’s sensitivity for predicting 30 day mortality was 97.0 % (95%CI = 82.5–99.8 %), while sensitivities for clinical rules ranged from 91 (PESI)-100 % (Hestia). Specificities of rules ranged between 29.6 % (Hestia)-39.8 % (PESI) and their C statistics ranged from 0.73 (sPESI)-0.81 (PESI). Additional file 1: Table S3 describes the characteristics of patients who died within 30 days of presentation and had discordant risk categorization across any of the 4 prediction rules.
Table 3
Prognostic test characteristics for 30 day mortality
 
IMPACT
PESI
sPESI
Hestia
Low-Risk Mortality
1/165
3/218
1/177
0/160
n/N (%)
(0.6 %)
(1.4 %)
(0.6 %)
(0 %)
High-Risk Mortality
32/408
30/355
32/396
33/413
n/N (%)
(7.8 %)
(8.5 %)
(8.1 %)
(8.0 %)
Sensitivity
97.0 %
90.9 %
97.0 %
100 %
(95 % CI)
(82.5–99.8 %)
(74.5–97.6 %)
(82.5–99.8 %)
(87.0–100 %)
Specificity
30.4 %
39.8 %
32.6 %
29.6 %
(95%CI)
(26.6–34.5 %)
(35.7–44.1 %)
(28.7–36.8 %)
(25.8–33.7 %)
PPV
7.8 %
8.5 %
8.1 %
8.0 %
(95 % CI)
(5.5–11.0 %)
(5.9–12.0 %)
(5.7–11.3 %)
(5.6–11.1 %)
NPV
99.4 %
98.6 %
99.4 %
100 %
(95 % CI)
(96.2–100 %)
(95.7–99.6 %)
(96.4–100 %)
(97.1–100 %)
C-statistic
0.804
0.805
0.731
0.791
(95 % CI)
(0.749–0.859)
(0.731–0.879)
(0.653–0.810)
(0.721–0.860)
CI confidence interval, IMPACT In-hospital Mortality for Pulmonary embolism using Claims data, NPV negative predictive value, PESI Pulmonary Embolism Severity Index, PPV positive predictive value, sPESI simplified Pulmonary Embolism Severity Index

Discussion

In this analysis, the claims-based IMPACT prediction rule displayed prognostic accuracy similar to that of commonly used clinical risk stratification rules, including PESI and sPESI (which have been prospectively validated for identification of low-risk PE patients) and the Hestia criteria (which was prospectively designed to identify patients whom could be treated as outpatients). The 4 rules evaluated in this study classified between 1/4th and 2/5ths of patients as low-risk. Each exhibited sensitivities >90 %, NPVs >98.6 % and specificities <40 % for predicting in-hospital or 30 day all-cause mortality, and these findings are consistent with prior derivation and validation studies. To our knowledge, this is the first external validation study of Hestia [ 5, 911]. Taken together, our results suggests each of the four rules can identify a cohort of low-risk PE patients whom are unlikely to die within the first 30 days of presentation. However, because a minority of patients with PE (<6 % in our study) die within 30 days, these prognostic rules classify a substantial number of patients who ultimately survive into higher-risk groups (hence their lower specificities). Prognostic tests seldom have both high sensitivity and specificity. Therefore, when using a prognostic test to decide upon implementing a less conservative treatment strategy (e.g., discharging a patient with acute PE directly from the emergency department) higher sensitivity and NPV values are preferable.
The American College of Chest Physicians and the European Society of Cardiology guidelines support early discharge and/or home treatment of PE patients at low-risk for early mortality as long as they have adequate home circumstances [ 1, 2]. These guidelines suggest that clinicians use validated clinical prediction rules to assist in identification and selection of low-risk patients. Although in this analysis IMPACT displayed a similar ability to identify low-risk PE patients as PESI, sPESI and Hestia; IMPACT was not originally derived or validated to assist in prospective clinical decision-making (and is relatively more computationally complex compared to clinical prediction rules), and it should not be used for individual patient decision-making [ 9]. However, IMPACT appears valid for retrospective identification of low-risk PE patients and therefore could be used to aid in the performance of real-world outcomes studies and to enable payer/institution benchmarking of rates of low-risk PE patients treated at home or following an abbreviated admission. Using claim s data as described by IMPACT may have advantages over obtaining highly granular clinical information from the EHR (including reduced time and effort requirements).
Our study has limitations that require consideration. First, this validation study was performed retrospectively and therefore may be subject to biases, particularly due to missing data. Nonetheless, our study had similar rates of missing data than reported in prior prospective derivation/external validation papers of clinical prediction rules [ 57]. Second, this was a single-center study limiting its generalizability and sample size. However, baseline characteristics and mortality rates were consistent with national estimates [ 10, 15] and our sample size (573–807 patients) was large relative to many previously published external validation studies of PE clinical prediction rules [ 5]. Third, we could not assess 30 day mortality in our entire study cohort due to restrictions on the availability of SSDI data [ 14]. Despite this, the sample size of patients with objectively confirmed PE available for final analysis was robust. Moreover, we are unaware of any programmatic changes in evaluation and treatment of patients with PE at our institution since February 2014. Consequently, the likelihood of selection bias resulting from the unavoidable exclusion of patients after this date is low. Next, the more subjective nature of certain criteria in Hestia (i.e., medical or social reason for hospital admission and the need for intravenous pain medication for >24 h), make retrospective scoring challenging. This being said, the proportion of patients in our study classified as higher-risk because of these “subjective” criteria was not inconsistent with the Hestia derivation study [ 8]. Lastly, the 48 h cut-off used to exclude patients undergoing thrombolysis and/or embolectomy is somewhat arbitrary. We excluded patients receiving thrombolysis and/or embolectomy in less than 48 h because such patients likely had hemodynamic instability at presentation and would not be considered low-risk per guidelines [ 1, 2]. Of note, numerous studies evaluating the prognostic accuracy of clinical prediction rules have similarly excluded patients undergoing early thrombolysis and/or embolectomy [ 5]. However, when these procedures are performed later in a hospital stay (day 3 onwards), they are more likely an indicator of a therapeutic failure resulting in a poor clinical course (i.e., respiratory failure or cardiac arrest). In addition, the need for and timing of thrombolysis and/or embolectomy can easily be detected in a claims database and a clinical setting, allowing it to be implemented in our evaluation of IMPACT and clinical prediction rules. The 48 h cut-off was chosen a priori to approximate the likely timing used for assessing the similar Hestia criterion (i.e., the Hestia study required discharge within 24 h of PE diagnosis, likely resulting in the assessment of the 11 Hestia criteria within 48 h of initial PE presentation) [ 8].

Conclusion

IMPACT identified low-risk PE patients with similar accuracy as PESI, sPESI and Hestia. While not designed for prospective clinical decision-making, IMPACT appears useful for identification of low-risk PE patient in retrospective claims-based studies.

Ethics approval and consent to participate

This study was approved by the Hartford Hospital institutional review board with a waiver of informed consent.

Consent for publication

Consent for publication: not applicable.

Acknowledgements

None.

Funding

This study was funded by Janssen Scientific Affairs, LLC, Raritan, NJ, USA
CIC has received grant funding and consultancy fees from Janssen Scientific Affairs, LLC; Bayer Pharma AG; and Boehringer-Ingelheim Pharmaceuticals, Inc. CC, JRS are employees of Janssen Scientific Affairs LLC. GJF has received grant funding for Novartis, Cardiorentis, Cardioxyl, Cempra Pharmaceuticals, Trevena, Intersection Medical, Siemens, The Mayday Foundation, Pfizer; and is on the advisory board and speakers bureau for Janssen Scientific Affairs. FWP has received grant funding and consultancy fees from Abbott, Alere, Banyan, Cardiorentis, Janssen Pharmaceuticals, Portola, Roche, The Medicine’s Company, Prevencio and Singulex.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://​creativecommons.​org/​licenses/​by/​4.​0/​), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://​creativecommons.​org/​publicdomain/​zero/​1.​0/​) applies to the data made available in this article, unless otherwise stated.

Competing interest

The authors declare that they have no competing interests.

Authors’ contributions

Study concept and design: ERW, CGK, GJF, WFP, CC, JRS, CIC. Acquisition of data: ERW, CGK, CT, DM, CIC. Analysis and interpretation of data: ERW, CGK, GJF, WFP, CT, DM, CC, JRS, CIC. Drafting the manuscript: ERW, CGK, CT, DM, CIC. Critical Revision of the manuscript for important intellectual content: ERW, CGK, GJF, WFP, CT, DM, CC, JRS, CIC. Administrative, technical or material support: ERW, CGK, CIC. Study supervision: CIC. CIC had full access to all the study data and take full responsibility for the integrity of the data and the accuracy of the data analysis. All authors read and approved the final manuscript. The authors meet criteria for authorship as recommended by the International Committee of Medical Journal Editors (ICJME) and were fully responsible for all content and editorial decisions, and were involved in all stages of manuscript development. All authors read and approved the final manuscript.
Zusatzmaterial
Literatur
Über diesen Artikel

Weitere Artikel der Ausgabe 1/2016

Thrombosis Journal 1/2016 Zur Ausgabe

Neu im Fachgebiet Innere Medizin

Mail Icon II Newsletter

Bestellen Sie unseren kostenlosen Newsletter Update Innere Medizin und bleiben Sie gut informiert – ganz bequem per eMail.

© Springer Medizin 

Bildnachweise