Introduction
Materials and methods
Search strategy
Inclusion criteria
Quality assessment
Missing data
Prognostic performance measures
Results
Search results
Study characteristics
Study design | Population | Models | Variables | Comparison | ||||
---|---|---|---|---|---|---|---|---|
Setting (Location)a | Study periodb | Nc/ICU Typed/Mortality%e | Model/Valid.f | SOFA Abstractionsg | Othersh | Standard Modeli | Mort.j | |
Toma et al (2008) [9] | 1 ICU (NL) | Jul 98 to Aug 05 | 2928/Mix/H = 24 | LR/Ind. | Seq of IOF1 | SAPS II | SAPS II | H |
Toma et al (2007) [8] | 1 ICU (NL) | Jul 98 to Aug 05 | 6276/Mix/H = 11 | LR/Ind. | Seq of SOFA2 | SAPS II | SAPS II | H |
Ho (2007) [4] | 1 multidisc ICU (AU) | Jan 05 to Dec 05 | 1311/Mix/H = 14.5 | LR/Ind. | TMS Adm Delta (TMS-Adm) | APACHE II | APACHE II | H |
Ho et al (2007) [11] | 1 multidisc ICU (AU) | Jan 05 to Dec 05 | 1311/Mix/H = 14.5 | No | TMS Adm Delta (TMS-Adm) | No | APACHE II, APS, RPH | H |
Holtfreter et al (2006) [12] | 1 ICU (DE) | 42 months | 933/Mix/H = 25/I = 23.9 | No | Adm | No | 16 variables, APACHE II | H |
Zygun et al (2005) [14] | 3 ICUs (CA) | May 00 to Apr 01 | 1436/Mix/H = 35.1/I = 27 | LR/NR | Adm TMS, Mean (ICU stay), Delta (TMS-Adm), Adm (i) | No | MODS | H/I |
Cabré et al (2005) [6] | 79 ICUs (75 ES, 4 L-Am) | Feb 01 to Mar 01 | 1324/Mix/H = 44.6/I = 37.3 | LR/NR | Min (MODS period), Max (MODS period), 5-day trend3 | Age | No | H |
Timsit et al (2002) [15] | 6 ICUs (FR) | 24 months | 1685/Mix/H = 30.3/I = 22.5 | LR/Ind.* | D1-7, D1-7 (mod) | No | LODS | H |
Pettilä et al (2002) [17] | 1 med-surg ICU (FI) | NR | 520/Mix/H = 30/I = 16.5 | No | Adm, D5, Max (5d), Delta (d5-d1), TMS | No | APACHE III, MODS, LODS | H |
Janssens et al (2000) [20] | 1 med ICU (DE) | Nov 97 to Feb 98 | 303/Med/H = 14.5/I = 6.3 | LR/NR | Adm, TMS, Delta (TMS-Adm) | No | No | H |
Khwannimit (2007) [10] | 1 ICU (TH) | Jul 04 to Mar 06 | 1782/Mix/H = 22/I = 16.4 | No | Adm | No | MODS, SOFA, LODS | I |
Rivera- Fernández et al (2007) [5] | 55 ICUs (EU) | 2 months in 97/98 | 6409/Mix/H = 20.6/I = 13.9 | LR/Ind. | Mean (ICU stay), Max(ICU stay) | SAPS II, diagnosis events | SAPS II | I |
Gosling et al (2006) [13] | 1 general ICU (UK) | Nov 02 to Oct 03 | 431/Mix/I = 20.9 | No | Adm SOFA | No | APACHE II, urine albumin and 5 other factors | I |
Kajdacsy- Balla Amaral et al (2005) [7] | 40 ICUs (1 AU, 35 EU, 1 N-Am, 3 S-Am) | 1 May 95 to 31 May 95 | 748 (6 countries)/Mix/I = 21.5 | LR/NR | Adm, TMS, Delta (48 h-Adm), Delta (TMS-Adm) | Different parameters | No | I |
Junger et al (2002) [18] | 1 operative ICU (DE) | Apr 99 to Mar 00 | 524/Surg/I = 12.4 | No | Max (ICU stay), TMS, Delta (TMS-Adm), Adm (mod) | No | No | I |
Ferreira et al (2001) [19] | 1 med-surg ICU (BE) | Apr 99 to Jul 99 | 352/Mix/I = 23 | No | Adm, 48 h, 96 h, Delta (48 h-Adm), Delta (96 h-Adm), Max (ICU stay), Mean (ICU stay), Total | No | No | I |
Moreno et al (1999) [21] | 40 ICUs (1 AU, 35 EU, 1 N-Am, 3 S-Am) | May 95 | 1449/Mix/H = 26/I = 22 | LR/NR | Adm, TMS, Delta (TMS-Adm), Adm (i) | No | No | I |
Bota et al (2002) [16] | 1 ICU (BE) | Apr to Jul99, Oct to Nov99, Jul to Sep00 | 949/Mix/U = 29.1 | No | Adm, 48 h, 96 h, Dis, Max (24 h), Adm (c), 48 h (c), 96 h (c), Dis (c), Max (c, 24 h) | No | APACHE II, MODS | U |
Missing data
Study quality
Study participation max 8 pts | Prognostic factor max 3 pts | Outcome measurement max 1 pt | Analysis max 8 pts | Total score max 20 pts | |
---|---|---|---|---|---|
Toma et al (2008) [9] | 8 | 3 | 1 | 7.5 | 19 |
Toma et al (2007) [8] | 8 | 2.5 | 1 | 8 | 19.5 |
Khwannimit (2007) [10] | 8 | 1 | 1 | 3.5 | 13.5 |
Ho (2007) [4] | 8 | 3 | 1 | 7 | 19 |
Ho et al (2007) [11] | 8 | 2 | 1 | 5 | 16 |
Rivera-Fernández et al (2007) [5] | 7 | 1 | 1 | 7.5 | 16.5 |
Holtfreter et al (2006) [12] | 8 | 1.5 | 1 | 5 | 15.5 |
Gosling et al (2006) [13] | 8 | 1.5 | 1 | 4 | 14.5 |
Zygun et al (2005) [14] | 8 | 2 | 1 | 5.5 | 16.5 |
Cabré et al (2005) [6] | 8 | 2 | 1 | 4 | 15 |
Kajdacsy-Balla Amaral et al (2005) [7] | 8 | 3 | 1 | 5 | 17 |
Timsit et al (2002) [15] | 8 | 2.5 | 1 | 7.5 | 19 |
Bota et al (2002) [16] | 7.5 | 1 | 0 | 3 | 11.5 |
Pettilä et al (2002) [17] | 8 | 1 | 1 | 7.5 | 17.5 |
Junger et al (2002) [18] | 7 | 2 | 1 | 3 | 13 |
Ferreira et al (2001) [19] | 8 | 2.5 | 1 | 3 | 14.5 |
Janssens et al (2000) [20] | 8 | 2 | 1 | 3.5 | 14.5 |
Moreno et al (1999) [21] | 8 | 2.5 | 1 | 3.5 | 15 |
Study results
Performance of single SOFA scores at a fixed time on and after admission
Admission SOFA | AUC | Brier | H/C-statistics | Compared with | AUC | Brier | H/C-statistics | Mort. |
---|---|---|---|---|---|---|---|---|
Ho et al (2007) [11] | 0.791 | 0.1 | C = 7.97 | APACHE II | 0.858 | 0.09 | H | |
p = 0.437 | APS | 0.829 | 0.09 | C = 2.9 p = 0.890 | H | |||
RPHICU | 0.822 | 0.09 | C = 4.7 p = 0.198 | H | ||||
Holtfreter et al (2006) [12] | 0.72 | APACHE II | 0.785 | H | ||||
Zygun et al (2005) [14] | 0.67 | U = 8.8 | MODS | 0.62 | H/C = 10.28 | H | ||
p = 0.38 | p = 0.17 | |||||||
Timsit et al (2002) [15] | 0.72 | U = 4.55 | LODS | 0.726 | H/C = 10.4 | H | ||
p = 0.8 | p = 0.16 | |||||||
Pettilä et al (2002) [17] | 0.776 | APACHE III | 0.825 | H | ||||
LODS | 0.805 | H | ||||||
MODS | 0.695 | H | ||||||
Khwannimit (2007) [10] | 0.8786 | LODS | 0.8802 | H | ||||
MODS | 0.8606 | |||||||
Gosling et al (2006) [13] | 0.61 | APACHE II | 0.62 | I | ||||
Zygun et al (2005) [14] | 0.67 | U = 11.66 | MODS | 0.63 | H/C = 14.29 | I | ||
p = 0.17 | p = 0.05 | |||||||
Moreno et al (1999) [21] | 0.772 | I | ||||||
Bota et al (2002) [16] | 0.872 | APACHE II | 0.88 | U | ||||
MODS | 0.856 | U | ||||||
Ferreira et al (2001) [19] | 0.79 | I | ||||||
Janssens et al (2000) [20] | 0.82 | SAPS II | 0.77 | H | ||||
Other scoring moments
|
AUC
|
Brier
|
H/C-statistics
|
Compared to
|
AUC
|
Brier
|
H/C-statistics
| Mort. |
Bota et al (2002) [16] 48 hours | 0.844 | MODS | 0.834 | U | ||||
Ferreira et al (2001) [19] 48 hours | 0.78 | I | ||||||
Bota et al (2002) [16] 96 hours | 0.847 | MODS | 0.861 | U | ||||
Ferreira et al (2001) [19] 96 hours | 0.82 | I | ||||||
Timsit et al (2002) [15], day 2 | 0.742 | U = 11.1 | LODS | 0.742 | H | |||
p = 0.2 | ||||||||
Timsit et al (2002) [15], day 3 | 0.762 | U = 9.94 | LODS | 0.762 | H | |||
p = 0.27 | ||||||||
Timsit et al (2002) [15], day 4 | 0.766 | U = 10.5 | LODS | 0.766 | H | |||
p = 0.23 | ||||||||
Timsit et al (2002) [15], day 5 | 0.746 | U = 13.6 | LODS | 0.746 | H | |||
p = 0.09 | ||||||||
Pettilä et al (2002) [17], day 5 | 0.727 | LODS | 0.76 | H | ||||
H | MODS | 0.744 | ||||||
Timsit et al (2002) [15], day 6 | 0.763 | U = 12.2 | LODS | 0.763 | H | |||
p = 0.14 | ||||||||
Timsit et al (2002) [15], day 7 | 0.746 | LODS | 0.764 | H | ||||
Bota et al (2002) [16], final | 0.897 | MODS | 0.869 | H |
Performance of sequential measurements of SOFA
Max SOFA | AUC | Brier | H/C-statistics | Comp. | AUC | H/C-statistics | Mort. |
---|---|---|---|---|---|---|---|
Pettilä et al (2002) [17], 5 days | 0.792 | LODS | 0.827 | H | |||
MODS | 0.795 | H | |||||
Junger et al (2002) [18], ICU stay | 0.922 | I | |||||
Bota et al (2002) [16], 24 hrs period | 0.898 | MODS | 0.9 | U | |||
Ferreira et al (2001) [19], ICU stay | 0.9 | I | |||||
Total Max SOFA
|
AUC
|
Brier
|
H/C-statistics
|
Comp
|
AUC
|
H/C-statistics
| Mort. |
Ho et al (2007) [11], ICU stay | 0.829 | 0.1 | C = 7.4 p = 0.496 | H | |||
Zygun et al (2005) [14], ICU stay | 0.7 | U = 9.2 p = 0.33 | MODS | 0.65 | 8.07 p = 0.43 | H | |
Pettilä et al (2002) [17], ICU stay | 0.816 | LODS | 0.839 | H | |||
MODS | 0.817 | H | |||||
Zygun et al (2005) [14], ICU stay | 0.69 | U = 7.30 p = 0.50 | MODS | 0.64 | 9.09 p = 0.33 | I | |
Kajdacsy-Balla Amaral et al (2005) [7], ICU stay | 0.84 | H: p = 0.95 C: p = 0.54 | I | ||||
Junger et al (2002) [18], ICU stay | 0.921 | I | |||||
Moreno et al (1999) [21], ICU stay | 0.847 | I | |||||
Janssens et al (2000) [20], ICU stay | 0.86 | H | |||||
Delta SOFA
|
AUC
|
Brier
|
H/C-statistics
|
Comp
|
AUC
|
H/C-statistics
| Mort. |
Ho et al (2007) [11], TMS – Adm | 0.635 | 0.12 | C = 20.2 p = 0.001 | H | |||
Zygun et al (2005) [14], TMS – Adm | 0.54 | U = 53.48 p < 0.01 | MODS | 0.55 | 31.2 p < 0.01 | H | |
Pettilä et al (2002) [17], day 5 – Adm | 0.6 | LODS | 0.633 | H | |||
MODS | 0.653 | H | |||||
Zygun et al (2005) [14], TMS – Adm | 0.51 | U = 98.01 p < 0.01 | MODS | 0.52 | 70.52 p < 0.01 | I | |
Junger et al (2002) [18], TMS – Adm | 0.828 | I | |||||
Moreno et al (1999) [21], TMS – Adm | 0.742 | I | |||||
Ferreira et al (2001) [19], 48 hrs – Adm | 0.69 | I | |||||
Ferreira et al (2001) [19], 96 hrs – Adm | 0.62 | I | |||||
Janssens et al (2000) [20], TMS – Adm | 0.62 | H | |||||
Mean SOFA
|
AUC
|
Brier
|
H/C-statistics
|
Comp
|
AUC
|
H/C-statistics
| Mort. |
Zygun et al (2005) [14], ICU stay | 0.77 | U = 22.66 p < 0.01 | MODS | 0.74 | 46.13 p < 0.01 | H | |
Zygun et al (2005) [14], ICU stay | 0.79 | U = 28.92 p < 0.01 | MODS | 0.75 | 42.72 p < 0.01 | I | |
Ferreira et al (2001) [19], ICU stay | 0.88 | I | |||||
Total SOFA
|
AUC
|
Brier
|
H/C-statistics
|
Comp
|
AUC
|
H/C-statistics
| Mort. |
Ferreira et al (2001) [19], ICU stay | 0.85 | I | |||||
Modified SOFA
|
AUC
|
Brier
|
H/C-statistics
|
Comp
|
AUC
|
H/C-statistics
| Mort. |
Timsit et al (2002) [15], Adm | 0.729 | U = 11 p = 0.2 | LODS | 0.733 | 11.3 p = 0.19 | H | |
Timsit et al (2002) [15], day 2 | 0.752 | U = 8.3 p = 0.4 | LODS | 0.748 | H | ||
Timsit et al (2002) [15], day 3 | 0.773 | U = 11.3 p = 0.19 | LODS | 0.761 | H | ||
Timsit et al (2002) [15], day 4 | 0.779 | U = 7.3 p = 0.5 | LODS | 0.76 | H | ||
Timsit et al (2002) [15], day 5 | 0.763 | U = 14.4 p = 0.07 | LODS | 0.749 | H | ||
Timsit et al (2002) [15], day 6 | 0.784 | U = 11 p = 0.17 | LODS | 0.79 | H | ||
Timsit et al (2002) [15], day 7 | 0.768 | U = 6.3 p = 0.62 | LODS | 0.746 | H | ||
Junger et al (2002) [18], Adm | 0.799 | I |
Performance of individual components of SOFA
Cardiovascular SOFA | AUC | Compared with | AUC | Mortality |
---|---|---|---|---|
Zygun et al (2005) [14], Adm | 0.68 | MODS | 0.63 | Hospital |
Khwannimit (2007) [10], Adm | 0.725 | LODS | 0.772 | ICU |
MODS | 0.726 | ICU | ||
Zygun et al (2005) [14], Adm | 0.74 | MODS | 0.64 | ICU |
Moreno et al (1999) [21], Adm | 0.802 | ICU | ||
Bota et al (2002) [16], Adm | 0.75 | MODS | 0.694 | Unspecified |
Bota et al (2002) [16], 48 hours | 0.732 | MODS | 0.675 | Unspecified |
Bota et al (2002) [16], 96 hours | 0.739 | MODS | 0.674 | Unspecified |
Bota et al (2002) [16], discharge | 0.781 | MODS | 0.75 | Unspecified |
Bota et al (2002) [16], max | 0.821 | MODS | 0.75 | Unspecified |
Respiratory SOFA
|
AUC
|
Compared with
|
AUC
|
Mortality
|
Khwannimit (2007) [10], Adm | 0.725 | LODS | 0.704 | ICU |
MODS | 0.71 | ICU | ||
Moreno et al (1999) [21], Adm | 0.736 | ICU | ||
Hepatic SOFA
|
AUC
|
Compared with
|
AUC
|
Mortality
|
Khwannimit (2007) [10], Adm | 0.539 | LODS | 0.563 | ICU |
MODS | 0.539 | ICU | ||
Moreno et al (1999) [21], Adm | 0.655 | ICU | ||
Renal SOFA
|
AUC
|
Compared with
|
AUC
|
Mortality
|
Khwannimit (2007) [10], Adm | 0.678 | LODS | 0.727 | ICU |
MODS | 0.659 | ICU | ||
Moreno et al (1999) [21], Adm | 0.739 | ICU | ||
Neurological SOFA
|
AUC
|
Compared with
|
AUC
|
Mortality
|
Khwannimit (2007) [10], Adm | 0.84 | LODS | 0.822 | ICU |
MODS | 0.839 | ICU | ||
Moreno et al (1999) [21], Adm | 0.727 | ICU | ||
Coagulation SOFA
|
AUC
|
Compared with
|
AUC
|
Mortality
|
Khwannimit (2007) [10], Adm | 0.623 | LODS | 0.59 | ICU |
MODS | 0.632 | ICU | ||
Moreno et al (1999) [21], Adm | 0.684 | ICU |
Performance of SOFA combined with other models and/or covariates
APACHE II | Given by | AUC | Brier | H/C statistics | Mortality |
---|---|---|---|---|---|
APACHE II | Ho (2007) [4] | 0.859 | 0.09 | C = 10 p = 0.189 | Hospital |
APACHE II + Total Max SOFA | Ho (2007) [4] | 0.875 | 0.086 | C = 10.1 p = 0.261 | Hospital |
APACHE II + Delta SOFA | Ho (2007) [4] | 0.874 | 0.086 | C = 7.5 p = 0.485 | Hospital |
APACHE II + Admission SOFA | Ho (2007) [4] | 0.861 | 0.09 | C = 9.3 p = 0.318 | Hospital |
SAPS II
|
Given by
|
AUC
|
Brier
|
H/C statistics
|
Mortality
|
SAPS II | Rivera-Fernández et al (2007) [5] | 0.8 | ICU | ||
SAPS II + Diagnosis | Rivera-Fernández et al (2007) [5] | 0.84 | ICU | ||
SAPS II + Diagnosis + Events | Rivera-Fernández et al (2007) [5] | 0.91 | ICU | ||
SAPS II + Mean SOFA + Max SOFA + Events | Rivera-Fernández et al (2007) [5] | 0.93 | ICU | ||
SAPS II + Mean SOFA+ Max SOFA + Events + Diagnosis | Rivera-Fernández et al (2007) [5] | 0.95 | H: 12.02 p > 0.05 | ICU | |
Other covariates
|
Given by
|
AUC
|
Brier
|
H/C statistics
|
Mortality
|
Min SOFA + Max SOFA+ SOFA trend over 5 days + Age | Cabré et al (2005) [6] | 0.807 | Hospital | ||
Max SOFA > 13 + Min SOFA > 10 + Positive SOFA trend + Age > 60 | Cabré et al (2005) [6] | 0.750 | Hospital | ||
Max SOFA > 10 + Min SOFA > 10 + Positive SOFA trend + Age > 60 | Cabré et al (2005) [6] | 0.758 | Hospital | ||
Total Max SOFA | Kajdacsy-Balla Amaral et al (2005) [7] | 0.841 | ICU | ||
Total Max SOFA + Infection | Kajdacsy-Balla Amaral et al (2005) [7] | 0.845 | ICU | ||
Total Max SOFA + Infection + Age | Kajdacsy-Balla Amaral et al (2005) [7] | 0.853 | C: p = 0.37 | ICU | |
H: p = 0.73 |
Brier score | ||||||||
---|---|---|---|---|---|---|---|---|
SAPS II + SOFA | Given by | Day 1 | Day 2 | Day 3 | Day 4 | Day 5 | Day 6 | Day 7 |
Recalibrated SAPS II | Toma et al (2007) [8] | 0.059 | 0.132 | 0.17 | 0.18 | 0.182 | ||
Recalibrated SAPS II | Toma et al (2008) [9] | 0.175 | 0.168 | 0.198 | 0.199 | 0.215 | 0.23 | |
Temporal SOFA model | Toma et al (2007) [8] | 0.058 | 0.128 | 0.161 | 0.171 | 0.166 | ||
Temporal SOFA model | Toma et al (2008) [9] | 0.168 | 0.17 | 0.195 | 0.183 | 0.206 | 0.211 | |
Temporal wSOFA model | Toma et al (2008) [9] | 0.166 | 0.175 | 0.199 | 0.19 | 0.21 | 0.224 | |
Temporal IOF model | Toma et al (2008) [9] | 0.161 | 0.166 | 0.187 | 0.175 | 0.195 | 0.216 | |
AUC
| ||||||||
SAPS II + SOFA
|
Given by
|
Day 1
|
Day 2
|
Day 3
|
Day 4
|
Day 5
|
Day 6
|
Day 7
|
Recalibrated SAPS II | Toma et al (2008) [9] | 0.761 | 0.746 | 0.692 | 0.66 | 0.643 | 0.645 | |
Temporal SOFA model | Toma et al (2008) [9] | 0.786 | 0.780 | 0.713 | 0.737 | 0.690 | 0.722 | |
Temporal wSOFA model | Toma et al (2008) [9] | 0.794 | 0.771 | 0.699 | 0.709 | 0.672 | 0.664 | |
Temporal IOF model | Toma et al (2008) [9] | 0.794 | 0.785 | 0.727 | 0.740 | 0.738 | 0.715 |
Performance of temporal SOFA models using pattern discovery
Discussion
Conclusion
Key messages
-
SOFA-based models evaluated on their prognostic performance fell under the categories: single SOFA scores at fixed times; sequential SOFA measurements; individual SOFA components; combination of SOFA with other covariates; and SOFA patterns automatically discovered from the data.
-
For predicting mortality SOFA-based models at admission seem to be competitive with severity of illness models limited to the first 24 hours of admission, and models based on sequential SOFA scores have comparable performance with other IOF scores.
-
The combination of SOFA-based models with admission-based models results in superior prognostic performance than each model alone.
-
Studies should use an independent validation set to assess performance and should apply multiple performance measures preferably covering discrimination, calibration and accuracy.