Skip to main content
Erschienen in: Intensive Care Medicine 2/2003

01.02.2003 | Original

External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study

verfasst von: Dieter H. Beck, Gary B. Smith, John V. Pappachan, Brian Millar

Erschienen in: Intensive Care Medicine | Ausgabe 2/2003

Einloggen, um Zugang zu erhalten

Abstract

Objective

External validation of three prognostic models in adult intensive care patients in South England.

Design

Prospective cohort study.

Setting

Seventeen intensive care units (ICU) in the South West Thames Region in South England.

Patients and participants

Data of 16,646 patients were analysed.

Interventions

None.

Measurements and results

We compared directly the predictive accuracy of three prognostic models (SAPS II, APACHE II and III), using formal tests of calibration and discrimination. The external validation showed a similar pattern for all three models tested: good discrimination, but imperfect calibration. The areas under the receiver operating characteristics (ROC) curves, used to test discrimination, were 0.835 and 0.867 for APACHE II and III, and 0.852 for the SAPS II model. Model calibration was assessed by Lemeshow-Hosmer C-statistics and was Χ2 =232.1 for APACHE II, Χ2 =443.3 for APACHE III and Χ2 =287.5 for SAPS II.

Conclusions

Disparity in case mix, a higher prevalence of outcome events and important unmeasured patient mix factors are possible sources for the decay of the models' predictive accuracy in our population. The lack of generalisability of standard prognostic models requires their validation and re-calibration before they can be applied with confidence to new populations. Customisation of existing models may become an important strategy to obtain authentic information on disease severity, which is a prerequisite for reliably measuring and comparing the quality and cost of intensive care.
Literatur
1.
Zurück zum Zitat Le Gall JR, Lemeshow S, Saulnier F (1993) Development of a new scoring system, the SAPS II, from a European/North American multicenter study. JAMA 270:2957–2963PubMed Le Gall JR, Lemeshow S, Saulnier F (1993) Development of a new scoring system, the SAPS II, from a European/North American multicenter study. JAMA 270:2957–2963PubMed
2.
Zurück zum Zitat Knaus WA, Draper EA, Wagner DP et al. (1985) APACHE II: a severity of disease classification system. Crit Care Med 10:818–829 Knaus WA, Draper EA, Wagner DP et al. (1985) APACHE II: a severity of disease classification system. Crit Care Med 10:818–829
3.
Zurück zum Zitat Knaus WA, Wagner DP, Draper EA et al. (1991) The APACHE III prognostic system: risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100:1619–1639PubMed Knaus WA, Wagner DP, Draper EA et al. (1991) The APACHE III prognostic system: risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100:1619–1639PubMed
4.
Zurück zum Zitat Wong DT, Crofts SL, Gomez M, McGuire GP, Byrick RJ (1995) Evaluation of predictive ability of APACHE II system and hospital outcome in Canadian intensive care patients. Crit Care Med 23:1177–1183 Wong DT, Crofts SL, Gomez M, McGuire GP, Byrick RJ (1995) Evaluation of predictive ability of APACHE II system and hospital outcome in Canadian intensive care patients. Crit Care Med 23:1177–1183
5.
Zurück zum Zitat Sirio CA, Tajimi K, Tase C, Knaus WA, Wagner DP, Hirasawa H et al. (1992) An initial comparison of intensive care in Japan and the United States. Crit Care Med 20:1207–1215 Sirio CA, Tajimi K, Tase C, Knaus WA, Wagner DP, Hirasawa H et al. (1992) An initial comparison of intensive care in Japan and the United States. Crit Care Med 20:1207–1215
6.
Zurück zum Zitat Markgraf R, Deutschinoff G, Pientka L, Scholten T (2000) Comparison of Acute Physiology and Chronic Health Evaluation score II and III and Simplified Acute Physiology Score II: a prospective cohort study evaluating these models to predict outcome in a German multidisciplinary intensive care unit. Crit Care Med 28:26–33PubMed Markgraf R, Deutschinoff G, Pientka L, Scholten T (2000) Comparison of Acute Physiology and Chronic Health Evaluation score II and III and Simplified Acute Physiology Score II: a prospective cohort study evaluating these models to predict outcome in a German multidisciplinary intensive care unit. Crit Care Med 28:26–33PubMed
7.
Zurück zum Zitat Castella X, Artigas A, Bion J, Kari A (1995) A comparison of severity of illness scoring systems for intensive care unit patients: results of a multicenter, multinational study. Crit Care Med 23:1327–1335 Castella X, Artigas A, Bion J, Kari A (1995) A comparison of severity of illness scoring systems for intensive care unit patients: results of a multicenter, multinational study. Crit Care Med 23:1327–1335
8.
Zurück zum Zitat Apolone G, Bertolini G, D'Amico R, Iapichino G, Cattanco A, De Salvo G, Melotti RM (1996) The performance of SAPS II in a cohort of Italian ICUs: results from GiViTI. Intensive Care Med 22:1368–1378PubMed Apolone G, Bertolini G, D'Amico R, Iapichino G, Cattanco A, De Salvo G, Melotti RM (1996) The performance of SAPS II in a cohort of Italian ICUs: results from GiViTI. Intensive Care Med 22:1368–1378PubMed
9.
Zurück zum Zitat Moreno R, Reis Miranda D, Fidler V, Van Schilfgaarde R (1998) Evaluation of two outcome prediction models on an independent database. Crit Care Med 26:50–61 Moreno R, Reis Miranda D, Fidler V, Van Schilfgaarde R (1998) Evaluation of two outcome prediction models on an independent database. Crit Care Med 26:50–61
10.
Zurück zum Zitat Rowan KM, Kerr JH, Major E, Mc Pherson K, Short A, Vesssey MP (1993) Intensive Care Society's APACHE II study in Britain and Ireland II: outcome comparisons of intensive care units after adjustment for case mix by the American APACHE II method. BMJ 307:977–981PubMed Rowan KM, Kerr JH, Major E, Mc Pherson K, Short A, Vesssey MP (1993) Intensive Care Society's APACHE II study in Britain and Ireland II: outcome comparisons of intensive care units after adjustment for case mix by the American APACHE II method. BMJ 307:977–981PubMed
11.
Zurück zum Zitat Zimmerman JE, Wagner DP, Draper EA et al. (1998) Evaluation of Acute Physiology and Chronic Health III predictions of hospital mortality in an independent database. Crit Care Med 20:1317–1326 Zimmerman JE, Wagner DP, Draper EA et al. (1998) Evaluation of Acute Physiology and Chronic Health III predictions of hospital mortality in an independent database. Crit Care Med 20:1317–1326
12.
Zurück zum Zitat Pappachan JV, Millar B, Bennett D, Smith GB (1999) Comparison of outcome from intensive care admission after adjustment for case mix by the APACHE III prognostic system. Chest 115:802–805PubMed Pappachan JV, Millar B, Bennett D, Smith GB (1999) Comparison of outcome from intensive care admission after adjustment for case mix by the APACHE III prognostic system. Chest 115:802–805PubMed
13.
Zurück zum Zitat Bastios PG, Sun X, Wagner DP, Knaus WA, Zimmerman JE (1996) Application of the APACHE III prognostic system in Brazilian intensive care units: a prospective multicenter study. Intensive Care Med 22:564–570CrossRefPubMed Bastios PG, Sun X, Wagner DP, Knaus WA, Zimmerman JE (1996) Application of the APACHE III prognostic system in Brazilian intensive care units: a prospective multicenter study. Intensive Care Med 22:564–570CrossRefPubMed
14.
Zurück zum Zitat Rowan KM, Kerr JH, Major E et al. (1994) Intensive Care Society's Acute Physiology and Chronic Health Evaluation (APACHE II) study in Britain and Ireland: a prospective, multicenter, cohort study comparing two methods for predicting outcome for adult intensive care patients. Crit Care Med 22:1392–1401 Rowan KM, Kerr JH, Major E et al. (1994) Intensive Care Society's Acute Physiology and Chronic Health Evaluation (APACHE II) study in Britain and Ireland: a prospective, multicenter, cohort study comparing two methods for predicting outcome for adult intensive care patients. Crit Care Med 22:1392–1401
15.
Zurück zum Zitat Livingston BM, MacKirdy FN, Howie JC, Jones R, Norrie JD (2000) Assessment of the performance of five intensive care scoring models within a large Scottish database. Crit Care Med 28:1820–1827PubMed Livingston BM, MacKirdy FN, Howie JC, Jones R, Norrie JD (2000) Assessment of the performance of five intensive care scoring models within a large Scottish database. Crit Care Med 28:1820–1827PubMed
16.
Zurück zum Zitat Beck DH, Taylor BL, Millar B, Smith GB (1997) Prediction of outcome from intensive care: a prospective cohort study comparing APACHE II and III in a UK intensive care unit. Crit Care Med 25:9–15PubMed Beck DH, Taylor BL, Millar B, Smith GB (1997) Prediction of outcome from intensive care: a prospective cohort study comparing APACHE II and III in a UK intensive care unit. Crit Care Med 25:9–15PubMed
17.
Zurück zum Zitat Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29–36PubMed Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29–36PubMed
18.
Zurück zum Zitat Hosmer DW, Lemeshow S (1989) Applied logistic regression. John Wiley, New York, pp 135–149 Hosmer DW, Lemeshow S (1989) Applied logistic regression. John Wiley, New York, pp 135–149
19.
Zurück zum Zitat Lemeshow S, Hosmer DW (1982) A review of the goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 115: 92–106PubMed Lemeshow S, Hosmer DW (1982) A review of the goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 115: 92–106PubMed
20.
Zurück zum Zitat Altman DC, Royston P (2000) What do we mean by validating prognostic models? Stat Med 19:453–473CrossRefPubMed Altman DC, Royston P (2000) What do we mean by validating prognostic models? Stat Med 19:453–473CrossRefPubMed
21.
Zurück zum Zitat Harrell FE, Lee K (1984) Regression modelling strategies for improved prognostic prediction. Stat Med 3:143–152PubMed Harrell FE, Lee K (1984) Regression modelling strategies for improved prognostic prediction. Stat Med 3:143–152PubMed
22.
Zurück zum Zitat Goldhill DR, Withington PS (1996) The effect of case mix adjustment on mortality as predicted by APACHE II. Intensive Care Med 22:415–419PubMed Goldhill DR, Withington PS (1996) The effect of case mix adjustment on mortality as predicted by APACHE II. Intensive Care Med 22:415–419PubMed
23.
Zurück zum Zitat Rowan KM, Kerr JH, Major E et al. (1993) Intensive Care Society's APACHE II study in Britain and Ireland I: variations in case mix of adult admissions to general intensive care units and impact on outcome. BMJ 307:972–977PubMed Rowan KM, Kerr JH, Major E et al. (1993) Intensive Care Society's APACHE II study in Britain and Ireland I: variations in case mix of adult admissions to general intensive care units and impact on outcome. BMJ 307:972–977PubMed
24.
Zurück zum Zitat Moreno R, Apolone G, Reis Miranda D (1998) Evaluation of the uniformity of fit of general outcome prediction models. Intensive Care Med 24:40–47PubMed Moreno R, Apolone G, Reis Miranda D (1998) Evaluation of the uniformity of fit of general outcome prediction models. Intensive Care Med 24:40–47PubMed
25.
Zurück zum Zitat Metnitz PGH, Vesely H, Valentin A, Popow C, Hiesmayr M, Lenz K, Krenn CG Steltzer H (1999) Evaluation of an interdisciplinary data set for national intensive care unit assessment. Crit Care Med 27:1486–1491PubMed Metnitz PGH, Vesely H, Valentin A, Popow C, Hiesmayr M, Lenz K, Krenn CG Steltzer H (1999) Evaluation of an interdisciplinary data set for national intensive care unit assessment. Crit Care Med 27:1486–1491PubMed
26.
Zurück zum Zitat Murphy-Filkins RL, Teres D, Lemeshow S, Hosmer DW (1996) Effect of changing patient mix on the performance of intensive care unit severity-of-illness models: how to distinguish a general from a speciality intensive care unit. Crit Care Med 24:1968–1973 Murphy-Filkins RL, Teres D, Lemeshow S, Hosmer DW (1996) Effect of changing patient mix on the performance of intensive care unit severity-of-illness models: how to distinguish a general from a speciality intensive care unit. Crit Care Med 24:1968–1973
27.
Zurück zum Zitat Lemeshow S, Teres D, Klar J et al. (1993) Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA 270:2478–2486PubMed Lemeshow S, Teres D, Klar J et al. (1993) Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA 270:2478–2486PubMed
28.
Zurück zum Zitat Beck DH, McQuillan PJ, Smith GB (2002) Waiting for the break of dawn? The effects of discharge TISS scores, discharge time and discharge facilities on mortality after intensive care. Intensive Care Med 28:1287–1293PubMed Beck DH, McQuillan PJ, Smith GB (2002) Waiting for the break of dawn? The effects of discharge TISS scores, discharge time and discharge facilities on mortality after intensive care. Intensive Care Med 28:1287–1293PubMed
29.
Zurück zum Zitat Miller ME, Hui SL, Tierney WM (1991) Validation techniques for logistic regression models. Stat Med 10:1213–1226PubMed Miller ME, Hui SL, Tierney WM (1991) Validation techniques for logistic regression models. Stat Med 10:1213–1226PubMed
30.
Zurück zum Zitat Zhu PG, Lemeshow S, Hosmer DW, Klar J, Avrunin J, Teres D (1996) Factors affecting the performance of the models in the Mortality Prediction Model II system and strategies of customization: a simulation study. Crit Care Med 24:57–63 Zhu PG, Lemeshow S, Hosmer DW, Klar J, Avrunin J, Teres D (1996) Factors affecting the performance of the models in the Mortality Prediction Model II system and strategies of customization: a simulation study. Crit Care Med 24:57–63
31.
Zurück zum Zitat Glance LG, Osler TM, Papadakos P (2000) Effect of mortality rate on the performance of the Acute Physiology and Chronic Health Evaluation II: a simulation study. Crit Care Med 28:3424–3428PubMed Glance LG, Osler TM, Papadakos P (2000) Effect of mortality rate on the performance of the Acute Physiology and Chronic Health Evaluation II: a simulation study. Crit Care Med 28:3424–3428PubMed
32.
Zurück zum Zitat Rivera-Fernandez R, Vazquez-Mata G, Bravo M, Aguayo-Hoyos E, Zimmerman JE, Wagner DP, Knaus WA (1998) The APACHE III prognostic system: customized mortality predictions for Spanish ICU patients. Intensive Care Med 24:574–581PubMed Rivera-Fernandez R, Vazquez-Mata G, Bravo M, Aguayo-Hoyos E, Zimmerman JE, Wagner DP, Knaus WA (1998) The APACHE III prognostic system: customized mortality predictions for Spanish ICU patients. Intensive Care Med 24:574–581PubMed
33.
Zurück zum Zitat Moreno R, Apolone G (1997) Impact of different customization strategies in the performance of a general severity score. Crit Care Med 25:2001–2008 Moreno R, Apolone G (1997) Impact of different customization strategies in the performance of a general severity score. Crit Care Med 25:2001–2008
34.
Zurück zum Zitat Beck DH, Smith GB, Pappachan JV (2002) The effects of two methods for customising the original Simplified Acute Physiology Score (SAPS) II for intensive care patients from South England. Anaesthesia; 57:785–793 Beck DH, Smith GB, Pappachan JV (2002) The effects of two methods for customising the original Simplified Acute Physiology Score (SAPS) II for intensive care patients from South England. Anaesthesia; 57:785–793
35.
Zurück zum Zitat Sirio CA, Shepardson LB, Rotondi AJ, Cooper GS, Angus DC, Harper DL, Rosenthal GE (1999) Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure. Chest 115:793–801PubMed Sirio CA, Shepardson LB, Rotondi AJ, Cooper GS, Angus DC, Harper DL, Rosenthal GE (1999) Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure. Chest 115:793–801PubMed
36.
Zurück zum Zitat Teres D, Lemeshow S (1998) As American as apple pie and APACHE (editorial). Crit Care Med 26:1297–1298 Teres D, Lemeshow S (1998) As American as apple pie and APACHE (editorial). Crit Care Med 26:1297–1298
37.
38.
Zurück zum Zitat Smith GB, Taylor BL, McQuillan PJ, Nials E (1995) Rationing intensive care: intensive care provision varies widely in Britain. BMJ 10:1412–1413 Smith GB, Taylor BL, McQuillan PJ, Nials E (1995) Rationing intensive care: intensive care provision varies widely in Britain. BMJ 10:1412–1413
39.
Zurück zum Zitat Metcalfe A, McPherson K. Sloggett A (1997) Mortality among appropriately referred patients refused admission to intensive care units. Lancet 350:7–11PubMed Metcalfe A, McPherson K. Sloggett A (1997) Mortality among appropriately referred patients refused admission to intensive care units. Lancet 350:7–11PubMed
40.
Zurück zum Zitat Goldfrad C, Rowan K (2000) Consequences of discharges from intensive care at night. Lancet 355:1138–1142PubMed Goldfrad C, Rowan K (2000) Consequences of discharges from intensive care at night. Lancet 355:1138–1142PubMed
Metadaten
Titel
External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study
verfasst von
Dieter H. Beck
Gary B. Smith
John V. Pappachan
Brian Millar
Publikationsdatum
01.02.2003
Verlag
Springer-Verlag
Erschienen in
Intensive Care Medicine / Ausgabe 2/2003
Print ISSN: 0342-4642
Elektronische ISSN: 1432-1238
DOI
https://doi.org/10.1007/s00134-002-1607-9

Weitere Artikel der Ausgabe 2/2003

Intensive Care Medicine 2/2003 Zur Ausgabe

Darf man die Behandlung eines Neonazis ablehnen?

08.05.2024 Gesellschaft Nachrichten

In einer Leseranfrage in der Zeitschrift Journal of the American Academy of Dermatology möchte ein anonymer Dermatologe bzw. eine anonyme Dermatologin wissen, ob er oder sie einen Patienten behandeln muss, der eine rassistische Tätowierung trägt.

Ein Drittel der jungen Ärztinnen und Ärzte erwägt abzuwandern

07.05.2024 Klinik aktuell Nachrichten

Extreme Arbeitsverdichtung und kaum Supervision: Dr. Andrea Martini, Sprecherin des Bündnisses Junge Ärztinnen und Ärzte (BJÄ) über den Frust des ärztlichen Nachwuchses und die Vorteile des Rucksack-Modells.

Häufigste Gründe für Brustschmerzen bei Kindern

06.05.2024 Pädiatrische Diagnostik Nachrichten

Akute Brustschmerzen sind ein Alarmsymptom par exellence, schließlich sind manche Auslöser lebensbedrohlich. Auch Kinder klagen oft über Schmerzen in der Brust. Ein Studienteam ist den Ursachen nachgegangen.

Aquatherapie bei Fibromyalgie wirksamer als Trockenübungen

03.05.2024 Fibromyalgiesyndrom Nachrichten

Bewegungs-, Dehnungs- und Entspannungsübungen im Wasser lindern die Beschwerden von Patientinnen mit Fibromyalgie besser als das Üben auf trockenem Land. Das geht aus einer spanisch-brasilianischen Vergleichsstudie hervor.

Update AINS

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.