Skip to main content
Erschienen in: Intensive Care Medicine 9/2005

01.09.2005 | Pediatric Original

Effect of training and strict guidelines on the reliability of risk adjustment systems in paediatric intensive care

verfasst von: Jolanda G. van Keulen, Reinoud J. B. J. Gemke, Kees H. Polderman

Erschienen in: Intensive Care Medicine | Ausgabe 9/2005

Einloggen, um Zugang zu erhalten

Abstract

Objective

Many risk-adjustment systems have significant interobserver variability in everyday clinical practice. This can be partly corrected by strict guidelines and training. These issues have not been well studied in the paediatric setting. We assessed the reliability of two widely used paediatric scoring systems, the Paediatric Risk of Mortality (PRISM) and Paediatric Index of Mortality (PIM), before and after a special training program.

Design

Prospective observational multi-centred cohort study.

Intervention

Twenty-seven physicians from five paediatric intensive care units (PICUs) assessed severity of illness in 20 selected patients using PRISM and PIM scores before and after a special training program. Physicians were divided according to level of PICU experience: intensivists (>3 years experience, n=12), fellows (6–30 months experience, n=6) and residents (<6 months experience, n=9). Intraclass correlation was used to compare scoring reliability before and after training.

Measurements and results

Wide variability in PRISM and PIM scoring was observed before training (intraclass correlation for PRISM scores 0.24–0.73, intraclass correlation for PIM scores 0.16–0.33). Training and implementation of guidelines led to significant increases in interobserver agreement (intraclass correlation 0.74–0.86 for PRISM and 0.88–0.95 for PIM scores), although some variability remained.

Conclusion

Our results show that the reliability of PRISM and PIM risk adjustment systems in daily clinical practice is much lower than expected. Training and guidelines can significantly increase interobserver agreement. These factors should be taken into account when using these systems for benchmarking, or to compare quality of care between different PICUs.
Literatur
1.
Zurück zum Zitat Pollack MM, Ruttimann UE, Getson PR (1988) Pediatric risk of mortality score (PRISM) score. Crit Care Med 16:1110–1116PubMed Pollack MM, Ruttimann UE, Getson PR (1988) Pediatric risk of mortality score (PRISM) score. Crit Care Med 16:1110–1116PubMed
2.
Zurück zum Zitat Shann F, Pearson G, Slater A, Wilkinson K (1997) Pediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med 23:201–207CrossRefPubMed Shann F, Pearson G, Slater A, Wilkinson K (1997) Pediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med 23:201–207CrossRefPubMed
3.
Zurück zum Zitat Pollack MM, Patel KM, Ruttimann UE (1997) The pediatric risk of mortality III: Acute Physiology Score (PRISM III-APS): a method of assessing physiologic instability for pediatric intensive care unit patients. J Pediatr 131:575–581PubMed Pollack MM, Patel KM, Ruttimann UE (1997) The pediatric risk of mortality III: Acute Physiology Score (PRISM III-APS): a method of assessing physiologic instability for pediatric intensive care unit patients. J Pediatr 131:575–581PubMed
4.
Zurück zum Zitat Gemke RJBJ, Bonsel GJ (1995) The Pediatric Intensive Care Assessment of Outcome (PICASSO) study group. Comparative assessment of pediatric intensive care: a national multicenter study. Crit Care Med 23:238–245CrossRefPubMed Gemke RJBJ, Bonsel GJ (1995) The Pediatric Intensive Care Assessment of Outcome (PICASSO) study group. Comparative assessment of pediatric intensive care: a national multicenter study. Crit Care Med 23:238–245CrossRefPubMed
5.
Zurück zum Zitat Slater A, Shann F, Gearson G (2003) PIM2: a revised version of the Pediatric Index of Mortality. Intensive Care Med 29:278–285PubMed Slater A, Shann F, Gearson G (2003) PIM2: a revised version of the Pediatric Index of Mortality. Intensive Care Med 29:278–285PubMed
6.
Zurück zum Zitat Tibby SM, Taylor D, Festa M, Hanna S, Hatherill M, Jones G, Habibi P, Durward A, Murdoch IA (2002) A comparison of three scoring systems for mortality risk among retrieved intensive care patients. Arch Dis Child 87:421–425CrossRefPubMed Tibby SM, Taylor D, Festa M, Hanna S, Hatherill M, Jones G, Habibi P, Durward A, Murdoch IA (2002) A comparison of three scoring systems for mortality risk among retrieved intensive care patients. Arch Dis Child 87:421–425CrossRefPubMed
7.
Zurück zum Zitat Gemke RJBJ, Bonsel GJ, van Vught AJ (1994) Effectiveness and efficiency of a Dutch pediatric intensive care unit: validity and application of the Pediatric Risk of Mortality score. Crit Care Med 22:1477–1484PubMed Gemke RJBJ, Bonsel GJ, van Vught AJ (1994) Effectiveness and efficiency of a Dutch pediatric intensive care unit: validity and application of the Pediatric Risk of Mortality score. Crit Care Med 22:1477–1484PubMed
8.
Zurück zum Zitat van Keulen JG, Polderman KH, Gemke RJBJ (2005) Reliability of PRISM and PIM scores in paediatric intensive care. Arch Dis Child 90:211–214PubMed van Keulen JG, Polderman KH, Gemke RJBJ (2005) Reliability of PRISM and PIM scores in paediatric intensive care. Arch Dis Child 90:211–214PubMed
9.
Zurück zum Zitat Polderman KH, Jorna EMF, Girbes ARJ (2001) Inter-observer variability in APACHE II scoring: effect of strict guidelines and training. Intensive Care Med 27:1365–1369CrossRefPubMed Polderman KH, Jorna EMF, Girbes ARJ (2001) Inter-observer variability in APACHE II scoring: effect of strict guidelines and training. Intensive Care Med 27:1365–1369CrossRefPubMed
10.
Zurück zum Zitat Gemke RJ, van Vught J (2002) Scoring systems in pediatric intensive care: PRISM III versus PIM. Intensive Care Med 28:204–207CrossRefPubMed Gemke RJ, van Vught J (2002) Scoring systems in pediatric intensive care: PRISM III versus PIM. Intensive Care Med 28:204–207CrossRefPubMed
11.
Zurück zum Zitat Shann F (2002) Are we doing a good job: PRISM, PIM and all that. Intensive Care Med 28:105–107CrossRefPubMed Shann F (2002) Are we doing a good job: PRISM, PIM and all that. Intensive Care Med 28:105–107CrossRefPubMed
12.
Zurück zum Zitat Jones GD, Thorburn K, Tigg A, Murdoch IA (2000) Preliminary data: PIM vs PRISM in infants and children post cardiac surgery in a UK PICU. Intensive Care Med 26:145 (Letter)CrossRefPubMed Jones GD, Thorburn K, Tigg A, Murdoch IA (2000) Preliminary data: PIM vs PRISM in infants and children post cardiac surgery in a UK PICU. Intensive Care Med 26:145 (Letter)CrossRefPubMed
13.
Zurück zum Zitat Taylor A, Butt W, Ciardulli M (2003) The functional outcome and quality of life of children after admission to an intensive care unit. Intensive Care Med 29:795–800PubMed Taylor A, Butt W, Ciardulli M (2003) The functional outcome and quality of life of children after admission to an intensive care unit. Intensive Care Med 29:795–800PubMed
14.
Zurück zum Zitat Slater A, Shann F, Pearson G (2003) PIM2: a revised version of the Paediatric Index of Mortality. Intensive Care Med 29:278–285PubMed Slater A, Shann F, Pearson G (2003) PIM2: a revised version of the Paediatric Index of Mortality. Intensive Care Med 29:278–285PubMed
15.
Zurück zum Zitat Polderman KH, Thijs LG, Girbes ARJ (1999) Interobserver variability in the use of APACHE II scores. Lancet 353:380CrossRef Polderman KH, Thijs LG, Girbes ARJ (1999) Interobserver variability in the use of APACHE II scores. Lancet 353:380CrossRef
16.
Zurück zum Zitat Polderman KH, Christiaans HM, Wester JP, Spijkstra JJ, Girbes AR (2001) Intraobserver variability in APACHE II scoring. Intensive Care Med 27:1550–1552CrossRefPubMed Polderman KH, Christiaans HM, Wester JP, Spijkstra JJ, Girbes AR (2001) Intraobserver variability in APACHE II scoring. Intensive Care Med 27:1550–1552CrossRefPubMed
17.
Zurück zum Zitat Polderman KH, Girbes ARJ, Thijs LG, Strack van Schijndel RJM (2001) Accuracy and reliability of APACHE II scoring in two intensive care units. Problems and pitfalls in the use of APACHE II and suggestions for improvement. Anaesthesia 56:47–50CrossRefPubMed Polderman KH, Girbes ARJ, Thijs LG, Strack van Schijndel RJM (2001) Accuracy and reliability of APACHE II scoring in two intensive care units. Problems and pitfalls in the use of APACHE II and suggestions for improvement. Anaesthesia 56:47–50CrossRefPubMed
18.
Zurück zum Zitat Chen LM, Martin CM, Morrison TL, Sibbald WJ (1999) Interobserver variability in data collection of the APACHE II score in teaching and community hospitals. Crit Care Med 27:1999–2004CrossRefPubMed Chen LM, Martin CM, Morrison TL, Sibbald WJ (1999) Interobserver variability in data collection of the APACHE II score in teaching and community hospitals. Crit Care Med 27:1999–2004CrossRefPubMed
19.
Zurück zum Zitat Fleiss JL, Shrout PE (1977) The effects of measurement errors on some multivariate procedures. Am J Public Health 67:1188–1191PubMed Fleiss JL, Shrout PE (1977) The effects of measurement errors on some multivariate procedures. Am J Public Health 67:1188–1191PubMed
20.
Zurück zum Zitat Altman DG (1991) Practical statistics for medical research. Chapman and Hall, London, pp 277–306 Altman DG (1991) Practical statistics for medical research. Chapman and Hall, London, pp 277–306
21.
Zurück zum Zitat Lemeshow S, Le Gall JR (1994) Modeling the severity of illness of ICU patients. J Am Med Assoc 272:1049–1055CrossRef Lemeshow S, Le Gall JR (1994) Modeling the severity of illness of ICU patients. J Am Med Assoc 272:1049–1055CrossRef
22.
Zurück zum Zitat Teres D, Lemeshow S (1994) Why severity models should be used with caution. Crit Care Clin 10:93–110PubMed Teres D, Lemeshow S (1994) Why severity models should be used with caution. Crit Care Clin 10:93–110PubMed
23.
Zurück zum Zitat Randolph AG (1997) Paediatric index of mortality (PIM): Do we need another paediatric mortality prediction score? Intensive Care Med 23:141–142CrossRefPubMed Randolph AG (1997) Paediatric index of mortality (PIM): Do we need another paediatric mortality prediction score? Intensive Care Med 23:141–142CrossRefPubMed
24.
Zurück zum Zitat Manns BJ, Lee H, Doig CJ, Johnson D, Donaldson C (2002) An economic evaluation of activated protein C treatment for severe sepsis. N Engl J Med 347:993–1000CrossRefPubMed Manns BJ, Lee H, Doig CJ, Johnson D, Donaldson C (2002) An economic evaluation of activated protein C treatment for severe sepsis. N Engl J Med 347:993–1000CrossRefPubMed
25.
Zurück zum Zitat Bernard GR, Vincent JL, Laterre PF, LaRosa SP, Dhainaut JF, Lopez-Rodriguez A, Steingrub JS, Garber GE, Helterbrand JD, Ely EW, Fisher CJ Jr, Recombinant Human Protein C Worldwide Evaluation in Severe Sepsis (PROWESS) Study Group (2001) Efficacy and safety of recombinant human activated protein C for severe sepsis. N Engl J Med 344:699–709CrossRefPubMed Bernard GR, Vincent JL, Laterre PF, LaRosa SP, Dhainaut JF, Lopez-Rodriguez A, Steingrub JS, Garber GE, Helterbrand JD, Ely EW, Fisher CJ Jr, Recombinant Human Protein C Worldwide Evaluation in Severe Sepsis (PROWESS) Study Group (2001) Efficacy and safety of recombinant human activated protein C for severe sepsis. N Engl J Med 344:699–709CrossRefPubMed
26.
Zurück zum Zitat Polderman KH, Girbes ARJ (2004) Drug intervention trials in sepsis: divergent results. Lancet 363:1721–1723CrossRefPubMed Polderman KH, Girbes ARJ (2004) Drug intervention trials in sepsis: divergent results. Lancet 363:1721–1723CrossRefPubMed
27.
Zurück zum Zitat van Zanten AR, Polderman KH (2004) Organizational changes in a single intensive care unit affect benchmarking. Ann Intern Med 140:674–675PubMed van Zanten AR, Polderman KH (2004) Organizational changes in a single intensive care unit affect benchmarking. Ann Intern Med 140:674–675PubMed
28.
Zurück zum Zitat Rosenberg AL, Hofer TP, Strachan C, Watts CM, Hayward RA (2003) Accepting critically ill transfer patients: adverse effect on a referral center’s outcome and benchmark measures. Ann Intern Med 138:882–890PubMed Rosenberg AL, Hofer TP, Strachan C, Watts CM, Hayward RA (2003) Accepting critically ill transfer patients: adverse effect on a referral center’s outcome and benchmark measures. Ann Intern Med 138:882–890PubMed
Metadaten
Titel
Effect of training and strict guidelines on the reliability of risk adjustment systems in paediatric intensive care
verfasst von
Jolanda G. van Keulen
Reinoud J. B. J. Gemke
Kees H. Polderman
Publikationsdatum
01.09.2005
Verlag
Springer-Verlag
Erschienen in
Intensive Care Medicine / Ausgabe 9/2005
Print ISSN: 0342-4642
Elektronische ISSN: 1432-1238
DOI
https://doi.org/10.1007/s00134-005-2716-z

Weitere Artikel der Ausgabe 9/2005

Intensive Care Medicine 9/2005 Zur Ausgabe

Ähnliche Überlebensraten nach Reanimation während des Transports bzw. vor Ort

29.05.2024 Reanimation im Kindesalter Nachrichten

Laut einer Studie aus den USA und Kanada scheint es bei der Reanimation von Kindern außerhalb einer Klinik keinen Unterschied für das Überleben zu machen, ob die Wiederbelebungsmaßnahmen während des Transports in die Klinik stattfinden oder vor Ort ausgeführt werden. Jedoch gibt es dabei einige Einschränkungen und eine wichtige Ausnahme.

Häusliche Gewalt in der orthopädischen Notaufnahme oft nicht erkannt

28.05.2024 Häusliche Gewalt Nachrichten

In der Notaufnahme wird die Chance, Opfer von häuslicher Gewalt zu identifizieren, von Orthopäden und Orthopädinnen offenbar zu wenig genutzt. Darauf deuten die Ergebnisse einer Fragebogenstudie an der Sahlgrenska-Universität in Schweden hin.

Fehlerkultur in der Medizin – Offenheit zählt!

28.05.2024 Fehlerkultur Podcast

Darüber reden und aus Fehlern lernen, sollte das Motto in der Medizin lauten. Und zwar nicht nur im Sinne der Patientensicherheit. Eine negative Fehlerkultur kann auch die Behandelnden ernsthaft krank machen, warnt Prof. Dr. Reinhard Strametz. Ein Plädoyer und ein Leitfaden für den offenen Umgang mit kritischen Ereignissen in Medizin und Pflege.

Mehr Frauen im OP – weniger postoperative Komplikationen

21.05.2024 Allgemeine Chirurgie Nachrichten

Ein Frauenanteil von mindestens einem Drittel im ärztlichen Op.-Team war in einer großen retrospektiven Studie aus Kanada mit einer signifikanten Reduktion der postoperativen Morbidität assoziiert.

Update AINS

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.