Updating the evidence for the role of corticosteroids in severe sepsis and septic shock: a Bayesian meta-analytic perspective

Randomised controlled trials in critically ill patients evaluating corticosteroid therapy versus no corticosteroid therapy in severe sepsis or septic shock were considered for inclusion. Only trials reporting mortality were included. We excluded: studies reporting only physiological endpoints (for example, changes in immunological variables); descriptive studies; retrospective cohort studies; studies in the pediatric population; and studies exclusively reporting series of meningitis, typhoid fever and pneumonia where sub-set analyses of patients of interest (for this meta-analysis) were not reported. Where there was missing data or ambiguity of data presentation, attempts were made to contact the study author(s) to resolve these issues.

An extensive computerized literature search was performed (SR) for the period of 1950 to September 2008 using the National Library of Medicine MEDLINE via OVID, OVID PreMedline, EBSCO Cinahl, OVID Embase, Cochrane Central Register of Controlled trials, Cochrane database of systematic reviews, American College of Physicians Journal Club, Health Technology Assessment Database and Database of Abstracts of Reviews of Effects. We restricted the search to studies on adult human populations and used the Mesh, Embase and Cinahl thesaurus in addition to free text searching. The following terms were identified as the most relevant: sepsis or bacteremia or fungemia or pneumonia or septicemia or septic shock narrowed down with the terms hydrocortisone or corticosteroids or adrenal cortex hormones or steroids. The set was then further limited to randomised controlled trials or clinical trials or multicenter study and trials published in English. A detailed search strategy is provided in Additional file 1.

We reviewed the abstracts of trials generated by the electronic search and the full text of trials pertaining to corticosteroids in sepsis and septic shock were retrieved for a more detailed evaluation. Review articles were examined to identify additional trials. In addition a hand search of the proceedings of scientific meetings of the following journals was performed: American Journal of Respiratory and Critical Care Medicine, Chest, Critical Care Medicine, European Respiratory Journal, Intensive Care Medicine and Thorax.

Three investigators (JLM, PLG, and AB) reviewed studies fulfilling inclusion criteria and pre-defined variables and outcomes were abstracted into standardized data abstraction forms. Quality assessment on the published studies was performed in an un-blinded fashion by two investigators (JLM, PLG) using the 11-point quality score of Cronin and colleagues [4]. Where there were differences in scoring, a consensus was reached. Extracted data was separately entered, reviewed and verified by two investigators (JLM, PLG) prior to analysis.

The primary outcome was mortality assessment at hospital discharge. Secondary outcomes were resolution of shock (or withdrawal of inotropes) at 7 to 28 days and corticotrophin responsiveness, secondary infections and non-infective (gastro-intestinal bleeding and new-onset hyperglycemia) complications.

Severe sepsis and septic shock were defined after the 1992 American College of Chest Physicians and Society of Critical Care Medicine Consensus Conference [19]. Pre-1992 studies were reviewed to establish consistency with this definition. Secondary infections were defined generally as a positive culture from a normally sterile site. The time span of the studies suggested that definitions for secondary infections would be subject to revision; for example, the use of quantitative cultures in more recent calendar years [20]. Shock resolution was defined as a stable hemodynamic state for a period of 24 hours or more after weaning of vasopressor support. Corticosteroid dose was converted into hydrocortisone equivalents (mg) and expressed as total maximum realizable dose [20] accounting for total time of exposure (therapeutic dose-time and tapering). Where patient corticosteroid dose-time schedule was unavailable due to death and/or reporting, we used median survival time from the published Kaplan-Meier curve.

The effect of corticosteroids compared with control on mortality; the proportion of patients experiencing shock-resolution at defined times; and infective and non-infective complications were assessed using Bayesian random-effects models [21], via WinBUGS software [22] using three simultaneous runs of the program with disparate starting values. The first 10,000 iterations were discarded and results were reported as the posterior median odds ratio (OR) with 95% credible intervals (CrI) on the basis of a further 100,000 iterations. As argued previously [20], the hazard ratio would have been the preferred metric for mortality effect due to varying event times. However, due to the variability in intra-trial reporting, this was not feasible. As the hazard ratio may be approximated from the OR [23], we chose the OR as an appropriate metric [24]. Bayesian parameter estimates, as opposed to frequentist, are probability distributions and hence there is no contradiction in computing both (i) a (median) point estimate and CrI and (ii) the posterior probability (P) that, say, the OR is more than 1 [25, 26]. That is, "Bayesian methodology also allows us to make statements about the probability that the ORs are greater than 1 in cases in which the associated 95% CrI includes 1" [27]. A probability of 50% suggests a null effect, while P of at least 90% signifies harm and P less than 10% indicates benefit for the mortality, infective and non-infective endpoints and vice versa for the shock reversal endpoint [28]. Analysis was undertaken by stratifying between 'high-dose' and 'low-dose' corticosteroid therapy, as in Annane and colleagues [6] and after the categorization of daily treatment doses of hydrocortisone by Marik (high-dose corticosteroid >1,000 mg per day) [29].

Bayesian meta-regression [21] was used to determine the relation between log odds mortality and (i) average patient age and (ii) control-arm risk, as log-odds mortality [17, 24]. The slope (β) with 95% CrI and the probability that β ≥ 0 (P_β) were presented. Heterogeneity was presented as the standard deviation, τ, between studies [30]; τ close to 0 indicates little heterogeneity, τ = 0.5 indicates moderate and τ > 1 reflects substantial heterogeneity [18].

For heuristic purposes we separately estimated: (i) pooled estimates with the Schumer [31] and Cooperative Study Group (CSG) [32] studies removed in a sensitivity analysis due to previous identification of the former as a potential outlier [7] and the remoteness of the latter 1963 trial from current therapeutic regimens; (ii) certain parameters of clinical import in the risk difference metric [21], albeit this metric may suffer from potential bias with varying time to event [24]; (iii) the mortality OR and probability (P) that the OR was 1 or more in the predictive distribution (that is, in the next 'new' study); (iv) the mortality OR for hypothesized studies of size 2,000 and 4,000 patients; (v) the Bayesian predictive P-value that the CORTICUS trial [8] was inconsistent with the other trials of the low-dose corticosteroid group; that is, the CORTICUS study was omitted from analysis (leaving n = 7 trials) and a replicate study of the same size as the CORTICUS study was drawn, with a replicate baseline, and a new treatment effect was established based upon the predictive distribution. A Bayesian predictive P-value was subsequently obtained, expressing the probability that the future study would be as 'extreme' as that observed.

Publication and the associated phenomenon of small-study bias were addressed using the approach of Peters and colleagues [33] via contour-enhanced funnel plots; formal quantitative testing for small-study-bias was performed using the approach of Harbord and colleagues [34], which has effective properties in the presence of appreciable heterogeneity. Implementation was via the R package 'meta' [35] and user-written routines.

Neither the low-dose nor high-dose cohort showed a significant steroid treatment effect for the mortality OR, although there was modest evidence of benefit in the low-dose cohort (P = 20.4%) (Table 5 and Figure 2). The odds of mortality (four studies [8, 36, 42, 45]), for both corticotrophin responders and non-responders was not significantly different compared with control (Table 5).

A contour-enhanced funnel plot showed no obvious asymmetry in terms of a lack of small studies with a null or adverse steroid effect (Figure 3), this was not rejected (at the 0.1 level) using the quantitative approach of Harbord and colleagues [34] (P = 0.146). The low-dose studies showed a degree of asymmetry of the contour-enhanced funnel plot (Figure 4), but the quantitative estimate did not confirm this (P = 0.113).

Median vasopressor time (six studies [8, 36, 37, 39, 42, 47]) ranged from 2 to 7 days for steroid-treated patients and 5 to 13 days for placebo. With respect to the number of patients experiencing shock reversal, there was no clear steroid treatment-effect (overall OR included 1) for high-dose studies (n = 2). However, there was a high probability of benefit for the low-dose cohort; moderate heterogeneity being present (Figure 5 and Table 5). Odds of shock-reversal were not substantially different for corticotrophin non-responders or responders; however, both had a high probability of benefit (Table 5).

Univariate metaregression of average age against log odds mortality yielded no significant effects although P_β was high and the slope positive for both low- and high-dose cohorts. This indicated some evidence that, on average, older study participants had increased odds of mortality under steroid treatment versus control (Table 5). Similarly, although the metaregression of underlying control-arm risk against log odds mortality yielded no significant effects, for the low-dose cohort, P_β was small and the slope negative, indicating a high probability that as the underlying risk of mortality increased the log odds mortality under steroid treatment decreased (Table 5). The removal of the CSG study [32] attenuated the negative slope of the line. In the risk difference metric, the intersection of the (meta)regression line with the line of null effect ('cross-over' point) occurred for age at 62 years and for control-arm mortality at 44%.

The complications of therapy were secondary infections, gastro-intestinal bleeding and steroid-induced hyperglycemia. No overall or low- or high-dose effects were demonstrated for any of the pooled endpoints (Table 5).

The considerable heterogeneity of the high-dose cohort (τ = 1.00, 95%CrI = 0.42 to 1.89) was diminished by the removal from analysis of the Schumer study [31] (Table 5), an effect previously noted by Minneci and colleagues [7]. This also lead to a high probability of harm in the high-dose studies (P = 89.3%) although the CrI for the OR still included 1. Removal of the CSG study [32] from analysis resulted in reduced heterogeneity among the low-dose studies (Table 5) and a high probability of benefit in the low-dose studies (P = 5.8%) although the CrI for the overall OR just included 1.

In the risk difference metric the absolute risk difference (treatment versus control) for nine trials in the low-dose cohort was -0.047(95%CrI = -0.197 to 0.077; P(RD > 0) = 21.9%); and for eight trials (CSG trial excluded [32]) was -0.072 (95%CrI = -0.202 to 0.018; P(RD > 0) = 5.3%), similar to the 6.6% reported by Annane and colleagues [48]. The mortality OR in the predictive distribution (from eight trials) was 0.703 (95%CrI = 0.156 to 2.198; P(OR > 1) = 19.9%). For hypothesized studies of size 2,000 and 4,000 patients, the mortality ORs were predicted to be 0.724 (95%CrI = 0.184 to 2.108) and 0.726 (95%CrI = 0.184 to 2.096), respectively. The Bayesian predictive P-value, reflecting the inconsistency of the CORTICUS study [8] with the remaining trials (n = 7; CSG trial excluded [32]) was 0.074.

Secondary outcome analysis was beset by selection bias in reporting [65], as witnessed by study numbers in Table 5; parameter estimates may be biased under such circumstances. The study list addressing low-dose corticosteroid mortality efficacy (n = 9) included a single study [32] in 1963, the others being from the period 1996 to 2005 (Figure 2). Plausible estimates of current therapeutic efficacy would suggest analysis excluding the former study, the result of which was to reduce heterogeneity of the mortality effect by 40% and to reveal a probability of corticosteroid efficacy of 94.2% (Table 5). The single-investigator single-centre Schumer study, conducted over a prolonged eight-year period, has been previously subject to substantive critique [7] and recent cautions regarding extended recruitment time [66] and inference from single-center studies [67] merits its consideration as an outlier.

That the inclusion of the large but null-effect CORTICUS trial [8] in the current meta-analysis did not extinguish a probable treatment effect deserves comment. The impact of the single large trial is undoubted, but the evidence produced by such a trial may be "less reliable than its statistical analysis suggests" [68]. We adopted a random effects methodology [69] in the presence of moderate between study heterogeneity (τ, Table 5); under these conditions large studies may have little impact upon a meta-analysis [70] and there may be virtue in (clinical) heterogeneity [71]. The degree of asymmetry of the contour-enhanced funnel plot in the low-dose cohort (see Results, Mortality outcome, above) raises concerns about a random effects methodology [69], but there was no quantitative evidence of small-study effects (at the 0.1 level) and the number of studies was small. In the presence of sparse data and moderate heterogeneity (Table 5), the interpretation of funnel plot asymmetry is problematic [34, 72] and exploration of the reasons for such heterogeneity is the preferred analytic focus [34].

With respect to the efficacy of corticosteroids in severe sepsis and septic shock, the divergent positions represented by the Annane and colleagues [36] and CORTICUS [8] trials remain unresolved. Two recent (calendar year 2009) updates [48, 73] of previous meta-analyses [6, 7] also merit comment. Both of the updated meta-analyses, using frequentist methodology, found efficacy of low-dose prolonged corticosteroids with respect to the mortality effect, Annane and colleagues [48] found a relative risk of 0.84 (95% confidence interval (CI) = 0.72 to 0.97; P = 0.02) and Minneci and colleagues [73] found an OR of 0.64 (95% CI = 0.45 to 0.93; P = 0.02), and shock reversal, the latter effect consistent with the estimates of the current study (Table 5). Study inclusions in these meta-analyses differed and were not the same as in our meta-analysis, which adopted a rigorous exclusion policy (Table 1). The frequentist meta-regression methods used by both meta-analyses [48, 73] to estimate the risk-related treatment efficacy of steroids are problematic [17, 24]. Although such methods may identify putative risk related treatment effects in meta-analyses they fail to allow for both regression to the mean (the difference between outcome and baseline being correlated with baseline) and the stochastic nature of the control rate (regression dilution bias). The stochastic characteristic of the control rate is also not addressed as the expected response in (ordinary) linear regression is conditional upon independent (fixed) variables and there is no inherent accounting for the random error in estimation of this control rate. Such problems are overcome by the use of Bayesian methods [17, 24].

Both meta-analyses were judicious in their conclusions about treatment efficacy and this was reiterated by an accompanying editorial [74]. However, neither study was able to attend to this uncertainty in a tangible manner. This is precisely what our Bayesian analysis quantifies: what was the probability of treatment efficacy. For example, our analysis demonstrated that the probability of adverse mortality outcome with low-dose corticosteroids (outlier excluded) was 5.8% (Table 5). The omission of such a probability statement cannot be justified by an appeal to "the nominal P values for these outcomes were very close to 0.05...." [48]. We have previously cautioned the against interpretation of 95% CI (and associated frequentist P values) as probability statements [75]. Furthermore, neither meta-analysis reported exploration of estimates from a predictive distribution, which may be considered as a more appropriate future treatment summary than the mean effect [18]. Such a capacity recommends Bayesian methodology, although meta-analytic prediction intervals, which address the "...dispersion of the effect sizes..." are computable from a frequentist perspective [76]. With respect to reservations expressed regarding the status of the CORTICUS study [29, 74], we found no compelling evidence (Bayesian predictive P-value 0.074) that this trial was inconsistent with the remaining (n = 7) trials.

Continued controversy and conventional wisdom [77] would appear to mandate the conduct of a large-(mega)-trial of this therapy in well-defined patient subsets; an absolute treatment effect of 7.2%, control arm risk of 54% and 90% power would suggest a total patient number of greater than 2,000. This being said our predictive estimates were unable to suggest efficacy for future 'large' trials, albeit the trial base from which these estimates were made was small.