A blind clinical evaluation committee should, in theory, make data of a randomized clinical trial stronger, not weaker

Excerpt

Sir: Firstly, we would like to thank Dr. Carlet for his comments regarding our article on the role, methodology and results from the clinical evaluation committee in PROWESS [1]. Evidence-based medicine is the art of extrapolating the results of randomized clinical trials (RCTs) into clinical practice; converting efficacy demonstrated in a controlled setting into effectiveness in the “real world”. Clinical evaluation committees (CEC) frequently aid us in making such extrapolations by assessing the quality of a successful study with respect to how well the protocol was implemented, how appropriate was concomitant care (e.g. adequacy of antibiotic therapy in sepsis studies), and how patient characteristics may have influenced the observed treatment effect. Dr. Carlet appropriately points out the value of these committees. However, Dr. Carlet’s suggestion that CEC analyses make “data stronger”, or that the optimal cohort defined by the CEC “should be the primary endpoint in RCTs” should be viewed with caution. Sprung et al. found that the mortality of the optimal cohort was better in the anti-TNF, INTERSEPT trial [2], but Dhainaut et al. reported no difference in mortality of the optimal cohort in the anti-PAF trial [3]. Additionally, these analyses, in common with all analyses that assess treatment effect in only a portion of the intent-to-treat population, are subgroup analyses and must be interpreted as such. That is, comparison of outcomes between subgroups should be assessed by interaction testing and not significance testing within a particular stratum. In the PROWESS study, interaction testing for the subgroup analyses referred to by Dr. Carlet indicated that the observed outcomes within each subgroup strata did not differ statistically from each other. In particular, the relative risk reduction associated with the administration of drotrecogin alfa (activated) to patients without underlying conditions, although numerically different, was not statistically different from that observed in patients with underlying conditions (11.7% vs. 27.0%; interaction p=0.09). Additionally, classification of patients into subgroups (e.g. optimal cohort) is based only on data contained in the case report form and not on a review of the patient’s actual medical record. Consequently, the robustness of such subgroups could be affected simply by the amount and type of information collected in the case report form, again confirming the exploratory nature of these comparisons. …