Part 3: Evidence evaluation process: 2010 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science with Treatment Recommendations☆

Worksheet template

As has been the process since the evidence evaluation for Guidelines 2000,⁵ volunteers from around the world were asked to perform systematic reviews of the literature. These reviews were conducted according to standardised instructions in completing the evidence worksheet template, and these worksheets were subsequently reviewed in a detailed iterative process (see http://www.ILCOR.org).⁶ A revised worksheet template (based on that used for the C2005 process) was created and accompanied by

Identifying the questions to ask

As with the previous evidence evaluation processes, the specific questions to be asked were informed by priorities identified by task forces and individual councils/organisations, review of the research gaps analysis,⁷ and a thorough systematic approach called “evidence mapping” based on the previous guidelines (http://www.evidencemap.org/about).⁸

The questions were allocated to worksheet authors by the relevant task forces, with 2 authors initially allocated to each worksheet. All potential

Formatting the questions (PICO)

The questions for each worksheet topic were structured into a standardised format (PICO: Population/Patient, Intervention, Comparison, Outcome; http://www.cebm.net/?o=1036).¹⁰ This process provided a clear statement about the components of the proposed topic, which facilitated the literature search, guided the accurate allocation of the LOEs, and identified areas of overlap (e.g., in-hospital versus prehospital, during arrest versus after arrest, cardiac arrest versus hemodynamic instability,

Clarifying the search strategy

Generic instructions were provided on the types of search strategy to be considered and databases to be searched. The initial search strategy was submitted electronically via the internet for initial review by one of the task force chairs and one of the worksheet experts. This was to ensure that the search appeared to be on track (e.g., addressing the desired question, reasonable combination of search terms), and the search strategy was subsequently returned to the author with comments. The

Selecting studies for further review

The authors were asked to review the titles and abstracts of all articles identified by their preliminary searches and assess the relevance of the articles to the question being asked. The worksheet author then retrieved the appropriate studies for further comparison with the previously developed inclusion and exclusion criteria. This allowed a reproducible final decision on articles to include in the worksheet.³ Inclusion of all relevant evidence including animal and manikin/model studies, as

Allocating levels of evidence

The LOEs used by any review process are a tool to create order and simplicity from the heterogeneity of published studies. There are many published classifications of LOEs. The international resuscitation community used seven LOEs in the preparation of the 2005 CoSTR.11, 12 For the 2010 CoSTR, we reviewed the literature on available classifications (Evidence Report/Technology Assessment No. 47, Systems to Rate the Strength of Scientific Evidence; Agency for Healthcare Research and Quality

Allocating quality

The process for assessing methodological quality was also reviewed. Many different techniques had been proposed (Evidence Report/Technology Assessment No. 47, Systems to Rate the Strength of Scientific Evidence; AHRQ publication No. 02-E016, available at: http://www.ahrq.gov/clinic/epcsums/strengthsum.htm).4, 13 None of these approaches (largely involving checklists) are appropriate for all settings, and the use of different approaches could result in apparently conflicting results. Several

Tabulating the evidence

All of the evidence identified and evaluated was displayed in standardised evidence tables. In each of three tables (supporting evidence, neutral evidence, and opposing evidence), studies were displayed according to LOE, methodological quality, and outcomes identified (Fig. 2).

Interpreting the evidence

The worksheet authors were asked to consider the internal and external validity of each study included and then to summarise the information they reviewed under the section entitled “Reviewer's Final Comments.” This is where authors, who had the most intricate understanding of the literature, could succinctly describe the results of their review, start to synthesise the information, tease out the contradictions, make observations, and propose solutions. To facilitate interpretation of the

Identifying the gaps

Worksheet authors were also asked to identify critical gaps in the literature, which were then incorporated into the final CoSTR document.

Iterative review process

Each worksheet was submitted electronically and underwent a rigorous iterative evaluation process. The layers of review before final acceptance of the worksheet included the task force chair and task force (in addition to a “domain leader” in the advanced life support task force), a worksheet expert, and the evidence evaluation expert. When the worksheet was finalised, it was posted on the Internet for public comment (http://www.americanheart.org/ILCOR; Fig. 3).

In general, multiple worksheets

Controversies in the evidence evaluation process

Four main issues arose from the evidence evaluation process. First, a recurrent theme was the allocation of an LOE based on study type rather than whether the study addressed the specific PICO question. Randomised controlled trials involving a therapeutic intervention in patients who were not in cardiac arrest were often initially allocated an incorrect LOE. If the worksheet PICO question specified the population as patients who were in cardiac arrest, then all randomised controlled studies in

Summary

The C2010 evidence evaluation process used the best evidence on critical appraisal to develop a unique process to incorporate all the peer-reviewed, published science underpinning resuscitation and emergency cardiovascular care. A detailed systematic, multilayered, iterative review of the individual topics (worksheets) has informed the final product: the 2010 CoSTR.