Systematic reviews to evaluate diagnostic tests

Abstract

Diagnostic testing and screening is a critical part of the clinical process because inappropriate diagnostic strategies put patients at risk and entail a serious waste of resources. It is being increasingly recognised that absence of clear summaries of individual research studies on the repeatability, accuracy and impact of tests, which are often scattered across many different journals, is a major impediment. Just as the need to develop means to systematically review research assessing the effectiveness of treatments has been pursued over the last decade, so more recently attention has focused on how research on diagnostic tests might also be systematically reviewed. These reviews present a huge methodological challenge. This paper describes the use of a systematic approach to collation, appraisal and synthesis of information contained in the primary literature about accuracy of diagnostic strategies.

Introduction

Clinicians are expected to make a diagnosis, to give a prognosis and provide treatment, typically in that sequence. Diagnosis is established by a combination of patient’s characteristics, history, examination and tests, and is the cornerstone of good clinical care. Without accurate diagnosis doctors cannot prognosticate accurately or choose the best treatment. Indeed a wrong diagnosis can harm patients by exposing them to the wrong therapy, while correct prediction of disease allows timely use of effective treatment. This is why clinicians need guidance about efficient diagnostic strategies and why evidence-based medicine clearly includes evidence-based diagnosis.

Lack of clear summaries of the individual research studies on the repeatability, accuracy and impact of tests, which may be scattered across many different journals, is a major impediment in optimising patient management. Just as the need to develop means to systematically review research assessing the effectiveness of treatments has been pursued over the last decade, so attention needs to be focused on how research on diagnostic tests might also be reviewed systematically [1], [2], [3].

Although diagnosis often raises more questions than therapy, primary research on evaluation of tests is generally poor in quality [4], [5], [6], [7]. The methodological deficiencies include biases arising from study design, the use of inappropriate patient groups, the unreliability of tests, lack of blinding and partial verification of outcome. Even when individual studies of good quality exist, they often lack power and require quantitative synthesis to produce reliable estimates of diagnostic accuracy. Authors of primary research studies claim more positive conclusions than can be supported by the rigour of the study design, conduct and analysis [8], [9], [10], [11]. This tendency to exaggerate about the findings of a study may be even greater in diagnostic research.

Although there are many reviews, commentaries and recommendations on diagnostic tests, they offer only limited guidance for practice because few apply scientific strategies to limit bias in their assembly, appraisal, and synthesis of primary studies [12], [13], [14]. There is a need to identify biases in primary diagnostic test research and to synthesise their results using quantitative methods in systematic reviews. At present there is a dearth of such reviews and in this commentary, we will demonstrate how the systematic approach can be used in this area to allow readers to critically appraise published review articles about diagnostic strategies. We do not intend to provide guidance on conducting reviews.

Systematic reviews should be conducted with the same rigour as primary research. Off-the-cuff commentaries by ‘content experts’ on the invitation of journal editors are no substitute. Conducting a research overview is conceptually analogous to conducting an epidemiological survey involving the steps shown in Table 1. A brief description of the basic steps is provided below.