Diagnostic accuracy of point-of-care testing for acute coronary syndromes, heart failure and thromboembolic events in primary care: a cluster-randomised controlled trial

We randomised 68 primary care practices in Zurich County (Switzerland) to diagnostic aid from a POCT analyser (n = 33 [39 physicians]; POCT group) or to conventional diagnosis employing best clinical practice (n = 35 [40 physicians]; controls). We randomised only practices at least 8-10 km from Zurich and Winterthur where laboratories with specialised diagnostic systems are less available.

All patients presenting with potentially cardiovascular chest pain or symptoms between May 2006 and August 2007 were invited to participate. Patients gave written informed consent and the study received approval from the cantonal ethics committee (Kantonale Ethikkommission Zürich), in line with the Declaration of Helsinki (1996) and Good Clinical Practice guidelines. Non-inclusion criteria were refusal of consent, presentation >5 days after symptom onset, recent anticoagulant treatment, severe renal dysfunction and cancer therapy. Rationales for the exclusion criteria were the normalisation of the cTnT level five days after ACS, the fact that cTnT may be increased even in the absence of clinically suspected acute myocardial ischemia in patients with renal insufficiency, and the unpredictable effect of anticoagulant treatment and cancer therapy on the biomarkers' concentration [12, 13].

Physicians examined all patients before making a working diagnosis of ACS, HF, TE, musculoskeletal or "other" (specified) problems based on the patients characteristics, medical history, symptoms, physical findings and, in the POCT group, the biomarkers (it was not mandatory to analyse all biomarkers: the physicians had the possibility to choose if and which biomarker test was necessary). At follow-up 3 weeks later the same physician reviewed the working diagnosis. The follow-up diagnosis was defined as the confirmed diagnosis. Follow-up data of patients requiring additional specialist visits or hospitalisation were provided by specialist or hospital reports.

POCT practices received a bedside Cardiac Reader^® (Roche Diagnostics, Switzerland), a 3-in-1 device that determines cTnT, NT-proBNP or D-dimer in heparinised venous whole blood within 8-12 min. Measurement was quantitative for each parameter over the ranges 0.05-2.00 ng/ml, 60-3000 pg/ml and 0.1-4.0 μg/ml, with validated positive/negative cut-offs of 0.1 ng/ml, 125 pg/ml and 0.5 μg/ml. Instruction to the POC-instrument always was performed by the same specialist from Roche Diagnostic. All GPs who received a POCT device received advice on the interpretation of test results. Test quality was monitored using the internal and external quality controls required by Swiss federal law and the Swiss Commission for Quality Assurance in the Medical Laboratory (QUALAB) [14].

Data were analysed using SPSS 14.0 for Windows. Intergroup comparisons of categorical data were performed using univariate logistic regression; standard errors and p values were adjusted for the effect of clustering utilising a generalised estimating equations approach. We detected an interclass correlation coefficient of 0.073 for the main binary endpoint of a correct versus incorrect working diagnosis.

To evaluate diagnostic test quality and performance we plotted receiver operating characteristic (ROC) curves, defined as plots of test sensitivity on the y axis vs. 1-specificity on the × axis. The area under the ROC curve (AUC) combining sensitivity and specificity was used to measure overall diagnostic test performance and was interpreted as the average sensitivity value for all possible specificity values.

Of the 369 patients recruited, 218 (59%) were enrolled in the POCT group (7 ± 10/practice) and 151 (41%) in the control group (4 ± 4/practice). Characteristics and blood parameters were similar in both groups (Table 1).

The interval between symptom onset and baseline visit was much longer in the POCT group because some (mainly POCT) patients presented over 5 days after symptom onset (non-inclusion criterion). We contacted the practices for specific explanations. In almost all cases the patients had visited the physician in the previous weeks/months (> > 5 days) with similar problems. Due to new or exacerbated symptoms (onsetting in the previous 5 days) they revisited their physician who reported the date of their first or previous visit incorrectly. After patients with incorrect symptom onset were excluded, the average interval between symptom onset and baseline visit became similar in the two groups.

Many patients reported mixed sensations of pain or discomfort. Both groups had high prevalence of acute chest pain, tightness, pressure or squeezing, dyspnoea and heartburn-like sensations. Other symptoms were relatively rare (Table 2). The groups statistically differed in rates of acute chest and calf pain.

The most frequent pre-existing diagnoses were hypertension, diabetes and angina. Only the proportion of patients with previously diagnosed HF differed substantially between the groups (Table 3). Pathological electrocardiograms (ECGs) were more prevalent in controls. Reasons for hospitalisation in the previous 12 months were diverse and only partially cardiovascular.

Some working diagnoses differed in frequency between the groups (Figure 1). In the POCT group ACS was less frequent, and HF more frequent, than in controls (p = 0.03 and p = 0.04). TE was similar in frequency (p = 0.81), as were musculoskeletal problems (p = 0.34) and "other" diagnoses (e.g. stable angina, pleuritis, psychiatric problems, viral infection; p = 0.97).

However, confirmed diagnoses did not differ in frequency (Figure 2). Intergroup ACS and TE were similar (p = 0.87 and p = 0.93), while HF remained substantially, but non-significantly, more frequent in the POCT group (p = 0.08). In both groups, two-thirds of patients had musculoskeletal problems (p = 0.49) or "other" diagnoses (p = 0.57).

In the POCT group 165/218 working diagnoses (76%) proved correct vs 90/151 (60%) in controls (p = 0.002). Seventy percent of the working diagnoses of ACS, HF and TE proved correct in the POCT group vs 45% in the controls (p = 0.002), with false-positives in 29 (30%) patients vs 40 (55%) patients, respectively. In contrast, correctness of the remaining working diagnoses (musculoskeletal or "other" problems) did not differ: 80% in the POCT group vs 73% in the controls (p = 0.31).

Except for ACS, where sensitivity was higher in controls, working diagnoses were more sensitive in the POCT group (Table 4). However, specificity was similar in both groups, except for ACS, where it was higher in the POCT group. NPVs of the ACS, HF, and TE diagnoses were near-identical. Overall these results confirmed greater diagnostic accuracy in the POCT group.

Individual test sensitivities and NPVs were generally higher when assessed using the confirmed diagnoses. Specificities, in contrast, differed only marginally.

The diagnostic power of the cTnT test was higher when assessed using the confirmed diagnoses: sensitivity and NPV were 17% and 6% higher (Table 5). Seven patients had a false-negative result and risked a wrong diagnosis. Two had a history of myocardial infarction and were polymedicated (statins, aspirin, diuretics, β-blockers); based on their history and medication they were correctly diagnosed with ACS. Another had a history of angina and was polymedicated: recurrent angina was suspected. A patient presenting 5 hours after symptom onset was diagnosed with stable angina. The final three patients were diagnosed correctly from their symptoms.

The diagnostic power of the NT-proBNP tests was high: all patients with HF also had a positive NT-proBNP, and while none of those with a negative NT-proBNP had HF. Specificity was 72%.

The D-dimer test had a sensitivity of 93%, specificity of 78% and NPV of 99%. The single false-negative was in a 65-year-old man with a history of cancer who was correctly diagnosed with deep vein thrombosis from his symptoms.

The cTnT ROC curve based on the confirmed diagnoses was somewhat oddly shaped because physicians often recorded the result simply as positive or negative (< 0.1 ng/ml). In such cases positive and negative cTnT values were arbitrarily entered as 0.1 ng/ml and 0.0 ng/ml. Very high test specificity accounted for the measured AUC, 82% (Figure 3). The NT-proBNP and D-dimer ROC curves were more regular, achieving high sensitivity and specificity, with AUCs of 94% and 93%.

In previous studies, atypical or absent chest pain has been associated with failure to recognise ACS by either physician or patient in up to one third of cases [15]. Our study shows a reverse tendency: the number of working diagnoses of ACS was unexpectedly high, in particular in controls. Perhaps the study topic alerted physicians to ACS as a possible cause of chest pain, causing them to overestimate its incidence, in particular in the unassisted control group. Many working diagnoses were overturned on follow-up, resulting in very high sensitivity and low specificity. In contrast, the POCT group suspected fewer ACS, resulting in high sensitivity and specificity. Despite two false-negatives in patients wrongly suspected of having stable angina, POCT was more accurate in stratification: ACS was correctly ruled in/out in 92% of POCT patients vs 78% of controls.

Overall, these findings support the notion that when combined with clinical symptoms and ECG screening, troponin assay improves diagnostic accuracy, in line with American College of Cardiology/American Heart Association guidelines on managing patients with unstable angina or NSTEMI [16]. Except for the sensitivity results discussed above, our data reflect the 94% sensitivity and 97% NPV of the cTnT test for ACS reported by Lüscher et al. [17] in 92 patients with chest pain, and the 100% sensitivity, 42% specificity and 100% NPV reported by Fehr et al. [18] in a cross-sectional study of asymptomatic haemodialysis patients. In our study, the high AUC value of the cTnT ROC curve, a measure of overall quality, was largely due to the very high test specificity. To rule out ACS on the basis of cTnT, a minimum of 5 hours (time to increase above the reference range) and a maximum of 5-7 days (time for biomarker level normalisation) must normally elapse after symptom onset. Table 1 confirms that most patients complied with these time limits.

The cTnT assay itself yielded seven false-negatives. Most patients concerned (5/7) were correctly diagnosed from their history and symptoms. Recurrent angina was suspected in the other two patients. High drug consumption in three patients, and the short interval (5 h) between symptom onset and presentation in one patient, may partially explain the false-negatives. Misuse of the POCT device may account for the other cases. That the same practice made 4/7 false-negative diagnoses supports this explanation, highlighting the importance of adequate training on POCT devices. In our study, training on the POCT instrument always was performed by the same Roche Diagnostics specialist, but some staff members were instructed by their colleagues only.

HF is a common and increasing public health problem [19]. Diagnosis in the primary care setting can be difficult and incorrect in up to 70% of cases [20]. Echocardiography is the gold standard for identifying and confirming left ventricular systolic dysfunction [21] but is relatively expensive and generally beyond the resources of general practitioners [22, 23]. Even where available, its appropriate use requires extensive training and experience.

In our study, diagnosis of HF in the POCT group showed high sensitivity, specificity and NPV. Sensitivity in the controls was relatively low, but both specificity and NPV were high. These results indicate that HF was better identified in the POCT group, presumably because of the NT-proBNP data. Assay sensitivity and NPV were perfect: all patients with HF had elevated NT-proBNP values, whereas HF was correctly excluded in all patients with normal values. The very high AUC of the NT-proBNP ROC curve (Figure 3) reflects the test's high diagnostic accuracy. These results confirm earlier studies: the NZ prospective Natriuretic Peptides in the Community Study, a randomised controlled trial of the effect of NT-proBNP on HF diagnostic accuracy in primary care, showing significant improvement in the BNP group over controls [24]; a UK study in 306 primary care referrals for suspected HF emphasising the high NPV of the NT-proBNP assay (ECG had no additional predictive value) [25]; and the prospective 600-patient N-terminal PRo-BNP Investigation of Dyspnea in the Emergency department (PRIDE) study which found the test with its 99% NPV to be a valuable addition to standard clinical assessment for identifying and excluding acute HF: a positive NT-proBNP was the strongest independent predictor of a final positive diagnosis of HF.

TE includes PE and DVT, which are closely interrelated since 90% of symptomatic PE arise from leg vein thrombi [26]. It's well recognised that TE cannot unequivocally be diagnosed from the history and physical examination alone, even in high-risk patients [27]. The recent development of non-invasive D-dimer blood tests with a very high NPV has markedly enhanced the accuracy of diagnosis [28].

In our study, the sensitivity, specificity and NPV of the TE diagnoses were good in both groups, making it difficult to determine whether diagnoses in the POCT group were more accurate. The D-dimer assay showed moderate specificity but very high sensitivity and NPV. The only patient with a false-negative D-dimer was a 65-year-old man with a history of cancer who was correctly diagnosed with a DVT based on his symptoms. The NPV confirmed the assay's power in excluding TE. While a negative D-dimer safely excludes TE, patients with a positive result should still be screened for PE and DVT, in particular in the presence of typical symptoms. Our results are consistent with those of other studies, e.g. Leclercq et al. or Schutgens et al. [29, 30].

Some limitations of the study require discussion. For example, we confined ourselves to Zurich County for practical reasons; whether the results can or cannot be extrapolated to less urbanised areas remains to be determined.

A second limitation was patient recruitment. Initially, recruitment was clearly higher in the POC group. We put this down to a difference in study awareness: the device on their premises made POCT physicians more study-aware. Control physicians, on the other hand, had only the study protocol and the questionnaires to remind them, and perhaps forgot to recruit; after we initiated regular telephone reminders, recruitment rates became almost equal in both groups.

The follow-up diagnoses were another limitation. In principle, they should have been performed by independent blinded assessors but this was not feasible. For practical and data protection reasons, they were made by the same physician who was primarily consulted by the patient. We are aware of the potential for bias, e.g. due to possible underreporting of incorrect baseline diagnoses leading to a false high rate of correct baseline diagnoses or a false low difference between study arms. On the other hand, for patients referred for further diagnostic work-up (including all patients at potentially high cardiovascular risk), GPs received a written report on the second-stage assessment as well as information on further clinical management, which substantially reduced the risk of bias.

A fourth limitation concerns possible underestimation of the number of HF diagnoses in the controls. Not only were there significantly more pre-existing HF diagnoses in the POCT group, but the total number of newly identified HF patients was also greater. We assume that the POCT physicians used the NT-proBNP value as diagnostic criterion to identify more patients with NYHA class I-II HF. Indeed, early stages of HF are often asymptomatic and difficult to identify without accurate screening. Some patients with mild HF may not have been diagnosed as a result, and this tendency would have been stronger in the controls.

As shown in Table 5 which compares biomarker performance in the POCT group to the working and confirmed diagnoses, it seems that in some cases the GP decided to ignore the results of the biomarker tests. It is unknown if there was a lack of confidence in the biomarkers, if there was a problem with the interpretation of the results, or if GPs gave more importance to physical findings, symptoms, and patient history. In any case, this finding emphasises the importance of training and continuing medical education.

Health-economic and cost-effectiveness studies of cardiovascular risk stratification using POCT are rare. Some studies have analysed the costs and benefits of a single biomarker for a defined disease. Thus Nielsen et al. [31] concluded that NT-proBNP testing in primary care patients with dyspnoea halved the need for echocardiographic screening. Gustafsson et al. [32] reported that a normal NT-proBNP effectively ruled out left ventricular systolic dysfunction in primary care patients referred for echocardiography, thereby avoiding unnecessary further investigation. Siebert et al. [33] found that NT-proBNP measurement improved patient outcome, reduced echocardiography use by 58%, prevented 13% of hospitalisations and reduced hospital stay by 12%. Similar studies have been performed for other biomarkers, but a more general evaluation is needed of the possible benefits of POCT for ACS, HF and TE in primary care.