Accuracy and precision of calibrated arterial pulse contour analysis in patients with subarachnoid hemorrhage requiring high-dose vasopressor therapy: a prospective observational clinical trial

Pulmonary arterial thermodilution has long been considered the clinical gold standard for the measurement of cardiac output (CO). Concerns about the inherent risks of pulmonary artery catheterization have driven the development of less invasive devices for monitoring CO [1] of which calibrated arterial pulse-contour analysis (PCCO) as implemented in the PiCCO™-device (Pulsion Medical Systems, Munich, Germany) has become increasingly popular [2]. Meanwhile, the accuracy of pulse-contour derived cardiac output (PCCO) measurements has been tested in a variety of validation studies [3‐9]. However, these studies are subject to different limitations. First, the vast majority of validation studies has been performed in cardiac surgical patients receiving if any, only minimal-to-moderate pharmacological hemodynamic support [3‐5, 9]. Hence, there are a paucity of data on the reliability of PCCO monitoring in situations with significant hemodynamic instability and in settings not related to cardiac surgery. Second, most of the studies assessed the validity of PCCO measurements by analyzing the correlation, bias and the limits of agreement with the chosen reference method [10]. In contrast, the percentage error as an important statistical measure was only reported in a minority of validation studies [11]. Moreover, as the total percentage error is a composite of both the tested and the reference method, a true interpretation of validation studies is only possible if the precision of the PCCO technique and the reference method is described separately [12]. To the best of our knowledge, the precision of PCCO measurements has not yet been reported [7, 8].

Furthermore, only a minority of validation studies addressed CO trending, with few investigators reporting concordance rates as an indicator of reliable trending detection [13‐15]. No data exist, however, analyzing the concordance between PCCO measurements and a reference method. Recently Critchley et al. suggested polar-plot analysis as the most valid method to assess trending ability [16], and this statistical technique has not yet been applied to PCCO measurements.

In the present study therefore, we analyzed the validity of PCCO measurements by comparison with intermittent transpulmonary thermodilution CO (TPCO) measurements. The study was performed in neurosurgical patients who, following the actual guidelines for the treatment of delayed cerebral ischemia (DCI) subsequent to subarachnoid hemorrhage (SAH), were treated with extensive vasoactive support (evidence level: Class I, Level B) [17, 18]. In the presence of vasoconstriction, pulse-contour analysis has been recently reported to exhibit insufficient accuracy [19‐22]. For the analysis of the agreement with the reference method, we employed the current statistical gold-standard methods, including the separate quantification of the precision of both the PCCO and the TPCO measurements. To investigate the trending ability of PCCO we determined the concordance as recently suggested by Perrino et al. [23, 24] and additionally employed the polar-plot technique [16].

A total of 20 patients were included in this study (Figure 1). Demographic and biometric data of the included patients are presented in Table 1. Hemodynamic data obtained at each time point are depicted in Table 2. Except for MAP at T₄₈, there were no significant changes during the observation period.

From the 20 patients included, a total of 136 sets of CO measurements were available for comparison of TPCO and PCCO. Due to technical problems, CO data pairs were available only for the first 24 hours in one patient, and in two patients only for the first 48 hours. TPCO ranged from 5.2 to 14.3 l · min^-1 and PCCO ranged from 5.0 to 14.4 l · min^-1 (Table 3).

Linear correlation analysis showed acceptable correlation between TPCO and PCCO only for absolute values (Figure 2). The correlation for the percentage changes in CO between each time point was poor (Figure 3).

A detailed statistical analysis of the comparison of TPCO and PCCO measurements is shown in Table 3, including the precision of the reference technique. For all data pairs, Bland-Altman analysis revealed a bias of −0.03 ± 0.82 l · min^-1 and a limit of agreement of 1.62 l · min^-1 (Figure 4), resulting in an overall percentage error of 18.8%. The precision of all PCCO measurements was 17.8% (Table 3).

Linear correlation analysis revealed no correlation between the bias between TPCO-PCCO and SVR (Figure 5).

Trending ability of PCCO as analyzed by the concordance method is shown in Figure 6. It reveals a concordance rate of 74% with an exclusion zone of 15%. Without applying an exclusion zone the concordance decreased to 67%. Figure 7 shows the results of polar-plot analysis. In our study (mean CO = 8.6 l min^-1) only 69.6% of the data lie within the 10% band, suggesting poor trending ability.

PCCO measurements using the PiCCO technology is a well-established and increasingly popular method of minimally invasive hemodynamic monitoring. The validity of PCCO measurements has been studied extensively [3, 5, 6, 9]. However, most of these validation studies have been performed in cardiac surgical patients requiring only low-to-moderate doses of vasoactive support [3, 5, 6, 9]. In contrast we analyzed the validity and accuracy of PCCO measurements in a setting outside of cardiac surgery and as recently demanded [31], in a select patient group under extreme cardiocirculatory conditions, that is, in neurosurgical patients requiring high-dose vasopressor support for the prevention/treatment of DCI due to SAH.

The results of our study demonstrate that in this patient population, CO measurements by calibrated arterial pulse-contour analysis revealed a percentage error of approximately 20% for agreement with the reference technique. Likewise, detailed statistical analysis demonstrated the precision of PCCO measurements to be below 20%. These results allow the consideration of the technology of PCCO measurements as clinically interchangeable with the reference technique, at least in this specified patient population.

Until today there is no consensus on the most appropriate statistical methodology for the validation of continuous CO monitoring-techniques [32]. In the first step, we used the method introduced by Critchley and Critchley [11] for the analysis of the agreement between PCCO measurements and TPCO as the reference technique. These authors originally suggested that the alternative method should have an equivalent precision to the chosen reference method to postulate interchangeability of the two methods. Critchley and Critchley proposed that the reference method has to yield a precision of approximately 20%. Hence (see also Equation 4 in Materials and methods), the percentage error derived from the Bland-Altman analysis should be <28.3% (or, as suggested by several authors, <30%).

As recently discussed, the rigid application of the ±30% cutoff for the percentage error can potentially hide important information as two separate levels of precision contribute to it, of which only the combination adds up to the cutoff value [12, 25]. Hence, a true interpretation of the total percentage error is only possible if the precision of each method is reported separately. In our study, both the precision of the TPCO (<3%) and the PCCO measurements (<18%) was lower than the precision of 20% as originally suggested by Critchley and Critchley, resulting in an overall percentage error <20%.

Analyzing the trending accuracy of the test method compared with the reference method revealed a concordance rate of 74% with a 15% exclusion zone. According to a recent overview by Critchley et al. [16], hence, the trending ability of the PCCO measurement can be considered poor. Because there is no consensus on an appropriate exclusion zone, and several different exclusion zones have been applied in the literature [14, 23, 33, 34], we also performed the concordance analysis using exclusion zones of 0%, 5% and 10%, respectively. This allowed the inclusion of substantially more data points into the analysis. However, narrowing the exclusion zone did not result in an improvement of concordance as expressed by concordance rates of 67%, 68% and 63%, most probably due to the fact that by this approach more central data points, and hence, more statistical noise, were included in the analysis. It has to be noted that the spread of percentage changes also affects the concordance rate. In our study, the majority of ΔCO values were rather small, which may have resulted in skewed distribution of the data points and might be associated with false estimation of the concordance rate. Therefore, we additionally performed a polar-plot analysis as recently suggested by Critchley et al. [16] that allows us to account for both the magnitude of the underlying CO changes and the degree of agreement. Also this analysis yielded an insufficient trending ability for the PCCO technique.

Owing to the underlying calculation algorithms, PCCO monitors have to be regularly recalibrated to obtain stroke volume by an independent reference method [35‐37]. However, as yet no consensus exists on the appropriate recalibration intervals for different patient populations. In our study, no major changes in either cardiac preload (as reflected by ITBV) or vascular tone (as roughly reflected by SVR) occurred throughout the observation period. In this situation, even recalibration intervals of up to 24 hours were not associated with a significant loss of accuracy for PCCO measurements. On the other hand, it is well-known that PCCO measurements in situations of hemodynamic instability with acute changes of vascular tone are often unreliable unless short recalibration intervals down to 1 hour are employed [35‐37]. Of note, more frequent recalibrations also allow updating of relevant hemodynamic information drawn from other thermodilution-derived variables [38].

Another interesting finding of our study was that the accuracy of PCCO measurements was not influenced by SVR, which is in contrast to the cardiac output measurements derived by auto-calibrating pulse-contour analysis, as recently reported by us using the same study design, and by other groups [20‐22, 25].

The present study has several limitations. Only a highly selected patient-collective was included in the study, so that extrapolation of the results to other patient populations is hardly possible. Moreover, all of our patients exhibited normal to supranormal cardiac outputs. The accuracy of PCCO measurements in patients with low cardiac outputs could therefore, not be analyzed and remains to be investigated. Finally, despite the high-dose vasopressor therapy all patients were in a hemodynamically stable state in which no major changes of cardiac preload or vasotonus occurred.

In neurosurgical patients requiring extensive vasoactive support, CO values obtained by PCCO showed a percentage error of <20% for the agreement with TPCO measurements as the reference technique. This error is commonly regarded as a criterion for method interchangeability. The precision of calibrated CO measurements was clinically appropriate and independent of SVR. However, owing to the poor trending ability of the PCCO device, caution is warranted when basing hemodynamic management solely upon the results of the pulse-contour analysis rather than performing frequent TPCO measurements.