Introduction
Handheld blood glucose meters or department-based blood gas analyzers are currently the preferred methods to measure blood glucose levels in intensive care unit (ICU) patients [
1,
2]. These intermittent glucose-monitoring techniques have variable accuracies [
3] but foremost lack useful trending because of the interval between consecutive measurements. Continuous glucose monitoring (CGM) is suggested to increase practicalities and safety of insulin titration in ICU patients [
1,
4], in particular when targeting normal or near-normal blood glucose levels when hypoglycemic episodes can be expected [
5-
13].
Glucose oxidase technique-based interstitial CGM devices have been used before in diabetic patients outside the ICU setting [
14]. It is uncertain, however, whether interstitial CGM devices are point accurate in critically ill patients [
1]. An altered relationship between blood and interstitial fluid glucose levels during critical illness could affect the point accuracy of interstitial CGM to reflect the blood glucose level [
15]. Several interstitial CGM sensor systems originally designed for non-ICU patients have been tested in the ICU setting in recent years [
16-
28]. Medtronic MiniMed (Medtronic Inc., Northridge, CA, USA) developed the Sentrino Continuous Glucose Management System, an interstitial CGM device that was especially designed for use in critically ill patients. This device was improved from previous models by creating the processor cable and pole-mounted monitor and by four sensing elements designed to increase responsiveness to glucose changes and to limit the influence from drug interactions.
The aim of this study was to test its point accuracy and reliability in a mixed medical-surgical ICU. We hypothesized that the device would provide an accurate reflection of the blood glucose level in ICU patients treated according to a local guideline for blood glucose control targeting blood glucose levels between 90 and 144 mg/dL. In addition, we determined its reliability, including duration of the device start-up, the need for calibration, skips in data acquisition, and number of and reasons for premature disconnections.
Methods
This was an investigator-initiated observational trial. The Institutional Review Board of the Academic Medical Center (Amsterdam, The Netherlands) approved the study protocol (study ID: NL41498.018.12). Medtronic MiniMed provided three devices for the duration of the trial and the necessary sensors but had no influence on study design or study reporting. Patients or next of kin had to provide written informed consent before the start of any study-related procedure.
Study population
Patients were recruited between October 2012 and February 2014 in a 30-bed mixed medical-surgical ICU of a large university hospital (Academic Medical Center). Patients were eligible for inclusion if they were at least 18 years old and had an anticipated life expectancy of more than 96 hours. Patients were excluded from participation if they had a platelet count of less than 30 × 109/L, had participated in a trial testing an investigational product or treatment within the past 30 days, were pregnant, or had a suspected or diagnosed medical condition which in the opinion of the investigators prevented the patient from completing the study.
Glucose control
ICU nurses performed glucose control with insulin by following a local guideline for blood glucose control targeting a blood glucose level between 90 and 144 mg/dL [
29]. Insulin titration adjustments were based on sliding scales. The local guideline for blood glucose control dictated nurses to perform blood glucose measurements at least every 4 hours and more frequently if blood glucose levels were out of range or were expected to change rapidly. For details, see Additional file
1.
During the study, ICU nurses were not allowed to change insulin infusion rate based on the readings by the investigational device. However, they were allowed to perform additional blood glucose measurements if the device suggested rapid changes in the glucose level or when there was a trend toward hypoglycemia.
The investigational device
The disposable glucose sensors of the device were glucose oxidase-based; each sensor had two probes, and each probe had two sensing elements. The individual measurement results were combined and displayed on the device monitor every minute. The signal was transmitted through the processor cable to the monitor. It was a single-patient single-use sensor, which could be used for up to 72 hours. The processor cable was reusable.
The sensor was inserted into the subcutaneous tissue by using two parallel introducer needles. The two needles automatically retracted when the introducer hub was pulled away from the sensor base; the sensor probes remained in the subcutis. Each new sensor needed calibration by using blood glucose levels after insertion and initialization and after 1 hour and 2 hours; thereafter, repeated calibrations were performed every 8 hours.
Study procedures
Sensors were inserted into the subcutis of the thigh. Successive sensors could be used for 72 hours, depending on length of stay in the ICU, but never for longer than 30 days. Arterial blood glucose levels were measured by using RapidLab 1265 blood gas analyzers (Siemens Healthcare Diagnostics, The Hague, The Netherlands), which were used for calibrations of the device. Not only did ICU nurses provide the mandatory calibration blood glucose levels, but also the routinely obtained blood glucose levels (that is, blood glucose measurements which were not requested by the device for calibrations but were taken by the nurses as dictated by the local guideline for blood glucose control) were entered into the device as well. Therefore, these measurements were also used for calibrations of the device. If the device displayed a message requesting an additional non-routine calibration to resolve a sensor performance issue (that is, a ‘Poor Sensor Signal’ alert), the nurses were permitted to disregard manufacturer recommendation and remove sensors rather than enter the requested calibration.
Each day, the place of insertion was photographed and inspected for redness, bruises, and swelling. In case the patient was awake, we questioned the patient whether it was painful. Every item could be scored as ‘none’, ‘minor’, or ‘major’.
Power calculation
We intended to enroll 50 patients to assess accuracy of the CGM device. With 50 patients, we expected to have at least 40,000 subcutaneous CGM device results and at least 1,200 blood glucose level measurements with the RapidLab 1265. Considering previous studies testing point accuracy, we assumed we would have a sufficiently high number of paired samples to enable evaluation of the point accuracy of the device.
Analysis plan
The glucose data collected with each new sensor were downloaded from the device after use in a patient; the arterial blood glucose levels were downloaded from the patient data management system. The arterial blood glucose levels in the patient data management system were compared with the entries for calibrations into the device. In case of an entry error, defined as a difference between the arterial blood glucose level in the patient data management system and the calibration entry of more than 9 mg/dL, the correct blood glucose level was used in the accuracy analysis. The subsequent pairs, though, were excluded from the accuracy analysis since these were influenced by the preceding entry.
For reporting point accuracy, we used analytical and clinical accuracy measures: that is, Bland-Altman plot with bias and limits of agreement (bias ± 1.96 × standard deviation of the bias) [
30], glucose prediction errors, and Clarke error grid analyses [
31]. According to International Organization for Standardization (ISO) criteria, 95% of the paired measurements should be within the glucose prediction error criteria; the consensus is that 95% of the values should be in zones A and 5% in zones B of Clarke error grid analyses. Finally, we expressed the linearity between the device glucose results and blood glucose results by the Pearson correlation coefficient and coefficient of determination, R
2.
In a
post hoc analysis, we also report point accuracy according to the recently published consensus recommendations [
1]. In this round-table meeting of ICU experts in blood glucose control, it was recommended to always report the mean absolute relative difference (MARD) when testing a CGM device, where MARD values should be less than 14%; values of more than 18% should be considered to represent poor accuracy [
32]. We added the MARD as a
post hoc analysis. Furthermore, we analyzed the point accuracy following the recently published surveillance error grid [
33]. For more details, see Additional file
1.
We also reported reasons for early disconnection, defined as the removal of a sensor before 72 hours. For details, see Additional file
1. The time between calibrations using an incorrect glucose value entry and the next calibration was extracted from the total connection time of the device. Definitions of the metrics used to assess device reliability, including those suggested by recent consensus recommendations [
1], are described in Additional file
1.
Statistical analysis
We reported data as mean (± standard deviation) or median (interquartile range, or IQR) where appropriate. To be considered for the statistical analysis, each patient needed to have at least four comparative blood glucose results for accuracy analysis. However, the excluded patients remained included in the reliability analysis.
In a
post hoc analysis, we used a linear mixed model to determine which variables influence the accuracy of the device. In addition, we stratified the accuracy results by diabetic status. For a detailed description of this model, see Additional file
1. Analyses were performed by using R (version: 2.15.1; R Foundation for Statistical Computing, Vienna, Austria).
Discussion
We determined the point accuracy and reliability of a device specifically designed for continuous real-time monitoring of interstitial glucose levels in critically ill patients. The analytic point accuracy of the device was low in a typical cohort of patients from a mixed medical-surgical ICU, according to ISO criteria and consensus recommendations. The clinical point accuracy was low according to Clarke error grid analysis but better according to surveillance error grid analysis. The device had few downtimes, but one third of the sensors were removed prematurely because of sensor- or device-related problems.
The present findings are in line with results from a previous trial testing the same device in cardiac surgery patients [
34]. In that study, the mean absolute relative difference was 12.2% with 95% real-time data. Similar results come from studies testing other devices for interstitial glucose monitoring that were originally designed for use in non-critically ill patients. Those studies were performed in cardiac surgery patients [
21,
24,
35], surgery patients [
26], patients with neurologic emergencies [
27], and non-surgical patients [
16,
22,
25], and only two reported more favorable accuracy results [
21,
22]. Taken together, these data suggest that point accuracy of interstitial glucose monitoring cannot replace blood glucose level measurements.
In contrast to our findings, a previous publication by Brunner
et al. [
18] suggests a better point accuracy of another interstitial CGM device in critically ill patients. This report combined data of two separate trials in medical ICU patients [
19,
36]. The tested device in that study was from the same manufacturer but was not specifically designed for use in critically ill patients. In addition, the sensor was used for up to 72 hours and never replaced. One important difference with the present study was that the sensors were placed exclusively under the skin of the abdomen in patients included in these two trials. In most other trials, sensors were inserted under the skin of the abdomen [
16,
18,
22,
24,
26,
28], thigh [
25,
26], or shoulder [
21]. Reported point accuracies do not suggest superiority of one of these sites. Certainly, there could be other unknown and unreported factors that could have resulted in the differences in performance.
We performed a mixed linear model to determine which factors could have influenced the point accuracy of the tested sensor. Rank order of measurement and presence of a history of diabetes affected the accuracy. The finding that rank order of measurement improved sensor performance is not new [
18] and certainly is not surprising: more calibrations may always increase accuracy of a sensor. A history of diabetes was the most important variable influencing point accuracy and deteriorated sensor performance by 34%. As yet, this effect remains unexplained. It could be that microcirculation alteration in patients with diabetes affects interstitial glucose level. However, in previous studies with interstitial devices, diabetes was not found to be significantly associated with poor sensor accuracy in critically ill and cardiac surgery patients [
16,
18,
28]. Moreover, in a recent study in cardiac surgery patients, an impaired microcirculation did not affect accuracy of two interstitial glucose sensors from two different manufacturers [
28]. The difference found between patients with and without diabetes might also be related to glucose variability. Patients with diabetes will have more glucose variability compared with patients without diabetes. Thereby, when the focus is percentage difference, a greater disparity could be found when variability differences are compared.
It should be stressed that we compared interstitial glucose measurements with glucose levels in arterial blood samples, which are far from comparable. Indeed, the interstitial glucose level is dependent on several factors other than the blood glucose level, such as the speed of glucose diffusion from blood to interstitial spaces, as well as the rate of glucose uptake by subcutaneous cells [
37]. Importantly, these factors are not constant, particularly in critically ill patients. Furthermore, there is a time lag between interstitial glucose and blood glucose measurement [
37]. Studies suggested that the interstitial glucose level decreases before the blood glucose decreases [
37,
38], although this was not confirmed in other studies [
39]. It is probably very difficult, if not impossible, to correct for factors causing a difference between interstitial and arterial blood glucose levels. Moreover, it is unknown whether differences between arterial and interstitial glucose levels are physiological.
Nevertheless, subcutaneous glucose monitoring could have advantages. One potential advantage is that continuous monitoring of interstitial glucose levels enables detection of trends in the blood glucose level [
32]. This could allow earlier responses to a rise or a decline of the blood glucose level. In both cases, knowledge of the direction of the trend may be more valuable than the exact blood glucose level.
It is clear that the tested device can never replace blood glucose measurements. First, initial calibrations are always necessary, as are calibrations every 8 hours thereafter. As nurses were allowed to perform additional blood glucose measurements and as we asked them to insert the values into the investigational device monitor where they were used for additional calibrations, the number of calibrations in this study was higher than mandated. In fact, this could have improved the accuracy of the investigational device: it is possible that with fewer calibrations, point accuracy becomes worse.
Our trial has several strengths and weaknesses. Strengths include the fact that we were able to use the sensors for several days in the participating patients. Moreover, we used accurate blood gas analyzer measurements for comparisons as well as for the calibrations. Furthermore, we were able to test the device in a typical mixed medical-surgical ICU. Weaknesses include the small sample size and the single-center design of the trial. Furthermore, we did not collect as many samples as we expected. A more important limitation of our trial, though, is that the vast majority of blood glucose levels were in a narrow range, preventing us from drawing firm conclusions regarding accuracy in the hypoglycemic range. Although the ICU nurses were not allowed to change insulin infusion rates, they could have anticipated hypoglycemia by performing new blood glucose measurements earlier than dictated by the local guideline for blood glucose control, allowing them to respond earlier to, for example, hypoglycemia. Still, some hypoglycemic events occurred, probably because not all nurses were paying attention to the readings of the investigational device. In addition, nurses could have noted that its point accuracy was not always good, so they could have mistrusted the device readings. Finally, we cannot exclude the possibility that hypoglycemia can occur even with the use of CGM. The latter possibility will be the subject of a planned trial. An accuracy analysis limitation was that the assessment focused on percentage difference comparisons between the continuous sensor and discrete reference points, evaluated by standards meant for discrete measurements for dosing. Another important limitation is that trend accuracy was not evaluated. Trending is the most interesting endpoint but mandates very short intervals (that is, a short as 15 minutes) between blood glucose reference measurements [
32,
40]. Trend accuracy should and will be evaluated in future studies.
Notably, length of stay in the ICU and sensor connection time were far from similar. This was caused by the fact that sensors could not be used before informed consent was obtained. Thus, we may have missed an important phase of glucose control (that is, the first day or days of stay in the ICU). In addition, because of sensor- or device-related factors, one third of the sensors were removed before sensor life ended. This is an important problem for the reliability of the device. However, nurses did not always attempt to solve sensor- or device-related problems that could have been solved. During conduct of the trial, they were always allowed to remove the sensor because of ‘Poor Sensor Signal’ alerts or recurrent alarms. With increasing device-specific experience, it could be that there are fewer early removals.
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Competing interests
RH reported consulting work for Medtronic Inc. and GlySure Ltd. (Abingdon, Oxfordshire, UK) and research support from Medtronic Inc. and OptiScan Biomedical (Hayward, CA, USA). TW reported consulting work for Medtronic Inc., GlySure Ltd., and OptiScan Biomedical. MS reported receiving consultant fees from Medtronic Inc., GlySure Ltd., Edwards Lifesciences (Irvine, CA, USA), and Roche Diagnostics (Basel, Switzerland) and financial support from Medtronic Inc. and OptiScan Biomedical; all fees and financial supports were paid to the institution. Medtronic MiniMed provided three devices for the duration of the study and the necessary sensors but had no influence on study design or study reporting. Medtronic MiniMed was only allowed to check the publication for company proprietary information. No financial support was received for this work. JL, JB, NJ, JH, JF, and ED-L declare that they have no competing interests.
Authors’ contributions
RH and MS contributed to study concept and design, acquisition of data, analysis and interpretation of data, drafting of the manuscript, and critical revision of the manuscript for important intellectual content. JL contributed to study concept and design, acquisition of data, analysis and interpretation of data, and critical revision of the manuscript for important intellectual content. TW, NJ, and JH contributed to study concept and design, acquisition of data, and critical revision of the manuscript for important intellectual content. JB contributed to study concept and design, analysis and interpretation of data, and critical revision of the manuscript for important intellectual content. JF and ED-L contributed to study concept and design and critical revision of the manuscript for important intellectual content. All authors read and approved the final manuscript.