Clinical utility of a non-invasive urine test for risk assessing patients with no obvious benign cause of hematuria: a physician-patient real world data analysis

Full methodological, case and participating physician details were reported in Darling et al. (2017) [12]. Briefly, patients were prospectively recruited and systematically selected to represent a broad patient demographic and clinical spectrum. Three hundred ninety-six physician-patient clinical decisions were evaluated where 12 urologists (participant physicians; Additional file 1: Table S1) with experience in the use and application of Cxbladder in their clinical settings were recruited. Participating physicians were offered honoraria as compensation for the time spent participating in the study. Experience in the use and application of Cxbladder was defined as using Cxbladder more than 10 times in real-world situations. Each participant physician individually evaluated the same 33 case-note clinical referral data from patients presenting with AMH systematically selected from the database of patients enrolled in previous prospective clinical studies of Cxbladder. Patients consented to the anonymous use of their urine sample and clinical information, IRB approval details are presented below.

All patients met the AUA definition of AMH with the possibility of UC requiring a urological work-up, including invasive diagnostic procedures [3]. Each participant physician was asked to review and recommend diagnostic procedures for each case based on the patient’s evaluation from normal referral data. Participant physicians were asked to select a set of urological investigation procedures from a list of investigation procedures recommended in the AUA guidelines [3]. For the purposes of this study and to enable the reduction of any systemic bias, participant physicians were instructed to consider all tests and procedures to be fully funded, with no additional cost incurred by the patient. Participant physicians were not informed of the final UC status of each case.

Participant physicians performed further recommended clinical evaluations for each case twice, firstly in the context of the information derived from Cxbladder Triage alone and secondly when both Cxbladder Triage and Detect were presented. The data format was consistent with the commercially available Cxbladder tests, including a report outcome, explanation of the Cxbladder test methodology and guide to interpreting the results [26, 27]. Outcomes were defined as low probability of UC, ‘negative’, or standard clinical workup, ‘physician-directed protocol’, for Cxbladder Triage and ‘normal’, ‘elevated’ or ‘high’ gene expression for Cxbladder Detect. Elevated and high results in the Detect modality were presented as ‘positive’ for UC for the purposes of this study.

The co-primary endpoints were changes in the total number of diagnostic procedures used over all patient cases and the number of invasive procedures requested following disclosure of diagnostic information from Cxbladder Triage and Cxbladder Triage and Detect. For the purposes of this study, cystoscopy (flexible or rigid) and CT scans (contrast or non-contrast) were defined as invasive diagnostic procedures whereas ultrasound, urine cytology and UroVysion® fluorescence in situ hybridization were defined as non-invasive procedures and tests.

Data were graphically presented as heatmap calculations to aid the visualisation of the results at the individual physician-patient interaction level. The heatmaps show the range of variation in decision making and the change in decisions with and without the addition of the Cxbladder result. The baseline heatmaps provide the total count of procedures at each of the 396 physician-patient interactions in the absence of availability of the Cxbladder result. The change, relative to this baseline, made when the Cxbladder result was disclosed reflects the change in physicians decision making. Heatmap columns represent individual physicians and rows represent individual patients with each row and column intersection becoming a physician-patient interaction. Patients (rows) are grouped by Cxbladder result with colour bar linkage, patient ID and test result. The baseline heatmaps include the actual number of procedures per interaction and the associated change heatmap includes the change relative to baseline per interaction. The colour code in the heatmap is used to indicate extent of the change in the number of tests and procedures used in the physician-patient interaction. An increase in the change is represented in shades of red or a decrease in the change is represented in shades of green or no change relative to baseline is in white. The greater the intensity of colour the greater the change in decisions.

Negative Cxbladder Triage results led to a net reduction in procedures in 79/182 (43.4%) clinical decisions where additional diagnostic procedures had been requested based on baseline data (Fig. 2a+b). This translated into a significant mean reduction in total diagnostic procedures requested per interaction (Table 1). Only one patient with a negative result had a net increase in procedures (Fig. 2b).

Presentation of negative results from Cxbladder Triage resulted in a net removal of invasive procedures in 93/169 clinical decisions (55.0%) where invasive procedures had been requested at baseline (Fig. 3b). Of the flexible cystoscopies and contrast CT scans requested for this group, 40.4% (67/166) and 46.5% (40/86), respectively, were abandoned (Additional file 2: Figure S1 and Additional file 3: Figure S2).

Following presentation of results from Cxbladder Detect for patients who were guided to a physician-directed protocol by Cxbladder Triage, 19/36 clinical decisions (52.8%), where a patient tested positive, resulted in a net increase in procedures, while three clinical decisions (8.3%) resulted in a net decrease (Fig. 2c). Overall, a positive result for Cxbladder Detect resulted in a significant increase in the total number of procedures per interaction (Table 1).

A net 26 invasive procedures were added across 17 clinical decisions for patients who tested positive using Cxbladder Detect, while three procedures were removed in a further three clinical decisions (Fig. 3c). No net change in the number of clinical decisions involving a request for a flexible cystoscopy occurred in this group (Additional file 2: Figure S1), but 9 additional rigid cystoscopies were requested. Furthermore, a net 13 clinical decisions resulted in the addition of a contrast CT scan (Additional file 3: Figure S2). One extra non-contrast CT scan was also requested.

Of the 96 clinical decisions resulting in a physician-directed protocol that subsequently tested normal for Cxbladder Detect, 31 had a net increase, and 17 a net decrease in total procedures (Fig. 2c). However, a normal result from Cxbladder Detect was associated with a significant mean reduction in total diagnostic procedures requested per patient interaction versus baseline (Table 1).

A net reduction in invasive procedures compared with baseline occurred in 32/66 (48.5%) clinical decisions following a normal result from Cxbladder Detect, while procedures were added in 14/96 (14.6%) clinical decisions (Fig. 3c). Overall, 28 (42.4%) fewer flexible cystoscopies were requested in this group (Additional file 2: Figure S1), but three rigid cystoscopies were added. Three fewer contrast CT scans (8.1% reduction versus baseline) and one less non-contrast CT scan were requested (Additional file 3: Figure S2).

Transition probability analyses indicate a de-escalation of procedures following a negative result from Cxbladder Triage (Fig. 4). For example, 72.5% (29/40) of patients with negative results who were allocated a contrast CT scan at baseline had this procedure substituted with ultrasound.

Transition analyses also indicate escalation of diagnostic procedures for patients following a physician-directed protocol result (Fig. 4). All five patients with these results who were allocated an ultrasound at baseline had this procedure substituted with a contrast CT scan. Six out of seven (85.7%) patients allocated a flexible cystoscopy at baseline were escalated to rigid cystoscopy.

In the absence of clear guidance or precedents for assessing the clinical utility of non-invasive diagnostic tests, the investigators considered the study design used here to be the most pragmatic method of evaluating the clinical utility of Cxbladder in a cohort of real-world patients presenting with AMH given the logistical, consistency and ethical challenges in prospectively investigating the selection of diagnostic tests in patients with AMH. Notably, the innovative methodology applied in this study acknowledges and addresses the concerns expressed in the literature by experts in the field regarding the potential risks of using non-invasive urine tests for UC in patients with AMH without risking patient health. Furthermore, the methodology has allowed the challenges in weighing the risk-benefit profile for individual patients using clinical judgement alone to be identified. Accordingly, this methodology has provided hypothesis-generating data to justify further clinical investigation into the use of the new and more sensitive, non-invasive tests such as Cxbladder in increasing clinical resolution of patients being worked up who have presented to the clinic with hematuria.

The design, by requiring each physician to evaluate all patients, takes out ‘patient’ as a variable, thereby permitting focus on the physician-patient interaction, whose focus is the main aim of the study.

While cases were systematically selected, to provide a broad representation of patients in the prospective studies, data was presented immediately and across a series of patients: the investigators acknowledge that the study may not be reflective of a true clinical scenario. The influence of pre- and intra-study experience influencing decision-making cannot also be ruled out. However, by using the same patients presented to all urologists the study removed any confounding effect caused by different patients for different urologists and accordingly the study measured urologist decision making independent of patient variance. Furthermore, the outcomes of this study have not yet been tested in the real-world where outside influences, such as patient selection, resource availability, costs, alternative diagnostic options and real patient outcomes could influence the translation of these outcomes into clinical practice.