Using predictive process monitoring to assist thrombolytic therapy decision-making for ischemic stroke patients

In clinical practice, doctors often need to make decisions based on their experience of diagnosis and treatment, as well as the specific situation of each patient. For example, they want to know whether thrombolysis therapy is necessary for patients with ischemic acute stroke. According to reference [1], the general doctor’s judgment of thrombolysis for stroke patients is not accurate, and the misperception for the rate of fatal intracranial hemorrhage using rt-PA may interfere with their willingness to endorse this treatment. Clinical guidelines (CGs) offer the best practice in medical activities and play an important role for improving medical quality as well as reducing risks. However, evidence in CG is essentially a form of statistical knowledge, which is used to capture the generalities of patient groups, rather than the peculiarities of a specific patient. Thus, several conditions are usually implicitly assumed by experts building a CG [2]:

However, influenced by various factors (such as economic ability, cultural concepts, etc.), doctors usually have low compliance with clinical guidelines in real medical environment [3, 4]. These factors need to be fully considered for providing clinical decision support.

Due to the application of medical information system such as EMR, a large number of valuable historical diagnostic and therapeutic data have been stored. Based on mining and analyzing these medical data of related patients in the past, it has practical application value for medical decision support. Many studies analyze big data to make prediction, which can be divided into two categories [5]: one is supervised method for specific application purposes to train and generate predictive models, such as classification, regression, deep neural network, etc.; the other is unsupervised method by measuring the distance between patients, establishing similar groups of patients, and predicting the health status of target patients with the characteristics of similar groups. Although both of them need to pre-process data such as variable selection and dimensionality reduction, the former often trains a prediction model with higher accuracy, and the latter has a broader prediction ability [6].

As an active branch of medical knowledge engineering and artificial intelligence research, clinical decision support system (CDSS) is always the focus of study and application. However, what can be accepted by doctors and put into clinical use is few in the current CDSS. The main reason is that current CDSS relies too much on clinical medicine knowledge inference rules (CG are essentially medical rules), while some new rules are difficult to be obtained and represented [7]. Due to paying little attention to diversity, variability and uncertainty factors of disease, these systems cannot help doctors in the case of complex patient and disease. Some problems of the rule-based reasoning (RBR) system can be solved to some extent by the method of case-based reasoning (CBR). CBR solves problems by searching and matching with features of previous cases. But there are some shortcomings when CBR is used only, for example, it is difficult to express the deep domain knowledge; case retrieval and similarity matching algorithm should be further improved. Through combining with new technologies and methods of academic subjects, CBR and RBR can play their respective advantages by integrating, improving the application effect of CDSS. Rossille et al. [8] present the overall framework of CDSS based on CBR and RBR, while they do not define the specific similarity metric.

Clinical processes can be seen as complex symbolic sequences, i.e., a sequence of events each carrying a data payload consisting of event attributes [9]. Both the sequence of medical events (control flow) and the attributes value of events (data flow) have a significant impact on medical outcomes. For example, a doctor may perform a certain type of surgery only if it is preceded by a preoperational screening; the event attributes include the age of the patient or the amount of glucose in a blood sample. Traditionally, the studies of medical data mining focus on cross-sectional data such as diagnosis, symptoms, examination, past history, drugs, surgery and so on, while the time information is seldom involved [6]. Process mining technology involves time factor on the basis of data mining, and is driven by the data of actual cases. It records specific activity information through event logs to reflect the actual business execution history and predict the status of next activities [10].

Taking thrombolytic decision-making of stroke patients for example, this paper proposes a method using predictive process monitoring, which could be complementary with clinical guidelines. When the clinical guidelines are not applicable, it could provide decision support for doctors by referring to related historical cases, and the accuracy and feasibility of this method are verified. The rest of this paper is structured as follows. Section 2 shows background knowledge including predictive process monitoring and clinical guidelines for acute ischemic stroke. The proposed framework is described in Section 3. Section 4 illustrates the data set, experimental design and results. Section 5 discusses related work and Section 6 draws conclusions.

The framework proposed consists of two phases: offline, to train a prediction model based on historical cases, and online, to make predictions on running process cases. The function of each phase is shown in Fig. 2. There are four steps in the offline phase. Firstly, given an event log, case prefixes are extracted and filtered. Next, the selected prefixes in control flow and attributes in data flow are encoded for classification. Finally, some supervised learning algorithms are used to train classify models. The online phase concerns the actual prediction for a running trace, by reusing the classifiers built in the offline phase.

The original data features of patients include age, sex, pulse, blood pressure, blood glucose, hypertension, symptom, etc. These data features are involved in structured data and unstructured data, which are stored in HIS, LIS, PACS, EHR or other information systems. Data extraction and conversion are needed at first. We use Beautiful Soup [21], which is a Python library for natural language processing, to extract information from unstructured text. For example, the time from onset to consultation is abstracted from patient’s complaint; neurological impairment score tested in FAST-ED [22] is computed based on history of present illness; history of stroke and intracranial hemorrhage are extracted from history of past illness.

The results drawn from the prediction models are compared with the actual thrombolysis cases(A and B) to evaluate the accuracy of the models. Because thrombolytic therapy for patients with ischemic stroke includes intravenous thrombolysis and arterial thrombolysis, two prediction (classification) models are generated. Secondly, thrombolysis and Non-thrombolysis are two separate decisions, so we should test their accuracy respectively. Thirdly, in order to evaluate the prediction effect of control flow, we compared the prediction models with control flow and without control flow. Moreover, we exploited DT and RF classification algorithms so as to improve the prediction performance. Lastly, clinical guidelines could be considered as another prediction model, so their ACC, precision, recall, and AUC can be calculated. The results on the test set by our predictive models are compared with those recommended by clinical guidelines(A and C), in order to assess whether the models are superior to those clinical guidelines.

We use a five-fold cross-validation method to evaluate the prediction performance. Compare with the actual results, the average accuracy of our prediction model (RF) on the test set is 0.96 and 0.91, the AUC is 0.93 and 0.85. Compared with the recommended results by clinical guidelines, the accuracy, recall and AUC of our predictive model are higher, which means our models are better at fitting the actual situation than the clinical guidelines. The precision for intravenous thrombolytic are also higher than recommendation from clinical guidelines, but the precision for arterial thrombolysis are lower than recommended by clinical guidelines. All of the experimental results are showed in Table 5.

Since we train prediction models based on actual historical data, the higher recall could indicates doctors are inclined the benefit got from thrombolysis therapy, while the clinical guidelines perhaps are concerned with the risk of bleeding. In practical application, our models could remind medical staff occasions where thrombolysis therapy is practically possible but does not recommended by the clinical guidelines. For example, there are 75 cases recommended by our models but not by clinical guidelines, and 30 of them are with contraindications. The performance for intravenous thrombolysis are better than for arterial thrombolysis, which may be due to the time window is longer(0–24 h), and doctors might have to consider more factors in making decisions.

The performance of DT with control flow is better than DT without control flow, could be because we added 4 events: anti-platelet therapy, statin treatment, MRA, and coagulation test. Their proportions in the thrombolysis group and the non-thrombolysis group are significantly different (CI 95%, P < 0.01). The distribution of thrombolysis and non-thrombolysis for each medical activity are shown in the Table 6.

At present, the building and update of clinical guidelines usually takes a long time. However, CG cannot cover all of the medical problems, even there are some contradictions between different clinical guidelines. In order for clinical practice guidelines to be effective, they need to be integrated with the care flow and provide patient-specific advice when and where needed. Hence, their formalization as computer-interpretable guidelines (CIGs) makes it possible to develop CIG-based decision-support systems (DSSs), which have a better chance of impacting clinician behavior than narrative guidelines. However, how does the patients’ personal context affect decision making, as well as developing process learning methods that could mine relationships between process context, need to be investigated further [26].

The past decade has seen an explosion in the amount of digital information stored in EMR. Over the same period, the machine learning community has seen widespread advances in the field of deep learning. A variety of deep learning techniques and frameworks are being applied to several types of clinical applications including information extraction and outcome prediction. For example, Long Short-Term Memory (LSTM) could accept sequence as input and produce better performance than DT or RF. However, a much larger data set is required for LSTM. On the other side, although in many circumstances predictive models are improved by using deep learning methodologies, model transparency is of utmost importance to clinical applications. Practitioners often do not take advice from clinical decision support tools that they do not understand [5].

The patient similarity analysis provides a general-purpose computer assistant clinical decision support framework, using the patient distance assessment. Up to date, this method has been initially approved in many medical domains such as cancer, endocrine diseases and heart diseases. Unlike the supervised methods used in building prediction models, patient similarity analysis adopts unsupervised or semi-supervised methods, then calculates the similarity between concepts according to the meaning of clinical concepts, to obtain the distance between patients, and finally predicts the runtime case using the most similar cluster of patients. At present, most of the studies in this field only use the data in static perspective for similarity analysis, lacking of time series characteristics, which cannot fully reflect the dynamic similarity between patients [27].

Although the overall performance of our models is better than that of clinical guidelines, it is not to replace clinical guidelines, but to provide a new reference tool for doctors. The method we presented synthesizes patient’s diagnosis and treatment process (control flow) as well as event attributes value (data flow). It could predict the next activities in medical decision points, and implicitly integrates clinical guidelines into the prediction models. Taking into account medical knowledge, cultural differences, economic costs and therapeutic effects, our predictive models could achieve a higher accuracy, which could remind doctors avoid missing patients who may receive thrombolytic therapy. Moreover, although we take thrombolytic decision-making of stroke as an example, this method could be extended to other clinical decision support applications.

The first limitation of our approach is that we assumed the historical medical records can fully represent various situations inconsistent with the guidelines. Secondly, the relative contraindications and those patients with complex complications are not taken into account, even if they are diagnosed with acute ischemic stroke. Lastly, this method relies on the accuracy and completeness of the electronic medical records in the information system.

This current research is just based on one hospital, and the number of samples is relatively small. Therefore, we are planning to combine with the National Stroke Prevention and Control Project in China, to conduct training and validation on the data sets of multiple advanced stroke centers (large general hospitals), to prove that our method is universally applicable and that our models could be migrated. On the other side, since the cooperative hospital has only started the mechanical thrombectomy operation from the last 2 years, we also plan to establish a predictive model for mechanical thrombectomy in future.