Predicting quetiapine dose in patients with depression using machine learning techniques based on real-world evidence

We included 474 patients with depression, who were treated with quetiapine and hospitalized in the First Hospital of Hebei Medical University, from November 1, 2019, to August 31, 2022.

The inclusion criteria included the following: (1) patients who were diagnosed with depression and (2) patients who took quetiapine orally for a long time (at least for 3 days) at the same dose, and the blood concentration reached steady state at the time of blood collection. The exclusion criteria were as following: (1) patients older than 60 years were deleted; (2) patients with missing information (e.g., patient ID, medication record, etc.) were deleted; (3) samples that contained quetiapine at levels below the lower limit of quantification of 20 ng·ml⁻¹ were eliminated; and (4) patients who were diagnosed with organic mental disorders or took psychoactive drug substance were deleted. The International Classification of diseases-10 (ICD-10) was used for diagnosis, and the supervising doctor made a diagnosis of depression. According to the Chinese Guidelines for the Diagnosis and Treatment of Mental Disorders 2020 Edition, antidepressants should be used as single as possible for patients with depression. When changing medicine is ineffective, combination therapy may be considered. The combination of two antidepressants with different mechanisms of action can be used, and other combinations include the combination of second-generation antipsychotics and lithium [24]. The First Hospital of Hebei Medical University is a tertiary hospital in local, and most of the patients who admitted in our hospital had poor effect after single antidepressant treatment and changing medicines in primary hospitals. Therefore, depending on the patient’s condition and guidelines, antidepressants combined with second generation antipsychotics (such as quetiapine) were commonly prescribed. Herein, the primary purpose of using quetiapine is the synergistic treatment of depression. All data were gathered from clinical paper records and computerized medical records held by the hospital for patients. Eventually, 308 eligible individuals were enrolled in this study. Figure 1 provides an illustration of the sample selection workflow.

Figure 2 provides an illustration of data collecting and processing. First, based on the database's available data, we collected 47 clinical variables, including quetiapine administration information (e.g., daily dose and concentration), demographic information (e.g., age, gender, weight, height), comorbidities (e.g., hypertension, diabetes, hyperlipidemia), combination medication (e.g., CYP3A4 enzyme inhibitors, and CYP2D6/CYP3A4 competitive substrates) and laboratory parameters (e.g., regular blood test, liver function, and renal function). Considering the missing rates or extremely unbalanced variables, we preprocessed the obtained data, and the variables' missing values were filled with the mean.

As depicted in Fig. 2, univariate analysis was used to screen the variables after data collection from all relevant samples. Ultimately, variables which had p < 0.05 were selected. Based on the final dataset, the whole dataset was randomly divided into training cohort and testing cohort at the ratio of 80%: 20%. The data of the training cohort is used to train the model, and the test cohort is used to verify the final effect of the model. In this study, 246 subjects were in the training cohort and 62 subjects were in the testing cohort. Following that, key variables with p < 0.05 were chosen, and the daily dose of quetiapine was defined as the target variable. We created and evaluated nine different machine learning and deep learning models to compare the prediction abilities, including XGBoost, LightGBM, Random Forest (RF), Gradient Boosting (GBDT), Artificial Neural Network (ANN), Lasso Regression (LR), Support Vector Machine (SVM), TabNet and Decision Tree (DT). Assessment indicators were used for model evaluation, including precision, recall, F1-score, accuracy, sensitivity, and specificity. At the same time, we evaluated the effectiveness (AUROC) of quetiapine at various doses (100 mg/d, 200 mg/d, 300 mg/d, and 400 mg/d). Among these evaluation indicators, precision denotes the proportion of false positives [25]. Recall/sensitivity measures false negatives against true positives [25]. The F1-score is the harmonic average of the precision and recall [25]. Specificity measures false positives against true negatives [25]. The area under the ROC curve, or AUROC, is a comprehensive measurement that reflects the sensitivity and specificity of continuous variables. Accuracy is the proportion of correct predictions over the output results [25]. By contrasting the models' overall average accuracy, we may assess how well these models perform in terms of classification. After that, the confusion matrix, a special table used to view a classification model's performance, was then used to evaluate the prediction findings [26]. To avoid model overfitting and reduce bias, we used grid search combined with tenfold cross validation for hyperparameter tuning. Parameters of all nine models are displayed in Additional file 2: Table S1.

IBM SPSS version 26.0 was used for statistical research. (IBM Corporation, Armonk, New York, USA). In the comparison between training cohort and testing cohort, Mann–Whitney U test (non-normal distribution) and independent t test (normal distribution) were used to analyze the various continuous factors. Categorical data were analyzed by the Chi-squared test (n ≥ 5) or Fisher's exact test (n < 5). Statistical significance was set at p value < 0.05. Windows Python 3.9.12 was used to create each machine learning model.

Table 1 displays the distribution of features across the complete dataset. This research included 308 depressed patients in total, 131 of whom were men and 177 of whom were women. Median (interquartile range, IQR) was used to characterize continuous variables, and frequency (percentage, %) was used to describe categorical variables. Patients’ average age was 19.00 (IQR 15.00–36.25) years. The median height and weight were 166.00 (IQR 160.00–172.00) cm and 65.00 (IQR 55.00–76.00) kg. Based on their daily quetiapine dose, the patients were separated into several groups, with 57 (18.51%) receiving a dose of 100 mg, 108 (35.06) receiving a dose of 200 mg, 74 (24.03%) receiving a dose of 300 mg, and 69 (22.03%) receiving a dose of 400 mg. The median value of the serum levels in the dataset was 213.06 (IQR 124.23–370.24) ng·ml⁻¹. Comorbidities including hypertension, diabetes, and hyperlipidemia occupied 8.44%, 10.00%, and 7.14%, respectively. Combination medicine usage rates for CYP3A4 enzyme inhibitors were 0.32%, CYP3A4 competitive substrates were 5.19%, and CYP2D6 competitive substrates were 9.74%.

Considering extremely unbalanced variables, including hypertension, diabetes, hyperlipidemia, CYP3A4 enzyme inhibitors/inducers/competitive substrates, CYP2D6 enzyme inhibitors/competitive substrates, and variables with a missing rate greater than 50%, including weight, height may influence the predicted results of quetiapine, we preprocessed the obtained data before determining the significant associations between univariates. This led to a total of 38 candidate predictors, and finally four variables were selected which had p < 0.05, including quetiapine TDM value, age, mean corpuscular hemoglobin concentration (MCHC), and total bile acid (TBA), described in Table 2.

We developed and validated prediction models based on the selected features using nine algorithms (including XGBoost, LightGBM, RF, GBDT, SVM, LR, ANN, TabNet, and DT). Table 3 displays the performance of these models in testing cohort. The metrics of the XGBoost model outperformed those of other models and achieved the best overall performance, with precision = 0.91 ± 0.07, recall = 0.68 ± 0.1, F1 score = 0.78 ± 0.09, AUROC = 0.93 ± 0.04, sensitivity = 0.68 ± 0.1, and specificity = 0.98 ± 0.01 for predicting the daily dose of 100 mg quetiapine; precision = 0.67 ± 0.05, recall = 0.76 ± 0.09, F1 score = 0.71 ± 0.03, AUROC = 0.77 ± 0.04, sensitivity = 0.76 ± 0.09, and specificity = 0.78 ± 0.05 for predicting the daily dose of 200 mg quetiapine; precision = 0.67 ± 0.17, recall = 0.47 ± 0.09, F1 score = 0.54 ± 0.1, AUROC = 0.77 ± 0.08, sensitivity = 0.47 ± 0.09, and specificity = 0.93 ± 0.04 for predicting the daily dose of 300 mg quetiapine; precision = 0.64 ± 0.1, recall = 0.79 ± 0.08, F1 score = 0.7 ± 0.07, AUROC = 0.86 ± 0.06, sensitivity = 0.79 ± 0.08, and specificity = 0.86 ± 0.05 for predicting the daily dose of 400 mg quetiapine, and accuracy = 0.69 ± 0.03 for the entire XGBoost model. As a result, XGBoost was chosen to forecast the daily dose of quetiapine.

On this basis, XGBoost calculated and ranked the importance scores of four selected variables, as shown in Table 4. Among them, the most important feature in the prediction model was discovered to be the quetiapine TDM value (importance score = 0.41 ± 0.02), followed by AGE (importance score = 0.23 ± 0.01), MCHC (importance score = 0.19 ± 0.01) and TBA (importance score = 0.18 ± 0.01).

Then, we evaluated the performance of XGBoost model with 4 variables (quetiapine TDM value, AGE, MCHC, and TBA) using a testing cohort of 62 patients. Figure 3 shows the AUROC values for XGBoost under different groups according to the daily dose of quetiapine. Typically, an AUROC has a value between 0.5 and 1.0, and the larger AUROC indicates the greater model classification effect. Based on different dose intervals, the patients were separated into four subgroups: those with a daily dose of 100 mg (11 cases), 200 mg (23 cases), 300 mg (14 cases), and 400 mg (14 cases). In different subgroups according to the quetiapine daily dose of 100 mg, 200 mg, 300 mg and 400 mg, AUROC were 0.99, 0.75, 0.93, and 0.86, respectively.

Figure 4 summarizes the model’s performance in the testing cohort (62 cases) through confusion matrix. The model accurately predicted the dose regimen of 100 mg, 200 mg, 300 mg, and 400 mg quetiapine for 9, 15, 9, and 10 individuals, respectively. The evaluation indicators of four subgroups in the XGBoost model were calculated. The model can predict the dose regimen of 100 mg quetiapin with 100% precision and 82% recall rate; the dose regimen of 200 mg with 75% precision and 65% recall rate; the dose regimen of 300 mg with 69% precision and 64% recall rate; and the dose regimen of 400 mg with 50% precision and 71% recall rate, respectively. The results showed that the predicted quetiapine dose metrics agreed well with those from the clinically delivered plans for these patients.