A comparison of efficacy of six prediction models for intravenous immunoglobulin resistance in Kawasaki disease

Kawasaki disease (KD) is a systemic self-limited vasculitis that mostly occurs in children under 5 years of age. The main complication of the disease is coronary artery abnormality (CAA), which is considered the leading cause of acquired heart disease [1]. Fifty years have passed since the first report of the disease and now 2 g/kg intravenous immunoglobulin (IVIG) together with large amount of aspirin is considered the golden primary treatment in the acute stage worldwide. However, approximately 10–20% patients do not respond to such treatment and presented with prolonged changes in presence of prolonged fever [1]. Moreover, alternative treatments after response failure to initial IVIG treatment are usually unsatisfactory, with a remarkable higher incidence of CAA [2, 3]. Thus, early identification of IVIG resistance is of great importance so that additional therapy in the early stage could be given to reduce the risk of CAA.

In 2006, Kobayashi established a prediction score model of IVIG resistance based on 676 study patients [4]. The related predictors were serum sodium (Na) ≤ 133 mmol/L, days of illness at initial treatment ≤4, aspartate aminotransferase (AST) ≥ 100 IU/L, N% ≥ 80%, c-reactive protein (CRP) ≥ 10 mg/dL, age ≤ 12 months and platelet count (PLT) ≤ 30 × 10⁴/mm³. Due to the racial heterogeneity, studies on IVIG resistance prediction score have amassed in recently years in different countries and areas [2, 5‐8].

Previously we have established a prediction model using the KD dataset in this area from 2006 to 2014 [9]. In the present study, we aimed to evaluate the efficacy of these prediction models using the medical data of KD patients hospitalized in Children’s Hospital of Soochow University during the last 2 years.

We prospectively collected the medical records of patients hospitalized in the Department of Cardiology in Children’s Hospital of Soochow University with a diagnosis of KD from Jan 2015 to Dec 2016. When we evaluated the Kawamura prediction model and parts of the Formosa model, the medical records were retrospectively collected. Children’s Hospital of Soochow University is a tertiary hospital that serves most children of this area and some of the children from surrounding cities. Kawasaki disease was confirmed when patients had fever for more than 5 days plus four of the following five characteristics. 1. Rash. 2. Bilateral conjunctive injection. 3. Cervical lymphadenopathy. 4. Changes of the extremities. 5. Oral mucosal changes. Incomplete KD was defined as patients with ≥5 days of fever and < 4 classic criteria, but who had CAA upon echocardiography or a set of suspect laboratory criteria according to the AHA guideline [10]. IVIG resistance was referred to when a patient developed recrudescent or persistent fever ≥36 h after the end of their IVIG infusion [10]. All febrile patients received 2 g/kg IVIG and 30-50 mg/kg aspirin as the guideline recommended [10]. The dosage of aspirin was decreased to 3–5 mg/kg/day until the patients were afebrile for 3–4 days. All parents of the patients gave their informed consent. The study was approved by the Ethics Committee of Children’s Hospital of Soochow University (No: 2014LW003).

Demographic and clinical characteristics were obtained on admission. Laboratory data regarding CRP, PLT, Na, N%, serum albumin (ALB), alanine aminotransferase (ALT), AST, total bilirubin (TB), neutrophils count, lymphocytes count were obtained within 24 h on admission. In our prediction score model, two points were assigned to age younger than 6 months and ALB lower than 35 g/L. One point was assigned to N% ≥80%, presence of rash and edema of extremities. Our model had a sensitivity of 71.4% and a specificity of 76.0%. Detailed information of the prediction models are shown in Table 1.

A total of 505 patients were diagnosed with KD during the study period. One patient failed to be treated with IVIG for her parents’ refusal. As a result, 504 patients were enrolled in the present study. 318 were males and 186 were females, yielding a male-to-female ratio of 1.71:1. 153 (30.3%) patients were under one year old and 467 (92.7%) were under five years old. The median age were 20 (11, 36) months. 322 (63.9%) patients were complete KD while the remaining 182 (36.1%) were incomplete cases. 25 (5.0%) patients were identified as IVIG resistance. The characteristics of the patients are shown in Table 2. Significant differences were demonstrated in age <  6 months, presence of rash and edema of extremities. For laboratory findings, patients with IVIG resistance had higher levels of CRP, AST, N% and TB. On the other hand, they tended to have lower level of ALB.

The scores of the prediction models were calculated for each patient, and then the patients were assigned to the high or low risk group. Table 3 shows the number of patients in the high and low-risk categories, together with the relative sensitivity, specificity, NPV, PPV and area of ROC curve of each model. Our prediction model had a sensitivity of 0.72 and a specificity of 0.75. Sensitivity of Kobayashi, Egami, Kawamura, Sano and Formosa were 0.72, 0.44, 0.48, 0.20, and 0.68, respectively. Specificity of these models were 0.62, 0.82, 0.66, 0.91, and 0.48, respectively.

In the present study, we evaluated the efficacy of the Kobayashi, Egami, Kawamura, Sano, Formosa, as well as our own prediction model using the medical data of KD in the past 2 years. We found Kobayashi and our prediction model had a relatively high sensitivity of 72% while our model had a higher specificity than Kobayashi. Sensitivity of Egami, Kawamura and Sano were all lower than 50%. Sensitivity of Formosa was intermediate with the lowest specificity. Our data suggested that our prediction model had a powerful predictive value in this area, followed by Kobayashi model while all the other prediction models had less excellent performances than ours. Actually, these findings might provide evidences for our future work on additional initial treatment in KD patients.

Kobayashi, Egami and Sano prediction models were mainly based on age, illness day and laboratory variables. The related predictive risk factors such as higher levels of CRP and N%, lower level of ALB were later proved to be effective predictors in different areas in our country [9, 11, 12]. However, the node and the point score in each prediction model were different due to a regional difference.

Developed in 2015, Formosa model was based on KD-associated manifestations besides laboratory characteristics. They developed the model in 181 patients and further validated the data in another 67 patients. The validation results showed the sensitivity and specificity of the prediction model were 71.4% and 81.0%, which was slightly lower compared with those in the development dataset [7]. Although two of the three predictors (N% and ALB) were also included in our prediction model, Formosa model didn’t have excellent performance in the present study. Our results were in consistence with Song’s study [13]. The assignment of 2 points to N% ≥60% in the Formosa model might increase the number of high-risk patients identified by that model.

Kawamura prediction model was also developed in Japan based on a single-center data. However, the prediction model was developed in 2016. That was to say, the prediction model was not fully established when we started the present study and thus we retrospectively reviewed the data according to the model at the end of our study. The prediction model was proved to have a sensitivity of 71% and a specificity of 69% in the combination of both neutrophil-to-lymphocyte ratio (NLR) ≥3.83 and platelet-to-lymphocyte ratio (PLR) ≥150 [8]. In another study including 437 KD patients which was carried out in the same medical center, the authors found the sensitivity and specificity of the prediction model were similar to Kobayashi model [14]. However, NLR and PLR could be calculated quickly and easily with the blood routine test alone. Neutrophils are the most abundant inflammatory cells, which may partly reflect the severity of the inflammation. In KD mouse models, influx of neutrophils and macrophages occurred 1 day after the stimulus, followed by a second neutrophil infiltration 28 days later, which persisted for several weeks [15]. In autopsied KD patients, neutrophils predominated in the earlier stage of the disease when macrophages predominated in the later stage [16]. The specific mechanisms remained obscure and were possibly related to the decreased apoptosis of neutrophils [17, 18]. Due to the activation of macrophages, elevated level of platelet was observed in most patients in the convalescent stage. On the other hand, lymphocytes also played a pivotal role in inflammatory processes and lymphocytopenia was associated with more severe inflammation owing to increased apoptosis [19, 20]. Although combination of NLR ≥3.83 and PLR ≥150 was more effective than either the NLR or PLR alone, an NLR of ≥3.83 or a PLR of ≥150 was an independent predictor of IVIG resistance [8]. Thus, patients with more than one predictor were assigned to high risk group and the remaining were assigned to low risk group. Unfortunately, the easily calculated prediction model had less powerful predictive efficacy. Factors such as different treatment regimens and methodology might be associated with the divergence.

It was not surprising that our prediction model had a higher efficacy in predicting IVIG resistance. We previously compared the clinical and laboratory characteristics of the patients with different IVIG responses. The sensitivity and specificity were 71.4% and 76.0% in the development dataset, respectively, which were similar to the results in the present study [9]. The divergences of sensitivity and specificity among these models and ours might be explained by different genetic backgrounds [1], definition of IVIG resistance and meteorological factors [21].

Though the incidences of IVIG resistance varied all of the world, mostly were between 10% and 20% [22‐24]. It was quite low in the present study, which was similar to the incidence of Shanghai in the latest survey [25]. We previously had found that 5.1% of all patients were IVIG resistant during the past 9 years [9], which was almost the same as the present study. We speculated the time of initial IVIG treatment, racial differences and lots of IVIG might attribute to the discrepancy in the incidences. Unfortunately, all of these six prediction models had a relatively low PPV. Our results were coincident with previous study [13], which could be explained by a low IVIG resistance incidence (5% in the present study).

All the prediction models described in the present study were developed in Asia populations, most of which were from Japan. However, all had shown moderate sensitivity. What was more, previous study had found a lower sensitivity in an American population [26]. Due to an unknown origin of KD, we speculated a prediction model combined clinical, laboratory characteristics, with biomarkers or other specific indicators might have a better performance.

It should be noted that a total of 505 patients were diagnosed with KD in 2015 and 2016, which were almost half of those diagnosed between 2006 and 2014 [9]. In 2014, the Chinese government introduced the second child policy, leading to a dramatic increase in the number of children aged under 2 years in recent years. Thus, it was not surprising that the number of KD patients were increasing rapidly.

This study had some limitations. First, part of the study was retrospective and the missing data might lead to selection bias. Second, we did not conduct stringent validation by using an independent KD cohort from other institutions in this area, although most KD patients were diagnosed and treated in our hospital. Third, we didn’t evaluate the efficacy of other models such as Tremoulet prediction model for the inconvenience to get such variables. Further large-scaled studies were warranted to further verify the efficacy of the prediction models and moreover, to develop a more efficient prediction model suited for this area.

Our prediction model had a powerful predictive value in this area, followed by Kobayashi model while all the other prediction models had less excellent performances than ours. The prediction model could help identify IVIG resistance in clinical work.