Selecting the embryo with the highest implantation potential using a data mining based prediction model

There is a growing concern about the risks of multiple pregnancies. Worldwide, the incidence of multiple pregnancies has increased and the main reason for this increase is the use of assisted reproductive technologies (ART) [1]. These multiple pregnancies are leading to a higher incidence of pregnancy complications [2‐6]. Single-embryo transfer (SET) after in vitro fertilization (IVF) is the only effective means to avoid multiple pregnancy in ART cycles. The success rate of SET is highly depending on the efficiency of the selection of the embryo with the highest implantation potential [7].

Ever since the introduction of ART, embryo selection has been based on the morphological characteristics of the cleavage stage embryo. At the early cleavage stage, the standard scoring system (SSS) based on the combination of several direct morphological parameters. Obviously an embryo has to reach a certain cell number at a given day, and can be further characterized by its morphological appearance based on the equal or unequal size of blastomeres and the degree of fragmentation. To date, cell number, proportion of fragmentation, symmetry of blastomere size, remains the main approach of embryo selection [8‐12]. Several studies focused on this topic. Holte [10] showed that blastomere number, the degree of fragmentation, the size variation of blastomeres and the symmetry of embryos were independent predictors of implantation potential of the embryo. Cleavage rate was reported to clearly provide powerful prognostic information for implantation [8, 13]. Severe fragmentation of the embryo and uniformity in blastomere size were associated with poor prognosis [8, 13]. However, the presence of an important intra-and inter-observer variability in the microscopic evaluation [14, 15], and the absence of a clearly defined standard method, make it essential to further improve the selection techniques.

New imaging technology using multilevel images combined with a computer-assisted scoring system (CASS) has the potential to overcome most of the disadvantages associated with the standard scoring system (SSS). This computer-assisted scoring system allows switching focuses every 5 μm and stores multilevel images of embryos under inverted microscope, which allows further evaluation and measurement of the morphologic and morphometric characteristics of early cleaving embryos in a semi- automatic and more precise manner. Based on an objective measurement method, CASS overcomes the subjectivity of SSS, reporting less intra-and inter-observer variability [15]. Furthermore a significant association between total embryo volume on Day 2 and 3 and pregnancy rate was reported by Paternot [16]. Both lower and higher volumes were associated with a lower probability of successful pregnancy [16]. In addition, it has reported that CASS gave a better prediction of the implantation potential and live birth rate than SSS with multiple logistic regression based on morphologic characters on Day 3 [16]. Although this prediction model was not validated using an external validation data set.

Apart from morphologic embryo characteristics, several studies investigated the impact of clinical and diagnostic characteristics of the patients on the outcome of an ART cycle. Female and male age were reported to be associated with ART outcome [17, 18]. The impact of patient characteristics on ART outcome has been investigated such as the impact of endometriosis. One study concluded that ART yielded similar pregnancy outcome in patients with different stages of endometriosis when compared to patients with tubal infertility [19], while Azem et al. [20] reported that patient with stages III and IV endometriosis have a poorer outcome of ART than patients with tubal infertility. Nevertheless, there is still a lack of solid data on how to rank and select embryos based [21] on embryological data in combination with clinical, patient-related data.

To date, over 21 prediction models integrating multiple characteristics have been reported [21]. However, only 1/3 of the studied included embryo characteristic. And most of the models haven’t gone through an internal or external validation, which are essential to confirm the reproducibility and generalizability of a model process [21]. Only three models had a good performance after external validation [22‐24]. Logistic regression is a generalized linear regression which has been the usual choice of method in binary medical events, such as mortality, pregnancy. Similar to linear regression, logistic regression may include only one or multiple independent variables the estimated probability of being in one binary outcome category versus the other, rather than representing an estimated continuous outcome [25]. To develop models with better discriminative capacity, a more advanced datamining method besides common used logistic regression method were implemented. Being different from logistic regression, which predefines a model, data mining methods learn a decision function by detecting underlying patterns without a predefined model. Next, the function is transformed into an objective function and then optimization methods are used to find the values of decision variables to reach the desired outcome with the most confidence. Data mining based clinical decision support systems are efficient tools that provide recommendations in medical diagnosis and treatment [26].

The first aim of this study was to construct a prediction model, using as well embryological data as clinical patient characteristics, for clinical pregnancy, with data mining approaches, and compare the predictive performance of models in SSS and CASS. We present a novel, intelligent, decision support system for IVF treatment, in view of the developmental and morphometric characteristics determined by a computer scoring system. This system is based on a multivariate adaptive regression splines (MARS) supervised embryo classification/ranking system that can assist on the selection of the most promising embryos for implantation in ART treatment. The second aim of the study was to compare selection made by SSS vs CASS.

Among the clinical variables, maternal age was considered as potential confounding variables. The returned value of h = 0 indicated that ks-test did not reject the null hypothesis that the studied factors were from the same continuous distribution. In univariate analysis of clinical variables, age of female and male were found to be associated with implantation (Table 1).

Prior to development of the predictive model, the intra-observer reliability of computer-assisted scoring system was evaluated. An excellent intra-observer agreement for the CASS was found for the evaluation of features on Day 1 and Day 2, and number of blastomeres on Day 3. The intra-observer agreement was good for the evaluation for the other characteristics on Day 3 (Table 2).

Descriptive analyses were performed on morphometric parameters of embryos from different categories. Firstly, the COD of embryos with different number of blastomeres were compared both on Day 2 and Day 3 (Table 3). On Day 2, the COD among embryos in different cell stage showed a significant difference. On Day 3, only 8 cell stage embryos showed significant (p < 0.05) difference when compared with other cell stages. Secondly, the COD of embryos that, implanted or did not implant were compared on day 3. A significant difference (p < 0.05) was only found in 8 cell stage embryos on Day 3 (Table 4), where implanted embryos were observed to have equally sized blastomeres. Thirdly, the correlation between TCV and degree of fragmentation was also investigated. As indicated in Fig. 3, total embryo volume was significantly negatively correlated to the degree of fragmentation.

The area under ROC curve (AUC) for each model in both training set and validation set is reported in Table 5 and Fig. 1. The area under the ROC curves were compared using “pROC” implemented in R. With CASS, the AUC for the LR model decreased from 0.67 in the training set to 0.64 (p = 0.64) in the validation samples. Similarly, the decline in ROC curve area between the training set and validation set negligible for the MARS model: from 0.71 to 0.69 (p = 0.71). For the models based on standard scoring system a greater decline was found. The ROC curve areas for the logistic regression models were significantly different, from 0.68 in training set to 0.55 in validation set (p = 0.03). For the MARS model, the AUC dropped from 0.73 to 0.54 significantly (p = 0.02). The decreases in AUC of on SSS datasets were indicative of a tendency of both models to be over-fitted on the training samples.

Detailed information of models based on CASS is presented in Table 6. In view of both training data and external validation data, the MARS model resulted in a significant (p = 0.02) better discrimination accuracy in training data (0.71) compared with LR model (0.67). However, the difference in AUC (0.69 and 0.64, respectively) in validation data was not significant (p = 0.30). The LR model consisted of 8 predictors: number and parity of blastomeres on Day 2, morphometric parameters (TCV, fragmentation and COD) on Day 2, number of blastomeres and COD on Day 3, and paternal age. Total cytoplasmic volume on Day 1 and Day 3, age of female were not significant and subsequently excluded. And an equation predicting the implantation occurrence was obtained in Table 7. The MARS model consisted of 22 basic functions, including all of the 12 predictors. The parameters were trained by the function of “areasparams” in ARESLab. The maximum number of basic functions included in model in the forward building phase was calculated automatically using formula min(200, max(20, 2d)) + 1, where d is the number of input variables according to user’s manual. The GCV penalty per knot was 3, as commended by Friedman [33, 34]. The rest parameters, whether to use piecewise-cubic type modeling, the level of piecewise-cubic modeling, threshold, prune, were determined by default according to user’s manual.

To compare the decision-makings of current embryo selection criteria and the model assisted method, 104 patient were randomly recruited. Thirty-six (34.6 %) of 104 cases retained to a same decision, while in 68 (65.4 %) cases the transferred embryos of current criteria were not the best option according to our prediction model.

There is an increasing interest in using statistical methods to predict ART outcome in clinical research. In the current study, we have developed an approach to rank embryos according to their implantation potential taking into account both embryological and clinical features. The proposed model is expected to assist embryologists to determine which embryo to transfer. The model established a moderate discriminative performance, both in the training set and in the separate validation set. Our data also suggested that prediction models based on computer assisted scoring system using morphometric data, rather than standard scoring system, had significantly more stability and reliability of predictive capability. The TCV from Day 1 to Day 3, number of blastomeres, COD and fragmentation on Day 2 and Day 3 are considered as important embryo features to evaluate the implantation potential.

It was indicated that a CASS may be superior to a SSS in the prediction of implantation and live birth [16]. To further refine and validate, using an external validation dataset, the model proposed by Paternot et al. [35], we decided to use logistic regression and multivariate adaptive regression splines in both SSS data and CASS data, to develop prediction models. For each of the four models retained, the AUC of training data and validation data were compared [31, 32] to evaluate the generalizability. In the two models derived from SSS data, we found a significant decline in prediction accuracy in the external validation data set, suggesting that both LR model and MARS model based on SSS had poor predictive capability and considerable variation. Models using CASS data showed a more stable discriminative capability. According to mutual information analysis (Fig. 2) which reflect the correlation between variables, morphological characteristics on Day 3 are supposed to be the most powerful embryo features to determine treatment outcome. Consequently, low correlation coefficients of evaluations on Day 3 [15] may result in the poor performance of the external validation data set. In the CASS dataset, the MARS model presented a better predictive power compared with LR model. Results of univariate analysis have showed that not all of the characteristics were linearly correlated to clinical pregnancy, such as number of blastomeres and COD on Day 3. In such cases, MARS model, regardless of the certain assumption about the underlying functional relationship between the dependent and independent variables, constructed the relation from a set of coefficients and basic functions that are entirely “driven” from the data itself and consequently resulted in better AUC [36].

Number of blastomeres on Day 2 and Day 3 were found to be a powerful predictor clinical pregnancy. On Day 2, the presence of 4 blastomeres was considered the ideal cleavage rate [10, 37]. The study by Van Loendersloot et al. [24] showed that faster cleaving embryos had a lower chance of implantation. However, in our dataset, the presence of 6 blastomeres on Day 2 resulted in a higher implantation rate (42.1 %) than 4 blastomeres (39.7 %) but these results have to be further confirmed in future studies because only 19 embryos at the 6-cell stage on Day 2 were transferred, with 8 embryos implanted. The ideal number of blastomeres on Day 3 was 8. A marked reduction in implantation rate was found to be associated with slow cleavage rate and a slight reduction was correlated with rapid cleavage rate. This finding corroborates other studies where numerous authors have reported that too slow or too fast embryo cleavage has a negative impact on implantation rate [8, 13, 38]. This non-linear relationship between number of blastomeres and IVF/ICSI outcome makes the transformation (abs(number-8.4)) essential model fitting.

The study showed that the difference in blastomere size had strong predictive power for implantation. This is in line with other predictive models [39]. We compared coefficient of diversity (COD) of synchronously divided embryos (2-, 4- or 8-cell stage embryos) to embryos in intermediate steps (3-, 5-, 6-, 7-, 9- or 10-cell embryos). Significant lower COD was observed to be associated with synchrony. This finding confirmed a previous study [35], where the lowest coefficient of diversity was found for 8-cell stage embryos on Day 3. The variation in blastomere size has been reported to be negatively correlated with implantation [38]. In our study, implanted embryos tended to have more uniform blastomere size than non-implanted embryos except for 6-cell stage embryos, although significant difference was only found in 8-cell embryos on Day 3. It also confirmed the results described by [40] and the division model published by Roux [41] that theoretically, 6-cell stage embryos should have unequally sized blastomeres and higher COD.

Fragmentation on Day 2 and Day 3 were indicated as independent significant predictors of implantation in the multivariate models by Van Loendersloot [24] and Holte [37], respectively, and this was confirmed in both the LR and MARS model.

Total cytoplasmic volume on Day1, Day2 and Day 3 were all included in the final MARS model while only TCV on Day 2 was included in the LR model. An earlier study [42] identified a significant association between TCV and implantation potential on both Day 2 and Day 3. This could be explained by the fact that volume regulation is an essential process in the embryo development, as a failure in volume regulation can result in blocked embryos [43]. Hnida [44, 45] reported a significant decrease in the mean blastomere volume with an increasing degree of fragmentation for all analyzed embryo stages. Our current data analysis shows a significant (p < 0.01) negative correlation between the TCV and the degree of fragmentation (Fig. 3), confirming the conclusion published by Hnida [44, 45]. In addition, in the VIF analysis, a retained valued of 92 indicated a moderate to strong (30 ~ 100) collinearity between TCV and fragmentation on Day 3. The collinearity did not affect the final LR model since both predictors were excluded by stepwise logit regression. Nevertheless, the final MARS model, including both TCV and fragmentation fitted the data significantly better than the model including only one of the two parameters. Several other embryo evaluation models have been published in the past several years. However, some studies suffered from inadequate data sets and concluded their input information was not sufficient to classify the outcome [8, 10, 46]. Some other studies managed to achieve a model with high accuracy, without validation in an external data set, which is an essential process for a prediction model [10, 47]. The model by Van Loendersloot et al. [24] was based on the investigation of a large number of patients and cycles, and acquired confirmation on external validation data. However, when integrating number of blastomeres, degree of fragmentation, and size difference between blastomeres, the scoring method introduced in this study has limited prediction value for embryos with the same score but different fragmentation and COD status (e.g., an embryo with no fragmentation but unevenly sized blastomeres and an embryo with fragmentation but evenly sized blastomeres).

Several other morphometric characteristics have been considered as predictor for developmental competence or implantation, such as zygote size, nuclear size, embryo area and perimeter, equivalent circle radius of the embryo, embryonic roundness, and zona pellucida thickness [48‐50]. Morphometric measurements may minimize the variability among different embryologists and clinics. Although very little relevant literature published on the predictive ability of morphometric parameters, further studies are worth to go. It may improve the understanding of basic biology controlling early embryonic development and how this is affected by clinical parameters.

According to our data, the type of female pathology and male pathology have limited influence on the success of IFV/ICSI treatment within the first cycle in univariate analysis. Notably, regardless of no significant difference on clinical pregnancy rate with the presence of endometriosis success rate in patient with severe endometriosis decreased remarkably in our research cohort. However, with the small population (49 in 871 cases), this factor did not contribute to the final model. Ages of male and female were interesting prognostic factors for implantation according to our models. Increasing age of both women and men has been reported to be associated with declining possibility of successful pregnancy chance [51‐54]. As one of the most important factors for success with IVF, age of female is commonly included in previous prediction models. Male age, for the first time, is recruited in the prediction model in our study. Univariate logistic regression analysis revealed no significant influence of type of infertility and duration of infertility on clinical pregnancy, which is in line with Bancsi’ s [55] study.

The best predictive capacity of our cohort was 0.71 on training set and 0.69 on validation set. Compared with the previous prediction models, most of which could hardly return an accuracy rate of 0.67, this model presented a more confident prediction. Data trained for the model was collected during a period of 6 years, while the validation was performed on a separate and more recent data set. Regardless of different period, this model showed a similar discrimination efficiency on validation set. It should be noteworthy that all the data used in this study was collected from a single source, it is important to evaluate the generalizability of the model to other clinics. The method needs to be confirmed by replicating experiments on different IVF data sets.

It also should be noted that only single embryo transfers were taken into account when we developing the model. This approach has the advantage that the embryo data can be directly linked to the outcome. The major weakness of the study is the fact that the transferred embryos were selected using the SSS and the CASS analysis was performed retrospectively. In clinical single embryo transfer practice, embryologist are often confronted with situations that more than one embryo are available. In these cases, the CASS based scoring method may provide better understanding of all the characteristics of the embryo and assist the embryologist to select the embryo with highest implantation potential. The comparison research carried out on the 104 patients illustrated that in the majority of cases, this model assisted scoring method retained to a different decision, indicating that an evaluation of the this prediction model on future clinical randomized trial would be necessary. However, in the absence of a clinical randomized trial, this prediction model can be used to counsel couples undergoing ART treatment on the chance of a pregnancy, as predictions made by clinicians on the basis of clinical experience have only slight to fair reproducibility and in consequence a poor predictive accuracy [56].

The results presented indicate a considerable variation in prediction accuracy of models based on standard scoring system. The significant decline in validation data compared with training data reveals a limited utility of prediction models in SSS. In comparison, computer-assisted scoring system and the related models present the advantage of less subjectivity and more generalizability.

In addition, the findings of this study show that combination of computer-assisted scoring system and data mining based prediction method has a promising benefit in the selection of best embryo to transfer in IVF/ICSI treatment. The results need to be confirmed in a prospective trial using the CASS for the final selection of the embryo to be transferred.