Deep Learning-Based Automatic Diagnosis of Keratoconus with Corneal Endothelium Image

This is a retrospective case–control study which was performed at the Refractive Surgery Center of West China Hospital (Chengdu, China). We targeted patients who had endothelium imaging records. This study included a keratoconus cohort and a normal cohort: the keratoconus cohort consisted of 221 patients (403 images) who were consecutively recruited from January 2021 to September 2022, and the normal cohort comprised 185 patients with myopia (370 images) who underwent implantable collamer lens surgery from September 2021 to August 2022.

The study followed the principles of the Helsinki Declaration and was approved by the Ethics Committee of West China Hospital of Sichuan University (No. 2023496). Exemption from informed consent was granted because this study only used previous examination and medical records, the privacy and personal identity information of the subjects were protected, and the research project did not involve personal privacy and commercial interests.

The diagnosis of keratoconus was made based on standard guidelines [6, 31], including visual symptoms (progressive myopia and irregular astigmatism), direct observation of keratoconus signs (Fleischer ring, Vogt striae, stromal scar, corneal thinning, and conical protrusion) on slit-lamp examination), obvious keratoconus corneal topography on Pentacam HR instrument (Oculus GmbH, Wetzlar, Germany), such as irregular astigmatism, elevation and pachymetry of the cornea, corneal steepening, skewed axes, and/or an asymmetric bowtie pattern on corneal maps. The Amsler-Krumeich classification was used to assess the stage of keratoconus, which was based on corneal power, astigmatism, corneal thickness, and corneal transparency measured using slit-lamp biomicroscopy and anterior segment OCT (AS-OCT). Subjects with inflammatory ocular disorders, ocular surface diseases, glaucoma, or systemic diseases with ocular involvement were excluded from the research. Patients who had a history of ocular surgery, contact lens usage, or use of topical (excluding artificial tears) or systemic drugs that might impact the cornea were excluded from the study.

The non-contact specular microscope (Topcon SP-3000P, Tokyo, Japan) was used to measure corneal endothelial images and morphometric parameters. The patient assumed a slit-lamp posture and was instructed to look straight ahead at the built-in fixation objects. The monitor displayed a targeting circle, and the device automatically focused the pupil picture until it was in clear focus within the circle. The measurements were taken from the central cornea, and three consecutive readings were obtained, with the subject blinking between each one. The examiner carefully validated the image with the highest contrast and illumination quality selected by the device. The manufacturer's built-in software was used to detect and count the cells automatically. The specular microscopy data collected included cell number (N), minimum cell size (MIN), maximum cell size (MAX), average cell size (AVG), standard deviation (SD), endothelial cell density (ECD), coefficient of variation (CV), percentage of hexagonal cells (HEX), and corneal thickness (CCT). Additionally, the endothelial images were saved as 8-bit gray JPG files, with a resolution of 160 × 356 pixels, and used to train and test DL algorithms for automated corneal endothelial cell segmentation and keratoconus diagnosis.

A Fully Convolutional Network model with a Resnet50 (FCN_ResNet50) was established to perform the endothelial segmentation. Fully Convolutional Networks (FCN) have recently demonstrated significant improvements in medical image segmentation applications. Shelhamer et al. first introduced FCN, which modifies a pre-trained convolutional neural network (CNN) for image classification to serve as the network's encoder module [32]. Convolution layers are created by re-utilizing the weights and biases of fully connected layers. To up-sample the feature maps into a full-resolution segmentation map, a decoder module with an inverted convolution layer is added. In the present study, Resnet50 is the FCN model’s basis network, which was adapted from the ImageNet dataset and has a great capability for feature extraction of images. The information extractor was made up of a succession of densely linked expanded convolution and concatenated operators, followed by an adoptive pooling layer. Ultimately, the produced feature map was evaluated by a decoder for the endothelial segmentation task. To assess the performance of FCN_ResNet50, the endothelial cell and background pixel accuracy, global accuracy, endothelial cell intersection over union (IoU), background IoU, and mean IoU were assessed to evaluate the accuracy of segmentation.

We firstly acquired the segmented corneal endothelial images from the FCN_ResNet50 framework, which were further classified into two groups: keratoconus and normal control. These images were randomly divided into separate training and validation datasets at a 7:3 ratio. The training dataset was used for model building and hyperparameter tuning, while the validation dataset was used to evaluate generalization performance. Both data augmentation and normalization were employed for training images; however, only normalization was used for validated images. In our investigation, we used random affine modification and horizontal patch flipping to enhance the data. After z-score normalization on RGB channels, the upgraded images were put into four dominant DL networks (Densnet121, Resnet50, Inception_v3, and Mobilenet_v2) for model training. The DL networks were fine-tuned using transfer learning, which involved freezing the weights of convolution layers optimized for identifying general image features and replacing the deeper layers with new fully connected task layers that were trained using backpropagation for the new tasks. After fine-tuning, the fully connected layers were used to extract DL features from the endothelial images. This approach of using transfer learning can improve the accuracy and efficiency of deep learning for new tasks by leveraging the pre-trained networks.

Moreover, we conducted principal component analysis (PCA) to respectively compress the four kinds of DL features into two main DL features (PC-0 and PC-1). Afterward, multiple machine learning algorithms (support vector machines [SVM], k-nearest neighbors classifier [KNN], random-forest, extra-tree, lightGBM, and XGBoost) combined with fivefold cross-validation were applied to train models for diagnosis of keratoconus. The receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) were adopted to assess the performance of the algorithms.

All statistical analyses were carried out using packages from the Python 3.8.0 and R 4.2.2 programming environments, respectively. Pytorch (v 1.10.1) in Python was used to implement all DL frameworks, which was worked on a GeForce RTX-3080 GPU with 10 GB of RAM. The “Sklearn” package in Python was used to run the machine learning algorithms. The “pROC” package in R was applied to draw ROC curves and estimate AUC values. In the statistical analysis for visual acuity, the logarithm of the minimum angle of resolution (logMAR) units was utilized. Mean ± standard deviation or median (interquartile range) was used to describe continuous data, whereas frequencies were used to describe categorical variables. When comparing two groups, the Wilcoxon test was employed, whereas the Kruskal–Wallis test was used when comparing more than two groups. The Chi-square test was used to assess the relationships between cohorts and clinicopathological characteristics.

Our retrospective study included 403 eyes from 221 patients with keratoconus (153 males and 68 females). The mean age of the patients was 22 (interquartile range [IQR] 18–27) years. The median of the sphere was − 4.20 (IQR − 7.00 to − 1.70) D, and the median of the cylinder was − 2.00 (IQR − 4.00 to − 1.00) D. The median corrected distance visual acuity (CDVA) was 0.19 (IQR 0.03–0.40) logMAR. The median of keratometric 1 (K1) power at the 3-mm zone was 45.2 (IQR 43.2–48.9) D; the median of keratometric 2 (K2) power at the 3-mm zone was 49.0 (IQR 46.0–53.0) D. Based on Amsler-Krumeich’s classification, the keratoconus cohort was divided into four stages: 93 eyes (23%) were at stage I, 90 eyes were at stage II (22%), and 220 eyes (55%) pertained to stage III&IV combined. The baseline characteristics of all patients with keratoconus are given in Table 1.

To match the keratoconus cohort, we retrospectively collected data on 185 patients with myopia (370 eyes) who had undergone specular microscopy measurement. The differences in morphometric parameters of endothelial cell between the keratoconus cohort and normal cohort are given in Table 2. According to the Wilcoxon tests, the corneal thickness (CCT), endothelial cell density (ECD), coefficient of variation (CV), and number of cells (N) in the keratoconus cohort were significantly decreased, while the percentage of hexagonal cells (HEX) was significantly increased. The differences in the morphometric parameters of endothelial indicated that the corneal endothelium may be a potential signature for the diagnosis of keratoconus.

We therefore acquired 773 high-quality corneal endothelial images for artificial intelligence application. First, we divided the endothelial cell images into a right eye dataset and a left eye dataset. Then we randomly stratified the right eye dataset into a 70% training set and a 30% test set. The left eye dataset was used as an external validation set (Table 3). The statistical results showed that there were no differences in clinical characteristics among training, test, and validation datasets.

The simplified process of segmentation is presented in Fig. 1a. First, we manually labeled the region of interest (ROI) in 100 corneal endothelial images before training the FCN ResNet50 segmentation model for 50 epochs with a batch size of 32 for each epoch. The entire training procedure took around 30 min to finish. As the number of training iterations increased, the training loss converged at the first 30 epochs (Fig. 2a). The endothelial cell pixel accuracy, background pixel accuracy, and global accuracy achieved nearly 90% at the first 30 epochs, meanwhile the IoU of endothelial cell and background, and mean IoU converged to about 80% at the first 30 epochs, which highlighted the effective segmentation of the FCN_ResNet50 model (Fig. 2b). Next, 773 corneal endothelial images were put into the FCN_ResNet50 model for automatic segmentation, which took around 10 min to finish. An example of automatic segmentation for endothelial images is shown in Fig. 2c.

The simplified process of classification is shown in Fig. 1b. The segmented endothelial images were respectively sent into four deep learning networks (Densnet121, Resnet50, Inception_v3, and Mobilenet_v2) for training 50 epochs, with a batch size of 32 per epoch. The optimizer was SGD (stochastic gradient descent) with a learning rate of 10^-2 and L2 regularization of 10^-5. The ROC curves showed that the AUC values of the Densnet121 model were 0.714 and 0.805 in the training and test datasets, respectively (Fig. 3a) and that the AUC values of the training and test datasets in the Resnet50 model were 0.722 and 0.819, respectively (Fig. 3b). In the Inception_v3 model, the AUC values were 0.729 and 0.808 for the training and test datasets, respectively (Fig. 3c), and in the Mobilenet_v2 model, the AUC values for the training and test datasets were 0.691 and 0.801, respectively (Fig. 3d). The Grad-CAM (gradient-weighted class activation mapping) technique can calculate the key regions that the model predicts for endothelial cell images. Therefore, we utilized it to visualize the heat maps of the final convolutional layers for the four different models and to overlay them onto the original images. The blue regions in Fig. 3e indicate the active areas where the model focuses on. We observed that our models were able to selectively focus on different endothelial regions (Fig. 3e). Each of the DL features of the endothelial image was then extracted from the fully connected layers in the four deep learning networks. Based on PCA analysis, we successfully compressed these DL features into eight main features (DS-0, DS-1, RS-0, RS-1, IN-0, IN-1, MB-0, and MB-1) (Electronic Supplementary Material [ESM] Table S1). following which the eight main features were transferred into six machine learning algorithms and combined with fivefold cross-validation for model training. The accuracy distribution of the six machine learning algorithms showed that the eight DL features can accurately distinguish between keratoconus and normal groups in the training set, test dataset, and validation set (Fig. 4a). Details on the metrics (sensitivity, specificity, precision, recall, and F1 score) of seven machine learning models are listed in ESM Table S2. The ROC curves showed that all machine learning algorithms can achieve > 98% AUC value to diagnose keratoconus, regardless of whether it is in the training set (Fig. 4b), test dataset(Fig. 4c), or validation set (Fig. 4d). According to the distribution of accuracy for models, we observed that the XGBoost model achieved the highest accuracy (Fig. 4a). Also, based on the score of eight DL features, we conducted neighborhood component analysis (NCA) to visualize the distribution of samples, which illustrated that keratoconus and normal samples could be well identified (Fig. 4e). The relative importance of the eight DL features was calculated by the XGBoost model (Fig. 4f). Finally, we developed a DL diagnostic formula: \(|\sum_{i=1}^{\mathrm{N}}\left({importance}_{i}\times {expr}_{i}\right)|\) with the eight DL features, which was used to calculate the DL-score of each sample. We further standardized these values to range from 0 to 1.

A novel nomogram based on the morphometric parameters and DL-score of our cohort of patients was created to provide a simpler and more accurate approach to diagnose keratoconus in clinical practice. First, the logistic regression analysis was applied to determine the diagnostic significance of the DL-score of endothelial images and morphometric parameters for keratoconus. The results revealed that DL-score, CCT, ECD, and the number of cells were significantly associated with keratoconus (Fig. 5a). As a consequence, these factors were employed to develop a new nomogram. Total points aggregated each variable's points, which can indicate the likelihood of keratoconus (Fig. 5b). The calibration curve of the nomogram performed well in comparison with the ideal model (Fig. 5c). The ROC curves were applied to estimate the sensitivity and specificity of the nomogram model, which showed that the AUC value was 0.945; this value was significantly higher than DL-score (AUC 0.91; p value < 0.001) (Fig. 5d). Decision curves were also developed to examine the practical application of the nomogram, and the investigation revealed that our nomogram and DL-score provided a superior clinical outcome than “treat-all” or “treat-none” practices. (Fig. 5e).

We conducted Spearman and subgroup analyses to explore the relationships among the DL-score, nomogram, and other clinical parameters. According to the associated heatmap, the DL-score and nomogram were positively connected with the keratoconus stage, but negatively linked with CCT, ECD, HEX, and the number of cells. Age, sphere, cylinder, CDVA, K1, and K2 were unrelated to the DL-score or the nomogram (Fig. 6a). In addition, the box plots revealed that the keratoconus group had a higher DL-score (Fig. 6b) and a higher nomogram score (Fig. 6c) than the normal group. Interestingly, compared to the stage I&II groups, the DL-scores (Fig. 6d) and nomogram scores (Fig. 6e) were significantly increased in the stage III&IV group.