Elsevier

Ophthalmology

Volume 125, Issue 8, August 2018, Pages 1199-1206
Ophthalmology

Original article
Efficacy of a Deep Learning System for Detecting Glaucomatous Optic Neuropathy Based on Color Fundus Photographs

https://doi.org/10.1016/j.ophtha.2018.01.023Get rights and content

Purpose

To assess the performance of a deep learning algorithm for detecting referable glaucomatous optic neuropathy (GON) based on color fundus photographs.

Design

A deep learning system for the classification of GON was developed for automated classification of GON on color fundus photographs.

Participants

We retrospectively included 48 116 fundus photographs for the development and validation of a deep learning algorithm.

Methods

This study recruited 21 trained ophthalmologists to classify the photographs. Referable GON was defined as vertical cup-to-disc ratio of 0.7 or more and other typical changes of GON. The reference standard was made until 3 graders achieved agreement. A separate validation dataset of 8000 fully gradable fundus photographs was used to assess the performance of this algorithm.

Main Outcome Measures

The area under receiver operator characteristic curve (AUC) with sensitivity and specificity was applied to evaluate the efficacy of the deep learning algorithm detecting referable GON.

Results

In the validation dataset, this deep learning system achieved an AUC of 0.986 with sensitivity of 95.6% and specificity of 92.0%. The most common reasons for false-negative grading (n = 87) were GON with coexisting eye conditions (n = 44 [50.6%]), including pathologic or high myopia (n = 37 [42.6%]), diabetic retinopathy (n = 4 [4.6%]), and age-related macular degeneration (n = 3 [3.4%]). The leading reason for false-positive results (n = 480) was having other eye conditions (n = 458 [95.4%]), mainly including physiologic cupping (n = 267 [55.6%]). Misclassification as false-positive results amidst a normal-appearing fundus occurred in only 22 eyes (4.6%).

Conclusions

A deep learning system can detect referable GON with high sensitivity and specificity. Coexistence of high or pathologic myopia is the most common cause resulting in false-negative results. Physiologic cupping and pathologic myopia were the most common reasons for false-positive results.

Section snippets

Methods

In the current study, 70 000 fundus photographs were downloaded by random sampling from the online dataset LabelMe (Healgoo Ltd. LabelMe dataset; 2016. http://www.labelme.org. Accessed February 16, 2016.), which contains more than 200 000 color fundus photographs collected from various clinical settings in China. Subsequently, 48 116 images with a visible optic disc were selected for the labelling of GON.

Approval from the institutional review board of Zhongshan Ophthalmic Center was obtained

Results

A total of 48 116 fundus photographs were graded for glaucoma by program-trained ophthalmologists, with each fundus photograph graded between 3 and 9 times. The median quantity of images per ophthalmologist graded was 3337 (range, 407–34 621). Twelve graders graded more than 3000 images. The median accuracy of the ophthalmologist graders (n = 15) for referable glaucoma was 82.1% (range, 76.0%–86.5%). A total of 5340 images were graded as poor quality and 3031 were labeled as poor location, and

Discussion

In this study, the efficacy of the deep learning method in identifying referable GON based on 48 116 fundus photographs was investigated. This deep learning algorithm showed a robust performance (AUC, 0.986; sensitivity, 95.6%; and specificity, 92.0%) for the detection of referable GON. Recently, several reports of automated methods for the evaluation of glaucoma have been published.30, 31, 32, 33, 34 Singh et al30 developed a technique using wavelet feature extraction techniques from segmented

Acknowledgments

The authors thank Zehong Zhou, International Department, Affiliated High School of South China Normal University, for his contributions to the data cleaning, integration, and convolutional neural network construction.

References (40)

  • A. Singh et al.

    Image processing based automatic diagnosis of glaucoma using wavelet features of segmented optic disc from fundus image

    Comput Methods Programs Biomed

    (2016)
  • A. Issac et al.

    An adaptive threshold based image processing technique for improved glaucoma detection and classification

    Comput Methods Programs Biomed

    (2015)
  • A.J. Tatham et al.

    Strategies for improving early detection of glaucoma: the combined structure-function index

    Clin Ophthalmol

    (2014)
  • M. Baskaran et al.

    The prevalence and types of glaucoma in an urban Chinese population: the Singapore Chinese Eye Study

    JAMA Ophthalmol

    (2015)
  • H.A. Quigley et al.

    The prevalence of glaucoma in a population-based study of Hispanic subjects: Proyecto VER

    Arch Ophthalmol

    (2001)
  • P. Gupta et al.

    Prevalence of glaucoma in the United States: the 2005–2008 National Health and Nutrition Examination Survey

    Invest Ophthalmol Vis Sci

    (2016)
  • B. Munoz et al.

    Causes of blindness and visual impairment in a population of older Americans: the Salisbury Eye Evaluation Study

    Arch Ophthalmol

    (2000)
  • Y. Shaikh et al.

    Burden of undetected and untreated glaucoma in the United States

    Am J Ophthalmol

    (2014)
  • P.P. Lee et al.

    A multicenter, retrospective pilot study of resource use and costs associated with severity of disease in glaucoma

    Arch Ophthalmol

    (2006)
  • J.F. Kirwan et al.

    Pharmacological mydriasis and optic disc examination

    Br J Ophthalmol

    (2000)
  • Cited by (0)

    Financial Disclosure(s): The author(s) have made the following disclosure(s): W.M.: Patent – Using deep learning models to process color fundus images

    M.H.: Patent – Using deep learning models to process color fundus images

    Supported in part by the Fundamental Research Funds of the State Key Laboratory in Ophthalmology, National Natural Science Foundation of China (grant no.: 81420108008); the Science and Technology Planning Project of Guangdong Province (grant no.: 2013B20400003); the University of Melbourne at Research Accelerator Program of Australia (M.H.); the CERA Foundation of Australia (M.H.); Victorian State Government of Australia (Operational Infrastructure Support to the Centre for Eye Research Australia); and Research to Prevent Blindness, Inc., New York (to Stanford University Eye Department). The sponsors or funding organizations had no role in the design or conduct of this research.

    HUMAN SUBJECTS: Human subjects were included in this study. The institutional review board of Zhongshan Ophthalmic Center approved the study and determined that informed consent was not required because of retrospective nature and fully anonymized usage of images in this study. The study was performed in accordance with the tenets of the Declaration of Helsinki.

    No animal subjects were used in this study.

    Author Contributions:

    Conception and design: Li, Meng, M.He

    Analysis and interpretation: Li, Keel, Chang

    Data collection: Li, Y.He, Keel, Meng

    Obtained funding: None

    Overall responsibility: Li, Y.He, Keel, Chang, M.He

    These authors contributed equally as first authors.

    View full text