The sensitivity and specificity of single-field nonmydriatic monochromatic digital fundus photography with remote image interpretation for diabetic retinopathy screening: a comparison with ophthalmoscopy and standardized mydriatic color photography

doi:10.1016/S0002-9394(02)01522-2

American Journal of Ophthalmology

Volume 134, Issue 2, August 2002, Pages 204-213

https://doi.org/10.1016/S0002-9394(02)01522-2 Get rights and content

Abstract

PURPOSE: To evaluate single-field digital monochromatic nonmydriatic fundus photography as an adjunct in the screening of diabetic retinopathy.

DESIGN: Prospective, comparative, observational case series.

METHODS: Patients with type I and type II diabetes mellitus (n = 197) were sequentially evaluated by three different techniques: single-field digital monochromatic nonmydriatic photography; dilated ophthalmoscopy by an ophthalmologist; and seven Early Treatment Diabetic Retinopathy Study (ETDRS) standardized 35-mm color stereoscopic mydriatic images. The seven stereoscopic color photographs served as the reference standard and were compared with either ophthalmoscopy or a single digital photograph transmitted electronically to a reading site. Levels of agreement were determined by κ analyses. The sensitivity and specificity of the three methods were compared based on a threshold for referral to further ophthalmologic evaluation (ETDRS level ≥35).

RESULTS: There was highly significant agreement (κ = 0.97, P = .0001) between the degree of retinopathy detected by a single nonmydriatic monochromatic digital photograph and that seen in seven standard 35-mm color stereoscopic mydriatic fields. The sensitivity of digital photography compared with color photography was 78%, with a specificity of 86%. Agreement was poor (κ = 0.40, P = .0001) between mydriatic ophthalmoscopy and the seven-field standard 35-mm color photographs. Sensitivity of ophthalmoscopy compared with color photography was 34%, with a specificity of 100%.

CONCLUSION: A single nonmydriatic monochromatic wide-field digital photograph of the disk and macula was more sensitive for diabetic retinopathy screening than mydriatic ophthalmoscopy, the currently accepted screening method. When adjudicated by standard seven-field color photographs, the higher sensitivity of digital photography primarily reflected the reduced sensitivity of ophthalmoscopy in detecting early retinopathy.

Section snippets

Design

In this prospective, comparative, observational case series, subjects were recruited from a large health care maintenance organization from July 20, 1996, to December 15, 1997.

Patients

Study protocols were presented to and approved by the Institutional Review Board for the Protection of Human Subjects at the Kaiser Foundation Research Institute (Oakland, California). A total of 403 consecutive subjects under care for diabetes mellitus were recruited from the Kaiser Oakland Medical Center and underwent digital monochromatic photography. Eligible patients included both type I and type II diabetic patients with any disease duration. Patients were ineligible if they were less

Results

A total of 403 patients underwent at least one screening modality, and 211 patients completed all three diagnostic modalities. After patients with unusable ophthalmoscopy records and unusable standard seven-field color photographs were excluded, a total of 197 participants were included in the data set (Table 1). The demographic characteristics of this urban diabetic population are summarized in Table 2. The exclusion of participants who failed to complete all three screening modalities did

Discussion

Our results suggest that single-field nonmydriatic monochromatic digital photography with remote interpretation is a sensitive and specific means for detecting diabetic retinopathy in at-risk populations. In this study, nonmydriatic wide-angle digital photography was superior to ophthalmoscopy in accurately grading lower levels of diabetic retinopathy, as performed in a typical outpatient setting. These results should be of interest to both the ophthalmoscopy and public health communities.

In an

References (40)

Vision problems in the U.S.A.statistical analysis
(1980)
R. Klein et al.
Vision disorders in diabetes
Photocoagulation treatment for proliferative diabetic retinopathy, the second report of Diabetic Retinopathy Study Findings
Ophthalmology
(1978)
Photocoagulation for diabetic macular edema. ETDRS report number 1
Arch Ophthalmol
(1985)
J.C. Javitt et al.
Preventive eye care in people with diabetes is cost-saving to the federal government. Implications for health-care reform
Diabetes Care
(1994)
J.C. Javitt et al.
Detecting and treating retinopathy in patients with type I diabetes mellitus, savings associated with improved implementation of current guidelines
Ophthalmology
(1991)
J.C. Javitt et al.
Cost-effectiveness of detecting and treating diabetic retinopathy
Ann Intern Med
(1996)
M. James et al.
Cost effectiveness analysis of screening for sight-threatening diabetic eye disease
BMJ
(2000)
H. Matz et al.
Cost-benefit analysis of diabetic eye disease
Ophthalmologica
(1996)
D.E. Singer et al.
Screening for diabetic retinopathy
Ann Intern Med
(1992)

S. Moss et al.

Are seven standard photographic fields necessary for classification of diabetic retinopathy?

Invest Ophthalmol Vis Sci

(1989)

Color photography versus fluorescein angiography in the detection of diabetic retinopathy in the Diabetes Control and Complications Trial

Arch Ophthalmol

(1987)

A. Schachat et al.

Comparison of diabetic retinopathy detection by clinical examinations and photographic gradings

Arch Ophthalmol

(1993)

A. Peters et al.

Cost effectiveness screening for diabetic retinopathy using a non-mydriatic retinal camera in prepaid health care setting

Diabetes Care

(1993)

V. Lee et al.

Diagnosis of diabetic retinopathyophthalmoscopy versus fundus photography

Ophthalmology

(1993)

J. Jacob et al.

A report on the use of technician ophthalmoscopy combined with the use of the Canon non-mydriatic camera in screening for diabetic retinopathy in the community

Diabet Med

(1995)

G. Phillipov et al.

Screening for diabetic retinopathy

Med J Aust

(1995)

J. Pugh et al.

Screening for diabetic retinopathythe wide-angle retinal camera

Diabetes Care

(1993)

S. Griffith et al.

Screening for diabetic retinopathy in a clinical setting: a comparison of direct ophthalmoscopy by primary care physicians with fundus photography

J Fam Pract

(1993)

H. Kalm

Non-stereo photographic screening and long term follow-up for detection of proliferative diabetic retinopathy

Acta Ophthalmol

(1992)

Cited by (300)

Diagnostic Accuracy of Artificial Intelligence-Based Automated Diabetic Retinopathy Screening in Real-World Settings: A Systematic Review and Meta-Analysis
2024, American Journal of Ophthalmology
To evaluate the diagnostic accuracy of artificial intelligence (AI)-based automated diabetic retinopathy (DR) screening in real-world settings.
Systematic review and meta-analysis
We conducted a systematic review of relevant literature from January 2012 to August 2022 using databases including PubMed, Scopus and Web of Science. The quality of studies was evaluated using Quality Assessment for Diagnostic Accuracy Studies 2 (QUADAS-2) checklist. We calculated pooled accuracy, sensitivity, specificity, and diagnostic odds ratio (DOR) as summary measures. The study protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO - CRD42022367034).
We included 34 studies which utilized AI algorithms for diagnosing DR based on real-world fundus images. Quality assessment of these studies indicated a low risk of bias and low applicability concern. Among gradable images, the overall pooled accuracy, sensitivity, specificity, and DOR were 81%, 94% (95% CI: 92.0-96.0), 89% (95% CI: 85.0-92.0) and 128 (95% CI: 80-204) respectively. Sub-group analysis showed that, when acceptable quality imaging could be obtained, non-mydriatic fundus images had a better DOR of 143 (95% CI: 82-251) and studies using 2 field images had a better DOR of 161 (95% CI 74-347). Our meta-regression analysis revealed a statistically significant association between DOR and variables such as the income status, and the type of fundus camera.
Our findings indicate that AI algorithms have acceptable performance in screening for DR using fundus images compared to human graders. Implementing a fundus camera with AI-based software has the potential to assist ophthalmologists in reducing their workload and improving the accuracy of DR diagnosis.
Effectiveness of Conventional Digital Fundus Photography-Based Teleretinal Screening for Diabetic Retinopathy and Diabetic Macular Edema: A Report by the American Academy of Ophthalmology
2024, Ophthalmology
This American Academy of Ophthalmology Ophthalmic Technology Assessment aims to assess the effectiveness of conventional teleretinal screening (TS) in detecting diabetic retinopathy (DR) and diabetic macular edema (DME).
A literature search of the PubMed database was conducted most recently in July 2023 to identify data published between 2006 and 2023 on any of the following elements related to TS effectiveness: (1) the accuracy of TS in detecting DR or DME compared with traditional ophthalmic screening with dilated fundus examination or 7-standard field Early Treatment Diabetic Retinopathy Study photography, (2) the impact of TS on DR screening compliance rates or other patient behaviors, and (3) cost-effectiveness and patient satisfaction of TS compared with traditional DR screening. Identified studies then were rated based on the Oxford Centre for Evidence-Based Medicine grading system.
Eight level I studies, 14 level II studies, and 2 level III studies were identified in total. Although cross-study comparison is challenging because of differences in reference standards and grading methods, TS demonstrated acceptable sensitivity and good specificity in detecting DR; moderate to good agreement between TS and reference-standard DR grading was observed. Performance of TS was not as robust in detecting DME, although the number of studies evaluating DME specifically was limited. Two level I studies, 5 level II studies, and 1 level III study supported that TS had a positive impact on overall DR screening compliance, even increasing it by more than 2-fold in one study. Studies assessing cost-effectiveness and patient satisfaction were not graded formally, but they generally showed that TS was cost-effective and preferred by patients over traditional surveillance.
Conventional TS is an effective approach to DR screening not only for its accuracy in detecting referable-level disease, but also for improving screening compliance in a cost-effective manner that may be preferred by patients. Further research is needed to elucidate the ideal approach of TS that may involve integration of artificial intelligence or other imaging technologies in the future.
Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
AI-Human Hybrid Workflow Enhances Teleophthalmology for the Detection of Diabetic Retinopathy
2023, Ophthalmology Science
Detection of diabetic retinopathy (DR) outside of specialized eye care settings is an important means of access to vision-preserving health maintenance. Remote interpretation of fundus photographs acquired in a primary care or other nonophthalmic setting in a store-and-forward manner is a predominant paradigm of teleophthalmology screening programs. Artificial intelligence (AI)-based image interpretation offers an alternative means of DR detection. IDx-DR (Digital Diagnostics Inc) is a Food and Drug Administration-authorized autonomous testing device for DR. We evaluated the diagnostic performance of IDx-DR compared with human-based teleophthalmology over 2 and a half years. Additionally, we evaluated an AI-human hybrid workflow that combines AI-system evaluation with human expert-based assessment for referable cases.
Prospective cohort study and retrospective analysis.
Diabetic patients ≥ 18 years old without a prior DR diagnosis or DR examination in the past year presenting for routine DR screening in a primary care clinic.
Macula-centered and optic nerve-centered fundus photographs were evaluated by an AI algorithm followed by consensus-based overreading by retina specialists at the Stanford Ophthalmic Reading Center. Detection of more-than-mild diabetic retinopathy (MTMDR) was compared with in-person examination by a retina specialist.
Sensitivity, specificity, accuracy, positive predictive value, and gradability achieved by the AI algorithm and retina specialists.
The AI algorithm had higher sensitivity (95.5% sensitivity; 95% confidence interval [CI], 86.7%–100%) but lower specificity (60.3% specificity; 95% CI, 47.7%–72.9%) for detection of MTMDR compared with remote image interpretation by retina specialists (69.5% sensitivity; 95% CI, 50.7%–88.3%; 96.9% specificity; 95% CI, 93.5%–100%). Gradability of encounters was also lower for the AI algorithm (62.5%) compared with retina specialists (93.1%). A 2-step AI-human hybrid workflow in which the AI algorithm initially rendered an assessment followed by overread by a retina specialist of MTMDR-positive encounters resulted in a sensitivity of 95.5% (95% CI, 86.7%–100%) and a specificity of 98.2% (95% CI, 94.6%–100%). Similarly, a 2-step overread by retina specialists of AI-ungradable encounters improved gradability from 63.5% to 95.6% of encounters.
Implementation of an AI-human hybrid teleophthalmology workflow may both decrease reliance on human specialist effort and improve diagnostic accuracy.
Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
RASS: Enabling privacy-preserving and authentication in online AI-driven healthcare applications
2023, ISA Transactions
Powered by the rapid progress of analytics techniques and the increasing availability of healthcare data, artificial intelligence (AI) is bringing a paradigm shift to healthcare applications. AI techniques offer considerable advantages for the evaluation and assimilation of large amounts of complex healthcare data. However, to effectively use AI tools in healthcare, key issues need to be considered and several limitations must be addressed, such as privacy-preserving and authentication of the healthcare data for analysis in training and inference procedures. Although various techniques ranging from cryptographic tools to obfuscation mechanisms have been proposed to provide privacy guarantees for data in AI-based services, none of them is applicable to online AI-driven healthcare applications. For they require a heavy computational cost on protecting privacy without offering authentication services for third parties. In this paper, we present RASS, an efficient privacy-preserving and authentication scheme for securing analyzed data in an AI-driven healthcare system. The security proofs of our construction indicate that its unforgeability and multi-show unlinkability can defend against the tempering and collusion attacks respectively. Finally, we conduct sufficient efficiency analysis, and the results show that RASS achieves the above security demands without introducing complex computation and communication costs.
Foundational Considerations for Artificial Intelligence Using Ophthalmic Images
2022, Ophthalmology
Citation Excerpt :
Indeed, it is a major strength of the Collaborative Community CCOI, and its disease-specific subgroups, that it has started discussing the development of such prognostic standards for disease areas of interest. Other factors that should be considered when evaluating potential reference standards, in addition to their validity or lack of thereof against outcome, include: (1) reproducibility of the reference standard (many studies have shown that multiple clinicians evaluate the same patient differently in 30%–50% of cases74–76); (2) repeatability (many studies have shown that the same clinician evaluates the same patient differently in 20%–30% of cases74–76; (3) diagnostic drift (studies have shown that clinicians from different regions, countries, or continents evaluate the same patient differently in up to 50% of cases, leading to vernacular medicine, as explained in the next section39; and (4) temporal diagnostic drift (studies have shown clinicians systematically evaluating the same hypothetical patient differently over generations of clinicians77). Because the evidence for a given treatment based on a given evaluation may have been derived decades ago, temporal drift in a prognostic standard may be hard to determine and difficult to correct for.
The development of artificial intelligence (AI) and other machine diagnostic systems, also known as software as a medical device, and its recent introduction into clinical practice requires a deeply rooted foundation in bioethics for consideration by regulatory agencies and other stakeholders around the globe.
To initiate a dialogue on the issues to consider when developing a bioethically sound foundation for AI in medicine, based on images of eye structures, for discussion with all stakeholders.
The scope of the issues and summaries of the discussions under consideration by the Foundational Principles of Ophthalmic Imaging and Algorithmic Interpretation Working Group, as first presented during the Collaborative Community on Ophthalmic Imaging inaugural meeting on September 7, 2020, and afterward in the working group.
Artificial intelligence has the potential to improve health care access and patient outcome fundamentally while decreasing disparities, lowering cost, and enhancing the care team. Nevertheless, substantial concerns exist. Bioethicists, AI algorithm experts, as well as the Food and Drug Administration and other regulatory agencies, industry, patient advocacy groups, clinicians and their professional societies, other provider groups, and payors (i.e., stakeholders) working together in collaborative communities to resolve the fundamental ethical issues of nonmaleficence, autonomy, and equity are essential to attain this potential. Resolution impacts all levels of the design, validation, and implementation of AI in medicine. Design, validation, and implementation of AI warrant meticulous attention.
The development of a bioethically sound foundation may be possible if it is based in the fundamental ethical principles of nonmaleficence, autonomy, and equity for considerations for the design, validation, and implementation for AI systems. Achieving such a foundation will be helpful for continuing successful introduction into medicine before consideration by regulatory agencies. Important improvements in accessibility and quality of health care, decrease in health disparities, and lower cost thereby can be achieved. These considerations should be discussed with all stakeholders and expanded on as a useful initiation of this dialogue.
Retina, Uveitis, Ocular Oncology Telehealth
2022, Ocular Telehealth: A Practical Guide

View all citing articles on Scopus

¹: InternetAdvance publication at ajo.com. April 12, 2002.

²: During the study, Ms Brothers was an employee of and Dr Blumenkranz was a consultant for Ophthalmic Imaging Systems, Sacramento, California.

^∗: The members of the Digital Diabetic Screening Group are listed at the end of the article.

View full text

Original articleThe sensitivity and specificity of single-field nonmydriatic monochromatic digital fundus photography with remote image interpretation for diabetic retinopathy screening: a comparison with ophthalmoscopy and standardized mydriatic color photography1

Abstract

Section snippets

Design

Patients

Results

Discussion

Vision problems in the U.S.A.statistical analysis

Vision disorders in diabetes

Photocoagulation treatment for proliferative diabetic retinopathy, the second report of Diabetic Retinopathy Study Findings

Ophthalmology

Photocoagulation for diabetic macular edema. ETDRS report number 1

Arch Ophthalmol

Preventive eye care in people with diabetes is cost-saving to the federal government. Implications for health-care reform

Diabetes Care

Detecting and treating retinopathy in patients with type I diabetes mellitus, savings associated with improved implementation of current guidelines

Ophthalmology

Cost-effectiveness of detecting and treating diabetic retinopathy

Ann Intern Med

Cost effectiveness analysis of screening for sight-threatening diabetic eye disease

BMJ

Cost-benefit analysis of diabetic eye disease

Ophthalmologica

Screening for diabetic retinopathy

Ann Intern Med

Are seven standard photographic fields necessary for classification of diabetic retinopathy?

Invest Ophthalmol Vis Sci

Color photography versus fluorescein angiography in the detection of diabetic retinopathy in the Diabetes Control and Complications Trial

Arch Ophthalmol

Comparison of diabetic retinopathy detection by clinical examinations and photographic gradings

Arch Ophthalmol

Cost effectiveness screening for diabetic retinopathy using a non-mydriatic retinal camera in prepaid health care setting

Diabetes Care

Diagnosis of diabetic retinopathyophthalmoscopy versus fundus photography

Ophthalmology

A report on the use of technician ophthalmoscopy combined with the use of the Canon non-mydriatic camera in screening for diabetic retinopathy in the community

Diabet Med

Screening for diabetic retinopathy

Med J Aust

Screening for diabetic retinopathythe wide-angle retinal camera

Diabetes Care

Screening for diabetic retinopathy in a clinical setting: a comparison of direct ophthalmoscopy by primary care physicians with fundus photography

J Fam Pract

Non-stereo photographic screening and long term follow-up for detection of proliferative diabetic retinopathy

Acta Ophthalmol

Original article
The sensitivity and specificity of single-field nonmydriatic monochromatic digital fundus photography with remote image interpretation for diabetic retinopathy screening: a comparison with ophthalmoscopy and standardized mydriatic color photography¹