Original articlePlus Disease in Retinopathy of Prematurity: A Continuous Spectrum of Vascular Abnormality as a Basis of Diagnostic Variability
Section snippets
Methods
This study was approved by the Institutional Review Board at Oregon Health & Science University and followed the tenets of the Declaration of Helsinki. Written informed consent was obtained from parents of all infants in the i-ROP study.
Results
Table 1 displays the distribution of plus disease classification (plus, preplus, or normal) for all 8 experts (labeled 1–8) and the RSD, ranked from least severe average grade (top) to most severe average grade (bottom) for dataset A (100 images). The 8 experts are displayed in the same order for dataset B (34 images). The average percent RSD agreement was higher in dataset A (82%; range, 77%–94%) than dataset B (65%; range, 29%–91%) because of the large number of normal images with good
Discussion
This study analyzes the classification of plus disease by ROP experts, with the goal of examining the pattern of diagnostic discrepancies. Key findings from this study are: (1) even among ROP experts, there is limited agreement on diagnostic classification of plus disease; (2) diagnostic discrepancy in plus disease reflects consistent systematic biases for each expert as to the appropriate cut points for plus and preplus disease; and (3) a continuous severity score, instead of discrete
References (47)
- et al.
Agreement among pediatric ophthalmologists in diagnosing plus and pre-plus disease in retinopathy of prematurity
J AAPOS
(2008) - et al.
Diagnostic Discrepancies in Retinopathy of Prematurity Classification
Ophthalmology
(2016) - et al.
Plus disease in retinopathy of prematurity: pilot study of computer-based and expert diagnosis
J AAPOS
(2007) - et al.
Measuring agreement in medical informatics reliability studies
J Biomed Inform
(2002) - et al.
Plus disease in retinopathy of prematurity: improving diagnosis by ranking disease severity and using quantitative image analysis
Ophthalmology
(2016) - et al.
Combining ROPtool measurements of vascular tortuosity and width to quantify plus disease in retinopathy of prematurity
J AAPOS
(2011) - et al.
Prognostic significance of vascular dilation and tortuosity insufficient for plus disease in retinopathy of prematurity
J AAPOS
(2000) - et al.
Detection of clinically significant retinopathy of prematurity using wide-angle digital retinal photography
Ophthalmology
(2012) - et al.
SUNDROP: six years of screening for retinopathy of prematurity with telemedicine
Can J Ophthalmol
(2015) - et al.
Dealing with inter-expert variability in retinopathy of prematurity: a machine learning approach
Comput Methods Programs Biomed
(2015)
Accuracy of retinopathy of prematurity image-based diagnosis by pediatric ophthalmology fellows: implications for training
J AAPOS
Training fellows for retinopathy of prematurity care: a web-based survey
J AAPOS
Challenges of ophthalmic care in the developing world
JAMA Ophthalmol
The International Classification of Retinopathy of Prematurity revisited
Arch Ophthalmol
Multicenter trial of cryotherapy for retinopathy of prematurity. Preliminary results
Arch Ophthalmol
Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial
Arch Ophthalmol
An international classification of retinopathy of prematurity
Arch Ophthalmol
Interexpert agreement of plus disease diagnosis in retinopathy of prematurity
Arch Ophthalmol
Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies
Arch Ophthalmol
Plus disease in retinopathy of prematurity: diagnostic impact of field of view
Retina
Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity: image analysis features associated with expert diagnosis
Transl Vis Sci Technol
Plus disease diagnosis in retinopathy of prematurity: vascular tortuosity as a function of distance from optic disk
Retina
Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis
JAMA Ophthalmol
Cited by (68)
Eye and Vision Disorders
2023, Avery's Diseases of the NewbornVariability in Plus Disease Diagnosis using Single and Serial Images
2022, Ophthalmology RetinaCitation Excerpt :Overall, most changes in diagnosis in our study (18 of 24 [75%]) were toward more severe disease, although clinicians were inconsistent in whether serial versus single image review changed their diagnosis. It has been well known that interexpert agreement in ROP classification varies with a single image because of differences among cut points of vascular abnormality required for plus disease, differences in the field of view considered, identification of different vascular parameters by different clinicians, and differences in training and education.6–12 Serial images have been previously used as a tool in telemedicine to assess follow-up after intravitreal anti-VEGF injection.31
ROP screening with the Pictor Plus camera: a telemedicine solution for developing countries
2022, Journal of AAPOSCitation Excerpt :This fact may explain the differences in sensitivity and specificity results among studies. It is important to note that there can be significant disagreements even among experts using BIO and wide-angle lens cameras for the diagnosis of plus and pre-plus disease.7,17,23 Previously published studies underscore the need for the development of technologies that improve accuracy and consistency in the diagnosis of retinal posterior pole vascular abnormalities: computer-based image analysis could be used with posterior pole images, such as the color and red-free images that are both automatically provided by the Pictor Plus digital camera, to generate quantitative and reproducible assessment scales with respect to vascular abnormalities, thus decreasing interpretive subjectivity.23
Sample complexity of rank regression using pairwise comparisons
2022, Pattern Recognition
Financial Disclosure(s): The author(s) have made the following disclosure(s): J.D.R.: Financial support – Novartis, Basel, Switzerland
M.F.C.: Consultant – Novartis, Basel, Switzerland; Scientific Advisory Board – Clarity Medical Systems, Pleasanton, CA
R.V.P.C.: Consultant – Visunex Medical Systems, Fremont, CA
Supported by the National Institutes of Health, Bethesda, Maryland (grant nos.: R01 EY19474 [J.K.C., D.E., S.O., K.E.J., R.V.P.C., M.F.C.], P30 EY010572 [J.P.C., S.O., M.F.C.], R21 EY022387 (J.K.C., D.E., M.F.C.], and T32 EY23211 (M.F.C., R.S.); the National Center for Advancing Translational Sciences at the National Institutes of Health, Bethesda, MD (Oregon Clinical and Translational Research Institute grant no.: TL1TR000129); Research to Prevent Blindness, New York, New York (J.P.C., S.N.P., J.D.R., M.X.R., S.O., K.E.J., R.V.P.C., M.F.C.); and the iNsight Foundation, New York, NY (R.V.P.C., K.E.J.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. No funding organizations had any role in the design or conduct of this research. Dr. Michael F. Chiang had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Members of the Imaging and Informatics in Retinopathy of Prematurity Research Consortium: Oregon Health & Science University (Portland, OR): Michael F. Chiang, MD, Susan Ostmo, MS, Kemal Sonmez, PhD, J. Peter Campbell, MD, MPH; University of Illinois at Chicago (Chicago, IL): R. V. Paul Chan, MD, Karyn Jonas, RN; Columbia University (New York, NY): Jason Horowitz, MD, Osode Coki, RN, Cheryl-Ann Eccles, RN, Leora Sarna, RN; Bascom Palmer Eye Institute (Miami, FL): Audina Berrocal, MD, Catherin Negron, BA; William Beaumont Hospital (Royal Oak, MI): Kimberly Denser, MD, Kristi Cumming, RN, Tammy Osentoski, RN, Tammy Check, RN, Mary Zajechowski, RN; Children's Hospital Los Angeles (Los Angeles, CA): Thomas Lee, MD, Evan Kruger, BA, Kathryn McGovern, MPH; Cedars Sinai Hospital (Los Angeles, CA): Charles Simmons, MD, Raghu Murthy, MD, Sharon Galvis, NNP; LA Biomedical Research Institute (Los Angeles, CA): Jerome Rotter, MD, Ida Chen, PhD, Xiaohui Li, MD, Kent Taylor, PhD, Kaye Roll, RN; Massachusetts General Hospital (Boston, MA): Jayashree Kalpathy-Cramer, PhD; Northeastern University (Boston, MA): Deniz Erdogmus, PhD; Asociacion para Evitar la Ceguera en Mexico (APEC) (Mexico City): Maria Ana Martinez-Castellanos, MD, Samantha Salinas-Longoria, MD, Rafael Romero, MD, Andrea Arriola, MD, Francisco Olguin-Manriquez, MD, Miroslava Meraz-Gutierrez, MD, Carlos M. Dulanto-Reinoso, MD, Cristina Montero-Mendoza, MD.
Author Contributions:
Conception and design: Kalpathy-Cramer, Campbell, Erdogmus, Sonmez, Swan, Chan, Chiang
Analysis and interpretation: Kalpathy-Cramer, Campbell, Erdogmus, Tian, Kedarisetti, Moleta, Reynolds, Hutcheson, Shapiro, Repka, Ferrone, Drenser, Horowitz, Sonmez, Swan, Ostmo, Jonas, Chan, Chiang
Data collection: Kalpathy-Cramer, Campbell, Erdogmus, Tian, Kedarisetti, Moleta, Reynolds, Hutcheson, Shapiro, Repka, Ferrone, Drenser, Horowitz, Sonmez, Swan, Ostmo, Jonas, Chan, Chiang
Obtained funding: Chiang, Erdogmus, Kalpathy-Cramer, Chan
Overall responsibility: Kalpathy-Cramer, Campbell, Moleta, Chan, Chiang
- ∗
Both authors contributed equally as first authors.