Advances in Retinal Imaging and Applications in Diabetic Retinopathy Screening: A Review

In terms of pupil dilation, there is no international consensus on the optimal approach to DR screening. Comparative trials of mydriatic and nonmydriatic fundus photography have demonstrated that mydriasis significantly reduces the proportion of ungradable photographs, from 19–26% to 4–5%, and improves both the sensitivity (77–86% vs. 81–94%) and specificity (77–95% vs. 86–98%) of DR detection compared with undilated photography [24, 30‐32]. Despite this, for patient convenience and logistical reasons, many programs employ nonmydriatic photography, with mydriasis being reserved for cases where ungradable images are obtained (Table 2).

Diabetic retinopathy severity grading, as established by the ETDRS with the modified Airlie House classification, requires a laborious process of capturing stereoscopic photos of seven standardized 30° fundus fields encompassing a horizontal fundus viewing angle of 75°. Photos are then mounted in plastic sheets and viewed against light boards using stereo viewers at 5× magnification [33]. While this approach remains the gold standard for photographic diagnosis and grading of DR, it is clearly inappropriate for large-scale population screening [34]. Dilated single-field 45° fundus photography has a sensitivity of 76–84% and specificity of 82–92% for DR [32]. In the UK, nonmydriatic fundus cameras are used for mydriatic photography with two 45° fields (one fovea centered and one disc centered) [35]. Most other programs employ single 45° field fundus photos or two- and three-field photography for screening purposes (Table 2).

Although many current screening programs are efficient in detecting referable DR, the accurate diagnosis of DME is more challenging. An especially problematic aspect of two-dimensional fundus photography for DME detection is the inability to reliably identify retinal thickening. The English National Screening Program (ENSP) for DR employs three photographic markers, namely (1) exudates within one disc diameter of the fovea, (2) circinates within the macula, and (3) microaneurysms or hemorrhages within one disc diameter of the fovea associated with best corrected visual acuity of worse than 6/12, as surrogates for DME [35]. This approach, while reasonable within the limits of currently used imaging platforms, has limited diagnostic accuracy. Only 17% of patients with hemorrhages or microaneurysms within one disc diameter of the fovea and reduced visual acuity had thickening on macular OCT, while only 6% had macular edema. Similarly, only 27% of those with macular exudates within one disc diameter of the fovea or circinates within two disc diameters of the fovea had thickening on macular OCT [36]. A cross-sectional observational study from Hong Kong recorded a false-positive rate of 86.6% for macular edema using conventional fundus photography screening methods [37]. Among eyes proven to have macular edema on OCT, approximately one quarter to one-third are missed by fundus photography screening [38].

Several studies have investigated the incorporation of macular OCT as part of a DR screening program to determine if this approach is beneficial from a medical and cost-effectiveness perspective. Prescott et al. found that use of OCT in cases where color fundus photos suggested the presence of macular edema resulted in direct cost savings of 16–17% because of fewer unnecessary referrals without a measurable decrease in medical benefits [19]. A cost-effectiveness study of DME screening in Hong Kong found that although the screening cost per patient increased by 35% if OCT was performed on all patients undergoing color fundus photography, the cost per quality-adjusted life-year was reduced by 45% (RW, unpublished). In contrast to these promising studies, more recent work from Australia that compared DR screening using a standard color fundus photography (CentreVue DRS fundus camera, Padova) with a combined fundus camera and OCT instrument (3D OCT-1 Maestro, Topcon, Tokyo) concluded that there was no added benefit of OCT [39]. This was due to the substantial increase in ungradable fundus photos obtained with the combined fundus camera/OCT scanner that required specialist referral, negating the benefit of reduced referrals for macular edema based on OCT imaging. This negative result, however, was largely specific to the scanning instrument chosen and would likely have been overcome with the use of a dedicated fundus camera and OCT scanner setup. That said, significant challenges remain to the addition of dedicated macular OCT scans to current DR screening protocols, such as the relatively low availability of OCT machines and expertise of operators in general outpatient clinics for screening due to cost constraints.

The degree to which DR severity and thus referral and treatment urgency is assessed varies has been categorized into four levels by the ATA. Category 1 includes programs that can distinguish no or minimal DR from more than minimal DR, while category 4 enables complete categorization of DR into mild, moderate, and severe nonproliferative disease, early and high-risk proliferative disease, and whether macular edema is present (Table 2). Current DR screening programs employ ATA categories 1–3 (Table 2); as a result, the incidence of DR and DR-related blindness have fallen dramatically [35]. In tandem, the implementation of telemedicine DR screening programs is cost-effective compared with physician-based DR screening approaches and generates similar health outcomes [40].

A relative scarcity of ophthalmologists necessitates the use of specially trained non-ophthalmologist and non-physician graders for DR screening. Such graders are used in most developed countries nowadays (Table 2), and previous work has demonstrated that this approach is adequate for DR detection [41‐44]. Despite this, the growing prevalence of diabetes has placed greater pressure on healthcare systems to increase screening throughput while minimizing costs. Automated grading for diabetic retinopathy was reported in the 1990s, when Gardner and colleagues described the use of an artificial neural network capable of detecting diabetic retinopathy with 88% sensitivity and 83% specificity relative to an ophthalmologist [45]. Their system and others like it employed pattern recognition algorithms trained to identify specific retinopathy features, such as microaneurysms, and as such required time-consuming pre-processing of training images to isolate clinically important features and develop mathematical descriptors of different lesion types.

Deep Learning (DL) is a new iteration of artificial intelligence (AI) that employs convolutional neural networks (CNNs) to interpret images by repititive analysis and comparison of the output with a standard (i.e., a human grader) and self-correcting if an error is made. DL is made possible by recent advances in software programming, the availability of large data sets used to train image recognition software, and high-resolution digital imaging systems [46‐50]. Multiple studies have shown successful results in developing DL algorithms that are capable of identifying referable DR without the prior need to teach the computer systems specific features of DR. The sensitivities and specificities of these approaches are in general > 90% and 95%, respectively [44, 47‐49]. Importantly, the negative predictive value of current DL algorithms for DR detection is now upwards of 99%, with less than 1% probability of missing severe NPDR, PDR, or macular edema [51]. Current generation DL algorithms that employ standard color fundus photos have already achieved grading performance similar to retinal specialists [52, 53]. A large-scale trial of a DL algorithm trained for DR screening with a multi-ethnic data set also included additional algorithms for detection of glaucoma and age-related macular generation (AMD) [48]. Detection of possible glaucoma had a sensitivity of 96.4% (95% CI 81.7–99.9%) and specificity was 87.2% (95% CI 86.8–87.5%), while detection of possible AMD achieved a sensitivity of 93.2% (95% CI 91.1–99.8%) and specificity of 88.7% (95% CI 88.3–89.0%). More recently, DL approaches were even used for automated interpretation of macular OCT scans and demonstrated diagnostic accuracy exceeding that of trained ophthalmologists for a variety of macular diseases including DME [54].

Several DL algorithms for DR screening, including the recently FDA-approved IDx-DR algorithm [51, 55], have been reported. These can broadly be grouped into lesion-based disease detection systems and image-based (“black box”) detection systems. Lesion-based systems are trained using known disease features of DR such as microaneurysms, blot hemorrhages, and exudates. In contrast, black box disease detection systems, such as the Google Brain system [47], are trained using fundus photographs that have been graded for DR, but the specific pathologic features being detected by the algorithm remain unknown, and output is given simply as a positive or negative response to a disease state.

Clearly DL approaches hold great promise for rapid, reliable, and cost-effective screening for diabetic retinopathy. However, several important challenges remain that complicate the introduction of these algorithms into large-scale screening programs. Despite the impressive performance previously reported by the Google Brain team using an image-based DR detection system, recently presented work from Lynch et al. [56] revealed that this “black box” approach to DR screening is highly vulnerable to seemingly innocuous alterations of fundus images. Using a series of reference DR fundus photos containing typical features of DR, slight pixel modifications (0.12–0.51% of total pixels) were introduced into the images that were essentially imperceptible to human readers and did not obviously alter the appearance of the DR lesions. These altered photos were then presented to an image-based CNN previously trained with half a million DR photos. The modified images were classified as normal by the image-based CNN, while the lesion-based system still detected DR. In light of this work, the potential value of image-based CNNs in identifying novel fundus signs of DR needs to be balanced against their potential failure to detect clinically obvious disease.

Another limitation of many current DL approaches is the binary nature of algorithm development. Most contemporary CNNs are trained using normal fundus images and images with a single disease, such as DR. The variety of retinal pathologies and differences in referral urgency make this approach inappropriate in a typical heterogenous population that would attend retinal screening programs. A recent attempt to use contemporary CNNs trained with a variety of retinal pathologies, including DR, AMD, vein and artery occlusions, hypertensive retinopathy, Coats disease, and retinitis, found that diagnostic accuracy for any particular pathology dropped precipitously with each additional disease added to the diagnostic algorithm, to as low as 20–30% after the addition of ten distinct retinal pathologies [57]. It remains to be seen whether these limitations will be overcome with the development of more sophisticated DL algorithms and more powerful computer hardware. Additionally, the usefulness of algorithms trained using retinal images from a particular population may not extend to other populations [48]. While the clinical findings of DR are largely similar between races, it is possible that DL algorithms are also recognizing new DR markers that are not shared among different populations.

The narrow field of view in ordinary fundus photographs (e.g., single-field 45° or even a 7-field standard ETDRS) may exclude peripheral DR lesions. This is significant when predominantly peripheral lesions (PPLs) are associated with retinal ischemia, DR progression, and development of proliferative retinopathy [58, 59]. Eyes with PPLs were shown to have a 3.2-fold increased risk of two-step or more DR progression and a 4.7-fold increased risk for progression to proliferative disease [58]. Ultrawide-field (UWF) imaging, which can image 80% of the retinal surface, is therefore a potentially important imaging modality for DR screening. Use of UWF imaging for DR screening can achieve a sensitivity up to 99% and specificity up to 97% for DR [34]. The Optos (Daytona, Optos) and Clarus 500 (Carl Zeiss, Oberkochen) are currently available UWF imaging platforms that both enable 200° field retinal imaging in only one or two images.

Optos is a scanning laser ophthalmoscope that produces retinal images of up to 200° in non-mydriatic eyes [60, 61], representing more than three times the retinal surface of traditional ETDRS seven-field photos [62]. Such images enable identification of peripheral retinal lesions of diabetic retinopathy that cannot be seen on conventional fundus photos [60] (Fig. 1). The recently released Clarus 500 captures 133° fundus photos or 200° composited photos but, unlike Optos, is able to produce true-color images that may potentially enable more accurate identification of DR lesions, although this has yet to be demonstrated in clinical trials.

The rate of ungradable fundus photos, a common cause of specialist referral in DR screening programs, was dramatically reduced when nonmydriatic UWF was employed compared with conventional nonmydriatic multifield color fundus photography in a US-based DR teleophthalmology program involving over 25,000 patients [63]. An ungradable rate per patient for DR was 2.8% with Optos UWF compared with 26.9% with multifield photography, while the ungradable rate for DME was 3.8% with Optos UWF compared with 26.2% with multifield photography. Overall, the study demonstrated an 81% reduction in ungradable fundus photos and almost twofold increased DR detection.

At present, the major constraint to widespread adoption of UWF systems such as the Optos and Clarus for DR screening programs is largely financial—such instruments cost an order of magnitude more than conventional fundus camera setups. Despite the impressive outcomes of clinical trials, it remains unclear whether the cost savings of reduced inappropriate referrals are sufficient to justify the financial outlay [64].

OCT angiography (OCT-A) is a novel noninvasive imaging modality that can detect blood flow within the retina (Fig. 2). In diabetic retinopathy, microaneurysms and retinal neovascularization can be detected using OCT-A noninvasively [65]. Although the clinical utility of OCT-A remains to be established [66], several recent works have highlighted its potential for incorporation into DR screening approaches. Accurate noninvasive grading of DR severity by OCT-A analysis has been demonstrated via analyses of foveal avascular zone acircularity [67] and retinal plexus densitometry [68‐70]. Automation of OCT-A image analysis has also been used for quantitative assessment of DR vascular changes that correlate with visual acuity and may be useful for monitoring DR progression [71].

Modern digital color fundus cameras, although widely available in developed regions, still typically cost many thousands of dollars, are large and cumbersome, and require specially trained ophthalmic photographers. This often limits their deployment in developing regions and in areas with underserved patient populations who are less likely to attend regular ophthalmic screening. In recent years, several manufacturers have attempted to address this problem by developing fundus photography attachments for smartphones with digital cameras. Modern smartphone cameras are capable of producing very high-quality digital images the match or exceed the resolution of dedicated clinical-grade retinal cameras, albeit with smaller image sensors. Contemporary smart phones feature ≥ 10 megapixel wide-angle and telephoto cameras with optical image stabilization and are a potentially useful alternative to conventional fundus cameras.

Fundus photography using a smartphone and handheld 20-diopter condensing lens has been successfully used to capture fundus photos [72], and since then several commercial developers have released hardware attachments and software to enable convenient fundus photography. Current smartphone-based imaging solutions are based on the incorporation of additional lens elements in line with the smartphone camera and typically come with custom software that enables capture, labeling, and secure transmission of fundus photos. Outcomes of recent prospective clinical trials with smartphone-based fundus photography systems used for DR screening are summarized in Table 3. In most reports the sensitivity for DR using images captured from smartphones is comparable to slit-lamp biomicroscopy or dedicated fundus camera imaging with the benefit of far greater affordability.

Given the obvious benefits of low cost and portability, there is clearly excellent potential in smartphone-based DR screening, though the questionable ability of these imaging systems to provide fundus images through undilated pupils poses a problem for screening programs that do not employ routine dilation. Concerns about acute angle closure precipitated by dilation, particularly in East Asian populations [73], may render this approach undesirable in certain regions. Portable nonmydriatic fundus cameras are widely available and have been successfully tested for DR screening (Table 3), with diagnostic performance comparable to smartphone-based systems. These devices offer somewhat better diagnostic performance than fixed fundus cameras in terms of image gradeability and diagnostic accuracy, but the benefit of portability cannot be overstated [74‐77]. Several large trials have demonstrated the potential of these devices for detecting referable DR in developing regions [78‐81]. It is likely that the incorporation of both smartphone-based and dedicated portable fundus cameras into DR screening programs will increase dramatically in the coming years, particular in developing countries with large rural areas where clinic review in often not feasible. Recent incorporation of AI-based automatic image grading to smartphone fundus images [82] offers potential for even further improvements to the cost-effectiveness and convenience of screening programs.