Use of whole slide imaging in surgical pathology quality assurance: design and pilot validation studies

Summary

By imaging large numbers of slides automatically at high resolution, modern automated whole slide imaging (WSI) systems have the potential to become useful tools in pathology practice. This article describes a pilot validation study for use of automated high-speed WSI systems for surgical pathology quality assurance (QA).

This was a retrospective comparative study in which 24 full genitourinary cases (including 47 surgical parts and 391 slides) were independently reviewed with traditional microscopy and whole slide digital images. Approximately half the cases had neoplasia in the diagnostic line. At the end of the study, diagnostic discrepancies were evaluated by a pathology consensus committee.

The study pathologists felt that the traditional and WSI methods were comparable for case review. They reported no difference in perceived case complexity or diagnostic confidence between the methods. There were 4 clinically insignificant discrepancies with the signed-out cases: 2 from glass slide and 2 with WSI review. Of the 2 discrepancies reported by the WSI method, the committee agreed with the reviewer once and the original report once.

At the end of the study, the participants agreed that automated WSI is a viable potential modality for surgical pathology QA, especially in multifacility health systems that would like to establish interfacility QA. The participants felt that major issues limiting the implementation of WSI-based QA did not involve image acquisition or quality but rather image management issues such as the pathologist's interface, the hospital's network, and integration with the laboratory information system.

Introduction

Quality assurance (QA) is an important part of surgical pathology practice and is mandated by the College of American Pathologists (CAP) in their accreditation of laboratories. Although the CAP publishes a series of checklists to guide surgical pathology QA, the specific implementation is intentionally left flexible to accommodate the wide variety of pathology practices that exist across the country. For example, although the CAP checklists require second review of surgical pathology reports in certain circumstances, the method and implementation of this QA are not specified. Typical approaches include (1) intradepartmental review, (2) second opinion requested by patient or clinician, (3) interdepartmental review, (4) extradepartmental review, and (5) cases reviewed (over-read) by a second pathologist as part of departmental QA policies [1].

Although many approaches to surgical pathology case review are possible, case over-reads, in which a second pathologist reviews cases as part of a departmental QA policy, represent the overwhelming majority of surgical pathology QA in general practice. In a typical practice, a percentage of signed-out cases are picked at random and distributed to a second pathologist for review. Discrepancies are noted and resolved between the pathologists, and if necessary, amended reports or addendums are issued. These QA programs can be quite extensive because as many practices review 10% (or more) of cases.

There is good reason for extensive QA programs in pathology. A wide range of anatomic pathology error rates is reported in the literature [1], ranging from 1% to 43% of all received specimens. This broad distribution reflects varied methods used to detect errors as well as differences in error qualification. Raab et al [1] have estimated that the actual error rate likely ranges from 1% to 5%, and in a recent study of self-reported discrepancies among 72 institutions, 6.7% of anatomic pathology diagnoses were found to be discrepant at second review. Significantly, in 1% of these cases (0.067% of all cases), a significant clinical event occurred as a result of these discrepancies.

Statistics based on second review (such as the previous data) likely reflect an underrepresentation of errors because as traditional case over-read is associated with a number of potential biases. For example, the reviewer often knows the original diagnosis and/or the identity of the sign-out pathologist. It has been shown that knowledge of the original diagnosis affects the sensitivity of review in cytopathologic specimens [2], and although not yet studied, it is likely that second review of surgical pathology specimens is similarly affected. Furthermore, reviews are usually done at the same facility at which the case was signed out. This could hide or even enhance local biases and could be a significant problem, especially for large, multiple facility health systems that would like to establish a uniform level of quality across their enterprise. A major hindrance to establishing a multifacility QA program is the expense and difficulty of moving and managing slides between facilities, especially if the QA is to be done close to the sign-out date.

Automated whole slide imaging (WSI), in which all the slides in a case are imaged in their entirety at high resolution and made available to pathologists on a network, is a modality that may prove useful in surgical pathology case review. A digitized case could allow a QA system to hide the original sign-out pathologist and, if desired, original diagnosis. More importantly, however, digital slides available on a network can mitigate the problems of glass slide logistics and, by so doing, enable routine multifacility surgical pathology QA.

In this article, we report the results of a pilot study in which surgical pathology case review based on digital slides created by an automated high-speed WSI robot is compared with traditional surgical pathology review based on direct microscopic examination of the glass slides.

Early attempts to incorporate digital imaging in surgical pathology practice centered on microscope-mounted cameras. These systems became popular in the early 1990s with the availability of moderately priced video/digital cameras, reasonably sized hard drives (tens to hundreds of megabytes), local area networks, and the Internet. For example, in 1994, Schubert et al [3] presented a “pathologist designed imaging system for anatomic pathology sign out, teaching and research.” The system, designed for permanent storage of static images and their incorporation in sign out and reporting, involved multiple cameras (640 × 480 pixels with 24-bit color), network connectivity to an image server and the pathologists' workstations, a RAID 5 (Dell, Inc, Round Rock, Tex) storage device and database, image capture, and image display software. The system was later integrated into the CoPath Laboratory Information System (LIS) (Cerner DHT, Inc, Waltham, Mass) [4]. Similar systems [5] are still widely used today for gross image management and for microscopic single-field documentation. However, their use is limited; at our institution, less than 1% of cases get a microscopic image. There are numerous reasons for this [6], but perhaps, the most important is that single-field, camera-on-microscope systems do not document the entire slide, forcing the pathologist to find and capture limited fields of interest. This causes the pathologist to become a photographer, takes significant time, and results in images that subsample the case and do not necessarily stand on their own.

Another approach to imaging in pathology was robotic microscopy; these systems have been reasonably successful in telepathology with tens of thousands of successful cases documented since the mid-1990s [7], [8], [9]. However, these are largely real time, remote control systems that require hands on involvement of the pathologist and do not provide (as part of the core operations) permanent image storage.

In the late 1990s, pathologists began to experiment with systems that imaged and permanently stored the entire slide (or parts of the slide) at reasonably high resolutions. In 1997, Joel Saltz and his group (Ferreira et al [10]), presented a system for “Enhanced Field Microscopy” in which a robotic microscope captured a large area of a slide, field by field, and a computer then “knitted” the individual fields together into a montage. The development was a significant advancement and was recognized as such by awards at the 1997 American Medical Informatics Association Fall Symposium. The system did have significant limitations, however, most significant was the long time (often many hours) required to capture a single extended field. That said, “slow” WSI based on traditional robotic microscopes are still used today for education and proficiency testing [11], [12].

Automated high-speed WSI began in 1999 when Art Wetzel and John Gilbertson, then of Interscope Technologies (Pittsburgh, Pa), developed a fully automated, high-speed device that can image entire slides at high resolution and at a reasonable cost. In late 1999, a prototype with a fully functional robot was developed at the University of Pittsburgh Medical Center (UPMC), and in March 2000, it was presented at the International Academy of Pathology Meeting in Nagoya. It was based on traditional microscope optics, a strobe light linked to a precision stage, and a digital video camera. With a primary magnification of ×20, a numerical aperture of 0.7, and square 6.6-μm pixels, it had a spatial sampling period (pixel size/optical magnification, a measure of resolution) of 0.33 μm/pixel and can image a slide in 5 to 10 minutes depending on the size of the tissue section and the amount of image compression desired [6].

Since 2000, there has been a small explosion of companies producing increasingly capable automated high-speed whole slide imagers. A typical imaging robot today can run in batch mode (reading barcodes on slides) and can capture and compress an image of a slide with a tissue section measuring 1.5 × 1.5 cm in approximately 6 minutes with spatial sampling periods of between 0.3 to 0.5 μm/pixel. The high-speed WSI robot industry is becoming highly diverse with a wide range of optics, detectors, slide handling devices, and software, resulting in an increasing range of capabilities and costs. Newer devices are implementing nontraditional optics, illumination, and sensors designed specifically for very high speed image capture [13] and should result in significant improvements in speed, throughput, and resolution in the months and years ahead, with different manufacturers eventually focusing on different aspects of the market.

The rise of automated, high-speed, high-resolution WSI should have a significant impact on pathology practice because, for the first time, pathologists have a device that can digitize large numbers of slides automatically. A digital slide could allow pathologists to better apply computational power and network connectivity to pathology practice, resulting in new capabilities and greater productivity similar or greater than those seen with the rise of the LIS and digitalization of the pathology report. However, although there are several published studies describing the use of WSI in education and research [14], [15], [16], there are only limited papers describing automated high-speed WSI in clinical practice and only a handful describing a direct comparison of automated WSI with traditional microscope examination in clinical activities [13]. This is particularly disappointing in that there are some clinical activities in which the advantages of a digital slide (eg, the ability to access the image across a wide network) may make WSI useful even if the image quality is not quite as good as direct examination of a slide under the microscope.

As mentioned previously, anatomic pathology QA offers such a potential niche. Anatomic pathology QA is an important clinical activity, and most institutions have policies that mandate some percentage of cases be reviewed (over-read) by a second pathologist. If discrepancies are noted, the case is discussed, and if necessary, the report is supplemented with an amendment or an addendum. Quality assurance is normally operated as an intrafacility activity even in institutions with multiple facilities such as UPMC or the US Air Force largely because of difficulties in glass slide logistics. A strong argument could be made that some amount of interfacility over-reads would be desirable and may help standardize pathology practice and reporting across large enterprises. Furthermore, it is reasonable to postulate that digital representations of the pathology cases, created by automated systems and readily available over networks, may be the tool necessary to make interfacility over-reads a practical reality. Finally, anatomic pathology QA is a mandated real-world clinical activity that has a set of protocols and outcomes that can act as a realistic background for the evaluation of automated WSI systems and the clinical space.

In this article, we report the results of a pilot study in which surgical pathology case review based on digital slides created by an automated high-speed WSI robot is compared with traditional surgical pathology review based on direct microscopic examination of the glass slides.