Consistency and Standardization of Color in Medical Imaging: a Consensus Report

The following sections represent descriptions of current challenges in the use of color for a variety of medical disciplines that rely on color images for diagnostic purposes. Within each section, we summarize the main consensus recommendations for addressing the issues and challenges described. Color applications in medical imaging include color for annotations and thresholding where color is not usually critical, pseudo-color applications where the displayed color has no correlation with the color of the object being imaged (e.g., visualization of calculated data and fused images), and, finally, true-color applications where the displayed color is intended to represent the actual color of the imaged object (e.g., dermatology and medical photography).

Although a variety of modalities in these disciplines utilize digital images including telepathology and digital microscopy, whole-slide imaging (WSI) is the modality most likely to drive the adoption of color standardization. Applications include research and clinical uses of pathology imaging for visual assessment, and (importantly) the automated assessment of images with image analysis software algorithms. Pathology samples are stained with a wide variety of colored dyes. With conventional light microscopy and human interpretation, variability in color is often not a critical issue, perhaps because in such a viewing environment, it is easy to adapt to differences and it is often easy to simply stain another piece of the tissue sample. The increasing use of digital pathology, however, highlights the color differences between samples, laboratories, and scanning and display systems (see Fig. 1). There are noticeable color differences in these situations, which are visible to the observer and affect the results of image analysis algorithms.

There is a significant variability in the color of images in this domain introduced at almost all stages of the process from sample acquisition through slide preparation, imaging, and transmission to the display device. A distinction can be drawn between color variation occurring before imaging (a significant task requiring standardization of tissue acquisition, fixation, processing, and staining) and during or after imaging (addressable using end-to-end color calibration techniques).

Initial efforts to standardize color using various color transformations have been described [1]. However, color accuracy is only one factor affecting the “fitness for purpose” of WSI for diagnostic use. Other factors include the resolution, contrast, and dynamic range of the image. Because there is significant variability in histochemical staining between and within laboratories, standardizing all pre-imaging sample handling steps (acquisition, fixation, processing, sectioning, and staining—known as “pre-analytic variables”) is a difficult task. Even with these additional variables, imaging systems should be able to generate an image with consistent color presentation irrespective of the instrumentation or devices used.

In addressing the variability introduced during and after imaging, current major roadblocks include the significant variation in performance between WSI scanners (even those from the same vendor), the lack of standardized color calibration, and the difficulty in designing and constructing appropriate test objects for scanners. Preliminary work in developing such a target has been conducted [1]. Key challenges include obtaining agreement between pathologists on what is the ideal color for a stain, defining end-to-end image standardization standards, and developing technology and infrastructure to apply standardization.

For WSI, end-to-end color management within the digital pathology workflow is required with special emphasis to the imaging chain stages of image capture, processing, and display. Ideally, this can be achieved with a multidisciplinary approach involving pathologists, color scientists, engineers, and digital pathology vendors with appropriate “targets” for scanner calibration. Previous work has showed that the application of appropriate color mappings can reduce interimage variability. In addition, there is no standard method of assessing dynamic range and there is some subjective evidence that the current implementations of WSI systems may not retain sufficient dynamic range from capture to display. The choice of the illumination condenser and light source plays a role in image quality. Although stains applied to tissue obey Beer’s law, where absorbance is proportional to concentration, some histologic preparations are impregnations or result in the deposition of particles that result in the refraction of light, rather than absorbance. As a result, silver-impregnations, and particle-based detection of IHC (DAB chromogen) are visualized based on their refractive properties. Depending on the optical path geometry and illumination design of the instrument, these objects may not replicate what is observed in manual microscopy.

Consistent color approaches in WSI require efforts to characterize and address the problem of color variation, including collaborative work on developing a test object. An intermediate step is to formally investigate the extent of color variation between scanners, and reasons for this variability. For example, appropriate test or calibration slides may reveal differences between scanner optics, on-board software or image storage and transmission. Such an effort will require cooperation from WSI users and vendors. Long-term aims for this work should include international standards for color reproduction, technology to apply standardization from the glass slide to the display, and a better understanding in the general pathology community of the importance of such work.

Furthermore, agreement on standardized tissue handling protocols and stain standardization may be useful at the preimaging stage including input from regulatory bodies such as the US Biological Stains Commission and equivalent international organizations. Further study of the causes of variation in color at the preimaging stage may also be necessary in the broader context of reducing variability in pathology images and automatic and computer quantitative analysis of images.

Endoscopes are used by almost every medical specialty: gastroenterology, pulmonology, gynecology, urology, orthopedics, anesthesia, ENT, thoracic surgery, and general surgery. Color plays an important role in endoscopic [2] as well as in laparoscopic [3] imaging. The main purpose of endoscopy/laparoscopy is to visualize a certain object inside the human body, such as internal organs, from outside. To visualize the internal organs, one must deliver the light into the human body to shine the object, image the scene, transmit the image data back to the outside, and finally reproduce the image for human readers. Normally, internal organs in vivo are not visible by a naked eye because the human body blocks visible light. Thus, the key of various endoscopy and laparoscopy technologies is to get around the barrier such that the light can be delivered and the image can be retrieved. Different devices address this problem with different approaches: the straightforward method is to perform surgery (e.g., thoracic or general) and gain direct access to the object. Laparoscopy is a less intrusive alternative to surgery. Endoscopy gains access through the cavities of the human body.

Conventionally, a tubular device is used to house both the light delivery channel and the image retrieval channel for either passing through or getting around the barrier. Capsule endoscopy takes advantage of the open circuit of the human gastrointestinal tract. The capsule travels the gastrointestinal tract passively while capturing and storing the images autonomously. Table 1 compares various endoscopic and laparoscopic methods by parameterizing each component in the imaging chain with examples.

A critical aspect for identifying color-related issues in endoscopic and laparoscopic systems is a review of the makeup of the imaging chain. The generalized imaging chain consists of object, illumination, light guide, detector, image guide, intermediate image or video file, video processor, display, and human reader. In laparoscopy systems, the illumination source is typically a xenon lamp. In endoscopy, the illumination source can be xenon/halogen light, filtered narrow-band light guided by a fiber optic bundle at the proximal end, or light-emitting diode lights embedded at the distal end. The endoscope itself can be rigid, flexible, or in a capsule form. The image detector can be an external video camera at the proximal end, or an embedded camera at the distal end. While most flexible endoscopes transmit image data electrically via internal wires, capsule endoscopes can transmit image data electrically via human body, electromagnetically via radiofrequency signaling, or physically via memory storage. The capsule systems also require the image data to be saved as intermediate files for off-line processing and review. The video data is rendered by the video processor in real time for flexible endoscopes, or off line for capsule endoscopes. The video data is presented to the endoscopist by an electronic display device during the procedure and can be archived in a picture archiving and communications systems (PACS) or as part of the electronic medical record (EMR) system for future review.

In these modalities, it is important to clarify related topics: color reproducibility, color consistency, color characterization, and color standardization. Color reproducibility means that the original scene can be faithfully reproduced, either colorimetrically or perceptually, on the final display. Color consistency means that the relationship between the input scene and the output image (i.e., the color mapping) remains constant. Color characterization means to measure and to determine the relationship between the input and output of a component/system with a certain color target. Color standardization means that the input and output of a component in the imaging chain are well defined such that products from different vendors are interchangeable (i.e., interoperability).

Surgeons who perform both open surgery and laparoscopic surgery may see slight differences in color when observing the same tissue. During open surgery, the surgeon looks at the tissue directly with his/her naked eyes while the tissue is illuminated by the lighting in the operating room. During laparoscopic surgery, the same surgeon observes the same tissue via the laparoscope and can therefore compare the color of the laparoscopic image to his/her recollection of how the tissue appeared when viewed directly with the naked eye. However, in a video-based (e.g., flexible or capsule) endoscopic procedure, the endoscopist observes a reproduced electronic image of the object illuminated by a nonstandard light source. If the endoscopist does not normally see this same tissue with his/her naked eye, but always uses an endoscope, the endoscopist believes the true color of the tissue is similar to the endoscopic image. Results from surveys of endoscopists and laparoscopists described at the Summit demonstrate that although a true color match between real-life color and the color on the display appears to be more important for laparoscopy than for endoscopy, most participants agreed that color standardization is beneficial for the advancement of the technology and its clinical applications. However, no misdiagnoses have been reported due to unfaithful color reproduction. Endoscopists rely on a pathologist’s examination of the endoscopic biopsy for the diagnosis. Therefore, nonideal color reproducibility of endoscopy/laparoscopy devices does not lead to safety concerns.

Achieving color reproducibility remains difficult since current endoscopes cannot emit light evenly yielding color variations with brightness. To date, the design goal of current endoscopy systems is not color reproducibility but user preference. In addition, since determining the original scene with existing endoscope products is not feasible, one can use the image shown by the onsite display to the endoscopist for establishing the diagnosis truth. However, color characterizing the onsite display is also challenging (see the Medical Display Section) and providing an identical or similar image on other display devices remains difficult even with a modification of the digital imaging and communications in medicine (DICOM) standard. This level of color consistency for the display of color images requires characterizing the input and output devices and adding new tags into the DICOM or ICC format, which are feasible but practically challenging. Current devices include quality assurance/quality control (QA/QC) procedures and criteria not based on the original scene but on the color consistency requirements imposed by the manufacturers.

In summary, color standardization is not critical in endoscopy if the components of the system are from the same vendor and managed within a unique framework. When systems consist of varying components or devices from different vendors, standardization should be helpful for achieving consistency. Even though the laparoscopic system is also designed in the same manner, it is physically possible to be composed of mix-and-match components from different vendors and thus color standardization would be preferred. In addition, color standardization is important if color consistency is required when endoscopy video is archived or recorded for future review with identical or completely different systems. Overall, faithful color reproduction is technically challenging for these modalities and currently not in demand by the user community.

Mydriatic fundus photography is one of the most prevalent forms of retinal imaging. This type of imaging is used in a clinical ophthalmology setting, most commonly as a mechanism of documenting the visual appearance of a patient’s retina on the day of their visit. It is also used as a secondary tool in clinical trials and research, as well as in teleophthalmology, as replacement for in-person patient visits. The relevance of color in all of these contexts pertains to accuracy as well as consistency [4, 5]. In order for the physician to be satisfied that the image is a reasonable reproduction of the patient’s retina as a medical record, it should match, as closely as possible, the view the ophthalmologist has during examination with an indirect ophthalmoscope. Consistency of color includes consistency between images from a single camera as well as between different cameras or cameras from different manufacturers [6‐9]. Digital fundus cameras can be compared by photographing the same eye with each camera and analyzing the differences between them using a known standard as reference with results showing significant variability from the same patient and same eye (see Fig. 2). Testing can also include photographing several known test color patches inside an eye model with all cameras and averaging the final captured color at the center of the image.

Typically, digital photographic cameras do not allow direct file access. Investigations on what differences may exist between processed (TIFF) and unprocessed (RAW) file formats in terms of color differences will contribute to understand the effect of proprietary processing after capture. The most significant barrier to the implementation of color correction and management in ophthalmology is the lack of standardization. Identification of a standard set of colors and a common method of imaging these colors in order to generate color profiles is needed. A common method inspired on established methodologies based on the Macbeth Color Checker (see Fig. 3 for an example test target proposed for system calibration), with an eye model and a standardized approach for profile generation, would result in more consistent color across cameras and systems.

Medical photography uses noninvasive visible-light photographs captured by off-the-shelf or specialized equipment. These include whole-body photographs and detailed close-up photography of the skin, including digital dermoscopy (see Fig. 4) and multispectral imaging [10], primarily for diagnostic purposes, requiring precise color rendering to avoid erroneous conclusions. Typical use cases are in telemedicine applications with a remote specialist having access only to the image and supplementary information about the patient or as input to a computer-aided diagnostic tool where color is a feature in a machine learning classification [11]. In these cases, precision and consistency of color reproduction is critical to achieve correct clinical decisions. Inconsistency and poor accuracy in color rendering in the recording, communication, and interpretation stages cannot only affect the telemedicine diagnosis, but also the subsequent evaluation of effectiveness of treatment based on image records. Color accuracy is sensitive to the image capture process and varies widely depending on the equipment, method, and lighting environment. Camera settings, most critically white balance, and ambient light conditions must both be considered in the capture process to obtain optimal color rendering [12]. Most casual users of photographic equipment are not trained in how to employ best practices for acquiring color images, which inevitably leads to unacceptably large variation in color. Greater training and a reliable process are needed to improve the color quality of medical images.

The establishment of best practices guidelines for the photography of color medical images and the definition of a set of reference test targets or images for various modalities can contribute to ensure consistent color renderings. In addition, efforts in this specialty should include the use of standard image post-processing color correction methods. This would allow an investigation into the way diagnostic errors relate to inconsistency of color reproduction in various conditions and modalities.

In dentistry, photographs are widely used by a diverse group of stakeholders who need to be able to communicate effectively with each other. Stakeholders include dental providers, technicians, dentists, specialists (including orthodontists), prosthodontics, periodontics, endodontists, oral and maxillofacial radiologists, pathologists and surgeons, dental manufacturers, laboratories, and camera hardware and software vendors. Color fidelity in dental photography is relevant in the clinical areas of treatment planning, esthetic dentistry and implant treatment, and in root fracture risk assessment [13-20]. In diagnosis and treatment planning, providers look for pockets, plaque deposits, and abscesses to treat diastema gingiva alteration, chin remold, and malocclusions alterations, all of which are affected by color techniques.

Prosthodontics use visible light photography for matching shade, fit and restoration of implants, crowns, bridges, veneers, inlays, and the complete and removable partial dentures. When making decisions about color, the clinician must consider the lighting conditions at the image acquisition stage including the color temperature of the light source. In periodontics, studying and treating diseases of the periodontium tissue as well as placement and maintenance of dental implants requires direct, real-time visualization and magnification of the subgingival tooth root surface and recessions, aiding the identification of deposits on the tooth surface.

Color is also of relevance in esthetic dentistry where color matching includes such nuances as hue, tint, translucency, refractive index, and these depth-related textures vary substantially across the visible crowns of individual teeth and need matching with their contralateral front counterparts. These subtle variations need to be recorded and accurately reproduced during preparation of prosthetic tooth crowns. Oral and maxillofacial surgery includes extractions, implants, and facial surgery. Surgeons diagnose and treat oral cancer and other diseases in the maxillofacial region. They use visible light photographs to monitor changes in color, detect white spot lesions, and gray and black discolorations to help with oral cancer assessment. In oral and maxillofacial pathology, both clinical disease progress of soft tissue lesions and also color in light microscopy of biopsy specimens are important. In endodoncy, tooth discoloration is an important determinant of pulp short-term bruising versus necrosis. Endoscopes also are commonly used to look inside the tooth roots during root canal preparation for root apex obituration.

In summary, proper handling of color in dentistry is hindered by lack of interconnectivity and consistency among proprietary systems. These effects can be minimized by providing education on color spaces and their influence on imaging tasks to dental health providers including training on the effect of the spectral characteristics of the ambient illumination through technical reports from recognized industry bodies such as the American Dental Association. In addition, implementing the DICOM WG 22 (Dentistry) color framework with appropriate targets and phantoms for color calibration of dental imaging systems will contribute to achieving consistency while ensuring interconnectivity among systems through integrating the healthcare enterprise (IHE) profiles.

Display systems are key components for the visualization of color medical images as evidenced by recently published reports. Display devices and approaches are relevant to the discussions of previous section in this article whenever the color image is to be presented to a human. Recent work demonstrated that color displays suffer from limited primary stability [21], which leads to color gamut shrinkage, color shift, gray imbalance, and contrast reduction. A quantitative method was provided to measure primary stability. In addition, color displays have large variability related to color of the white point, color point of neutral gray values, and even color differences within the entire color gamut, even when color management experiments attempt to reproduce different tone curves and white points on the display screen adjusted to DICOM grayscale standard display function (GSDF) [22].

Color calibration of displays for medical applications is necessary to guarantee stability of display systems over time and to ensure similar behavior between different display brands and types. There is no strong clinical evidence that such calibration increases diagnostic efficacy. However, some studies indicate significant improvement in practitioner efficiency. In fact, the grayscale DICOM GSDF was defined exactly for this purpose. Furthermore, appropriate measurement methods and the quality sensors for display characterization and calibration with accurate color calibrations of at least 11 bits for the internal display pipeline are beneficial. It is unclear what type of calibration is needed for medical applications, and whether each modality or specialty should have its own calibration method variant. Some modalities (e.g., dermatology, ophthalmology, and medical photography) would benefit from an absolute color calibration such that visualized images closely reflect reality, for instance, using color calibration charts commonly employed in professional photography and print [5]. However, there is no agreement concerning the type of calibration required for WSI, digital microscopy, histopathology, endoscopy, and laparoscopy. Absolute color calibration typically results in a reduction of color gamut, contrast, and luminance [23]. Therefore, absolute color calibration may not be the best alternative for these modalities. Perceptually, uniform calibration approaches resulting in color differences appearing equal in strength seem to be beneficial in all cases.

In addition, it has to be noted that display calibration is sensible only when the entire imaging chain is calibrated. In that context, architectures with a flexible end-to-end calibrated imaging chain are useful (see Fig. 5). For example, ICC profiles [24] describing the color space (range and metrics of the colorimetry of the device) and color behavior of the system or components could be used throughout the imaging chain. This would likely require defining a new rendering intent for medical content. At the display side, it would require defining a color extension of the widely accepted GSDF [22]. The display systems would need to be calibrated to reflect the newly defined color display function. However, the calibration of tablet and smart phone displays may prove to be particularly difficult, since most of these devices do not currently support ICC profiles.

Telemedicine uses digital images for remote diagnosis and management of medical conditions to reduce patient and clinician travel, improve access to subspecialist care, decrease wait times, and to reduce costs. Optimal acquisition and display of color medical images is critical to the diagnostic process as it impacts accuracy, discrimination, and consistency. Three aspects are relevant to color imaging in telemedicine as follows: acquisition, rendering, and display.

Teledermatology (real-time and store-forward) shows why acquisition is important [25]. Providers send patient history and image data to a dermatologist who provides diagnostic and treatment recommendations, second opinions, monitoring, and/or e-learning, generally with a high degree of diagnostic accuracy (see Fig. 6). Conditions impacting color quality include lighting, camera settings (e.g., focus, flash, and white balance), compression, and views. Image rendering can improve quality, for example by using a spectrum-based reproduction system that produces high-fidelity color reproduction in different lighting environments. This type of system uses color or multispectral cameras (e.g., six-band video camera), devices for illumination spectrum measurements, calibrated color displays, and spectrum-based color conversion [26, 27]. Multispectral images from dermatology, surgery (see Fig. 7), and pathology have been rated by clinicians as achieving higher color reproducibility, better image fidelity, and superior appearance of material surface compared to conventional RGB-based images. Multispectral images also allow for quantitative color analyses that could improve diagnostic interpretations. For display, calibration and characterization protocols can be considered for high color accuracy. One pathology study [28] has revealed a slight advantage diagnostically for a properly calibrated and color-managed display and a significant advantage in terms of workflow.

Although teleclinicians emphasize the importance of accurate color in diagnoses and other clinical applications that use color images, there is a significant lack of data showing that poor or inappropriate calibration or standardization of acquisition and display devices can affect diagnostic performance and therefore there is no universally accepted image quality program for color displays. Although some technical telemedicine guidelines do exist, they rarely incorporate color calibration issues.

To optimize acquisition and display of color medical images, the consensus recommendations included standardization of data formats, exploration of automated color correction options, the development of methods and criteria for color reproduction capabilities of input and display devices, and performing studies to quantify the impact of color quality on a variety of clinically relevant diagnostic tasks from the perspectives of both the human observer and computer-based image analysis schemes.

To facilitate consistent and appropriate handling of color in telemedicine, the key recommendations included developing training methods for improving basic medical photography skills, incorporating standardized yet application-relevant color considerations into standards and guidelines documents, and increasing awareness of the importance of color quality.

Standards are critical to color in medical imaging as they enable regulatory review and facilitate innovation. A well-specified implementation pathway with good quality, calibrated display systems supports greater diagnostic effectiveness and consistency. There is a complex web of standards-making bodies active in topics relating to color in medical imaging. Foremost among these are DICOM, American Association of Physicists in Medicine (AAPM), ICC, International Commission on Illumination (CIE), and International Engineering Consortium (IEC). DICOM specifies an extensible medical imaging file format that supports a variety of imaging modalities, can carry many different types of metadata, and is associated with an object-oriented network protocol and storage and retrieval services [29]. AAPM is responsible for standards in the areas such as radiology, cardiology, pathology, and dermatology. ICC defines the file format that carries color transforms (look-up tables) [17], and also documents the architectures and workflows for using the transforms. CIE is the international standards body for color and illumination, with interests in color, vision, lighting, and imaging applications. Other standards-making bodies active in this area include IEC, Video Electronics Standards Association (VESA), International Committee for Display Metrology (ICDM), and American College of Radiology (ACR). Although many of the necessary standards exist, they are not widely implemented in medical imaging. DICOM in particular supports advanced color capabilities through embedded ICC profiles, yet few if any actual products have the capability to write ICC profiles into the DICOM file, or to parse the profile and apply the transform correctly in the workflow. Another area where medical color applications would benefit from more rigorous application of standards and performance testing is in color displays. There is also some development work needed to define best practices for ambient lighting, multispectral imaging (imaging with an extended range and multitude of color sources), and archival color. A path toward improved standardization should include the implementation guidelines for each DICOM imaging modality and corresponding conformance tests, and the availability of source code for baseline implementations, possibly by connecting the ICC’s open source “Sample ICC” color management module (CMM) with a suitable open source DICOM viewer.

Beyond the need for implementation support of existing technologies, DICOM standards need to provide support for camera RAW image data to derive the colorimetry of a scene and for multispectral image data. In addition, other standards activities including AAPM and IEC work to evaluate color spaces and white points (due to report in 2014), and studies for objective comparison of colorimeters used in display calibration and profiling are in progress. These efforts will likely be useful if the standards bodies coordinate activities through liaisons in technical committees.

It is clear from the number of attendees and level of participation in the color summit that experts from many areas of medical imaging realize the need for some degree of standardization of color in medical imaging. In some areas, color communication is to some extent successful based on the adoption (either deliberately or by default) of a sRGB model. In most of the identified areas, such a model is limited and a broader framework is needed. This broader framework should be able to provide support for sRGB images while at the same time providing for new technologies such as wide-gamut displays, scene capture, and multispectral imaging. The ICC model seems to be a good candidate and this was one of the reasons that led to the setting up of the ICC medical imaging working group to provide a focus for efforts to standardize color and to ensure that work in different medical imaging disciplines is well coordinated.

It is unrealistic to expect instant universal adoption of any proposed color framework. That said, for some modalities such as digital histopathology and ophthalmology, the lack of a well-defined color framework is limiting the use of digital tools that could maximize the utilization of medical devices for improved diagnostics. The recommendations in this paper are being implemented with the leadership of the recently created ICC medical imaging working group. The role of the ICC will be to provide a forum for discussion and implementation in close liaison with other medical imaging communities in order to ensure good adoption of well-defined color framework.

Moreover, user education in the area of color capture, processing and display, and future conferences with a focus on color should be considered for in-depth coverage of color issues for each imaging modality. In addition, the development of standard test methods for common functions and opportunities for manufacturers to test interoperability of systems can contribute further to a solid color framework across medical imaging applications with close collaboration with relevant standardization committees.

In summary, a standard framework for color would provide several benefits for medical imaging applications. In some application areas, this benefit is expected to be substantial; whereas in other areas, the improvement is likely to be marginal. As a result of the discussions during the summit, the following candidate work items were identified.