Technical Challenges of Enterprise Imaging: HIMSS-SIIM Collaborative White Paper

Consumer photographic devices such as smartphones, tablets, as well as point and shoot and low cost single lens reflex (SLR) cameras make for inexpensive and apparently easy image capture of both still photographs and video. Additionally, many of these devices open the options for dedicated medical image recording applications. However, there are features of these devices that may insidiously interfere with the medical image acquisition requirements. A full best practices discussion would deserve its own white paper but here are some basics:

Staff should enlist the aid of a professional medical photographer to create a cheat sheet of settings for each device and the room environments in which they are used.

Consumer devices are often limited in the range of file formats that they are capable of storing. They may be network connected (e.g., via Wi-Fi or cellular connections) to enable file transfer, but do not usually implement healthcare-specific interoperability standards, though smartphones and tablets may support dedicated healthcare image and video capture applications.

Many health care enterprises have a dedicated medical photography department staffed by certified professionals, which provides clinical, educational, and research imaging services for other specialties The history of medical photography long predates the availability of digital cameras [24]. Professional medical photographers need expertise not only in photographic technique but also the relevant anatomical and procedural information [25].

In some cases, high volume service lines utilizing visible light imaging such as ophthalmology have their own dedicated medical photography section. While some specialties and applications may not require photography-trained staff, for other specialties, medical photographers play a critical role in a patient’s care. The sophistication of the photographic techniques used may mean the difference between an image that is usable for follow-up care and one which is strictly evidentiary. Over or underexposure will cause loss of detail that may result in missing important information. Proper white balance, lighting, and flash use are necessary to avoid variation between images, in specialties such as wound care where, for example, documentation of burn color or wound traits over time may be crucial. To reduce or eliminate glare, lighting diffusion techniques may be required. Measurements of objects within a photograph require the presence of a reference object of known size or metadata recording the image acquisition geometry (such as the physical size of the field of view in the same plane as the measured object).

Imaging is used in dermatology for comparing lesions and skin conditions over time, discussion with patients, remote review (teledermatology), as well as teaching and research. They are not usually used for primary diagnosis, which is performed visually or by biopsy, except in some uses of teledermatology [26, 27].

Instruments used for recording images are digital cameras and microscopes with digital image output. Medical photography in dermatology is important for following skin conditions over time, rather than relying on subjective, written descriptions and inconsistent memories, as well as for education and publication [28]. Still though, many dermatologists do not use photography for reasons of cost and complexity [29]. To mitigate such issues, the use of low cost, reasonable quality, readily available digital cameras has been encouraged and the importance of correct photographic technique and the identification of the patient, date, and anatomic region emphasized [30]. The presence of identifying information and anatomic landmarks in the images is useful in the absence of formal systems that capture this as explicit metadata, but does not facilitate indexing for retrieval, and does present an issue when de-identification is necessary. I.e., for enterprise distribution of such images, a more sophisticated workflow that enables metadata capture with minimal burden to the user may be necessary.

Other than hand-held photography, specific dermatological devices are available for specific visible light applications, including dermatoscopes for epiluminescence microscopy of individual skin lesions [31], and systems for whole body integumentary photography [32], which can produce digital images. Other modalities than visible light are also used in dermatology, including ultrasound biomicroscopy and OCT [33].

The use of DICOM rather than consumer industry formats like JPEG offers advantages for dermatology, particularly with respect to identification and anatomical location metadata as well as interoperability with other enterprise imaging systems, and standard DICOM Visible Light Information Object Definitions (IODs) may be used for most applications [26, 27].

Legal issues related to the use of digital images in dermatology are non-trivial, but beyond the scope of this paper [34‐37].

Endoscopy is not a single discipline, but rather a family of technologies used in many specialties. Every cavity or potential space in the human body is a potential candidate for indirect visualization using a device with a light source and a means of conveying and image. Originally, the technology relied on rigid or flexible light pipes (e.g., fiber optics), but with the advent of chip-based cameras, digital acquisition occurs at the tip of the endoscope (distal sensors) rather than at the eyepiece [38].

Gastrointestinal applications included upper and lower gastrointestinal endoscopy, the latter including both colonoscopy and proctosigmoidoscopy, use in conjunction with radiology, such as endoscopic retrograde cholangiopancreatography (ERCP) and proctosigmoidoscopy. Surgical endoscopy has long been used for cystoscopy and arthroscopy, thoracoscopy and mediastinoscopy, rhinoscopy, laryngoscopy, esophagoscopy, and bronchoscopy, and more recently has become a mainstay in general surgery in the form of laparoscopy. In obstetrics and gynecology, amnioscopy and fetoscopy, colposcopy, hysteroscopy, and falloposcopy are performed. Endoscopes may even operate independently, as in the form of the wireless capsule endoscope (WCE), which is swallowed and passes through the gut passively.

The opportunity to digitally store endoscopy videos has been recognized for several decades, and specific requirements, PACS, and medical record systems have been developed for the purpose [39]. The importance of integration with enterprise systems to assure correct patient identification is recognized [40]. Departmental medical record systems may or may not support capture and archival of images and video [41]. Professional societies have issued specific guidance on imaging systems for their own domain, such as gastrointestinal endoscopy, which address such matters as video recording devices, quality, compression, storage format, and both real-time and store-and-forward tele-endoscopy [42].

A special concern for recording endoscopy video is determining the start and end of a procedure, which can be automated to avoid dependence on error prone manual control or requiring post-procedure editing [43]. WCE applications have additional challenges, given the transit time through the gut, and the need to detect and associate video segments with specific events needs to be addressed [44, 45].

Many systemic diseases, including tumors and tumor-like conditions, endocrine, metabolic, infective, inflammatory, dysplastic, granulomatous, and demyelinating diseases, have ocular manifestations with imaging findings [46, 47]. The availability and accessibility of imaging performed in ophthalmology is crucial in maintaining a holistic medical record, pertinent to other providers as part of the overall clinical evaluation, and therefore an important part of an enterprise imaging strategy.

Ophthalmic visible light applications have historically made use of film photography. With the conversion to the universal use of professional digital cameras, the availability of specific digital acquisition modalities such as for retinal fundoscopy [48], as well as a long history of tele-ophthalmology applications (such as for diabetic retinopathy screening), digital imaging is now ubiquitous. Compared to film, digital imaging raises specific concerns for quality as well as opportunities for processing [49]. In addition, various image-like applications, such as measurement of visual fields as well as measurements for refractive correction and various surgical applications, can be considered to produce a graphic representation of data in the form of an image or a document that is managed like an image. Ophthalmology-specific image management systems are available, as is EHR integration [50‐52].

Most visible light photography ophthalmic imaging device types utilize the concept of field of view (FOV) to establish accuracies in measurement. Various imaging sensor technologies, such as charge coupled device (CCD) or complementary metal–oxide–semiconductor (CMOS), are used by device vendors and provide advantages based upon acquisition light levels or noise factors. Optical acquisition physics vary across device types such as slit lamp, OCT, and visual field depending upon the specific structures of the eye being imaged.

The fluorescein angiography (FA) technique differs from ordinary retinal fundoscopy and requires the intravenous injection of a fluorescent dye, and imaging with blue light as an exciter and capture of the yellow-green light emitted, recorded as a series of monochrome frames.

OCT using near-infrared light and high-frequency ultrasound both provide cross-sectional imaging capabilities, and both types of images need to be supported in addition to conventional photographs.

Similar caveats to those applying to digital dermatology and medical photography apply to digital ophthalmology. In a deployment for Enterprise Imaging, storing the images in DICOM format (encapsulation) will be advisable [53] as is the use of DICOM work lists [54]. However, many ophthalmic imaging device vendors do not provide for raw data for analytics and progression evaluations in DICOM format and only provide a post-processed image or graphical representation of data report in DICOM format. Other vendors only offer a post processed image or report in other file formats such as a PDF or XML, requiring the receiving PACS to offer OCR report functionality or customizable XML data indexing capability.

Similar to radiology device vendors 20 years ago, when MIP/MPR reconstruction of a CT scan required vendor-specific reconstruction software or workstations, many ophthalmic vendors today require vendor-specific software to perform any form of analysis, well illustrated by OCT. The collected OCT raw data, not to be confused with RAW visible light imaging, is not only often proprietary, but there can be a variety of methods to both capture and interpret the data, though a DICOM standard has been developed and is supported by most vendors [55]. While some organizations will restrict device purchases to a specific vendor or two and manage image storage and distribution through the respective vendor’s PACS, other organizations, including those with heavy research participation or employing a best-of-breed approach, are forced to maintain multiple applications for each vendor.

Due to the proprietary nature of some ophthalmic device vendor products, challenges include requiring vendor-specific systems to provide patient or exam schedules to devices from separately distributed ADT or order message feeds. For some vendors, company size and device market cost contribute to additional challenges in support of standards like HL7 or DICOM or general support of functionality like patient or exam schedules.

Quality control processes are essential to provide reproducible, accurate, and precise results. Specifically trained ophthalmic medical photography personnel are necessary, and as with other medical specialties, certifications are available from various professional societies.

The history of digital pathology is long and convoluted [56] and fraught with intellectual property issues, some of which affect enterprise imaging [57]. Initial efforts to capture and distribute histopathology images were based on use cases involving selected images, e.g., to illustrate the pathology report [58, 59].

By contrast, whole slide imaging (WSI), in which the relevant part of the entire glass microscope slide is scanned at high resolution, is being considered as a potential replacement for use of the microscope for primary interpretation, i.e., virtual microscopy [60‐64]. Regulatory issues, uncertainty over image quality and observer performance, and practical productivity and infrastructure issues continue to be a barrier to widespread deployment [65‐72].

Slide scanners may be used for single slice scanning, but can also be deployed in automated multi-slide and batch mode systems, processing and scanning 200 slides per hour or more. The images may be stored in several different resolutions for display on a computer monitor at different magnifications. Some systems can store multiple focal planes as multilayer objects, which significantly increases the size of the images.

Standardization of the formats and protocols used for WSI and the ability to reuse existing enterprise infrastructure for storage and distribution are regarded as crucial issues, so DICOM Working Group 26 addressed storing digital slides within a PACS archive in DICOM Supplement 145 [63, 73‐75]. Despite the availability of the DICOM standard, proprietary formats, many based on TIFF, are in widespread use and interoperability between different systems from different vendors is limited [76]. There are also competing standard image formats developed that are specific to the WSI application, such as OME-TIFF [77, 78] from the Open Microscopy Environment [79]. Various commercial and open source platforms exist to convert proprietary formats into common formats exist, to enable vendor-neutral viewing and image processing [80‐83]. The use of JPEG 2000, with its inherently multi-resolution structure in the wavelet domain, seems like a natural representation for WSI, but it is not used by vendors, who prefer traditional JPEG compression of tiles, and has seen limited application in virtual microscopy viewers [84, 85] with or without use of the associated of the standard JPEG Interactive Protocol (JPIP) to retrieve selected regions for interactive viewing [86].

As a practical matter, whole slide images are so large, and the manner in which they are viewed is so specific (i.e., using a “virtual microscopy” navigation paradigm that is more like a whole earth geographical system such as Microsoft Terraserver [87] or Google Earth [88]), that dedicated systems may be required, and it may be inappropriate to burden an Enterprise Imaging Platform with this class of images. While data volumes for WSI might be tractable if used only for remote consultations, case conferences, and teaching, if all of the slide output of a busy pathology laboratory were to be digitized, the archived data volume might exceed that of all other enterprise imaging applications combined. For example, one site describes producing 380,000 slides per year (1500 slides per working day) [89], which at 0.4 GB per slide (after lossy compression, expect an average range of 200–650 MB [69]), is approximately 150 TB per year. Another site, a large tertiary referral center generates 1.4 million slides per year, which at the 0.4 GB per slide estimated file size would be 560 TB per year, compared with the same facility’s current volume of images from all existing DICOM sources of 150 TB JPEG losslessly compressed (280 TB/year uncompressed; 80 % is 2.1 million radiology studies per year, and remainder cardiology, obstetrics and gynecology, etc.). Scalability is not only a requirement for archiving and distribution, but also for processing [90].

Lossless and lossy compression may be applied at different stages in the lifecycle of an image. The image may be acquired in a compressed form, at least from the perspective of the user. That is, a digital camera or an ultrasound device may emit images that have already been lossy compressed, and this may or may not be configurable by the operator. An acquisition device may create uncompressed images but during transmission to the repository, lossless or lossy compression may be used as appropriate to reduce transmission time.

On receipt in a repository, lossless compression is often applied to uncompressed images to save storage space, and already losslessly compressed images may be recompressed with a different, more effective, scheme.

During retrieval from the repository by a viewer, if it is appropriate for the task for which the images are being viewed, lossy compression may be applied to reduce transmission time, or progressive lossy to lossless compression may be used to reduce initial response time.

Over time, as the retention period for images expires, rather than discarding them, they might be lossy compressed or lossy compressed again to a smaller size. Managing different “versions” of the same images that have been compressed in different ways is not without its challenges, especially with respect to the unique identifiers of each image and the maintenance of referential integrity [123].

There may be significant reduction in image quality if an image that is lossy compressed is compressed again in a lossy manner, especially with a different scheme. For example, it is not generally appropriate to decompress lossy JPEG images and re-compress them with JPEG 2000. By definition, losslessly compressed files may be decompressed and recompressed losslessly without any change.

Compression schemes are frequently confused with the file formats that use the compression schemes, but it is an important distinction. Many file formats are just the stored form of the bit stream produced by the encoder for a particular compression scheme. Other file formats involve the use of some sort of “header” or “container” structure that encapsulates the compressed bit stream. Some file formats are tied to a single compression scheme or variants of a single scheme. Other formats are containers for an extensible variety of compression schemes. The software or hardware that performs the compression and decompression is referred to as a “codec” (COder DECoder). Even though an extensible container format is supported by a system, there is no guarantee of being able to decompress a particular scheme if the codec used for that particular image is not supported on the particular device that is being used to view the images. This is particularly true for the plethora of proprietary video codecs that are encountered in the consumer domain.

In general, the selection of a standard compression scheme encoded in a standard container format is preferable to the use of exotic or proprietary methods. Otherwise, interoperability is compromised. This is particularly important for archives, since the hardware or software to decompress non-standard schemes may not be available in the future.

Container formats not only define the type of compression scheme used, but may also be capable of encoding metadata (including data about how the image was acquired). Multimedia container formats that are common in the consumer applications include formats such as AVI (Audio Video Interleaved) and MOV (QuickTime). For professional and scientific applications, the TIFF format is common.

DICOM is itself a multimedia container format, with patient and medically specific metadata fields, and makes use of standard compression schemes as well as allowing for the use of proprietary schemes.

Well-known standard lossless compression schemes include JPEG lossless, JPEG-LS, and JPEG 2000. JPEG 2000 will generally offer 1.5× the lossless compression ratio of JPEG for the same image but requires more computation power.

PNG is a file format with its own lossless compression scheme, which is widely used for web browser applications, and was intended to replace the proprietary GIF format.

BMP is an uncompressed file format that offers no compression and is wasteful of storage, but is commonly used in desktop applications.

The TIFF format supports a variety of compression methods, but most commonly uses some type of lossless compression in consumer applications. However, it is generally better to select a different and more widely supported format such as PNG or JPEG lossless if offered.

Common standard lossy schemes include those in the JPEG and MPEG families. If a scheme is referred to without an explicit statement that it is lossless or lossy, lossy should be assumed (especially in the case of JPEG). Video images are almost always lossy compressed to a bitrate that can be communicated over a specified channel continuously.

Whether or not the degree of compression applied is under the user’s control, and what choices should be made, depend very much on the type of image and the use case. For radiology images, there has been considerable research on the subject and various guidelines developed, though its use remains controversial [114]. Similar literature, if not professional society guidance, exists for the various specialties using visible light images, but is beyond the scope of this article to review.

For hand-held photography with consumer phones and cameras and professional cameras, various quality factors via either a numeric (0–100) scale or with varying textual descriptions may be available through configuration or within a particular image or video capture application. Be sure to consult the device’s documentation, the scientific literature and professional society recommendations, and most importantly test with local typical images to see which settings are acceptable to the users.

For still images, the image pixel matrix size (number of rows and columns) and aspect ratio (columns to rows ratio—e.g., 4:3, 5:4, 16:9) is usually independent of the compression scheme or file format. The more common standard video compression schemes may constrain the matrix size and aspect ratio and other parameters by defining profiles.

Common still frame compression schemes include the following:

Common video compression schemes include the following:

Photographic image output formats vary across camera manufacturers, models, and smart device apps. Camera RAW format [124] is not a single format, but rather a way to refer to higher quality proprietary image files created by the camera manufacturers. The manufacturer-specific RAW format provides the most information about an image. It is the data recorded by the camera’s sensor with little if any post-processing, and may be thought of as the digital equivalent of an analog photographic negative.

RAW files generally provide greater dynamic range via higher bit depth (9–16 bits per color, but typically 12–14 bits) and can be uncompressed, lossless compressed, or lossy compressed depending on the implementation.

The RAW formats encode metadata about the photo capture including exposure, picture control, and focus. Unless it is being viewed and processed in specialized software, the RAW format requires conversion to a consumer file type such as a JPEG or TIFF. This is often an automatic post-processing event, although most camera manufacturers provide software to manually process the image including white balance, saturation, hue, contrast, sharpening, and compression, and third-party software is available, such as UFRAW.

Many of the proprietary RAW formats are supported by photo and video editors, and this software is regularly updated to support new versions and camera models. Sometimes support for older formats is dropped in recent software, creating a problem for reuse of archived material. RAW formats are not backed as a standard by a standards body such as NEMA, JPEG, or used in the DICOM standards. The RAW formats are intended for advanced image editing including changing color correction, gamma, brightness contrast, and many other parameters prior to producing final output. There is no industry standard way to specify or record metadata for these processing parameters; it is assumed that a final output image in a consumer format will be produced as part of the editing process.

DNG (Digital Negative) format was developed by Adobe as an extension to the TIFF format specifically to address the storage and archiving of raw camera data and avoid the plethora of camera manufacturer-specific RAW image formats, and hence be more interoperable and archivable. It has rich photographic metadata, comparable to EXIF. However, at this time, it is not widely supported as a native format by the major camera vendors (whether SLR or “point and shoot”). This means that a conversion step from RAW to DNG format is required. The advantage of doing this conversion vs. conversion to a consumer format or DICOM is questionable. Depending on the parameters of the conversion, DNG files may be slightly smaller than RAW files. DNG files can maintain the full original RAW data, providing a means for viewing the image without losing the ability to reprocess the image in its original form. Any metadata not recognized by the DNG converter is irreversibly stripped from the original RAW file so care must be taken to examine the true RAW data before any DNG conversion process is initiated.

For final clinical distribution, educational, and publication purposes, consumer formats or DICOM usually suffice, even if the RAW or DNG files archived. Many publications require TIFF images, for example. Currently, DICOM does not have standardized support for encoding the RAW or DNG pixel data or metadata, though theoretically the Raw Data Storage SOP Class could be used for this purpose.

It is interesting to note that the Reuters news agency has banned the use of RAW format in favor of JPEG images produced directly by the camera, ostensibly in the interests of realism rather than artistic manipulation, as well as speed due to the greater transfer time of the much larger RAW images [125]. Both authenticity and speed are important for medical applications too, but need to be balanced against the appropriateness for the use case. JPEG and DICOM images can of course be manipulated too, but protection against this (e.g., with digital signatures or embedded watermarks) has not been a priority, despite there being standards and mechanisms available.

DICOM defines standard data elements to encode identifying, clinical and technique metadata, as well as the structural elements describing the pixel data itself (matrix size and number of frames, color space, bit depth, type of compression used, etc.).

Note that the discussion of the metadata is meant to be illustrative rather than exhaustive. Refer to the DICOM and IHE standards for a complete discussion.

The demographic metadata for the patient and study can be obtained from a query or manually entered, as has been discussed under the topic of Workflow and Sources of Reliable Metadata. Using a query mechanism dramatically improves the long-term maintainability and comparability of DICOM images as opposed to the use of error-prone, inconsistent, manually entered data. In enterprise imaging, the DICOM images will rarely stand on their own, but will be linked to clinical notes, observations, results, or reports in the EMR, further emphasizing the need for reliable, accurate metadata. In some cases, a “convenience copy” of a report may be kept in DICOM format in the image repository (e.g., to aid in the selection of prior comparison studies), but the image repository is generally not the system of record.

DICOM defines not only standard data elements but also standard value sets for them, many of which can be extended by the site or the users. These may be used to index and retrieve selected images based on standard or site-specific descriptions for the procedure and type of imaging study, the anatomical region (body part), laterality, etc. Usually patients, studies, series, and images may be browsed hierarchically using the more common standard data elements as well as the textual descriptions at each level, and some systems also support free text search of DICOM data elements.

DICOM defines modality and acquisition-specific IODs and corresponding SOP classes, which define the metadata and type of content for specific use cases. Some IODs and SOP classes are intended for specific medical devices (e.g., CT and MR scanners); others are intended for specific applications of visible light imaging (e.g., whole slide imaging) and another set are available for general purpose visible light still-frame and video applications. Finally, there is a family of secondary capture IODs, which are generally applicable, though lacking in visible light-specific descriptive fields, but are widely supported and easily exchanged, even with older PACS systems and viewers.

The entire set of composite image storage IODs and SOP classes share the same patient identifying and order and encounter metadata (i.e., have a common composite context), so they may all be indexed in the Enterprise Imaging Platform in the same way.

When choosing the IOD and SOP class for a particular image capture application, consider both the application-specific metadata (does it have the necessary descriptive fields) as well as the implications for interoperability, e.g., do downstream systems, both internal and external, support the selected SOP class?

Tables 1, 2, 3, and 4 describe some suggested IODs and SOP classes suitable for general purpose and some specific visible light applications in enterprise imaging [129]. Note that when using an MPEG family DICOM Transfer Syntax to encode a video object, the audio channels may be encoded in the MPEG video bit stream [130].

An additional consideration, especially for VL images, is whether or not the “payload”, e.g., the compressed still frame or video stream, needs to be accessible downstream.

Maintaining native files may be preferred for organizations utilizing standard multimedia editing software for video and still image manipulation. The extraction of native format from DICOM or XDS-SD CDA objects requires additional software.

DICOM viewers may also be limited in the range of compressed pixel data that is supported. Typical PACS workstations may not support exotic video formats, for example, and even supposedly “universal” enterprise viewers have their limits. Though the repository must support DICOM, the potential cost of additional licenses for DICOM capable devices downstream needs to be considered.

The DICOMweb (WADO-RS) services provide a simple means of gaining access to the encapsulated payload that can be returned in a consumer format media type.

Dedicated medical devices make careful choices about the encoding schemes used to record pixel data, and these are subject to decisions about suitability for the intended use as well as regulatory oversight.

When non-medical devices are used for image capture, there are often multiple image formats to choose from, and the application and the operator may or may not have control over the choice of format and/or parameters that affect image quality.

With each conversion between formats that use lossy (irreversible) compression schemes formats, there may be a further loss in image quality. DICOM provides a choice of standard compression schemes (defined in Transfer Syntaxes), which allow the already compressed data to be encapsulated, rather than transcoded. Conversion of uncompressed and lossless compressed data is always lossless by definition, therefore is harmless. Conversion may be necessary when the receiver does not support a particular compression scheme.

Table 5 summarizes some of the current capabilities of DICOM with respect to encapsulation and conversion of various consumer compressed image formats. The media type that may be requested and returned by the DICOMweb (WADO-RS) services for pixel data extracted as bulk data and separated from the DICOM metadata is listed; note that a media type transfer-syntax parameter may provide further information about the particular compression scheme used when the media type itself is ambiguous [131]. There are also less commonly used MPEG4 stereo video formats available. An exhaustive listing of the transfer syntaxes supported by DICOM is defined in DICOM PS3.5 [130].

Traditionally, DICOM has used its own dedicated protocol for sending DICOM objects over the network. This is the DICOM C-STORE operation [133]. It operates over TCP/IP like most other reliable Internet protocols and involves make a “connection” between the sender and recipient. It is most commonly employed within a LAN or via a VPN connection, but can also be used over TLS, though this is not common. Each image object is transferred one at a time, though multiple asynchronous operations can be negotiated (uncommon) and of course multiple connections can be made simultaneously. Each connection begins with a negotiation to assure that the recipient supports the type of object that is being sent and what compression options are available for the transfer. DICOM refers to connections as “associations” using the ISO OSI terminology. Each storage operation is followed by an indication from the receiver to the sender of the success or failure of the transfer (e.g., in case what is received cannot be stored for some reason). Note that using multiple parallel connections is a powerful means of overcoming network latency as a limiter of transfer speed.

More recently, in order to ease the burden for implementations of lightweight sending applications, such as on mobile devices, the DICOMweb STOW-RS (STore Over the Web) has been defined [134]. This allows the sender to use an HTTP POST operation to send one or more DICOM objects, either encoded as ordinary DICOM PS3.10 binary files, or as separate XML or JavaScript Object Notation (JSON) metadata and separate image or video pixel data in various Internet media file formats (e.g., image/jpeg or video/mpeg2).

The STOW-RS service is the basis of the IHE WIC integration profile [106]. WIC attempts to simplify the sender’s task by not only allowing DICOM objects to be sent but also allowing images or videos in a non-DICOM format to be transmitted, along with minimal DICOM metadata in XML or JSON encoding, and places the burden on the server to populate the DICOM metadata that describes the pixel data.

DICOM objects can also be sent using IHE XDS or XDS-I and related XDR and XDR-I mechanism, using the same SOAP-based transaction that is used for transmission of documents in XDS. For XDS-I, an additional requirement is the presence of a manifest of objects, which is a DICOM Key Object Selection (KOS) instance.

The FHIR ImagingStudy resource [135] provides another mechanism for storing DICOM objects, together with a selected set of metadata that can be accessed through the FHIR API (i.e., as elements of the FHIR resource).

Traditional DICOM provides three services for querying and retrieving DICOM objects:

Of the two retrieval services, C-MOVE is far more widely implemented, but does require that the sender be pre-configured with the IP address or hostname, and port, of the recipient. Hence, it is less suitable for use on a large scale than C-GET for ad hoc requests from non-preconfigured devices from which access could be controlled on the basis of user identity.

DICOMweb also provides query (QIDO-RS) and retrieval (WADO-RS) using the RESTful approach of defining studies, series, instances, and frames as resources specified in the URL of an HTTP GET operation [136]. Both QIDO-RS and WADO-RS allow for the retrieval of XML or JSON metadata, and WADO-RS allows for retrieval of the pixel data (bulk data) in various Internet media file formats (e.g., image/jpeg or video/mpeg2). A traditional C-GET-based DICOM service for retrieving metadata without bulk data is also available but it has not been widely implemented [137].

Both XDS and XDS-I support the query and retrieval of DICOM objects. In the case of XDS, the DICOM images can be indexed in the registry in the same manner as any other object. The XDS-I profile describes the use of an intermediate document, the manifest, which is actually the document stored in the registry. The DICOM instances listed in the manifest can then be retrieved from a separate source, the Imaging Document Source, which may or may not be co-located with the Document Repository or Document Registry. The transactions used to retrieve the images may involve either traditional DICOM C-MOVE operations, DICOM WADO-URI retrieval, or an IHE-specific SOAP-based transaction. As yet, support for WADO-RS has not been added directly to IHE XDS-I, but there is an MHD-I profile in development that makes use of WADO-RS and a FHIR resource-based manifest [138].

In addition, XDS-I extends the limited XDS registry metadata to include imaging-specific metadata including the type of imaging procedure, the modality, and the anatomic region.

The FHIR ImagingStudy resource [135] also provides another mechanism for querying for and retrieving DICOM objects, using the set of metadata that can be accessed through the FHIR API as elements of the FHIR resource. The retrieval of the images identified in the FHIR resource may be through the use of DICOM, WADO-RS (DICOMweb), or WADO-URI using the unique identifiers, or via URLs to WADO-RS or WADO-URI endpoints, though theoretically the images can be included in the resource by inline encoding as media attachments.

IHE XDS supports the storage of objects in any format as documents, though consideration should be given to the use of IHE Scanned Document (XDS-SD) to provide a persistent metadata wrapper.

As described earlier, non-DICOM objects can also be sent using DICOM STOW-RS and IHE WIC, by creating a very minimal set of DICOM metadata in JSON or XML form.

Conversion to DICOM has already been described.

Another standard option that may be considered is the use of the FHIR Media resource [139]. This allows for the provision of content inline (an attachment) or as identified by a URL (perhaps a WADO-RS URL). This resource is related to, but not dependent on, the FHIR ImagingStudy resource [135].

In addition to the use of healthcare-specific systems for use as enterprise image repositories, whether they be dedicated to the purpose or a component of an EMR or other information system, it is also possible to use generic content management or object storage systems for enterprise storage. In general, these make use of proprietary protocols, since there seems to be little standardization of interfaces in that industry. These systems have in common the concepts of separating the payload from the metadata, and supporting indexing and queries on user extensible metadata [140]. This begs the question of the need for a standard for that metadata in order to be interoperable.

There are a variety of proprietary protocols that can be envisioned for ingestion of images. They can be classified based on the interchange type such as follows:

However, for each of these there is a near equivalent DICOM or XDS service that could be used instead. It is recommended that when considering the use or creation of a proprietary ingestion method, that strong consideration be given to standards-based transfers and a high bar be placed for approval and implementation of a proprietary method. The content management system vendor should be contacted to ascertain when standards-based methods for ingestion will be provided.

At a minimum, the new system must also provide standards-based methods for storage, retrieval, and viewing in order to avoid a technological “lock-in” to a vendor specific proprietary strategy with no easy escape mechanism. What is particularly problematic is the situation where the images are saved in an opaque proprietary format (database blobs or data files) rather than either standard photographic image formats (e.g., JPEG, JPEG 2000, TIFF) or better yet, DICOM.

Consideration must be given to image exchange and interchange with other institutions as well as eventual migration to a new content management system, VNA or XDS infrastructure [4].

For all traditional DICOM operations, though it is possible to communicate user level authentication information when establishing the connection, this is not often done. DICOM supports various types of user authentication, including username and password, Kerberos tokens, and SAML assertions [141].

Recipients can be configured to be “promiscuous” (accept objects or requests from any source) or restricted to accept connections or requests objects from a named set of sources. DICOM uses the ISO OSI terms “Application Entity” (AE) and “Application Entity Title” (AET) to identify endpoints for communication.

When using DICOM over TLS, client and server certificates can be used for more sophisticated control over individual connections and the information used for selective access control and to record in audit trails, as defined in the IHE Audit Trail and Node Authentication (ATNA) integration profile [142].

The DICOMweb transactions can make use of normal HTTP security mechanisms, including those for confidentiality (e.g., TLS) and for user authentication.

IHE defines various profiles related to user authentication and authorization for SOAP and non-SOAP transactions for enterprise [143], cross-enterprise [144], and Internet [145] applications.

The decision to acquire and store imaging in DICOM or native format may impact image management workflows.

An Enterprise Imaging Platform may or may not implement explicit rule-based lifecycle management, to purge itself of stale content when retention periods expire. Some organizations are satisfied with keeping everything forever, especially in the face of growing rates of image acquisition volume that make the size of historical images a decreasing fraction; others are not. The policy behind such decisions is out of scope for this paper, but the need to communicate in a standard manner to other systems that images have been or are to be deleted for life cycle management purposes may be a consideration.

For DICOM images, IHE defines support for object lifecycle and correction management in the Image Object Change Management (IOCM) profile [146].

Non-DICOM images stored using IHE XDS profiles do not account for many image management needs in a real-world implementation.

Many enterprises have already experienced the challenges of migration of large-scale systems such as PACS, departmental and hospital information systems, and EMRs.

For enterprise systems, the scope of migrations to be considered needs to include the incorporation of data formerly stored only in departmental systems, the replacement of legacy systems, the strategy for integration or replacement of systems encountered during organizational mergers and acquisitions, as well as a long-term plan for replacement of new systems being acquired when they reach the end of their life or support for them is terminated. Other factors to consider for specific types of enterprise image data include the purpose for which the image was recorded (e.g., diagnostic, procedural, evidentiary) as well as any legal retention prior requirements, which may be specialty-specific. I.e., perhaps not all data needs to be migrated or made available in the replacement system with the same level of accessibility.

The importance of reliable, clean image metadata efficiently accessible in a standard format is well recognized. For radiology PACS, the migration-related benefits of both encoding images in DICOM [147] and providing access to them using the DICOM network protocol [148] are recognized. The challenges of migrating images encoded in obsolete or proprietary formats include constraints on the utility of converted data in the absence of sufficient metadata [149] and the loss of information in translation [150]. These lessons are equally applicable to enterprise imaging. The decision to use native format encoding rather than DICOM, and to store the metadata in a proprietary database, even if the latter is accessible using XDS or FHIR services, may create difficulties for future migration that need to be carefully considered.

Organizations planning an Enterprise Imaging strategy or embarking on integration of imaging specialties not currently storing to a centralized platform can benefit from pre-determining deployment based upon their workflow drivers and existing acquisition device and information system capabilities or options. Table 6 is an example of a matrix outlining specific decisions used to guide imaging strategy. While this example may not represent the drivers and system or device capabilities associated with the every organization, it provides a customizable overview. It is an example of a methodology for decision-making, not a recommendation.

In many cases, enterprise images require some editing prior to finalizing the content for the medical record. For still images, this may include cropping the image, adjusting the brightness or contrast, applying some color correction or image processing (e.g., sharpen), or obscuring a region (e.g., facial features). For video, it may be required to edit the full-length video into one or more concise clips or apply some compression.

Within clinical areas, there may be a requirement to maintain some content for a period of time, while “publishing” a defined set of content for use within the EMR. For example, it may be required to maintain a full surgical video for use by surgeons, using a specialized application, while also providing selected clips to the EMR for inclusion in the patient’s medical record.

The workflow as to whether this editing is applied at the time of capture (e.g., medical photo on a mobile device) or afterward (e.g., post surgery) may vary and should be understood.

The final presentation of the images may be created at the time of acquisition, rather than retrospectively. Given that in many cases the photos or movies are documenting a procedure or status of the patient during the visit, it is important to save the “as viewed” images, with key images flagged.

If images are acquired raw, then there are the additional steps of post processing and creating the “for presentation” processed images that are then saved for ongoing general use. The original raw images may be saved as well but should be flagged as “for processing.” Note however that outside of DICOM there is no formal linkage between “for presentation” and “for processing” images. Having two versions of the same image can lead to confusion for later users and raises some patient safety/risk concerns regarding confusion of images, and time points or references.

In some cases such as surgery or endoscopic videos, there may be a long, constant recording during the entire procedure. Some systems provide the ability to edit out clips or still images but this is a time consuming process and can entail an additional loss in image quality due to decompressing a recompressing the images. Strong consideration should be given to a workflow that is to “capture only what you intend to keep.”

For some use cases, such as teaching and research, information that may potentially identify the patient, or potentially be used in conjunction with other information to re-identify the patient, needs to be removed.

For most radiology and cardiology use cases, such information is usually confined to the DICOM metadata, though for some modalities it may be burned into the pixel data and need to be edited out, manually or automatically.

In some cases, the images themselves may be identifiable. It has been suggested that high resolution CT and MR images that include the face might be recognizable using manual or automated methods [151‐153].

The incorporation of visible light images in enterprise systems increases the risk that identifying information might be present in the metadata (even of consumer formats like JPEG), as well as present in the pixel data (e.g., when labels are present in the photograph to identify the patient).

Full-face photographs and any comparable images are explicitly mentioned in the US HIPAA Privacy Rule [154]. Manual processes and photo editing tools are necessary to ensure de-identification of identifiable patient features within a medical photograph.

A special consideration for ophthalmology images is that retinal and iris images may also be inherently identifying, since they are used in biometric identification systems [155].

The normal considerations for metadata de-identification apply equally to enterprise imaging [156, 157]. Another special case unique to visible light images that may be overlooked is the presence of geolocation information (such as GPS coordinates) that may be present in the EXIF headers of JPEG files.