Multispecialty Enterprise Imaging Workgroup Consensus on Interactive Multimedia Reporting Current State and Road to the Future: HIMSS-SIIM Collaborative White Paper

A city description enriched with an annotated map of streets and neighborhoods, embedded images and videos of landmarks, tables and graphs of citizenry demographics, and corresponding hyperlinks to additional content provides a better understanding of the city than would a textual description alone. To-scale timelines of a city’s past events effectively convey the city’s history to readers. Multimedia integration with textual elements has been shown to increase the quality and quantity of knowledge transferred because different forms of information are processed and integrated by a consumer’s brain [1‐3]. Information consumers shift their attention between visual and textual elements integrated together in a single location or user interface depending on their needs at that moment. Organizations with experience studying captivating educational presentations recommend the integration of multimedia with descriptive tables, graphs, captions, and annotations in order to create engaging content [4, 5]. Similar visual communication principles guide content creation in the advertising and entertainment industries [6, 7].

In the fields of healthcare education and knowledge sharing, reputable medical research journals encourage manuscript writers to include illuminating graphics, tables, and online videos to effectively and efficiently convey results and conclusions to readers [8, 9]. Similarly, providing hands-on 3-D printed tactile anatomic models of patient abnormalities improves understanding of disease processes and ultimately surgical care over cases with exclusively textual descriptions of abnormalities [10‐12]. As an interactive online map integrates information sources to help a consumer comprehend and use complex city information, clinical interactive multimedia reporting integrates several forms of textual and imagery data to help physicians comprehend a patient’s condition. Unfortunately, most clinical still image, video, and sound creation today is accompanied by only a separate exclusively textual description, rather than with interactive multimedia and text integration.

In August 2019, a workgroup of volunteers from the Healthcare Information and Management Systems Society (HIMSS) and the Society for Imaging Informatics in Medicine (SIIM), the HIMSS-SIIM Enterprise Imaging Community, convened the interactive multimedia reporting (IMR) workgroup to address this technical and clinical opportunity. The IMR workgroup’s purpose was “to serve as a multidisciplinary forum to share interactive multimedia reporting successes and challenges, and to spur innovation in across imaging-centric medical imaging subspecialties.” In doing so, the workgroup adopted a consensus definition of IMR as “interactive medical documentation that combines clinical images, videos, sound, imaging metadata, and/or image annotations with text, typographic emphases, tables, graphs, event timelines, anatomic maps, hyperlinks, and/or educational resources to optimize communication between medical professionals, and between medical professionals and their patients.”

Medical still images, video, sound, and modality-generated multimedia may be created from [13] the following:

Textual elements and communication aids which may supplement and enrich the above multimedia may include the following:

Shortly after Roentgen’s November 8, 1895, discovery of X-rays and their medical applications, imaging findings, and clinical consultative impressions were conveyed via handwritten summaries on paper prescription forms [24]. The evolution of these original handwritten reports progressed to human transcribed and/or typewritten prose textual summaries. In the late 1990s and early 2000s, the voice-recognition prose of today grew more common [25‐27]. Radiology reporting increasingly includes structured elements based on standard lexicons in addition to the traditional prose reporting style, enabling research, interoperability, and better communication [28‐31]. Radiology interactive multimedia reports and their precursors have long been described but remain underutilized [32‐38].

Virtually all cardiac imaging can be thought of as radiology extensions dedicated to the visualization of cardiovascular structures. As with radiology, traditional reporting in cardiology has been prose, with the verbose narrative of procedures and findings remaining the predominant procedure reporting format. Nonetheless, structured reporting is slowly replacing traditional verbose reporting, including the incorporation of IMR concepts to enhance communication and more efficiently convey content [70]. Like radiology, the transformation in cardiology brings parallel opportunities and challenges as those described for radiology above.

An additional driver of structured reporting in cardiology has been the need for “good data” about clinical processes, care delivery, and patient outcomes for quality assessment and performance improvement purposes. Many of these improvements in care delivery have specifically been enabled by disease and device registries that require the submission of well-formed, high quality, semantically consistent information as discrete data [71]. Integral to the function of registries are data dictionaries that define the clinical concepts to be captured; beyond registries, multiple professional society initiatives have further extended the defined cardiovascular lexicon [72, 73]. Given the tens of thousands of clinical concepts just in cardiovascular medicine necessary for structured reporting, the natural need is for workflow tightly integrated with dataflow such that data captured at the point of care by the entire healthcare team is used to generate clinical documentation, for registry data submission, and for other secondary analyses. This structured reporting approach reduces redundancy and clinician burden while allowing all members of the healthcare team to practice at the full extent of their education, training, and experience.

Anatomic pathology, which includes both histopathology and cytopathology, is, by its very nature, a highly visual discipline with many opportunities for integrating image-based multimedia into reports. Opportunities exist in clinical pathology as well, including in the microbiology, hematology, flow cytometry, and cytogenetics labs. In many respects, pathology was an early adopter of multimedia reporting as laboratory information systems have long been able to incorporate images into reports. Interactive multimedia, however, has generally been absent from pathology reports, but the recent introduction of digital pathology into routine pathology workflows will increase opportunities to integrate multimedia into pathology reports.

Multiple IMR opportunities exist in visible light laparoscopy, arthroscopy, bronchoscopy, colonoscopy, endoscopic ultrasound, and other types of endoscopic imaging during the examination and treatment of maladies in various body cavities. For example, during knee arthroscopy, evidentiary images are commonly captured of normal or abnormal menisci, articular cartilage, cruciate ligaments, and any structures after surgical repairs have been performed. Capturing still images of a red mass behind the tympanic membrane or video of vocal fold mobility during phonation are increasingly common practice in otolaryngology. DICOM standards and IHE Endoscopy Domain profiles currently support these applications, but as in other disciplines, forward progress is slow and often heterogeneous [95‐97].

Therapeutic radiation doses are delivered in daily fractions through linear accelerators or brachytherapy remote afterloaders that derive input from treatment planning systems. Treatment planning systems import CT and MRI DICOM images for the radiation oncologist to segment (i.e., contour) anatomical structures. The radiation oncologist targets radiation at diseased tissue, such as the tumor or involved lymph nodes, while, if possible, avoiding healthy tissue to minimize treatment-related toxicities. Through a highly computational process, the treatment planning system simulates the placement and shaping of radiotherapy beams iterating variations on the consequent impact on radiation dose to structures of interest until an optimal dose distribution is obtained. The resulting treatment plan is then transferred to delivery devices. Controlling systems monitor the daily delivery of the radiation treatment as originally prescribed and planned.

Dermatologic consultation may occur live in clinic, or electronically through real-time or asynchronous teledermatology. Imaging in dermatology is used as an adjunct to documentation to represent particular skin conditions at a given moment in time. Most dermatology images are acquired with off-the-shelf phones and tablets not using the DICOM standard. Less commonly, higher-resolution compact or digital single-lens reflex (DSLR) cameras are used, also without DICOM. Some clinics invest in 2D or 3D total body photography for patients with multiple suspicious or cancerous lesions to be tracked [108]. While 3D topology images are becoming more widespread and contain quantitative calibrated information, they typically require proprietary viewers with limited interoperability, and come with an increase in cost and complexity.

Dermatology patient privacy protection and privileged access are especially important considerations in total body photography, imagery of faces, graphic or sensitive body parts, and unique tattoos. The imaging data is ideally obtained for immediate transmission utilizing a secured wired or wireless network to an EHR, PACS, or vendor neutral archive (VNA) without local device storage. Manual transmission, such as via a portable memory card, to an intermediary system continues to be common but is not recommended due to security and patient-matching concerns.

Like most medical specialties that depend on reliable point of care visible light image acquisition, storage, and analysis, dermatology suffers from local and cross-institution workflow and dataflow variation. Workflows vary for review and annotation of acquired images depending on the clinical scenario, the archive and viewing platform, and location. Forums like the HIMSS-SIIM Enterprise Imaging Community build technical and workflow consensus and raise awareness. Encounter-based photos are typically annotated within an EHR office visit document (sometimes referencing photos or lesion IDs directly). Order-based workflows are usually cumbersome in dermatologist offices due to the frequency of captured photos obtained on-demand and without antecedent EHR orders placed [109].

Reporting of encounters may occur in the EHR, in a dedicated teledermatology platform, and/or in informal messaging such as email. Salient clinical photographs may or may not be pasted alongside the clinician report summary and management. Anatomic site labeling has historically been loosely based on SNOMED CT terminology to encourage consistent anatomical descriptions [110, 111]. Similar to other specialties, dermatology is establishing a specific lexicon that can encourage more structured reporting for many aspects of image annotation, including anatomic site, histopathologic diagnosis, and clinician impressions [112].

Ophthalmology, like pathology, endoscopy, and dermatology, relies heavily on visible light images. DSLR cameras capture external imaging, such as the eyelids and conjunctiva. Specialized digital retinal cameras capture color surface and angiographic retinal images. Slit-lamp micrography captures corneal, lens, and iris imaging. Ophthalmologic optical coherence tomography (OCT) is analogous to radiology ultrasound, but it uses light waves instead of sound. OCT reveals reflective differences within the layers of the retina, creating a 4-dimensional high-resolution cross-section of the retina and optic nerve.

To illustrate the importance of IMR using an ophthalmology use case, we will use glaucoma, a common eye disease, and one using several imaging modalities. Ophthalmologists evaluate the cup, contour, and color of the optic nerve head in diagnosing and following the course of glaucoma. Like other phenotypic body parts, the optic nerve head appearance varies throughout the healthy and glaucomatous populations. Tracking the optic nerve head over time helps determine if surgical intervention is warranted or if medical management is sufficient [118]. Timelines associated with IMR allow for visualizing and analyzing data from different diagnostic modalities while providing interconnectivity between report text and images and annotations over multiple studies. Unfortunately, most ophthalmology practices cannot create even rudimentary interactive multimedia reports because of incomplete integration and interoperability between imaging modalities, image viewers, and EHRs. Currently only a few manufacturers have imaging devices across multiple imaging technologies (e.g., optical coherence tomography, fundus photography, visual fields). While a vendor’s proprietary viewer systems may support IMR-like display from their modalities, integration for other ophthalmology devices or vendors is not common. The norm being multiple vendors deployed at individual clinic sites, interoperability suffers.

The current field of ophthalmology is overburdened with a high demand of monthly clinic visits with ophthalmologic imaging required to determine if the patient requires treatment for diabetic retinopathy or age-related macular degeneration. Instead of being done in a clinic environment, a more efficient tele-ophthalmology approach includes point-of-care image acquisition at a local pharmacy or optometrist office, or even at home via imaging devices attached to mobile phones [119, 120]. Also as with teledermatology, ophthalmology image data privacy and security are critical, since any images in the glaucoma patient use case above depicting the iris or retina are typically considered PHI [121, 122].

Physical Medicine and Rehabilitation, also known as PM&R, physiatry, or rehabilitation medicine, works closely with physical therapy, occupational therapy, and preventive medicine. Together with telehealth, these disciplines help enhance and restore functional ability and quality of life to those with physical impairments or disabilities, including, but not limited to, traumatic brain and spinal cord injury, stroke, and musculoskeletal injuries. Rehabilitation medicine aims to maximize individual independence in activities of daily living and return patients to optimal physical performance.

As the HIMSS-SIIM Enterprise Imaging Community IMR Workgroup has discussed in this paper, IMR is generally poorly adopted across the different image-producing specialties. In fact, specialties vary widely in the overall adoption of imaging healthcare IT (HIT) and electronic reporting. For IMR to evolve beyond home-grown and specialized deployments requires HIT technical standard development, integration, implementation, and adoption.

In the past, HIT standards, such as DICOM and HL7, and profiles including those standards, such as IHE’s Scheduled Workflow (SWF), enabled interoperability of image transfer, scheduled patient worklists, and simple report content [128]. The decades-long penetration of these and similar efforts show the value of integrating HIT technical standards. Careful extension or profiling of existing standards or creation of new standards or profiles are similarly needed to address the challenges of advanced interactive reporting. Successful standards development involves not just product vendors and technical experts, but clinicians who can articulate the use cases to be addressed, identify product and standards gaps, and mobilize community adoption.

Although technical standards tend to be designed for a limited number of focused use cases, they are often still moderately flexible, supporting a range of uses. Some of these uses depend on capabilities that the standard declares as optional. Vendors are often reluctant to invest in implementing optional features since there is no guarantee that other systems with which their product must communicate will also implement that optional capability. Even if implemented, systems are often configured to only use baseline capabilities to ensure maximal compatibility. For example, HL7 version 2 report messages (ORUs) support unformatted text content and an HL7-defined simple formatted text content. The messaging standard also supports embedding PDF, RTF, HTML, and other kinds of documents. Even though HL7 v2 supports these advanced features, most software applications utilizing HL7 v2 do not adopt them. Even if the vendor does include the optional, advanced capabilities in a commercial product, hospitals often do not or cannot implement them when even one surrounding application (like PACS or the EHR) in the hospital IT ecosystem cannot interoperate with those features. Even with full technical interoperability between standards and commercial products, end-user physicians may simply choose not to adopt them because they have limited incentives beyond their own specialty. Thus, stitching several standards together into beneficial workflows that can be broadly deployed requires physicians and other users signaling receptiveness to the opportunity, and commercial vendors developing toward meeting those desired workflows. As healthcare systems consolidate and physician reimbursement tilts toward value-based, patient care outcomes-based plans, the HIMSS-SIIM Enterprise Imaging Community believes beneficial cross-specialty technologies like IMR in the EHR will become more appealing.

Organizations like IHE address those gaps through convening expert clinical and technical volunteers to “profile” how the basic and advanced features of multiple standards can be integrated to accomplish cross-standard, innovative use cases beneficial to patient care. IHE has recently proposed profiles for point of care ultrasound and handheld camera encounter-based imaging workflow (EBIW) [129]. With EBIW, the foundational standards already exist for IMR. DICOM Key Object Selection, and Structured Reports, can be used to communicate key images and findings (including finding location in a volume, and lesion tracking between studies). Interactive report content could be encoded in HTML, PDF, RTF, or with additional semantics in HL7 Clinical Document Architecture (XML) or HL7 FHIR DiagnosticReport or Composition [130]. Individual findings can be encoded with HL7 v2 OBX segments in ORU messages, FHIR Observation, or using DICOM Structured Reports. Images and annotations can be retrieved by EHRs, reference viewers, or patient portals via DICOMWeb [131]. Coordination between various systems can leverage HL7 Context Management Specification (CCOW), DICOM UPS (Universal Procedure Step) or the UPS-RS web-based variant, or FHIRcast [132]. The IHE RFD (Retrieve Form for Data Capture), IHE SDC (Structured Data Capture), or its FHIR-based equivalent could be used to assist with the collection of more computable report content [133‐135]. Advanced, structured encodings can be leveraged by machine learning and artificial intelligence applications, user-appropriate report rendering, report pre-population, and more efficient contribution to population-based registries.

It is believed that the foundational technical standards for IMR exist; however, we are not yet at the point where a cross-vendor standards-based IMR implementation has been implemented. Interoperability will remain limited unless all systems creating, using, and consuming content agree on terminology for procedure codes, anatomy, feature descriptors, and similar elements. In many cases the need is beyond simple terminology and into ontologies or complete data models and associated search or matching capabilities. Profiles are needed to specify how information flows between modality, PACS, report authoring tool, information systems, and the EHR. These profiles need to document how reports are encoded, expectations of report storage and display systems, how findings and key images are communicated between systems, how the activity context is identified, how users are authenticated, how users create interactive multimedia reports, how users add to existing interactive multimedia reports “owned” by other specialties, and many other aspects. This profiling needs to define how a specific standard or multiple standards can be used to accomplish IMR.

The HIMSS-SIIM Enterprise Imaging Community IMR workgroup has a technical section developing a follow-up paper outlining technical considerations in interactive multimedia report creation, distribution, and consumption. This technical workgroup is scoping a “minimum viable product” IHE Radiology IMR profile for upcoming development.

The HIMSS-SIIM Enterprise Imaging Community IMR workgroup has outlined the benefits of more visual, interactive reporting of imaging procedures and interventions that output imaging data. These benefits include easy to understand aggregations of multispecialty data, more efficient and engaging data presentation, fewer abstraction or data entry errors, structured and labeled data for further research and downstream quality assurance processes, and likely better collaboration between provider groups. We have defined and demonstrated the current state and future directions of interactive multimedia reporting in several imaging-centric medical specialties through example reports, including images, videos, rich text, tables, graphs, patient event timelines, and hyperlinks. Unfortunately, most specialties today utilize only plain textual descriptions of imaging in dedicated reports or buried in clinical documentation. Multimedia and interactive reports remain uncommon regardless of specialty. Those sites that have adopted IMR subsequently experience limited functionality after that report has transferred into the EHR.

Several critical limitations are preventing widespread adoption of IMR. Heterogeneity in clinical workflows and dataflows in image capture, storage, viewing, and reporting semantics are the first. These exist across specialties and within specialties. Particularly challenging areas include structured data creation by providers to support IMR, EHR integration of native IMR functionality, purchasable applications to aggregate such widespread metadata and multimedia, and having a consistently used imaging format as described above. Interestingly, heavily DICOM-based imaging specialties appear more apt to adopt widespread IMR than do specialties with currently low DICOM penetration. DICOM-based specialties having better capabilities for inherently rich metadata, native support for annotations and key image tagging, image viewing, and image sharing across applications and across physical locations are closer to deep IMR adoption than are those specialties where non-standard, proprietary image formats or consumer image formats without metadata persist.

Incentives may be another reason for limited IMR adoption. With healthcare reimbursement still largely fee-for-service and productivity-based, any activity impairing productivity, even minimally, is opposed. In addition, the personnel and technology needed for the necessary image capture, storage, and curation can be a cost-prohibitive upfront investment in some clinical practices and use cases—dermoscopy in dermatology and whole slide imaging in pathology, for examples—that have no opportunity to offset costs with related charge codes [136‐138]. As more healthcare providers and hospitals are financially motivated toward higher quality, higher efficiency care, IMR may become a competitive advantage for practices with interoperable systems that facilitate it, especially if the value of improved communication and patient care can be quantified. As those competitive advantages grow, more practices may grow interested in adopting the workflows, dataflows, and applications supporting IMR. A sub-workgroup emerged from this HIMSS-SIIM Enterprise Imaging Community workgroup focused on the needed IMR technical developments so that industry is prepared as clinicians become more interested in interactive multimedia reporting.