Design and Implementation of an Open Source Indexing Solution for a Large Set of Radiological Reports and Images

Indexing a large set of reports requires fast access to all text files to reduce the time needed to build such an index. Our radiology information system (RIS) did not comply with this rule because reports were stored in an Oracle database using a Lempel–Ziv related compression scheme.2 To counter this problem we decided to set up an independent report warehouse that did not have an impact on RIS performance.

We started out by recycling two 5-year-old servers and fitting both of them with enough hard disk capacity (160 gigabytes) and a small uninterruptible power supply. We chose to run the open-source Free Berkeley Software Distribution (FreeBSD) operation system,3 which had the important ability to maintain disk integrity after a crash or power outage. This technique, called soft updates, works by properly ordering file system metadata-writes to guarantee consistency and was introduced by Marshall Kirk McKusick.4 Both servers were configured as a high-availability cluster.

We achieve synchronization with the RIS through two parallel processes. First, a structured query language (SQL) statement interrogates the RIS to find out which reports were cancelled since the last synchronization. Subsequently, these cancelled reports are removed from the report warehouse. The second process is similar to the first; a list of new reports and reports updated with an addendum is obtained. Next, by means of another SQL statement, the report binary large objects are retrieved from the RIS database and decompressed. Alternatively, we could have been using a standardized, but more complex, method of capturing outbound Health Level 75 observation results unsolicited messages originating from RIS. At the time of implementation this method had the major disadvantage of only supporting converted flat text representations of the reports; meanwhile, our RIS gained the possibility to send out rich text format (RTF), retaining report layout and style. In an ideal situation, indexing should be built right into RIS; this way both extracting and synchronizing reports and data become obsolete. Our report warehouse only contains validated reports by policy.

With the advent of the Internet and ever-increasing computing power, indexing software became widely available. Our report warehouse made it possible to plug in and evaluate several search engines with a limited set of reports. Due to our characteristic European setting, language independence was needed to support Dutch, French, English, and German reports. The ability to include RIS data in the index as report metadata was also an important factor; this allows searching for (or sorting on) specific authors and requesting physicians and departments and other administrative information. Four more features we required were phrase and wildcard searching, Boolean logic, and the possibility to exclude common stop words. The section on semantics in this paper contains a detailed motivation for why these last four features are essential in a medical context.

Both commercial and open-source indexing engines were tested and compared, but in-depth evaluation reports have not been included here because they have become seriously outdated after more than 3 years. Commercial engines were difficult or impossible to integrate due to their closed-source nature. Most engines could be discarded quickly because of a missing feature or a time-consuming indexing process. Note that the Google Search Appliance6 was not available in Europe at that time. We stopped looking for other candidates after we evaluated the highly recommended Swish-e indexing engine,7 which had all the desired features and a very efficient indexing process.

Our evaluation showed that the filters provided by most search engines, including Swish-e, had issues with importing our reports. Most problems were related to file formats; old formats were converted incorrectly, international characters often disappeared, and automatically generated text fragments (Microsoft Word AutoText) were not being filled in. One issue was observed across file formats: hyphenated words did not always make it into the index in one piece.

Solving these problems required programming import filters for all historically and currently used file formats: both American Standard Code for Information Interchange flat text and flat text containing multiple international character sets, WordPerfect, Microsoft Word, and RTF. These in-house developed filters properly undo hyphenation and combine the imported report with corresponding RIS data into a single Extensible Mark-up Language (XML) file ready for indexing, an example of which is presented in Figure 2. At our institution there is no mandatory structure for radiological reports. Additionally, we have put a quality assurance process in place; whenever word processing software or a report template is changed, the import results are tested on international characters, automatically generated fragments, and hyphenation.

Indexing without tuning becomes very computing-intensive as the collection of document content, and thus the number of word/document combinations, grows. Fortunately, limiting the size of the index itself strongly decreases the computing power needed. In our situation, patient and study IDs in the radiological reports were safely excluded from the index without loss of information; using the metadata from the RIS yielded the same results and was more efficient.

A second technique consists in excluding some, but certainly not all, of the very frequent words present in almost all documents. These common stop words were found by building an index and evaluating the significance of the thousand most frequent words. Amongst the 39 manually excluded stop words, good examples like “a,” “the,” “of,” and closing statements like “greetings” were found. The very frequent word “no” was not excluded because its meaning was highly significant in combination with other keywords.

In the section on semantics we will show that excluding common stop words leads to semantic advantages. We will also discuss building indexes in an incremental fashion in the section on scalability.

Due to our completely digital and highly optimized reporting workflow,8 the only good place to put the search interface was right at the radiologists’ fingertips. To attain this, we created a full front-end integration with RIS and the picture archiving and communication system (PACS). By means of a generic search button, the interface can be launched from any RIS or PACS context. The integration also includes a security layer serving two distinct purposes. Firstly, our existing single-sign-on/single-sign-off mechanism allows radiologists and researchers to provide their authentication credentials using an encrypted hardware key, defying the need to type in many usernames and passwords. The second function of the security layer consists of auditing the search queries and the displayed reports.

Because Internet technology was being used, it was straightforward to make a web-based interface that resembles well-known Internet search engines. Such simple and user-friendly interfaces lower the threshold for new radiologists. As shown in Figure 3, users simply enter the desired keywords, possibly accompanied by parentheses and the binary operators “and,” “or,” and “not.” Wildcards are denoted by an asterisk, and quotes can be used for phrase searching. Between keywords or phrases, an implicit “and” is assumed. The user can quickly pick a previous query from his complete query history. The results can be sorted on a calculated page rank (default), descending and ascending report time, and RIS metadata. Useful query feedback is shown: the number of results, the query duration, and the excluded stop words. Fast reviewing of reports is essential; to meet this end, the user can choose between displaying summarized or full reports—in both cases, keywords are highlighted and the matching ranking score is displayed graphically. With a single click of the mouse, radiologists can open both images in PACS and the historical reports of the selected patient.

Alas, trying to comprehend the ranking code of Swish-e is not for the faint of heart because of its inherent complexity and the speed optimizations used. The algorithms take into account the binary structure of the search query, details of keyword occurrences, and the relative frequency of words in the index. Luckily, unlike searching the Internet, queries in a radiological context tend to be very specific (see results), making fine-tuning of ranking unnecessary.

The search engine was consulted 7,071 times by our radiologists mainly to find interesting cases for research or training purposes. Occasionally, a query is performed to find a recent clinical exam in a fast and convenient manner.

Table 1 presents the usage of search engine features. Apparently, 61% of the radiologists are aware of and use at least one of the features despite the lack of formal training; 39% of them have never used a search engine feature. Nine in ten queries consist of keywords only, which means that results are mostly found without having to resolve to more complicated queries. Users tend to start out with a simple query and quickly skim across the clearly presented and highlighted results. Binary operators turned out to be the most popular feature, especially the “and” operator, which is used in 4.6% of the queries to narrow the returned results. Phrase and wildcard searching each are used in 3% of the queries, whereas parentheses account for 0.6%.

Querying the index is very fast; the median query duration is 31 ms. The most complex query took 2.6 s: results for each keyword need to be binary combined and ranked; a processing delay is noticeable in the case of very frequent keywords or when broad wildcards are used. The users enter fairly specific queries; medially, only 36 results are returned. A statistical analysis of the query duration and the number of results can be found in Table 2. The cumulative frequency distribution of the query duration, presented in Figure 5, can give the reader an idea of the response rate of the search engine. Caching performed by the operating system has a strong influence on the query duration because 36% of the queries yield results within the 13-ms average access time of the disk drive used; Figure 6 visualizes the relevant detail of the frequency distribution.

Our indexing solution is a prime example of an open-source success story; extended use of this technology dramatically reduced both implementation time and cost. Designing and building the system took a mere 6 weeks, which was less than the “red tape” time needed to approve a commercial offering. Apart from the implementation man-hours, the total cost of ownership consists of low-budget hardware and support costs. Support calls are rare due to the high-availability design of the system and the excellent stability of the Swish-e search engine. In fact, users almost never call; the interface is very user friendly and similar to well-known interfaces of Internet search engines. Furthermore, passwords cannot be forgotten because of the single-sign-on mechanism. About four times a year, manual intervention is required when the synchronization link with the RIS breaks down; in this case, the administrator is automatically notified by e-mail. There has only been one hardware failure during the system’s lifetime, a broken power supply, but this did not cause any downtime.

All software we developed in the course of this project has been made available10 to the open-source community covered by an unrestricted BSD-style license. The source code included linking and synchronization with a RIS, five import filters, the security layer and audit logging, the complete user interface, and the integration code for both RIS and PACS.

Our decision to make use of Internet indexing technology in a medical context was straightforward because of its ubiquitous presence, the available software components, and the maturity this technology has gained over the years. Three generations of automated Internet indexing systems can be identified as to their methods of compiling their data sets, their search interfaces, and their associated etymological metaphors and mythologies.11 Firstly, the Archie search engine dating back to 1990 allowed for single keyword and regular expression searches of the file lists of anonymous File Transfer Protocol sites. Gopher, a distributed document network protocol preceding the World Wide Web,12 featured a second generation; the Veronica system allowed for Boolean queries of directory information and filenames. Thirdly, search engines that compile searchable databases of information accessible via the World Wide Web are in wide use today, such as the popular Google.13 Indexing technology slowly found its way into computer operating systems like Microsoft Corporation’s Windows Vista (released in 2007) and Apple Corps’ Mac OS X 10.4 (released in 2005).

Two aspects of Internet indexing technology could not be applied to our medical documents because of absent hyperlinking. The first was “spidering”: robots crawling the web to search for new and updated material by investigating hyperlinks. However, this did not pose a problem because the RIS kept track of new and updated reports. Secondly, page-ranking algorithms could not use rules based on the number of referrals by other documents.

A nationwide study conducted by Vorbeck et al14 in Austria, apart from the Alps, a country similar to Belgium, showed that radiologists were already familiar with the Internet in 1999. Today, all radiologists at our institution have a broadband Internet connection at home, including secure access to the electronic medical records and the PACS.

Generally, the duration of building an index should be less than the desired timeframe in which new or updated documents show up in the system. Build times can increase beyond this point when document access speeds decrease or when the total size of the document collection grows. To counter this limitation, indexes can be built incrementally; however, care must be taken because some indexers do not recalculate frequencies of very common words for automatic exclusion. Similarly, indexes can be built in parallel; both mechanisms make use of index merging. Judging by our current results and the obsolete hardware used, we estimate that our indexing system scales up by two orders of magnitude, the main bottleneck being the nondistributed index.

To scale a cross-patient search engine, hospital-wide privacy considerations have to be taken into account. The hospital’s policy on patient privacy should be implemented in strict access rules based on document metadata.

A disadvantage of keyword-based searching is the simple fact that radiologists or researchers want to find the meaningful concepts behind their keywords. To make word indexing more or less suitable for concept searching, four search engine features are necessary. Firstly, because medical terms mostly consist of more than one word, it has to be possible to search for phrases. Page-ranking algorithms can help by giving a higher score to documents with adjacent keywords, but in practice, the sole use of this technique alone is not specific enough. Secondly, very frequent words with low significance can hamper phrase searching and should not be indexed; when “of” and “the” are excluded, the phrases “MRI of the brain” and “MRI brain” yield the same results. A third helpful feature is Boolean logic to facilitate binary operations like “and,” “or,” and “not” in combination with parentheses. Finally, a word-stemming technique can be used to find all words with a common root. Although we did not implement word stemming because our multilanguage set-up required building a separate stemmed index for each language, the use of wildcards turned out to be an adequate alternative. Word stemming is part of the broader method of fuzzy indexing, which can be used to find words with similar pronunciations.

To find conceptual information, users build their queries in multiple steps; hence, the user interface should allow for fast reviewing of results. Table 3 shows a typical user refining his query. Note that structured reporting, which uses standardized information concepts, does not have the semantic problems of free-text reporting.

A lot of research regarding medical semantics has been conducted, ranging from natural language processing techniques15‐17 to building unified medical lexica18,19 for multiple languages.20 The authors hope to combine the indexing solution with some of these scientific methods and technology regarding the semantic web21 in the near future.