CustusX: an open-source research platform for image-guided therapy

As an IGT platform, CustusX should be focused toward research and clinical trials in the operating room (OR). The purpose of the system should be to aid the users during the clinical procedure, from planning through intraoperative guidance to postoperative control. Components used during therapy, such as visualization and navigation, should be made with the intention of being simple, intuitive and for the most part automated. Pre- and postoperative features may be limited, and such features can instead be covered by good interoperability with other image analysis tools. The system should be robust, i.e., be up and running during the entire clinical procedure, and it must deliver functionality within the limited time frame available in the OR.

From the developer point of view, the platform must be easy to extend while retaining a stable core. Unstable extensions must not impede the stability of the rest of the system. The platform should support specific clinical procedures and present a customized user interface when used in the OR.

CustusX uses a main window user interface based on the Qt QMainWindow class, which provides menus, toolbars, configurable widgets, and a central view area, as shown in Fig. 1. During navigation, the view area is where the clinician interacts with the patient using surgical tools, pointers and US probes. The main window is highly customizable, and a specific setup can be stored for later use.

A clinical procedure (e.g., neurosurgery, endovascular interventions, laparoscopy) is modeled as a workflow, consisting of several discrete steps. Each step involves its own operations and visualization.

The user moves between these steps during the procedure, and each one has an associated custom user interface that again can be personalized by storing the main window setup. As the customization of the graphical user interface (GUI) can be done beforehand, this minimizes the need for setup in the OR.

CustusX uses a layered architecture, as shown in Fig. 2. The external libraries contribute to much of the functionality, with CustusX being an integrating platform. The resource layer provides what cannot be found externally, in addition to defining service interfaces. Plugins contain the majority of the code and provide discrete chunks of features on a high level of abstraction.

As mentioned earlier, in CustusX the clinical procedure is modeled as a workflow dividable into a sequence of steps, as shown in Fig. 3. The following sections describe features that are related to each of the workflow steps. As CustusX is dedicated to IGT, this is also the most important features of the system.

The planning of CustusX development is done in cooperation between clinical researchers, technological researchers, and developers, who cooperate closely, both by discussions, demonstrations, and by developers attending clinical procedures. Biannual planning sessions attended by all three parties determine the major features to implement. The work is then done in iterations allowing for corrections along the way, thus following an agile process [21]. The developers use techniques such as pair programming, daily standup, unit testing, and continuous refactoring to increase quality. Some of these techniques are only available because the core team is situated in the same location. With the recent conversion to open source, this will be augmented with a distributed process using github¹¹ as a basis.

In order to get feedback from the tests as quickly as possible, a continuous integration methodology is used [22]. This requires that the build system is fully automated and that a suite of tests are in place. Every commit made to the main git branch is picked up by a Jenkins server [23], then built and tested on all target platforms (currently Windows 8, Ubuntu 14.04, and MacOSX 10.8). Because of the sensitivity with regard to choice of hardware such as GPU, the tests are performed on physical machines similar to those used in the OR. Figure 6 shows the test flow.

The Python superbuild framework is used to perform these operations. Catch¹² is used to run unit tests, all of which are run as a single process in order to maximize speed. CTest is used to run integration tests, as they usually require each test to be run as a stand-alone process. Smoke tests involve starting all executables to validate the deployment.

A dashboard gives feedback on which tests that failed or passed. In addition, we have created extreme feedback devices [23] in the shape of red or green gummy bear lamps reporting the status to the developers in a very visual way.

The CustusX project operates with three types of releases: alpha, beta, and full. The difference is mainly the level of manual testing involved. Automated tests give a basic confidence, while structured manual tests give more complete coverage on the system level.

All releases are one-click operations. The alpha releases can be retrieved from the build server, while beta and full releases are created using a Python script that attaches the correct version tag, runs the Jenkins tests, and moves the installer to a release server.

The overall clinical accuracy in IGT is often referred to as the navigation system accuracy (NSA) and is defined as the difference between the location of a surgical tool relative to some anatomical structure as displayed in the image information and the true physical location relative to the same structure within the patient. This accuracy is difficult to assess in a clinical setting due to the lack of fixed and well-defined landmarks inside the patient that can be accurately reached with a pointer. Common practice is therefore to estimate the system’s overall accuracy in a controlled laboratory setting using accurately built phantoms. In order to conclude on the potential clinical accuracy, the differences between the clinical and the laboratory settings must be carefully examined.

A good system response, i.e., high frame rate and low latency, is important for the operator. The frame rate is the rate of screen updates, while the latency is the total delay from an event occurs in the physical world to it is displayed on the screen.

Accurate delineation of tumor borders throughout brain tumor resection (Fig. 11) and identification of feeding arteries in arteriovenous malformations (AVM) are examples of challenges where navigation technology and intraoperative ultrasound imaging can provide valuable information to the surgeon. In these and other similar procedures, we have used CustusX together with an optical tracking system and a US scanner to perform feasibility studies and evaluations of new technology. For most neurosurgical applications, we import preoperative MR images of the patient and perform an image-to-patient registration using fiducial markers or anatomical landmarks. During surgery, multiple freehand 3D US volumes are typically acquired for initial delineation of the target and to monitor the progress of surgery (Fig. 5). CustusX has been used to investigate the ability of 3D US to identify cerebral vasculature and vascular pathologies such as aneurysms and AVMs. Vascular structures seen in preoperative images, and intraoperative images can also be used to correct for brain-shift during the procedure [15]. For transsphenoidal surgery of pituitary tumors, the US imaging has been performed with prototype probes designed for the purpose of intrasellar imaging [32]. An example of 3D intrasellar ultrasound imaging is shown in Fig. 12. Other applications include data collection for more basic US research such as Doppler angle-independent 3D blood flow and velocity methods [33]. CustusX has proven to be a versatile platform for IGT research in neurosurgery, facilitating support for new ultrasound imaging techniques, prototype ultrasound probes, and development and evaluation of methods for brain-shift correction.

The laparoscopic resection of abdominal soft tissue tumors from, e.g., the liver or kidney are challenging surgical procedures. The aim of the procedures is to completely resect one or several lesions with a security margin and at the same time resect as little healthy parenchyma as possible. IGT and laparoscopic ultrasound can help localize the exact intraoperative location of the tumor, its relative position to important vessels and the boundaries of vascular regions making laparoscopic surgery more precise and safer [34, 35]. One of the challenges with navigation technology in laparoscopic surgery is the movements of soft tissue organs, which have to be accounted for in IGT [36].

We have investigated how CustusX can resolve these issues by providing navigation and US support. Preoperative images and extracted models of, e.g., the liver surface, tumors, and vessels have been imported. Navigated US has been made possible by mounting an electromagnetic position sensor close to the tip of a 4-way flexible laparoscopic US probe (Vermon, France). This has been used for both 2D real-time US and 3D US volumes reconstructed from these. Slicing in arbitrary directions and image fusion are used for visualization.

The system has been used with success in a number of laparoscopic procedures [37], where US gives exact localization information, while the preoperative data give a valuable overview with a lower accuracy. The main advantages of using a navigation system like CustusX compared to conventional laparoscopic US are the faster localization of the tumors, the better interpretation of the US images, and thus also in deciding the resection borders [34].

During endovascular procedures, the vascular tree is accessed through percutaneous punction in the extremities or via small incisions in, e.g., the groin, and catheters and small flexible tools are maneuvered inside the blood vessels. For the interventional team to perform these procedures safely, accurate real-time imaging of the vascular anatomy is needed. Traditionally, this has been provided by X-ray fluoroscopy. This results in ionizing radiation to the patient and staff and also requires use of nephrotoxic contrast agents which can be a health hazard in patients with comorbidities such as diabetes mellitus and chronic renal disease [38]. 2D image modalities like X-ray fluoroscopy also make navigation in a 3D vascular structure a challenging task requiring highly competent operators.

We have investigated methods using CustusX as an image guidance platform during endovascular procedures. Preoperative CT and intraoperative cone beam CT were used as image input. Electromagnetic position sensors were integrated in catheter and guidewires and provided real-time position information for the tools. With this setup, we have been able to accurately and safely navigate the vascular anatomy of animal models [39] as shown in Fig. 13. Using a navigation platform like CustusX can potentially reduce the radiation dose and amount of contrast used during an endovascular procedure like endovascular aneurysm repair (EVAR) [40]. CustusX provides real-time navigation in an updated 3D anatomical model which also can facilitate complex endovascular procedures, like fenestrated or branched stent grafts for aortic aneurysms making them safer and easier to perform.

A challenge with bronchoscopes is the external diameter of 5–6 mm preventing advancement beyond the third-fourth division of the airways, in addition to the multiple bifurcations, making it difficult to localize the tumor. Even with fluoroscopy guidance, the biopsy diagnostic success rate for bronchoscopically non-visible tumors is as low as 15 % compared to 80 % for visible tumors [41]. Tracking of the position of the bronchoscope and associated tools increases the success rate up to 80 % for peripheral targets [41]. In addition to an increase in the efficiency in lung cancer staging, tracking of tools also provides new possibilities for procedure documentation, e.g., of the biopsy positions.

CustusX has been used as a guiding system during guided bronchoscopic examinations. The airways and their corresponding centerlines are automatically segmented from the CT images using a method integrated in the navigation platform [13]. Tumors and lymph nodes, which are the targets for the staging procedures, are visualized in the navigation scene.

In endobronchial US transbronchial needle aspiration (EBUS-TBNA), a linear US probe is positioned on the tip of the bronchoscope, allowing visualization of anatomical structures outside of the tracheobronchial wall. Using EBUS-TBNA in combination with CustusX allows fusion of US and CT images, which is likely to improve the efficiency and accuracy in localizing the designated target for needle aspiration. Figure 14 shows the CustusX navigation scene during EBUS-TBNA.

In lung navigation with CustusX, an automatic image-to-patient registration is used, which matches the trajectory of the bronchoscope tip with the centerline of the airways extracted from the CT [42]. The algorithm, a modified version of an ICP algorithm, utilizes in addition to the positions the orientation of the bronchoscope and the running direction of the CT centerline in the alignment process. Positions of the bronchoscope tip for the registration procedure can, for instance, be acquired during the typical anesthetic procedure at the initial part of the examination, thus not adding any extra time or work load compared to a regular bronchoscopy.

Several factors are important in order to use a research system in the OR, where accuracy is one of them. Either the operator must have an accuracy below what is needed in the procedure at hand, or the data must clearly be marked as inaccurate. We have shown that the overall NSA is \(<\)1.1 mm for ultrasound-based navigation, which is acceptable for the procedures we have studied.

Furthermore, the system must be responsive, i.e., frame rates about \(>\)20 fps and latency about \(<\)200 ms for navigation and video. While the typical frame rate for CustusX of 20 fps is acceptable, the video latency of up to 285 ms will be noticeable for fast movements. However, this has proved to be sufficient during normal movement.

Software stability and quality are also important issues in the OR. Systems that seem to work well in controlled conditions of the laboratory are not necessarily stable enough to be used in the OR, where rapidly changing requirements due to the procedural flow can cause software to fail in unexpected ways. A good quality process is therefore crucial. Over the last years, we have spent a considerable amount of time and effort on the quality system with the result that CustusX can now be used for full day procedures without problems. It must be remembered, however, that there will always be a fundamental difference between research and commercial systems: A research system should be in front of the development and bring new features into the clinic faster than commercial systems which concentrate on stability and ease of use.

The architecture of CustusX has been changed several times over the years. The latest refactoring into an OSGi-based plugin framework was an important and successful milestone in the CustusX development. This refactoring was a major task, but has greatly simplified the architecture and module dependencies. Startup and shutdown of the system are now trivial because of the temporal nature of the services. Extensions of the system through plugins have also become simple because of the standardized framework and the similarity to other systems that also use CTK [1, 3] . As much of the new code are written in plugins, the overall complexity of the system increases more slowly than before and it is easier to isolate code of varying quality or with a large degree of specialization. The architecture is also suited for toolkit-like usage. Several customized applications are currently being developed using CustusX as a basis.

The CustusX development process revolves around the needs of the clinic and clinical studies, and we try to avoid the drive to implement new technology per se. This process is enforced by maintaining a tight cooperation between software developers, technological and clinical researchers through informal discussions, weekly meetings, frequent attendance of technological researchers in the OR, and joint attendance at national and international conferences.

The integration of IGT with the use of intraoperative US requires the introduction of high-tech equipment in the OR, changes and adaption of clinical practices, and a good understanding of US image interpretation [35]. We therefore organize clinical courses to spread technology and clinical methods.

There are several other platforms that operate in the same field as CustusX. The two major ones are 3D Slicer and MITK. 3D Slicer is application centered, but extensible via plugins, while MITK can be extended by plugins and embedded in new applications. Both are generic platforms for medical image analysis research.

For comparison with CustusX, 3D Slicer can be combined with the PLUS framework to create an IGT platform. MITK-US gives MITK many of the same features that are found in CustusX and is an example of how a generic system can be converted to IGT. The advantages of both of these systems are that they draw upon the resources of platforms with large feature sets and a well-defined quality process. CustusX is dedicated to IGT, which means that it is ready for use in the OR, the user interface and features are tuned toward intraoperative use, and it has a complete feature set for intraoperative US. Each of these systems have their strengths and some of the functionality is overlapping. The implementation is also shared to a large extent because they are all built upon the same open-source libraries. As part of the CustusX development, we have started to integrate with the PLUS framework through OpenIGTLink as an optional tracking implementation, and we cooperate with both through the CTK library.

We believe that the best way to cooperate is through common libraries and interfaces. This way every platform can develop freely while sharing the majority of the code. A multitude of applications and platforms also strengthen the common libraries they are built upon through different usage patterns.

CustusX has been developed and used for more than 15 years, but only recently published as an open source. The publication of the source code comes as a result of a shift in the software vision in our institution. Earlier, code was seen as property and a tool used to produce knowledge. Now, source code is seen as part of the knowledge.

There are several important points supporting open-source software: Code is an integral part of the produced knowledge; thus, the credibility of scientific results and published papers is strengthened. It also contributes to reproducible research results as others can perform experiments using the same tools. Better access to necessary tools can also trigger more research in the field. Public code will be viewed by more people, displaying strengths and weaknesses. Ultimately, this will lead to better code. Most of our research projects are funded by public sources; thus, the results should be available to the public. Lastly, the ultimate goal of doing research is to contribute to the global body of knowledge.

By publishing our code, we declare ourselves as part of the open-source community: All knowledge should be open, including papers, software, and data. Our intent is to continue this path by also publishing parts of our extensive databases of clinical image data accumulated over the years. While the open-source strategy applies to our own work, software partially owned by other companies may still be kept proprietary.

CustusX will be continuously developed and improved in the years to come to meet new research requirements and adapt to new technologies such as new tracking devices, new ultrasound scanners, 3D and wireless US probes, and flexible tools. With the current platform in place, CustusX will be used for different purposes including development of new applications customized to specific clinical procedures, prototyping of new modules for validation in the OR then possibly being integrated into commercial systems, and in basic research where new ideas need an implementation in order to be evaluated.