Application of failure mode and effects analysis to treatment planning in scanned proton beam radiotherapy

New technologies have been introduced in radiation therapy (RT) with the aim of improving treatment outcome by means of dose distributions which conform more closely to the target volumes. Highly conformal dose distributions allow for dose escalation in the target volumes without increasing the dose to neighbouring normal tissues, or for a reduction in the dose to normal tissues without decreasing the dose to the target. In particular, actively scanned proton beams represent a novel irradiation technique taking full advantage from the physical interaction properties of these particles with tissues and advanced delivery modality to generate very sharp dose gradients in three dimensions, with many degrees of freedom available at the planning level.

The increased complexity related to the technological and process changes places new demands on quality assurance (QA) programs, as well as innovative instrumentation and detectors for beam characterization and checks [1‐5]. Moreover, new approaches of safety culture are required, since complexity may also increase the sensitivity to uncertainties and risk for accidental exposures.

Radiotherapy-related errors are unfortunately not uncommon, even in the countries with the highest level of health-care resources [6]. A number of accidents in conventional external radiotherapy have been extensively investigated and the lessons learned have been disseminated by the International Atomic Energy Agency (IAEA), as well as the International Commission on Radiological Protection (ICRP) [7, 8]. In order to fully assess and manage the risks of accidental exposures deriving from the use of innovative radiotherapy methodologies, retrospective approaches are not fully adequate, since they have the intrinsic limitation of being confined to the reported experiences, thus leaving unreported events or latent risks unaddressed. This is particularly true for new methodologies, for which safety reports may not be available. Prospective approaches, widely applied in high-risk industry, have to be implemented to find out all the elements that could go wrong and identify, a priori, all the potential hazards that might occur during a radiotherapy treatment. Recently, the interest in using these methodologies for safety assessment in complex medical practices, like modern radiotherapy, is gaining importance and the literature on this topic is rapidly increasing [9‐13].

Following the general guidelines proposed by the World Health Organization (WHO) [6], the RT treatment process can be divided into ten stages: 1) assessment of patient, 2) decision to treat, 3) treatment protocol prescription, 4) positioning and immobilization, 5) simulation, imaging and volume determination, 6) planning, 7) treatment information transfer, 8) patient set-up, 9) treatment delivery, 10) treatment verification and monitoring.

The aim of this work was the application of the Failure Mode and Effects Analysis (FMEA) prospective approach to actively scanned proton beam radiotherapy, representing the most advanced irradiation modality using this type of particle. The specific processes implemented at CNAO Foundation (Centro Nazionale di Adroterapia Oncologica) was considered for the analysis. The study was focused on the treatment planning stage, since it was considered one of the most critical phases within the whole RT process, as also reported in the WHO Technical Manual [6]. The applied procedure included the definition of the involved sub-processes and fault trees, the assignment of a score for each potential failure mode and finally the suggestion of additional safety measures for process improvement. Risk analysis for the remaining nine stages, as well as for the commissioning of the treatment machine and treatment planning system (TPS), is out of the scope of this work and deserves being addressed elsewhere.

The process tree of the treatment planning stage is shown in Figure 1. Thirty-four sub-processes were identified, starting from the selection of the reference CT scan for planning, up to the transfer of the approved treatment plan to the Oncology Information System (OIS). Twenty-two of these sub-processes (65% of cases) were judged to be potentially prone to one or more failure modes. The O, S and D indexes for each failure mode were assigned by taking into account the current specific functionalities of Syngo RT TPS as well as the following ten main strategies applied in the clinical practice at CNAO for risk mitigation: (a) definition of detailed site-specific planning protocols and check lists, (b) refusal of enrolling patients with metallic implants disabling accurate delineation of volumes of interest and/or creating unacceptable artefacts in planning CT data, (c) independent double-checking of the main planning parameters used for each individual patient, (d) treatment plan review by the radiation oncology and the medical physics staff before plan approval, (e) skill-based qualitative evaluation of plan robustness, (f) patient treatment position reaching in automatic mode, (g) daily patient set-up verification in the treatment room, (h) isocenter marking on patient’s thermoplastic mask and visual check using lasers (CT-simulation and treatment rooms), (i) regular and successful performing of Department QA checks, (j) clinical utilization of TPS restricted to qualified planners (dosimetrists and medical physicists) who have been trained on its use and limitations.

Following the independent assignment of O, S, and D values for the identified failure modes by each member of the working group, global consensus was easily reached, although a full-day plenary session was needed. Each index was discussed in detail until definitive score achievement, starting from the average of the individual values previously assigned. More than trying to establish universally reliable and objective assignments of the indexes, the attention of the members was paid to reach a proper evaluation consistency within all the failure modes. As already pointed out by the Task Group 100 of the AAPM [14], the RPN consensus values here reported should not be regarded as directly applicable to other centres, unless carefully reviewed by taking into account local specificities.

Globally, forty-four failure modes were identified. In twenty-one cases (48%), quite a low RPN value (range: 18–75) was estimated, so these failures were considered of little concern and reported in Table 1 in a condensed form: they included, for example, the lack of removal of metallic markers from body contour and incorrect localization of the origin of coordinates identified by lasers. Among the potential causes of failure for such minor events, we respectively recognized human error, lack of communication and inadequate operator skill, while their effects consisted of wrong dose distribution (i.e. deviating from the planned one) delivered to the patient, unintended normal tissue irradiation, target geographical missing or underdose, low plan robustness and sub-optimal plan quality.

Twenty-three failures modes (52% of cases) were characterized by an RPN score equal to 80 or higher, as shown in Table 2. The mean values of O, S and D parameters were 3.6, 6.9 and 4.7, respectively, suggesting that such events, on average, are infrequent (once or few times a year) and not too difficult to detect, but potentially severe in terms of patient safety. The threshold of 125 for RPN was exceeded in five cases only. The highest RPN values, equal to 192 and 196, were associated to an outdated representation of patient anatomy and incorrect proton stopping power assignment to body regions, due to artefacts, altered structures, or metal implants within the planning CT scan, respectively. Both failure modes appeared moderately infrequent, potentially severe and difficult to detect.

Three cases (failures n. 7, 8 and 11) could lead to very severe injuries or even patient death under worst conditions (delivered biological dose approximately deviating by more than 15% or 10 GyE from the total prescribed one) and consisted of lack or wrong localization of couch origin of coordinates by a large amount (more than 3 mm, corresponding to the typical organ-at-risk safety margin) and wrong dose fractionation scheme setting. On the other side, the estimated overall RPN for those failure modes did not exceeded the value of 90, since they appeared easily detectable by means of plan review, in-room patient set-up verification procedures and pre-treatment patient-specific dosimetric QA checks.

The most critical sub-process within the treatment planning stage appeared related to the delineation and correction of artefacts in planning CT data (n. IV in Table 2): the high RPN values estimated for the three failures associated to that sub-process reflect the very strong sensitivity of particle beams to range uncertainties, unlike in conventional photon RT [20‐22]. Potential causes of these failures are represented by human error made by the planner and lack of exhaustive clinical documentation about the previous surgical procedure. Two additional strategies were suggested by the working group for risk mitigation. The first one simply consisted of systematically asking preliminary information to each patient and collecting documentation about any known implanted prosthesis. Secondly, to investigate the utilization of dedicated image reconstruction algorithms on CT/MRI scanners or megavoltage CT imaging to decrease the effects of artefacts and allow more accurate delineation, associated to appropriate material over-writing, of the artefacts themselves and metal implants.

As a further priority, additional safety measures were also investigated to mitigate the risk for the remaining three failure modes (n. 2, 13 and 16) characterized by an RPN higher than 125. Concerning the outdated representation of patient anatomy, due to changes occurring in the period between CT scan and treatment start, the proposed solution simply consisted of enforcing a short interval time (i.e. not more than 7–10 days) between the two phases and, in doubtful cases, plan recalculation on a new CT scan acquired just a few days before the first treatment session. This latter strategy could at least half the D index and consequently the RPN itself.

For the failure mode consisting of improper selection of beam direction, leading to an otherwise avoidable situation in which the spread-out-Bragg-peak (SOBP) stops against one or more organs at risk, the only additional strategy appearing able to significantly reduce the risk consisted of quantitative evaluation of treatment plan robustness, that is the determination of the degree of sensitivity of the plan to the uncertainties involved in the treatment process. They mainly include range, patient set-up, dose calculation and delivery deviations. Nonetheless, that solution did not seem easily and immediately applicable: although several methods (such as minimax, worst case or multi-criteria optimization, probabilistic treatment planning) have been recently proposed for handling the uncertainties in proton therapy [21‐26], the inclusion of robustness in the plan optimization process does not yet represent the state-of-art of commercial TPSs for protons. Therefore, additional human and technological resources have probably to be provided to implement those techniques in the clinical practice. In parallel, the role of Monte Carlo simulations is strongly increasing as a support to TPS analytical dose calculation engines [23, 27]: Monte Carlo approach seems to represent a flexible modality to analyse plan robustness, by simulating several combinations of uncertainties. In this case too, efforts and time are required to achieve the needed expertise, but at least more complex plans will benefit of it, in terms of reduced sensitivity to the uncertainties, including those due to biological effects (i.e. the increase in the RBE at the distal part of the spread-out-Bragg-peak, while TPSs typically assume a fixed RBE value equal to 1.1) [23].

Finally, the failure mode related to the improper selection of physical beam model or dose calculation grid was recognized as the result of a human error, on its turn due to time pressure or inadequate skill of the planner. The implementation of independent methods for dose calculation, such as again Monte Carlo simulations [23, 27], as well as procedures aiming at plan accuracy verification under realistic conditions, as recently proposed by Albertini et al. [20], in our opinion could represent additional strategies to increase both the probability of failure detection and the levels of attention and awareness of the planners.

The application of FMEA to the treatment planning stage in scanned proton beam RT lead to the identification and deep investigation of several failure modes; the assignment of a score assessing the potential risk for each event allowed to rank these failure modes in order of importance and define priorities for risk mitigation with the aim to optimize quality management workflow. In addition to the safety strategies already adopted in the clinical practice and reported in this work, novel solutions have been proposed to increase patient safety. The multi-disciplinary and multi-institutional approach followed in this study appeared quite useful as a mutual experience exchange in a relatively new and complex field, such as actively scanned particle beam RT.

This study was carried out considering the specific processes implemented at CNAO, therefore, the detailed definition of failure modes and the assignment of RPN scores, strongly depend on the specific process under investigation and on the current strategies/solutions locally applied. However, the process and fault trees here delineated can be easily adapted by other users to their local scenario or, at least, be useful as a starting reference point, thus minimizing the workload impact of the FMEA analysis on the involved team.