Where and how much is explainable AI relevant in radiology
In 2021 an interesting article written by van Leeuwen K.G. and colleagues was published, containing an analysis of 100 commercially available AI radiological products and related scientific evidence [
29]. The paper underlines how despite the CE-marked products being analyzed, only a few had related studies on clinical impact (18/100) and, only 36/100 had peer-reviewed efficacy papers published. An aspect that needs a comment on this paper is the transparency aim of the manuscript, despite a clear xAI section is lacking. However, the Authors provide an online up-to-date tool to deepen the approved AI (
www.aiforradiology.com), where it is possible to search for a specific AI tool and related information on how it works, its trustworthiness, and its clinical efficacy. From the analysis of this example, one of the main problems concerning AI in radiology and its explainability emerges. In fact, despite the availability of much CE-marked software, having already been released and started to be used, only a few have been analyzed regarding their explainability. The xAI problem could be marginal in the case of AI software that performs individual tasks, such as segmentation or lesion detection, where radiologists have the ability to check and modify the AI output before signing a report. However, in the case of more complex tasks that combine different medical areas and yield results in terms of prognosis or therapeutic strategies, based on different AI approaches, may radiologists be able to critically interpret the output? In this contest, the black-box approach lacks trustworthy and xAI is necessary to assure radiologists, other specialists, and patients the essential tool to merge AI software in real life. Of course, this process is not easy, and it needs time to be assimilated and integrated into the clinical use of AI.
In addition, it is hard to believe that radiologists might have all the knowledge to understand xAI but, some efforts are necessary to acquire some fundamental expertise and principle of xAI to improve the transparency and explainability of AI software that has the potential of decision-making in the medical area. Moving on to a different radiological topic it is possible to make explicit this concept. For instance, not all radiologists know all the Magnetic Resonance Imaging (MRI) functioning or components even if they interpret it for a clinical purpose MRI imaging. However, radiologists that use MRI as a diagnostic tool are aware of the ghosting artifact mechanism, and that allows them not to misinterpret an aortic pulsation on the liver parenchyma as a lesion. Taking this example, it seems important for AI applications in Radiology to have the possibility to understand how outputs are generated to reduce the risk of “dogmatic medicine”, far away from the “evidence-based approach” that drives progress in science now [
5,
30].
For that reason, all the available AI tools in the Radiology field, such as segmentation, detection, lesion characterization, and prognosis, need end-users' side attention regarding the artificial neural network data processing accessibility also after the output is obtained. Of course, this extremely challenging task deserves attention and collaboration also from the other actors in the process (data scientists, developers, engineers, product companies, etc..) to improve the xAI process of merging AI tools in clinical practice, by avoiding conscious or unconscious errors that will damage the patient’s health or trust in AI. Nevertheless, pushing xAI to extreme transparency and explainability contains a very complex intrinsic limit. With increasing transparency, interpretability and explainability comes the risk of reducing the performance of these algorithms based on the true deep learning process. Therefore, once the benefits and limitations of xAI in Radiology are clear, we need to start implementing this process on a large scale of users to test the benefits of AI in clinical practice and to adapt the process itself to reality.
An interesting approach to the evaluation of the explainability of an AI system is the one called Z-Inspection; an initiative to assess trustworthy AI in practice [
31,
32]. The Z-Inspection procedure has three main phases: (1)
Set Up phase, during which necessary preconditions are clarified, the team of investigators are defined, the boundaries of the assessment are delineated, and a protocol is created; (2)
Assess phase, during this phase the use of AI system is inspected, the potential ethical, as well as technical and legal issues are identified, which are further extended to the trustworthy AI ethical values and requirements; (3)
Resolve phase, this phase engages with the raised issues in the sense of possible ethical tensions, and recommends appropriate procedures.
The adoption of the Z-Inspection process is important to settle on an AI in clinical practice, since it follows the Assessment List for Trustworthy Artificial Intelligence (ALTAI) outlined by the Ethics Guidelines for Trustworthy AI [
33].
Implications of xAI for the radiological profession
Large-scale benefits potentially derivable from AI in medical care are enormous, in terms of process optimization, personalized treatment, and technical implementations, but all these possible scenarios are far from being realized, if possible, drawbacks are not recognized and corrected properly [
5]. Being aware of these aspects and realizing the actuality of the thematic is central to preserving the rigor of the medical process as we built it up to now. In fact, AI is taking space not only in the research field but also in clinical practice as mentioned above. In this context, the xAI plays a bridging role in combining the rapid development of AI and its use in practice, in particular in the radiological profession. Thus, being conscious of xAI in the radiological profession implies some changes in the profession itself to avoid a possible catastrophic epilogue such as the one hypothesized by the AI precursor Geoffrey Hinton in 2016: “People should stop training radiologists now. It's just completely obvious that within 5 years deep learning will do better than radiologists.” Luckily, the process of AI implementation has been revealed to be more complex than expected and the role of radiologists is still fundamental; on the other hand, this profession will need implementation and modification to be part of the paradigm shift process. In fact, an important role of radiologists will be, as already happened in the past, to expand their knowledge and merge them with prior expertise. In fact, radiology since its beginning has faced up a wide multitude of technological changes and consequent adaptations that succeeded one other very rapidly, an emblematic example is represented by the X-rays phenomenon described by Roentgen to their clinical application soon after [
34]. Therefore, one of the main implications of xAI for radiologists is to keep expanding knowledge in this field to take confidence with this new topic strictly related to medicine and in particular radiology, for improving trustworthiness for them and for patients. In fact, a translational approach is more than ever required in medical disciplines to enhance the benefits of progress and minimize potential drawbacks. Within these considerations, two more aspects need to be highlighted. Firstly, how to use the time obtainable from the automation of certain processes that are currently carried out by the radiologist? Secondly: how to implement knowledge of xAI in radiological practice and during radiology training?
Radiologists’ working schedules will probably evolve in a direction prone to solving more complex cases, where uncertainties or atypical situations make the AI application less performant, or to increase multidisciplinary meetings to merge all the information derived from different AI tools. In fact, as figures are more prone to technology, the role of radiologists in terms of explainability and transparency should be central in the next few years.
In addition, it is emerging how radiologists in training suffer from the lack of adequate training regarding AI [
35]. An interesting reflection is provided by Forney et al. [
36] regarding how much knowledge is the minimum acceptable for radiologists in training to give them the necessary tools to interpret AI in terms of input and outputs produced. In fact, by doing so, new generations of radiologists will be able to critically assess AI tools and be aware of a large number of biases present in this new entity (e.g., prevalence bias, automation bias, detection bias, negative set bias, etc.). Soon, it will be desirable to assure a basic standardized comprehensive education regarding AI and xAI during the training of radiologists, to prevent a new generation of radiologists from getting lost in the path of integrating AI into their discipline, but on the contrary, to become conscious and critical users of it [
37].
The responsibility of the radiologist
Together with the great hype around the blooming of AI, the role of the radiologist is loaded with additional responsibilities concerning the various steps of the AI workflow. In fact, one of the prerequisites of training AI systems is access to a huge amount of data, in the case of imaging data as ground truth. The first concern regarding the accessibility of these medical data regards data ownership, and informed consent. In fact, it is critical to establish, according to countries’ laws, who is the final owner of this data. Community-dedicated laws are necessary to support physicians in that direction [
38]. This aspect is also very sensitive, especially since private companies might use such data to develop AI tools that soon after will generate profit [
39]. Strictly linked to data ownership there is another important aspect that radiologists need to know and consider. It regards patients’ privacy and informed consent. In fact, it is essential for privacy protection that data injected into the system are anonymized or pseudo-anonymized to avoid tracing back to individual patients. This aspect is deeply connected with the role of radiologists. Before sharing data, radiologists need to be prepared about which data are trackable or recognizable to a single person, and what is needed to be maintained as data to improve AI system efficiency: examples of data protection are the use of pseudo-anonymized information of patient’s age instead of the more conductible date of birth, or to prefer a system that avoids facial recognition obtainable with a volumetric reconstruction of head and neck [
40].
After all these aspects have been managed, another fundamental step needs the radiologists’ attention. In fact, an essential step that assures a good development of AI tools regards the clinical question and consequently the type of data that will be used for training the systems. This will help to reduce potential pitfalls that might affect the AI development and further use of AI tools event if they are built with xAI approach. To reduce biases that might impact outputs, xAI will help radiologists and AI tools developers to choose which data are useful to train the software to solve the clinical question.
Parallel to this, radiologists need also to be aware of data labeling. In fact, careful annotation of imaging data that will be used for training, validation, and testing has a central role in AI tool development. In addition, also the definition of the ground truth deserves important consideration: for some pathologies, in fact, a single radiological modality is sufficient to define the diagnosis (e.g., pulmonary CT for pneumothorax) while, some other abnormal entities need a different images modality, imaging follow-up or support from other specialties (e.g., atypical pneumonia to confirm with a second CT after medical therapy). This issue intrinsically contains the risk of weakening the AI training due to the image findings are not directly sufficient, in real life, for the final diagnosis and so, the risk of higher uncertainty for the algorithms or error is high [
39].
Another important responsibility of radiologists is transparency with both AI solution developers and patients: xAI, in fact, will support these aspects that will improve trustworthiness and will improve the use of AI in a clinical setting. With transparency, another crucial aspect needs consideration: the responsibility of the medical diagnosis. A large debate is present about the final responsibility of AI tools, but the main direction is prone to consider the radiologist that uses the AI-support as the final responsible [
11,
39]. Of course, this aspect is more acceptable with xAI than with black box, and lots of steps need to be taken to consolidate this position and ensure protection for both radiologists and patients.
All the consideration above-mentioned are important to reduce the possibility of pitfalls in the use of AI tool in radiology, even if it is xAI. Another aspect to be considered is actual limit of xAI that cannot be applied at the moment on every AI tool and that explainability is not coincident with high level-decision in every approach in medical practice. Radiologists need to be aware of these limitations to avoid potential biases in the xAI usage [
27].
Finally, the most important responsibility for radiologists, encompassing all the others, is to remain critical of the software itself, AI developers, and all the users. In fact, only with constructive critical collaboration among all the professional figures and patients, it will be possible to improve the comprehension of the benefits and limits of AI on specific tools [
41].