Radiologists in the loop: the roles of radiologists in the development of AI applications

In the first step, developers define the medical use case: a problem to be solved and certain improvements to be achieved in radiology work. The use case should be medically relevant and offer a viable solution that can be implemented and worked with. We observed three roles of radiologists in this step. First, as problem finders, radiologists can initiate the idea of the use case by identifying a particular problem, need, challenge, or improvement, which can be supported, expedited, or fully automated by AI solutions. Second, radiologists may be engaged after the definition of the problem, to offer their insights and feedback on the potential tool’s functionalities, features, considerations, limitations, and challenges of the idea (problem shaper). Third, radiologists sometimes become the dominant actor in making the final decisions on requirements and features (problem dominator). We observed that these three roles are often based on occasional consultations, instead of continuous, ongoing interactions. Although radiologists were involved in the definition of the use case, they were often absent in specifying the conceptual design of the solution, such as defining the operations, designing the interface, and determining how to integrate the application to the radiology workflow.

Most of the companies occasionally consulted external radiologists for sourcing and curating data (e.g., through their connections with medical institutes). Nevertheless, large companies had resources to hire radiologists to play this role. Radiologists were able to use their professional relations with PACS managers, and data warehouse managers in radiology departments, to get the data that are needed for the development of the AI solution. Sometimes, they had to decide how to collect and curate data. They also facilitated the interaction with privacy officers and ethical committees for ensuring secure and eligible handling of data.

Before training the algorithm, the acquired data should be labelled in order to set a performance standard for the algorithm for (semi-)supervised algorithms. Next to labeling, developers need to establish the ground truth, against which they can assess the accuracy and performance of their algorithm. AI vendors considered the roles of data labeler to be played by radiologists. Experienced data labelers came forward from long-lasting relations with medical institutions. Sometimes, vendors used automated techniques or used trained coders (who are not radiologists) for labeling data. These trained labelers were part of the technological team (e.g., software developers, data scientists), who were already familiar with the field of radiology and with the conceptual design of the AI application. This was mostly the case for small vendors, who had limited resources to hire radiologists for labeling. Then, they relied on radiologists as data quality controllers who checked the labeled data for accuracy and consistency and resolved the complex cases.

After training, the developers need to ensure that the algorithm is accurate and stable when it is used on broader datasets that represent the diversity and complexity of the real-world settings (validation) and can be effectively worked with under the clinical conditions (testing). During this step, developers need to critically check the algorithm and its application based on feeding multiple, heterogeneous datasets. For that, radiologists perform the role of algorithm tester by suggesting how to validate the algorithm on real-life datasets (e.g., different patient profiles and different modalities), checking the biases and sensitivity of the algorithm, examining the false-positive and false-negative cases and identifying common issues, testing the algorithm on complex cases, and specifying the boundary conditions and limitations of it. Since there is no universal standard database and procedure for validating algorithms, radiologists play a crucial role in defining the composition of validation datasets and the procedures that should be followed to ensure an accurate and mindful validation. All these enable the technical team to revise the application and algorithm in order to reach the expected performance. In addition, vendors relied on radiologists as AI researchers to design and execute scientific tests and produce scientifically sound evidence to endorse the performance of the AI application. In this role, they are also active in writing scientific publications (especially when the development is in the research phase) and crafting scientific sections of the legal documents (when the application is going to be approved, e.g., by FDA for commercial use).

We see that radiologists play several other roles than merely labeling data. In fact, companies are increasingly relying on other ways to annotate their data than only recruiting radiologists (e.g., using semi- or unsupervised algorithms, using trained employees to annotate data, using automatic tools for data labeling, crowdsourcing the labeling). Hence, companies can benefit from the scarce, valuable knowledge of radiologists as domain experts in other steps of the development. Many of these roles reappear along the way of using AI. For instance, evaluation and validation of AI tools is not a one-time event, but an ongoing process, since regular updates of the software and using new types of equipment require revalidation of the algorithm. Radiologists should define how the tools can and should be evaluated in the clinical environment.

Next to offering their expertise, radiologists can play important roles in establishing connections between the companies and medical institutions (e.g., for getting access to data) and creating trust among their fellow radiologists. Next to their medical knowledge, their technical engagement is also essential to guide the development process. In addition, they can provide guidance with the seamless integration of an application in the radiology workflow, as well as ongoing evaluation of the AI at work. This way, radiologists can contribute to the establishment of relations with the prospective clients, the introduction of the AI solutions to the clinical practice, and the definitions of viable business cases.

Looking across the cases, we see that several factors influence the active engagement of radiologists in the development process. Companies with strong medical background and connection with radiology departments engage radiologists from the beginning of the development process. Nevertheless, when companies are dominated by the technological teams and have a tech-heavy background, they tend to engage radiologists limitedly and in later steps of the development. This sometimes causes major challenges down the road². As organizations become more mature in the development of their products, the roles of radiologists become more diverse throughout the development process. In addition, having resources available for hiring radiologists and systematically engaging them in the development is a key factor. Especially companies cannot rely only on occasional consultations and expect radiologists to help them besides their full-time medical work. Next to the resources, the active engagement of radiologists is sometimes hampered due to limited expertise of radiologists (e.g., knowing how deep learning algorithms work or how data characteristics can impact the outcomes of the algorithms), combined with their skepticism regarding how AI applications can help them.

Although we examined a diverse sample of companies, future studies can expand the empirical scope and diversity of the cases by comparing how radiologists play roles across geographical contexts and different modes of production (e.g., in-house development, commercial companies, and co-creation). Future studies can dig into the skills and capabilities that radiologists (need to) learn (before and) during their engagement in the development process. We can still learn from following the various career trajectories of radiologists who pursued different ways of engaging in the development of AI (e.g., as external advisers, as in-house medical experts, or as leading actors in the development). Radiologists can play important roles in the “implementation” and “use” phases of the AI systems (e.g., as local champions and AI supervisors), which deserves further investigations. Finally, it is important to study the radiology community to see how they themselves perceive and play various roles in the development of AI applications.