Developing an e-learning course on the use of PRO measures in oncological practice: health care professionals’ preferences for learning content and methods

Based on the results of the scoping review, a guideline for semi-structured interviews was developed (see supplementary material). In addition to open-ended questions, participants were asked to rate the relevance of predefined topics (i.e. concept/definition of PRO, available PRO measurement tools, choice of PRO measurement tools, timing and frequency of PRO assessments, choice of PRO measurement tools, interpretation of PRO scores, predefined examples of how to react to PRO data in clinical use and implementation issues) to be included in the course. Rating was performed on a Likert scale, from 1 (not at all relevant) to 4 (very much relevant).

Between December 2019 and May 2020, 73 semi-structured interviews were conducted with participants from Austria (n = 20), The Netherlands (n = 21), Germany (n = 19), United Kingdom (UK; n = 7) and France, Belgium, Norway, Denmark, Israel and Malaysia (n = 1 each), including physicians (n = 27), nurses (n = 21), psycho-oncologists (n = 10), patient representatives (n = 7), IT specialists (n = 4) and other professions (n = 4). Four participants (6%) were < 30 years of age, 39 participants (53%) were between 30 and 50 years of age, and 30 (41%) were older than 50 years. Overall, 16.4% (12/73) of all participants indicated not having any kind of prior experience with PROs.

The international anonymous online survey was open between June 3 and July 19, 2020. The survey was completed by 233 HCPs from 33 countries, mainly aged between 30 and 50 years (62.2%) and 52% female. The sample represents the intended target group of HCPs, of whom 28.3% (66/233) indicated not having prior experience with PRO assessments in clinical practice (see Table 1).

Qualitative analysis of the data assessed in the semi-structured interviews resulted in four main categories (A–D in Supplementary material Table 2) and ten subcategories (1–10 in Supplementary material file 2). A summary of categories and subcategories with representative quotes is presented in Supplementary material file 2. In the interviews, the embedded relevance ratings of the predefined themes showed highest ratings for basic knowledge on PROs (concept and definition) and self-management advice (95%), how to communicate (89%) and interpretation of PRO data and timing and frequency of assessments (88%). The categories were merged into content clusters representing eight topic areas that were subsequently presented in the online survey. The analysis of the online survey confirmed the results of the interviews with respect to the suggested topics and methods of presentation (see additional quotes in Supplementary material file 2). The relevance of all 8 categories listed was rated > 90%, with interpretation of PRO data (226/233, 97.0%), clinical benefits of assessing PRO data (222/233, 95.3%), implementation of routine PRO data assessment (221/233, 94.8%) and the use of PROMs and PRO data in clinical practice (220/233, 94.4%) receiving the highest relevance ratings (see Table 2). These were followed closely by basic information on PROs in clinical routine (219/233, 94%).

Qualitative and quantitative findings of both methods are presented in a joint display (see Fig. 1). Table 2 provides a detailed summary on participants’ ratings of relevance of suggested topics for the content and the methods of the e-learning course from both the semi-structured interviews and the online survey. On these findings, we based the final selection of topics for the learning content and structure and didactics of the e-learning programme, including methods of presentation (see Table 3).

With respect to topics that need to be addressed in the e-learning course, HCPs indicated the highest relevance for the interpretation of PRO data, clinical benefits of assessing PRO data, implementation of routine PRO data assessment and the use of PROMs and PRO data in clinical practice. The wish for more standardisation of procedures and increasing the sense of ownership of and the involvement in the process setup was expressed. These results are in line with recent findings on HCPs’ perspectives on implementing PROs in routine clinical care in oncology [22‐24]. HCPs included in our study recognise the potential to promote earlier intervention and more holistic approaches to oncology care but were concerned about cases when they might feel unable to interpret and adequately respond to the issues identified (e.g. if feasible solutions were unavailable). The need to educate providers about the benefits and value of PROs beyond current clinical approaches and to develop coherent implementation strategies also became apparent [22, 23]. Further implementation aspects raised by participants of our study correspond to key issues reported by Snyder, e.g. goals of assessments, selection of questionnaires, timing and mode of assessments, interpretation of the scores and clinical consequences [24].

The general question of how to motivate other colleagues and decision-makers to opt for routine PRO assessment emerged from most interviews and does generate different answers depending on the respective HCP target group. While a possible survival benefit was reported to be especially motivating for oncologists, standardised care pathways were indicated to support the engagement of nursing professionals. Critical comments emphasised the necessity to address the fundamental question of how PROs, and thus the patient perspective, can be integrated into clinical care in a way that is “actionable” for routine clinical settings. Attitudes towards PROs are often tainted by the fear of losing the human connection with the patient as the centre of physician–patient-interaction, often referred to as empathy [25].

Both the literature and our study show that the following three premises have still not reached the collective knowledge of HCPs, at least in this sample: (1) the patient’s perspective is valid, (2) the patient is an important source of subjective symptom information that is otherwise not available, and (3) empathy is an essential aspect in the encounter between people (and therefore also in the clinical encounter). However, a structured measurement of symptom burden will still supplement symptom assessment in the medical interview and allow monitoring over time. This aligns with Stephen [12], stating that when approaching the development of (nursing) curricula, person-centred care should be embedded within several courses and incorporated across the curriculum. According to the latter, areas of interest should include knowledge on how to facilitate PRO assessments, how to implement care based on the knowledge gained through PRO data, how to evaluate care based on the feedback given according to PRO data as well as competence in administering PRO instruments, reviewing the following assessments and care planning. Likewise, this should include a balanced presentation of organisational requirements for the adaptation of an effective PRO-based assistance and benefits of the latter for patient outcomes.

Considering a digital learning format, HCPs expressed the importance of examples of integration of PROs into the clinical workflow, clinical actions based on PRO data, successful implementations in different clinical settings and self-management recommendations as well as case vignettes about patients with different diagnoses. Integration of examples of successful best practice was suggested but also those of unsuccessful implementations to increase awareness of potential problems and pitfalls. Interactive components such as video examples of simulated consultations with oncologists using PRO data have been previously reported as desired learning formats [11, 13, 18].

The preferences for a modular approach and tailoring course levels to learning needs were also stated in the context of other digital learning tools where HCPs expressed the importance of a simple and visually attractive e-learning environment with a combination of (evidence-based) theory, modelling videos and illustrations [18]. Different HCP target groups with differing clinical obligations, service settings and associated fields of interest might require different content foci, e.g. general planning and organisation of the implementation process, including awareness of structural barriers and limited resources, might be more interesting for HCPs who are in a position to decide on and plan implementations.

Strengths of the project are the stepwise design comprising a scoping review of the relevant literature, qualitative interviews and the online survey resulting in quantitative data, corresponding to the “CeHRes Roadmap” for the development of eHealth technologies [16‐18]. The high participation rate enabled a rich qualitative and quantitative database. The purposeful sampling provided perspectives of professionals of various professional backgrounds as recommended [16]. The international sample of the online survey allowed a more detailed insight and interpretation of the results for the intended international target population, differing in their level of familiarity with and expertise in assessment and interpretation of PROs and different health care systems.