Development of a framework and research impact capture tool for nursing, midwifery, allied health professions, healthcare science, pharmacy and psychology (NMAHPPs)

Framework development followed the three stages outlined by McMeekin et al.: i) identifying relevant data, ii) developing the framework, iii) validating, testing and refining the framework [31].

Relevant data related to the reported impacts of research activity by NMAHPPs. This included our previous systematic review [26], qualitative interviews with healthcare managers and research-active clinicians [27] and qualitative interviews with research participants and patient advisors [28]. Additional evidence was identified using a targeted literature search limited to the period after the previous systematic review (January 2020 to October 2022). The newly identified literature primarily focused on clinical academic careers and developing research capability and capacity [13, 17, 19, 20, 32‐38].

To develop the framework, the findings from our existing research were pooled and organised by the authors to create key impact themes. These were cross referenced with data identified in the additional literature. The preliminary framework was refined, validated and tested by the patient and public advisory group (PAG), the NHS Trust Clinical Academic Research Committee (CARC) and the NHS Trust Postgraduate Research Forum (PGRF). Four PAG members were recruited nationally, and all had experience as NMAHPP research participants and/or research advisors. These individuals were reimbursed for their time. CARC consists of research and clinical leads covering pharmacy, psychology, healthcare science and allied health professions within the Trust and linked organisations. The PGRF is a group of research-active NMAHPP clinicians who are affiliated with or working at the Trust and involved in research that is supported by various funding sources.

Table 1 shows the basic framework that was created through discussion, refinement and review. The reasoning behind the inclusion of each element is presented, along with the evidence used to inform and support inclusion.

This framework was translated into a series of questions to capture each element. Questions and response options were reviewed by the three stakeholder groups discussed above (PAG, CARC and PGRF) and refined based on this feedback. The questions were incorporated into an online research impact capture tool (Qualtrics XM) and pilot tested within the NHS Trust with healthcare managers from the NMAHPP professions, research-active clinicians at pre-doctoral, doctoral and post-doctoral levels and PAG members. Further refinements were made at this stage and the questions were linked back to the framework elements to ensure that all aspects were included.

The pilot research impact capture tool is available via the Open Science Framework [39]. All questions were framed in relation to research activity and impact within the previous 12 months. This period was chosen to facilitate accurate recall and to enable the questions to be repeated annually to monitor change over time.

Approximately 55 research-active clinicians within or affiliated with the Trust were invited to complete the questions by email. Individuals known to be involved in research were contacted using existing databases and the leads for each of the NMAHPP professions were also asked to disseminate the invitation using their own records of research-active individuals. Up to two reminder emails were sent.

Responses were analysed using Stata (version 15.1, StataCorp LLC). Additional comments were reviewed by the research team and grouped into similar themes.

The call for responses was open during November 2022. Thirty individuals responded from the approximate total of 55, yielding an estimated maximum response rate of 54.5%. Respondent demographics are provided in Table 2. All respondents were registered healthcare professionals. Respondents were asked about their academic qualifications and were categorised as ‘pre-doctoral’ if their highest qualification was at bachelor’s or master’s level, ‘doctoral’ if they were currently working towards a PhD or professional doctorate, and ‘post-doctoral’ if they had completed a PhD or professional doctorate. The majority of those at NHS band 5–6 (entry grade and first senior level) were pre-doctoral (63%), and the majority of those at band 7-8a (clinical specialist or local team lead) were currently undertaking a PhD or clinical doctorate (50%). All individuals working at NHS band 8b and above (department lead or consultant clinician) were at a post-doctoral level, however there were individuals who had completed a PhD or professional doctorate across all clinical grades.

Half of respondents had been employed by their current organisation for more than 5 years (n = 15), compared with 27% (n = 8) for 2–5 years and 23% (n = 7) for less than 2 years. Duration varied according to academic level; more than 55% of doctoral and post-doctoral respondents had been at their current organisation for more than 5 years versus 33% of pre-doctoral respondents.

Those who moved to a new organisation within the last 2 years were asked to select the reasons why they chose to move. Those who had worked at the same organisation for at least 2 years were asked to select the reasons why they chose to stay. Responses are summarised in Table 3.

Respondents were also asked to identify the most important reason for joining or remaining at their organisation. For both groups, the majority selected ‘research and clinical academic opportunities, including access to research funding’ (< 2 years 29%, ≥ 2 years 40%).

Fourteen respondents (47%) reported that at least one person had applied for a role in their department within the past 12 months because of the research culture and opportunities. A further 10 (33%) were unsure.

Nearly three quarters of respondents held an honorary contract linked with their research activity (n = 22, 73%). The majority were NHS employees holding honorary contracts with a university (n = 18, 82%). The remainder were either NHS employees holding contracts with additional NHS organisations, or university employees holding an honorary contact with the NHS. Honorary titles could be classified into six groups: research fellow or clinical research fellow, research officer, research assistant or associate, clinical lecturer, consultant, and professor.

All respondents were asked about their ideal career progression within the next 5 years. The most commonly selected option was ‘clinical academic – based in the NHS’, as chosen by 21 respondents (70%; Fig. 1). Seven individuals also provided additional comments. All focused on a desire to establish combined clinical academic roles that enabled specific time for clinical activity, research capacity building, and research.

Within the previous 12 months, respondents reported involvement in a combination of research, service evaluation, audit and quality improvement (QI) projects. These activities are summarised in Table 4. Compared with pre-doctoral respondents, a greater proportion of doctoral and post-doctoral respondents were involved in research leadership and related supervisory and governance roles.

Twenty-two respondents (73%) had allocated research time through a research fellowship or grant. Three individuals (16% of those employed by the NHS) reported protected research time as part of their NHS job plan. This compared with four individuals with protected research time as part of their university job plan (36% of those employed in the university through any type of contract).

Twelve respondents (40%) had secured backfill to enable protected time for research. Another 12 reported that backfill was not applicable for their research activity and role because research was separate to any other duties. Of the 12 who had secured backfill, this was provided at the same clinical grade for six individuals, at a lower grade for five individuals, and one individual was unsure of the grade. Additionally, four respondents reported that no backfill funding was available, and one reported that their backfill funding was used as a cost-saving, rather than to appoint another person.

There were six additional comments regarding backfill (five from nurses/midwives and one from an allied health professional). Respondents covered all academic levels and were employed within the NHS or on joint or separate NHS/university contracts. Comments centred on the difficulty of recruiting to short-term, part-time roles that were often in a highly specialised clinical setting. Respondents recalled how this led to “clinical work being absorbed by the current team” (ID25), resulting in increased “pressure on the service and other team members” (ID15). It was identified that there was “little acknowledgement within the clinical role of the contribution made by research work” (ID35) and that research-active clinicians felt guilty because research was viewed as a “foolish extravagancy when [the team] is so short staffed” (ID28).

Twenty-nine individuals completed the sections relating to research capacity building. Reported activities are shown in Fig. 2.

Twenty respondents reported changes to their own clinical practice as a result of their research involvement in the previous 12 months. The most commonly reported changes were increased confidence in discussing treatment ambiguities with colleagues (12 individuals) and supporting their clinical service to change practice based on research (12 individuals; Table 5). Respondents were also asked about the broader impacts of their research across local, national, and international settings. Sixteen individuals identified changes to practice in these settings, the most common responses were new or different pathways of care at a local level (eight individuals) and the use of new or existing clinical guidelines at a national level (eight individuals). There were two additional comments; both highlighted potential future impacts of the individuals’ current research and mentioned that it was too early in the research process for research translation.

The majority of respondents reported involving patients and the public in their research. Two (7%) reported informal discussions and engagement activities, while 20 (67%) reported formal involvement through advisory groups and co-investigator roles. Involvement activities are shown in Fig. 3. Six respondents (20%) reported no patient or public involvement of patients or the public within their research activities.

Eight respondents (29%) provided their research participants with payment or a voucher, and six (21%) provided reimbursement for travel expenses. Of the 20 respondents who had formally involved patients or public in an advisory role, 10 (50%) provided payment or a voucher for this contribution, six (30%) provided travel or other expenses, while four (20%) did not provide any form of financial compensation.

Comments relating to the involvement of patients and the public focused on the beneficial impact of including different perspectives, for example, it was “extremely important to help the study to be patient friendly and for constructive feedback” (ID19) and “securing PPI [patient and public involvement] from diverse communities is one of the great things going on in this organisation” (ID28). However, respondents commented that it was difficult to set up payment for patient/public advisors within NHS systems “payment of PPI group attendees is complicated… still figuring out how paying each person will work” (ID15), or that they were advised to only offer expenses “I was advised against payment/vouchers for participation or involvement, so the best I could offer was expenses” (ID21).

Reported methods of sharing study findings with participants are outlined in Table 6. None of the respondents reported using a study website to share findings or providing a summary of any changes to practice. Only two respondents (both post-doctoral) reported updating their participant about any new research developments that arose from the existing project.

Twenty five respondents provided information about their research dissemination activities in the past 12 months (Fig. 4). The most commonly reported activities were providing feedback to their local department (16 individuals) and giving a presentation via a submitted abstract at a national or international conference (14 individuals).

Respondents were also asked about any prizes that they had been awarded in the past 12 months. Four had won prizes for poster presentations and four for conference free paper (oral) presentations. Two individuals had been awarded research-related prizes from their own organisation, one had been awarded fellowship of their professional body, and two had been awarded governmental honours.

Two respondents (both pre-doctoral) reported cost-savings, improvement in efficiency or income generation as a result of their research activities. An additional 12 (40%) were unsure whether these impacts had occurred. Respondents were separately asked about the availability of support to carry out economic assessments; 17 (57%) reported that it was not relevant for their research; nine (30%) reported that they would have liked support but were unable to access it; and two (6%) reported that they had accessed either formal or informal support with health economic assessments.

Seven individuals provided additional comments regarding income generation and health economics. These focused on plans to run future workshops and teaching to generate income for the organisation (ID7, ID35); the idea that economic evaluation would occur at a later stage in their research (ID13, ID9, ID29); and not knowing where to start with health economics (ID30, ID28).

Twelve individuals reported applying for new research funding in the last 12 months. The outcomes of these applications are summarised in Table 7.

Twenty-two respondents reported that they had developed new research-related collaborations in the past 12 months. The nature of these collaborations is summarised in Table 8.

Our research impact capture tool has several important limitations. Firstly, it relies on self-reported data. However, this is a common format for other systems of recording impact, including REF, ResearchFish® and VICTOR [42]. The timing of data collection needs to be carefully considered to avoid potential survey fatigue. Our pilot period coincided with the NHS Staff Survey, which has organisation- and department-level targets for completion [52]. Despite this, we achieved a response rate of > 50%, illustrating high levels of engagement. However, we plan to move the completion window for the next round to avoid this overlap.

Secondly, our sample population was not fully representative of the wider population of research-active NMAHPPs within the Trust and affiliated organisations. The self-selected sample may have been biased towards those who were interested in capturing the impact of research activity, especially due to the limited opportunities currently available for those wishing to pursue a clinical academic career in the UK. Therefore, the reported findings may over-estimate the impacts of research activity compared with a wider population. Additionally, we had no responses from healthcare scientists or psychologists, only one respondent was male, and there was limited ethnic diversity. We will further refine our sampling strategy and engagement activities for subsequent rounds to ensure that all NMAHPP disciplines are aware of the purposes of the impact capture tool and their inclusion within the NMAHPP acronym. The lack of ethnic diversity in our sample may also be representative of wider patterns of limited diversity within healthcare disciplines, for example in 2022, only 12% of allied health professionals came from Black, Asian or Minority Ethnic (BAME) backgrounds compared with 20% across the NHS workforce [53]. Initiatives such as FAIResearch have been established to support fair, accessible, inclusive research opportunities for allied health professionals [54], and demographic data from the research impact capture tool could be used to record this aspect of research involvement.

Thirdly, there is currently no method to link the data captured from this tool with measures of experience or feedback from patient and public advisors or research participants. A possible future expansion could include direct data capture or data triangulation with participants or other public contributors.

Finally, we did not collect data about individual departments within employing organisations. This would help identify multidisciplinary areas of research collaboration and is something we plan to add for the next iteration. This will allow learning from research-active departments and enable additional support to be provided for areas with lower research involvement or impact.