Background
The RE-AIM framework
RE-AIM Dimension | RE-AIM questions | Study research question |
---|---|---|
Reach | What percentage of potentially eligible participants are recruited, and how representative are they? | How many people are judged to be at risk for IGR, and how many were recruited? How representative are those recruited of the local population? How many people were recruited from high-risk populations? Which treatments did participants enter? Which referral route was most effective in recruitment of eligible participants? |
Effectiveness | What impact did the programme have on targeted outcomes? |
Clinical outcomes for individuals were not evaluated in this study.
|
Adoption | Which setting and intervention agents were involved and how representative are they? | What features of the participating services were considered crucial to delivery? Did engagement vary across the different services? What characteristics of services are required if the intervention was to be adopted elsewhere? |
Implementation | Was there fidelity to the intervention, and what local adaptation occurred? | What were the barriers and facilitators to delivering the intervention as planned? How did contextual factors impact on delivery? How was the intervention tailored to different populations? Was delivery adapted over time, and if so why? |
Maintenance | Individual level – extent to which behaviours are maintained long term; Institutional level – extent to which programme is sustained over time. | Individual: which factors are associated with adherence to treatment – does referral route impact on this? Institutional: What factors will impact on continuation of the programme? What resources are considered necessary to improve delivery? |
NPT and RE-AIM constructs and questions
Adoption: how engaged are providers? What factors impact on their engagement?
Implementation: to what extent was the intervention implemented as planned? What were the barriers to implementation?
Methods
Quantitative Evaluation
Design
Population and data sources
Analysis
Qualitative Evaluation
Design
Population and data collection
Community referral service | 16 |
---|---|
Decision makers and service leads | 8 |
Primary care | 6 |
Exercise | 2 |
Complete sample size | 32 |
Analysis
Topic | Exemplar Data | Initial interpretation & analysis | Interpretation with NPT Constructs |
---|---|---|---|
1. Adoption: engagement of providers | |||
1a) Coherence: consensus, agreement and congruence around shared purpose | “GPs [General Practitioners] are interested in health promotion and disease prevention, but actually they’ve already got a load of stuff to do with people who are already ill.” Primary focus group 1 “I think we’re very restricted time-wise because of staffing to sit down with patients to impart that information and utilising the services that are setup.” Primary 3 | Value of prevention recognised but accompanied by perception that practices themselves are under- resourced to deal with this. | High coherence, with all participants agreeing on the need for a prevention programme with patients at risk of diabetes, and agreement that existing practice resource was insufficient. |
1b) Cognitive participation: roles and relationships | “You need to be a healthcare professional, yeah … You’ve got to interpret the blood results, look at the patient records to see if they’re suitable.” Lead 5 “Having somebody that knew about diabetes … and able to make decisions at a higher level. You need that gravitas of someone who’s clinically trained. The patients respect that more.” Primary focus group 1 “I put together flowcharts for our surgery about what to do around blood sugar, I adapted the stuff that the nurse facilitator had originally sent me and I did a flowchart for staff on how to refer so I’ve done quite a lot of work … It’s taken months of implementing.” Primary 3 | The support of the NF (Nurse Facilitator) was viewed by practices as providing additional full capacity to co-ordinate and deliver referrals (as opposed to supporting the practices to do this themselves). Perceived as essential that the role was for a Nurse, who could interact with the clinical systems and with patients in a clinical capacity. Practices which did not receive full support reflect on complexity and burden of referral process. | Cognitive participation was straight-forward given that consensus was that the NF needed to act independently rather than requiring additional work by the practice staff. There was also agreement that the nurse facilitator was the appropriate role, as the person offering support needed to be clinically trained. In those practices where the full support of the NF was not provided, practices instead reflected on the burden created by referral, again demonstrating that cognitive participation for these participants centred around the difficulty of practices themselves trying to do the work and therefore needing full support. |
2. Implementation and barriers to implementation | |||
2a) Collective action: relationships and confidence in each other | “A lot of [GPs] felt that the education they already gave them [patients] was adequate … so their question was why should we refer in?... But when you then explain how you go into things in great detail [in the telephone service] and what we actually do, they could see what’s going on.” Lead 5 | The NF role as part of the telephone service increased understanding of the service. | The model of NF was simple to implement in comparison to the community route given that a need for collective action was largely avoided – the role of the NF in conducting the work in practices and liaising directly into the telephone service meant the interactions were simplified. The NF’s dual role as part of the telephone service itself further embedded trust in the process by being able to directly communicate value to GPs. |
2b) Reflexive monitoring: extent to which there is a shared understanding about the intervention. | “It’s certainly built up relationships. I still get emails now from doctors in different practices and practice nurses, so they’re now aware of the service which before they may not have been... … they now know that they’ve got somebody to contact if they need the help … I think once you’ve got that link they will be more receptive to referring in and the services and helping the patients.” Lead 5
“I’d refer to our Health Care Assistant [HCA] … they’d get a face-to-face with a HCA. I could understand if I was getting some diabetic input … If it’s just for a chat on the phone then I wouldn’t refer. If it was just to a health care person then I’d do that in-house.” Primary focus group 1
| Participants again reflected on the value of the NF being integrated with the telephone service itself, which provided reassurance and an accessible way to clarify issues regarding referral and the service. Possible changes to the telephone service intervention itself (removing a first contact with a Diabetes Specialist Nurse) may threaten the evaluation of the service as useful, and consequently the need for referral. | Reflexive monitoring was supported by the NF being the person performing the referrals and also part of the telephone service itself, providing a clear means for staff to understand the value of the service. However, this may be undermined if primary care staff felt that patients would not benefit from additional specialist input, demonstrating how congruence (perceived value) can be reflectively reassessed and may decrease cognitive participation and collective action. |
Topic | Exemplar Data | Initial interpretation & analysis | Interpretation with NPT Constructs |
---|---|---|---|
1. Adoption: engagement of providers | |||
1a) Coherence: consensus, agreement and congruence around shared purpose | “[The CCG – Clinical Commissioning Group] I think are really supportive of community based interventions. They appreciate that it brings something different to the table.” 0I9 (Community Service Lead) “The intervention we’re being asked to signpost into is an intervention that’s based in secondary care services at the hospital, and yet these people aren’t ill...So the whole fundamental way they think about it is to treat people in a clinical way, and it goes against the ethos and the way that we would work” 0030 (Community referral provider focus group) “It’s really frustrating for our staff and demotivating when they’re seeing budget cuts continually in this service and then they’re seeing another service being commissioned to do what they were doing” 0030 (Community referral provider focus group) | Buy-in to the need for community referrals at all levels. Both Community agencies referred to similar benefits in comparison to clinical referrals (proactive, raising awareness, being more approachable) and similar requirements (flexibility, the importance of local knowledge) Tensions between the community and clinical services regarding their different approaches, and tensions between the two community services in the context of limited funding, Unclear on value of both community services working together, what this added. | There was coherence from all participants (frontline workers, managers, and commissioners) for the need for community referral as a different way of working and the likely benefits. However, there was a lack of coherence regarding how this would be achieved and the value of the two community services working together. |
1b) Cognitive participation: roles and relationships | “I think communication and an understanding of peoples roles, and a understanding of how people fit in, into the whole process, I think that was our kind of key. And some people maybe had a misunderstanding of what the role was … I would of liked … the expectations in place and then start, … getting expectations drawn up between the different parties” 033 (Community referral provider co-ordinator) “I didn’t get...personally didn’t get my head round Care Call until much later into the pilot. How does it work? How does that link up with the case finding?” 019 (Community referral provider service lead) | Lack of clarity about how the collaboration would work in practice and how the different services were expected to work together. | Cognitive participation was problematic. The services involved did not share understanding about the work to be done (both the collaboration between the community services and the interaction between the community services and Care Call) and the processes required, and in the early stage of the pilot had not been brought together to collectively resolve the issues |
2. Implementation and barriers to implementation | |||
2a) Collective action: relationships and confidence in each other | “It’s really fragmented and broken down. There’s loads of different people involved in it and nobody knows what anybody else is doing.” 0030 (Community referral provider leads focus group) “One of the problems has been communication …. I think sometimes when you’ve got a service like [Care Call]that’s very clinical and it’s almost quite a closed system and they know the referrals they’re getting through is very clean, I don’t think they understand how difficult it is for us to actually case find in the community.” 0030 (Community referral provider leads focus group) “After speaking to [leads] they just said to me get out as much as you can and get as many people as you can … the fact of working with [the other service] and them referring in, I don’t really get where all that’s working together.” (042 Community referral provider frontline worker) “While we are trying to target sometimes you don’t try to target too much. You’ve got to hit the numbers.” (028 Community referral provider frontline worker) | The lack of agreement on how to work together meant that provision was fragmented rather than collaborative, and divisions between the services were maintained. This had 2 impacts on the referral process itself 1. Care Call were unprepared for the different referrals received from the community services. The community services felt that Care Call did not understand the work they were doing and had unrealistic expectations about what could be provided (for example, NHS Numbers) which led to delays in referrals being processed. 2. The focus on targeting was obscured by a focus on each service trying to “make up the numbers” rather than working effectively to co-ordinate the work required. | Collective action was not achieved – the division of labour and mechanisms for working together were unclear. The services did not have confidence in each other which led to delays and meant that the focus on targeting high-risk areas was partly abandoned. |
2b) Reflexive monitoring: extent to which there is a shared understanding about the intervention. |
“What’s been useful is the meetings that we’re having with [clinical service] around this project because we’ve relayed that information and we’ve actually had people from there attend and we’ve been able to put those to them and say, you know, we want the feedback.”
(Community referral provider frontline focus group) “When we’re [community agencies] working together it’s really important that we know what they want but it’s also very important for them to tell us what they want... So communication is, we’ve learnt a lot from this about how different we work together” (Community referral provider frontline focus group) | Over the course of the pilot, the services all made efforts to improve their communication and understand each others’ roles. This led to an appreciation of the need to collectively understand the issues from each others perspective in order to resolve them. | Reflexive monitoring emerged as key over the course of the pilot, with services needing to come together directly to share their experiences and preferred ways of working, and revise processes where required, including establishing mechanisms for further feedback. The learning in terms of understanding each service and how to work together was highly valued but took time to develop. |
Results
Reach – comparison of community and primary care identification
How many people are judged to be at risk for IGR, and how many were recruited?
Diabetes risk score category | Number with risk score (% of total) | Number given blood tests (% of those with risk score) |
---|---|---|
No risk score done | 1 (0%) | 0 (0%) |
Low/increased risk | 371 (32%) | 6 (2%) |
Medium risk | 517 (44%) | 468 (90%) |
High risk | 274 (24%) | 272 (99%) |
Total | n = 1163 | n = 746 (64%) |
How representative are those recruited of the local population?
Diabetes prevalence
Reach – comparison of recruitment into the intervention
How many people were recruited?
Source of referral | Referred | Started (% of referred) | Completed (% of started) |
---|---|---|---|
GP nurse facilitator | 633 | 288 (45%) | 212 (74%) |
Other GPs | 55 | 38 (69%) | 27 (71%) |
Community | 36 | 8 (22%) | 4 (50%) |
Total | 724 | 334 (46%) | 243a (73%) |
Source of referral | Referred | Started (% of referred) | Completed (% of started) |
---|---|---|---|
GP nurse facilitator | 24 | 17 (71%) | 14 (82%) |
Other GPs | 3 | 3 (100%) | 2 (67%) |
Community | 22 | 12 (55%) | 9 (75%) |
Telephone DPP | 90 | 63 (70%) | 55 (87%) |
Total | 139 | 95 (68%) | 80 (84%) |
Which referral route was most effective in recruitment of eligible participants?
Adoption and implementation
-
Perceived value of the new referral methods
-
Tensions in regard to inter-organisational working
-
Discrepancies between the service leads focus on targeting and how this was described by front line workers for the community route
-
The need for direct communication between the organisations to resolve process issues
Maintenance
Individual: which factors are associated with adherence to treatment – does referral route impact on this?
Discussion
Summary of findings
RE-AIM Dimension | Key Findings |
---|---|
Reach | • The community campaign completed diabetes risk scores with 1162 people, and blood tests with 746 people, of which 71 were diagnosed with NDH and 66 (6%) were referred to a local DPP. The conversion rate was disappointing, suggesting that the community campaign was not particularly effective. • There were 883 referrals to the DPP from primary care A nurse facilitator undertook electronic searches and/or clinics in 16 practices and thisresulted in the referral of referred 774 (88%) patients to the DPP. The remaining 109 (12%) were referred from the 30 practices without support from the nurse facilitator. This suggests that the addition of the nurse facilitator was effective in producing more referrals. • Within thecommunity referral route, of the completed diabetes risk scores, 46% were with people over 70, 65% were for women, 7% were for someone from an ethnic minority and rates of completion were higher in the least deprived wards and those with lower rates of diabetes. This suggests that further targeting to high risk groups would be beneficial. • The community campaign led to 8 people starting the telephone DPP (22% of those referred). The facilitated GP route (16 practices) led to 288 people starting the telephone DPP (45% of those referred). The GPs without extra facilitation (30 practices) led to 3 people starting the telephone DPP (100% of those referred). |
Effectiveness |
Not assessed in this study.
|
Adoption | • Adoption of the intervention itself was strongly supported by the professionals involved in delivery, with consensus around the need for additional resource to support identification in primary care, and the need for community-focused organisations to expand identification beyond clinical settings. • However, in the community service, a lack of buy-in to the need for collaborative working hindered inter-agency collaboration in the early stages of delivery. |
Implementation | • The facilitated GP route was comparatively easier to implement, with the role of the nurse facilitator well understood and integrated into existing processes. The community services, due to lack of consensus around the value and processes of collaborative working, did not work in the integrated way intended. Resulting pressures impacted Reach, as there was a lack of fidelity to the intended focus on high-risk patient populations. • Adaptations over time had the potential to impact both routes, both positively and negatively. In the community services, collaboration was enhanced through deliberate efforts to improve inter-agency working. In the GP route, changes to staff involved may undermine trust in the process. |
Maintenance | Once people had started in a DPP programme, the retention rates were fairly high, with 73% of people completing the telephone DPP and 84% completing the exercise DPP. Retention rates were lower among the community referrals (50% in the telephone DPP and 75% in the exercise DPP), but this is based on small numbers of community referrals, so the result must be treated cautiously. |