Integrating research and system-wide practice in public health: lessons learnt from Better Start Bradford

In 2015 the Big Lottery Fund launched “A Better Start” across 5 sites in the UK. The 10 year programme aims to give children the best start in life by offering interventions to pregnant women and children aged 0–3 years. One of the selected sites was Bradford, a socio-economically deprived and ethnically diverse city in the North of England. Within the city the programme is being delivered by Better Start Bradford, a community led partnership involving key organisations delivering children’s services in the area including the National Health Service (NHS), public health, Local Authority, Voluntary and Community sector (VCS) organisations [4]. The Better Start Bradford programme is implementing more than 20 interventions that are delivered by a range of statutory and VCS organisations (Table 1). The lack of a strong evidence base for early life interventions means that the majority of these interventions have been defined in a recent review [2] as ‘science based’ (e.g. developed using the best available evidence, but not tested or proven effective using robust methods of evaluation), rather than ‘evidence based’ (e.g. tested and proven effective using robust study designs) [2].

A unique feature of Better Start Bradford is its partnership with Born in Bradford (BiB), a birth cohort study following 12,500 families. BiB began in 2007 and its remit is to better understand and to improve the health and wellbeing of children in the city and beyond [20, 21]. Together, Better Start Bradford and BiB established the Better Start Bradford Innovation Hub, a centre for monitoring and evaluation of multiple complex interventions within Better Start Bradford. To facilitate this, the Better Start Bradford Innovation Hub has established an innovative experimental birth cohort: Born in Bradford’s Better Start (BiBBS) [4]. BiBBS seeks consent from families living in the Better Start Bradford areas to follow them through linkage to their routine health, education and social care data and to monitor their participation in Better Start Bradford interventions [4]. The Innovation Hub offers an opportunity to efficiently evaluate multiple complex early life interventions through planned controlled experiments and quasi-experimental methods using routinely collected data from partners in health, social care, education, and the interventions themselves, to provide information on baseline characteristics, exposures and outcomes. It also aims to conduct implementation evaluations using best practice, as defined by the Medical Research Council [9], including complementary qualitative work.

Strategies and tools were developed using the principles of co-production (e.g. [22]) defined in this case as working in partnership or ‘with’ key stakeholders including commissioners (Better Start Bradford), implementers (e.g. statutory and VCS Organisations), and service users (community representatives). First, a series of workshops were held with representatives from all groups to identify the key challenges of integrating research and practice, and develop appropriate strategies and tools to overcome these. A summary of these challenges and agreed strategies can be seen in Table 2. Once developed, the first iteration of the strategies and tools were shared at a full day workshop including community representatives, commissioners and service providers from a wide range of health, local authority and VCS organisations in Bradford, as well as academics. Feedback from the workshop was used to refine the strategies and tools. The end result of this process was the production of a series of practical, pragmatic strategies and tools usable by researchers, service providers and commissioners to overcome the challenges of integrating research into public health practice. These strategies are described in detail here and the corresponding tools and templates are available from the BiB website [23].

Successful community and stakeholder engagement requires their involvement from the beginning, and at all stages of intervention design, delivery, evaluation and dissemination. Establishing a Community Advisory Group (CAG) made up of local people (e.g. local parents/patients, volunteers and local business leaders) will facilitate this integrated involvement. The CAG can be involved at every stage of intervention design and evaluation development including setting evaluation objectives and outputs, developing the wording of surveys, developing information sheets and consent forms, and advising on appropriate methods for engaging with and recruiting local parents. The group can also play a key role in the interpretation and dissemination of findings before they are made public. Alongside a CAG, consultation events and focus groups can be conducted as and when specific guidance from the community is required to shape the work. Having a presence in the local community by attending events, contributing to local newsletters, newspapers, radio programmes and through social media is also a good facilitator.

The support and commitment needed to integrate research into practice goes beyond the obvious research and practice teams, to include senior management, commissioners and data teams within different organisations. The starting point is a careful mapping out of all key stakeholders followed by regular and effective communication, preferably face-to-face. This contact enables all stakeholders to begin to learn how to work together, develop a common language, and gain a shared understanding of the pressures and priorities of all sides that enables shared objectives to be agreed. Working closely with commissioners and stakeholders enables a mutual understanding of the requirements for robust evaluation by researchers, the range of factors that impact on commissioning and de-commissioning decisions including the timelines required for different decisions, and the practical challenges facing service providers. We have also developed an evaluation framework (see “Evaluation” section) in which evaluations are staged to allow short-term evaluations around implementation and trends that can fit into commissioning timelines, ahead of long-term effectiveness findings.

Within Better Start Bradford, each intervention undergoes a service design process involving the commissioner, a provider with expertise in delivering the service, the local community, and researchers. Service design describes the process by which all aspects of an intended intervention or service are specified, from referral and recruitment through to data capture, monitoring and evaluation. It helps to ensure that all parties involved in the intervention or service are clear as to the rationale behind it, how it will be delivered, what resources are required, what the intended outcomes are, and how the intervention or service should be monitored or evaluated.

We have developed a pragmatic operational guide that provides a framework to take an intervention through the service design process in a number of sequential phases [23]. This allows the expectations of all parties to be clear from the outset and ensures that all requirements are considered in a logical order including: specification of the complex components of the intervention; consideration of the practical challenges and service constraints; the needs of the local community; recruitment and referral pathways; identification of measurable and appropriate outcomes through a logic model; and clarification of the data needed to measure these outcomes. Our guide contains a series of templates to help with these processes including: A questionnaire to clarify the components of the intervention; a referral and recruitment pathway; a logic model template; and a minimum data-set to ensure appropriate and meaningful data capture (See Fig. 2 for an example).

Ongoing monitoring of performance during the delivery of an intervention is important to ensure that the intervention is being implemented as intended, and thus an evaluation of its effectiveness can ultimately be conducted when the intervention and systems are fully developed and operational. Identifying three key performance indicators (which we term progression criteria) and agreed boundaries that allow performance on these indicators to be rated on a “traffic light” symbol of “Red, Amber, Green” will facilitate this process. It is important that selection of the criteria and boundaries is a shared process including service providers, commissioners and evaluators. Agreed key performance criteria provides a simple way to obtain a regular overview of performance and also allows for early identification of areas of success that can be shared with other interventions, as well as areas of potential concern that can then provide the basis for discussions on support or adaptations as required. By developing these criteria in collaboration with local services there is acceptance across partners when issues are identified and this allows a solution focussed discussion to occur. In-depth description of this process will be published shortly, and an example of the benefits of using progression criteria can be seen in Table 7.

Many guidelines about gathering evidence start at the point at which an intervention is ready for evaluation [8, 15] but there is often a lot of work required to get interventions to this point.

We have proposed strategies in our operational guide and toolkit [23] that are relevant to this process. In addition, we have worked closely with our partners and stakeholders to produce a monitoring and evaluation framework that highlights the steps needed to build up the evidence base for an intervention [23]. This framework takes a staged approach to evaluation based upon the logic model for the intervention (activity, input, output, short and long-term outcomes), allowing each stage to be clearly defined as a part of the process. An example of this approach is described in Table 8. The framework sets expectations of all partners by outlining what each step will and will not be able to tell us and what is required to be able to complete that level of evaluation. To support this framework we have adapted an evaluability checklist to aid decisions and set expectations about when an intervention is ready for an effectiveness evaluation. The evaluation framework provides a flexible approach to evaluation, meaning that decisions can be made based on the quality of current evidence, logistical constraints such as the time period that an intervention is commissioned for, ethical constraints and implementation performance.