How do we engage people in testing for COVID-19? A rapid qualitative evaluation of a testing programme in schools, GP surgeries and a university

Universal, repeated, weekly testing followed by strict household isolation after a positive test and continued normal life after a negative test, was promoted in the UK as an important exit strategy from the COVID-19 (SARS-CoV-2) pandemic [1‐5]. A novel pilot programme to assess the feasibility of weekly community RT-LAMP (Reverse transcription loop-mediated isothermal amplification) testing [6, 7] for the SARS-CoV-2 virus using saliva samples collected at home was developed and piloted in a unique partnership (the Southampton COVID-19 Saliva Testing Programme) between Southampton City Council and the University of Southampton. Immediately on waking in the morning, the saliva samples were self-collected at home into specimen pots, which participants then dropped off at collection points for transfer to the testing laboratory. Direct RT-LAMP testing was undertaken using the OptiGene SARS-CoV-2 assay [8]; an evaluation of the assay found an overall sensitivity of 79% and specificity of 100%, with a sensitivity of 94% in samples with a higher viral load(RT-qPCR ORF1ab CT values of < 25) [6]. Between June and October 2020, participants from two general practices in Southampton, staff and students at the University of Southampton and staff and pupils from one infant, one junior, one primary and one secondary school in the city were invited to participate in two phases of a pilot of the Southampton COVID-19 Saliva Testing Programme. The four schools had catchment areas in more deprived parts of the city, and the pupils were from diverse ethnic backgrounds. More than a quarter of students at the University of Southampton come from outside the UK. Interactive engagement activities took place alongside the testing programme to maximise uptake of saliva testing, with extensive development and deployment of educational materials targeting different age groups of students and pupils in the schools, alongside work to engage university students.

Guidance from the UK Scientific Advisory Group for Emergencies (SAGE) draws on experiences of previous public health emergencies and identifies the need for high levels of engagement with communities and individuals in the creation of successful mass testing programmes [9]. They suggested that high levels of engagement with the community builds trust, shared goals, and a sense of fairness [10, 11]; engagement also bridges cultural and language gaps [12]. Concern was expressed about the lack of community engagement in the government’s response to COVID-19 [13]. In previous epidemics, highly engaged communities and community organisations were found to play important and active roles in prevention and control of infection [13]. Data are lacking on the factors that improve engagement with efforts to prevent and control infection, including community-testing, in the context of the COVID-19 pandemic.

Rapid qualitative evaluation can be used when the public sector requires immediate feedback on programme process and impacts in order to shape policy [14]. Hence, this type of approach lends itself to providing insights into the response to public health recommendations and programmes during the COVID-19 pandemic. Data generated describing people’s lived experiences of the infection, their behaviour and responses to the government efforts to contain the virus have complemented and explained the epidemiological data [15]. Other studies involving rapid qualitative methods during the COVID-19 pandemic have provided ‘real-time’ feedback on current public health practices [16] and have informed the design of the UK’s COVID-19 response [17]. This approach was established to inform iterative development and testing of programmes and interventions and has been shown to produce findings comparable to those of more traditional qualitative research and thematic analysis [18, 19].

To generate insights and inform the design and modification of the Southampton COVID-19 Saliva Testing Programme and the next phase of future mass-testing, we conducted a rapid qualitative evaluation exploring the experiences of individuals and organisations who took part, of those individuals who declined to take part, and of those in the educational and healthcare organisations involved in the pilot testing who were responsible for roll-out.

Between 4th June and 7th November 2020, we conducted 77 interviews and 20 focus groups with 223 staff, pupils, students and household members, from four schools, one university, and two GP practices belonging to one Clinical Commissioning Group in the city of Southampton in the South-East of England. This evaluation of the Southampton COVID-19 Testing Pilot Programme was guided by the NIHR framework for community engagement and involvement in health research and, as such, was underpinned by a participatory approach, modifications recommended to the programme were driven by the participants’ priorities and we endeavoured to be inclusive of all ages, ethnicities and genders [20].

Three groups of participants, selected purposively, took part in interviews and focus groups: i) individuals who had been approached about taking part in the pilot Saliva Testing Programme including some who tested positive; ii) those who were approached but declined to take part; and iii) senior university, primary care and school representatives responsible for delivering the Saliva Testing Programme in their organisations. The majority of participants in this qualitative study were recruited via the testing programme’s online registration process, a feedback form on the Southampton City Council website, during a phone call with the testing programme team to those who tested positive or inconclusive, or through the schools’ weekly newsletters about the programme. Senior university and school representatives and a minority of other participants were recruited through the testing team’s and participants’ professional networks. Participants gave permission for their contact details to be shared with the qualitative evaluation team and were offered a small financial incentive to take part. Participants aged under 18 years and their parents signed consent forms. Participants from all groups except those who declined to take part continued to be recruited and interviewed until the point of data saturation was believed to had been reached. Those who declined to take part were more difficult to recruit and so the number of these participants interviewed was determined by those who were available within the timeframe. Participants’ characteristics are outlined in Table 1.

Twelve experienced qualitative researchers were involved in this evaluation. Each researcher observed at least one focus group or interview to ensure our approach was standardised. Focus groups, averaging 1 h long, were conducted using Zoom Pro and audio-recorded using Open Broadcast Software; telephone interviews, averaging 30 min long, were recorded using Olympus recorders and a microphone earpiece. Participants gave verbal consent for discussions to be recorded at the start of each interview and focus group. Focus groups and interviews were conducted using semi-structured discussion guides (see Additional files) developed by experienced qualitative researchers informed by the needs of the programme team and revised iteratively in the light of insights gained. Observation guides were used to capture key findings and contextual factors.

Findings from the focus groups and interviews were rapidly synthesised and added to a report, which was edited collectively by the evaluation team and continuously updated as new data and insights were generated [16, 21, 22]. This report was used to provide immediate feedback to the programme team enabling them to make modifications to the running of the programme, and to offer insights to other groups in the UK developing similar testing programmes. At the end of the pilot phases of the programme, rapid analysis of the key messages gathered in the shared report was conducted to produce a framework of themes which addressed three questions [23]: (i) what made people engage with the testing programme?; (ii) how could engagement with the testing programme be improved?; and (iii) what were the impacts on participants of engaging with the testing programme? A descriptive approach was taken to the analysis, meaning that notes taken during the focus groups and interviews were coded into the framework of themes. Findings under each theme were summarised and presented in the findings section as they answer the three questions above [24]. Quotes from participants were transcribed directly from the audio-recordings [22] and illustrate the range of responses as they relate to the three questions are provided in Table 2. The themes and key messages were synthesised and discussed widely with authors to form recommendations (see Table 4).

In a final phase of analysis, a consultation activity was undertaken with public contributors hosted by the Wessex Public Involvement Network, during which our interpretation of the data was validated and promoted transparency and reflectivity [25]. Theory was not directly applied to the conceptualisation of the evaluation or the data analysis in the first instance. An inductive approach to analysis was taken to ensure that the codes and themes were grounded in the views of participants themselves. Psychological theory was, however, applied retrospectively to assist with interpretation of the findings described in the discussion [26]. Theory on promotion- and -prevention-focused motivation was used to explain why people did and did not take part. Promotion- and -prevention theory suggests that people balance the motivation to fulfil basic needs with the desire to maintain safety and security [27]. Promotion activities generate gains for people in terms of social connectedness and reductions in anxiety, for example, where prevention activities are those that minimise risk and loss of those gains. Bandura’s concept of collective efficacy was applied to descriptions of differences in engagement between the organisations and communities involved [28]. We used COREQ guidance to structure our reporting [29].

Participants in the rapid qualitative evaluation were staff or students at the University, pupils, parents and staff from four schools and employees at General Practices in the city.

Individual participants and organisational representatives provided valuable insights into the experience of engaging with the Southampton COVID-19 Saliva Testing Programme, leading to changes in programme processes and communications, which undoubtedly increased engagement and adherence to programme requirements. They also made important observations about the benefits they experienced from engaging with the testing programme. This is known as promotion-focused motivation which describes a focus on what can be gained from a health behaviour or intervention [27]. Participants suggested that promotion-focused motivation for engaging in testing should be emphasised to potential participants by those running the programme. One of the benefits that participants in the programme emphasised was increased knowledge of the science behind management of the pandemic that school and university students gained from the educational engagement activities. As well as providing insights, it was clear from the focus groups and interviews that the process of seeking feedback from participants also increased the engagement of local people with the testing programme. Prevention-focused motivation theory would suggest that programme promotors should understand and address the value to potential participants of preventing the loss of what they already have [27]. In the case of the saliva testing programme, this would refer to their ability to go to work or school or to socialise if they tested positive. It was felt by decliners and by those who knew people who had declined to take part, that it might be best to remain ignorant of their viral status if it meant that they could continue earning money, for example. Programme implementors need to work on schemes that provide support such as financial aid to those who cannot work if they test positive and to increase emphasis of the benefits such as being able to socialise on receipt of a negative test and continuing to follow national guidance.

Involving local people and organisations in the development and piloting of programmes in this way may well have been important in ensuring community buy-in [30]. They appreciated the chance to contribute to this programme, which they saw to be of national importance. The process of holding the conversations reported in this paper may also have helped to address two key underlying issues affecting engagement with the testing programme: the need for trust between participants and testing programmes, and the role of ‘collective efficacy’ within organisations. Local organisations, such as schools and the University were seen to be answerable to local people and hence more trustworthy than more national ‘faceless’ and private organisations such as NHS Test and Trace. Collective efficacy is a group’s shared belief in its capability to organise and execute actions required to achieve goals [28], and a sense of collective efficacy appeared to be stronger in the smaller, more cohesive organisations such as the schools and GP practices than in the University, for example. Schools and GP practices had staff and student communities within which members through daily conversation and mutual encouragement made regular testing the ‘social norm’. The educational engagement activities appeared to play an important role in this too. Students’ increased knowledge appeared to make them more engaged with the programme and motivated by the need to protect their communities, as well as making them feel like they had more agency in controlling the spread of the virus and its damaging consequences. Schools and the University were thus building collective efficacy for regular testing.

A rapid qualitative evaluation enabled real-time feedback of insights into perceptions and functioning of the testing programme, improving the participants’ experience and was therefore likely to increase uptake of testing. Whilst some of these insights confirm what has been learnt during previous epidemics, our study was necessary to understand more precisely how to improve specific features of the programme which would facilitate participation. Table 3 details how insights gathered through the interviews and focus group discussions led to modifications to the programme in order to increase engagement. Insights gained were also used to inform university, schools, and city-wide strategies for managing the pandemic and to feed-back to national and local government [31]. This included evaluation reports to the UK Department of Health and Social Care [31] and presentations to the UK Department for Education based on the findings in this rapid qualitative evaluation. Figure 1 illustrates the way in which the rapid qualitative evaluation influenced Saliva Testing Programme development and wider testing strategy. This study has also produced actionable recommendations for improving engagement with COVID-19 testing programmes across the UK (Table 3). See Table 4 for details of these recommendations.

High levels of communication, trust and convenience were necessary to ensure people’s engagement with the programme. Community leaders and stakeholder organisations should be involved throughout programme development and implementation to optimise these features of the testing. Participants’ and stakeholders’ motivations, challenges and concerns need to be understood and these insights used to modify the programme in a continuous, real-time process to ensure and sustain engagement with testing over the extended period necessary.