How context can impact clinical trials: a multi-country qualitative case study comparison of diagnostic biomarker test interventions

The CRP POCT trials were situated in Southeast Asia, which is deemed a global AMR epicentre due to widespread unregulated antibiotic use in humans and animals [43‐45]. The qualitative data were collected in a cross-sectional design as part of the clinical trials, involving participating HCWs and patients (see Table 1 for an overview of the clinical trials). The initial intention of this qualitative data collection was to broaden the evidence base and help explain the results of each clinical trial individually [46]. More specifically, the objective of the qualitative data collection in Chiang Rai and Yangon was to contextualise the CRP POCT: “within an existing system of practices at the patient–health system interface” [47]; the Hanoi data were collected to “assess the acceptance of CRP [point-of-care] testing to reduce inappropriate antimicrobial use for self-limiting [acute respiratory infections] among patients [including guardians of child patients] and HCWs in Vietnam” [48]. Although the initial intention of the qualitative data collection was not to carry out a cross-case comparison, the qualitative data sets contained extensive and compatible context-related information (e.g. patients’ conceptions of illness, HCWs’ perceptions of patient expectations) that enabled a secondary analysis across the three trial case studies. We therefore synthesised the qualitative data from the three trials and compiled a large qualitative data set, using inductive thematic analysis (i.e. without predefined themes) to identify factors that accounted for cross-case variation in the effectiveness of the CRP POCT trials [49].

The target populations for the clinical trial in Hanoi were patients with non-severe acute respiratory infections, which represented a group of particularly high (and potentially ineffectual) antibiotic use in Vietnam [50]. The trials in Chiang Rai and Yangon focused on febrile patients, considering the regional endemicity of malaria alongside the limited range of diagnostic aides to guide treatment for undifferentiated fevers [32]. These target groups are comparable in their clinical presentation and represent the commonest causes of attendance in primary care. The target samples of the clinical trials involved 1200 patients each in Chiang Rai and Yangon and 2000 in Hanoi, each comprising 50% adults and 50% children (4446 trial participants were recruited in total). As part of the consent process, all trial patients were provided with brief information in the local language about the role of antibiotics and the risk of antibiotic resistance.

The clinical trials assessed the effectiveness of a CRP POCT on reducing antibiotic prescriptions at urban, peri-urban, and rural primary-care-level healthcare facilities. As a diagnostic tool, CRP indicates whether a patient is likely to have a bacterial infection, and the CRP POCT was intended to discourage HCWs from prescribing an antibiotic below a certain threshold that indicated the absence of bacterial infections (10 and 20 mg/L in children and adults in Hanoi, respectively; and two intervention arms using 20 and 40 mg/L thresholds in Chiang Rai and Yangon). The CRP POCT was administered as a finger-prick blood test, and a reader device (NycoCard II® reader) reported CRP levels within 5 min (for a more detailed description of the process, see [47]). The participating primary healthcare facilities in Chiang Rai comprised public health centres staffed with nurse practitioners and public health technical officers. In Yangon, the outpatient department of a government hospital and clinics run by a local non-governmental organisation were staffed with medical doctors, and so were the public health centres and the public district hospital in Hanoi. Healthcare staff in all three sites received the CRP POCT results from dedicated study staff, namely study nurses in Chiang Rai and study doctors in Yangon and Hanoi. The HCWs would then use the CRP POCT results to complement their clinical judgement.

The qualitative sampling strategies and the ensuing samples are summarised in Table 2. In Chiang Rai, all participating HCWs were recruited, whereas the Yangon and Hanoi studies sampled at least one HCW from each participating site depending on their experience with the CRP POCT and their availability. Patient sampling in Hanoi involved a random sample of the patients in the treatment groups (including guardians when the patients were children), recruited successively until data saturation was reached (defined as no new themes arising from two consecutive focus group discussions/semi-structured interviews, which can already occur after 10–12 interviews [51, 52]). In Chiang Rai and Yangon, purposive maximum variation samples were obtained specifically to capture differences across the target population. The variables guiding the patient selection were the randomised allocation to the trial group, antibiotic prescription, sex, age, and education level. To understand the intervention context more comprehensively, the Chiang Rai and Yangon samples also included febrile patients who did not participate in the clinical trial (sampled through patient logs of the healthcare facilities). The resulting qualitative data set involved 130 participants and comprised semi-structured interviews (SSIs, lasting 30–90 min) and focus group discussions (FGDs, lasting 1–2 h).

The topics of the data collection instruments were based on the literature around frontline HCWs, treatment seeking and antibiotic use in LMICs, and rapid diagnostic testing for malaria [53‐55]. SSI and FGD guides covered similar topics, which are summarised in Table 3 (the complete interview guides are presented in [47, 48]). The FGDs focussed on triangulating insights from the SSIs and therefore did not explore the topics in the same level of depth. All interviews and discussions were conducted in the local languages (Burmese, [Northern] Thai, Vietnamese) and yielded approximately 98 h of audio-recorded material. The audio records were transcribed and translated by the study team members who conducted the interviews. The transcripts were complemented with written notes describing for instance the interview setting and non-verbal communication. The resulting 977,000 words of written material formed the basis of our qualitative data analysis.

Considering the scarcity of knowledge on contextual factors and their “subjectivity” [3], we chose an inductive thematic analysis approach for this cross-case comparison. This means that we developed themes from the narratives of patients and HCWs who were involved in the trials (although different from a “grounded theory approach,” this is sometimes referred to as “grounding” themes in the qualitative data [56]; the inductive approach is opposed to a deductive approach that would explore themes derived from theory or the literature [49]).

The trial outcomes that we considered were (1) patient adherence and (2) HCW (or “clinical user”) adherence to the biomarker test results and (3) patient exclusion, to gauge the potential “impact” of CRP point-of-care testing on the population level (e.g. even if user and patient adherence were high, the trial impact may be diminished if the CRP POCT only reaches a small fraction of the population). Following the process description for qualitative cross-case comparisons by Bazeley [40], the analysis took place in 3 stages: first, description of each individual case study; second, identification of similarities and differences of each case in relation to the trial outcomes through pairwise comparisons; and third, interpretation of key variables influencing the trial outcomes in the three case studies.

In stage 1, all textual material was read and coded using Nvivo 11 [57]. Codes were assigned in relation to any of the 3 outcomes, starting with the SSI transcripts as our primary data source, which we triangulated with FGD transcripts (interview notes provided an overview and contextualising information). The triangulation through FGD transcripts involved checks of completeness and representation of statements from the SSIs, whereby we analysed participants’ contributions individually in the context of the dynamic evolution of the discussion (see [58]; this corresponds to participant-based group analysis, as opposed to whole-group analysis [59]). In order to ensure consistency of the coding frame across the three case studies, this stage involved two coding rounds of all textual material and was conducted in a single-coder approach by the lead social scientist (MJH; ambiguities in meaning in the translated transcripts were clarified during this stage through the local study team members NC, NTTD, YKZ).

In stage 2, each individual case study was compared pairwise against the remaining two cases, the similarities and differences of which were tabulated in a cross-case comparative matrix. After a first round of comparative coding by the lead social scientist, the matrix was validated and amended by the local study team members who were involved in the qualitative data collection (NC, NTTD, YKZ), and by the team members who coordinated the clinical trials across the three countries (TA, NTTD). This validation process involved the critical interrogation and challenge of the identified themes (e.g. in light of the original-language transcripts) and the consideration of alternative or omitted themes. This joint deliberation entailed a further round of analysis of all textual material relating to challenged or omitted themes.

In stage 3, the study team derived jointly the key themes of the analysis, namely contextual domains, outcomes, and the pathways linking them. In light of space constraints, we focussed our presentation of the results on the key themes and the cross-case comparative matrix; an in-depth description of each individual case is provided in Additional file 1. We further refrained in this paper from linking quantitative outcome measures to the qualitative themes, considering the breadth of the latter (in statistical terms, we would have fewer observations than variables to specify the exact relationship).

The first contextual influence involved HCWs’ perceptions and fears of potentially life-threatening infections and the extent to which they owned and managed the associated treatment risks (e.g. through referral). A doctor in Hanoi suggested for instance that antibiotic prescription decisions related partly to individual risks that the clinical users (i.e. HCWs) of CRP POCT faced in their treatment:

“If we prescribe antibiotics, we would not be blamed for any problem the patients might have. If we don’t prescribe antibiotics, the patients might get worse. In this case, we would not be able to explain to their relatives. And they would not accept our explanation.” (Doctor, Hanoi, FGD)

Perceptions of risks may thereby correspond only partially to the actual epidemiological environment. For example, a survey in Vietnam by Minh [60] detected very low rates of pneumonia of 1.2% among 563 outpatients with acute respiratory infections visiting a paediatric tertiary referral hospital [60]. Yet, doctors in Hanoi repeatedly expressed the need to protect themselves from potentially under-treating such infectious diseases and “co-infections of both viral and bacterial infections” by over-prescribing (Doctor, Hanoi, SSI).

We observed similar tendencies in the Yangon clinics, which catered especially to patients with tuberculosis (TB) and patients with HIV and which were situated among squatter populations with poor hygiene and environmental conditions [61, 62]. The participating doctors would thus describe common risks of co-infection and that: “If [the CRP test result is] low and [the patient’s] condition is bad, and there is bacterial infection, what we fear most in the bacterial infection is the pneumonia. So for that we would give [antibiotics] even if the CRP is low” (Doctor, Yangon, SSI). Considering the possibility of co-infection, antibiotics were also often prescribed as prophylactic treatment especially in patients whom the doctors considered to be high-risk groups (e.g. children, malnourished patients, or those from lower socio-economic backgrounds: “some [patients] are very weak, so then I give antibiotics, but for patients with asthma or heart failure by birth, or children, I give antibiotics even if they are not weak, because they are more prone to infection” Doctor, Yangon, SSI).

In contrast to the experiences from Yangon and Hanoi, respondents in Chiang Rai would only occasionally express a need for prophylactic prescriptions, considering the different patient profiles resulting for example from higher average wealth and access to improved sanitation facilities (e.g. access to improved sanitation in Thailand was 93% in 2015, compared to 78% in Vietnam and 80% in Myanmar [63]; see Additional file 1 for details). However, when for example, pneumonia was suspected, one nurse explained that: “mostly we’d have to refer and a medical doctor will take care of it because that’s something quite serious” (Nurse, Chiang Rai, SSI). Referral mechanisms therefore mitigated the remaining risks for the nurses when patients had low CRP but suspected pneumonia, which underlined the role of the health policy environment as a further determinant of the risks that HCWs perceived when not prescribing an antibiotic.

In summary, perceived infectious disease risks undermined user adherence with CRP POCT insofar as they created a fragile balance between clinical judgment and the fear of missing a bacterial infection. Perceived infectious disease risks when refusing patients antibiotics on the basis of a negative CRP POCT could include social pressure and adverse patient outcomes. Such situations appeared more pronounced in Yangon and Hanoi: doctors more commonly articulated fears of undertreating potentially life-threatening infections and they owned treatment risks to a greater extent than the nurses in Chiang Rai.

The third and final domain of contextual factors related to the demand side of healthcare services, influencing patients’ adherence to the CRP POCT and exclusion from the intervention. Patient adherence was affected when patients challenged the authority and decisions of HCWs. This was especially pronounced in Chiang Rai, where nurses rather than doctors were involved in the clinical intervention (e.g. “they’re not doctors here [at the health centre], they’re nurses” (patient, Chiang Rai, SSI); versus “I have gone to hospital whenever I am ill. I trust in doctors” (patient, Hanoi, FGD)). In addition, in both Yangon and Chiang Rai (for which we had more comprehensive qualitative data), patients with less formal education, from lower socio-economic strata, or with ethnic minority backgrounds would appear less assertive and more compliant with HCWs’ treatment decisions, stating for instance that: “I don’t have any knowledge, so I’d take anything. I’d take whatever they advise” (patient, Chiang Rai, SSI). Healthcare workers echoed this observation and described these patients as being “easy to talk to” (Nurse, Chiang Rai, SSI) and that they “don’t understand about medicines, so they do accept the treatment we give” (Doctor, Yangon, SSI). Based on these examples, we hypothesise that patient adherence is higher in settings where the distance in power between HCWs and patients is larger.

Adherence to the CRP POCT results could be further undermined if patients’ conceptions of illness and medicine were at odds with the implicit logic of the intervention (viz. a conceptual distinction between bacterial and non-bacterial causes of illness to guide antibiotic prescription). With few exceptions, SSI and FGD respondents in Hanoi articulated a working concept of “bacteria” and “viruses” as disease-causing agents. This conception was less prevalent among our respondents in Chiang Rai and Yangon, who would often link illness to an “inflammation” of the body (Chiang Rai) or to an infection with generic “germs” (Yangon). Respondents in these two sites also had a wider range of notions of antibiotics, which would include “anti-inflammatory medicine” (Chiang Rai), “germ killers” (Chiang Rai, Yangon), or “pesticides” (Yangon), and some patients especially in Yangon did not: “quite understand what germ killers [i.e. antibiotics] are for” (patient, Yangon, SSI). As local conceptions of illness and medicine in Chiang Rai and Yangon more often contradicted the biomedical logic of the CRP POCT, the information to explain the test might have been less effective than in Hanoi. At the same time, we observed a common pattern in Chiang Rai and Yangon that patients misinterpreted and over-estimated the capabilities of the CRP POCT as a comprehensive blood test (e.g. the finger-prick test indicating: “[…] whether this disease is good or bad, or if it’s very serious or not. And we get to know what disease it is […]” (patient, Yangon, SSI)). Ironically, this discrepancy appeared to increase rather than undermine patient adherence (see [47, 73] for more discussion on this point).

Local conceptions of illness and medicine also affected exclusion from the CRP POCT: first, local approaches to self-treatment with antibiotics were common in all three sites, and they could potentially involve strategies as elaborate as described by a patient in Hanoi:

“Sometimes I give [my daughter] ampi [ampicillin], small capsule. After replacing it by cefexim, I found [the treatment] better. Since then, I often treat her with cefixim at home, normally for 3-5 days. If she doesn’t have fever, I will treat her at home or buy medicines from [the] drug store.” (Patient, Hanoi, SSI)

While self-treatment with antibiotics was shaped partly by local conceptions of illnesses and their corresponding remedies, it was also an expression of barriers to accessing healthcare: “If [the patients] are really a hill tribe member, I don’t see them participate [in the trial], I don’t think. Because it’s hard for them to come down [from the mountain], something like that. It’s hard, it’s inconvenient” (Nurse, Chiang Rai, SSI)

Second, a mismatch emerged in Chiang Rai and Yangon between patients’ expectations about antibiotics and the focal condition of the test: neither patients nor HCWs would commonly demand antibiotic treatment for a fever, unless accompanied by other symptoms (see [71] for an analysis of administrative primary-care-level data from Chiang Rai):

“Anti... anti-inflammatory [i.e. antibiotic]; if they have a fever only—fever or cold—I wouldn’t prescribe [an antibiotic]” (Nurse, Chiang Rai, SSI).

Question (Q): “Right. And when you have a fever, do you normally take anti-inflammatory [i.e. antibiotic]?”

Response (R): “For just fever, no, only Para.”

Q: “There has to be a sore throat.”

R: “Yes, if there’s an irritation, I’d take it right away.” (patient, Chiang Rai, SSI).

“Here they don’t ask for germ killers [i.e. antibiotics]. Because people that come here don’t have much knowledge, they might not even know that what they are taking are germ killers.” (Doctor, Yangon, SSI).

“I don’t take medicine [for a fever]. I usually have a sponge bath, if I have doubts [that I have fever], I take a sponge bath. I don’t usually take medicine.” (patient, Yangon, SSI).

Owing to the incongruency between fever and antibiotic demand, a doctor in Yangon reflected that: “I don’t think that it [i.e. CRP POCT] can change much the amount of antibiotics [on the clinic level] based on whether or not to give antibiotics to those 5 or 10 people [out of 200 patients/day]” (Doctor, Yangon, SSI).

In summary, a smaller distance in power between HCWs and patients and discrepancies between the intervention logic and the local conceptions of the target population appeared to undermine patients’ adherence to the CRP POCT results. In addition, incongruencies between local forms of antibiotic use and the disease/healthcare provider focus of the CRP POCT intervention could diminish the potential overall impact at the population level.

The objective of this paper was to contribute to the understanding of how contextual factors influence the implementation and operation of medical interventions. We compared three case studies of CRP POCT trials involving qualitative research with 130 healthcare workers and patients across Yangon (Myanmar), Chiang Rai (Thailand), and Hanoi (Vietnam). The qualitative cross-case comparison demonstrated how perceived infectious disease risks, health system factors, and the demand-side context systematically influenced clinical trial outcomes (adherence of HCWs and patients to the test results, and target group exclusion from the clinical trial). From a HCW perspective, less pronounced fears of undertreating infectious diseases by withholding antibiotics, stricter prescription monitoring, and the promotion of alternatives to antibiotic treatment appeared to reinforce adherence to the CRP POCT in Chiang Rai. The opposite was the case in Yangon and Hanoi, where absent or unenforced AMR policies appeared to undermine compliance. Patient adherence to CRP-POCT-based treatment was affected positively in Yangon and Chiang Rai, where patients tended to interpret the CRP POCT as a comprehensive blood test and therefore had a higher degree of trust in the intervention. A final example was the disease focus of the trial, which did not correspond closely with expectations about antibiotic treatment among doctors and patients in Chiang Rai and Yangon. This mismatch may have entailed a relatively higher degree of exclusion of antibiotic users among the target population compared to Hanoi.

The documented variation in contextual factors demonstrated how similar clinical trials operated differently across countries and different parts of their target populations. The operational variations could influence the interpretation and generalisability of trial findings. For example, a CRP POCT trial implemented without a complementary policy environment or out of sync with local expectations of antibiotic use may yield less significant findings than otherwise, which could hinder the pursuit of further research in single-site trials unless the source of the contextual impact is clear. Likewise, trial results could appear positive yet emerge as unsustainable in routine practice if healthcare workers reverted to their accustomed behaviours during the workload-intensive monsoon season.

While these implications were specific to the CRP POCT trials, we could also distil more general lessons for AMR-related interventions and for other clinical trials more broadly. For AMR-related interventions, health system factors appeared to be of fundamental importance and echoed related social sciences research on malaria rapid diagnostic testing in low-income and middle-income countries. The literature has highlighted such factors as the drug supply environment, healthcare workers’ prior experience with diagnostic testing, or the availability of alternative treatment options in case of negative test results [14, 74, 75]. If AMR interventions failed to appreciate the local context (e.g. the nature of the drug procurement and monitoring systems, existing informal practices of healthcare staff), then they risk duplicating other solutions, competing with existing practices, or producing unintended consequences that potentially undermine the purpose or sustainability of the intervention. The parallels between our study and the literature on malaria therefore underscore that interventions to manage and reduce antibiotic prescriptions need to respond and adapt to the local health system context [76].

The same logic holds for the demand-side context of AMR-related interventions. We highlighted the role of local conceptions and how the resulting interpretations of the CRP POCT affected patient adherence with the intervention. The theme of language and the translation of scientific into popular knowledge is not new and had been raised as an issue in the field of AMR-related behaviour change communication as well [77, 78]. Language and popular conceptions of illness thereby emerged as an important pointer for contradictions between implicit assumptions of the AMR intervention and local realities, which our comparative case study confirmed.

These points may apply to other types of clinical trials, although we cannot sustain this hypothesis without further research. Nevertheless, our work did relate to fundamental contextual impact on clinical trials in diverse settings. One problem is the potential disjunction between interventions’ assumptions and local realities. This was for instance reported in the case of directly observed therapy (DOT) as the World Health Organization’s recommended strategy for TB treatment [79, 80], which faced numerous ethical and logistical challenges like insufficient healthcare resources and insensibility to the socio-economic constraints of patients [81‐83]. Our case study was a further example of such tension between internationally recommended guidelines for disease management and local health systems - manifest in the often unrealistic delayed strategy of antibiotic prescription for uncertain diagnoses [84].

Another element of common relevance for clinical trials was target group heterogeneity. Our cross-case comparison illustrated how socio-economically disadvantaged parts of the relevant target population behaved systematically differently from majority groups and struggled with participation in the trial. Such situations may not be uncommon in other low-income and middle-income contexts, where poverty often renders health expenditure catastrophic [85]. Our case study therefore related to methodological arguments in the clinical trials literature according to which the treatment effect of a trial may be shaped by population heterogeneity and selection biases, and average effects may not correspond to effects of the intervention on sub-populations [86‐88]. We argue that the characteristics and behaviours of the target group are important demand-side factors that require complementary qualitative evidence to help design and interpret clinical trials within the local context.

The main limitations of our research related to the slightly different implementation of the clinical trials across the three countries, to the varying depth of the qualitative data across our field sites, and to the inductive thematic analysis of our case studies. First, the trial implementation differed for example in terms of the staff involved. Nurses worked at the primary care units and dedicated study nurses supported the clinical trial in Chiang Rai, while doctors and dedicated study doctors participated in the trials in Yangon and Hanoi. In Chiang Rai, the study nurses voluntarily fulfilled other functions that helped ease the workload at the health centres. This unintended trial design effect (which we did not observe in the other sites) may have increased nurses’ compliance with the trial during peak times, ceteris paribus. The trial specifications therefore evidently contributed to part of the variation across our case studies. While this limited the comparability of the trials to a certain degree, it was also partly a result of necessary local adaptation, it highlighted the interactions between trial specifications and context [10], and the broadly comparable interventions enabled us to isolate contextual factors nonetheless.

Second, our themes potentially over-emphasised the cases of Chiang Rai and Yangon, where more extensive qualitative data were collected. However, all data sets involved oral accounts from a wide array of healthcare workers and patients, and all data were analysed by local native speakers with knowledge of the CRP POCT trials to provide as much depth to the interpretation as possible. Because the emerging themes applied across all three case studies, we were confident that our analysis uncovered relevant contextual factors.

Third, inductive thematic analysis was chosen to identify unforeseen and locally specific factors on the basis of our informants’ narrative accounts. This meant that we were unlikely to produce an exhaustive list of all contextual factors at work in the three clinical trial case studies - especially if our respondents did not allude to them directly or indirectly, or if they applied equally across Myanmar, Thailand, and Vietnam. For example, while regional and national trade policy regimes may have influenced trial operations through the demand and supply of antibiotics [89, 90], they did not emerge as a theme in the narratives of patients and healthcare staff. Further mixed-method research is therefore necessary to establish a geographically and conceptually comprehensive knowledge base of contextual factors affecting clinical trial outcomes.