Implementation of a novel malaria management strategy based on self-testing and self-treatment in remote areas in the Amazon (Malakit): confronting a-priori assumptions with reality

Malakit is a research project involving three countries. The sponsor of the research project was the Hospital of Cayenne (French Guiana) which also had a role in the implementation of the intervention. In Suriname, the National Malaria Program and the Foundation for the Advancement of Scientific Research in Suriname (SWOS) were responsible for the investigation and implementation. In Brazil, the institution involved in investigation was Foundation Oswaldo Cruz (Fiocruz) and the non-government organization (NGO), DPAC fronteira was in charge of the implementation [28]. The context, the content of the intervention, the players, and the steps of the project development phase have been described in previous articles [26, 28, 34]. Figures 1 and 2 describe the logic model of the Malakit intervention and its principle. The study population included individuals over the age of 18 and individuals aged 15–17 with parental consent, who go to French Guiana’s illegal gold mining sites to work, or accompany someone who works there: miner, machine owner, cook, housekeeper, canoe operator, driver, hawkers or shopkeepers/vendors, sex workers, etc., whether their activity is itinerant or fixed.

Properly defining the type of research study carried out is useful, since clarity can help avoid duplications, funding inefficiencies, and difficulties in seeking and understanding information encountered by the end-users of research evidence [37]. The Malakit project, by developing an unprecedented approach to malaria management, can be classified as intervention research. A before-after study design was developed using measurable, realistic, and comparable variables. The main evaluation criterion was based on proportion of gold miners who declared good diagnostic and treatment practices, measured cross-sectional surveys before and after the intervention [26, 28]. To complete these indicators, continuous and longitudinal data collection was implemented to assess the correct use of the kit [34]. Between April 2018 and March 2020, 4,766 kits were distributed to 3,733 participants. Six hundred and thirty one of them returned to a distribution site to answer questions about their experience during follow-up visits, among whom 223 used at least one malakit [38]. The main outcomes were analyzed and published independently of the Malakit implementation evaluation [4, 38].

The boundaries between intervention and implementation research are not always clear and may closely overlap [37]. Indeed, Malakit could also be considered as an implementation of a test-and treat strategy relying on rapid diagnostic tests (RDT) and artemisinin-based combination therapy (ACTs). In support of this, several implementation research outcomes such as coverage or acceptability were also included in the evaluation from the outset. Therefore, the term of type 2 effectiveness-implementation hybrid trial can be applied to the Malakit study [39‐41]. No process evaluation plan was elaborated before the launch of the intervention but the need to report on what was delivered and how, as well as on barriers and levers, became evident during the roll-out of the intervention, in order to complement the effectiveness evaluation outcomes and thus improve validity and inform on applicability and transferability.

Quantitative and qualitative data were collected from the studies performed as part of the Malakit project, i.e. pre- and post-intervention surveys, Malakit intervention study, an independent qualitative study, and as part of a medical outreach mission carried out alongside the French army at a remote gold mining site known as Repentir (see Table 1). The qualitative study aimed at exploring: 1) the opinion, perception and responsiveness of participants, facilitators, as well as key actors who are community members not eligible for the intervention, 2) levers and barriers to the use of the “malakit”, 3) the role of the facilitators, 4) contextual elements [42]. Other sources of information were also used to complete the overall picture (see Table 1). Supervisory visits were carried out in the field by project implementers, among whom members of the sponsor team, to observe first and follow-up visits and to hold discussions with facilitators [28]. Seventeen supervisory visits were carried out in Suriname and 15 in Brazil. The total duration of the interventional research was 24 months in Suriname and 18 months in Brazil, between April 2018 and March 2020. In both countries, the first supervisory visit took place within one month after the project launch and the final visit took place one month before the end of the study, i.e. just when it became impossible to travel due to border closings in the context of the Covid-19 pandemic. In Brazil, supervisory observations were conducted on 21 first visits and 10 follow-up visits. In Suriname, 15 first visits and four follow-up visits were reported, although more were actually observed.

Finally, information were extracted from more informal sources i.e. implementer debriefings conducted throughout the project and video self-recorded by facilitators based on a list of questions (Additional file 2).

A modified version of the Conceptual Framework for Implementation Fidelity was chosen retrospectively to present all these outcomes [43]. Therefore, implementation outcomes (“Adherence”) were separated from process outcomes (“Moderators”). Adherence was subcategorized into “Content” and “Dose/Exposure”. However, strictly speaking, adherence with regard to frequency was not assessed, as there was no determined target value regarding the number of training sessions, visits, or kits distributed, due to the lack of knowledge on population size and flows at this time. The moderators presented are “Participant Responsiveness”, which concerned both participants and intervention deliverers as described by Hasson, 2010, “Quality of Delivery”, and “Facilitation Strategies”, but “Intervention Complexity” was not assessed [44]. One element was added, based on a modified model used by Hasson in 2010 to systematically evaluate the implementation fidelity of complex interventions in health and social care, i.e. “Context” [44]. Factors related to the research setting were integrated in this last aspect. The research questions and the sources of the answers are summarized in Table 2.

Ethical clearance has been described previously [28, 38]. They were obtained from National Ethics Committee from the countries where the project was implemented, in Brazil—Approval from the Fiocruz Ethics Committee (Opinion Number 2.831.534)—and in Suriname: Approval from the CMWO (Commissie voor Mensgebonden Wetenschappelijk Onderzoek) (Opinion Number VG 25–17)—for the Malakit study, and in Brazil—Approval from the Fiocruz Ethics Committee (Opinion Number 2.560.415)—and in Suriname—Approval from the CMWO (Opinion Number DVG-738)-, for the post-intervention survey.

Adherence is defined in implementation science as the extent to which “a program service or intervention is being delivered as it was designed or written” [45].

Moderators are factors or mediators which can influence the degree of implementation fidelity [43].

Based on the observation data, it is possible to assert that the content of the messages and the way they were transmitted complied satisfactorily with what was planned, even if rectifications were necessary at the beginning.

Dane and Schneider suggest [49] that lack of confidence or experience, as well as not being professional – i.e. being a paraprofessional or lay person –, are predictors of poor program integrity in preventive intervention research. Conversely, in a study assessing the fidelity of implementation of malaria care for children by community health workers (CHWs) in Nigeria, adherence to the diagnostic, treatment, and counseling protocol by CHWs was found to be equal or higher to that of the medical staff who served as gold-standard comparators, and was not related to age, level of education, or primary occupation. In the Malakit intervention, previous experience in health care or health mediation did not seem to be an asset – also since, compared to other interventions, no clinical evaluation was performed –, and overconfidence was actually a barrier to compliance with what was planned, to the point where facilitators had to be replaced (see Additional file 3, “Terms and conditions to be included”). This is consistent with WHO’s finding that CHWs can be men or women, young or old, literate or illiterate, as long as they blend into the culture of the community and ensure its acceptance [46].

Ongoing training is a recommended practice, but can be linked to dissatisfaction when format, frequency, quality, etc. are judged inappropriate or insufficient by CHW [50]. Facilitators did not express such discontent, despite many opportunities to do so. The use of mobile technologies in particular was quite well accepted [34]. In low and middle income countries, they are increasingly seen as an opportunity to better train and improve worker performance remotely [51, 52]. This approach, also known as “Mobile Learning for Development”, is the subject of recent studies that concluded that there is a need for further research to better assess and adapt approaches [53, 54]. In Kenya, in a very similar manner to the Malakit intervention, an intervention included a WhatsApp group to strengthen “supervision, professional development and team building”, and also found that quality assurance, information sharing, and the creation of a supportive environment were useful [55]. More broadly, social interaction and peer assessment have been found to be associated with better guideline implementation and clinical practice change [56, 57]. In the present project, the peer-to-peer form of supervision within the WhatsApp group was not observed. Facilitators in the two countries knew each other slightly or not at all, due to the limited number of joint training sessions or meetings (all of them needed a visa to enter French Guiana). That is why they may not have felt comfortable enough to ask questions and share difficulties, and tended instead to share successes. In-person peer supervision, which at one point was considered, can be a way to further foster performance, but could not be implemented.

The geographical distance issues identified here as a main constraint to implementation and monitoring may be encountered in other contexts involving several countries and should be addressed. In addition to instant messaging debriefings, field supervision and refresher training, which are very time-consuming when two days are needed to reach a site, should be assigned to someone dedicated solely to those tasks. This person should actively collaborate in designing and developing training contents and data collection tools with the principal investigators. The development of refresher training tools for facilitators using a participatory approach – as used for participant training tools –, in order to adapt content to their literacy and needs, could also alleviate distance issues.

Constant and long-term efforts to maintain quality are essential to adapt to evolving contexts, including beyond scale-up. Indeed, while resources allocated for research can be sufficient to ensure integrity, for example through continuous in-person and remote supervision, decrease in fidelity is more likely when interventions are adopted and sustained [49]. Further qualitative research is planned during the sustainability phase in Suriname to add and or improve communication tools and ways of delivering messages to enhance quality of delivery.

Although coverage was acceptable after two years of intervention, it could have been improved by better allocating funds and resources to adjust to the gold miners’ mobility in a timely manner, especially given the increasing heterogeneity of malaria transmission among gold mining regions. While the penetration of the intervention was very good in a remote gold mining region where traveling to reach a distribution site is costly and time consuming, lack of access remains a barrier to better coverage, as mentioned in the qualitative study and the quantitative results of the post-intervention survey [38, 42].

The excellent appropriateness – defined as “perceived fit of the innovation to address a particular problem” already observed during the feasibility study as well as during participatory development of communication tools, good “adoption” or “uptake” by the study population and finally great acceptability definitely boosted reach [58]. Adjusting the length of training could be a mediator to increase acceptability, especially at certain distribution sites where time is a limiting factor. Furthermore, the findings underline the importance of factors that contribute to the population’s trust in the project, especially with wary communities. Research requirements in particular can negatively impact the community’s perception of the project, which should not be underestimated. Although the strategy was still at the experimental stage, articulation with existing care services should have been further developed to avoid competition being felt instead of complementarity. Finally, diminution of malaria prevalence may imply decrease of participant responsiveness more or less depending on the place of distribution [58]. Maintaining community uptake could be achieved by expanding services offered by facilitators, as seen in Myanmar, where the management of non-malaria febrile illnesses and the referral of severely ill patients complemented “malaria only” CHW prerogatives [59].

While the main defect of the present assessment is the absence of quantitative indicators for content adherence, the main strength is the regularity of supervisory visits throughout the project and not only during specific periods. The distinction between the “core components” and adaptable elements of an intervention can only be discerned through practice and mispractice over time as the intervention is more widely deployed and replicated in other contexts, as explained in the Consolidated Framework of Implementation Research [60]. Since Malakit was an innovative strategy, several choices and adaptations were made during the implementation itself. Thus, the objective of this article was not to create fidelity measures to assess an evidence-based intervention, but to capitalize on this unprecedented field experiment to contribute to future process evaluation or implementation research on the strategy. This is why no proper observation grids were designed to assess adherence to content or quality of delivery and why program differentiation, which is apart from fidelity, was not performed [43].

A workshop bringing together all facilitators and supervisors and led by an external assessor was planned in April 2020 but was cancelled due to the COVID-19 pandemic. The self-recorded videos requested of facilitators to replace their presence at the final meeting of the project were another way to give them a voice. Although this format was not anonymous, which may have inhibited the free expression of opinions, the videos made it possible to confirm or complete information on moderating factors, such as the influence of the length of the training on participation.

One limitation of the post-intervention survey was an over-representation of people frequently traveling to resting sites linked to an overestimation of the coverage. Lack of knowledge on the study population size and flows also made it difficult to assess the estimation of coverage based on distribution figures. The findings of the medical outreach mission in a hard-to-reach gold mining site provided some information on penetration despite the small size of the sample. In both of these quantitative data collections, biases were also over-representation of health-conscious individuals and expected response bias with over-reporting of positive opinion on the project. The qualitative study carried out by an external assessor allowed for increased freedom of expression and to some extent made it possible to alleviate this last bias [38].

Although it is not independent and external, feedback from players who were engaged in the protocol (intervention design and evaluation) and tools development, training, and close field supervision, can constitute in-depth information to complete qualitative and quantitative data collection.