Background
A number of proven standard interventions for controlling priority diseases in low- and middle-income countries (LMICs) exist, for example the large-scale distribution of insecticide-treated bed nets and ambulatory treatment of tuberculosis (TB) patients. At the same time, the effective implementation of these interventions requires adaptation to specific contexts, highlighting the importance of operational and implementation research (OR/IR) [
1‐
4]. Indeed, an important aim of OR/IR is to provide an evidence base for context-specific implementation of globally proven interventions and to identify issues that prevent programmes from operating effectively, thereby providing evidence-based solutions for improving programme performance [
5‐
8]. OR/IR intends to generate locally relevant answers and solutions that can be used by a specific programme but evidence may also be relevant far beyond [
9,
10]. It is well documented that policy-making and decision-taking are not always evidence-based although local evidence is arguably the most compelling [
11,
12]. It is the understanding of many global health stakeholders, including the Special Programme for Research and Training in Tropical Diseases (TDR), that OR/IR should be an integral part of disease control programme activities to maximize their outcome and impact [
13‐
15].
The present article analyses the role that OR/IR plays for optimising programme performance of international health assistance through the example of the Global Fund to Fight AIDS, Tuberculosis and Malaria (“Global Fund”). Of note, the Global Fund is the single largest external funder of control programmes targeting TB and malaria in LMICs [
16]. In 2008, the Global Fund – together with TDR – started to promote the inclusion of OR/IR activities in disease control programmes it supports. These efforts were linked to the development of a series of relevant guidelines, frameworks and toolkits [
17‐
19]. The active promotion of OR/IR was subsequently reduced, and the extent to which countries currently request Global Fund support for OR/IR remains unclear. In 2011, the Global Fund introduced the “New Funding Model” which, among others, provides guidelines on the proven priority interventions eligible for Global Fund support. While the most recent version of these guidelines provides a detailed description of the scope of eligible interventions, hardly any reference is made to OR/IR and the role research may play for dealing with context-specific implementation problems [
20]. More generally, the mechanisms determining to what degree countries take advantage of OR/IR funding are not well documented, let alone understood [
1].
We conducted a situation analysis with the goal to deepen the understanding of the extent to which OR/IR activities are included in Global Fund grants and the determinants resulting in the observed patterns. Emphasis was placed on malaria and TB, while HIV was excluded. High disease burden countries in Africa and Asia receiving significant and sustained support by the Global Fund were purposefully selected to represent a range of conditions. The specific objectives of the situation analysis were: (1) to assess the extent to which Global Fund grants are being utilized to support OR/IR projects and activities; (2) to understand the context, funding sources, capacities, challenges and considerations regarding OR/IR within the selected countries; (3) to explore the priority that different stakeholders in-country and at the Global Fund give to OR/IR; and (4) to identify gaps, priorities and potentials that can be activated in the short- and medium-term.
Methods
Study approach
The situation analysis was conducted in the second half of 2015. A mixed methods approach was chosen, consisting of a review of key documents as well as interviews with a broad range of stakeholders. The data collection included five main activities: (i) review of publicly available documents pertaining to OR/IR and the Global Fund; (ii) review of publicly available (deadline: September 2015) grant proposals for the six selected countries; (iii) analysis of grant budgets made available to the research team by the Global Fund; (iv) interviews with representatives from the Global Fund (the fund portfolio manager (FPM) and/or respective country teams); and (v) interviews with in-country key informants from different stakeholder groups.
Country selection
The selection of the study countries was based on the following criteria: (i) high malaria and TB burden; (ii) representation of different geographic regions; and (iii) Swiss Tropical and Public Health Institute (Swiss TPH; Basel, Switzerland) not acting as a Local Fund Agent in the country. Three countries in Africa (Democratic Republic of the Congo, Ethiopia and Zimbabwe) and three in Asia (India, Indonesia and Myanmar) were selected.
Document review
The document review consisted of three sequential steps. First, a search on the Global Fund website was conducted to identify key documents on OR/IR and support of such activities through Global Fund grants. Second, all publicly available Global Fund proposals from Round 1 up until and including the New Funding Model that had been submitted by the six study countries were obtained and screened for evidence of OR/IR using the keywords ‘research’, ‘study’ and ‘studies’. Initially, the identified paragraphs were differentiated between ‘OR/IR’ or ‘other research’ based on the context. OR/IR-related paragraphs were classified into three categories: (1) epidemiological, behavioural and knowledge, attitude and practices (KAP) studies, and household and health facility surveys; (2) capacity building and training in OR/IR; and (3) support for the establishment of national research agendas and OR/IR review committees. Third, the available budgets were analysed with regards to the proportions spent on monitoring and evaluation (M&E) and specifically OR/IR.
Stakeholder interviews
Interviews with key stakeholders were conducted by the authors at the Global Fund secretariat (one per country) and during visits in all study countries. At the Global Fund, the FPM and/or other country team members participated in the interview, either face-to-face or by phone. A semi-structured questionnaire guided the interview which addressed the following topics: (i) attitudes towards OR/IR and related experiences at the Global Fund; (ii) strategic perspectives on OR/IR, including funding tendencies and outlook on its relevance; (iii) barriers and challenges for successful application for, and implementing of, OR/IR; and (iv) any complementary information that might be of relevance. Answers were categorized according to pre-determined terms (for example yes/no; stakeholder type) or as suggested by interviewees (for example perceived OR/IR capacity), with explanations and answers to open questions captured in writing during the interviews. Once completed, the key points and messages of the interview were summarised and shared with the respondents for validation.
In-country, interviews were conducted face-to-face or by telephone, as appropriate. Respondents represented the following stakeholder groups: country coordinating mechanism (CCM), principal recipients and sub-recipients of Global Fund grants, national disease control programme managers, M&E and research coordinators, other project implementation partners, academic and governmental research institutions and international organizations with a stake in global health (Table
1). The following topics were addressed by the semi-structured interview guide: (i) national OR/IR policies and available funding; (ii) organisations/partners involved in OR/IR related to Global Fund-supported projects; (iii) capacities and challenges for OR/IR; (iv) dissemination of results of OR/IR and knowledge management; (v) missed opportunities and promising topics for OR/IR; and (vi) tendencies and suggestions for future OR/IR. The questionnaires are available as Additional files
1 and
2.
Table 1
Number of stakeholder interviews on OR/IR within Global Fund grants, stratified by respondent type and study country
Democratic Republic of the Congo | 2 | 7 | 6 | 2 | 17 |
Ethiopia | 1 | 3 | 4 | 7 | 15 |
India | 1 | 6 | 2 | 0 | 9 |
Indonesia | 1 | 7 | 3 | 5 | 16 |
Myanmar | 0 | 5 | 1 | 1 | 7 |
Zimbabwe | 3 | 4 | 5 | 6 | 18 |
Total | 8 | 32 | 21 | 21 | 82 |
The interviews were analysed by reviewing the answers to specific topics across respondents with a focus on the detection of patterns that could be linked to respondent characteristics (for example function, represented organization type and country).
Discussion
The Global Fund provides detailed guidance on eligible interventions under the New Funding Model, emphasises the essential role of M&E systems but does not specifically refer to the role that OR/IR may play in identifying context-specific solutions to implementation challenges. The Global Fund is a central funder with ramifications for the strengthening of health information systems and implementation of large-scale population-representative surveys pertaining to malaria, TB and HIV/AIDS [
24,
25]. Basic and general research as well as individual career development grants, on the other hand, cannot be supported by the Global Fund. However, the Global Fund is committed to fund OR/IR studies that directly address operational questions and aim at identifying practical solutions to implementation bottle-necks with a view to optimize programme performance and outcomes. Exceptionally, it also supports thematic grants, which may include substantial research agendas such as the RAI that aims at interrupting the spread of artemisinin resistance in the Greater Mekong sub-region through a high coverage with long-lasting insecticidal nets, improved diagnosis and treatment, as well as strengthened surveillance.
A key finding of the current situation analysis is that there are large variations in the demand and absorption capacity for OR/IR and consequently also in available funding across the countries and programmes reviewed here. The following determinants for OR/IR related to national malaria and TB control programmes were identified: availability of a national or disease-specific research strategy, particularly one that emphasises the value of OR/IR, in-country human and technical research capacity, access to local and international technical assistance, coordination of research activities by a designated body, and involvement of all relevant stakeholders including research institution representatives in the concept note development. Of note, research needs as well as resource availability are often not well mapped out, and priorities may be poorly aligned. Similarly, a challenge appears to be the process of results dissemination, both in-country to policy makers and across countries. Last, no central database of OR/IR studies supported by the Global Fund exists, and hence, it is difficult for country teams to locate examples for proposed studies, take previous results into account, and learn from prior experience. The country teams of the Global Fund are not actively involved in concept note development as this is a country-driven process. However, interviewed in-country stakeholders expressed a wish for feedback and guidance regarding OR/IR by the Global Fund. It is conceivable that the findings from a specific country can be extended to other diseases within that country but no conclusions regarding the situation in countries not covered in this study can be made, as illustrated by the considerable variation between study countries.
Reportedly, proposing concrete OR/IR studies at the time of concept note development is often not feasible since operational challenges typically only become apparent during project implementation. While this argument is relevant for disease control programmes that are implemented over an extended period by the same in-country organisations, it might point towards an insufficient documentation of implementation challenges over the years, or inadequate involvement of relevant stakeholders in the concept note development. The results and outcome-driven funding model operated by the Global Fund means that OR/IR studies should directly contribute to improved programme performance under the same grant to be fully justified. This demand is not fully compatible with traditional research approaches under which study results are first published in academic circles and expected to be reviewed in the light of other studies to ultimately impact policy while direct feedback to relevant stakeholders is less well established.
The present situation analysis revealed that OR/IR was better integrated into grants and more prominently represented in countries with (a) an established research coordination body [
26‐
30]; (b) a strong autochthonous capacity to design and implement research projects; (c) an inclusive approach to concept note development; and (d) a country team at the Global Fund secretariat with an active interest in integrating OR/IR studies into grants. Importantly, countries appear to often make a rational choice when confronted with the question of whether or not to include requests for support for OR/IR studies in Global Fund grant applications. Important considerations in the process are budget restrictions and the availability of alternative funding sources for OR/IR: the needs of the programme and the chances or approval are carefully assessed to maximise overall support, and as a result countries may opt to obtain a maximum of commodities and operational support through Global Fund grants, while funding for OR/IR is obtained from other sources with different funding priorities.
Our situation analysis has a number of shortcomings that are offered for consideration. First, a comparable representation of different stakeholders was sought for all countries and country teams at the Global Fund secretariat but the total number of interviews per country and the profiles of the interviewees varied slightly depending on the availability of potential respondents and the situation in-country. Second, as in all studies primarily relying on self-reported information, a certain bias due to the background and personal interest of the respondents and their awareness of the study objectives cannot be excluded. Third, the availability and detail of grant proposals and budgets varied, and their screening with a unified set of keywords may have resulted in the erroneous exclusion of relevant items if a different wording had been used. Finally, no conclusions can be drawn regarding HIV-related grants since these were excluded from the study. However, there is no indication that the Global Fund systematically applies a different OR/IR policy to HIV as opposed to malaria and TB.
Conclusions
Demand and significance of OR/IR vary from one country to another, mainly determined by in-country capacities to coordinate and implement studies and perceived needs. In-country stakeholders express a desire for more specific guidance pertaining to OR/IR from the Global Fund and more flexible administrative procedures. Global Fund representatives generally limit themselves to react to demands from countries and focus on evidence and outcomes as main measures for success. The results of this situation analysis informed a consultation organised by TDR in December 2015 [
31] that included Global Fund representatives and other interested stakeholders. Based on the discussions during the stakeholder consultation, a set of recommendations aimed at promoting OR/IR with the overarching goal to improve programme performance was developed (Table
5). In December 2016, the Global Fund published an Information Note that explicitly mentions that operations research can be included in applications to strengthen country health information systems [
32].
ᅟ | ᅟ |
Global Fund secretariat |
• Global Fund to provide specific guidance on inclusion of OR/IR in concept notes and grant budgets. |
• Global Fund to ensure necessary flexibility to fund small-scale OR/IR studies identified only after grant signing, e.g. through flexible OR/IR allocation within the M&E budget. |
• (Re-)establishment of an inventory of OR/IR studies supported by the Global Fund. |
Technical partners |
• Technical partners including WHO country and regional offices to promote and actively support the inclusion of OR/IR in country health strategies, strategic development plans, guidance documents and disease-specific agendas to increase awareness and to align Global Fund concept notes including OR/IR studies with all relevant guidance. |
Countries |
• Countries to increase awareness of the importance of OR/IR within national disease control efforts and inclusion of OR/IR in concept note development. Hence, academic and research stakeholders should be closely involved in the elaboration of concept notes. |
• Countries to strengthen capacities at all levels to coordinate research, develop research agendas as well as to plan, conduct, oversee and disseminate OR/IR. Improve communication strategies to disseminate findings to relevant stakeholders to influence policy and translate findings into improved program performance. |
• Countries to improve results dissemination and uptake in-country, and – with support from funders and development partners – facilitation of results dissemination across programs and countries (e.g. regional). |
Acknowledgements
The authors acknowledge the willingness of the interviewees in the Global Fund to Fight AIDS, Tuberculosis and Malaria and in the six countries to make available their precious time for interviews, to share their thoughts and to answer specific follow-up questions.