Operations research (OR) is a discipline that uses advanced analytical methods (e.g. simulation, optimisation, decision analysis) to better understand complex systems and aid in decision-making.
Summary
Herein, we present a scoping review of the use of OR to analyse issues in global health, with an emphasis on health equity and research impact. A systematic search of five databases was designed to identify relevant published literature. A global overview of 1099 studies highlights the geographic distribution of OR and common OR methods used. From this collection of literature, a narrative description of the use of OR across four main application areas of global health – health systems and operations, clinical medicine, public health and health innovation – is also presented. The theme of health equity is then explored in detail through a subset of 44 studies. Health equity is a critical element of global health that cuts across all four application areas, and is an issue particularly amenable to analysis through OR. Finally, we present seven select cases of OR analyses that have been implemented or have influenced decision-making in global health policy or practice. Based on these cases, we identify three key drivers for success in bridging the gap between OR and global health policy, namely international collaboration with stakeholders, use of contextually appropriate data, and varied communication outlets for research findings. Such cases, however, represent a very small proportion of the literature found.
Conclusion
Poor availability of representative and quality data, and a lack of collaboration between those who develop OR models and stakeholders in the contexts where OR analyses are intended to serve, were found to be common challenges for effective OR modelling in global health.
The online version of this article (doi:10.1186/s12961-017-0187-7) contains supplementary material, which is available to authorized users.
Abkürzungen
aNAAT
automated nucleic amplification test
ARV
anti-retroviral
DES
discrete-event simulation
DSS
decision support system
EPI
Expanded Programme on Immunization
LMICs
low- and middle-income countries
OR
operations research
TB
tuberculosis.
Background
‘Global health’ broadly refers to “an area for study, research, and practice that places a priority on improving health and achieving equity in health for all people worldwide” [1]. From population-based prevention to individual-level clinical care, global health encompasses health issues and solutions that transcend borders, and involves a collaborative and interdisciplinary effort [1]. The goal of achieving equity in health, namely the absence of systematic disparities in health or in the major social determinants of health between groups with different levels of underlying social advantage/disadvantage [2], has become a particularly important part of the post-2015 development agenda [3‐6]. Globally, major progress has been made towards certain Millennium Development Goals and targets; however, many low- and middle-income countries (LMICs), especially in sub-Saharan Africa and Asia, continue to experience high health inequities, both within and between countries [3, 4]. Further, these countries carry most of the world’s burden of morbidity and mortality; for example, more than 99% of under-5 child deaths in 2010 occurred in LMICs, and although mortality rates fell in most monitored countries, 15 countries experienced increases in the absolute number of deaths, with 12 of these countries being in sub-Saharan Africa [7].
Operations (or operational) research (OR) is a discipline that uses advanced analytical methods to better understand complex systems and aid in decision-making [8, 9]. OR uses a wide range of problem-solving techniques and computational methods, including computer simulation, mathematical optimisation, statistics and decision analyses, to help improve the operations of organisations. With its orientation towards improving efficiency, cost-effectiveness and decision-making, OR is particularly useful for analysing complex global health issues – especially in settings where the burden of disease is high but health systems are weak and resources limited. Despite the growing use of OR in global health, it is unknown how much of an impact OR is having in this important area as publications rarely discuss whether their findings were implemented or were influential in policy- or decision-making [10, 11].
Anzeige
The objective of this scoping review is to examine the extent, range and nature of operations research activity in global health, specifically within healthcare settings, health services delivery, and population health in LMICs. Our goal is to highlight the breadth of healthcare applications of OR in global health, both geographically and across application areas, and – through select case studies – discuss the impact such studies can have on improving healthcare and healthcare equity for communities and populations globally. We aim to encourage OR researchers and global health practitioners alike to continue to apply OR in global health, particularly in areas where OR-based studies may currently be lacking, and to consider sharing the impact of OR work more broadly so that others can learn from challenges and successes.
It should be noted that, in the context of global health, the term ‘operations research’ is sometimes synonymous with implementation research [12] and is used broadly to encompass cross-sectional, case-control, retrospective or prospective cohort analyses [13‐15], as well as qualitative research methods [12, 16], all of which are valuable in studying the effectiveness of health services and programs within the day-to-day operating environments of routine practice. In this review, however, we focus on studies where analytical methods or modelling are used to explore health research questions with an orientation towards decision-making or the consideration of ‘what-if’ scenarios – in other words, modelling studies that are prescriptive in their recommendations.
The modelling realm of OR is of particular interest because it can help address global health questions not easily answered with other methods. For example, OR is beneficial in situations where conducting a real-world study might be considered impossible, impractical, too costly or unethical, such as when choosing between implementing policy ‘A’ or policy ‘B’, when controlled trials to compare a wide variety of available options would be unreasonable, when the disease or illness of interest takes years or decades to progress and the process of evaluating long-term outcomes would be long and expensive, or when simulating virtual cohorts of patients allows researchers to explore questions without ethical consequences. OR is also useful for framing complex financial evaluations, for example, determining the most cost-effective intervention among many options, establishing the optimal way to allocate a limited budget across multiple competing needs, or deciding whether a new intervention (e.g. a vaccine) can be implemented sustainably with limited funding. In LMICs, such OR analyses, which help narrow down the number of possible options or help inform where to focus efforts for more targeted studies, are even more important due to limited resources.
While OR in healthcare in the developed world has been extensively studied in recent years [17‐22], the latest review of OR in healthcare in developing countries was published in 1993 [10]. A few recent review papers and bibliographies have explored the use of OR in developing countries; however, these did not specifically focus on healthcare and included several other sectors such as agriculture, energy and transport [11, 23‐25]. Others have reviewed the use of OR within a very narrow area of global health (e.g. infectious diseases, particularly HIV) [26‐30]. Several survey papers and special issues of journals have recently focused on the use of OR to address global health or humanitarian issues, but these were not based on systematic reviews of the literature [16, 31‐33]. Given that the existing literature on this topic is sporadic, not comprehensive in the search strategy, and lacks depth in the analysis of thematic areas, we have chosen a broad scoping review approach. With this approach, we aim to build upon previous work by providing a systematic and comprehensive landscape overview of the use of OR in global health with a more rigorous analytic framework than has been previously performed.
Anzeige
The results of this scoping review are presented in four main sections. First, we present a global overview of the literature, which includes the distribution of OR studies across countries of different income classifications, over time, and across different methodological approaches. Then, we explore the use of OR in four global health application areas with concrete examples in each category. In this review, we consider the four main application areas of global health to be (1) health systems and operations, (2) clinical medicine, (3) public health, and (4) health innovations – from the local to global level. Next, health equity, which is integral to the concept of global health and transcends all four application areas, is explored as a separate overarching theme using a subset of included studies. Health equity is not only a topic of growing interest globally, but is compelling to explore through an OR lens. For example, when health equity is operationalised and quantified using meaningful and measurable criteria [2], OR methods can be used to analytically find solutions that improve or maximise equity. To our knowledge, the use of OR to analyse issues of health equity has not yet been explored through a systematic review. Finally, by way of selected cases, we present a discussion on implementation and impact, i.e. how OR has influenced real health policy change or aided in decision-making by stakeholders. We highlight common factors among these studies that likely helped contribute to their effective translation into policy or practice, and discuss barriers and challenges to bridging the gap between OR and health policy. We conclude the paper with a discussion of key insights and implications of this review.
Methods
We followed the scoping review framework set out by Arksey and O’Malley [34] and by others who have proposed refinements to this approach [35‐37]. Specifically, we followed these five stages:1 (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; and (5) collating, summarising and reporting results.
Stage 1: Identifying the research question
This scoping review seeks to identify the extent, range and nature of OR in global health (i.e. geographical, over time, methodological and across application areas), with an in-depth exploration of literature addressing questions of health equity and literature having made a specific impact in decision-making or policy change.
Stage 2: Identifying relevant studies
The databases HealthStar (a subset of Ovid Medline focused on health systems research), Scopus, Web of Science, Inspec and Compendex were chosen to capture literature from multidisciplinary sources across health research and engineering. Individualised search strategies were designed for each database.2 We searched titles, abstracts and keywords for combinations of search terms in the following categories: OR modelling and methodologies; healthcare settings, health services delivery, and population health; LMICs and regions (e.g. sub-Saharan Africa, South East Asia, etc.), including specific country names in these income categories; and policy- and decision-making. The search strategy including all search terms for the Web of Science database is provided in Box 1 as an illustration, and all others are provided in Additional file 1: Tables S1 to S4. Only papers published in the year 2000 and later were included. This search strategy was refined and validated by ensuring the search captured a set of 15 ideal target papers [38‐52] known to the authors. Librarians specialising in both engineering and health sciences literature were consulted when designing the search strategy. Search results were downloaded in August 2014. We also hand-searched 19 review papers and special issue articles [11, 13, 16, 17, 19, 23‐33, 53‐55] for additional references.
Box 1 Example search strategy for Web of Science database
Database: Web of Science (WOS)
Strategy: Keyword search. “Topic search” (TS) was used, which searches all words in article titles, abstracts, author keywords, and “KeyWords Plus” fields. Sub-search categories: (a) model types, (b) geographic focus, (c) health, and (d) decision-/policymaking. Terms within categories were combined with “or”, and categories were combined with “and”. All LMIC names individually listed since country names are not controlled vocabulary in WOS. Results pre-2000 and unrelated WOS categories were excluded.
Sub-search categories:
(a) Model types
TS=(“operation* research”) OR
TS=(model* NEAR/5 (mathematical or queu* or inventory or scheduling or demand or forecast* or comput* or network or stochastic or decision* or delivery or simulation or optimi?ation or non-linear or nonlinear or linear or Markov or cost-effectiveness or agent-based)) OR
TS=(optimi?ation$ NEAR/5 (mathematical or nonlinear or non-linear or linear or network or discrete or multicriteria or multi-criteria or stochastic or problem or minimi?ation or maximi?ation or location or allocation)) OR
TS=(simulation NEAR/3 (comput* or discrete or agent-based or system$))
AND
(b) Geographic focus
TS=(“developing countr*” OR “low-income countr*” OR “middle-income countr*” OR “developing world” OR “developing nation*” OR “low-resource setting*” OR “resource-constrained setting*” OR “resource-poor setting*” OR “limited-resource setting*” OR “resource-limited setting*” OR “under-developed countr*” OR “least-developed countr*” OR “less-developed countr*” OR LMIC* OR Africa* OR (Asia* NEAR/2 south) OR (Asia* NEAR/2 east) OR “latin America*” OR “central America*” OR “south America*” OR Caribbean OR “middle east”) OR [all low- and middle-income country names listed out]
AND
(c) Health
TS=(health* or medical or hospital or clinic* or treatment) OR AD=(health or hlth)
AND
(d) Decision-/policy-making
TS=(polic* or decision-mak* or decision-support or decision-process or decision-aid* or implement* or impact) OR AD=(policy)
Search results from each database were combined and duplicates were removed. An initial screen of the remaining 14,518 references eliminated studies that were clearly not relevant to this review. This initial screen was largely based on title and keywords; if additional information was required to judge relevance, the abstract was consulted. A large proportion of papers rejected at this stage fell into one of two categories, either (1) field studies and implementation research that did not have a modelling element, or (2) health-related modelling studies that were purely explanatory or descriptive in nature and did not have an orientation towards policy- or decision-making. There were 1408 abstracts remaining after this initial screening, including 31 articles from hand-searching review papers (Fig. 1).
Fig. 1
Systematic search results and screening process
×
Stage 3: Study selection
A second screen was conducted whereby a more rigorous set of inclusion criteria was applied to identify the final set of papers for the review. Two co-authors independently reviewed each abstract against the following four key inclusion criteria: (1) the study clearly used methodologies common to the field of OR; (2) the problem or research question was of an OR nature; (3) the study was related to a healthcare or public health issue; and (4) the geographic focus was on LMICs and/or regions (Box 2). For each main criterion, at least one of the sub-points had to be true in order for a paper to be included in the review. Papers for which both reviewers were in agreement were automatically included. Discrepancies were resolved through discussion (with a third reviewer if necessary), or by downloading and reading the full-text. Co-authors met periodically during this stage to discuss any uncertainties related to study selection and ensure consistency in applying the criteria. After applying the inclusion criteria, 1099 papers remained – these comprised the set of studies for our global overview.
Box 2 Inclusion criteria for second screen of review process
1) An operations research (OR) technique is used
↑
or
↓
□ Simulation model (e.g. discrete event, agent-based, etc.)
□ Optimisation model (e.g. linear and non-linear programming, goal programming, location-allocation models)
□ Decision analysis, decision tree models
□ Stochastic, compartmental, state transition models (e.g. Markov)
□ General mathematical or probabilistic models of disease progression and/or transmission will be considered if Criteria 2 is met
and
2) An OR problem is exploreda
↑
or
↓
□ Several competing interventions and/or policy options are modelled/simulated/compared to propose the best/optimal strategy (incl. comparing current status quo vs. a new option)
□ The cost-effectiveness of a treatment or intervention is explored through hypothetical cohorts of patients or through decision-analysis techniques to estimate costs per person
□ Outcomes are modelled for different treatment or therapy scenarios/options
□ Issues of logistics, supply chain, distribution, scheduling are explored; including studies that highlight operational inefficiencies or poor performance
□ ‘What if’ scenarios are tested, e.g. what if anti-retroviral therapy coverage was increased?
and
3) The OR problem has a healthcare delivery or public health focus
↑
or
↓
□ Health services delivery (hospital/clinical services, primary care, treatment or diagnostic options, health technology management or integration, etc.)
□ Public or population health (vaccination policy, mass screening for health conditions, transmission prevention and/or reduction, etc.)
and
4) The study focuses on a low- or middle-income setting
↑
or
↓
□ Country-focused: a look-up table was provided to co-authors to determine whether a country was low income, lower-middle income or upper-middle incomeb
□ Regional: Africa, sub-Saharan Africa, South East Asia, Latin America, South America, etc.
aThis list does not cover the full extent of OR-type problems, but these criteria describe the types of OR problems of interest for this reviewbAccording to World Bank classifications as of July 1, 2014. Low-income economies are defined as those with a gross national income (GNI) per capita of $1045 or less in 2013; middle-income economies are those with a GNI per capita of more than $1045 but less than $12,746; lower-middle- and upper-middle-income economies are separated at a GNI per capita of $4125
Anzeige
To identify papers among the 1099 studies in the global overview that explore the specific theme of health equity, we searched for the following keywords within titles, abstracts and author addresses: (in)equit*, (in)equalit*, pro-poor, poorest, socio-economic, marginalized, stigm*, quintile*, disparit* and gender. Two co-authors assessed each abstract and collaboratively decided if they addressed an issue aligned with the definition of health equity as described by Braveman and Gruskin [2]. Full-texts for health equity-themed papers were downloaded and read.
Identifying studies for the impact theme was less straightforward. As noted by others who have reviewed OR in global health or LMICs [10, 11, 23, 24], many OR studies are published before any evidence of having influenced policy- or decision-making has been demonstrated. Thus, it would be misleading to assess the impact of OR studies based solely on a review of published literature. We therefore took the approach of providing select case examples of studies where impact was described in the publication in order to gain insight from their experiences, with the caveat that additional OR studies have likely had an impact on improving global health. For our purposes, ‘impact’ implies that study results meaningfully informed a policy decision, or that model recommendations were implemented in a real-world situation. Full-texts for impact-themed papers were downloaded and read.
Stage 4: Charting the data
‘Charting’ is “a technique for synthesising and interpreting qualitative data by sifting, charting and sorting material according to key issues and themes” [34]. For each of the studies included, the following items of information were charted by one co-author and cross-checked by another: (1) country or region of focus; (2) income classification of that country3 – low income, lower-middle income, or upper-middle income; (3) OR methodology or type of OR model developed/used for the analysis; (4) health issue studied (HIV/AIDS, malaria, childhood pneumonia, etc.); (5) application area of global health – Health Systems and Operations, Clinical Medicine, Public Health, or Health Innovation; and (6) level at which the study was targeted – local, national, regional, global or general. These characteristics were gathered from the abstract, but in cases where this information was not clearly stated, the full text was consulted.
The Health Systems and Operations category refers to studies that looked at the logistics related to the provision of services, the allocation of resources or the operations of health facilities. Clinical Medicine was distinguished from public health in that these studies focused primarily on disease diagnosis, treatment or care for the individual patient (e.g. treatment regimens, case management, etc.), whereas Public Health studies emphasised disease prevention and health promotion at the community or population level (e.g. vaccination policy, mass screening, etc.). The Health Innovation category was reserved for studies that explored healthcare innovations or technologies in the pre-implementation stages of development (e.g. vaccines still in early clinical trials and not yet accepted for widespread use, hypothetical future discoveries – in diagnostic, treatment or vaccine technologies). The study’s target level refers to the level at which the model recommendations would be or were intended to be implemented. ‘Regional’ refers to global regions (e.g. sub-Saharan Africa, South-East Asia) and not sub-national regions. The ‘general’ category was reserved for those studies that considered ‘low-resource settings’ as the target in a very general sense or where the level of intended implementation was not clear.
Anzeige
For the most part, studies were easily categorised; however, a small fraction fell into grey areas. Where there was overlap, a determination was made based on what was deemed to be the dominant category. For example, studies that were based on a local setting but were intended to inform national policy- or decision-making were counted towards the national category because proposed changes would be made at the national level. Similarly, some studies bridged clinical and public health (e.g. screen and treat programs). We considered any study with broad public health goals, regardless of whether treatment was included, as public health.
Stage 5: Collating, summarising and reporting results
Based on Arksey and O’Malley [34], we present our findings in two ways. First, through basic numerical analysis of the extent, nature and distribution of studies across various dimensions (i.e. global overview, application areas), and second by organising a subset of the literature thematically (i.e. for the themes of health equity and impact).
Results
Global overview
In this section, we present a global overview of the 1099 studies that met the inclusion criteria, including a breakdown of OR studies according to country income classifications, geographic regions, year of publication and methodology.
Figure 2 shows the breakdown of OR studies by country income level. The majority of studies (74%) were focused on a specific low, lower-middle, or upper-middle income country; however, several studies (20%) were targeted towards LMICs broadly, and a small proportion (6%) looked specifically at a grouping of countries or regions that spanned several income categories (Fig. 2a). Among the 817 studies that had a single-country focus (Fig. 2b), low-income countries made up 17% whereas middle-income countries (lower-middle and upper-middle) made up 83%, with the majority being in the upper-middle income category. Using number of countries and population as benchmarks,4 our findings suggest that lower-middle-income countries are under-represented in the literature, upper-middle-income countries are over-represented, and the representation of low-income countries is roughly proportional to these benchmarks. Lower-middle-income countries make up about 34% of all LMICs and 44% of the LMIC population but only 18% of the literature, while upper-middle-income countries make up 40% of all LMICs and 41% of the LMIC population but 65% of the literature. For comparison, low-income countries, which make up 17% of the literature, represent approximately 26% of all LMICs globally and 15% of the LMIC population.
Fig. 2
Breakdown of operations research studies according to World Bank income classification of the country of focus – low-income (L), lower-middle-income (LM) or upper-middle-income (UM) – for (a) all studies (n = 1099) including studies about low- and middle-income countries in general or some combination of regions and/or L-, LM- and UM-income countries; and (b) studies focused on a single country only (n = 817)
×
Anzeige
Figure 3 provides a more detailed geographical view of the distribution of OR studies across the developing world. Almost 40% of the literature reviewed was focused on just six LMICs. China, Brazil and South Africa were the most frequently studied, and collectively accounted for 25.4% of the studies reviewed. India, Mexico and Thailand accounted for 14.5%; all were classified as upper-middle-income countries, except India, which was a lower-middle-income country. These countries represent just 4.4% of all LMICs, but account for about 52% of the total LMIC population. The low-income country most studied in the OR literature was Uganda, with 26 studies. More papers were focused on Asia and South America than sub-Saharan Africa (excluding South Africa). Approximately 50 LMICs were not studied in any of the global health OR publications identified; these countries account for approximately 5% of the total LMIC population, or approximately 303 million people.
Fig. 3
Number of operations research studies by country. Note that only studies that focused on a single country (n = 817) or multiple specific countries (n = 55) are represented in this figure. Studies that considered multiple countries are counted once for each country represented
×
As Fig. 4 suggests, low- and lower-middle income countries have historically been less frequent targets for global health-related OR compared to upper-middle-income countries. Despite a steady increase in the absolute number of studies focused on low-income countries since 2000, the proportion of such studies relative to all global health-related OR has plateaued at approximately 14% since the year 2006. This figure also suggests a trend towards more country-specific analyses rather than studies that consider LMICs in general or groupings of countries (see Other category in Fig. 4). A possible explanation for the drop in number of papers for 2013 is the lag between when a paper is published versus when it has been indexed in databases. The year 2014 was not included in Fig. 4 since our review does not encompass the entire year.
Fig. 4
Proportion of operations research (OR) studies per year in different country income classifications (bars, left axis); low income (L), lower-middle income (LM), upper-middle income (UM) and Other (includes studies targeted at LMICs in general or some combination of L-, LM- and UM-income countries). Total number of OR papers per year also displayed (line, right axis). Note that 2014 was excluded as this review covers studies indexed up until August 2014 only
×
A breakdown of OR studies according to methodology is shown in Fig. 5. A wide range of OR methods have been used to study global health issues, and no single method appears to be dominant. In the section that follows, examples of different methods are provided within the context of four application areas of global health.
Fig. 5
Breakdown of operations research methodologies. The Stochastic category includes Markov models (e.g. state-transition and decision process models) and Monte Carlo methods. The Mathematical category includes deterministic models, epidemiological compartmental models, and other analytical models described by governing equations. The Other category includes all remaining smaller categories including artificial neural networks, inventory models, spreadsheet models with no analytical formulation, etc. See Box 2 Criterion (1) for additional details about the methodologies included. DEA data envelopment analysis
×
Global health application areas
In this section, we explore the volume and breadth of OR literature found across two dimensions of global health; the global health application area and the level at which the analysis was targeted (Fig. 6). These application areas were chosen because we felt they were broad enough to cover the full gamut of global health challenges. At the same time, studies within categories would carry a similar flavour in the types of problems studied. Other categorisations could also have been appropriate [10, 16]. Similarly, we felt it important to distinguish between different levels of focus as the types of problems, analytical approaches, and scale of implementation would be different across these levels. Detailed examples of OR studies in the four areas of global health are described in more detail in the sub-sections that follow.
Fig. 6
Percentage of operations research studies (n = 1099) by application area of global health and by analysis target level (local, national, regional, global or general)
×
The majority (58.3%) of OR literature explores clinical medicine and public health issues at a national level, with locally-focused health systems and operations studies being the next most frequently studied area (11.0%). Since policies affecting clinical medicine and public health are typically mandated by national ministries of health or implemented by national public health programs, it makes sense that the OR studies in these areas have been targeted at the national level. Although very helpful for exploring the impact of interventions on a macro scale or adding to discussions on global priority-setting, fewer studies were targeted at the regional or global level (12.2%) or towards LMICs in general (6.8%).
Health systems and operations
About 20% of the literature was related to health systems and operations, and most of these studies were focused on the local or national level. At a local level, common analyses included improving the day-to-day operations of health facilities (e.g. patient flow and wait times in health facilities [49, 56‐60] and emergency departments [61‐64], facilities layout planning [65, 66], inventory planning [67, 68], nurse rostering [69‐75], and surgical scheduling [76‐79]) and health services planning (e.g. location-allocation of emergency medical services [80‐82] or new health facilities [83‐86]). The majority of these types of problems were analysed using simulation (25.0%) or optimisation (35.0%), or a combination of both methods (3.3%). For example, discrete-event simulation (DES) was used to analyse patient wait times in health clinics in Zambia [49] and Colombia [59], and a combination DES-optimisation model was used to study ambulance positioning and response time for an urban city in Brazil [81].
At the national level, questions related to supply chains and logistics were often explored using methods such as agent-based simulation, DES and other types of microsimulation. For example, a series of recent studies used the HERMES (Highly Extensible Resource for Modeling Event-Driven Supply Chains) software to develop DES models of the vaccine supply chain in Niger and Thailand; researchers explored the impact on vaccine availability of introducing new vaccines into the supply chain [87, 88], changing vaccine vial size or replacing multi-dose vials with single-dose vials [46, 89], removing the regional level of distribution [90], and trade-offs between augmenting transport versus increasing cold storage capacity [47, 91].
The lack of adequate resources represents a major constraint for health systems in LMICs, thus the efficiency with which available resources are being used was another common theme. A small subset of OR literature (2%) used data envelopment analysis at both a local and national level to analyse the efficiency of health facilities and systems in many low-resource settings. ‘Technical efficiency’ is typically defined as a ratio between a weighted sum of outputs (e.g. number immunisations provided, number of antenatal visits, etc.) and a weighted sum of inputs (e.g. human and financial resources, supplies, beds, etc.); a less than ideal efficiency, as indicated by an ‘envelope’, indicates that a health facility can potentially expand their outputs without changing the quantity of inputs used [92]. Data envelopment analysis studies have been set in India [93‐95], Kenya [96, 97], Sierra Leone [98], Angola [92] and Zambia [99], and helped identify inefficiencies in health services delivery, as well as opportunities to better use existing resources. Other studies exploring efficient resource allocation included a simulation model used to provide insight into better resource utilisation (e.g. personnel and physical resources) in an emergency department in Malaysia [100], and an optimisation model used to explore the optimal allocation of resources in a region in Tanzania given different health objectives (e.g. minimise number of deaths, minimise disease incidence, minimise loss of quality of life, etc.) [101].
One area of health systems and operations that was not often studied using OR was medical equipment and health technology management. In addition to two previously mentioned studies about increasing vaccine cold storage equipment [47, 91], we identified just nine OR studies related to medical equipment. Some examples include a cost-utility analysis of introducing PET scanning technology for lung cancer diagnosis in Iran [102]; a DES model of a mammography clinic in Brazil that took into account equipment failures and maintenance [103]; a queuing model developed to improve response and turn-around time of equipment repair work orders in a clinical engineering department in Cuba [104]; and models to help inform general medical equipment purchasing [105] and replacement schedules [106] in LMICs.
Clinical medicine
Most clinical medicine studies were focused at the national level. Common themes were assessing the cost-effectiveness of adopting new treatment or diagnostic strategies, comparing outcomes or cost-effectiveness of competing treatment options, and estimating the benefits of scaling-up treatment access.
Almost 45% (168) of studies in this category were related to HIV/AIDS, malaria or tuberculosis (TB). For example, STDSIM [107] – a microsimulation decision-support model – has been used in several studies to analyse the impact of expanding anti-retroviral (ARV) access [108, 109] as well as treating other curable sexually transmitted infections in order to prevent HIV infection [110‐112]. Shillcutt et al. [113] used a decision-tree model to evaluate the relative cost-effectiveness of presumptive treatment, field standard microscopy, or rapid diagnostic tests for malaria diagnosis in different sub-Saharan African settings. A combination decision analysis and Markov model was used by Mandalakas et al. [114] to compare the cost-effectiveness of different TB prevention strategies using WHO-recommended isoniazid preventive therapy for children in close contact with infectious TB cases.
Stochastic models, such as Markov models, were common methods for clinical studies, representing almost 26% of the studies in this category. Such models are useful for simulating cohorts of patients with a specific illness as they transition from one disease state to another throughout the course of an illness or even their lifetime. For example, the cost-effectiveness of different treatment options for patients with chronic hepatitis B was studied using Markov disease models in China [115, 116], Brazil [117, 118], Turkey [119] and India [120], over time horizons ranging from 20 to 40 years.
Interestingly, 53 of the 70 studies related to the diagnosis or treatment of cancer, cardiovascular disease or diabetes were published in the past 5 years (between 2009 and 2014), consistent with increased global attention on such non-communicable diseases in LMICs [121, 122]. For example, a Markov model was developed to compare the cost-utility, in terms of quality-adjusted life years, of four different treatment options for lung cancer in Thailand [123]. DES models were used to analyse the cost-effectiveness of saxagliptin as a treatment for type II diabetes in both Argentina [124] and Brazil [125]. The treatment of mental health issues is one area that has not been studied extensively with OR – we found only 13 studies in the clinical medicine category that focused on mental illnesses in LMICs such as depression and schizophrenia.
Public health
Public health, specifically at the national level, was the most common global health area explored using OR. Vaccination policies, particularly the introduction of vaccines into routine child immunisation programmes, and other disease prevention strategies such as screening programs (e.g. for cervical cancer), were among the most common types of problems explored.
An example vaccination model is the TRIVAC decision-analysis model from the Pan American Health Organization ProVac initiative, which was used to assess the cost-effectiveness of adding vaccines (e.g. pneumococcal conjugate vaccine, Hib and rotavirus) to the routine child immunisation schedule in LMICs, particularly in Latin America [126, 127]. Among preventative public health measures, studies exploring screening and/or vaccination combinations were common. For example, Demarteau et al. explored efficient combinations of cervical cancer prevention strategies (e.g. screening and/or vaccination against human papillomavirus) using a combination Markov and optimisation model, in both Brazil [128] and Nigeria [129]. The Markov model estimated the costs and outcomes of different strategies, which was used as input to an optimisation model that determined the combination of prevention strategies that minimised cervical cancer cases for a fixed budget.
Similar to the clinical medicine category, HIV/AIDS, malaria and TB were a common focus for public health studies, with approximately 30% of all studies in this category dedicated to these illnesses. Simulation platforms, such as OpenMalaria [130] and STDSIM [107], have provided the modelling foundation for several public health-oriented OR studies related to such illnesses, at both a national and regional level. STDSIM was used to analyse focused public health interventions for high risk groups such as commercial sex workers [131, 132]. The OpenMalaria model was used to simulate the impact of interventions such as indoor residual spraying in the highlands of western Kenya [133].
Global-level studies represented only 2% of studies, and most of these (52%) were in the public health category. Examples of such studies include a model to recommend the required size and resulting cost of an international stockpile of cholera vaccine to enhance efforts to mitigate cholera outbreaks in the wake of natural disasters [134], and a comparison of the potential impact of rotavirus versus human papillomavirus vaccination across 72 countries eligible for support from the Global Alliance for Vaccines and Immunization (GAVI), taking into account affordability, cost-effectiveness and distributional equity [135].
One area of disease prevention that lay at the intersection of clinical medicine and public health is the prevention of mother-to-child transmission of HIV. Although some prevention strategies are of a clinical nature (e.g. administering ARVs or nevirapine), we considered this a public health issue as there are other behavioural considerations as well (e.g. recommendations for early weaning or avoidance of breast-feeding). Examples of such studies include a DES model used to evaluate relative benefits of ARVs at childbirth and/or bottle-feeding in Tanzania [45], a mathematical model comparing different feeding recommendations (i.e. exclusive replacement-feeding versus exclusive breast-feeding for durations of 4 or 6 months) at different compliance levels in Uganda and Kenya [136], and simulation studies exploring the cost-effectiveness of implementing the WHO’s 2010 guidelines for the elimination of mother-to-child transmission in Zimbabwe [137, 138].
Health innovation
Innovation was the least studied category of global health-related OR, with only 47 papers. The majority of these studies (89%) were related to vaccines, either in the early phases of clinical trials or yet to be developed, and were predominantly focused on HIV and malaria. Common themes were modelling the potential impact of imperfect or partially effective vaccines [139‐143] or vaccines with rapidly waning protection [144, 145], modelling changes in behaviour (i.e. adopting riskier or relaxed behaviour) with the introduction of a newly developed vaccine [146‐149], modelling the cost-effectiveness or willingness-to-pay thresholds of a new vaccine [150‐157], forecasting demand for a new vaccine [158], or combinations of these issues [159‐165].
Some studies explored the best ways to implement or roll-out a new vaccine should it become available (e.g. through the Expanded Programme on Immunization (EPI), school-based programmes, mass vaccination campaigns, targeted high risk groups, planning for follow-up boosters, etc.) particularly in cases where initial supplies are expected to be limited [166‐171], as well as how a partially effective vaccine would measure up against existing prevention strategies [172] (e.g. male circumcision in the case of HIV [173] or insecticide-treated nets in the case of malaria [174]). Lee et al. [175] used a DES model of Niger’s vaccine supply chain to analyse the impact of developing thermostable versions of six currently available EPI vaccines, an innovation that could relieve bottlenecks in the cold chain. They found that thermostable versions of any of the EPI vaccines, either individually or in combination with other vaccines, would decrease cold storage and transport utilisation and increase the availability of all vaccines, even non-thermostable ones. Levin et al. [176] also explored thermostable vaccine introduction in Cambodia, Ghana and Bangladesh – their model was a spreadsheet-based decision tree and costing analysis.
Other studies examined innovations in drugs and new diagnostic technologies [177‐179]. For example, Dowdy et al. [178] used a decision analysis model to estimate the cost-effectiveness of a novel point-of-care TB diagnostic tool in comparison to existing methods in South Africa, Brazil and Kenya. Cost-effectiveness was sensitive to the specificity and cost of the new test, but its introduction was estimated to avert almost 50% more disability-adjusted life years per 1000 TB suspects [178].
The examples provided in this section highlight how OR can be a useful tool for informing health policies and decision-making in low-resource settings – from studies with local health facility-level implications to analyses that are global in scope, exploring issues that span all application areas of global health. We have highlighted areas where there has been a strong OR focus; for example, national-level studies focused on clinical and public health and studies about infectious diseases such as HIV/AIDS, malaria and TB. Areas where OR analyses have been lacking include health technologies and non-vaccine-related innovation, and non-communicable diseases such as cancer, diabetes and mental health.
Health equity theme
This section focuses on an important goal of global health – achieving equity in health for all people worldwide. The challenge in studying health equity is that there is no single way to identify or measure it within a community or population. We felt it would be compelling to discuss how issues of health equity have been analysed using an OR approach, especially given this challenge.
Out of the 1099 papers included in this review, we identified 44 studies that considered health equity as an important part of the research question being explored. Due to our review’s focus on healthcare provision and public health, rather than wider social determinants of health, the studies in this section are primarily focused on healthcare equity, specifically as it relates to socially disadvantaged groups. These studies spanned all four application areas of global health (health systems and operations (n = 16), clinical medicine (n = 4), public health (n = 22) and innovation (n = 2)) and all target levels (local (n = 6), national (n = 29), regional (n = 3), global (n = 4) and general (n = 2)). Geographically, studies were predominantly focused on South Africa (n = 10), China (n = 5) and India (n = 4); all other locations had just one or two studies.
Studies differed in how they operationalised (i.e. defined the measurement of) healthcare equity. Some studies defined inequity as a quantifiable disparity in a specific health indicator across different social groups (e.g. mortality risk across wealth quintiles [180], malaria incidence in children and pregnant women vs. adults [181]) and estimated how this indicator might change with a more equity-centred approach to an intervention. Other studies parameterised equity as a model variable that ranged between two extremes – from least to most equitable (e.g. percent coverage of an intervention [182, 183], measures of spatial accessibility [184] or a modified Gini coefficient [185]) – allowing researchers to explore the circumstances under which this parameter was less than ideal or even how to maximise it. Some applied a single ethical principle when operationalising equity (e.g. Wilson et al. [183] took an egalitarian approach), whereas others explored their research question through multiple ethical lenses [185, 186].
We also looked at the distribution of healthcare equity studies across groups with different levels of underlying social advantage/disadvantage, including wealth, geographic location, sex or other social status (Fig. 7).
Fig. 7
Number of equity-themed OR papers by topic area. Note that some studies looked at equity across several categories; these were counted for each relevant category. Marginalised groups include people living with HIV or other stigmatised illnesses. High risk groups include men who have sex with men, commercial sex workers, or people considered to be in high risk age groups for certain diseases
×
Accessibility of healthcare, for both the financially and geographically disadvantaged, was a common theme among equity-related papers. The impact of health insurance and/or universal coverage [185, 187‐189], user fees [190] and subsidies [191] on equitable healthcare accessibility and affordability was one of the most prominent themes. For example, Waters et al. [185] used a statistical probit model to analyse the potential impact of a health insurance program and various insurance eligibility standards on both overall access to healthcare, as well as equitable access to healthcare across all economic quintiles in Ecuador. Economic status was also considered by Pagel et al. [192], who explored how different community-based strategies to prevent post-partum haemorrhage affected women of different economic quintiles in Malawi, and Carrera et al. [193], who showed that an equity-focused approach to child health that prioritises the poorest and most marginalised populations could lead to higher decreases in child mortality while being more cost-effective than traditional approaches.
Geographic accessibility and distribution of health services, and the identification of geographic disparities in health, were explored by a number of resource location-allocation studies [183, 184, 186, 194‐196]. For instance, Moore and Stamm [184] built a location optimisation model for cholera treatment facilities in Haiti, using the Enhanced Two-Step Floating Catchment Area method. They present their model with five unique objective functions, including one that minimises inequitable access, in order to explore the trade-offs between adequate, equitable and efficient coverage of treatment centres. Similarly, a resource allocation model of a Zambian health service delivery program parameterised equity in the objective of their optimisation model for decision-making based on resource efficiency and equity across varying geographic locations [186]. In India, a location-allocation model was used to propose new health facility locations for improved geographic access to healthcare [194].
Health equity issues related to sex [146, 197‐204], high-risk groups (e.g. commercial sex workers) [205‐207] or marginalised groups (e.g. people living with HIV) [182, 208‐212] were commonly associated with health issues such as HIV and cardiovascular diseases. For instance, upon recognising that women often lack the power to negotiate safe sex in developing countries and can be exposed to HIV against their will, studies have analysed the effects of post-circumcision changes in male condom use [198, 200] and women-initiated vaginal microbicides [199] on gender health equity in Southern Africa. The issue of high HIV burden among sex workers was analysed using a deterministic model that compared the impact of several interventions, including equitable access to ARVs and community empowerment programs that educate female sex workers about preventive measures against HIV [207]. Two studies applied an equity lens to mathematical optimisation problems exploring optimal HIV treatment strategies in South Africa [182, 183]. Wilson et al. [183] formulated an ‘equity objective function’ to propose ARV allocation strategies that would ensure each individual infected with HIV has an equal chance of receiving ARVs. Cleary et al. [182] parameterised the concept of health equity as the percent coverage of treatment in HIV/AIDS patients. By placing different constraints on this parameter in the model, they were able to highlight the trade-off between maximising equity versus maximising health outcomes, where the ‘opportunity cost’ is QALY’s forgone in the former scenario, and higher proportions of unmet need in the latter [182].
These examples highlight the utility of OR for informing equitable health policy decision-making in low-resource settings. Equity is not a concept easily measured, nor will it be possible to achieve consensus on how it should be measured. A major contribution of OR is that it allows for equity to be quantified in different ways, often within the same modelling framework, such that trade-offs and consequences can be explored more systematically, opening up important discussions about how best to reduce systematic disparities in health for all people worldwide.
Impact theme
In this section, we highlight seven OR studies in which the authors described how their work was implemented or was influential to specific health policy changes or decisions. This compilation is not an exhaustive list; however, studies describing implementation or impact represent a very small fraction of all papers in this review (we estimate less than 10% based on our review of abstracts). In the sub-section that follows, we explore several features of these studies that may have helped contribute to the effective translation of model recommendations into policy or practice, and discuss barriers and challenges to bridging the gap between operations research and health policy.
Case examples of OR impact
The first four studies are examples of impact at the national or global level. Dowdy et al. [213] used a decision tree model to estimate the cost-effectiveness of serological testing for active TB in India. Serological tests are widely used in India and other developing countries because they are fast, simple and readily available; however, no international guidelines recommend their use over other diagnostic tests such as sputum smear microscopy. The study found that serology tests can result in more secondary infections and false-positive diagnoses, and cost more per-patient, compared to sputum smear microscopy. Their findings, which were presented to a WHO Expert Group on TB in 2010, were influential to the WHO’s subsequent policy statement recommending against the use of commercial serological testing for active TB [213].
Hutton et al. [38] developed a combination decision tree and Markov model of hepatitis B infection and progression, which compared options for hepatitis B screening, vaccination and treatment in the United States and China. In China, they found that providing catch-up vaccination for children under 19 would improve health outcomes as well as save healthcare costs in the long run due to the number of infections averted. Their modelling work in 2008 was influential in China’s decision to expand free catch-up vaccination to all children under 15 in April 2009 [38].
In the wake of a global debate to shift the significant resources being used for polio eradication towards effective control [214], a systems dynamics disease outbreak model for polio developed by Thompson and Tebbens [215, 216] demonstrated that shifting to a control strategy would not only be more costly in the long run, but would lead to more cumulative cases as populations become more susceptible to new outbreaks [215]. After the results of their model were presented to global stakeholders at a WHO-convened consultation in 2007, experts were convinced that efforts towards completing eradication must continue; for example, the director of the global polio-eradication initiative at the WHO in Geneva commented that Thompson’s work put “a nail in the coffin for the idea that there is a cheap and painless way out”, and a representative from the global immunisation program at the United States Centers for Disease Control and Prevention commented that this analysis showed there is no viable control option and that we need to intensify eradication efforts [217]. As eradication efforts continue today, there is hope that complete eradication can be achieved in 2016; in 2015, there were fewer cases in fewer countries than ever before, and in January 2016, India marked its fifth year without a case of polio [218].
A DES model was developed by Langley et al. [39] to evaluate the impact of automated nucleic amplification test (aNAAT), a new TB diagnostic test, compared to existing techniques in Tanzania. The model recommended several combinations of TB diagnostic options incorporating aNAAT testing that were cost-effective in both urban and rural settings. At the time of publication, policymakers in Tanzania were considering specific sites for a trial of the new aNAAT technology, and results from the DES model were going to be used to inform the implementation plan for the trial [39].
The following three studies are examples of implementation on a more local level. Cruz et al. [104] developed a queuing simulation model to help enhance medical equipment repair service quality for the clinical engineering department of a 600-bed hospital in Cuba. Simulation results showed that service quality enhancements (i.e. reduced work order backlogs and service times) could be achieved without hiring new personnel. Clinical engineering management implemented two proposed strategies and major service improvements were observed over a 2-year period, as predicted by the model [104]. Perez et al. [59] used a combination DES-optimisation model to reduce wait times in the admissions centre of a health centre in Colombia with relatively low additional cost. The solutions proposed by the model were subsequently implemented, and although not explicitly measured, experts in the admissions centre noticed relevant improvements in wait times [59]. Finally, Friedrich et al. [72] developed a decision support system (DSS) using linear programming to upgrade the nurse scheduling process at a hospital in South Africa in order to improve the quality of healthcare and nursing services. The model’s objective was to minimise nurse dissatisfaction by better taking into account nurse preferences. Although the system had not been fully implemented at the time of publication, feedback provided through user validation was positive and enthusiastic. Staff managers reported that, in just a few seconds, the system performs the same time consuming computations they carry out manually each month, with improved nurse utilisation and reduced overtime [72].
Factors contributing to success in ‘bridging the gap’ between OR and impact
Based on these cases, three key drivers for bridging the gap between OR and impact have emerged, namely (1) engagement of local or expert stakeholders in model design and validation, particularly those in policy- or decision-making roles; (2) use of contextually representative data; and (3) a concentrated effort on communication of research findings. All selected cases demonstrated all three of these key drivers even if not explicitly cited in the discussion that follows.
(1) Local or expert stakeholders involved in OR model design and validation
Active participation of local stakeholders has been suggested by others as a key to strengthening health research and policy linkages [219, 220] and the examples provided in this section are evidence of this in the field of OR. Such collaborations are important for several reasons. First, the engagement of stakeholders facilitates the identification of relevant and appropriate global health research questions. Thompson and Tebbens advise that “modelers need to focus on working effectively with the people who need and can use the results” [215]. Langley et al. [39], who underwent a comprehensive review of the questions that policymakers need to address when assessing different TB diagnostic strategies, are a great example of this. Situating their work within this ‘Impact Assessment Framework’ [221] not only helped identify the questions that their model should answer, but also informed the appropriate choice of modelling methodology to achieve their goals. Identifying relevant research questions is perhaps more easily accomplished for OR studies based on local settings. For example, Friedrich et al. [72] conducted a root cause analysis of challenges faced at the hospital they wanted to help, and developed their decision-support solution in response to several identified problems around nurse scheduling. Cruz et al. [104] and Perez et al. [59] also worked with local collaborators to identify and model relevant problems for the health facilities they worked with.
Second, it is critical to involve local or expert collaborators in the model conception and design because they are intimately knowledgeable about the context, and can help ensure that the model and analysis accurately describes the health issue and addresses the policy questions or decisions they face. This type of collaboration was demonstrated by Hutton et al. [38], who formed a multi-disciplinary team, including the director of Stanford’s Asian Liver Centre, for their research on hepatitis B in China. They also used an iterative approach to model development, beginning with a very simple representation of patient health states, with details added incrementally based on suggestions from experts until they were satisfied that their model appropriately represented the policy problem [38]. Friedrich et al. [72] also underwent an iterative design process, whereby their nurse scheduling DSS model was tested with users and continuously improved throughout development. Langley et al. [39] worked with experts from Tanzania and Malawi to ensure the input parameters and control logic for their DES model were valid.
Third, engaging local stakeholders throughout the development process can also facilitate trust-building and implicitly lead to capacity building, helping to address the lack of technical ability to interpret findings common in LMICs [219] and empower the policymakers to take ownership of the process. For example, as a result of the work by Langley et al. [39], policy advisors in Tanzania have requested the ability to be able to use the simulation model themselves to evaluate alternative diagnostic strategies in the future. A pilot study is underway to demonstrate whether the model is sufficiently user friendly for this type of use [39]. In the case of Friedrich et al. [72], the users were pleased that their input was used so extensively in the development of key features of the nurse scheduling DSS, including the interface design and the data validation function that prevents them from entering invalid data; users even requested further training in using the DSS [72].
If collaboration between local policymakers, researchers and implementers is important for impact, then the lack thereof can be a major barrier to impact. Yet, much of the OR literature reviewed did not have a collaborator or partner in the context where their model was intended to serve. Although some studies did express a desire to use their models and results as a basis for further research in partnership with healthcare organisations, in general, few of the studies in this review mentioned showing (or even the intention of showing) their model or results to relevant stakeholders.
(2) Use of contextually representative data
In addition to having relevant stakeholders involved in the model conception and design, contextually representative data is also likely an important factor in generating OR analyses that have impact or are implementable. The featured case studies are examples of the use of appropriate data. The studies by both Thompson and Tebbens [216] and Dowdy et al. [213] were focused on India, and to the extent possible, were populated with input data relevant to the Indian context, from state-level statistics on population, polio incidence, etc. [216] to costing data obtained from local labs [213]. These nationally-focused studies were compelling enough to capture the attention of global stakeholders, such as the WHO, leading to broader global implications. Hutton et al. [38] conducted a comprehensive review of over 250 published papers to populate the data for their model on hepatitis B in China. Input data for the TB diagnostics model in Tanzania came from a range of sources, including the National TB and Leprosy Programme, diagnostic centre laboratory records, and local managers [39].
For locally-focused studies, especially when local collaborators are involved in the research exercise, it is often possible to prospectively collect the data required for modelling. For example, Cruz et al. [104] used data collected in the hospital’s electronic technology management system over a 3-year period for both the development of their equipment service simulation model and for the validation of their model recommendations post managerial improvements. For the study to reduce wait times at a health centre in Colombia, the time between patient arrivals and service time in the admissions centre were collected for a short period of time in order to build the simulation model. The model was further validated using admissions data provided by health centre managers [59].
For many of the other studies in this review, approximations and assumptions were required to estimate certain model parameters. Often researchers had to resort to the use of unrepresentative data, for example data from neighbouring regions or developed countries. These data assumptions and compromises, which are often unavoidable, should be taken into consideration when applying model results and recommendations to a given context.
Availability of reliable data is one of the challenges that sets low-income countries apart from middle-income countries with regards to modelling [32]. In this review, the studies in some middle-income countries (e.g. China [38] and Brazil [43]) were able to make use of centralised hospitalisation information systems and national household surveys, enhancing the validity and robustness of their models and analyses. In fact, access to data may be a reason for location selection on the part of researchers, possibly explaining our finding that low-income countries are less often targets for OR. For example, one study stated “India was selected for this simulation because it is one of the largest developing countries and sufficient data on breast cancer epidemiology to construct a reliable and valid model were available” [222].
(3) Emphasis on communication of research findings
Publication is important, but not sufficient, for the effective communication of research findings, whether from OR or other types of analyses. Communication in various forms beyond the journal publication was an important part of the case studies that influenced change. Perhaps the best example of this was Thompson and Tebben’s work on polio eradication [215], which they had the opportunity to present at a WHO stakeholder meeting in early 2007. For their presentation, they did not focus on explaining the model, equations, and diagrams in detail, but communicated the key insights in the simplest way possible [215]. For the study on TB serology testing, two of the authors were affiliated with the Stop TB Partnership’s New Diagnostics Working Group and had the opportunity to present their study findings to a WHO expert panel on TB serological testing, an audience that would be receptive to their work [213]. One study was actually translated into another language [38], and for another [72], user validation interviews were conducted in the local language in order to get the most accurate feedback possible from users.
Many of the OR models had user-friendly interfaces or used visual simulation environments as a means of communicating their model applications and results in a more personalised or accessible fashion. For example, Hutton et al. [38] specifically used Microsoft Excel because their intent was to develop a model that could easily be shared with policymakers. They “incorporated sufficient detail to capture important characteristics of hepatitis B disease progression and treatment so that the model would be believable to a clinical audience” [38] but tried to keep it simple enough so that those who lack modelling expertise could easily understand it. Langley et al. [39] also stressed the importance of a visual representation of the modelled processes in order to improve engagement and assist in the validation of their model with experts in Tanzania; the simulation software they chose afforded this possibility. The output of the nurse scheduling DSS model was formatted similarly to the hospital’s previous manual scheduling process so that unit managers would more easily adopt and transition to the new system [72].
Overall, the continued expansion of research reach and influence requires sustained efforts to communicate findings through different channels, with engaged outreach to, and personal connections with, policymakers and public health officials.
Discussion
“Scoping studies aim to map the literature on a particular topic or research area and provide an opportunity to identify key concepts; gaps in the research; and types and sources of evidence to inform practice, policy-making, and research” [35]. The goal of this scoping review was to provide a broad overview of the use of OR in global health, with several concrete examples showing the breadth and depth of how this field of research is being applied to important global health challenges worldwide. We also explored the theme of health equity, demonstrating the unique opportunities the field of OR can contribute to this increasingly important area of global health. Cases where OR has had an impact on policy- or decision-making were also highlighted, with examples ranging from the implementation of local-level changes related to the day-to-day operations of health facilities, to decisions about national vaccination policies, to influencing international WHO policies and global perceptions about disease eradication. These cases serve as excellent examples of the importance of collaboration, data and communication for affecting change at the local and global level.
Limitations and challenges of the review
We faced some challenges and limitations when conducting this review. Given the broad interpretations of what constitutes OR, a lack of consistent terminology for OR, and the variety of journals where OR literature in healthcare tends to be published, our search terms were broad in reach and scope, with the consequence that a large amount of literature was captured that was not relevant. Additionally, the geographic search tailored to LMICs was not straightforward; country names are not always considered controlled vocabulary, so every individual country name had to be included as keywords.
We also had to be pragmatic at the outset about the coverage of the review. We chose to focus on papers published in the year 2000 and later. As such, there is some overlap with the hand-searched literature [11, 16, 17, 23, 24, 26‐28, 32], but not with the review of OR in global health by Datta [10], which at the time of our review was over 20 years old. To fill this gap would have been an onerous task. Due to rapid advancements in computing technology, it could be argued that OR models developed before 2000 are out-dated, as are any global health data used to populate them. Further, we also had to be selective when setting the inclusion criteria for the types of ‘OR’ and ‘health’ studies explored given these terms have such broad definitions. We restricted OR studies to those where modelling or analytical methods were used with an orientation towards decision-making. Our health focus was largely on healthcare provision and public health, and did not include the wider social determinants of health; this focus was inherently reflected in the subset of studies explored for the health equity-theme as well. As it was, due to the volume of literature included, we could not summarise or cite all of the studies found; however, we hope this review has provided enough of a landscape overview to prompt further exploration of the utility of OR in this context. A complete database of the 1099 studies is provided as Additional file 2.
Income categories for countries were based on 2014 World Bank classifications, regardless of how a country may have been classified historically. It would have been difficult to track shifts in income classification for every country and every paper included in this review. Furthermore, we felt the interpretation would be simpler knowing that all studies related to a specific country (e.g. Brazil) were consistently counted towards its current income category (e.g. upper-middle) rather than split across multiple categories.
Our criteria for ‘impact’ when selecting case examples was that the study meaningfully informed a policy decision or the recommendations were implemented in a real-world setting. We were unable to make any inferences about the magnitude of improvement in health that may result from these changes. Only Hutton et al. [38] presented estimates that 170 million children would be vaccinated for hepatitis B in China as a result of their model recommendations, preventing almost 8 million infections and 70,000 deaths, and saving the equivalent of $1.4 billion over the lifetime of these children.
A final limitation of this review was the restricted search for published literature alone. Those papers that did not describe a particular policy change could have indeed influenced decision-making after their publication. Future work could involve searching grey literature for case studies and policy documents suggesting that knowledge gained from OR was influential in decision-making processes. The lack of published evidence that OR is impacting policy change represents a major missed opportunity for the academic community to learn and better engage in impactful OR work. It has been mentioned that fora are needed where these findings can be discussed [219] and success stories of policy transfer shared with a broader community [14], if not in peer-reviewed literature, then elsewhere.
In general, scoping reviews take a considerable amount of time and skill. Balancing feasibility, breadth and comprehensiveness can be a challenge given available time, funding and resources [35, 37]. Although not explicitly tracked, we thought it would be informative to provide an estimate of the amount of time that it took to conduct our review. The most time intensive stages were stages 2 to 4 – designing the search, identifying and selecting studies and charting data. Stage 2 was conducted over a period of approximately 4 months by one author on a near full-time basis (estimated 500 person-hours), and stages 3 and 4 were carried out by four co-authors over a period of approximately 8 months, all on a part-time basis (estimated 340 person-hours). This is consistent with the findings of Pham et al. [36], who reported scoping reviews have taken anywhere from 2 weeks to 20 months to complete.
Global overview and global health application areas
Despite these limitations and challenges, this scoping review, consisting of 1099 studies, is to our knowledge the most comprehensive review of OR in global health to date. Our overview highlighted that low-income countries are less frequently studied using OR compared to middle-income countries, a trend that does not seem to be improving with time. Furthermore, a large proportion of healthcare-related OR in LMICs has focused on just six middle-income countries. If population is an appropriate yardstick for research focus, then perhaps this representation is reasonable; however, the disparity between volume of literature and number of countries in each income category is much more pronounced. That being said, 84 LMICs around the globe have been the focus of at least one OR study since 2000; hopefully, increased global coverage will continue.
Although the aggregate data did not show that any particular OR method was dominant, we found that certain types of research questions were more amenable to specific OR methods than others (e.g. local and national-level health systems problems were commonly studied with simulation or optimisation methods). This highlights the importance of a collaborative and interdisciplinary approach to applying OR in global health, such that those with modelling expertise in specific methods can apply their expertise where it can make the most impact.
We found that the majority of OR literature explores clinical medicine and public health issues at a national level, and that, although very helpful for exploring the impact of interventions on a macro scale or adding to discussions on global priority-setting, fewer studies were targeted at the regional or global level. Since OR models tend to describe the dynamics and interdependencies of actual systems, it likely gets more difficult to develop accurate models as complexity increases (i.e. from local or national systems to regional or global systems). OR models are also highly dependent on input data, which is likely easier to obtain at the local and/or national level. Studies targeted at several countries or a whole region have themselves cautioned that more specific country-level analyses with more representative data are needed for country-level decision-making, which is perhaps why few studies are targeted at these levels [135, 193].
One area of global health that OR has made a unique contribution to is the analysis of new health innovations. This small subset of studies highlights the value of OR for analysing global health interventions that cannot yet be trialled or implemented on the ground because the scientific breakthroughs have yet to be achieved. Understanding the potential impact or possible implementation challenges of new innovations is important to their successful roll-out when they do become available. OR can also help highlight important design criteria targets for the development of new innovations, in terms of both cost and minimum levels of efficacy or specificity required to achieve desired outcomes.
Infectious diseases (e.g. HIV/AIDS, malaria, TB) continue to be a major focus of OR globally; however, there has been an increase in the number of OR studies about non-communicable diseases over the past 5 years. Neglected areas representing an opportunity for future OR include analyses focused on low- and lower-middle income countries, non-communicable diseases (particularly mental health), and medical equipment and technology planning.
Additional study characteristics that could have been analysed include the funding sources, the academic institutions of the lead investigators and the quality of the studies themselves. Funding sources have been identified as a potential external influence on both research and policy agendas, but in some cases resources dedicated to an OR study can lead to positive change. For example, two studies from this review [43, 223] were funded by the Brazilian Ministry of Health [223], indicating interest on the part of the government to use OR as a tool to answer a key health systems-related questions. Exploring funding sources in more detail could be an area of future consideration. The quality of the studies themselves could also be highly relevant to successful research uptake, but we did not undertake critical appraisal for this review as quality assessment does not typically form part of the scoping study remit [34]. Others have reported on the quality of simulation models in healthcare by applying strict quality criteria during the review process [18].
Health equity: an opportunity for future OR
The healthcare equity-themed studies featured in this review demonstrate the utility of OR for informing equitable health policy decision-making in low-resource settings. These studies, however, represented a relatively minor proportion (4%) of all global health-related OR in the time period studied. Some have argued that a major shortcoming of the Millennium Development Goals was a failure to address equity [6], and that “looking forward, equity analyses and actions need to be an integral part of programme strategies rather than an afterthought” [7]. We believe a huge opportunity exists to apply the tools and techniques of OR to study health equity in the post-2015 era.
First, OR is extremely versatile in how the concept of health equity can be operationalised. The examples in this review demonstrate a variety of different ways to quantify equity - from measured disparities in specific health indicators to parameterised model variables. Although beyond the scope of this review, these principles can be used to explore the broader social determinants of health as well. As no single measure is sufficient to assess inequities, those applying OR to health equity could benefit from integrating established health equity frameworks into their approach (e.g. the PROGRESS [224, 225] and PROGRESS-Plus [226] frameworks) to help ensure the explicit consideration of important equity factors in the design of OR models and analyses. Second, OR allows for the comparison of different equity goals (e.g. using different ethical principles or comparing efficiency versus equity), often within the same modelling framework. For example, a utilitarian perspective aims to maximise overall societal benefit, whereas an egalitarian approach would strive to achieve equal distribution of, or access to, resources for every person [183]. OR allows for a more systematic analysis of the trade-offs and consequences of viewing equity from these different perspectives. Third, OR models can study the effect that different policies or decisions might have on marginalised populations without the ethical implications of a real-world study. Arguments could be made that, to achieve health equity, certain groups should be valued over others; for example, perhaps high-risk groups or the least advantaged should be prioritised, rather than treat all cases alike regardless of social standing. OR can aid in testing such sensitive policy hypotheses a priori without unintentional consequences.
OR can be applied at national and sub-national levels, across different socioeconomic groups and marginalised populations, and through different ethical lenses, in order to inform interventions and health policy decisions that will promote better equity in health going forward. We hope the equity-themed literature highlighted in this review can help open up important discussions about how best to model and analyse systemic disparities in health for all people worldwide.
OR impact: recommendations for bridging the gap between OR and policy or practice
Studies describing OR implementation or impact represent a small proportion of the literature reviewed – there is still a far way to go for OR to reach its full potential in global health. Our finding that few papers present details of implementation or impact is consistent with the experience of others who have reviewed OR in developing countries [10, 11, 23, 24]; for example, Datta [10] reported that less than 5% of studies reviewed discussed implementation. From the numerous studies included in this review that did not appear to have any influence on policy or decision-making, several persistent challenges emerged as common themes, including lack of local or expert stakeholder engagement in model conception and design, challenges in acquiring reliable and representative data, and a lack of communication strategy beyond the journal publication.
The lack of appropriate packaging of research findings or exclusive dissemination within academic circles was also found by others to be barriers to research uptake in LMICs [219, 227]. In recent years, several theories, frameworks and practical handbooks or ‘toolkits’ have been developed by agencies such as the Overseas Development Institute [228‐230], the International Development Research Council [231], the WHO [232], and the Institute of Development Studies [233], to help guide and make more effective the translation of research into policy. Specifically, there is a focus in this literature on the effective communication of research findings, which extends beyond just communication products (e.g. policy brief, stories of change, etc.) to a whole body of research on knowledge sharing, knowledge transfer and knowledge translation. The use of packaging and language that are more appropriate and targeted towards implementation can help enhance the impact of OR.
Yet, there have been many success stories of global health research effectively bridging the research-policy gap [13, 234]. For example, Zachariah et al. [13] identified several impactful operational research studies that had implications for policy and practice; however, all of these studies were field studies that did not involve modelling and thus did not meet the inclusion criteria for this review. Additional guidance to be gleaned from these studies for the OR modelling community include (1) the generation of research questions from within existing programs, which are focused and of simple design; (2) working with partners to ensure that sufficient resources (human and financial) are available for an engaged and motivated research process that extends beyond models and analyses; (3) setting realistic expectations of research impact; (4) investing in long-term research and policymaker relationships; and (5) helping build capacity of end-users to use research to demand policy change [13, 234].
Promisingly, within the OR community, there is a growing movement towards impact-driven research and publication. The “Doing Good with Good OR” paper series and research award, offered by the Institute for Operations Research and the Management Sciences, and the “OR in Development” prize, offered by the International Federation of Operational Research Societies, are efforts to recognise OR for the impact of the analysis, in addition to its analytical rigor. In general, there is a need for incentivising the engagement of researchers in problems that are relevant and timely to important policy issues [219]. Hopefully, these efforts, paired with efforts within developing countries to increase end-user capacity to use OR [11, 25] will help bridge the gap between OR and impact in LMICs.
Conclusion
There is a tremendous opportunity for OR researchers and global health practitioners alike to continue to apply OR in global health, particularly in areas where such studies may currently be lacking. We hope the findings of this scoping review, which represents the most comprehensive compilation of OR literature in global health to date, are of interest to a wide-ranging group of stakeholders engaged in global health policy and practice. For government bodies and administrators of health programs and services, we hope to have showcased the utility of the OR approach in modelling policy and programme changes to improve efficiency, particularly when resources are limited. We also hope funders of international development research see value in allocating funding to operations research within broader global health programs. We hope those currently engaged in OR can benefit from the impactful studies highlighted in this review, and we encourage them to share the impact of their work more broadly so that others can learn from challenges and successes.
Acknowledgements
Not applicable.
Funding
Not applicable.
Availability of data and materials
All data generated or analysed during this study are included in this published article [and its supplementary information files].
Authors’ contributions
BDB conceived the study idea, designed the systematic search, conducted the initial screen of search results, reviewed papers for the second screen of search results, categorised papers, contributed to the interpretation of data, and drafted the first manuscript. TJ, AT-V, BK reviewed papers for the second screen of search results and categorised papers, and TJ contributed to writing the paper. TJ, TCYC and Y-LC contributed to the study design and interpretation of data, and critically revised the draft. AT-V and BK provided comments on the draft. All authors read and approved the final manuscript.
Authors’ information
BDB is a doctoral candidate at the Centre for Global Engineering and the Collaborative Doctoral Program in Global Health at the University of Toronto. Y-LC is Director of the Centre for Global Engineering at the University of Toronto. TCYC is the Canada Research Chair in Novel Optimization and Analytics in Health, and Director of the Centre for Healthcare Engineering at the University of Toronto.
Competing interests
The authors declare that they have no competing interests.
Consent for publication
Not applicable.
Ethics approval and consent to participate
Not applicable.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
