Description of the context and intervention
Chad is a landlocked country in the Sahelian belt, suffering from an annual hunger gap between June and October. The Ouaddaï region in eastern Chad, with Abeche as the capital city, experiences some of the highest rates of childhood acute malnutrition in the country. This area additionally suffers from a limited number of humanitarian actors, due to ongoing security concerns [
30]. In 2010, below-average cereal production and high food prices led to increased food insecurity in eastern Chad [
31].
In the first half of 2010, UNICEF reported that an increasing number of areas were affected by SAM, including hundreds of thousands of internally displaced people and refugees in eastern and southern Chad [
30]. ACF surveys in Abeche reported that in mid-2009, wasting prevalence was 20.6% with 3.2% severe wasting and in early 2010, wasting prevalence was 16.8% with 2% severe wasting [
32,
33]. These rates are consistently above the WHO threshold of 15% [
34], demonstrating a need for intervention. Moreover, project documents noted that admissions to nearby CMAM (community-based management of acute malnutrition) programs were higher and later in the season than expected given annual trends [
35].
Table
1 presents the baseline nutrition status of children participating in the program. While acute malnutrition (in terms of WHZ and MUAC) is not at emergency levels, prevalence of anemia is above 60%. There is no baseline measure of diarrheal morbidity; however a feasibility study conducted by ACF in February 2010 in Abeche indicated that poor sanitation and hygiene, common causes of childhood diarrhea, were serious issues affecting children’s health and nutrition status, particularly in the most vulnerable neighborhoods where the intervention was targeted. Poor sanitation was exacerbated by severe rainfall and flooding in Abeche during July and August of 2010.
Table 1
Sample characteristics at baseline
MUAC, cm, mean (SD) | 14.0 (1.0) | 14.0 (1.0) |
Wasting, n (%) | | |
WHZ < −2 | 59 (13.4) | 81 (13.6) |
WHZ < −3 | 1 (0.2) | 3 (0.5) |
Anemia (hemoglobin <110 g/l), n (%) | 270 (61.5) | 370 (61.9) |
Program implementation took place in 7 administrative sectors of Abeche, which were divided into 14 geographical clusters. These clusters were then randomized during an official ceremony with community leaders. Seven clusters received only Food Assistance (FA) rations consisting of staple foods (sorghum, legumes, palm oil, sugar and salt, approx. 1,800 kcal [
18]) without a fortified blended food such as CSB, and 7 received FA with additional provision of RUSF (FA + RUSF). Operational research was implemented within this structure for 5 months (June-October 2010) with an RCT, with a protocol approved by the Ethics Committee of the University Hospital of Ghent in Belgium and Chadian authorities; study methods have been described elsewhere [
18]. Acutely malnourished children were enrolled in the standard protocol for management of acute malnutrition, available at health center level and technically supported by ACF.
Starting 2 months before implementation commenced, a Sensitization Project Manager raised awareness about the program by creating sensitization materials and discussing with community leaders and health officials within the intervention area. This was an important step due to the lack of humanitarian projects in the city, and community suspicion of such projects. Beneficiary selection was conducted with the assistance of community leaders on the basis of ACF’s vulnerability criteria; ACF subsequently verified the final household lists.
Each month, food distributions were conducted at 5 distribution sites accessible to the participating communities. Since communities donated distribution sites, constructions were built temporarily and broken down when distribution was finished at each site. Monthly, distribution took between 7 and 8 working days (over approximately 2 weeks) with additional days required for setting up and breaking down the distribution sites or rebuilding them due to weather damage. During the remainder of each month, program staff were engaged in Food for Training sessions (for the first 2 months only, as described below), raising community awareness about the date of the next distribution, replenishing food rations for the subsequent distribution, entering data and conducting surveys (including a baseline survey, two SMART surveys, and Post Distribution Monitoring). All beneficiaries entered the general distribution site, and were channeled into different “circuits” based on whether their distribution card identified them as being part of the FA or FA + RUSF group. Data on nutrition and morbidity status was collected at distribution sessions: anthropometry was measured monthly, episodes of diarrhea were recalled for the prior week on a monthly basis, and hemoglobin concentration was measured both at baseline (June) and end of intervention (November) or when the child was discharged from the study.
During the first 2 months of implementation, rations were distributed to all beneficiaries (FA and FA + RUSF groups) conditional on attendance at Food for Training sessions on hygiene-related themes. These trainings may have improved beneficiary practice to some extent, however due to a deteriorating security situation in Abeche, these sessions were discontinued and by August the program was limited to unconditional food distribution.
The FA program and the RUSF component shared many activities and resources. Both program components shared the same intervention area (14 geographical clusters) and general program structure. Households targeted for the FA program were also included in the RUSF component if they were located in one of the 7 FA + RUSF clusters, and had a child who fit the admission criteria. The FA program consisted of 5 monthly distribution sessions; at these same sessions, RUSF was distributed to qualified households.
In terms of staff, each program component had its own dedicated personnel, including a Head of Project (along with an assistant in the RUSF component) who was responsible for managing and overseeing activities within their program component. Additionally there were some shared staff who implemented activities common to both interventions; these included supervisors, distribution monitors and a community mobilizer who sensitized the community about the entire program.
While the RUSF component benefited from sharing some of the infrastructure of the FA program, given its different focus it also required separate trainings for dedicated staff, and additional supplies (e.g. MUAC strips, sensitization materials for beneficiaries receiving RUSF, etc.). Other resources were employed specifically for the RUSF component, related to anthropometric measurement, including staff and equipment, and several surveys.
The RUSF component of the program was directly related to the operational research; therefore many program staff spent a portion of their time on research-related activities. Time spent on research was estimated and excluded so that this analysis would only account for time spent implementing the project.
Analytical strategy
This study assessed the incremental cost-effectiveness of the addition of RUSF to a FA ration, for improving selected child health and nutrition outcomes. Both total FA program costs and incremental costs of the RUSF component were estimated from the societal perspective, thereby including all costs related to program implementation and participation incurred by institutions and communities. The World Food Program (WFP) provided food rations and support for the food distribution sessions. All other institutional costs were covered by ACF. Costs were calculated with a combination of accounting records and “ingredients” estimates using unit costs and quantities of inputs [
36].
Activity-based costing (ABC) is a method of cost categorization and analysis in which all costs of a program are allocated to its activities [
36,
37]. Traditional accounting cost centers organize costs by input category (e.g. personnel, medical supplies, capital costs) [
36]. Activity-based costing takes analysis of costs one step further by allocating these input costs to activity-based cost centers, representing the activities for which the input was used. This enables assessment of the specific resource use of various program activities.
An ABC methodology was applied to all costs in this analysis, including both accounting costs and ingredients estimates; this methodology was used to achieve multiple analytical goals. First, the ABC methodology assisted in separating all program costs into FA and RUSF program components, thereby facilitating an analysis of incremental costs of the RUSF component. Second, the ABC method was used to allocate costs of all inputs that were shared between the FA and RUSF program components, and for which the allocation was not straightforward (i.e. staff, vehicles, office running costs). Application of the ABC methodology will be further described in the following sections.
Effectiveness data was taken from the operational research study connected to the program [
18]. Incremental costs were calculated per case of anemia averted and episode of diarrhea per child-month.
Cost estimation
Institutional costs
Costs of inputs used in the program, which were not recorded in accounting records, were estimated with data from key informant interviews and project documentation. These included cost of storage for FA rations and RUSF, transport of rations, and staff involved in implementation and distribution support but not recorded in program accountancy. WFP provided the cost of staple rations.
Communities contributed time and resources to participate in this program. Direct and indirect beneficiary costs were estimated via key informant interviews with implementing staff familiar with local travel distances and transportation prices. Estimates were calculated for differential time spent at monthly distribution sessions by beneficiaries receiving FA and FA + RUSF respectively; and for round-trip travel to and from the distributions. It was assumed that on the way back home from the distribution point, when carrying their FA rations and RUSF, one-half of beneficiaries would take local transportation while the other half would walk home with a family member. The intervention occurred during harvest time, and many beneficiaries (40% according to program documentation and key informants) had to travel from their fields into town for the distributions. It was assumed that these beneficiaries would travel via bus for two hours one-way door-to-door to return to town from their fields.
Cases of moderate anemia identified during the program were referred to the local health clinic for treatment consisting of mebendazole, iron and folic acid supplements. Sixteen out of the 45 children referred actually accepted the referral. It was assumed that accessing this treatment would take a caregiver one-half day of travel and waiting time, that no other direct costs would be incurred, and that all other costs would be covered by the clinic. Estimates do not include any direct or indirect costs incurred by households for provision of additional care after the clinic visit.
Sites for the monthly food distributions were donated by communities (schools, personal yards, stadiums). While private properties were the only suitable distribution sites available, it was difficult for owners to give up this land for all distribution days. Costs for these sites were estimated for all 38 distribution days as the daily rental price for a room used for meetings at a private center in Abeche town (60 EUR), multiplied by 6 to account for the total space needed; this price was discounted by 25%, as a best-guess estimate to account for its being a charitable donation from the community.
Finally, 7 community members per quarter were engaged in a 3-week process to select beneficiaries. Key informants estimated that heads of quarter (n = 2 per quarter) spent 3 weeks full time, while the remaining members of this committee spent 30% of their time on this activity.
Cost of community members’ time was valued in various ways. Beneficiary time was valued using an average agricultural daily wage estimated from previous studies in different African countries and different years (Mali and Zambia: 2006; Ghana and Malawi: 2009) [
38‐
40] at 1.45 EUR per day (0.23 EUR per hour). Community committee members’ time was valued at the daily wage paid by the ACF Abeche mission to casual laborers at 6.10 EUR per 8-hour workday.
Data analysis
Assembling cost data
Accounting data was adjusted to arrive at the final estimates.
Costs for the supplementary food (RUSF) include only the food used during the program. Therefore RUSF shipping costs were adjusted to account for this lower volume. This entailed using a higher international shipping rate than was actually charged to the program to account for the higher cost per unit charged for shipping smaller amounts of product, and a lower cost for truck rental and fuel for local shipping. The potential cost savings of using locally produced RUSF were explored during sensitivity analysis.
The program incurred many research costs related to the RCT, which were excluded as they did not contribute to program outcomes. Baseline and SMART surveys produced information that informed implementation, therefore costs for these surveys were included. Costs of routine program monitoring were included. To reflect costs of a more typical program scenario, one-half of some research-related costs were included as program costs, e.g. double measurement of anthropometric indicators taken to ensure data quality for the RCT, including cost of anthropometrists, data entry clerks, wooden measuring boards, scales and batteries for scales.
Food for Training activities were initiated but not completed. The cost of these activities was included assuming that even limited exposure to training could influence beneficiary behavior and therefore program outcomes.
Cost of capital items was amortized using standard tables (3 years for computers, 5 years for other equipment), discounted at a rate of 3%, and one year’s value was included, given the program’s short duration.
Costs from the accounting system were regularly converted to Euros from Central African Francs using official bank rates, and were not adjusted for inflation as they covered less than one year. All costs are presented in 2010 EUR.
Applying activity-based costing in cost allocation
The ABC methodology was applied to all costs, including both accounting costs and ingredients estimates, and costs to both institutions and communities. During the ABC analysis, all costs were considered as “inputs” required when implementing a particular activity.
Activity-based cost centers in this analysis represent the overarching program components (FA and RUSF) which are comprised of both component activities (e.g. distributions, sensitization, training) and the inputs required for activities in each cost center (e.g. personnel, staple rations, distribution sites).
The ABC method assisted in isolating costs of the RUSF component from costs of the overarching FA program, enabling an analysis of incremental costs of the RUSF component. First, costs were allocated to the FA or RUSF cost centers based on direct utilization where possible (i.e. the cost of RUSF was allocated to the RUSF cost center). Second, shared costs, such as management, shared equipment and other program support, for which allocation was not straightforward, were allocated to cost centers separately using the activity time allocation of staff. Staff people implement programs, and their time allocation to various activities within a program is therefore intended to represent the relative “resource-intensiveness” of each activity. For example, if a program with 2 activities has a staff time allocation of 50% to each activity, then each activity requires an equal amount of staff time and, therefore, of resources for program support. Support costs included in this analysis represent those contained within the ACF accounting records for only this program, and therefore do not include costs related to supporting other programs implemented by ACF concurrently.
Key informant interviews were conducted with key implementing staff, along with management and coordination staff involved with the program, to determine the proportion of their time devoted to either the FA program or to the RUSF component and its related management and logistics. Time dedicated to research activities was excluded, and a scale factor was applied to these estimates so proportions summed to 100%. An average time allocation proportion was then calculated for each activity, to reflect the overall staff time proportion required for each activity. The effect of these proportions of support costs on relative cost-effectiveness of FA versus RUSF was explored during sensitivity analysis.
Cost analysis
Ingredient estimates were entered and analyzed, and accounting data adjusted and allocated, using Microsoft Excel software [
41]. Costs were analyzed first in terms of input category, using those categories indicated in the accounting records, and then by allocating these inputs to the activity-based cost centers.
Cost-effectiveness analysis
Cost per child was calculated using the number of children included in the program at baseline, regardless of outcome. This included 1,071 children total, a subset of whom received only FA (n = 458) and another subset who received FA + RUSF (n = 613). Cost per child of FA alone was calculated by dividing total costs of the FA program by all participating children (n = 1,071). An incremental cost per child was calculated by dividing the incremental cost of the RUSF component by the number of children receiving RUSF (n = 613).
Incremental cost-effectiveness ratios (ICERs) represent the additional cost per improved outcome achieved by the RUSF component compared to FA alone. ICERs were calculated by dividing the cost per child (calculated as total cost of program/number children in program) by the difference in morbidity outcomes (i.e. cases of anemia or episodes of diarrhea per child-month in FA or FA + RUSF/number of children in FA or FA + RUSF). Since both numerator and denominator of the ICER are divided by the number of children in the program, the ICERs represent the cost per case of child morbidity averted.
Costs and effects were modeled with TreeAge Pro 2012 software [
42], using a decision tree with two branches: one for the FA program, and one for the additional FA + RUSF component. The model assessed the incremental cost per case of anemia averted and episode of diarrhea per child-month averted in the FA + RUSF program area compared to FA alone, assuming various levels of “willingness to pay” to achieve these outcomes. Willingness to pay refers to the value of the ICER that society would consider acceptable to achieve program outcomes [
43].
Sensitivity analyses were conducted to determine whether significant changes would occur in the base case ICER estimates, given changes in various parameters of interest. In the first phase, univariate sensitivity analyses were conducted for individual variables, representing both costs and effects, over a plausible range of values. In a second phase, probabilistic sensitivity analyses were conducted with 100,000 replicates per analysis to assess variation in multiple variables simultaneously.
During sensitivity analyses, best and worst case scenarios were modeled using a range of +/− 25% on base case observations. For the diarrhea episode outcome, an additional 10 percentage points were added to the variance in the FA group to account for a loss to follow-up that was 10% higher over the course of the study than the FA + RUSF group. Diarrhea episode data was collected on a monthly basis; the loss to follow-up affects monthly incidence data, but not anemia prevalence data, which was collected at the end of the program. Loss to follow-up was experienced in the first follow-up measurement and was corrected by the third follow-up after caregivers in the FA group were given incentives (e.g. laundry soap, mosquito nets) for bringing their children to distribution sessions.
For one-way sensitivity analyses, the incremental cost per child of the RUSF program component was broken down into several elements to model various scenarios. To test the potential of locally-produced RUSF to improve cost-effectiveness, a scenario was modeled using available data on price differentials between imported and locally produced products. The “worst case” scenario used estimates from the base case analysis of RUSF imported from Nutriset (2.83 EUR/kg) and all international and local shipping costs included from the program accounting records. The best case scenario was modeled using published data from Malawi [
44], suggesting that local product would cost 1.89 EUR/kg (adjusted for inflation and currency exchange), representing a 33.2% decrease from the base case. This estimate includes duty and shipping for imported ingredients (multivitamin mix), however it may not include other costs of local production such as quality assurance efforts for aflatoxin testing, etc., and may therefore underestimate the real costs of local production. An additional scenario was modeled using a cost estimate from a local RUSF producer in Niger of 3.39 EUR/kg (Nutriset, communication to ACF), representing a 19.8% increase from the base case. For these modeled scenarios, only the difference in purchase price of local product was used; it was assumed that the local product was available in Chad, and so local shipping costs were assumed to be the same as in the base case, with no international shipping costs incurred.
Support costs allocated to the program were varied over a plausible range. The base case included 35% of support costs, based on percent of staff time allocation to the RUSF program component, in accordance with the activity-based costing methodology. The worst case scenario included 50% of support costs while the best case scenario included 15% of support costs, as an assumed plausible range.
All other program costs were modeled using a range of +/− 25% on base case observations.