Cost and cost-effectiveness analysis of a community mobilisation intervention to reduce intimate partner violence in Kampala, Uganda

SASA! - An Activist Kit for Preventing Violence against Women and HIV [8] is an intervention designed by Raising Voices and implemented by Center for Domestic Violence Prevention (CEDOVIP), two non-governmental organisations based in Kampala, Uganda. The overall aim of the intervention is to prevent violence against women and reduce HIV/AIDS risk. SASA! recognises that IPV and HIV risk occurs in a community context and is influenced by attitudes, norms and power imbalances between men and women. A community mobilisation approach is used to stimulate change at individual, relationship, community and societal levels using a four stage process, represented by the acronym SASA! [9, 10].

Through each of these stages, the SASA! materials provide the framework for the delivery of mutually reinforcing messages delivered through both formal and informal sources. Thus, community members are repeatedly exposed, either directly or indirectly (for example through other community members who have attended activities) as the intervention gains momentum. Through this process, new ideas, attitudes and norms that promote more equitable relationships diffuse throughout the community and behaviours and community responses to violence are expected to shift.

In intervention communities, four types of activists delivered SASA! programming; (i) community activists (CAs), (ii) Ssengas (traditional marriage counsellors), (iii) drama groups and (iv) community based organisations. CAs and Ssengas received bi-monthly training and support from CEDOVIP staff members to plan, host or implement activities including public events such as community dramas and poster discussions, small group activities, and one-on-one ‘quick chats’ [9]. Drama group members and community based organisations also participated in regular training over the course of the intervention and contributed to the dissemination of SASA! materials and messaging through drama and participation in community events. Regular training was also provided to community stakeholders including health care workers, police and institutional leaders who incorporated the knowledge and skills gained through SASA! into their everyday work responding to violence, preventing violence, working to improve service delivery and influencing policy [9]. CEDOVIP staff members delivered all training sessions, supported activists in planning activities, acted as mentors and conducted field visits to assess and assist in the delivery of community based activities. Staff also organised public screenings of SASA! films. Raising Voices staff led the monitoring and evaluation activities including daily field visits, data entry, analysis and reporting. CEDOVIP management oversaw the overall implementation of the intervention with support from Raising Voices on a regular basis.

The SASA! approach is unique in that all activists and stakeholders are involved on a volunteer basis, receiving no stipends or payments in-kind in exchange for their involvement. This contributes to the development of a core of individuals in each intervention community with the skills necessary to continue disseminating SASA! messaging beyond the life of the intervention without the risk that once payments stop, the messaging will as well.

The SASA! study is a cluster randomised controlled trial conducted in eight communities in two administrative divisions in Kampala, Uganda. Sites were pair matched, with one from each pair randomly allocated to the intervention group and the other designated a control. Cross-sectional surveys of community members (men and women aged 18–49 years) were conducted at baseline (2007) and endline (2012) in both intervention and control sites. Over this four year time period programming was interrupted several times due to political disturbances, meaning that outcomes relate to approximately 2.8 years of intervention implementation [11]. Primary outcomes included measures of the social acceptance of gender inequality and intimate partner violence, women’s past year experience of intimate partner violence, community responses to women experiencing violence and sexual risk behaviour [11]. A cluster-level analysis compared outcome prevalence in intervention versus control communities at follow-up. The intervention showed positive impacts across all outcomes measured, as demonstrated by the adjusted risk differences presented in Table 1. For the purposes of this analysis we have used a 90 % confidence interval to characterise uncertainty around outcomes. This narrower confidence interval was used in place of the 95 % confidence intervals presented in the main trial analysis where the study was not powered to detect statistically significant changes in these outcomes. This was caused by a combination of having few clusters in the trial along with an unexpected increase in between-cluster variation in prevalence of the IPV outcomes between baseline and follow-up. Details of the study protocol [6], a full description of survey methodology and main trial results [11] are available elsewhere.

We estimated total costs related to (i) the development of the SASA! Activist Kit and (ii) implementation of the SASA! intervention in Kampala, Uganda. The time period considered for development of the SASA! Activist Kit was 2005–2008. Costing of SASA! implementation corresponds to approximately 4 years of programming from 2008–2012 and includes some start up activities conducted in 2007.

A provider perspective was adopted and both financial and economic costs considered. Resource utilisation data, including number of SASA! activities and activity inputs (equipment, supplies and personnel time) were collected retrospectively using both financial records and routine monitoring and evaluation data. Using complete financial records, a top-down costing approach was used to allocated financial costs associated with development of the SASA! Activist Kit, the SASA! Intervention and administrative and overhead costs. A bottom-up approach was used to quantify Activist time associated with implementation of community activities. Costs to individuals related to attending SASA! activities were not considered.

All costs were considered in the year in which they were incurred. Costs incurred in Ugandan Shillings were converted to US dollars using the average exchange rate [12] in the year in which expenditure occurred and converted annually to US dollars. Final costs were adjusted to 2011 US dollars using the World Bank GDP deflator [13]. Capital costs were annualised over the expected useful life for each item and a 3 % discount rate was applied throughout.

An incremental approach was used to assess the cost of developing SASA! materials over the course of three years from 2005 and 2008. Recurrent costs included: personnel costs, building and overhead, activities, supplies, and other. Other recurrent costs include technical support provided by Raising Voices to CEDOVIP, translation, artwork, film production, printing, shipping, field testing of materials and piloting activities. Capital costs included: building costs, equipment and materials development. Personnel costs were estimated based on a proportion of annual gross salary for Raising Voices and CEDOVIP staff or stipend for volunteer activists. Capital costs were valued using purchase prices obtained from financial records or replacement value estimated based on purchase price of similar items in the same financial year and annualised over their expected useful life using a discount rate of 3 % with the exception of building cost which were valued using annual rental rates. All costs were allocated toward SASA! materials development based on estimated proportion of use. Overhead costs were allocated using a simple step-down methodology whereby the cost of support services were allocated to final cost centres based on programme areas.

Overall, the cost of developing SASA! was treated as an initial investment expected to yield benefits beyond both the time periods in which costs were incurred and the specific intervention setting. As such, the total cost associated with materials development was treated as a single capital item. Annual total costs were annualised year on year over the course of development and the final total costs were annualised over 10 years using a 3 % discount rate. The final total was divided across the estimated number of roll out sites using SASA! materials annually between 2008 and 2011 (50 sites in 2008, 60 sites in 2009, 70 sites in 2010 and 80 sites 2011). The number of roll-out sites per year was estimated based on interviews on with Raising Voices staff responsible for distributing materials and providing technical support to organisations using the materials.

Implementation costs were estimated using a full economic costing approach covering the period 2007 to 2011. As with the cost of developing SASA! materials, cost categories included the same capital and recurrent costs with the addition of transport and training costs. Similarly, all costs were allocated based on proportion of use related to SASA! implementation, overhead and support costs allocated using a step-down approach and capital items were annualised over their expected useful life using a discount rate of 3 %.

As has been previously described, the SASA! intervention model relies heavily on activists to implement programming in the community on a volunteer basis. Activists also attended bi-monthly training sessions over the course of the intervention. The opportunity cost of this labour was valued using an estimated local hourly wage for temporary unskilled labour of 1500 Ugandan Shillings (2011 US$0.60) reflecting the reality that many activists were either casual labourers or not otherwise employed [14]. This is consistent with valuing volunteer time based on the market value of labour used in other community based interventions [15‐18]. The total value of this contribution was calculated using an ingredients-based approach whereby the estimated time per activity was multiplied by the number of activities and the estimated hourly wage. Community activists and local leaders were supported in conducting SASA! activities by paid staff members whose time was allocated toward intervention activities based on individual interviews and confirmed by both Raising Voices and CEDOVIP management.

All research costs were excluded, however; routine monitoring and evaluation costs were included as it is expected that these costs would also be incurred in replication and scale up.

Unit cost estimates for the economic analysis include the cost per activist supported (excluding community based organisations), cost per community member in intervention areas, cost per SASA! activity conducted. Cost effectiveness was estimated as the cost per case of physical IPV averted (year free of physical IPV).

The number of activists supported and number of activities conducted were obtained from routine monitoring and evaluation data. Monitoring data were compiled based on a combination of monthly activity planners submitted by each CA, reports of completed activities submitted by activists and quality assessment reports completed by CEDOVIP and Raising Voices staff based on observed field activities.

Monitoring data showing the number of SASA! activities conducted by activists was available from June 2009 to December 2011 based on individual reports of activities completed. The total number of activities conducted between January 2008 and May 2009 were estimated based on available data by calculating the average number of activities for the corresponding month in subsequent years and assuming that the number of activities were 40 % lower in 2008 and 20 % lower in 2009 in order to reflect an upward trend of programming over the intervention period. A similar approach was used to estimate the total number of activities conducted by Ssengas, drama groups and community based organisations where data were only available from 2010–2012. The incomplete monitoring data in the earlier part of the intervention reflects the challenges of implementing a complex community based intervention with reliant on individual activist reports. In the latter part of the intervention, a focus on routine data collection and monitoring protocols lead to substantial improvements in reporting. Reports may be biased by either over or under reporting; however, data are consistent with planned activities and field monitoring visits, making significant over reporting unlikely. Therefore, estimates presented in the base case are more likely to be an underestimate. Best and worst case scenarios are presented in sensitivity analysis.

The intervention’s ‘target population’ is considered the best proxy for number of people reached by SASA!, as the model of social diffusion on which the intervention is built means that community members may be exposed through indirect routes as well as through direct exposure to SASA! materials and activities. Indeed intervention impacts on trial outcomes were seen at the community level (not just among those directly exposed to SASA! materials or activities).

The target population for prevention of past year experience of IPV was considered to be women aged 18–49 years who had been in a regular or casual partnership in the past year, and who lived in a census enumeration area (EA) in which a SASA! community activist was operating. It should be noted that this is a conservative interpretation of the intervention’s target population - in reality, people outside this age bracket and beyond the boundaries of these enumeration areas will have been reached. However, since the trial examined intervention effects within this circumscribed population, this is the target population deemed appropriate to use for unit cost and cost-effectiveness estimates.

To estimate the target population size, we used data from both the SASA! follow-up survey and the SASA! household mapping exercise in which we listed every household in each EA in which a community activist was operating [10, 11]. The number of 18–49 year old women in the target population was estimated as the number of 18–49 year old women in the sampled households (data collected in the follow-up survey) divided by the sampling fraction (proportion of total mapped households completing the survey). Follow-up survey data was then used to estimate the proportion of these women who would have been partnered in the past year.

The measure of past year experience of physical IPV was based on a series of questions on whether the woman had experienced different physical acts by a partner – those who reported having experienced one or more of these acts in the past 12 months were coded as having past year experience of physical IPV (as described in the main results paper [11]). The number of cases averted was estimated as the difference between the number of cases that we would have expected to occur in the intervention communities in the absence of the intervention, and the number of cases actually observed in intervention communities. Observed cases were extrapolated by applying study estimates of IPV prevalence in the intervention communities, to the target population size. The adjusted risk difference in prevalence of past year IPV between intervention and control communities at follow-up was then used to estimate the additional number of cases that would have been observed in intervention communities had SASA! not been implemented (expected number).

Number of cases averted is then represented as follows:

Where R_IPV _e is the expected risk, R_IPV _o is the observed risk and Pop is the number of people in the target population.

The adjusted risk difference between intervention and control communities for IPV was calculated using a cluster-level analysis controlling for site pair, age and marital status, akin to the analysis of relative risk described in the main trial paper [11]. Ninety percent confidence intervals around the estimates of risk difference were used to calculate upper and lower bounds for the number of cases of IPV averted.

The cost effectiveness of the SASA! intervention was assessed compared to a ‘do nothing’ alternative, which in this case is represented by the costs and outcomes associated with the control groups in the intervention. The cost effectiveness ratio (CER) was calculated as follows:

A number of the parameters for both the costing and the estimation of outcomes were characterised by uncertainty. To quantify the impact of uncertainty around assumptions related to costing methodology and inputs on unit costs, univariate sensitivity analysis was performed by varying the following key parameters:

Sensitivity analysis around the impact of uncertainty in outcome parameters on cost effectiveness was performed by varying estimates of the number of cases averted (using the 90 % confidence interval around the intervention-control risk difference to determine the upper and lower bounds).

The total estimated cost of developing the SASA! Activist Kit is 2011 US$138,598 (see Table 2). Recurrent personnel made up 76 % of the total costs (2011 US$105,322). Recurrent building and overhead costs, transportation, activity costs, supplies, training and other recurrent costs made up 21 % of the total costs (2011 US$29,165). Capital costs related to materials development comprised approximately 3 % of the total cost (2011 US$4052).

The annual and total costs of SASA! implementation are presented in Table 3. The estimated total cost of delivering the intervention is approximately 2011 US$553,252. Of this total, approximately 97 % are recurrent costs and 3 % are capital costs. Personnel costs comprise 42 % of recurrent costs with the estimated economic cost of community activists’ time contributing approximately 3 % to the total.

Over the course of the intervention 351 community activists and local leaders worked to deliver approximately 11,877 SASA! activities. The estimated total cost of supporting activists to deliver programming over the course of the intervention is 2011 US$1576, or 2011 US$394 annually. The estimated cost per activity conducted is 2011 US$47 per activity (see Table 4).

The total target population (women aged 18–49 years who had been in a regular or casual partnership in the past year and who lived in a census enumeration area in which a SASA! community activist was operating) was estimated as 10,333. The total cost per person in the target population is therefore estimated to be approximately 2011 US$21, or 2011 US$5 annually. Expanding this to individuals 15 years of age or more or aged 10 and above, the estimated unit cost is 2011 US$16–18 (see Table 4).

Approximately 1202 cases of past year physical IPV were averted (90 % CI: 97–2307). This translates to an average cost of 2011 US$460 per case of past year IPV averted.

The results of univariate sensitivity analysis showed that unit cost estimates are robust to variations in key parameters (see Table 5). With the exception of the cost per activity, the lowest unit cost estimates are associated with setting the value of volunteer time equal to the financial cost (which is zero), and the highest unit cost estimates are associated with allocating the cost of SASA! materials entirely to the intervention sites.

Sensitivity analysis for the cost effectiveness analysis was extended to consider the impact of uncertainty in the measure of intervention effectiveness; these results are presented in Fig. 1. Overall, the results are not sensitive to changes in assumptions around cost and input parameters, but are highly sensitive to uncertainty around the outcome estimate. Using the upper and lower bounds of the 90 % confidence interval around the estimate of the number of cases of IPV averted the cost per case of past year violence averted ranges from 2011 US$327–982.