Creating spaces for dialogue: a cluster-randomized evaluation of CARE’s Community Score Card on health governance outcomes

Social accountability is defined by the World Bank as “…an approach toward building accountability that relies on civic engagement, i.e., in which it is ordinary citizens and/or civil society organizations that participate directly or indirectly in exacting accountability” [1]. The approach seeks to engage citizens in the governance of public sector services and hold governments and health service providers accountable for the quality and equity of these services. Despite general agreement that social accountability involves the intersection of citizen participation and engagement with governance or public sector oversight [1, 2], there have been limited efforts to develop theoretical frameworks for social accountability or to measure its impacts empirically [3, 4]. To address this gap, CARE developed a theory of change and a set of empirical measures for governance concepts that helps unpack this “black box” of what happens in between the implementation of a social accountability approach and health-related outcomes [5]. CARE’s theory of change posits governance outcomes along the casual pathway; these include the empowerment of community members and health workers, plus the creation or claiming of negotiated spaces in which power-holders, health workers, and community members can all interact in a safe, supportive, and equitable environment (see Fig. 1). CARE’s Community Score Card© (CSC) process seeks to facilitate and improve the quality of negotiated spaces so community members and health workers can come together to voice issues, craft solutions, and hold each other mutually accountable for improvement.

Civic participation in governance at the local level can help ensure the equity, quality, and inclusiveness of services, but only if government institutions are willing to engage the public and be responsive to demands [6]. Critical to the success of civic participation and engagement in governance is the composition of who participates, the issues around which they engage [6], and how local government accommodates and responds to this engagement [6, 7]. Although the nature of civic engagement could be adversarial, CARE’s CSC aims to facilitate collaborative engagement between service users and providers. Through this engagement, CARE aims to build relationships and trust and increase power sharing, because interpersonal relationships, trust, and power relations are highly intertwined with patients’ satisfaction and utilization of health services [8‐11] and adversarial or oppositional tactics can be detrimental to the long-term goals [12].

Thus, the CSC process intentionally promotes the strengthening of relationships between health workers and community members. The process intends to alter power dynamics and build trust by creating spaces for interaction and negotiation, so that communities and health workers can take mutual responsibility and hold each other accountable for the quality and equity of service provided. Because equity and quality often depend on who participates in negotiated space and how the voices of marginalized groups are heard [13], the CSC process emphasizes the inclusion of women, adolescents, and other frequently marginalized groups. The concerns of both community members and health workers receive equal weight in the CSC process as it strives to improve governance by building trust, power-sharing, collective efficacy, and mutual responsibility between community members and health workers.

In this paper, we use endline data from a cluster-randomized evaluation to test the effect of CARE’s CSC on a set of governance measures that our theory of change suggests will be directly influenced by the intervention and are important milestones along the causal pathway between the CSC intervention and health-related outcomes.

Similar to Malawi’s population overall, approximately 50% of the women in our sample were under the age of 25 years (see Table 2). Less than two-thirds were functionally literate. Approximately one-half had three or more living children. Similarly, about 50% lived over one hour from the nearest health facility with basic emergency obstetric care available. Reflective of the Ntcheu district, nearly all the women were of Ngoni ethnicity and were married or co-habitating with their partner.

The CSC facilitates the development of local committees or groups that enable community members to share experiences, coordinate their voices, and interact with health workers and local government officials around maternal and newborn health issues of concern to them. In communities where such committees existed prior to the intervention but were not fully functioning, the CSC supported and facilitated the reinvigoration of these committees. At endline, significantly more women in the intervention than the control communities reported the existence of these Community Action Groups or Safe Motherhood Committees (p = .003) (Table 3).

Among women in the intervention areas who were aware of the CSC intervention, active participation in the intervention was positively associated with 7 of our 13 governance constructs (Table 4), including joint monitoring and transparency as well as equity and quality. However, the trust in health workers and mutual responsibility scales were negatively associated with active CSC participation.

We further analyzed these women to evaluate the relationship between our governance measures and health-related outcomes. Several of the governance measures were significantly associated with one or more of the health-related outcomes. Eight of our 13 governance measures were significantly associated with receiving a home visit from a health worker (Table 5). Four of our governance measures that predicted home visits were also significantly associated with modern family planning use (Table 6). While six of our 13 governance measures were significantly associated with service satisfaction, four of these were negative associations indicating that women who actively participated in the intervention were more dissatisfied with the services they received (Table 7).

Trust in health workers was significantly associated with home visits (p = .02) and satisfaction with services (p < .001), but not modern family planning. Collective efficacy was similarly associated with home visits (p = .02) and satisfaction (p = .01), but not modern family planning. Joint monitoring and transparency of health services was positively associated with home visits (p = .04) and modern family planning (p = .01), but not with satisfaction. Similarly, perceiving the health advisory committees as an effective bridge between the community and the health services was positively associated with home visits (p = .03) and modern family planning (p < .01), but not with satisfaction. Interestingly, perceiving direct outcomes of collective action was positively associated with home visits (p = .001) and modern family planning (p = .05), but negatively associated with satisfaction with services (p = .009).

To our knowledge, this is the first empirical study that attempts to unpack the “black box” of what happens in between the implementation of a social accountability approach and the resulting reproductive health-related outcomes. Our theory of change suggests that a critical domain is the creation or claiming of negotiated spaces in which power-holders, health workers, and community members can all interact in a safe, supportive, and equitable environment. Therefore, we set out to develop and assess governance constructs in this domain (i.e. power sharing, trust, links between the community and health providers, quality of negotiated spaces etc.). At endline, significantly more women in the intervention than the control communities reported the existence of groups that serve as critical links between the community and facility, Community Action Groups or Safe Motherhood Committees. Our findings are consistent with previous research showing that social accountability approaches can create space for dialogue and negotiation between stakeholders [22, 23].

Among women in the intervention areas who were aware of the CSC intervention, active participation in the intervention was positively associated with 7 of our 13 governance constructs. We documented significant relationships between those who actively participated in the intervention and perceptions of equity and quality of the negotiated space. While this measure focuses on perceptions of equity, we need to do more work in the future around understanding who has a voice, who participates in these platforms, and whose voice gets taken into account in social accountability platforms such as the CSC. We also found positive relationships with governance measures of actions resulting from the negotiated space such as joint monitoring and transparency, collective action, and availability of community help.

Interestingly, however, we found CSC participation negatively associated with trust in health workers despite other evidence that the CSC improved relationships between health workers and community members [17]. Program documentation and discussions revealed some unfulfilled promises made by the District Health Management Team (DHMT) (i.e. promising an ambulance that had not yet been delivered) lowered the trust between the community and health workers. Despite the fact that some actions the DHMT was charged to fulfill (like the ambulance) would require an extended timeline and additional funds. Based on our experience in Malawi, we recommend that the CSC process place more emphasis on sharing local governments’ constraints with the community, on higher-level advocacy, and on setting realistic expectations so that failure to achieve all goals does not damage relationships.

In addition to the CSC’s influence on select governance measures, our results highlight the links between these governance measures and important health-related outcomes that the CSC improved [17]. Over half of the governance measures were positively associated with having received a home visit from a health worker including collective efficacy and trust. We tailored the items in our collective efficacy scale to focus specifically on women’s belief that the community could work together to influence issues around maternal newborn health and improve government-provided services. Others have also documented the importance of collective efficacy as part of the process to mobilize communities and improve governance [24, 25]. Similarly, trust has been cited as critical for success in the social accountability literature [26]. Trust is an important element of the patient-provider relationship, predicting crucial outcomes such as utilization of prevention services and treatment adherence [27, 28].

The four governance measures positively associated with modern family planning, including joint monitoring and transparency and collective action, all reflect actions resulting from the CSC process. Our joint monitoring and transparency scale is an important governance measure because of its ability to gauge perceptions about the ability of all participants – community members, health workers, and local authorities – to voice concerns and make plans to improve services. Similarly, community collective action has been identified as an important influence on health worker accountability [29]. Here, our collective action measure focuses specifically around maternal newborn health concerns.

The relationship between our governance measures and women’s satisfaction with services is more complicated than with home visits or modern family planning as several measures were negatively associated with service satisfaction. Satisfaction is an important outcome for social accountability interventions because a woman’s satisfaction with services drives her health behaviors and service utilization [9, 30]. Women who indicated that there were meetings in which community members, health workers and local government officials discussed problems and made improvement plans were less likely to be satisfied with health services than their counterparts. Perhaps these meetings highlighted service problems and areas that needed improvement which in turn negatively influenced women’s perceptions of and satisfaction with services. As a result of participation in these meetings, women may no longer be satisfied with the status quo of health services because they know that they have a voice and that there is an opportunity to improve. Importantly, however, trust in health workers was positively associated with service satisfaction, similar to others’ findings [27, 31]. Further research needs to unpack this relationship between governance and satisfaction with services. Given the difficulty in measuring satisfaction accurately, our measure of satisfaction may need further refinement.

A few of our governance measures – power-sharing, mutual responsibility, and Community Action Groups or Safe Motherhood Committees providing maternal newborn support – did not show significant relationships with any of our health-related outcomes. One possibility for this finding is that these measures may reflect long-entrenched power and relationship dynamics between the community and health system that may take longer to change before women in the community perceive that they are on equal footing with health workers. Another possibility is that we may need to refine these measures. Even though the reliabilities were within the acceptable range, power sharing and mutual responsibility did have the lowest Cronbach’s α of all our scales. Mutual responsibility (α = .65) performed nearly identically at baseline (α = .64) [5], suggesting that there is indeed room to enhance measurement of this concept. Through the CSC process, the role of the community, health workers, and the DHMT is clarified. Perhaps, the CSC process clarified for women legal responsibility for various aspects of care and where ultimate responsibility lies, but these nuances around women’s perception of power sharing and mutual responsibility need to be refined.

The CSC process appears to influence key governance-related constructs that we theorize are important milestones (or causal links) between the intervention and major health outcomes. CARE’s CSC intervention enhanced these governance constructs by creating and facilitating safe, inclusive, and equitable negotiated spaces. Community members, health workers, and local government officials collaborated within these spaces to achieve higher-quality, more transparent, and more accountable health services. Active participation in the CSC ensures a safe, inclusive space to voice concerns and work together to improve health services and outcomes. These spaces enable interpersonal exchanges between community members and health workers so that together they can affect change. In order to understand the true impact of social accountability and how it works to improve health outcomes, we suggest measuring the governance processes affected by such interventions. We also need continued research to measure the sustainability of the intervention and these outcomes, especially after CARE steps back from its facilitation role in the CSC. We encourage those who evaluate social accountability programs to employ similar governance measures; knowledge of how these interventions change the relationships between community members and health workers can, ultimately, help to improve health services.