Commentary | Open | Open Peer Review | Published:
Studying social accountability in the context of health system strengthening: innovations and considerations for future work
Health Research Policy and Systemsvolume 17, Article number: 34 (2019)
There is a growing body of research on the role of social accountability in bringing about more accessible and better-quality healthcare. Here, we refer to social accountability as “citizens’ efforts at ongoing meaningful collective engagement with public institutions for accountability in the provision of public goods” (Joshi, World Dev 99:160–172, 2017). These processes have multiple interrelated components and sub-processes and engage a range of actors in community-driven, often unpredictable and context-dependent actions, which pose many methodological challenges for researchers. In June 2017, scientists and implementers working in this area came together to share experiences, discuss approaches, identify research gaps and consider directions for future studies. This paper shares learnings from this discussion.
In particular, participants considered (1) how best to define and measure the complex processual nature of social accountability; (2) the study of social accountability as an inherently political process; and (3) the challenges of generalising unpredictable, community-driven and context-dependent processes. Key among a range of consensus areas was the need for researchers to capture a broader range of outcomes and better understand the nuances of implementation processes in order to effectively test theories and assumptions. Furthermore, power relationships are inherent in social accountability and the research process itself. In presenting details on these deliberations, we hope to prompt a wider discussion on the study of social accountability in health programming.
Social accountability is gaining rapid acceptance as a way to address health systems inefficiencies and improve basic public health performance, including planning and service delivery, and to contribute to the attainment of the highest possible standards of health. Here, we refer to social accountability as “citizens’ efforts at ongoing meaningful collective engagement with public institutions for accountability in the provision of public goods (, p. 161).”Footnote 1 For example, the use of community scorecard processes to improve maternal health services, whereby health providers and community members assess their existing entitlements in service delivery and then prioritise the issues they face in accessing and delivering services [2, 3]. These priorities are then jointly shared in multi-stakeholder meetings with health officials, health facilities and the community to identify local solutions and actions for service improvement and to promote mutual responsibility and accountability. Other social accountability processes use a range of tools to trigger such changes, including social audits, public expenditure tracking systems, information campaigns, public hearings, participatory budgeting, and social movement to name a few. Interest in accountability has spurred a proliferation of studies on social accountability and other participatory processes in the context of health programmes. Yet, there has been less discussion on how to go about studying these dynamic and complex change processes.
The evidence on social accountability and health is growing fast. Early reviews found mixed evidence for the approach, partly due to the varied definitions of accountability used and the range of studies undertaken [4, 5]. Another set of reviews summarised the quantifiable benefits of social accountability for service improvement, service utilisation and health outcomes using more linear cause–effect measures [4, 6,7,8,9].
The definition of success encompasses more than directly measurable health-related outcomes and includes a wider range of governance outcomes such as empowerment, participation, and the responsiveness of duty-bearers [10,11,12,13,14,15,16,17]. Joshi notes the expanding number of outcomes related to social accountability “are expected to unfold, and range from immediate short-term improvements in public services, to more durable long-term changes in states and societies” (, p. 162). Evaluating a broader range of interrelated outcomes poses several methodological challenges.
In response to these methodological challenges, recent studies on social accountability in health service delivery have attempted to document and assess the complexity of activities and outcomes. Among the methodologies or mixes of methods used are randomised control trials to measure the impact on health and service delivery outcomes [2, 3], realist evaluations of social accountability programming [18, 19], process evaluations of the implementation processes , psychometric measures of governance outcomes , and Most Significant Change approaches to trace change [21, 22], all of which are applied retrospectively and prospectively. These approaches have been developed independently and present a wealth of innovation and experience in their conceptualisation, design and execution. There is, therefore, much to be learned from existing research practice.
In June 2017, researchers and implementers had the opportunity to share experiences, methodologies and findings to help guide future endeavours. WHO’s Department of Reproductive Health and Research, which includes the UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction, convened an event on studying social accountability in Geneva, Switzerland. In this forum, researchers and implementers of social accountability in the context of health, and specifically in reproductive, maternal, newborn and child health, shared experiences, challenges and successes (see Acknowledgements section for list of participants). This paper shares learnings from this discussion.
In particular, participants considered (1) how best to define and measure the complex processual nature of social accountability; (2) the study of social accountability as an inherently political process; and (3) the challenges of generalising unpredictable, community-driven and context-dependent processes.
Defining and measuring the complex processual nature of social accountability
Social accountability processes aim to support service users voice their needs and make claims to their entitlements and hold those responsible for the provision of quality services to account. Accountability processes feature multiple and interrelated components, steps and actors, with several simultaneous processes of change, triggering collective changes. This process adheres to the definition of a complex intervention [23, 24]. Therefore, the object of study is a highly unpredictable process with a range of outcomes, and it accordingly requires some methodological innovation. Difficulties in defining the boundaries of the intervention and outcomes of interest can prove challenging in the conceptualisation of research and the development of appropriate study designs. Given this complexity, the study of social accountability interventions can risk either being reduced to mechanistic formulas or dismissed as too context-specific and lacking a generalisable value that would allow replication.
One challenge is that social accountability is relational and focuses on the relationship between rights-holders and duty-bearers; therefore, interventions change and adapt as they are rolled out in specific localities and relationships . This unpredictability may lead to activities and strategies falling out of line with pre-set interventions. To address this, many researchers described ways they documented and examined implementation to better accommodate adaptive interventions. Process evaluations are increasingly being used to effectively explore fidelity, dose, reach and cost, and realist evaluation methodologies have also been used to assess fidelity, context, mechanisms, and provide lessons for replication and scale-up .
Descriptive and analytic work on implementation is further complicated by the broad and interrelated set of outcomes that together facilitate change. Outcomes may include both routine health and service delivery outcomes, alongside changes in self-efficacy, social cohesion, trust and responsiveness. Less conventional outcomes in public health, such as increased responsiveness of service providers and policy-makers or the emergence of new norms around empowerment of community members and community capabilities [18, 26], pose their own challenges because the metrics used to capture them are not standardised or widely accepted. Relatively new outcomes of interest need formative work to distil the mechanisms of effect and the most appropriate way to measure them. There have been efforts to capture the newer constructs (e.g. empowering service providers, mutual responsibility, increased confidence of service users) using psychometric measures and structured observations, which have been tested and are now moving into multi-country validation [2, 14, 20]. Others have been using approaches such as the Bellwether approach, outcome mapping and process tracing to assess changes in duty-bearers’ actions as a consequence of such interventions [3, 27, 28]. These innovations can be learnt from and built upon.
The composite nature of complex interventions presents yet more methodological challenges. Even when the outcomes are delineated, it may be difficult to attribute them to a specific part of a multi-component intervention . By their very nature, social accountability processes are not easily standardised in their design and delivery as they are highly influenced by local context. Fox’s reinterpretation of the evaluation of evidence moves beyond treating social accountability as a bounded, tactical intervention built for field experiments, toward social accountability as multi-pronged process that utilises multiple strategies to encourage collective action and voice . This means that the pathways between intervention and outcome are not linear or easy to attribute, and that a mix of methodologies and a range of data sources may be required to sufficiently describe an intervention’s effects [24, 29]. Part of the challenge is determining whether the intervention was delivered as intended .
Evaluation strategies need to respond to this complexity and researchers are developing creative solutions. A recommended approach to studying complex interventions is to develop a theoretical understanding of the change process based on existing evidence and theory, and then model the complex intervention to inform the intervention and evaluation design [24, 29, 31]. Mixed methods approaches enable the collection of data which not only describes outcomes but also the processes, shedding light on factors affecting implementation, while still measuring key effects.
A range of tools and tactics are used to support social accountability processes, from community scorecards, audio-visual documentation of rights violations, village health committees and citizen charters. They may vary in focus, looking either broadly at health systems or focusing on specific service delivery points, and they vary in engagement type from collaborative problem solving to more adversarial approaches. These diverse social accountability processes share three broad components as a part of their theory of change, namely information, collective action and official response . To date, elements of this theory of change have been evidenced, while others remain theoretical and need further work. Ideally, theories of change should be adaptive to the perceptions of participants and to changing conditions on the ground so that they are participatory and responsive.
Not only are social accountability processes unpredictable and the outcomes varied, but a host of activities, outcomes and ‘unintended outcomes’ (negative and positive) may not be recognised in many forms of data capture. Methodologies are needed to intentionally account for these; for example, the Most-Significant Change approach uses personal accounts of change to determine what is considered most significant, or diffusion of innovation research that studies the adoption of a practice among a group of stakeholders [21, 32]. Local social hierarchies and power dynamics influence the implementation of social accountability efforts. These are often unspoken, easily elude the most ‘robust’ study design and scuppering the tidiest field experiment design intervention, and speak to the value of qualitative research . Ethnographic methods, such as long-term observation, can be helpful in capturing the implicit and taken-for-granted hierarchies of daily life .
The study of social accountability as an inherently political process
The study of social accountability is never neutral as it is explicitly concerned with changing the power relationship between ‘citizens’ and the local duty-bearers, and are inherently political even if they do not intend to be. Many quantitative or clinical study designs attempt to control for and reduce complexity, which often means that the political dimensions of an intervention’s implementation are masked. These dimensions not only need to be recognised as germane to the trajectory and success of social accountability efforts, but also need to account for how the research process itself reflects or perpetuates power dynamics. Involving local political scientists in the evaluation processes can bring expertise and understanding of how power penetrates the capillaries of communities.
It is critical to acknowledge that those who actively challenge the authorities, even in research conditions, risk reprisals and social harm. The social, political and economic repercussions are substantial and, for many, disrespectful treatments may be pervasive and expected [11, 35]. Ethical research conduct should extend to these very real-life negative consequences for ‘research participants’ and requires the provision of safeguards and protections. For example, community-based organisations often set up a telephone hotline for supporters/participants when working on accountability. Fields such as anthropology and psychology commonly employ codes of practice (see the American Anthropologist Association’s Principles of Professional Responsibility and the American Psychological Association’s Ethical Principles of Psychologists and Code of Conduct). Given the political character and ethical considerations of studying social accountability, a similar code of conduct could be relevant.
Moreover, researchers could move beyond just accounting for power to examining their role in power relationships. This could entail supporting the communities that they work with, and addressing community members’ concerns and issues, even if this is by simply raising the issue with the relevant authorities, and even if those issues are beyond the study’s objective. There are always dynamics around who sets the research agenda – who decides what questions to ask, how to define impact and who is implicitly entitled to do this. There needs to be openness to the different roles that researchers, implementors and communities can play. More concrete examples of how and where this has been done in a way that considers interests of ‘researchers’, ‘implementers’ and communities affected by the issue/intervention/research, would be beneficial.
The challenge of generalising unpredictable, community-driven and context-dependent interventions
The final area of consideration at the convening was on how best to discern generalisable findings from differently situated social accountability processes that adapt to local context. This area is relevant for replication, scale-up and sustaining successes after the intervention. There was consensus that, to assess an intervention’s potential for generalisability and replication, there needs to be consensus on which elements make for success. This involves unpacking the mechanisms of effect and determining what the ‘active ingredients’ are and how they relate to each other [23, 36].
Firstly, identifying scalable ‘active ingredients’ requires moving away from cause–effect models and towards mixed methods, which can relate thick qualitative descriptions with impact measures. As such, across the board, recent well-resourced projects have successfully built intervention monitoring and/or process evaluations into studies of impact. This has varied from better documenting the exact inputs for an intervention, to more detailed process evaluations, including extensive in-depth interviews and observations. Moreover, these processes do not happen in a vacuum but rather in real world settings, where other programmes and interventions may be operating in the same area along with uncontrollable factors external to the programme. Accordingly, many studies map out existing programming as well as other contextual factors (e.g. elections) at play using political economy analysis and process mapping [18, 37,38,39]. Contexts change and context analysis should capture context over time rather than a one-off activity that provides a snapshot.
Yet, even when we better understand ‘what works’, other factors may hinder these processes being replicated, scaled-up and sustained after the intervention. There are no guarantees that the stakeholders key in supporting any scale-up would be interested in doing so or have the appetite for scaling-up processes that challenge the status quo or authorities.
To bring about accessible and better-quality healthcare, it is important to recognise both the value and complexity of social accountability interventions and operate accordingly. In doing so, we should test our theories and assumptions by building on what we know, capturing a broader range of outcomes, and accommodating unpredictable implementation processes. This work happens in the real world, with influential power relationships and high stakes for participants. By moving towards co-design and horizontal approaches, anticipating exclusion and ensuring that safeguards are in place to prevent and mitigate exclusion, the power inherent in the research process itself can change. Finally, we must acknowledge – and advocate – that this work takes time and requires context-specific adaptations.
We recognise the limits of the word ‘citizen’, which excludes non-citizens who also have the right to make claims. We use the term ‘citizen’ as shorthand to refer to all rights-claiming people.
Joshi A. Legal empowerment and social accountability: complementary strategies toward rights-based development in health? World Dev. 2017;99:160–72.
Gullo S, Galavotti C, Sebert Kuhlmann A, Msiska T, Hastings P, Marti CN. Effects of a social accountability approach, CARE's Community Score Card, on reproductive health-related outcomes in Malawi: A cluster-randomized controlled evaluation. PLoS One. 2017;12(2):e0171316 https://doi.org/10.1371/journal.pone.0171316.
Blake C, Annorbah-Sarpei NA, Bailey C, Ismaila Y, Deganus S, Bosomprah S, Galli F, Clark S. Scorecards and social accountability for improved maternal and newborn health services: A pilot in the Ashanti and Volta regions of Ghana. Int J Gynecol Obstet. 2016;135:372–9.
Gaventa J, McGee R. The impact of transparency and accountability initiatives. Dev Policy Rev. 2013;31:3–28.
Joshi A. Do they work? Assessing the impact of transparency and accountability initiatives in service delivery. Dev Policy Rev. 2013;31:29–48.
Ringold D, Holla A, Koziol M, Srinivasan S. Citizens and Service Delivery: Assessing the Use of Social Accountability Approaches in Human Development Sectors. Washington DC: World Bank; 2011.
Molina E, Carella L, Pacheco A, Cruces G, Gasparini L. Community monitoring interventions to curb corruption and increase access and quality in service delivery: a systematic review. J Dev Effect. 2017;9(4):462–99. https://doi.org/10.1080/19439342.2017.1378243.
Lynch U, McGillis S, Dutschke M, Anderson M, Arnsberger P, Macdonald G. What is the Evidence that the Establishment or Use of Community Accountability Mechanisms and Processes Improves Inclusive Service Delivery by Governments, Donors and NGOs to Communities? Systematic Review, EPPI Centre. 2013. https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=3425. Accessed 22 Mar 2019.
Molyneux S, Atela M, Angwenyi V, Goodman C. Community accountability at peripheral health facilities: a review of the empirical literature and development of a conceptual framework. Health Policy Plan. 2012;27:541–54.
Altman L, Kuhlmann AK, Galavotti C. Understanding the black box: a systematic review of the measurement of the community mobilization process in evaluations of interventions targeting sexual, reproductive, and maternal health. Eval Program Plann. 2015;49:86–97.
Fox J. Social accountability: what does the evidence really say? World Dev. 2015;72:346–36.
Edstrom J. Engaging Citizens in Health Service Delivery: A Review of the Literature. Washington DC: USAID; 2015.
Paina L, Vadrevu L, Manzoor Ahmed Hanifi SM, Akuze J, Rieder R, Chan KH, Peters DH. What is the role of community capabilities for maternal health? An exploration of community capabilities as determinants to institutional deliveries in Bangladesh, India, and Uganda. BMC Health Serv Res. 2016;16(Suppl 7):621. https://doi.org/10.1186/s12913-016-1861-0.
Joarder T, Mahmud I, Sarker M, George A, Dipankar Rao K. Development and validation of a structured observation scale to measure responsiveness of physicians in rural Bangladesh. BMC Health Serv Res. 2017;17:753. https://doi.org/10.1186/s12913-017-2722-1.
Lodenstein E, Dieleman M, Gerretsen B, Broerse JE. A realist synthesis of the effect of social accountability interventions on health service providers and policymakers’ responsiveness. Syst Rev. 2013;2:98.
Lodenstein E, Dieleman M, Gerretsen B, Broerse JE. Health provider responsiveness to social accountability initiatives in low- and middle-income countries: a realist review. Health Policy Plan. 2016;32:125–40.
Sebert Kuhlmann AK, Gullo S, Galavotti C, Grant C, Cavatore M, Posnock S. Women's and Health Workers’ Voices in Open, Inclusive Communities and Effective Spaces (VOICES): measuring governance outcomes in reproductive and maternal health programmes. Dev Policy Rev. 2017. https://doi.org/10.1111/dpr.12209.
Schaaf M, Topp SM, Ngulube M. From favours to entitlements: community voice and action and health service quality in Zambia. Health Policy Plan. 2017;35(2):289–311 https://doi.org/10.1093/heapol/czx024.
Van Belle S, Mayhew S. Public accountability needs to be enforced – a case study of the governance arrangements and the accountability practices in a rural health district in Ghana. BMC Health Serv Res. 2016;16:1.
Steyn P, Boydell V, Cordero JP, McMullen H, Habib N, Nai D, Shamba D, Kiarie U. Community and Provider Driven Social Accountability Intervention (CaPSAI) Project: Study Protocol. Australian and New Zealand Clinical Trial Registry. 2019. https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=376797&isReview=true. Accessed 27 Mar 2019.
Ho LS, Labrecque G, Batonon I, Salsi S, Ratnayake S. Effects of a community scorecard on improving the local health system in Eastern Democratic Republic of Congo: qualitative evidence using the most significant change technique. Confl Heal. 2015;9:27. https://doi.org/10.1186/s13031-015-0055-4.
Dasgupta J, Sandhya YK, Lobis S, Verma P, Schaaf M. Using technology to claim rights to free maternal health care: lessons about impact from the My Health, My Voice pilot project in India. Health Hum Rights. 2015;17:135–47.
Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation in complex public health intervention studies and the need for guidance. J Epidemiol Community Health. 2014;68(2):101–2. https://doi.org/10.1136/jech-2013-202869.
Medical Research Council. Developing and Evaluating Complex Interventions. London: Medical Research Council; 2006. https://mrc.ukri.org/documents/pdf/complex-interventions-guidance/. Accessed 26 Mar 2019.
Moncrieffe J. Relational Accountability: Complexities of Structural Injustice. London: Zed Books; 2008.
George A, Scott K, Sarriot E, Kanjilal B, Peters D. Unlocking community capabilities across health systems in low- and middle-income countries: lessons learned from research and reflective practice. BMC Health Serv Res. 2016;16(Suppl 7):631.
Miller RL, Reed SJ, Chiaramonte D, Strzyzykowski T, Spring H, Acevedo-Polakovich ID, Chutuape K, Cooper-Walker B, Boyer CB, Ellen JM. Structural and community change outcomes of the connect-to-protect coalitions: trials and triumphs securing adolescent access to HIV prevention, testing, and medical care. Am J Community Psychol. 2017;60:199–214.
Nove A, Hulton L, Martin-Hilber A, Matthews Z. Establishing a baseline to measure change in political will and the use of data for decision-making in maternal and newborn health in six African countries. Int J Gynaecol Obstet. 2014;127:102–7.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655 https://doi.org/10.1136/bmj.a1655.
Hawe P, Shiell A, Riley T. Complex interventions: how out of control can a randomised controlled trial be? BMJ. 2004;328(7455):1561–3 https://doi.org/10.1136/bmj.328.7455.1561.
Petticrew M, Rehfuess E, Noyes J, Higgins JP, Mayhew A, Pantoja T, Shemilt I, Sowden A. Synthesizing evidence on complex interventions: how meta-analytical, qualitative, and mixed-method approaches can contribute. J Clin Epidemiol. 2013;66(11):1230–43 https://doi.org/10.1016/j.jclinepi.2013.06.005.
Turcotte-Trembly AM, Gali-Gali IA, De Allegri M, Ridde V. The unintended consequences of community verifications for performance-based financing in Burkina Faso. Soc Sci Med. 2017;191:226–36.
Social science approaches for research and engagement in health policy & systems (SHaPeS) thematic working group of Health Systems Global, Regional Network for Equity in Health in East and Southern Africa (EQUINET), and Emerging Voices for Global Health, et al. Fair publication of qualitative research in health systems: a call by health policy and systems researchers. Int J Equity Health. 2016;15:98.
Marston C, Renedo A, McGowan CR, Portela A. Effects of community participation on improving uptake of skilled care for maternal and newborn health: a systematic review. PLoS One. 2013;8(2):e55012. https://doi.org/10.1371/journal.pone.0055012.
Dasgupta J. Ten years of negotiating rights around maternal health in Uttar Pradesh, India. BMC Int Health Hum Rights. 2011;11(Suppl 4):S4.
Pawson R, Tilley N. Realistic Evaluation. London: Sage Publications; 2004.
Boydell V, Neema S, Wright K, Hardee K. Closing the gap between people and programs: lessons from implementation of social accountability for family planning and reproductive health in Uganda. Afr J Reprod Health. 2018;22:73–84.
Biradavolu MR, Burris S, George A, Jena A, Blankenship KM. Can sex workers regulate police? Learning from an HIV prevention project for sex workers in southern India. Soc Sci Med. 2009;68:1541–7.
Hanson C, Waiswa P, Marchant T, Marx M, Manzi F, Mbaruku G, et al. Expanded Quality Management Using Information Power (EQUIP): protocol for a quasi-experimental study to improve maternal and newborn health in Tanzania and Uganda. Implement Sci. 2014;9:41.
The authors are writing this paper on behalf of the Community of Practice of Studying social accountability in the context of health programming and would like to acknowledge the members of the Community of Practice for their significant contributions, specifically Anayda Portela, Department of Maternal, Newborn, Child and Adolescent Health, WHO; Angela Bailey, Accountability Research Centre, American University; Anne-Marie Turcotte-Tremblay, Institut de recherche en santé publique de l’Université de Montréal; Asiya Odugleh-Kolev Health Systems and Innovation, WHO; Carolyn Blake, Sexual and Reproductive Health Unit, University of Basel; Elizabeth Allen, GOAL Uganda; Eveline Geubbels, Ifakara Health Institute; Fletcher Tembo, Independent; Heather McMullen, International Planned Parenthood Federation; James Kiarie, Department of Reproductive Health and Research, WHO; Joanna Cordero, Department of Reproductive Health and Research, WHO; Kamil Fuseini, Population Council Ghana; Marta Schaaf, Averting Maternal Death and Disability Program, Colombia University; Mary Awelana Addah, Ghana Integrity Initiative; Nanono Nulu, Reproductive Health Uganda; Niranjan Saggurti, Population Council, India; Patrick Kinemo, Sikika; Petrus Steyn, Department of Reproductive Health and Research, WHO RHR; Rajat Khosla, Department of Reproductive Health and Research, WHO; Sara Gullo, CARE USA; Suzanne Cant, World Vision; Victoria Boydell, International Planned Parenthood Federation; and Walter Flores, Centro de estudios para la equidad y gobernanza en los sistemas de salud.
The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated.
This work was produced with the support of the UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction, a cosponsored programme executed by WHO. The content is solely the responsibility of the authors and does not necessarily represent the official views of the authors’ employers or funders. Any opinion, finding, and conclusion or recommendation expressed in this material is that of the authors.
Availability of data and materials
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.