What, why and how do health systems learn from one another? Insights from eight low- and middle-income country case studies

All health systems face challenges managing complex and changing health needs, with these challenges being the greatest in low-income countries due to the larger health needs, faster population growth [1] and least availability of financial resources for health [2]. At the same time, these governments are committed to progressing towards universal health coverage as part of the Sustainable Development Goals [3], within a context of more constrained development assistance [4, 5]. The use of evidence from other countries may result in health system reforms that are more efficient and effective [6–8]. This study seeks to understand policy transfer and evidence use around health systems in low- and middle-income settings in order to inform investments in improved learning between countries.

Globalisation and the activity of international organisations involved in the design, implementation and analysis of regional and domestic policies have facilitated dialogue and sharing of ideas and experiences across actors in different settings. The process of using the ideas, content and lessons from implementing policy from other countries, or what this study terms ‘learning across systems’, falls under the broader literature on policy transfer. Transfer is defined as the intentional process through which “knowledge about policies, administrative arrangements, institutions etc. in one time and/or place is used in the development of policies, administrative arrangements and institutions in another time and/or place” [9]. Unintentional emulation of policies, on the other hand, may be considered to be merely a ‘convergence’ of policy rather than a process in which one actor deliberately seeks and uses lessons from other actors [10].

A small but growing set of literature seeks to understand policy transfer processes in the health sector of low-income countries. Mechanisms of policy transfer that are identified include learning, coercion, socialisation and competition [11]. Financial assistance, identified as the most dominant form of coercion, has also led to changes in in-country policy, in many cases the adaptation of policy specifically to receive aid [12]. Significant attention in the literature has been placed on the role of international organisations, while questions around individual country-to-country transfers are not as well understood [12]. The bulk of relevant literature appeared in the 1990s and early 2000s, suggesting that research has de-prioritised this topic. This presents a missed opportunity to understand the mechanisms involved in policy transfer, especially those between low-income countries and those that are specific to the health sector.

Studies of evidence-based policy-making link to the policy transfer literature by highlighting the types of evidence that currently (or from the perspective of many researchers, should) inform policy, including systematic or scientific research, practical experience and political judgement [13]. Many also recognise that evidence is used in different ways, including instrumentally (using evidence to problem-solve in policy and to improve policy outcomes), conceptually (evidence contributes to knowledge on a particular issue) and symbolically (for example, when evidence is used by politicians to legitimise themselves or to support political claims) [14–19]. Further, it is now widely recognised that policy-makers make decisions in rational and emotional (e.g. political, value-based) ways (using ‘bounded rationality’), which require different forms of evidence [20]. Recommendations for improving the uptake of evidence include pursuing the systematic examination of research that more holistically identifies past lessons and experiences [21]; using research that targets multiple stages of the policy process, for instance, to inform agenda-setting, examining alternatives and outcomes [22]; evaluation of policies that considers political factors [23]; and giving greater attention to the institutional and capacity factors that favour uptake of evidence [24].

Drawing from eight country case studies, this article seeks to supplement existing literature by drawing on the insights and experience of policy-makers in low-income countries and assessing their demand for evidence, how it is met (or not) and what barriers they perceive to exist. It aims to understand how learning has occurred in these case studies of health policy reform and what could be done to strengthen it. It was undertaken by Oxford Policy Management to inform the Bill and Melinda Gates Foundation on priorities for investment in supporting cross-country learning on effective health system policies.

The wider study within which the case studies were nested started in early 2017 with three scoping literature reviews focussed on (1) the content of learning across health systems, in terms of which topics comparative health systems literature has addressed since 2000 and using which methods [25]; (2) a review of institutions and platforms that currently exist and aim to facilitate learning across health systems [26]; and (3) international health policy transfer studies [12]. These background reviews and meetings fed into the design and framework for analysis of the case studies, which were overseen by an expert advisory group of researchers. The study was approved by the Ethical Review Committee of the lead UK institution.

Case studies were selected from countries that were categorised as low income in 2000 and performed well in meeting Millennium Development Goal targets by 2015 (had achieved at least 1.5 on the Centre for Global Development’s health score) [27]. From those countries that met these criteria (23 in total), eight were selected as initial case study candidates on the further criteria of geographic spread, inclusion of Anglophone and Francophone African countries, and feasibility of access to appropriate interviewees. The counties selected were Bangladesh, Burkina Faso, Cambodia, Ethiopia, Georgia, Nepal, Rwanda and Solomon Islands. Five of these are currently classified as low income, while three are now lower middle income.

One to two policies or programmes (policy is used as shorthand for both in this article) were chosen per country. The policy selection criteria were simply that the reforms were undertaken within the past decade (to ensure recall by interviewees), and involved significant change to at least one health system block. The studied reforms (described further in Box 1) cover a wide range of reforms, including the Health Extension Programme (HEP) in Ethiopia, the Sector Wide Approach and community clinics in Bangladesh, Health Equity Funds and Special Operating Agencies in Cambodia, hospital privatisation and health sector financing reforms in Georgia, the Integrated Management of Maternal, Neonatal and Childhood Illness programme in Nepal, the Role Delineation Policy in Solomon Islands, health financing reforms, including community-based health insurance (CBHI, the Mutuelles de Santé) in Burkina Faso, and CBHI and performance-based financing in Rwanda.

The findings below are structured according to (1) conceptualisation; (2) uptake or implementation; and (3) further policy development, once a policy has been implemented. We then examine what respondents told us about the mechanisms of learning, which operate at international, regional and national levels. This follows the themes that emerged inductively from the interviews conducted and reflects not just findings on the specific tracer policies but also respondent’s wider comments on learning and evidence use. Finally, we present cross-cutting themes in relation to facilitators and barriers to learning, which are grouped into factors focussed on the demand for and supply of evidence.

Many of the case study findings are consistent with the broader literature on health policy transfer in low- and middle-income countries [36, 37]. Both case studies and the literature illustrate that evidence is used in conceptualisation through the identification of a problem or policy need, facilitated by relationships that exist through policy networks, and sometimes through advocacy of international agencies, and is facilitated by the alignment of goals between relevant stakeholders [9–11, 38]. However, the case studies illuminate many aspects of health policy transfer that are either differently or under-represented in current literature. These aspects include political economic factors, how policies are implemented and the types of evidence that are used to inform implementation, and the kinds of practical mechanisms that are useful for policy-makers. The mechanisms highlighted are very varied but those which are most frequently cited – study tours and face-to-face interaction with development partners – highlight the importance of experiential learning, which allows for sharing of not just technical but also political insights. This article also complements existing literature by starting from a national and sub-national perspective (not the ‘push’ approach adopted by much of the literature on how international actors can promote evidence uptake, which tends to take a normative stance) and using a range of low- and middle-income settings to draw a broader analysis.

By starting from actual policy decisions (rather than from questions about use of international evidence), we find that many of our studied policies were home-grown or at least heavily home-incubated (for example, in Ethiopia, Rwanda, Bangladesh, Nepal and Cambodia). We can speculate that this may link with their subsequent good performance, either due to higher ownership and/or a correlation between the capacity to innovate and the capacity to manage implementation well. Social factors determining the effectiveness of policies, such as cultural norms changing how maternal health policies in Nepal and community clinics in Bangladesh are implemented, were understood by policy-makers. As a result, formative and technical recommendations from international agencies that conflict with these norms are generally rejected or adjusted by policy-makers.

The commitment of ‘national elites’ to policy transfer is commonly cited throughout both literature and the case studies as crucial for the success of policy implementation [39, 40]. Elites may consist of politicians, leaders of government agencies and organisations, as well as individuals who are employed by or participate in their home government but interact with international policy communities [40]. Discussions between international actors and national elites around the Sector-Wide Approach in Bangladesh and the formation of strategic plans in the health sector in Cambodia, as well as the integral role of policy-makers in small countries like the Solomon Islands, with a high turnover of development partner staff and limited numbers of senior level officials, demonstrate that buy-in from in-country policy-makers is crucial for reforms to be adopted and subsequently for resources to be mobilised around scaling up those reforms (see also Shroff et al. [41]).

International agencies are often cited as important since they mobilise interest and resources around issues that affect when and how a policy is conceptualised [42]. Bennett et al. [11] describe the role of agencies as being between advocates and neutral facilitators in the transfer of policy. This is a theme which emerges throughout the literature as agencies either impose or neutrally act as the medium through which policy is transferred. While most criticism of international agencies in the literature centres around the issue of coercion and how agencies and their financing have been used as a means for wealthy countries to shape policy formation for their own agendas [38, 43], the case studies present a more nuanced understanding of the role of international agencies as being influencers rather than controllers of policy conceptualisation, uptake and development. The case studies acknowledge that international agencies have their own mandates and agendas that, in some cases, differ from the governments they work with; however, the impact of agencies is met with the motivations of key in-country decision-makers. This greater agency given to domestic actors may reflect the make-up of our respondents to some extent, although more likely it relates to the country selection and the bias towards ‘strong performers’. Country leadership is also not stable over time – cases like Cambodia have seen a growth from a low base to current greater national confidence. These stages will very much influence demand for and use of international and local evidence.

The case studies overwhelmingly suggest that local political and economic factors determine when and what international evidence is used and whether that use is instrumental or conceptual (symbolic use was not raised in these case studies) [14–16]. Political power often supersedes the influence of international agencies and national technical elites, having earned support from the wider public and established social groups [44]. Unlike other policy transfer stakeholders, political parties have the ability to manoeuvre both public and private (e.g. corporate) interests [45]. In addition, other stakeholders often rely on political support to influence policy decisions, including those who provide financial, programmatic, and technical services [46–48].

It is also striking that conflict or crisis had propelled reforms in the majority of our case studies (Rwanda, Ethiopia, Solomon Islands, Nepal, Bangladesh and Cambodia), presumably creating the need and motivation to innovate, as well as a momentum to reduce inertia, challenge path dependencies and willingness to risk policy errors [35]. Resource constraints were also seen as having encouraged creativity in adopting new policies in some cases.

The existence of policy networks is another mechanism that is widely identified as important in conceptualisation by both published studies and the case studies. Policy networks consisting of formal or informal relationships between governments and other policy stakeholders [49] are understood to be useful for promoting dialogue and learning between stakeholders [11, 50, 51], and are enhanced by political and social connections between decision-makers and other actors [38, 39]. As would be expected, the case studies demonstrate that, while the influence of external information is typically stronger at earlier stages of the policy cycle, i.e. conceptualisation, implementation is strongly influenced by internal learning within policy networks, although external actors, especially consultants and technical assistants, remain important for the operational stage. The case studies point towards consistent dialogue between stakeholders as a mechanism of evidence uptake in conceptualisation, for instance, through discussions and consultations which led to health financing reforms in Georgia, the facilitation of learning through professional connections between officials and development partners in the Solomon Islands, and interactions between health officials in regional meetings and study tours in Burkina Faso and Rwanda.

Other studies on health policy transfer suggest that evaluation is needed to improve dissemination of progress in health policy reform and implementation [52], follow-up and management [53], alignment of policy goals and messages across stakeholders [54], and quality of health services provided through transfer [55]. The case studies in Cambodia, Nepal, Burkina Faso, and Rwanda show that evaluation can inform conceptualisation of policy by identifying weaknesses in health policies and informing policy development from pilot project outcomes and impact evaluations. However, in some cases, evaluations were blocked for political reasons. As evaluations present a more summative judgment, they are potentially more threatening to high profile policies than feedback from continuous monitoring.

Our findings highlight the importance of continuous learning and many positive examples of institutions which are doing this in different contexts. This is an important supplement to current literature, which does not provide much insight into how continuous learning affects uptake of evidence in health policy transfer. Most of the findings highlight the importance of developing the domestic incentives and demand for evidence – areas of gap in supply of evidence were reportedly more minor by comparison, as seen from the national level, though this is not to deny on-going access barriers. Smith et al. [56], for example, analyse more than 3000 papers in almost 1000 journals dealing with global health, and conclude that only 39% of papers published in a journal have open access, and 42% of scholarly articles require a subscription, although there is an increasing wealth of evidence available in grey and open-access sources.

The case studies illustrate how evidence that is used to inform policy is not ‘systematic’ in nature, in that evidence is often not systematically collected, examined or applied. This supports the views of most practitioners and many academics [23] that the ideal type of evidence-based policy – in which policy-makers are comprehensively rational, have the ability to systematically rank policy alternatives, and prioritise robust and critically appraised evidence – is unrealistic. Cairney and Oliver [20] suggest that researchers can be most effective when combining the principles of evidence and governance. They argue that the weight of value-driven arguments can be just as important, if not more so, to policy-makers than the importance of evidence and, therefore, evidence could be packaged to accommodate policy-makers’ social, political and ideological predispositions and motives [20]. The case studies illustrate the variety of forms and processes through which evidence is used, and imply that evidence is best conceptualised as one element feeding into decisions, which are dominated by the interests and outlooks of the most influential actors. Perceived fit to local needs and context is key.

There are some important limitations to note, which include that the countries were selected as relatively strong performers which had undertaken some form of significant health system reform in recent years. The selection allowed for the study of how evidence had, or not, informed policies. However, the findings may not be representative of a wider set of countries that may be less active in policy innovation. It should also be noted that each case study was conducted in a limited time, and thus not all perspectives are reflected and included. We can therefore regard the findings as a rich snapshot, rather than as a complete account. We also highlight our inductive approach to analysis, which meant that a structured comparison of learning across systems within a pre-determined theoretical framework was not undertaken.

However, the article can inform the future development of a conceptual framework for learning health systems, which should include not just internal factors (such as alignment of actors, incentives, capacities and resources) but also openness to and mechanisms for filtering international experiences and evidence (personal, organisational and institutional, explicit and tacit, strategic) by different actors and for different purposes (strategic, political and technical). Existing frameworks do not adequately reflect the agency we found for local decision-makers, as much of the focus is on ‘push’ models, such as policy transfer (which emphasises the transfer of specific ideas) and evidence-based policy-making (which emphasises getting research into practice), both neglecting a more active role of demanding, shaping and co-creating knowledge in the local arena.

This article reviews the experiences of eight low- and low–middle-income countries which have adopted health system reforms in the past two decades. Using key informant interviews with those directly engaged in the reforms at different periods of time, it probes whether and how international policy transfer occurred, how evidence informed the different stages of the policy cycle, what mechanisms were effective for learning and what barriers and facilitators were perceived by the participants. Extra focus was placed on unpacking the role of learning from other countries throughout the reform process. The findings emphasise the agency of national and sub-national players and the importance of developing local institutions for gathering, filtering and sharing evidence, locally as well as south–south. Developing demand for and capacity to use evidence appears more significant (in terms of current barriers) than augmenting the supply internationally, although specific gap areas were identified by respondents, especially in relation to more operational and practical questions. The case studies also highlighted that, beyond an initial sharing of information, a lot of work is needed to adequately contextualise and internalise ideas in a new setting. The overall learning process (including conceptualisation, uptake and development) is a long-term and complex endeavour, in some cases taking 15 to 20 years before a lesson can be said to be ‘in action’ at a national level. The findings also highlight the importance of the local political economy in setting the parameters within which evidence is considered and the importance of trusting relationships between national and international individuals and organisations. Finally, we highlight the need for a theoretical framework within which to further analyse learning across health systems.