Skip to main content

Advertisement

Embedding implementation research to enhance health policy and systems: a multi-country analysis from ten settings in Latin America and the Caribbean

Article metrics

Abstract

Background

Progress towards universal health coverage requires health policies and systems that are informed by contextualised and actionable research. Many challenges impede the uptake of evidence to enhance health policy implementation and the coverage, quality, efficiency and equity of health systems. To address this need, we developed an innovative model of implementation research embedded in real-world policy and programme cycles and led directly by policy-makers and health systems decision-makers. The approach was tested in ten settings in Latin America and the Caribbean, supported under a common funding and capacity strengthening initiative. The present study aims to analyse ten embedded implementation research projects in order to identify barriers and facilitators to embedding research into policy and practice as well as to assess the programme, policy and systems improvements and the cross-cutting lessons in conducting research embedded in real-world policy and systems decision-making.

Methods

The multi-country analysis is based on the triangulation of data collected via three methods, namely (1) document review, (2) an electronic questionnaire and (3) in-depth interviews with decision-makers. Data from the document review was charted and narratively synthesised. Data from the questionnaire was used to assess three characteristics of the decision-maker’s participation in embedded research, namely (1) level of engagement in different stages of research; (2) extent to which their capacities to conduct and use research were developed; and (3) the level of confidence in undertaking implementation research activities. Interview data was analysed using a thematic approach.

Results

The main barriers to effective delivery or scale-up of health interventions identified in the research projects were inadequate financing, fragmentation of healthcare services and information systems, limited capacity of health system stakeholders, insufficient time, cultural factors, and a lack of information. Decision-makers’ experience in embedded research showed strong engagement in protocol development, moderate engagement in data collection and low engagement in data analysis. The in-depth interviews identified 17 facilitators and 8 barriers to embedding research into policy and systems. The principal facilitating factors were actionability of findings, relevance of research and engagement of decision-makers, whereas the main barriers were time and political processes. In Argentina, the research led to the development of new monitoring indicators to improve the implementation of the perinatal health policy, while in Chile, empirical findings supported the establishment of a training programme on reproductive rights, targeted to municipal health facilities.

Conclusions

This multi-country analysis contributes to the evidence base for the embedded research approach to support health policy and systems decisions-making. Embedding research into policy and practice stimulates the relevance and applicability of research, while promoting decision-makers’ engagement and likelihood to use research evidence in policy-making and health systems strengthening.

Contributions to the literature

  • Improving implementation and scale-up of effective health interventions is critical to support universal health coverage schemes globally.

  • Co-production of implementation research by researchers and end-users has the potential to reduce wastage of research and enhance evidence-informed policy and practice.

  • Empirical knowledge on the impact and processes of co-production approaches remains limited.

  • This multi-country study suggests that implementation research led by decision-makers and embedded into policy and practice stimulates the relevance and applicability of empirical findings, while promoting decision-makers’ engagement and likelihood to use evidence for implementation improvements.

  • Key facilitators to embedding research into policy and practice include actionability of findings and relevance of implementation research questions, whereas the main barriers include policy implementation timeframes and complex political processes.

  • Embedding research to support implementation raises questions around the competing interests of stakeholders and the need for capacity strengthening, particularly in low- and middle-income countries.

Introduction

Progress towards universal health coverage and the health-related Sustainable Development Goals requires health policies and systems informed by timely and relevant research. The use of empirical knowledge is a critical component of evidence-informed policy-making [1], yet many challenges impede the uptake of research in health policy development and implementation [2]. Not least is the conduct of research that is non-responsive to policy-makers’ priorities and poorly aligned with the needs of real-world decision-makers [3]. In recent years, various strategies have been developed and tested to bridge the gap between research and policy, largely focusing on knowledge translation, synthesis and exchange. Although some strategies have yielded positive results, conventional knowledge translation approaches have been criticised for failing to adequately consider the complexity of health decision-making processes and the inherent power dynamics [4, 5]. The prevailing practices are necessary but not sufficient to bridge the knowledge-to-action gap and to effectively support evidence-informed decision-making

Greater attention is now provided to the collaboration between researchers and knowledge users, including patients, providers, policy-makers and health system decision-makers [6,7,8]. There is increasing consensus on the importance of engaging stakeholders, on the premise that co-design and co-production of research has the potential to improve the alignment of science and policy processes and reduce wastage of research [9].

Demand-driven research fosters the relevance of empirical work and increases the propensity of policy-makers to use research findings to support health policy planning and implementation [10, 11]. Yet, most engagement approaches put forth a limited vision of collaboration restricted to dialogues and ad hoc communications [9], and there is a dearth of knowledge on effective stakeholder engagement approaches and determinants [12]. The knowledge gap also concerns good practices to engage policy-makers in the conduct and use of health policy and systems research in various settings, particularly in low- and middle-income countries (LMICs).

In order to improve the policy relevance of research, the Alliance for Health Policy and Systems Research (henceforth ‘Alliance’), an international partnership hosted by WHO, developed an innovative model of research embedded in real-world policy and programme cycles and led directly by decision-makers. The Alliance partnered with PAHO to spearhead the embedding of implementation research in support of health policies and programmes in the Americas. The initiative, entitled Improving Programme Implementation through Embedded Research (iPIER), was implemented in 2014–2015 in Latin America and the Caribbean [13]. One of its crucial components is the focus on implementation research ‘embedded’ (or integrated) in health policy and programme processes. For the purpose of this initiative, ‘embedded’ research corresponds to the engagement of policy-makers and implementers as leaders of the research and their involvement in all phases of the empirical endeavour. The innovative approach placed policy-makers and programme managers in the position of co-principal investigators of the research, with the objective of stimulating demand-driven empirical work [14]. The collaboration is designed to increase the policy-relevance of research questions being addressed and enhance policy-makers’ and implementers’ ownership of research. Policy-makers and health system decision-makers are the actors who are the best positioned to ensure that findings are integrated in real-time to support the development and implementation of health policies and the performance of health systems.

Embedded implementation research aims at scientifically studying the implementation of health interventions, including policies, programmes and services, in different real-world settings and within the existing range of health systems [15]. Engaging closely with decision-makers in a meaningful and timely manner also bears the potential to generate feasible recommendations and increase the use of research to support implementation and health policy-making [6, 9, 16, 17].

The approach was tested in ten settings in Latin America and the Caribbean, supported under a common funding and capacity strengthening initiative. The embedded implementation research teams were selected after an open competitive call for proposals managed by the Alliance and PAHO. The research projects were aimed at supporting existing policies, programmes and interventions using implementation research embedded in real-world decision-making. As such, the focus was not on developing external interventions to enhance the performance of health systems, but rather to use empirical tools and findings to enhance the implementation of active interventions or policies already developed and prioritised by decision-makers.

The teams benefited from training on implementation research and health systems research approaches through a protocol development workshop (Washington, DC, September 2014) and a data analysis workshop (Rosario, Argentina, 2015), where the decision-makers and researchers leading the projects were present. Ongoing scientific and technical support was also provided by the Instituto de Efectividad Clinica y Sanitaria (IECS, Buenos Aires, Argentina), acting as the iPIER technical assistance centre.

The objective of this multi-country study is to analyse embedded implementation research projects in order to identify facilitators and barriers as well as the policy, programme and systems improvements and cross-cutting lessons in conducting research embedded in real-world policy and systems. Our study focuses on ten embedded implementation research projects conducted in eight Latin American and Caribbean countries, namely Argentina, Bolivia, Brazil, Chile, Colombia, Mexico, Peru and Saint Lucia.

Methods

Over the course of the embedded implementation research initiative, a multi-disciplinary team from the Alliance, PAHO and IECS carried out a continuous monitoring and evaluation of the process of the research endeavours. We periodically collected information on the research activities as well as the use of research findings to improve the implementation of the health policies and programmes under study. The multi-country analysis is based on the triangulation of data collected via three methods — (1) document review, (2) electronic questionnaire and (3) in-depth interviews with decision-makers.

Document review

We conducted a document review to synthesise and chart the data in relation to the embedded research process and impact of the implementation research. Data was extracted from the research protocol, routine progress reports and the final technical report for each project submitted to the Alliance during their implementation between 2014 and 2016 as well as from scientific papers published in 2017 based on the studies [18,19,20,21,22,23,24,25,26,27]. Therefore, five documents were reviewed for each study, making a total of 50 documents, representing all available project documentation from the studies. The document review was conducted by the assessment team (AM, VE) in 2016 and 2017. Data was extracted using a data extraction form that included the headings that are provided in Tables 1 and 2, such as the research question, implementation research variables, implementation research strategies and policy/programme impact, among others. Data extracted were then directly validated in writing on two occasions by the decision-makers and researchers involved in the research in 2016 and 2017. We narratively synthesised the results to highlight key empirical findings and characteristics of the embedded research process as well as policy, programme and health systems improvements.

Table 1 Characteristics of embedded implementation research projects in Latin America and the Caribbean
Table 2 Embedded implementation research findings and health policy/programme impact

Self-administered questionnaire

A standardised electronic questionnaire that contains a series of Likert scale questions was sent to ten decision-makers that were purposefully selected for their involvement in the embedded implementation research projects. Respondents were approached to participate in the questionnaire through email including a link to the survey. Eight respondents completed the survey during January 2016, leading to a response rate of 80%. The survey included 28 Likert-scale questions to assess (1) the level of engagement in different stages of the embedded research; (2) the extent to which their capacities to conduct and use research was strengthened; and (3) the level of confidence in undertaking implementation research activities. All survey responses were included in the analysis.

In-depth interviews with stakeholders

We conducted post-hoc in-depth interviews with decision-makers who were purposefully selected for their involvement in the research projects. Respondents were approached through email and invited to participate in the interviews. A semi-structured interview guide was used that included a series of open-ended questions on the experience and perceptions of carrying out the embedded research projects throughout the research process, including facilitators and barriers. The questions for discussion during the interviews are presented in Additional file 1. The thematic areas under consideration included (1) conceptualisation and conduct of the research, (2) embedded research approach, (3) uptake of research findings and resulting impact, and (4) perceptions of research.

One interview with each decision-maker (n = 10) was conducted by at least two members of the assessment team (AM, VE, LR) using teleconference services, at a time and place agreed upon with the respondent — in most cases, their place of work during working hours. In some cases, the researcher participated in the interviews. Prior to starting the interview, AM had no previous relationship with the respondents, while VE and LR were involved with management of the initiative at the time of the study. The assessment team has received training in qualitative approaches and methods through their education and employment. Respondents were provided with information about the research, including the purpose and reasons for doing the research, prior to obtaining oral consent to participate. Each interview lasted approximately 1 hour and took place between November 2015 and April 2016. The interviews were audio recorded, directly transcribed and translated into English. Notes were taken and recorded during the interviews to summarise and ensure understanding, emphasise important points made during the interviews and identify areas for follow-up questions. The translated transcriptions were uploaded into Nvivo 11 for data management and analysis. Interview data was analysed and coded by one member of the assessment team (AM) using an inductive thematic approach to identify emerging themes derived from the data on the facilitators and barriers to implementation of the embedded research projects [28, 29]. We also put forth a participant validation approach by sharing the results and discussion sections of the study with the country respondents in 2018 to explore the credibility of findings and check for resonance with their experiences. Given the small sample size, data saturation was not assessed as all decision-makers engaged in the initiative were approached and the findings are meant to contribute a spectrum of experiences and perceptions for further discussion as the embedded research approach is becoming more common. Qualitative findings were reported following the Consolidated criteria for reporting qualitative research (COREQ) guidelines (see Additional file 2). This study was approved by the Ethical Review Committee of PAHO and WHO (PAHO-2016-02-0005).

Results

Document review

The characteristics of the embedded implementation research projects are provided in Table 1, which presents the validated results from the document review, and a summary is provided below. The majority of research projects were conducted at national level and consisted of mixed methods implementation research studies. The key issues studied focused on maternal and child health and infectious diseases, for instance, the integration of health services to prevent and control co-infection by tuberculosis and HIV. Table 2 summarises the implementation research findings, implementation strategies, dissemination methods and impact at policy, programme and systems-levels.

The majority of studies identified strong impediments to implementation, with the main barriers to effective delivery or scale-up of health interventions identified being inadequate financing, fragmentation of healthcare services and information systems, limited capacity of health professionals and health system stakeholders, insufficient time, cultural factors and lack of information. The principle facilitators reported were adequate communication incentives, training of human resources, and existence of an appropriate implementation strategy. All studies produced improvement strategies supported by the research findings, including implementation strategies (n = 5), operationalisation improvements (n = 4) and action plans (n = 2). Health policy and programme improvements involved these evidence-informed products to optimise the implementation and scale-up of health interventions.

The majority of studies reported positive impacts on the implementation of health policies and programmes in their respective health systems settings. In Argentina, for instance, implementation research conducted on the perinatal health regionalisation policy, along with a deliberative dialogue discussing the research findings, led to the development of new monitoring indicators to support policy implementation [20]. In Chile, the findings of implementation research to improve access to contraceptive services among adolescents informed the development of a training programme on gender, sexual and reproductive rights for professionals working in municipal health facilities [25].

Questionnaire findings

The questionnaire was circulated to all decision-makers involved in embedded research projects and 8 out of 10 decision-makers responded. Table 3 shows the characteristics of questionnaire respondents. Briefly, the respondents included 4 men and 4 women, and were from Argentina, Chile, Colombia, Mexico, Peru or Saint Lucia. The respondents were from the national (n = 4), provincial/state (n = 3) and municipal (n = 1) levels.

Table 3 Questionnaire respondent characteristics

The results of the questionnaire on decision-maker engagement in the embedded research approach are presented in Fig. 1. Responses showed strong engagement in protocol development (n = 6), moderate engagement in data collection (n = 3) and low engagement in data analysis (n = 5). According to the results of the questionnaire, almost two-thirds (n = 5) of decision-makers thought the study generated evidence that was useful for addressing the implementation barrier that programmes are facing. The majority (n = 5) of decision-makers felt that the options for action supported by the findings of the study were feasible to implement and could be used to improve health policies and programmes. Three-quarters (n = 6) were willing to consider the findings of the study when making decisions and/or were willing to advocate for the findings of the study to other decision-makers responsible for implementation, and the same proportion was willing to lead further research projects when facing implementation barriers.

Fig. 1
figure1

Engagement of decision-makers in the stages of embedded implementation research

Respondents reported that the embedded research approach substantially or fully built their research generation and use capacities, with the highest capacity improvements observed for ‘presenting research findings to policy and decision-maker audiences’ (n = 6), while improvements for ‘carrying out research on implementation’ and ‘elaborating options for action that are supported by the results’ were reported by four decision-makers. However, few respondents reported being ‘fully confident’ in research activities, with the greatest level of confidence reported for ‘carrying out research on implementation’ and ‘elaborating options for action that are supported by the results’. The least confidence was reported for ‘assessing failures within the health system’ and ‘bringing about changes in programmes and systems’.

In-depth interview results

All stakeholders involved as co-principal investigators in the embedded research initiative were contacted to conduct in-depth interviews. Ten decision-makers (6 men and 4 women) were interviewed, including all respondents who completed the questionnaire. As outlined in Table 4, respondents were decision-makers working at national (n = 4), provincial or state (n = 5) and municipal (n = 1) level, and represented the following countries: Argentina, Bolivia, Brazil, Chile, Colombia, Mexico, Peru and Saint Lucia.

Table 4 In-depth interview respondent characteristics

Results of the in-depth interviews with decision-makers revealed 17 facilitators and 8 barriers to embedded implementation research projects and its application in policy and practice. The most common facilitating factors and barriers are assessed below.

Facilitators

Actionability

Respondents from nine projects spoke about the actionability of the research. Six respondents spoke about using the research to inform a decision, proposal, plan or policy for implementation. One respondent said:

“This project made it possible to find out whether what we are proposing will be good or not, or if there was a better way to do it.” (Respondent from national health system level)

Four spoke about using the research to advocate for change or gain support:

The study allows us to produce evidence of where the difficulties are, the gaps at which this policy should aim, so that when we present the policy to the authorities […], it is much better to have evidence to support and back up that policy, and we have been able to get this with the study results.” (Respondent from national health system level)

Relevance

Eight respondents mentioned issues related to relevance as a driver of the research. One respondent said:

The [embedded implementation research] project [generated] the evidence that the programme needs [to learn] about how to change.” (Respondent from national health system level)

According to another respondent:

Everyone agreed [with the research] because it was in line with the work that we were doing. The issue of implementation […] was bothering us a bit […] and this research came in exactly that […] moment […] so we agreed that this was the ideal combination between research and governance so as to guide the adjustments that had to be made in the implementation.” (Respondent from provincial or municipal health system level)

Engagement

Eight respondents spoke about engagement of stakeholders; specifically, four described how the participation resulted in greater acceptability or openness to the research or change. Three respondents talked about engaging the necessary stakeholders to bring about change:

This research has made it possible for all these separate [stakeholders] to come to an agreement to improve.” (Respondent from provincial or municipal health system level)

Maintaining the attention of the authorities was also mentioned by two respondents. In describing the engagement of high-level stakeholders, one respondent said:

I think [engaging stakeholders] is obviously how we have been working and that has made it possible to maintain the attention of the authorities.” (Respondent from provincial or municipal health system level)

Barriers

Time

Six respondents spoke about issues related to time. Three respondents spoke about the challenge of time in carrying out the implementation research. One respondent mentioned that the short timeframe for the projects caused the research to be restricted, while another discussed the time it took to receive ethical approval for the project causing delay. Notably, one respondent also shared the challenge of finding the time to carry out the research and balancing the extra activity with other programmatic responsibilities. Two respondents each spoke about the challenge of time related to bringing about changes or impact and/or reaching agreement among stakeholders. As one respondent said:

The time that we may need for chang[ing] their habits and the date they have to improve […] were limited.” (Respondent from provincial or municipal health system level)

When describing the engagement of stakeholders in the research, another respondent stated:

It takes a bit of time because it is more demanding, for all of us to agree on one strategy.” (Respondent from provincial or municipal health system level)

Political process

Four respondents identified issues related to working with the government. Of these, three respondents spoke about the challenge of turn-over and government changes affecting the carry out of the work or resulting in delays. As one respondent explains:

We don’t really know which are the interests of the new government, but this is really putting some new barriers that were not thought before because we were in the same line and the same objectives with the public health and different kinds of things…and now…” (Respondent from national health system level)

Three also mentioned political influences in the government that affected the projects or use of the empirical results. In referring to how decisions are made in government, one respondent said:

There are a lot of factors that play into this kind of research and this kind of implementation of interventions, so it is difficult.” (Respondent from national health system level)

Discussion

Our assessment of the ten implementation studies conducted in the Latin America and Caribbean region provides key insights on the innovative embedded research approach tested in different health systems settings. Our study shows an overall satisfaction of decision-makers engaged in the implementation research process as well as a positive perception of embedding research in policy and systems decision-making. Across our study settings, satisfaction of policy-makers is associated with the actionability, relevance and context-sensitivity of research findings, perceived as a good fit-for-purpose to support health policy and programme implementation. The perceived usefulness of the embedded research approach in the Americas is consistent with previous evidence on the benefits of embedding research in real world policy- and decision-making in other settings [30, 31].

By promoting the active engagement of policy-makers from the onset, and by ensuring continuous policy engagement throughout the research, the embedded research model addresses two important barriers to the uptake of evidence — the engagement and ownership of end-users, and the applicability of relevant and contextualised research [2]. The embedded research approach puts forth an early engagement of policy-makers at the protocol development stage and stimulates a meaningful collaboration throughout the research cycle, thus contributing to the policy-relevance of research findings. As such, the embedded research approach in Latin American and Caribbean countries echoes previous experience showing that knowledge uptake to support health policies and systems is catalysed by on-going engagement in the research continuum as well as participation and trust among stakeholders [6, 32].

There is an increasing empirical base to support the co-production of research within the health sector, including the engagement of patients, communities, healthcare providers and health systems decision-makers [7, 8, 13, 21, 33,34,35,36,37]. The approach of embedding research into policy and practice also builds on the experience and methods put forth by the field of participatory action research. In addition, co-production of research has been spearheaded in other sectors, including environment, education and social welfare [38], speaking to its potential to advance the Sustainable Development Goals.

Engaging policy-makers in developing the research objectives and empirical questions ensures that research addresses key priorities and evidence needs for local health systems. The embedded component stimulates a demand-driven process, thus addressing recent calls for greater efforts to stimulate the generation of research questions by end-users [10, 11] and co-production of scientific evidence [9]. Furthermore, active engagement of decision-makers in the conduct of the study increases the relevance and practical application of the research outputs — both in terms of content and format — and the potential of research to bring about real-world improvements in health systems, which is a fundamental principle of implementation science [15]. In Argentina, for instance, a deliberative dialogue discussing the research findings provided recommendations to improve the implementation of the perinatal healthcare policy [20]. These collaborative processes are an opportunity to illuminate the relevance of research findings, explore the options for changes and improve the decision-maker’s perception on the usefulness of empirical evidence. Such hands-on and actionable evidence increases the likelihood that research is used to support real-world implementation and health systems strengthening.

Addressing health systems failures

By documenting the systemic problems that are contributing to the suboptimal implementation of health interventions, the embedded studies also shed light on important systemic failures in health systems arrangements and performance. In Chile, for instance, the research underlines that suboptimal implementation of the national programme for clinical practice guidelines is associated to a lack of systemic and standardised processes for evidence-informed development and management of guidelines [21]. By identifying the underlying dysfunctions in healthcare services organisation, embedded research provides critical insights on correlations between implementation barriers and health systems failures in specific LMIC contexts. In addition, documenting the complex root causes and applying a health systems lens further addresses a critical knowledge gap to understand complex adaptive health systems in LMICs [39]. Yet, our study highlights the difficulty in enhancing capacity to apply systems thinking for health systems strengthening, and the need for further investments in capacity strengthening activities in this regard. Furthermore, in the context of embedded research, the time and resources required to explore systemic factors need to be balanced with the need to produce actionable results for implementation and policy improvements.

Challenges for embedded research

Our study also shows that complex political and systems processes can at times act as an impediment to the uptake of embedded research findings. Although the scientific community is advocating for greater uptake of research in health policy and practice [10, 11], empirical findings remain only one component of complex decision-making processes, thus potentially limiting the impact of embedded research. Our study suggests that building consensus around implementation improvements informed by embedded research also requires intensive engagement and time from decision-makers, who are grappling with various competing interests. These findings corroborate the body of evidence highlighting the role of power dynamics as a critical factor in evidence-informed policy-making [40, 41].

In addition, our study highlights the need for realistic expectations of embedded research endeavours, as enhancing policy implementation and health systems strengthening require time and resources that often go beyond the lifecycle of the research. The scope of embedded research initiatives and the need to engage different stakeholders thus need to be carefully considered from the onset to strike a balance between timely improvements to the implementation of interventions and longer-term strengthening of health systems.

Furthermore, embedding research in policy and practice also raises important ethical and empirical questions about potential biases of decision-makers when assessing policy implementation or systems performance, which is a potential caveat of the embedded approach. To mitigate this process and uphold scientific validity of implementation research, our model showcases the engagement of researchers as co-principal investigators of the research project. Yet, there is a need to further document and study the dynamics of co-production schemes in health systems research, to inform good practices and to learn from the challenges in applying this approach in different contexts. The need for evidence thus pertains to greater evaluations of decision-maker engagement models in both primary research and evidence synthesis [42]. In turn, this evidence will inform further guidance on when and how to embed research in policy and practice, recognising that health systems investments also require external and independent evaluation of policy effectiveness and implementation in different circumstances [43].

Way forward in embedding research in policy and systems

Our multi-country analysis shows that action-oriented research outputs and active engagement of knowledge-users contribute to a strong buy-in of stakeholders involved in health systems strengthening at national and sub-national levels. For instance, the implementation research findings identified by the team in Peru shed light on the fragmentation of healthcare for patients co-infected by tuberculosis and HIV [19]. In turn, the results informed the development of a comprehensive model for integrated healthcare services for this vulnerable population, which is now being scaled-up nationally with additional support from the Global Fund to Fight AIDS, Tuberculosis and Malaria. The decision-makers’ favourable perception of the embedded model, including potential return-on-investment for implementation research, is also aligned with previous experiences showing that modest costs associated with researching implementation barriers can generate a magnifier effect by extending the impact of health interventions [44].

As the knowledge base on embedded health systems research remains limited, our multi-country analysis contributes to the development of this priority research agenda for global health. As such, there is a need to better assess and document critical issues of management, leadership and championship, and how they might influence the use of knowledge to support policy and programme implementation. Further knowledge is also needed to understand the optimal approach, level and intensity of engagement of decision-makers, including policy-makers and programme managers, in the conduct of all research phases, considering the resources and time required for empirical work and the competing interests of daily implementation obligations. In addition, further research is needed on the pathways through which engagement and leadership of decision-makers lead to greater uptake of research in health systems. Previous research links stakeholder involvement to higher trust and confidence in research outcomes [45]. Value-based assessment of research priorities [3] and integration of experiential knowledge of end-users [46] have also been associated with evidence use, yet more research is needed to understand the mechanisms and determinants of effective engagement approaches to strengthen evidence-informed decision-making.

Strengths and limitations

The strengths of this study include triangulation of the findings using different data sources and a reflexive member checking of results with the country participants, aiming to uphold data trustworthiness. A limitation of this multi-country analysis is the absence of critical appraisal of the implementation studies using validated tools. This embedded research experience in Latin America and the Caribbean reveals important challenges in the conduct and support of this type of empirical endeavour, not least pertaining to understanding the complexity and power dynamics of decision-making within health systems and government processes. Research addressing real-world contexts is inherently complex and requires intensive efforts to unpack and understand the political economy, structures and processes underlying health policies and programmes. Although some teams have addressed elements of complexity, for instance, Colombia’s stakeholder analysis for the implementation of clinical practice guidelines for sexually transmitted infections [18], most teams did not have sufficient time nor resources to thoroughly appraise the complex policy and health system settings. Further limitations of this study include the small sample size and a potential social desirability bias of answers provided during the in-depth interviews and questionnaires. In addition, the direct assessment of the findings by a WHO team might have influenced the interpretation of the data.

Conclusion

Our study contributes to enhance the limited empirical and evaluative evidence base on co-production of health systems research, with a view of improving the implementation of health systems interventions. Findings from the embedded implementation research initiatives provide relevant and context-specific knowledge to improve the implementation of health policies and programmes, while shedding light on the underlying performance of health systems. Our multi-country analysis further contributes to strengthening the knowledge base for the embedded approach in the field of health policy and systems research. Our study provides support on the usefulness of embedding research in health policy and systems decision-making, adding to previous requests for greater stakeholder engagement and co-production of knowledge in health systems research [47, 48]. At the same time, it highlights the challenges and potential caveats of embedding research in complex policy and systems decision-making, and the need for capacity strengthening efforts, particularly in LMICs.

The embedded research experience in the Americas also shows the interest, political will and readiness to invest in active engagement and capacity strengthening models to conduct and use health policy and systems research. By changing the traditional modus operandi of health research, the embedded research approach puts forth an innovative way of developing science and engaging decision-makers in research. As such, embedding research in health policy and programme requires a change of mindsets in both researchers and health system decision-makers. This is critical, as embedding research into real world policy and practice bears the potential to improve implementation and scale-up of effective health interventions, thus contributing to the relevance of research to support universal health coverage schemes globally.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Alliance:

Alliance for Health Policy and Systems Research

IECS:

Instituto de Efectividad Clínica y Sanitaria

iPIER:

Improving Programme Implementation through Embedded Research

LMIC:

Low- and middle-income country

References

  1. 1.

    Bosch-Capblanch X, et al. Guidance for evidence-informed policies about health systems: rationale for and challenges of guidance development. PLoS Med. 2012;9(3):e1001185.

  2. 2.

    Oliver K, et al. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14:2.

  3. 3.

    Deverka PA, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res. 2012;1(2):181–94.

  4. 4.

    Murphy K, Fafard P. Taking power, politics, and policy problems seriously: the limits of knowledge translation for urban health research. J Urban Health. 2012;89(4):723–32.

  5. 5.

    Greenhalgh T, Wieringa S. Is it time to drop the ‘knowledge translation’ metaphor? A critical literature review. J R Soc Med. 2011;104(12):501–9.

  6. 6.

    Langlois EV, et al. Enhancing evidence informed policymaking in complex health systems: lessons from multi-site collaborative approaches. Health Res Policy Syst. 2016;14:20.

  7. 7.

    Boivin A, et al. Patient and public engagement in research and health system decision making: A systematic review of evaluation tools. Health Expect. 2018;21(6):1075–84.

  8. 8.

    Olivier J, Whyle E, Gilson L. Embedded Health Policy and Systems Research: A Rapid Scoping Review. 2017; Available from: https://www.who.int/alliance-hpsr/resources/Alliance-embedded-hpsr-BriefingNote-WEB.pdf.

  9. 9.

    Greenhalgh T, et al. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q. 2016;94(2):392–429.

  10. 10.

    World Health Organization. Research for Universal Health Coverage: World Health Report 2013. http://www.who.int/whr/2013/report/en/. Accessed 27 May 2019.

  11. 11.

    World Health Organization. WHO Strategy on Health Policy and Systems Research: Changing Mindsets. 2012; http://www.who.int/alliance-hpsr/alliancehpsr_changingmindsets_strategyhpsr.pdf. Accessed 27 May 2019.

  12. 12.

    Ryan M, et al. Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol Assess. 2001;5(5):1–186.

  13. 13.

    Langlois EV, Tran N, Ghaffar A, Reveiz L, Becerra-Posada F. Embedding research in health policy and systems in the Americas. Rev Panam Salud Publica. 2017;41:e68.

  14. 14.

    Tran N, et al. Embedding research to improve program implementation in Latin America and the Caribbean. Rev Panam Salud Publica. 2017;41:e75.

  15. 15.

    Peters D, Tran N, Adams T. Implementation Research in Health: A Practical Guide. In: Alliance for Health Policy and Systems Research. Geneva: World Health Organization; 2013.

  16. 16.

    Oliver K, Lorenc T, Innvaer S. New directions in evidence-based policy research: a critical analysis of the literature. Health Res Policy Syst. 2014;12:34.

  17. 17.

    Bawah AA, et al. The child survival impact of the Ghana Essential Health Interventions Program: A health systems strengthening plausibility trial in Northern Ghana. PLoS One. 2019;14(6):e0218025.

  18. 18.

    Moreno JHR, et al. Evaluation of tools for the implementation of clinical practice guidelines on sexually transmitted infections. Rev Panam Salud Publica. 2017;41:e49.

  19. 19.

    Garcia-Fernandez L, Benites C, Huaman B. Access barriers to comprehensive care for people affected by tuberculosis and human immunodeficiency virus coinfection in Peru, 2010–2015. Rev Panam Salud Publica. 2017;41:e23.

  20. 20.

    Simioni AT, et al. Regionalization of perinatal health care in the province of Santa Fe, Argentina. Rev Panam Salud Publica. 2017;41:e38.

  21. 21.

    Herrera P, et al. Clinical practice guidelines: qualitative study of their implementation in the Chilean health system. Rev Panam Salud Publica. 2017;41:e67.

  22. 22.

    Klein K, et al. Strategy to improve access to etiological treatment of Chagas disease at the first level of care in Argentina. Rev Panam Salud Publica. 2017;41:e20.

  23. 23.

    Tinajeros F, et al. Health-worker barriers to syphilis screening in pregnant women in Bolivia's Los Andes network. Rev Panam Salud Publica. 2017;41:e21.

  24. 24.

    Velazquez M, et al. Evaluation of the teleconsultation process from the perspective of the provider (Oaxaca Telehealth Program, Mexico). Rev Panam Salud Publica. 2017;41:e22.

  25. 25.

    Ramirez GR, et al. Adolescents’ access to contraception: perceptions of health workers in Huechuraba, Chile. Rev Panam Salud Publica. 2017;41:e77.

  26. 26.

    de Paula SHB, et al. Evaluation of implementation of the protocol for managing tuberculosis/ human immunodeficiency virus coinfection in specialized care services in ceara state. Rev Panam Salud Publica. 2017;41:e48.

  27. 27.

    Alexander S, et al. Knowledge of and attitudes toward heel prick screening for sickle cell disease in Saint Lucia. Rev Panam Salud Publica. 2017;41:e70.

  28. 28.

    Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nurs Health Sci. 2013;15(3):398–405.

  29. 29.

    Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

  30. 30.

    Vindrola-Padros C, et al. The role of embedded research in quality improvement: a narrative review. BMJ Qual Saf. 2016;26(1):70–80.

  31. 31.

    Tugwell P, Knottnerus JA. Benefits of embedding researchers in a health service setting. J Clin Epidemiol. 2017;85:1–2.

  32. 32.

    Keown K, Van Eerd D, Irvin E. Stakeholder engagement opportunities in systematic reviews: knowledge transfer for policy and practice. J Contin Educ Heal Prof. 2008;28(2):67–72.

  33. 33.

    Concannon TW, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med. 2014;29(12):1692–701.

  34. 34.

    Domecq JP, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.

  35. 35.

    Shippee ND, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66.

  36. 36.

    Ghaffar A, et al. Strengthening health systems through embedded research. Bull World Health Organ. 2017;95(2):87.

  37. 37.

    Peters DH, Bhuiya A, Ghaffar A. Engaging stakeholders in implementation research: lessons from the Future Health Systems Research Programme experience. Health Res Policy Syst. 2017;15(Suppl 2):104.

  38. 38.

    Olivier JSV, Molosiwa D, Gilson L. In: Baker K, Dd S, Adam T, editors. Embedded systems approaches to health policy and systems research. In: Applied Systems Thinking for Health Systems Research: A Methodological Handbook. Maidenhead Berkshire: Open University Press; 2017.

  39. 39.

    de Savigny D, Adam T. Systems Thinking for Health Systems Strengthening. Geneva: Alliance for Health Policy and Systems Research; 2009. p. 107.

  40. 40.

    Erasmus E, Gilson L. How to start thinking about investigating power in the organizational settings of policy implementation. Health Policy Plan. 2008;23:361–8.

  41. 41.

    Sriram V, et al. 10 best resources on power in health policy and systems in low- and middle-income countries. Health Policy Plan. 2018;33(4):611–21.

  42. 42.

    Tricco AC, et al. Engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process: a scoping review. Implement Sci. 2018;13(1):31.

  43. 43.

    Peters DH. Health policy and systems research: the future of the field. Health Res Policy Syst. 2018;16(1):84.

  44. 44.

    Hales S, et al. Reporting guidelines for implementation and operational research. Bull World Health Organ. 2016;94(1):58–64.

  45. 45.

    Burton H, et al. Developing stakeholder involvement for introducing public health genomics into public policy. Public Health Genomics. 2009;12(1):11–9.

  46. 46.

    Saunders C, et al. Operationalising a model framework for consumer and community participation in health and medical research. Aust New Zealand Health Policy. 2007;4:13.

  47. 47.

    Boote J, Telford R, Cooper C. Consumer involvement in health research: a review and research agenda. Health Policy. 2002;61(2):213–36.

  48. 48.

    Elwyn G, et al. Identifying and prioritizing uncertainties: patient and clinician engagement in the identification of research questions. J Eval Clin Pract. 2010;16(3):627–31.

Download references

Acknowledgements

The authors wish to thank the participants in this study, including the co-principal investigators in each of the country teams, namely: 1) Argentina (2014): Alberto Tomas Simioni, Provincial Directorate for Child, Adolescent, Sexual and Reproductive Health; Ministry of Health of the Province of Santa Fe. Edgardo Abalos, Provincial Directorate for Child, Adolescent, Sexual and Reproductive Health; Ministry of Health of the Province of Santa Fe. 2) Argentina (2015): Karen Klein, Researcher of the Institute for Clinical Effectiveness and Health Policy (IECS), Department of Mother & Child Health Research. María Soledad Burrone, Head of the National Chagas Program, Technical Coordinator of Communicable Diseases by Vectors, Ministry of Health. 3) Bolivia (2015): Rita Revollo, Ministry of Health and Sports, La Paz, Bolivia. Freddy Tinajeros Guzmán, International Consultant on STI/HIV and AIDS, Epidemiology and Health. 4) Brazil (2014): Sheila Maria Santiago Borges, Tuberculosis Technical Area, Secretariat of State for Health of Ceará, Ceará, Brazil. Silvia Helena Bastos de Paula, Institute of Health, Secretariat of State for Health of São Paulo, São Paulo, Brazil. 5) Colombia (2014): Jaime Hernán Rodríguez Moreno, Institute of Technological Evaluation in Health, Bogotá, Colombia. 6) Chile (2014): Paloma Herrera Omegna, Department of Health Technology Evaluation, Ministry of Health, Chile. 7) Chile (2014): Giovanna Rojas Ramírez, Health Department, Municipality of Huechuraba, Chile. 8) Mexico (2014): Mauricio Velazquez, National Center of Health Technology, Secretariat of Health, Mexico City, Mexico. Miriam Silva, National Center of Health Technology, Secretariat of Health, Mexico City, Mexico. Adrian Pacheco Lopez, National Center of Health Technology, Secretariat of Health, Mexico City, Mexico. 9) Peru (2014): Lisset García-Fernández, National Health Strategy on the Control and Prevention of Sexually Transmitted Infections and HIV/AIDS. Ministry of Health, Peru. Byelca Huamán, National Health Strategy on the Control and Prevention of Sexually Transmitted Infections and HIV/AIDS. Ministry of Health, Peru. 10) St Lucia (2015): Sharon Belmar- George, Medical Officer of Health, Ministry of Health and Wellness. Alisha Eugene, Senior Medical Officer for Infectious Diseases in the Ministry of Health, Ministry of Health and Wellness, St Lucia.

We wish to acknowledge Abdul Ghaffar, Nhan Tran, Carissa Etienne, Francisco Becerra-Posada and Marcelo D’Agostino for their support and guidance in developing and implementing iPIER. Authors also wish to thank Ms. Verena Weissenbacher for her input in the background research to support this paper, as well as Ms. Yurani Sandoval for the coordination of activities pertaining to the iPIER research projects.

Funding

This study was funded by the Alliance for Health Policy and Systems Research, WHO, Geneva, with support from the Norwegian Government Agency for Development Cooperation (Norad), the Swedish International Development Cooperation Agency (Sida) and the UK Department for International Development (DfID).

Author information

EVL and AM developed the assessment design and study tools. AM, LR and VE implemented the questionnaire and conducted the in-depth interviews and AM led the qualitative data analysis. EVL and AM developed the first version of the manuscript. All authors reviewed the paper critically for important intellectual content. All authors read and approved the final manuscript.

Correspondence to Etienne V. Langlois.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Ethical Review Committee of PAHO and WHO (PAHO-2016-02-0005). The study respondents were provided with information about the research, including the purpose and reasons for doing the research prior to obtaining oral consent to participate.

Consent for publication

Not applicable.

Competing interests

EVL and AM are employed by the WHO Alliance for Health Policy and Systems Research and LR and VE are employed by the Pan-American Health Organization (PAHO). The Alliance and PAHO are supporting the embedding of research in policy and practice in different low- and middle-income countries.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Langlois, E.V., Mancuso, A., Elias, V. et al. Embedding implementation research to enhance health policy and systems: a multi-country analysis from ten settings in Latin America and the Caribbean. Health Res Policy Sys 17, 85 (2019) doi:10.1186/s12961-019-0484-4

Download citation

Keywords

  • Implementation research
  • co-production
  • embedding research
  • health systems research
  • universal health coverage
  • policy-making
  • maternal health
  • engagement
  • health systems strengthening