Skip to main content

Cross-cutting lessons from the Decision-Maker Led Implementation Research initiative

Abstract

Background

Almost 20 million children under one year of age did not receive basic vaccines in 2019, and most of these children lived in low- and middle-income countries. Implementation research has been recognized as an emerging area that is critical to strengthen the implementation of interventions proven to be effective. As a component of strengthening implementation, WHO has called for greater embedding of research within decision-making processes. One strategy to facilitate the embedding of research is to engage decision-makers as Principal Investigators of the research. Since 2015, the Alliance for Health Policy and Systems Research within the WHO and the United Nations Children’s Fund have supported decision-maker led research by partnering with Gavi, the Vaccine Alliance, in an initiative called "Decision-Maker Led Implementation Research". This synthesis paper describes the cross-cutting lessons from the initiative to further understand and develop future use of the decision-maker led strategy.

Methods

This study used qualitative methods of data collection, including a document review and in-depth interviews with decision-makers and researchers engaged in the initiative. Document extraction and thematic content analysis were applied. The individual project was the unit of analysis and the results were summarized across projects.

Results

Research teams from 11 of the 14 projects participated in this study, for an overall response rate of 78.6%. Most projects were carried out in countries in Africa and conducted at the sub-state or sub-district level. Seven enablers and five barriers to the process of conducting the studies or bringing about changes were identified. Key enablers were the relevance, acceptability, and integration of the research, while key barriers included unclear results, limited planning and support, and the limited role of a single study in informing changes to strengthen implementation.

Conclusions

Decision-maker led research is a promising strategy to facilitate the embedding of research into decision-making processes and contribute to greater use of research to strengthen implementation of proven-effective interventions, such as immunization. We identified several lessons for consideration in the future design and use of the decision-maker led strategy.

Background

Immunization is recognized as one of the most effective and economical public health interventions, preventing between two and three million deaths each year [1, 2]. Despite its promise, almost 20 million children under the age of one did not receive basic vaccines in 2019 [2]. Most of these children lived in low- and middle-income countries (LMICs) where coverage of vaccines remains lower than in high-income settings [3, 4]. For example, coverage of diphtheria-tetanus-pertussis-containing vaccine was 21% lower in low-income countries compared to high-income countries in 2019 [4]. Disparities in coverage of vaccines also exist within countries, with rural areas, urban slums, poorer households, and indigenous communities often experiencing lower coverage [5, 6].

Implementation research has gained attention as an emerging area within the field of health policy and systems research (HPSR) that is critical to strengthen the implementation of proven-effective interventions toward the achievement of international health goals [7, 8]. Implementation research is defined as the “scientific inquiry into questions concerning implementation—the act of carrying an intention into effect, which in health research can be policies, programmes, or individual practices (collectively called interventions)” [9]. Since implementation research focusses on issues of implementation that occur within real-world contexts and systems, there is significant value in engaging the actors involved in implementation in efforts to understand and strengthen it.

Since 2012, WHO has called for the greater embedding of research within health policy and programme implementation and systems strengthening efforts as a means to move closer towards universal health coverage [10, 11]. Embedding is an innovative approach to HPSR in which the research is carried out as an integrated component of decision-making processes [12]. The embedded approach is relevant to implementation research as efforts to address challenges and strengthen implementation ultimately rely on the decisions and actions of decision-makers and stakeholders operating in these systems.

Several strategies and techniques to facilitate the embedding of research have been employed, one of which is to engage decision-makers as Principal Investigators of the research (decision-maker led) [12, 13]. Since 2015, the Alliance for Health Policy and Systems Research (AHPSR) within WHO and the United Nations Children’s Fund (UNICEF) have supported decision-maker led research by partnering with Gavi, the Vaccine Alliance (GAVI) in an initiative called “Decision-Maker Led Implementation Research” (DELIR). Through this initiative, a total of 14 decision-maker led implementation research projects were completed in 10 LMICs with the aim of strengthening the implementation of immunization interventions and improving coverage.

This paper describes a study to synthesize the cross-cutting lessons from the initiative to further understand and develop future use of the decision-maker led strategy. Our objectives are: (1) to describe and summarize the projects, methods, activities, use, and perceived changes resulting from the projects; and (2) to examine the experiences and perceptions of decision-maker led research by both decision-makers and researchers engaged in the projects to identify enablers and barriers to conducting the research and using the results to bring about changes.

Methods

The study was qualitative and consisted of both a document review and in-depth interviews with decision-makers and researchers engaged in the DELIR initiative. The initiative and data collection methods for this study are described in more detail below.

Decision-Maker Led Implementation Research initiative

A full description of the DELIR initiative, its rationale, and processes is included in the editorial for this journal series [14]. For the projects supported through the DELIR initiative, the Principal Investigator had to be a decision-maker directly involved in the implementation of an immunization intervention (decision-maker led) working in collaboration with a researcher who was affiliated with a local academic or research institution. An overview of the supported projects participating in this study is provided in Table 1.

Table 1 Project summary

Document review

We conducted a review of relevant documentation from the initiative including final protocols, progress reports, final reports, policy briefs, and other research outputs for the studies shared with AHPSR, UNICEF, and GAVI.

Final protocol

Teams completed and submitted protocols based on a template. Space was made in the background section for describing the context and health system, immunization intervention, current implementation barrier and systems failure, and the knowledge that was needed to strengthen implementation. Emphasis was also placed on describing the plans for knowledge use, dissemination, and stakeholder engagement in the protocol.

Progress report

Approximately one year after the start of their projects, each team submitted a progress report. These reports included a description of the team’s activities and the progress that had been made towards achieving their study goals and outcomes. Teams also identified any challenges they were experiencing in carrying out their project and their plans for the next period.

Final report

Teams submitted a final report at the end of the research. These described all activities carried out during the project as well as the results in relation to the study goals and objectives.

Research outputs

Teams developed additional research outputs such as manuscripts, policy briefs, and presentations that were submitted along with their final reports [15,16,17,18,19,20,21,22].

In-depth interviews

Respondents for in-depth interviews were purposively selected based on their role in the project. All decision-makers serving as Principal Investigators and their researcher counterparts were approached through an email introducing them to the study and inviting them to participate (two per project totaling 28). If a selected team member was unable to participate, they were invited to refer another team member to be approached for an interview.

Interviews were conducted individually with participating decision-makers and researchers using Skype or Zoom by one of the authors (SM). SM had no prior relationship with the study teams. The interviews followed a semi-structured guide (Additional file 1: In-depth interview guide questions). Interviews were audio-recorded and directly transcribed for analysis. Notes were also taken during each interview. Each interview lasted approximately one hour. No repeat interviews were carried out and the transcripts were not returned to participants. Since the purpose of the interviews was to understand the diversity of experiences and perceptions, saturation was assessed at the time of data analysis to determine the pervasiveness of emerging themes.

Data analysis

Project documentation and interview transcripts were uploaded to NVivo 12© (QSR International, Doncaster, Australia) for organization and management. The primary unit of analysis was the project. First, characteristics of the projects were extracted and summarized across the projects by one of the authors (AM) using a data extraction form. Then, thematic content analysis was applied by AM in consultation with authors SM and ZS [23,24,25]. For the content analysis, initial codes for enablers and barriers were first developed. Factors were considered enablers when respondents identified them as contributing positively to carrying out the study or bringing about change. Factors were considered barriers when respondents discussed them as negatively influencing the study or their ability to bring about change or identified them as an aspect that could have been improved in their study. If a factor was discussed both positively and negatively by respondents, the number of respondents was used to determine whether it was considered to be an enabler or barrier and these different views were noted. This was followed by an iterative process of reading, examination, reorganization, and consultation to produce a final list of enablers and barriers. The final list included those enablers and barriers that were most pervasive in the data or most significantly influenced the process and change from the projects. This approach allowed themes to emerge from the data that were then summarized and presented across the projects.

Results

In this section, we first describe participation of the research teams in this study and provide an overview of the projects. We then identify the enablers and barriers to the process of conducting the studies and bringing about changes.

Participation of the research teams

Research teams from 11 of the 14 projects agreed to participate in this study, for an overall response rate of 78.6%. A total of 17 interviews were conducted with seven decision-makers and 10 researchers. For six projects, both the decision-maker and researcher participated in an interview. Only the researcher participated in an interview for four projects and only the decision-maker for one.

Project characteristics

The participating projects (n = 11) were conducted in seven countries in four WHO regions, mostly Africa (n = 8; 72.7%) (Table 1). The intervention (policy, programme, or service) for almost all projects was the Expanded Programme on Immunization (n = 10; 90.9%). The greatest number of projects focused on health and immunization systems as the priority issue from the call (n = 4; 36.4%). Most projects were at the substate/subdistrict level (ex. local government area, subcity, urban slum) (n = 5; 45.5%) followed by the state/district (n = 4; 36.4%).

Almost all projects were exploratory (n = 9; 81.8%) and only two were interventional (n = 2; 18.2%) (Table 2). Almost all projects employed a mixed methods study design (n = 8; 72.7%). The most frequent primary data collection methods were qualitative, including focus group discussions (n = 10; 90.9%), key informant interviews (n = 7; 63.6%), and in-depth interviews (n = 6; 54.5%). Coverage remained the most common implementation outcome (n = 7; 63.6%), but acceptability (n = 6; 54.5%), adoption (n = 5; 45.5%), and sustainability (n = 4; 36.4%) were also frequently assessed.

Table 2 Research methods

For almost all of the projects, the aim was to identify facilitators and barriers to implementation (n = 10; 90.9%) (Table 3). Seven (n = 7; 63.6%) projects focussed on describing an immunization-related problem. Only four (n = 4; 36.4%) and two (n = 2; 18.2%) projects, respectively, sought to explore potential implementation strategies or assess or test an implementation strategy. The projects described multiple challenges related to the implementation of immunization interventions (n = 7; 63.6%) and identified both demand- and supply-side barriers (n = 6; 54.5%). A few implementation strategies were explored and tested by the projects, such as a tailored communication strategy and participatory action research.

Table 3 Study results

The Ministry of Health and local governments were the most common audiences targeted for dissemination (Table 4) (n = 9; 81.8%). Meetings (n = 8; 72.7%) and dissemination workshops (n = 4; 36.4%) were the most frequent dissemination activities. Policy briefs (n = 11; 100%) and publications (n = 10; 90.9%) were the most frequently used dissemination tools. Respondents from almost all projects described perceived changes because of the project at the time of the study (n = 10; 90.9%) (Table 5). Respondents from more than half of  the projects described how the results of their research had been directly used at the time of the study (n = 6; 54.5%). The results from some projects were also used to develop tangible products to strengthen implementation (n = 5; 45.5%).

Table 4 Stakeholder engagement and dissemination
Table 5 Use and perceived changes

Enablers and barriers

In this section, we present the enablers and barriers to the process of conducting the decision-maker-led research projects, as well as bringing about changes. The project process consisted of all stages from conceptualization through implementation and dissemination, while changes referred to the outcomes of the projects in terms of the use of the findings or perceived changes resulting from the projects. In each section, enablers are presented first, followed by the barriers.

Factors influencing the project process

Enabler: existing relationship or prior experience in study setting.

Almost all decision-makers and researchers knew each other prior to the study (n = 10; 90.9%). Some research teams also had prior experience in the study area (n = 4; 36.4%). Respondents described the importance of building off these existing relationships and prior experiences to facilitate their projects. According to one respondent:

“The key aspect that really made this easy is our previous relationship and also our previous partnership. […] So, I think this environment of mutual trust and also not only mutual trust but mutual learning.” (Researcher)

Enabler: decision-maker identified problem or country priority for research.

For some projects, the decision-maker learned about the call first, identified the problem or need to be addressed, and approached the researcher to collaborate (n = 5; 45.5%). Even when the researcher initiated the study, the problem or need for research was often identified by the decision-maker (n = 3; 27.3%). According to one respondent:

“Because then you know the programme and you analyze it routinely on a day-to-day basis and you know where the implementation challenges are. So [to] really formulate your hypothesis, which is not [a] theoretical hypothesis but […] based on the practical observations and gaps that we have identified.” (Decision-maker)

When the problem was not identified by the decision-maker or the research did not address a country priority, projects experienced challenges at later stages in engendering change (n = 3; 27.3%).

Enabler: Interest in and support for decision-maker led research.

Decision-makers from all projects had prior experience or expressed interest in engaging in research. Researchers from most projects (n = 9; 90.9%) also expressed support for decision-maker led research or recognized the benefits of engaging with decision-makers in research. Both decision-makers and researchers felt that this interest and support was critical to engaging in decision-maker led research and for the success of their projects. As one respondent described:

“[Decision-makers] are experts in their own field and they are experts in the...they know what government is all about. They know what will work and what will not work. So if you get into a place and you need to do some work with the government, it is better to let them really direct you. That way you are more likely to be successful.” (Researcher)

However, some researchers also expressed concern about the appropriateness of decision-maker led research for all purposes and contexts. They described the need to consider when decision-maker led research is appropriate or not before engaging in it.

“I think it depends on […] the decision-maker and their demand for evidence and their ability to use evidence, and a bit on the context. […] So I think trying to assess what is the best context in which a decision-maker may be the best lead for research is important and in situations which the researcher will be the lead but the research to be designed in a way that also uses the capacity of the decision-maker to demand and be able to use research.” (Researcher)

Enabler: combination of research and practice.

As conditioned in the call for proposals, decision-makers and researchers had to partner together in the project to be eligible to apply for funding. In general, researchers provided scientific and technical input to the proposal and did most of the data collection, analysis, and writing. Decision-makers, on the other hand, provided a practical and context-specific perspective that was instrumental in identifying research questions, aligning the proposal with country priorities, understanding current implementation of the immunization intervention, and facilitating the research project including engaging stakeholders and disseminating the findings. Most respondents described this combination of research and practice as positive, complementary, and contributing to the success of the project. However, a few respondents also described challenges in aligning the research project with country priorities and collaborating during the project. Respondents discussed differences between decision-makers and researchers in terms of their perspectives, approaches, priorities, and interests for engaging in research.

Enabler: external stakeholder engagement and dissemination throughout study.

Most projects engaged external stakeholders other than the decision-maker or researcher throughout the study period (n = 7; 63.6%). Some disseminated their findings during the study in addition to at the end (n = 6; 54.5%). Many respondents perceived this ongoing engagement and dissemination as beneficial by aligning the research project with country priorities and needs, leveraging resources for the project, improving acceptability of the findings, and identifying new opportunities to facilitate changes following the project. As one respondent described:

“Although we thought we would disseminate at the end [in] the conventional way, the dissemination, it started [a] little earlier. […] It was very common to be asked about what we have found so far. Questions about how we can improve what is happening. And sometimes in answering those questions, we were disseminating our data.” (Researcher)

For a few projects, stakeholder engagement and dissemination were planned throughout the project as a component of the research strategy (n = 3; 27.3%). In these studies, data collection took place in multiple phases. Each phase was followed by dissemination of the findings from the previous phase. Data collection during the next phase built on what had been found in the prior phase. This enabled studies to not only identify strategies and solutions to strengthen implementation but explore stakeholders’ perceptions about them and in some cases, test whether they worked.

Barrier: limited engagement of the decision-maker or researcher throughout study.

While external stakeholder engagement was identified as an enabling factor for the project, respondents from several projects also mentioned challenges regarding meaningful, ongoing engagement with their decision-maker or researcher counterparts throughout the project. Some respondents described challenges such as scheduling time to meet with the decision-maker and meaningfully engaging them in all study activities. Decision-makers described or were perceived to have competing priorities outside of the research project as part of their routine work. Some respondents also felt that decision-makers did not have the capacity to engage in certain research activities, specifically for data collection and analysis. Respondents from a few projects also discussed challenges around the meaningful engagement of the researcher. Some respondents felt that the researchers had other projects or activities that they were leading and not enough time to commit to providing adequate support necessary for the decision-maker led project.

Barrier: limited involvement of national-level decision-makers and international partners.

Respondents from many projects felt that they would have been more successful if they had involved more national-level decision-makers and international partners in the project. This limited involvement of national-level decision-makers was perceived to limit the scope of the changes that could be made and potential scale-up following the project. While the Ministry of Health was often a stakeholder targeted for dissemination activities, only one decision-maker serving as the Principal Investigator was affiliated with the Ministry of Health (n = 1; 9.1%). Respondents from this project felt the involvement of a decision-maker from the Ministry of Health was one of the main factors that contributed to their success. According to one respondent:

“So if we could have added another decision-maker, probably from the Ministry of Health, I think that dissemination and then the [uptake] of the results would have been more national rather than limited to my setting of practice.” (Decision-maker)

Other respondents described the limited involvement of international partners, specifically the country offices of WHO and UNICEF. Respondents felt that this involvement of international partners was a missed opportunity that could have been leveraged by the projects to strengthen them and support changes.

Barrier: insufficient capacity and resources for qualitative research methods.

All projects employed some type of qualitative research methods to generate knowledge needed to inform implementation. Respondents from some projects discussed how they were unprepared for the substantial task of undertaking qualitative research methods. Respondents discussed the need for strengthening capacities and obtaining adequate resources for qualitative research methods, specifically for the analysis and interpretation of their data. As one respondent described:

“And also another challenge was in terms of data analysis. As I said, we collected a lot of data. And analysis of qualitative data was quite difficult. Maybe that is also related to our backgrounds, as that was not our background.” (Researcher)

Factors influencing changes

Enabler: relevant and acceptable findings to inform implementation.

Respondents from many projects discussed the relevance and acceptability of the findings for informing implementation of the immunization intervention. Respondents described the research findings as useful for the decision-maker to inform implementation of the immunization intervention. They also felt that the findings were applicable in the real-world context in which they were operating. Respondents also perceived the findings to be more acceptable to other stakeholders, in part because the way the research was carried out with their ongoing engagement. This was especially important when the findings informed changes that were outside the decision-maker’s individual ability to act upon and other stakeholders were critical to their success in bringing them about. According to one respondent:

“I think one of the advantages […] was we had buy-in for lack of a better word. […]. They knew about the study and it was easy to walk to them and say […] we suspect we've found one aspect that doesn't work well, and they were more receptive of critique.” (Researcher)

Enabler: integrated into decision-making processes and ongoing or routine work.

Respondents from some projects discussed the integration of their project into decision-making processes and ongoing or routine work in the country. Respondents from a few projects described how the project came about at a time when they or other stakeholders were concerned about a problem and that they undertook the study to inform their decision on how it could be addressed. Respondents from one project discussed how their study was integrated into an ongoing project that enabled them to leverage networks and resources to implement changes. Respondents from another project described how the project and changes resulting from it were perceived to be part of the routine work of health systems actors. This integration was perceived to facilitate changes informed by the findings of the study. One researcher said:

“I think having a funded project in the district also working in vaccination, in line with what we were trying to understand, this was really useful. […]. So in this implementation phase they had funding for [interventions] to increase vaccination. And they literally took all […] the results from our qualitative analysis […]. They took them all, in the [intervention].” (Researcher)

Barrier: results did not support a clear action to improve implementation.

Respondents from several projects discussed challenges when their findings did not support a clear action to improve implementation. Some respondents felt that the focus of the projects was on understanding implementation rather than exploring or testing solutions to improve it, and this focus made changes difficult to bring about. Others felt that the findings revealed aspects or barriers that were not easy to address solely within the context of immunizations. Others felt that the changes that were needed to strengthen implementation were outside of the decision-maker’s ability to address. Some of the changes relied on decisions that needed to be made by other stakeholders that had not been engaged in the implementation research project.

Barrier: limited planning or support through the process of making changes.

Many respondents discussed how there was limited planning or support through the process of making changes. Respondents discussed the need for better conceptualization and planning for change from the beginning of the projects since the aim was to inform implementation. While the specific results and changes that these results might inform could not be known prior to the study, respondents felt that further thought should have been given to the type of results that were needed, how they might be used, who might need to be engaged, and what resources might be needed for these efforts before the study started. They also felt that the project did not end when the study period was over, but rather continued through a process of informing and making changes to strengthen implementation. The continuation of the projects through this process necessitated additional time and resources that were not accounted for in the planning for the study. They also described the need for commitment or resources from various partners to support the process of making changes informed by the results of the study.

Barrier: limited role of a single study in informing changes to strengthen implementation.

Some respondents described how one project was limited in terms of making or sustaining changes to strengthen implementation. Respondents described how the findings from this project generated new research questions that needed to be answered before a decision could be made. While this project may have been used to understand the barriers to implementation, further research might be necessary on specific barriers, the strategies that could be used to overcome them, and the impact of implementing changes. They felt that multiple rounds or phases of research or funding might be necessary to strengthen implementation, fully engage with decision-makers in research, and support the institutionalization of research into decision-making processes.

Discussion

This study summarizes the study characteristics along with the experiences and perceptions of decision-makers and researchers across the research projects supported through the DELIR initiative, revealing several factors that influenced the process of carrying out the projects or contributed to their ability to bring about changes to strengthen the implementation of immunization interventions in different LMICs. The identification of these factors enabled us to deduce several lessons from across the projects. These lessons may be used to inform future use and design of decision-maker led strategies towards greater embedding and integration of research as a part of decision-making processes. This study builds upon prior calls and efforts to better understand the use of embedded implementation research across settings [8, 26].

Several factors were critical for the success of the projects. First, our study emphasized the importance of the research addressing a problem identified by the decision-maker or a country priority for research. Addressing these types of problems or priorities supported the relevance or usefulness of the research and has been identified as a facilitating factor for facilitating uptake of research and bringing about changes in other studies [26,27,28,29]. In addition to the decision-maker and researcher partnership, the engagement of external stakeholders throughout the study was also identified as important for the success of the projects. This ongoing stakeholder engagement and dissemination not only facilitated the process of carrying out the research projects but contributed to the acceptability of the findings. Acceptability of the findings has been shown by this study and others to contribute to research uptake and use [26,27,28,29]. We also identified the importance of engaging national-level stakeholders, specifically the Ministry of Health, as many respondents identified the limited engagement of these stakeholders as a limitation of their study.

Respondents identified the generation of results that did not support a clear action to improve implementation, limited planning and support throughout the change process, and the limited role of a single study in informing changes to strengthen implementation as barriers to bringing about changes. These barriers underscore the substantial challenge of engendering change within systems, especially when the changes supported by the research are beyond the scope of the project, the decision-maker leading the study, or  the stakeholders engaged to address. They highlight the importance of considering the research purpose and scope, the engagement and dissemination strategies to be employed, and planning for the change process from the conceptualization and design stages of the project. On the other hand, changes were facilitated when the project was integrated into decision-making processes and ongoing or routine work. This integration of research into the real-world context has been identified by others as an important consideration for this type of work [8]. These findings suggest that research done for the purpose of informing decisions and changes to strengthen implementation should be conceptualized and designed in a way that is responsive to the needs, activities, resources, and context in a particular setting.

Our findings also show that this collaborative approach to research enabled the projects to leverage existing relationships, networks, and prior experiences in country settings and bring together the perspectives of both decision-makers and researchers that contributed to their success. While this combination of research and practice was perceived by respondents to be beneficial, they also shared that collaboration was sometimes challenging or inappropriate. The issues that caused tension or conflict between decision-makers and researchers when engaging in research should be further explored and considered when planning decision-maker led strategies. Consideration should also be given to the appropriateness of the research question for decision-maker led research. Some types of questions may build upon decision-maker’s strengths in understanding implementation and context, while others may be more suited to different strategies of engaging with decision-makers or need to be carried out independently of decision-maker engagement.

The large number of proposals received in response to calls for decision-maker led research show the extent by interest of decision-makers in engaging in research. We show that this interest is important for the success of the projects. We also show that projects were more successful when decision-makers had prior experience in research. As more decision-makers have the opportunity to engage in research, their experience with research may grow and contribute to more meaningful engagement and capacity in research as well as better institutionalization of research as a part of their decision-making processes [11]. Similarly, researchers may have more opportunities to engage with decision-makers and recognize the benefits of engagement for their work. Further research is needed on the influence and impact of this engagement between decision-makers and researchers on the study and health system. This may include considerations such as how research questions evolved in response to the tacit knowledge and input of decision-makers, how the results of the research were or continue to be used to inform and strengthen implementation, whether the relationship between the decision-maker and researcher was sustained after the projects ended, and whether the decision-maker led strategy facilitated the longer term integration of research into decision-making processes.

This study has several limitations. First, the participation of more researchers than decision-makers as respondents for the interviews may have given greater emphasis to the researchers’ perspective. Second, the diversity of projects and experiences examined for this analysis made the identification of shared perspectives difficult. Some perspectives that were unique to individual projects may not be shared by others, but were still important to influencing the level of success in terms of bring about changes that those projects achieved. There may also be some social desirability and recall bias to the results as respondents may not have felt that they could openly talk or accurately recall their experience retrospectively. Given the time at which this study was carried out, i.e. shortly after some of the projects were completed, we may not have captured longer term changes resulting from the projects.

Conclusions

Decision-maker led research is a promising strategy to facilitate the embedding of research into decision-making processes and contribute to greater use of research to strengthen implementation of proven-effective interventions, such as immunization, toward the achievement of health goals. Our analysis of the factors that influenced the process of carrying out the projects and making changes enabled us to extract several lessons from this initiative for consideration in the future design and use of the decision-maker led strategy.

Availability of data and materials

The data generated and analysed during the study are included in this published article or available from the corresponding author on reasonable request.

Abbreviations

AHPSR:

Alliance for Health Policy and Systems Research

GAVI:

Gavi, the Vaccine Alliance

HPSR:

Health policy and systems research

LMIC:

Low- and middle-income country

References

  1. 1.

    WHO. Immunization agenda 2030: A global strategy to leave no one behind. Geneva: WHO; 2020.

    Google Scholar 

  2. 2.

    WHO. Immunization coverage. 2020. https://www.who.int/en/news-room/fact-sheets/detail/immunization-coverage. Accessed 7 Aug 2020.

  3. 3.

    United Nations Children's Fund, WHO. Immunization coverage: Are we losing ground? New York: United Nations Children's Fund; 2020.

  4. 4.

    WHO. Global Vaccine Action Plan: 2011–2020. Geneva: WHO; 2013.

    Google Scholar 

  5. 5.

    Chopra M, Bhutta Z, Blanc DC, Checchi F, Gupta A, Lemango ET, et al. Addressing the persistent inequities in immunization coverage. Bull World Health Organ. 2020;98:146–8.

    Article  Google Scholar 

  6. 6.

    Restrepo-Méndez MC, Barros AJD, Wong KLM, Johnson HL, Pariyo G, Franca GVA, et al. Inequalities in full immunization coverage: trends in low- and middle-income countries. Bull World Health Organ. 2016;94:794–805A.

    Article  Google Scholar 

  7. 7.

    Sanders D, Haines A. Implementation research is needed to achieve international health goals. PLoS Med. 2006;3(6):0719–22.

    Article  Google Scholar 

  8. 8.

    Theobald S, Brandes N, Gyapong M, El-Saharty S, Proctor E, Diaz T, et al. Implementation research: new imperatives and opportunities in global health. Lancet. 2018;392:2214–28.

    Article  Google Scholar 

  9. 9.

    Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Implementation research: what it is and how to do it. BMJ. 2013;347(f6753):1–7.

    Google Scholar 

  10. 10.

    WHO. Changing mindsets: strategy on health policy and systems research. Geneva: WHO; 2012.

    Google Scholar 

  11. 11.

    WHO. Research for universal health coverage. Geneva: WHO; 2013.

    Google Scholar 

  12. 12.

    Ghaffar A, Langlois EV, Rasanathan K, Peterson S, Adedokun L, Tran NT. Strengthening health systems through embedded research. Bull World Health Organ. 2017;95:87.

    Article  Google Scholar 

  13. 13.

    Olivier J, Whyle E, Gilson L. Embedded health policy and systems research: within the system, for the system, used by the system. Geneva: WHO; 2017.

    Google Scholar 

  14. 14.

    Shroff ZC, Mancuso AB, Sharkey A, Shahabuddin ASM, Kumar B, Johnson H, et al. Decision Maker Led Implementation Research on Immunization: learning from LMICs. Heal Res Policy Syst. 2021;19(Suppl 2).

  15. 15.

    Akwataghibe NN, Ogunsola EA, Popoola OA, Agbo AI, Dieleman MA. Using participatory action research to improve immunization utilization in areas with pockets of unimmunized children in Nigeria. Heal Res Policy Syst. 2021; 19(Suppl 2).

  16. 16.

    Abakar MF, Seli D, Lechthaler F, Crump L, Mancuso A, Tran N, et al. Evaluation of the feasibility and sustainability of the joint human and animal vaccination and its integration to the public health system in the Danamadji health district, Chad. Heal Res Policy Syst. 2021;19(Suppl 2).

  17. 17.

    Nair AT, Nayar KR, Koya SF, Abraham M, Lordson J, Grace C, et al. Social media, vaccine hesitancy and trust deficit in immunization programs: a qualitative enquiry in Malappuram District of Kerala, India. Heal Res Policy Syst. 2021;19(Suppl 2).

  18. 18.

    Yazdani AT, Muhammad A, Nisar MI, Khan U, Shafiq Y. Unveiling and addressing implementation barriers to routine immunization in the peri-urban slums of Karachi, Pakistan: a mixed-methods study. Heal Res Policy Syst. 2021;19(Suppl 2).

  19. 19.

    Etokidem A, Nkpoyen F, Ekanem C, Mpama E, Isika A. Potential barriers to and facilitators of civil society organization engagement in increasing immunization coverage in Odukpani Local Government Area of Cross River State, Nigeria: an implementation research study. Heal Res Policy Syst. 2021;19(Suppl 2).

  20. 20.

    Zewde T, Teklu A, Bedada D, Tsehaye Y. The influence of linkages, feedback mechanisms, and caregiver mobility on immunization follow-up visits in Lideta Sub-city of Addis Ababa, Ethiopia: a qualitative study. Heal Res Policy Syst. 2021;19(Suppl 2).

  21. 21.

    Tilahun B, Teklu A, Mancuso A, Endehabtu BF, Gashu KD, Mekonnen ZA. Using health data for decision-making at each level of the health system to achieve universal health coverage in Ethiopia: the case of an immunization programme in a low-resource setting. Heal Res Policy Syst. 2021;19(Suppl 2).

  22. 22.

    Mafigiri, DK, Iradukunda C, Atumanya C, Odie M, Mancuso A, Tran N, et al. A qualitative study of the development and utilization of health facility-based immunization microplans in Uganda. Heal Res Policy Syst. 2021;19(Suppl 2).

  23. 23.

    Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2007;62(1):107–15.

    Article  Google Scholar 

  24. 24.

    Krippendorff KH. Content analysis: an introduction to its methodology. Thousand Oaks, CA: SAGE Publications, Inc.; 2004.

    Google Scholar 

  25. 25.

    Cavanagh S. Content analysis: concepts, methods and applications. Nurse Res. 1997;4(3):5–16.

    CAS  Google Scholar 

  26. 26.

    Langlois EV, Mancuso A, Elias V, Reveiz, L. Embedding implementation research to enhance health policy and systems: a multi-country analysis from ten settings in Latin America and the Caribbean. Heal Res Policy Syst. 2019;17(85):1–14.

    Google Scholar 

  27. 27.

    Innvaer S, Vist G, Trommald M, Oxman A. Health policy-makers’ perceptions of their use of evidence: a systematic review. J Heal Serv Res Policy. 2002;7(4):239–44.

    Article  Google Scholar 

  28. 28.

    Humphries S, Stafinski T, Mumtaz Z, Menon D. Barriers and facilitators to evidence-use in program management: a systematic review of the literature. BMC Health Serv Res. 2014;14(171):1–15.

    Google Scholar 

  29. 29.

    Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14(2):1–12.

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the project teams from the DELIR initiative and specifically the decision-makers and researchers who participated in this study. The authors would also like to acknowledge Dr. Abdul Ghaffar, Dr. Nhan Tran, and Dr. Kumanan Rasanathan for their guidance and support in the conceptualization, design, and implementation of the DELIR initiative.

About this supplement

This article has been published as part of Health Research Policy and Systems 2021, volume 19, supplement 2: Decision maker led implementation research on immunization. The full contents of the supplement are available online at https://health-policy-systems.biomedcentral.com/articles/supplements/volume-19-supplement-2.

Funding

This study was funded by the AHPSR. The funding body provided input on the design of the study through its Scientific and Technical Advisory Committee, but had no role in the data collection, analysis and writing of the manuscript. Publication costs were funded by the AHPSR within WHO and UNICEF with funding from GAVI.

Author information

Affiliations

Authors

Contributions

AM conceptualized the study and developed the protocol. SM collected the data. AM, SM and ZS analyzed and interpreted the results. AM developed the draft manuscript. SM, AS, ASM and ZS provided input on the manuscript. All authors read and approved the final manuscript.

Authors’ information

AM coordinated the DELIR initiative at the AHPSR. ZS is a staff member of the WHO. The views expressed in this article are theirs alone and do not necessarily represent the views, decisions, or policies of the WHO.

Corresponding author

Correspondence to Arielle Mancuso.

Ethics declarations

Ethics approval and consent to participate

This study was reviewed by the institutional review board at Johns Hopkins Bloomberg School of Public Health and determined to be non-human subjects research. All respondents voluntarily provided oral consent to participate in the interviews and have the session recorded prior to engaging in data collection activities. Oral consent was obtained from participants as the study posed minimal risk and was deemed non-human subjects research as participants were sharing their experiences and perceptions in their professional capacity. Interviews were also conducted using web-conferencing rather than in person.

Consent for publication

Not applicable.

Competing interests

AM and ZS were employed by the AHPSR within WHO and AS and ASM were employed by the UNICEF at the time of the study. The AHPSR and UNICEF are supporting the embedding of implementation research in LMICs.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

In-depth interview guide questions

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mancuso, A., Ahmed Malm, S., Sharkey, A. et al. Cross-cutting lessons from the Decision-Maker Led Implementation Research initiative. Health Res Policy Sys 19, 83 (2021). https://doi.org/10.1186/s12961-021-00706-0

Download citation

Keywords

  • Immunization
  • Vaccination
  • Implementation Research
  • Embedded Research
  • Decision-Maker Led Research
  • Africa
  • Asia