Skip to main content

Developing a workbook to support the contextualisation of global health systems guidance: a case study identifying steps and critical factors for success in this process at WHO

Abstract

Background

Global guidance can help countries strengthen their health systems to deliver effective interventions to their populations. However, to have an impact, guidance needs to be contextualised or adapted to local settings; this process includes consideration of health system arrangements and political system factors. To date, methods to support contextualisation do not exist. In response, a workbook was designed to provide specific methods and strategies to enable the contextualisation of WHO’s ‘Optimizing health worker roles to improve maternal and newborn health’ (OptimizeMNH) guidance at the national or subnational level. The objective of this study was to describe the process of developing the workbook and identify key steps of the development process, barriers that arose and facilitators that helped overcome some of these barriers.

Methods

A qualitative single case study design was carried out. Interviews, documents and a reflexive journal were used. Constant comparison and an edit-style of organisation were used during data analysis to develop concepts, themes, subthemes and relationships among them.

Results

Thirteen interviews were conducted and 52 documents were reviewed. Three main steps were identified in the process of developing the workbook for health systems guidance contextualisation, namely (1) determining the need for and gaining approval to develop the workbook, (2) developing the workbook (taking on the task, creating the structure of the workbook, operationalising its components, undergoing approval processes and editing it), and (3) implementing the workbook both at the WHO level and at the national/subnational level.

Five barriers and/or facilitators emerged relevant to each step, namely (1) having well-placed and credible champions, (2) creating and capitalising on opportunities, (3) finding the right language to engage various actors and obtain buy-in, (4) obtaining and maintaining meaningful buy-in, and (5) ensuring access to resources.

Conclusions

Understanding the key steps and the critical factors involved in the process of developing the workbook could help in the planning of similar and other tools aimed to support the implementation of WHO guidance. A plan for dissemination and implementation needs to be addressed during the preparation of these tools.

Peer Review reports

Background

Many low-resource and effective clinical and public health interventions (e.g. immunisations, kangaroo mother care for low birthweight infants) are not reaching people who need them most, especially in low- and middle-income countries (LMICs), because of fragmented and overburdened health systems [1,2,3,4]. This also leads to the inability to implement evidence-based guideline recommendations. ‘Health systems guidance’, defined as “systematically developed statements produced at global or national levels to assist decisions about appropriate options for addressing a health systems challenge in a range of settings” [1], can be used to strengthen health systems by addressing issues such as the lack of trained health workers for the delivery of effective interventions. WHO is the leading producer of health systems guidance to help countries that are facing the same or similar system-level issues. Global guidance can be used to develop policies at the global level (e.g. funding policies of international organisations) and guidance or policies at the national or subnational levels (e.g. health human resources planning for HIV treatment) [5].

However, the process of using this evidence-based guidance is complex. In order to have an impact, the topic of the guidance needs to get on the government’s agenda, guidance has to inform policy development, and the policy needs to be implemented [5]. These steps are determined by whether a government agrees to prioritise a particular framing of a problem and its causes, whether it agrees that the recommendations make sense for its health system and in the context of its political system, and whether it has the commitment and resources to implement it [1, 5, 6]. Therefore, to increase the relevance, acceptance and implementation of global guidance, it needs to be contextualised or adapted to fit a particular setting [5]. Indeed, there are a few striking examples where the context has been noted to shape the implementation of guidelines [7,8,9,10]. For example, Bollini et al. [7] showed that the content of policies around HIV prevention in prisons in four different settings varied and reflected the thinking and strategies of HIV prevention and care in their respective communities despite the availability of international guidelines on this topic.

There has also been a lack of support for users of health systems guidance (i.e. policy-makers, stakeholders and researchers) for combining global recommendations with national/subnational assessments of local problems and their causes, as well as of existing health system arrangements that may need to be changed and political system considerations that need to be taken into account [5]. Part of this challenge has been noted to include a lack of implementation plans in WHO guidelines or specific strategies where WHO recommendations can be appropriately and systematically tailored for each jurisdiction [11, 12].

To address these challenges, the development of a workbook was proposed to help users of WHO recommendations be able to contextualise recommendations to their own setting or situation, with the ultimate goal of increasing the acceptability and implementability of the recommendations. This study describes the process of how this tool was developed, including the steps in the process and the critical factors leading to success or non-success in each step. An edited version of the workbook can be found at the McMaster Health Forum website at https://www.mcmasterforum.org/lets-collaborate/resources [13].

Methods

Context for this study

Between 2010 and 2012, WHO developed a guidance document with recommendations for optimising health workers’ roles (through regulation, training and support) to improve access to and utilisation of key interventions for improving maternal and newborn health in LMICs (OptimizeMNH guidance) [14]. The development of these guidelines is outside the scope of this paper, but was performed following WHO Guideline Review Committee processes and can be found at http://optimizemnh.org/. During the development of the guideline, it was recognised that tailoring recommendations to the health system context and political environment (e.g. agreement on the need for regulatory change for health cadres) (personal correspondence, Alvarez) was required to optimise their implementation. In response, the guidance developers requested a tool or resource to advise users on how to contextualise global recommendations to the unique situations in specific jurisdictions. A workbook for contextualising health systems guidance (henceforth ‘workbook’) was developed de novo by two of the authors of this paper (EA and JL). This study was pragmatic in that there was an opportunity to study the course of the development of the workbook while it occurred.

Framework underpinning the workbook

The workbook was based on scholarly and applied work in the field of health systems and policy [5, 15,16,17,18,19,20,21,22,23,24,25,26,27,28] and addressed (1) clarifying the problem and its causes, (2) framing options for addressing the problem, (3) identifying implementation considerations, (4) considering the broader health system context, (5) considering the broader political system context, (6) refining the statement of the problem, options and implementation considerations in light of health system and political system factors, (7) anticipating monitoring and evaluation needs and, finally, (8) making national or subnational policy recommendations or decisions [29]. These steps made up the ‘health systems guidance contextualisation framework’, which was created with the workbook.

Study approach

Given the diversity of people developing health systems guidance within an international health organisation such as WHO, and given that this was a new process within WHO, an exploratory holistic single case design was chosen to allow analysis of the case and its context. This study design captures comprehensive and meaningful characteristics of complex real-life social phenomena [30]. Because multiple people were involved in the development of this workbook, a constructivist paradigm, where multiple realities are expected based on each person’s view, was fitting for this study [30, 31].

The ‘case’ in this investigation was the process of developing a workbook for the contextualisation of global health systems guidance. For the study to remain focused [30], it was bounded by context, so only information relating to the creation of the workbook was used; participants who took part in the process of developing the workbook; and time, starting when the workbook was conceptualised and ending when the workbook was posted online along with the OptimizeMNH guidance document (Fig. 1). WHO guidance documents, which would include the workbook, incorporate a continuous review process and therefore it was important to set a time limit for this study as changes could be ongoing for many years.

Fig. 1
figure 1

Timeline of events in the development of the workbook for contextualising health systems guidance

Sampling and recruitment

Intensity sampling [32] was used to find participants who could provide in-depth information about the case, given that they were involved in the idea for or the actual design of the workbook. Respondent-driven sampling was also used to find other policy-makers, stakeholders or researchers who were considered to be information-rich sources about the case. Recruitment was conducted through personalised emails.

Criterion sampling [32] was used to find documents that related specifically to the process of developing the workbook, for example, meeting minutes. This was especially relevant because the development of the workbook was occurring in parallel with the development of the OptimizeMNH guidance, and some people were involved in both. These documents were found through literature searches, personal emails or interviews.

Data collection

Informed consent was obtained prior to conducting the interviews. Semi-structured interviews were conducted by a single interviewer (EA) over Skype, and these were recorded with a digital recorder and transcribed. The interviewer asked participants about their role in, and the process of, the development of the workbook, challenges that arose during the process and how these challenges were overcome, other contributors to this work, and other documents for review. In addition, participants were asked to describe what their hopes were for a tool to help countries contextualise global guidance and the challenges they foresaw with such a tool.

Documents came from the interviews, from personal emails or through online searches by the principal investigator, and most were publicly available (Additional file 1). Finally, a reflexive journal was kept by the principal investigator (EA) throughout the course of the study to keep track of events and their contexts (e.g. who was involved, what and why decisions were made, etc.), to provide insights into the role of the principal investigator as a participant-observer, and to allow for personal reflection of the events [31]. Entries in this journal were also used as data.

Data analysis

The coding of data was mainly conducted by the principal investigator (EA), with input from JL on the codes, themes and subthemes, and on the presentation of the data. Data analysis occurred concurrently with data collection to help direct further data collection. A code structure was developed using an integrated approach, where both deductive and inductive methods were used [33, 34]. A deductive organising framework or ‘start list’ [33, 34] was used, which included steps in the process of developing the workbook, challenges and mitigating strategies. Additional steps and sub-steps, challenges and mitigating strategies were added to the start list in a temporal sequence following an edit-style organising process using Microsoft Word [30, 34]. As this code structure was being developed, concepts, themes, subthemes and their relationships emerged inductively through constant comparison, where new information was evaluated against previous data. These concepts and themes were then verified across each main step in the process. The code structure and interview transcripts were reviewed to ensure no new themes or disconfirming evidence were found. Data saturation was achieved. Member checking with two interviewees and peer debriefing helped refine the concepts and their presentation.

Results

Participants

Overall, 13 of the 17 individuals approached participated in the study (76% response rate); 3 did not respond to the invitation and 1 was not persuaded they could make a meaningful contribution. For confidentiality purposes, identifying information is not presented, but there was diversity in age, sex, seniority and roles, and the participants had collective experience at the district, national and international levels as well as representing every WHO region except for South-East Asia. Broadly, for the purposes of this study, participants are described as belonging to one of three categories, namely members of the Secretariat of the guidance panel on task shifting (those involved in developing the WHO OptimizeMNH guidance who either worked at WHO or at an outside institution – 3 participants), staff of WHO not part of the Secretariat of the guidance panel on task shifting (3 participants), or health system and policy analysts (7 participants).

One interview was conducted with each participant. These interviews ranged from 19 to 65 min in length, with an average duration of 38 min.

Documents

A total of 52 documents were reviewed, including 27 journal articles, 7 presentations and 18 other documents (e.g. meeting agendas, reports, etc.). Multiple personal emails and a reflexive journal (Volume I – 195 pages, volume II – 127 pages, and volume III – 93 pages) were also used as data (Additional file 1).

Process

Three main steps, and various sub-steps, were identified in the process of developing the workbook for health systems guidance contextualisation (Fig. 1). These were (1) determining the need for and gaining approval to develop the workbook, (2) developing the workbook (taking on the task, creating the structure of the workbook, operationalising its components, undergoing approval processes and editing it), and (3) implementing the workbook both at the WHO level and at the national/subnational level.

The first step helped explain the purpose of the workbook and gave a concrete reason for the workbook to be developed, which was to contextualise the OptimizeMNH guidance. The second step ranged from the time approval was gained to move ahead with this work to the time the workbook was published online by WHO. The third step continues and encompasses the dissemination, implementation and institutionalisation of the use of the workbook both at the level of WHO and at the national/subnational level. This last phase is critical for improving the workbook itself through user testing but also for helping improve the uptake of health systems guidance recommendations, which is the purpose of the workbook. While uptake has not been universal, the workbook has been applied and evaluated in Peru and Uganda (in review).

Critical factors affecting the development of the workbook

At each of these three steps, five critical factors surfaced from the data as barriers and/or facilitators, including (1) having well-placed and credible champions, (2) creating and capitalising on opportunities, (3) finding the right language to engage various actors and obtain buy-in, (4) obtaining and maintaining meaningful buy-in, and (5) ensuring access to resources. General descriptions of each of these factors, along with their relationships, are provided (Fig. 2, Table 1). The implications of the presence or absence of these critical factors at each step are shown on Additional file 2, and a full description can be found elsewhere [35].

Fig. 2
figure 2

Relationships among critical factors influencing the development of the workbook for contextualising health systems guidance

Table 1 Themes, subthemes and descriptions of critical factors influencing the process of workbook development

Having well-placed and credible champions

Through the interviews and reflexive journal, several people emerged as being more prominent in moving ideas forward and/or devoting their time and other resources to completing tasks in the process. Without these champions, the workbook may not have come to fruition. While champions may be thought of as people with clout or within existing leadership positions, our definition sees champions as being leaders from any position, but does require that they go beyond their day-to-day tasks in moving the ideas forward [36]. Champions can have different levels of involvement, and someone who is a champion at one stage of the process may not be a champion at another.

…so I think my role was more kind of being the person pushing for this to be brought about, identifying the opportunity for [the person taking on this work as participant-observer], and then after that I moved into much more of a supporting role.” (001, Health systems and policy analyst)

It took [the head of the Secretariat of the WHO guidance panel on task shifting] a lot of work in the background in the WHO to try and bring everyone on board [to using innovations in the guidance development process].” (002, Member of Secretariat of WHO guidance panel on task shifting)

Creating and capitalising on opportunities

The champion(s) either sought out or created opportunities to move the work forward, or other actors presented opportunities to the champion(s) who then capitalised on these opportunities.

We knew that the recommendations from the guidance were going to be directed to policy-makers and that is why we involved [the health systems policy expert] in the panel.” (003, Member of Secretariat of WHO guidance panel on task shifting)

There were times when the work could have moved forward, but it did not, resulting in missed opportunities. One example is that there were several opportunities to promote the workbook to attendees at a WHO-African Regional Office meeting in Addis Ababa, Ethiopia, in 2012. However, a prior champion of this work did not bring up the potential to use and evaluate the workbook at the country level as was expected, creating missed opportunities. Lastly, there was a potential opportunity identified by a participant that could be capitalised upon in the future, involving identifying gaps in the evidence through the contextualisation process, which could then be fed back into and inform research processes.

Finding the right language to engage various actors and obtain buy-in

Compiling information from the interviews and journal, the concept surfaced that the champion(s) used persuasive language to gain buy-in from those who were involved in the process and who could have created barriers. A more subtle way to help obtain buy-in was to first build a shared understanding of the concepts or issues about contextualisation by using common terminology that resonated with the target audience. This level of shared understanding about what is meant by contextualisation and its importance was seen as especially useful for trying to communicate the relevance of the work or how to apply the work.

...he [the head of the Secretariat of the WHO guidance panel on task shifting] gave me the opportunity to pitch the idea to the committee members, which I did in the form of a brief presentation and they seemed to be quite excited about the idea. And I also, because in sharing the meeting…after two days, I was able to insert a bunch of examples that had come back, come directly from them, that were the type of things that a good workbook would flag for people.” (001, Health systems and policy analyst)

Communication challenges within WHO were noted as a barrier, especially for disseminating the use of the workbook, including departments working in silos and products developed in one department not necessarily being communicated for use by other departments or at the regional office level through internal means. In addition, several participants stated that following standard academic or scientific language could be a facilitator when working with specific audiences, such as WHO’s Guideline Review Committee, but this could be a barrier for other audiences, such as policy-makers, who may not be familiar with this language. Likewise, the workbook had to be approved by the Guideline Review Committee and had to follow an academic style. Evaluating the use of the workbook will help determine if the workbook is easy to understand by research and lay audiences who will be using the workbook directly.

I think one of the criticisms we have had of the Optimize guidelines is that it feels, some people have said it feels very academic. You have done quite a good job of pulling together all those current literatures all methodologically sound and all that, but you know it is quite difficult to digest as a user in the field. I can see where they are coming from because we’re – we are researchers and we’re also trying to adhere to these WHO standards which require sort of – kind of requirements of certain kinds of language and so on. I think that is probably true of all the evidence coming out of this project, that’s how we kind of make those things more accessible.” (002, Member of Secretariat of WHO guidance panel on task shifting)

Obtaining and maintaining meaningful buy-in

The need to obtain and maintain buy-in in the development of new products was mentioned by one participant, but the concept resonated with what many of the other participants stated.

It is not the member state, it’s not WHO, it’s not the NGOs [non-governmental organisations], it takes many to tango. It’s more like carnival, it’s not like a tango. But it’s different at the same time because if you want to overcome those obstacles that exist, this petty politics and things like this, you really have to… persevere, it requires lots of perseverance. You have to talk all the time, you have to go to the right people all the time, you have to convince a higher up, a senior person, and then this person gets distracted and you have to go back and press, and you have to ask your friends to call this person. It’s relentless.” (007, Staff of WHO not part of Secretariat of guidance panel on task shifting)

As noted in the journal, reflecting on these concepts during the analysis led to thinking about institutionalisation as a type of ‘permanent buy-in’ by an institution. Unless processes of including tools to contextualise guidance at the WHO level or of using these tools at the country level are institutionalised, a change in leadership could result in previous buy-in being lost. Therefore, either continuous communication with new leadership is required to secure meaningful buy-in or institutionalising a process by one group or leader could bypass the future need to obtain buy-in. One participant remarked that the difficult part can be knowing if there is meaningful buy-in or if the buy-in is for a secondary purpose (e.g. advancing other work). This could manifest itself as appearing to have meaningful buy-in at one stage of the process but not at another.

My sense is that the panel which had more health systems and policy people on it, was very very supportive… but from the WHO staff people we continued to have this problem that if they come from clinical epidemiology backgrounds, their sense is their usual way of doing by and large can be aloofness, but they recognise that they sometimes still need to have people like me in the room to make it look like they are doing things differently, but I am not convinced at the end of the day that they are committed to doing things differently.” (001, Health systems and policy analyst)

However, ascertaining this level of information could require immense transparency on the part of the actors involved as it seems unlikely for people to be open about secondary motives. As was noted in the journal, the problem of buy-in could also be difficult to distinguish from a separate problem of lacking resources (e.g. buy-in from one group of people may not secure resources from other groups of people).

Ensuring access to resources

Most participants noted the need for human resources, finances and time for different steps in the process of developing the workbook and contextualising the guidance. In addition, several participants mentioned that having people from different organisations involved in the work with WHO also meant there were different agendas, timelines and priorities involved in developing the guidance and the workbook and in implementing the guidance recommendations.

So right now, you know, everyone puts all the effort into the front end and then the guidance is there and there is no energy or money to see it implemented….we really need to re-think how we develop critical paths for developing guidance and the guidance development process ends relatively early in that timeline and then we have lots of time and resources left to do all this other stuff. Otherwise the whole exercise is for naught.” (001, Health systems and policy analyst)

Human resources include the people involved in doing the work. Further attributes of these individuals, which arose from the data, include motivation and knowledge. Motivation to work on a particular topic is necessary when there are competing demands on an individual or on an agency. This can be seen as prioritising specific work. Knowledge can be intrinsic and come from existing knowledge of the individuals involved in the work (e.g. expertise) or can be extrinsic and found through searches for information (e.g. library resources). Finances include salaries or payments for those carrying out the work and funding for traveling and for securing supplies. Time is required for individuals to do the work, and individuals and agencies have timelines for getting the work done. Technology can also support (e.g. find evidence) or be a barrier (e.g. complex or limited access) in advancing the work.

Discussion

Principal findings

Three discrete steps were identified in the process of developing the workbook for health systems guidance contextualisation, namely (1) determining the need for and gaining approval to develop the workbook, (2) developing the workbook and (3) implementing the workbook both at the WHO level and at the national/subnational level. Five factors affecting each step in the development of the workbook surfaced as barriers and/or facilitators, namely (1) having well-placed and credible champions, (2) creating and capitalising on opportunities, (3) finding the right language to engage various actors and obtain buy-in, (4) obtaining and maintaining meaningful buy-in, and (5) ensuring access to resources.

Each of these factors was found to be present when proceeding from one step to the next in the process. However, some factors played larger roles at certain points. For example, in developing the workbook, the sub-step of taking on the task relied heavily on champions, language and buy-in, while the sub-step of creating the structure of the workbook relied more on resources. Steps 1 and 2, determining the need for and gaining approval to develop the workbook and developing it, were both successful in that the workbook, along with the OptimizeMNH guidance, was published online. Barriers encountered during these two steps were overcome. However, the implementation of the workbook at the WHO level and at the national/subnational level have encountered many barriers and have yet to occur. The exceptions are Peru and Uganda, where the use of the workbook has been evaluated (separate study).

Several points are noteworthy. Another instance was identified where WHO-led work has not been taken up systematically. The development of the Handbook for Developing Health Systems Guidance [37], while not central to this study, came up sufficiently during the interviews to enable observations to be made about its process of development. The Handbook was supported in its production by WHO, but it was ultimately not published or endorsed by WHO and has not had systematic uptake within WHO. The extent of its use by departments in WHO is therefore unknown. This represents another lost opportunity for advancing the work in health systems strengthening. Instead, as evidenced through the interviews, groups within WHO are trying to distinguish between health systems guidance and clinical guidelines without referring to the work of the taskforce that developed the Handbook. It is unknown at this time if the workbook will have consistent uptake or if future WHO guidance documents will require the incorporation of tools to help with contextualisation.

Fit within existing literature

As mentioned previously, the opportunity to study the development of the workbook was pragmatic in that events were taking place before there was enough time to conduct a full literature search to identify all the frameworks around the creation of a new knowledge translation tool, especially within similar settings to WHO. Instead, events were noted as they occurred. A critical interpretive synthesis of the literature around contextualisation of guidance (separate study) identified many frameworks and tools for the dissemination and implementation of clinical practice guidelines [38,39,40,41,42,43,44,45,46,47]. Many of these tools (e.g. ADAPTE, MAGICapp, EtD) are used at the development, dissemination, adaptation or implementation of guidelines or recommendations at the service or provider levels. Many do not account for the health (e.g. referral systems, supply chain management, regulatory changes for providers) and/or political systems (i.e. institutions, interests, ideas and external factors) changes that may be necessary to support the implementation of guidelines. This was the main reason a new tool was sought to support the implementation of the OptimizeMNH guidance. As opposed to adapting recommendations directly, the workbook addresses how possible policy solutions can be developed to support the implementation of guidance recommendations at the systems level by considering these contextual factors. The study by Gagliardi et al. [48] highlights the importance of context in the implementation of integrated knowledge translation (IKT), or collaboration between researchers and policy-makers. IKT may be seen as a tool in the use of research evidence for decision-making, and each organisation needs to develop its own approach to IKT. Challenges and enablers affecting the success of IKT were similar to those found in this study and included champions, opportunities, organisational endorsement, resistance to change, resources, motivation and time. Further, findings in this study align with some of the concepts from Greenhalgh et al.’s [49] review of diffusion of innovations frameworks, including champions, leadership and vision, enablement of knowledge sharing via internal and external networks, values and goals, power balance, innovation-system fit, dedicated time/resources, and motivation. These frameworks may be especially useful in addressing the dissemination and implementation of workbooks at the WHO and national/subnational levels, or the third step in our process. However, this study extends the findings from these frameworks to the multiple steps in the creation of new tools, not just at the diffusion or implementation stage. Each step may be seen as its own process, which is helpful for those planning to undertake a similar task. This study also reinforces the use of qualitative methods to examine the process and context of developing a new knowledge translation tool.

Strengths and limitations of the study

The main strengths of this paper revolve around the choice of methods. Following a qualitative case study method, including the use of multiple sources of data such as interviews, documents and a reflexive journal, allowed for an in-depth look at the process of developing the workbook. The role of participant-observer added to the richness of the findings in that EA and JL had first-hand accounts of the process. Member checking and peer debriefing helped strengthen the analysis and increased rigor. Finally, the reflexive journal helped identify the investigator’s biases (e.g. beliefs that health systems guidance and the contextualisation of guidance hold promise in strengthening health systems) and account for them to mitigate their influence, such as by asking for both positive and negative feedback on the workbook during the interviews.

There is one main weakness of this study, which is the potential for recall bias with retrospective interviews (ranging from 6 months to 4 years). However, having multiple interviewees and triangulating information with documents helped decrease the impact of this recall bias. One main challenge with the case study methodology is that findings cannot be generalised from a single case to other cases, as the context of the work done within and alongside an international organisation such as WHO may be unique. Therefore, readers would need to consider their own contexts before applying these findings to their settings. A limitation includes that testing the workbook was not conducted through this study. This work was instead conducted through a separate study evaluating the use of the workbook in Peru and Uganda (in review).

Implications for policy and practice

There are three main implications for policy and practice that were found through this study. First, lack of implementation of guideline recommendations is still a significant barrier to improving care, and there has been a call to increase attention to implementation strategies within documents developed by WHO [11]. This study provides some clear and direct steps that can be used to develop new tools for WHO and similar organisations. Practically, the workbook was developed to be generic enough to use for any policy topic, yet incorporates examples drawn from the OptimizeMNH guidance. Subsequent versions of the workbook would not take as long to develop as it would simply rely on drawing examples for other particular guidance documents. Second, as was seen in this study, and in the case of the Handbook, a plan for dissemination and implementation of the tools themselves needs to be addressed in their preparation. Not having a plan for implementation can severely limit the use and therefore the usefulness of these labour- and resource-intensive products. Third, many considerations were listed as barriers to the implementation of workbooks at the WHO and national/subnational levels. These are all areas that could be addressed in practice so that guidance recommendations have a better chance of being implemented and can have a positive impact on the health of populations. Furthermore, the role of various stakeholders, such as guideline development at the WHO level versus a more tailored approach to guideline development at the regional or national level, is an area that needs further elaboration to determine the right balance of resource use and context-tailored recommendations. This is especially important in light of equity considerations.

Implications for research

First, using qualitative methods to study processes of tool development can help not only to describe the process but also its context to tease out facilitators and barriers in these processes. Second, the use of the workbook has been evaluated in two countries, Peru and Uganda (in review). However, in order to improve guidance contextualisation processes at the national/subnational levels, further evaluations of the workbook for various topics, and in different contexts, will be needed to help refine the language and structure to make the workbook more user-friendly and therefore more useful.

Conclusions

The key steps and critical factors found through this study would help in the planning of other tools at WHO. Additionally, a plan for dissemination and implementation needs to be addressed during the preparation of these tools. As this study highlights, there are many barriers in the implementation of this workbook at the WHO level and at the national/subnational level, which can be addressed in practice. This study also provides insights into improving the workbook from the perspective of international guidance developers.

Abbreviations

IKT:

integrated knowledge translation

LMICs:

low- and middle-income countries

OptimizeMNH:

Optimizing health worker roles to increase access to and utilization of key interventions to improve maternal and newborn health

References

  1. Bosch-Capblanch X, Lavis JN, Lewin S, Atun R, Røttingen J-A, Dröschel D, et al. Guidance for evidence-informed policies about health systems: rationale for and challenges of guidance development. PLoS Med. 2012;9(3):e1001185.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder A, et al. Overcoming health-systems constraints to achieve the Millennium Development Goals. Lancet. 2004;364:900–6.

    Article  PubMed  Google Scholar 

  3. World Health Organization. Everybody’s Business: Strengthening Health Systems to Improve Health Outcomes: WHO’s Framework for Action. Geneva: WHO; 2007.

    Google Scholar 

  4. World Health Organization. Ministerial Summit on Health Research, Mexico City, Mexico - November 16–20, 2004. 2011. http://www.who.int/rpc/summit/en/. Accessed Nov 2014.

    Google Scholar 

  5. Lavis JN, Røttingen J-A, Bosch-Capblanch X, Atun R, El-Jardali F, Gilson L, et al. Guidance for evidence-informed policies about health systems: linking guidance development to policy development. PLoS Med. 2012;9(3):e1001186.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Kingdon J. Agendas, Alternatives, and Public Policies. 2nd ed. New York: HarperCollins College Publishers; 2003.

    Google Scholar 

  7. Bollini P, Laporte J-D, Harding TW. HIV prevention in prisons. Do international guidelines matter? Eur J Pub Health. 2002;12(2):83–9.

    Article  Google Scholar 

  8. Eamer GG, Randall GE. Barriers to implementing WHO’s exclusive breastfeeding policy for women living with HIV in sub-Saharan Africa: an exploration of ideas, interests and institutions. Int J Health Plann Manag. 2013;28(3):257–68.

    Article  Google Scholar 

  9. Gomez P, Gutman J, Roman E, Dickerson A, Andre Z, Youll S, et al. Assessment of the consistency of national-level policies and guidelines for malaria in pregnancy in five African countries. Malar J. 2014;13:212.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Mulligan JA, Mandike R, Palmer N, Williams H, Abdulla S, Bloland P, et al. The costs of changing national policy: lessons from malaria treatment policy guidelines in Tanzania. Tropical Med Int Health. 2006;11(4):452–61.

    Article  Google Scholar 

  11. Wang Z, Norris SL, Bero L. Implementation plans included in World Health Organisation guidelines. Implement Sci. 2016;11:76.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Hoffman S, Lavis J, Bennett S. The use of research evidence in two international organizations’ recommendations about health systems. Healthc Policy. 2009;5(1):66–86.

    PubMed  PubMed Central  Google Scholar 

  13. McMaster Health Forum. Workbook on Contextualizing Guidance Supports Localized, Evidence-based Policy Development. https://www.mcmasterforum.org/new-at-the-forum/news-item/2016/04/27/Workbook-on-contextualizing-guidance-supports-localized-evidence-based-policy-development. Accessed 25 Oct 2017.

  14. World Health Organization. OptimizeMNH: Optimizing Health Worker Roles to Improve Access to Key Maternal and Newborn Health Interventions Through Task Shifting. 2012. http://apps.who.int/iris/bitstream/10665/77764/1/9789241504843_eng.pdf. Accessed 20 Dec 2012.

  15. Lavis JN, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP). Health Res Policy Syst. 2009;7(Suppl 1):I1.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP) 1: What is evidence-informed policymaking? Health Res Policy Syst. 2009;7(Suppl 1):S1.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Lavis JN, Wilson MG, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for Evidence-informed Health Policymaking (STP) 4: Using research evidence to clarify a problem. Health Res Policy Syst. 2009;7(Suppl 1):S4.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Lavis JN, Wilson MG, Oxman AD, Grimshaw J, Lewin S, Fretheim A. SUPPORT Tools for Evidence-informed Health Policymaking (STP) 5: Using research evidence to frame options to address a problem. Health Res Policy Syst. 2009;7(Suppl 1):S5.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Fretheim A, Munabi-Babigumira S, Oxman AD, Lavis JN, Lewin S. SUPPORT Tools for Evidence-informed Policymaking in Health 6: Using research evidence to address how an option will be implemented. Health Res Policy Syst. 2009;7(Suppl 1):S6.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Lavis JN, Oxman AD, Souza NM, Lewin S, Gruen RL, Fretheim A. SUPPORT Tools for Evidence-informed Health Policymaking (STP) 9: Assessing the applicability of the findings of a systematic review. Health Res Policy Syst. 2009;7(Suppl 1):S9.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT Tools for Evidence-informed Health Policymaking (STP) 10: Taking equity into consideration when assessing the findings of a systematic review. Health Res Policy Syst. 2009;7(Suppl 1):S10.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Oxman AD, Fretheim A, Lavis JN, Lewin S. SUPPORT Tools for Evidence-informed Health Policymaking (STP) 12: Finding and using research evidence about resource use and costs. Health Res Policy Syst. 2009;7(Suppl 1):S12.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Lavis JN, Permanand G, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for Evidence-informed Health Policymaking (STP) 13: Preparing and using policy briefs to support evidence-informed policymaking. Health Res Policy Syst. 2009;7(Suppl 1):S13.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Lavis JN, Boyko JA, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for Evidence-informed Health Policymaking (STP) 14: Organising and using policy dialogues to support evidence-informed policymaking. Health Res Policy Syst. 2009;7(Suppl 1):S14.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Oxman AD, Lewin S, Lavis J, Fretheim A. SUPPORT Tools for Evidence-informed Health Policymaking (STP) 15: Engaging the public in evidence-informed policymaking. Health Res Policy Syst. 2009;7(Suppl 1):S15.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Oxman AD, Lavis JN, Fretheim A, Lewin S. SUPPORT Tools for Evidence-informed Health Policymaking (STP) 16: Using research evidence in balancing the pros and cons of policies. Health Res Policy Syst. 2009;7(Suppl 1):S16.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Lewin S, Oxman AD, Lavis JN, Fretheim A, Marti SG, Munabi-Babigumira S. SUPPORT Tools for Evidence-informed Policymaking in Health 11: Finding and using evidence about local conditions. Health Res Policy Syst. 2009;7(Suppl 1):S11.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Fretheim A, Oxman AD, Lavis JN, Lewin S. SUPPORT Tools for Evidence-informed Policymaking in Health 18: Planning monitoring and evaluation of policies. Health Res Policy Syst. 2009;7(Suppl 1):S18.

    Article  PubMed  PubMed Central  Google Scholar 

  29. World Health Organization. Annex 8. Contextualizing the Guidelines – Workbook. Geneva: WHO; 2012. http://www.optimizemnh.org/Annexes/Annex_8_Contextualizing_Workbook.pdf. Accessed 20 Dec 2012.

  30. Yin R. Case Study Research: Design and Methods. 4th ed. Thousand Oaks, CA: Sage; 2009.

    Google Scholar 

  31. Baxter P, Jack S. Qualitative case study methodology: Study design and implementation for novice researchers. Qual Rep. 2008;13(4):544–59.

    Google Scholar 

  32. Creswell J. Qualitative Inquiry & Research Design: Choosing Among Five Approaches. Thousand Oaks, CA: Sage; 2007.

    Google Scholar 

  33. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Serv Res. 2007;42(4):1758–72.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Crabtree B, Miller W, editors. Doing Qualitative Research. 2nd ed. Thousand Oaks, CA: Sage Publications, Inc.; 1999.

    Google Scholar 

  35. Alvarez E. Developing and Evaluating the Use of a Workbook for Contextualizing Health Systems Guidance. Hamilton, ON: McMaster University; 2016.

    Google Scholar 

  36. Lachance J, Oxendine J. Redefining leadership education in graduate public health programs: prioritization, focus, and guiding principles. Am J Public Health. 2015;105(S1):S60–4.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Bosch-Capblanch X. Handbook for Supporting the Development of Health System Guidance. Swiss Tropical and Public Health Institute. 2011. https://www.swisstph.ch/fileadmin/user_upload/WHOHSG_Handbook_v04.pdf. Accessed 18 Aug 2014.

  38. Gagliardi AR, Brouwers MC, Bhattacharyya OK. A framework of the desirable features of guideline implementation tools (GItools): Delphi survey and assessment of GItools. Implement Sci. 2014;9:98.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Gagliardi AR. “More bang for the buck”: exploring optimal approaches for guideline implementation through interviews with international developers. BMC Health Serv Res. 2012;12:404.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Kastner M, Estey E, Hayden L, Chatterjee A, Grudniewicz A, Graham ID, et al. The development of a guideline implementability tool (GUIDE-IT): a qualitative study of family physician perspectives. BMC Fam Pract. 2014;15:19.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Bahtsevani C, Willman A, Khalaf A, Ostman M. Developing an instrument for evaluating implementation of clinical practice guidelines: a test-retest study. J Eval Clin Pract. 2008;14(5):839–46.

    Article  PubMed  Google Scholar 

  42. Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G, et al. The GuideLine Implementability Appraisal (GLIA): Development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005;5:23.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM. How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011;6:26.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Harrison MB, Graham ID, van den Hoek J, Dogherty EJ, Carley ME, Angus V. Guideline adaptation and implementation planning: a prospective observational study. Implement Sci. 2013;8:49.

    Article  PubMed  PubMed Central  Google Scholar 

  45. The ADAPTE Collaboration. The ADAPTE Process: Resource Toolkit for Guideline Adaptation. Version 2.0. 2009. http://www.g-i-n.net/document-store/working-groups-documents/adaptation/adapte-resource-toolkit-guideline-adaptation-2-0.pdf. Accessed 15 Feb 2015.

  46. Schunemann HJ, Wiercioch W, Brozek J, Etxeandia-Lkobaltzeta I, Mustafa RA, Manja V, et al. GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT. J Clin Epidemiol. 2017;81:101–10.

    Article  PubMed  Google Scholar 

  47. MAGICapp - Making GRADE the Irresistible Choice - Medical Guidelines. https://www.magicapp.org. Accessed 6 Feb 2018.

  48. Gagliardi A, Webster F, Brouwers M, Baxter N, Finelli A, Gallinger S. How does context influence collaborative decision-making for health services planning, delivery and evaluation? BMC Health Serv Res. 2014;14:545.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Fadi El-Jardali, Kaelan Moat and Mita Giacomini for their input into the interpretation and representation of the data, and Majdi Qutob for reviewing the manuscript. Consolidated criteria for Reporting Qualitative research (COREQ) guidelines were followed in this manuscript [50].

Funding

This research was supported through an International Development Research Centre (IDRC) grant and a Vanier Canada Graduate Scholarship, which helped support EA’s graduate studies. These funding bodies were not involved in the design of the study, any aspect of data collection, analysis or interpretation, or in the writing of the manuscript.

Availability of data and materials

The datasets (interview transcriptions, documents and reflexive journal) used during the current study are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

Authors

Contributions

EA developed the study design with input from the other authors. EA conducted the interviews and maintained the reflexive journal. EA coded and analyzed the data with input from JL. EA wrote the manuscript and all other authors read and provided input into the manuscript.

Corresponding author

Correspondence to Elizabeth Alvarez.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was obtained from the Hamilton Integrated Research Ethics Board (HiREB), Hamilton, Ontario, Canada, prior to data collection. In addition, each participant signed an informed consent form prior to being interviewed.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Documents reviewed for the study. (DOCX 18 kb)

Additional file 2:

Critical factors influencing each step of the process of workbook development. (DOCX 19 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alvarez, E., Lavis, J.N., Brouwers, M. et al. Developing a workbook to support the contextualisation of global health systems guidance: a case study identifying steps and critical factors for success in this process at WHO. Health Res Policy Sys 16, 19 (2018). https://doi.org/10.1186/s12961-018-0297-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12961-018-0297-x

Keywords