Volume 7 Supplement 1

SUPPORT Tools for evidence-informed health Policymaking (STP)

Guide
Open Access

SUPPORT Tools for evidence-informed health Policymaking (STP) 2: Improving how your organisation supports the use of research evidence to inform policymaking

  • Andrew D Oxman1Email author,
  • Per Olav Vandvik2,
  • John N Lavis3,
  • Atle Fretheim4 and
  • Simon Lewin5
Health Research Policy and Systems20097(Suppl 1):S2

https://doi.org/10.1186/1478-4505-7-S1-S2

©  Oxman et al; licensee BioMed Central Ltd. 2009

Published: 16 December 2009

Abstract

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

In this article, we address ways of organising efforts to support evidence-informed health policymaking. Efforts to link research to action may include a range of activities related to the production of research that is both highly relevant to – and appropriately synthesised for – policymakers. Such activities may include a mix of efforts used to link research to action, as well as the evaluation of such efforts. Little is known about how best to organise the range of activity options available and, until recently, there have been relatively few organisations responsible for supporting the use of research evidence in developing health policy. We suggest five questions that can help guide considerations of how to improve organisational arrangements to support the use of research evidence to inform health policy decision making. These are: 1. What is the capacity of your organisation to use research evidence to inform decision making? 2. What strategies should be used to ensure collaboration between policymakers, researchers and stakeholders? 3. What strategies should be used to ensure independence as well as the effective management of conflicts of interest? 4. What strategies should be used to ensure the use of systematic and transparent methods for accessing, appraising and using research evidence? 5. What strategies should be used to ensure adequate capacity to employ these methods?

About STP

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers. The series is intended to help such people ensure that their decisions are well-informed by the best available research evidence. The SUPPORT tools and the ways in which they can be used are described in more detail in the Introduction to this series [1]. A glossary for the entire series is attached to each article (see Additional File 1). Links to Spanish, Portuguese, French and Chinese translations of this series can be found on the SUPPORT website ). Feedback about how to improve the tools in this series is welcome and should be sent to:http://STP@nokc.nohttp://www.support-collaboration.org.

Scenario

There is a new Minister of Health in the Ministry where you work. One of the goals of the new Minister is to improve the capacity of the Ministry to use research evidence to inform decisions about how the health system is organised, financed and governed. You have been asked to put forward a proposal for strategies to improve how the Ministry supports its use of evidence to inform policy decisions.

Background

In this article, we present five questions that policymakers and those who support them could ask when considering how to improve support for the use of research evidence to inform health policy decisions. Such questions could, for instance, be asked by any of the people in the scenario outlined above.

A number of theories have been proposed to explain the role of research evidence in policymaking. In addition, common wisdom about how to improve the appropriate use of research evidence is abundant. However, empirical evidence to support such ideas is difficult to find [2]. While increasing numbers of studies are being undertaken in low- and middle-income countries [3–8] most evidence still comes from interview studies in high-income countries [9, 10]. Systematic reviews of these studies suggest that [9, 10]:

  • Interaction between researchers and policymakers increases the likelihood of research being used by policymakers

  • Good timing and timely research increase (and poor timing or lack of timeliness decrease) the likelihood of research being used by policymakers

  • When policymakers have negative attitudes towards research evidence, the likelihood of research being used by them decreases

  • When policymakers lack relevant skills and expertise, the likelihood of research being used by them decreases

  • Policy networks and trust in researchers increase the likelihood of research being used by policymakers, and

  • A lack of perceived relevance, the use of jargon, and the production of publications aimed at a scholarly audience are all factors that decrease the likelihood of research being used by policymakers

Activities aimed at improving the use of research evidence to inform policy have been referred to in various ways. These terms include: knowledge translation, knowledge transfer, knowledge exchange, research utilisation, implementation, diffusion, and dissemination [11]. Considerable confusion and misunderstanding exists about the definition and scope of these concepts, and the literature related to these issues is diverse and widely dispersed [12]. Several frameworks have been proposed as ways to organise these approaches and thus improve the use of research evidence by policymakers [11–20]. These frameworks have overlapping purposes and concepts.

One of these frameworks focuses on assessing country-level efforts to link research to action. This framework provides an inventory of a range of activities that can be considered when developing organisational arrangements to support the use of research evidence to inform health policy decisions [20]. It includes four elements: the general climate for research use, the production of research that is both highly relevant to – and appropriately synthesised for – policymakers, the mix of efforts used to link research to action, and the evaluation of efforts to link research to action. Within this framework, efforts to link research to action are categorised in four clusters of activities. These are: push efforts (efforts to communicate research findings which may include, for example, the tailoring of messages by researchers according to policymaker needs), efforts to facilitate user pull (such as rapid-response units to meet policymaker needs for research evidence), user pull (efforts to facilitate research use, such as efforts to train policymakers in how to access research evidence), and exchange efforts (partnerships between researchers and policymakers in which relevant questions are jointly asked and answered).

Little is known about how best to organise such a range of activities and, until recently, relatively few organisations were responsible for supporting the use of research evidence in developing health policy [21, 22]. The questions that we propose in this article focus on the lessons learned from the experience of organisations engaged in activities to support evidence-informed health policymaking [21]. The evidence from which these lessons were drawn was collected from a survey of 176 organisations, followed by telephone interviews with 25 of these, and site visits to eight. The lessons are:

  • Establish strong links between policymakers and researchers, and involve stakeholders in the work undertaken

  • Be independent and manage conflicts of interest among those involved in the work

  • Use appropriate methods and be transparent in the work

  • Collaborate with other organisations

  • Start small, have a clear audience and scope, and address important questions

  • Build capacity among those working in the organisation

  • Be attentive to implementation considerations even if implementation is not a remit

Questions to consider

Drawing on the above lessons, we suggest five questions that can be asked when considering how to improve support for the use of research evidence to inform health policy decisions. These questions address key strategies to improve how support for evidence-informed health policymaking is organised, as illustrated in Figure 1. They do not address broader questions about government policymaking processes and how these can be designed to promote the use of evidence. The questions are:
  1. 1.

    What is the capacity of your organisation to use research evidence to inform decision making?

     
  2. 2.

    What strategies should be used to ensure collaboration between policymakers, researchers and stakeholders?

     
  3. 3.

    What strategies should be used to ensure independence as well as the effective management of conflicts of interest?

     
  4. 4.

    What strategies should be used to ensure the use of systematic and transparent methods for accessing, appraising and using research evidence?

     
  5. 5.

    What strategies should be used to ensure adequate capacity to employ these methods?

     
Figure 1

Strategies to improve how support for evidence-informed health policymaking is organised

1. What is the capacity of your organisation to use research evidence to inform decision making?

In order for organisations to improve the degree to which their decisions are well-informed by research evidence, sufficient capacity is needed to recognise the need for research evidence. This is necessary for acquiring research when it is needed, critically appraising it, using it to inform decisions, and measuring the impacts of policies and programmes that are implemented [20, 23–27]. Capacities in these different areas vary widely both in governmental and non-governmental organisations [21, 28]. A first step in the process of improving organisational capacity is therefore the assessment of an organisation’s current capacity.

There are a number of validated instruments for measuring the competence of individuals to practice evidence-based medicine e.g.[29–31]. However, in this article, our focus is on organisational capacity rather than the competence of individuals. The Canadian Health Services Research Foundation (CHSRF) has developed a self-assessment tool for healthcare organisations to assist in identifying ways in which research can be gathered and used, as well as potential ways in which this can be improved [23, 24, 32, 33]. This tool includes four key areas for the assessment of research use: the acquisition, assessment, adaptation and application of evidence. Lavis and colleagues have proposed a framework for assessing country-level efforts to link research to action [20]. Their framework includes a number of areas not covered by the CHSRF tool [23]. These include the extent to which the general environment supports the linking of research to action, the production of research, efforts to communicate research findings (so-called ‘push’ strategies), and efforts to facilitate the use of research findings (so-called ‘user pull’ factors).

The self-assessment tool that we present in Additional File 2 draws on both of these frameworks, and the CHSRF tool in particular. It addresses the key steps needed to ensure the appropriate use of research evidence to inform decisions related to health policies and programmes. This tool is intended to help organisations assess and improve their capacity to use research evidence. It has not been formally tested. Instead, it has evolved through iterative revisions based on workshops involving a variety of groups.

Additional File 2 is a ‘scorecard’ intended to provide the basis for discussion and for reaching agreement about the priorities and strategies needed for improvement. Often people in the same organisation have divergent perceptions of how well the organisation is doing. This is illustrated in Additional File 3, which summarises the application of a scorecard to the assessment of an organisation’s performance. Identifying and discussing these discrepancies can help to develop a shared vision and a plan of action. The scorecard shown in Additional File 2 can be applied across departments in a large organisation (as highlighted in the illustrative example shown in Additional File 3), as well as within a department, or a combination of both. The scorecard can also be used to monitor how well an organisation is doing in its efforts to improve its use of research evidence.

2. What strategies should be used to ensure collaboration between policymakers, researchers and stakeholders?

Many organisations that support the use of research evidence in policymaking commonly involve policymakers in the selection of topics and the services undertaken. Personal communication between policymakers and researchers has been found to be particularly important, both by policymakers and those who support their use of research evidence [21]. Organisations that support evidence-informed policymaking view their close links with policymakers as a strength [21]. However, this strength brings with it a related challenge: the need to manage conflicts of interest that can emerge in any close relationship between researchers and policymakers.

Strategies that can help to ensure collaboration between policymakers and researchers include:

  • Locating those who support the use of research by policymakers (by accessing, appraising and summarising evidence) within or close to those organisations responsible for policymaking

  • Involving policymakers on an advisory board or steering committee in instances when organisations are located outside government or policymaking organisations

  • Formal agreements linking academic organisations to policymaking organisations

  • Using trusted individuals as ‘knowledge brokers’ to build relationships among researchers and policymakers [34]

  • Involving policymakers in research processes such as the preparation of policy briefs [35]

  • Involving researchers in policy-informing processes such as policy dialogues [36]

  • Skill development programmes for both policymakers and researchers [37–39] including exchanges where researchers are seconded to a policymaking organisation and policymakers are seconded to a research organisation

An illustration of the need to manage potential tensions between policymakers and researchers who are working together is provided in Table 1.
Table 1

A case study of the need to manage tensions between policymakers and researchers in a long-term collaboration

Since the early 1990s, policymakers in the provincial government of the Free State in South Africa have worked closely with researchers on health and health policy-related topics, including the monitoring and evaluation of antiretroviral (ARV) therapy rollouts [21]. The evidence from these studies has exposed major deficiencies in the ARV rollout, and concerns have been raised that if the research findings become too critical, the privileged data access offered to researchers, and the collaboration offered on evaluations, may simply end. This has led to tensions in the relationship between the researchers and the provincial Health Department with both sides being very direct about these concerns. While acknowledging that it is challenging to manage the tensions, both the policymakers and the researchers are committed to learning how to manage this kind of conflict. From the Health Department’s perspective, this is essential in order to evaluate and improve the services delivered by the provincial government. From the researchers’ perspective, this is motivated by “a feeling that you are doing research that is actually relevant and addressing actual needs as opposed to just driving publications” [21].

Organisations that support the use of research evidence in policymaking also frequently cite the involvement of stakeholders as a key strength [21]. Stakeholder organisations include, for example, patient organisations, community groups, coalitions, advocacy groups, faith-based organisations, charities or voluntary organisations, professional associations, trade unions and business associations [40].

However, managing stakeholder involvement can be both challenging and demanding. There is a paucity of evidence comparing alternative ways of involving stakeholders in policymaking or research processes including [41]:

  • The degree of involvement (consultation or collaboration)

  • Different forums for communication (e.g. committee membership, permanent panels, town meetings, interviews, written consultation)

  • Different methods for recruiting stakeholders (e.g. targeted personal invitations, advertisements, or the use of mass media)

  • Different ways of training and supporting consumers or other stakeholders to ensure effective involvement

  • Different degrees of financial support to facilitate the involvement of consumers or other stakeholders

There is a range of different types of collaboration that may be appropriate for different stakeholders. For some groups, ongoing interaction may be more useful than involving them directly in policymaking (e.g. groups that have an interest in one aspect of a policy, such as professional regulatory issues). For other groups, it may be desirable to keep them at arms length (e.g. pharmaceutical companies with a vested interest in a policy decision). For certain groups, it may be justifiable to exclude them completely from deliberations (e.g. tobacco companies that have falsified research results on the harmful effects of tobacco).

Strategies that can help to ensure appropriate levels of stakeholder involvement are similar to those highlighted above for ensuring collaboration between policymakers and researchers. These may include, for example, the involvement of stakeholders on an advisory board or steering committee, in research processes, and in policymaking processes. They may also include consultation with stakeholder groups, the use of skill-development programmes for stakeholders [42–44], and the communication of evidence to the wider public via the mass media [45].

An example of the use of extensive strategies for involving stakeholders by a public agency is provided in Table 2.
Table 2

An example of stakeholder involvement in healthcare decisions: the National Institute for Health and Clinical Excellence (NICE)

Few organisations have sought to integrate stakeholders (especially patients and their caregivers) more thoroughly than the National Institute for Health and Clinical Excellence (NICE) in England and Wales [50]. NICE has created effective strategies to involve stakeholder groups including [50–52]:

• A programme within the Institute with dedicated staff responsible for patient and public involvement

• The identification and recruitment of stakeholders, including lay people, to NICE’s independent advisory committees

• The provision of training and support to lay people on NICE’s committees

• The registration of stakeholder groups, which are then routinely consulted electronically and through meetings

• The involvement of stakeholders throughout the development of guidance and decisions from topic selection to reviews of draft guidance, through to consultation and active participation on committees

• Systematic and transparent responses to stakeholders’ comments on drafts

• The development and dissemination of lay versions of NICE’s guidance, versions for key stakeholder groups, and mass media briefings, as well as versions for clinicians and managers, and

• The involvement of stakeholders in guidance implementation

NICE’s experience suggests that the involvement of stakeholders in healthcare decision making is possible and can work well, but requires strong commitment and specific arrangements. It can also be costly. Although NICE’s investment in stakeholder involvement is widely valued, it is uncertain whether the right stakeholders are involved, both in terms of which stakeholder groups engage in the process and in terms of the extent to which the individuals who become involved appropriately represent various stakeholders. It is also uncertain whether the strategies they use are as efficient as they could be – in other words, whether the resources invested in those processes represent good value for money [51]. There are also concerns about the growing burden of managing stakeholder input. Although the number of submissions from stakeholders has been increasing, involvement at the individual level within stakeholder organisations may be less than desired.

3. What strategies should be used to ensure independence as well as the effective management of conflicts of interest?

Independence is the most commonly cited strength of organisations that support the use of research evidence in policymaking [21]. Conversely, conflicts of interest are seen as a key weakness. Financial and intellectual independence and freedom from government and industry influence are viewed as the key strengths of such organisations. But these need to be balanced against the desirability of arrangements that can ensure collaboration between policymakers and researchers. Independence is, of course, relative. No organisation is entirely independent.

Mutually agreed processes and methods are essential in order to manage possible competing tensions arising from the demands of both collaboration and independence. They are also important as ways to ensure the systematic and transparent access and appraisal of evidence as an input into the policymaking process.

Conflicting interests frequently underlie tensions arising between policymakers, researchers and other stakeholders. Although there is little empirical evidence to guide arrangements for managing conflicts of interest, the key options that warrant consideration include [46]:

  • Specific, detailed, structured disclosure forms that solicit as much information as possible about the nature and extent of competing interests. Minimal or open-ended formats for disclosure forms are likely to be uninformative

  • Explicit criteria to make decisions easier about whether a disclosed interest constitutes a conflict of interest

  • A range of management strategies to address disclosed conflicts of interest, ranging from the public disclosure of conflicts associated with each meeting as a minimum prerequisite, through to the recusal of conflicted individuals as the most extreme measure

  • A standard policy requiring all financial ties to be made public (e.g. that they be recorded in meeting minutes), may reduce the number of problematic cases

  • A standing committee to review all financial disclosure statements prior to the commencement of committee meetings or hearings, to make management recommendations when necessary, and which can help to ensure that conflict of interest policies are enforced

Organisational arrangements should ensure responsiveness to the information needs of policymakers. At the same time, it is important to ensure independence with respect to the methods used to access, appraise and summarise research evidence. Arrangements to ensure that independence is maintained may include:

  • Financial arrangements that minimise the risk of inappropriate influence on what evidence is summarised, or how it is summarised

  • Management arrangements, including the involvement of independent stakeholders in advisory boards or steering groups

  • Mechanisms for managing disputes such as independent arbitrators or appeal processes, particularly for governmental agencies that fund the work and for industry

  • Ensuring that decision making is transparent in terms of how evidence is accessed, appraised, summarised and publicly reported

4. What strategies should be used to ensure the use of systematic and transparent methods for accessing, appraising and using research evidence?

The majority of organisations supporting the use of research evidence in policymaking use systematic reviews [21]. In addition to their independence, such organisations commonly state that their use of systematic and transparent methods (sometimes they are referred to as “being evidence-based”) is one of their key strengths. However, organisations that support governments to use research evidence in the development of health policies and programmes are less likely to have guidelines describing the methods they use. They are also less likely to conduct or use systematic reviews relative to organisations that produce health technology assessments (HTAs) or clinical practice guidelines. In addition, using systematic and transparent methods brings a related challenge: the time-consuming nature of using more rigorous methods. As a consequence, many organisations, particularly HTA agencies, have attempted to develop more rapid methods that are “quick but clean enough” [47].

Given that evidence-informed health policymaking is characterised by the use of systematic and transparent methods to access and appraise evidence as an input into the policymaking process, it therefore follows that the use of agreed-upon methods for doing this is key for any organisational arrangement to support evidence-informed policymaking. Such methods should be described in easily accessible documents. Moreover, although organisational arrangements are likely to vary widely, a great deal of commonality in the methods that are used is likely, as is the case for clinical practice guidelines, for example [48]. Thus, in addition to helping to ensure the use of agreed-upon methods, accessible manuals that describe these methods can also benefit other organisations with similar interests.

Stakeholders who feel that they have lost out as the result of a particular decision are still likely to challenge the methods used if there is a substantial amount at stake, irrespective of the rigour and transparency applied. Nonetheless, the use of agreed-upon methods that are described in easily accessible form can make it easier to respond to such challenges.

An illustration of efforts to ensure the use of systematic and transparent methods to develop recommendations and policies is provided in Table 3.
Table 3

An example of ensuring the use of systematic and transparent methods in an international organisation

The World Health Organization (WHO) has had guidelines for guidelines since 2003, emphasising the use of systematic reviews for the evidence of effects, processes that allow for the explicit incorporation of other types of information (including values), and evidence-informed dissemination and implementation strategies. However, until 2007 systematic reviews were rarely used for developing recommendations [53]. Instead, processes usually relied heavily on experts in a particular specialty, rather than representatives of those who have to live with the consequences of those recommendations, or experts in particular methodological areas. To address these problems and to ensure the use of systematic and transparent methods, WHO has taken a number of actions, based on a review of its own work and the methods used by others [21, 26, 40, 53–55]. These actions include:

• Revising and updating a manual describing the methods that are to be used, which is updated and revised based on both WHO’s experience and new developments

• Establishing a committee with a mandate to review and approve plans for developing recommendations prior to initiating the work, and recommendations prior to their publication

• Developing checklists for assessing recommendations and plans for developing recommendations based on the manual

• Establishing a secretariat and a network to provide training and support to implement the methods described in the manual, and

• Monitoring and evaluating the impacts of these arrangements to ensure the use of systematic and transparent methods

5. What strategies should be used to ensure adequate capacity to employ these methods?

The most commonly cited weakness of organisations that support the use of research evidence in policymaking are a lack of financial and human resources. How adequate funding for supporting the use of research evidence can be ensured is a major challenge, particularly in low- and middle-income countries. Partly, this may be because this function falls between two stools – it is typically not funded by research funders, or by those interested in strengthening policymaking. Identifying appropriate sources of funding is critical to developing and sustaining adequate capacity for supporting evidence-informed health policymaking.

Three of the key messages that emerged from a review of these organisations relate to ensuring adequate capacity [21, 49]:

  • Collaborate with other organisations, both informally and formally, to learn from their experience in order to avoid the unnecessary duplication of efforts, to draw on their capacity, and to help build capacity (see Table 4 for examples of international collaboration)

Table 4

Examples of collaborations with other organisations

The following are examples of international collaborations that help to build capacity and support for the use of research evidence in health policymaking:

EVIPNet (the Evidence Informed Policy Network) – initiated by the World Health Organization and the Ministries of Health in 25 countries, its aim is to promote the use of research evidence in health policy formulation in order to strengthen health systems [22, 56]. At the country level, EVIPNet takes the form of partnerships between policymakers, researchers and civil society and focuses on facilitating the use of research evidence. Launched in 2005, EVIPNet now supports activities in Africa, Asia and the Americas.

Region of East Africa Community Health (REACH) policy initiative – established within the East African Community (EAC) (Kenya, Tanzania and Uganda, with the recent addition of Rwanda and Burundi) to bridge the gap between evidence and health policy and practice [57]. Its mission is to access, synthesise, package and communicate evidence required for policy and practice and to influence policy-relevant research agendas for improved population health and health equity in each of the member countries.

Reforming States Group (RSG) – since 1991, leaders in health policy from the legislative and executive branches of state government, with the financial support and staff collaboration of the Milbank Memorial Fund, have shared their experiences and have worked on practical solutions to shared healthcare problems. They have focused increasingly on the use of research evidence to inform health policy decisions [38, 39, 58]. The RSG now also includes members outside the United States of America. The Center for Evidence-based Policy, which works with RSG members, was established in 2003 by former Oregon Governor, John Kitzhaber, to address public policy challenges by identifying and applying the best available evidence through self-governing communities of interest [59].

Cochrane Collaboration – a global network whose aim is to improve healthcare decision making through the preparation and updating of systematic reviews of the effects of healthcare interventions. The Cochrane Collaboration ensures that these reviews are made accessible. See http://cochrane.org/

  • Build capacity among those working in the organisation through training, making the best use of available staff (numbers are often limited), and actions aimed at retaining skilled staff, and

  • Start small, have a clear scope, and address important questions in order to ensure that available resources are focused on areas where they are needed most

As noted above, another strategy that many organisations identified was the use of more rapid methods that are rigorous but less resource-intensive – especially those that would result in a reduction in the time required of skilled staff.

Conclusion

A scorecard, such as the one shown in Additional File 2, can be used to assess the capacity of an organisation to support its use of research evidence. This can provide a useful basis for discussion and for establishing consensus about an organisation’s strengths, weaknesses, priorities and the strategies necessary for improvement. Although people in the same organisation often have divergent views about how well it is performing, identifying and discussing these discrepancies can help to develop a shared vision and plan of action. This may be achieved, for example, by sharing information within or across different sections or levels within the organisation, clarifying what different sections of the organisation can or should be doing, addressing misunderstandings, resolving communication problems or identifying information that is needed to resolve disagreements.

There is limited evidence regarding the effects of different strategies to improve how support for evidence-informed health policymaking is organised. Organisational arrangements should logically be tailored to address specific aims and circumstances. Nonetheless, a number of lessons can be drawn from the experience of organisations around the world. Reflection on the questions discussed in this article can help policymakers and those who support them to improve organisational arrangements supporting the use of research evidence to inform health policy decisions.

Resources

Useful documents and further reading

Links to websites

  • Evidence-Informed Policy Network (EVIPNet): http://www.evipnet.org/php/index.php – EVIPNet promotes the systematic use of health research evidence in policymaking. Focusing on low- and middle-income countries, EVIPNet promotes partnerships at the country level between policymakers, researchers and civil society in order to facilitate both policy development and policy implementation through the use of the best scientific evidence available. EVIPNet comprises networks that bring together country-level teams, which are coordinated at both regional and global levels.

  • Alliance for Health Systems Policy and Research: http://www.who.int/alliance-hpsr/en/ –The Alliance for Health Policy and Systems Research is an international collaboration based in the WHO, Geneva. It has its origins in the recommendations of the 1996 report of WHO’s Ad Hoc Committee on Health Research which identified a lack of health policy and systems research as a key problem impeding the improvement of health outcomes in low- and middle-income countries. It aims to promote the generation and use of health policy and systems research as a means to improve the health systems of developing countries.

  • Canadian Health Services Research Foundation: http://www.chsrf.ca – The Foundation brings researchers and decision makers together to create and apply knowledge to improve health services for Canadians. It is an independent, not-for-profit corporation, established with endowed funds from the federal government and its agencies.

Declarations

Acknowledgements

Please see the Introduction to this series for acknowledgements of funders and contributors. In addition, we would like to acknowledge Sara Bennett, Shawna Bourne-Shields, Judith Healy and Anita Kothari for helpful comments on an earlier version of this article.

This article has been published as part of Health Research Policy and Systems Volume 7 Supplement 1, 2009: SUPPORT Tools for evidence-informed health Policymaking (STP). The full contents of the supplement are available online at http://www.health-policy-systems.com/content/7/S1

Authors’ Affiliations

(1)
Norwegian Knowledge Centre for the Health Services
(2)
Norwegian Knowledge Centre for the Health Services, Norway and Department of Medicine, Innlandet Hospital Health Authority
(3)
Centre for Health Economics and Policy Analysis, Department of Clinical Epidemiology and Biostatistics, and Department of Political Science, McMaster University
(4)
Norwegian Knowledge Centre for the Health Services, Section for International Health, Institute of General Practice and Community Medicine, Faculty of Medicine, University of Oslo, Norway
(5)
Norwegian Knowledge Centre for the Health Services, Health Systems Research Unit, Medical Research Council of South Africa

References

  1. Lavis JN, Oxman AD, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP). Introduction. Health Res Policy Syst. 7 (Suppl 1): I1-10.1186/1478-4505-7-S1-I1.Google Scholar
  2. Granados A, Jonsson E, Banta HD, Bero L, Bonair A, Cochet C, Freemantle N, Grilli R, Grimshaw J, Harvey E: EUR-ASSESS Project Subgroup Report on Dissemination and Impact. Int J Technol Assess Health Care. 1997, 13: 220-86. 10.1017/S0266462300010370.View ArticlePubMedGoogle Scholar
  3. Sevene E, Lewin S, Marino A, Woelk G, Oxman A, Matinhure S: Can a drug be too cheap? The unavailability of magnesium sulphate for the treatment of eclampsia and preeclampsia in Mozambique and Zimbabwe: systems and market failures. BMJ. 2005, 331: 765-10.1136/bmj.331.7519.765.PubMed CentralView ArticlePubMedGoogle Scholar
  4. Trostle J, Bronfman M, Langer A: How do researchers influence decision-makers? Case studies of Mexican policies. Health Policy Plan. 1999, 14: 103-14. 10.1093/heapol/14.2.103.View ArticlePubMedGoogle Scholar
  5. Working Group on Research to Action and Policy: Lessons in Research to Action and Policy – Case studies from seven countries. 2000, Council of Health Research and Development (COHRED), [http://www.cohred.org/libarchive/content/696.pdf]Google Scholar
  6. Tomson G, Paphassarang C, Jonsson K, Houamboun K, Akkhavong K, Wahlstrom R: Decision-makers and the usefulness of research evidence in policy implementation--a case study from Lao PDR. Soc Sci Med. 2005, 61: 1291-9. 10.1016/j.socscimed.2005.01.014.View ArticlePubMedGoogle Scholar
  7. Aaserud M, Lewin S, Innvaer S, Paulsen EJ, Dahlgren AT, Trommald M, Duley L, Zwarenstein M, Oxman AD: Translating research into policy and practice in developing countries: a case study of magnesium sulphate for pre-eclampsia. BMC Health Serv Res. 2005, 5: 68-10.1186/1472-6963-5-68.PubMed CentralView ArticlePubMedGoogle Scholar
  8. Albert MA, Fretheim A, Maiga D: Factors influencing the utilization of research findings by health policy-makers in a developing country: the selection of Mali's essential medicines. Health Res Policy Syst. 2007, 5: 2-10.1186/1478-4505-5-2.PubMed CentralView ArticlePubMedGoogle Scholar
  9. Lavis JN, Davies HTO, Oxman A, Denis JL, Golden-Biddle K, Ferlie E: Towards systematic reviews that inform healthcare management and policymaking. J Health Serv Res Policy. 2005, 10: 35-48. 10.1258/1355819054308549.View ArticlePubMedGoogle Scholar
  10. Innvaer S, Vist G, Trommald M, Oxman A: Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002, 7: 239-44. 10.1258/135581902320432778.View ArticlePubMedGoogle Scholar
  11. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N: Lost in knowledge translation: time for a map?. J Contin Educ Health Prof. 2006, 26: 13-24. 10.1002/chp.47.View ArticlePubMedGoogle Scholar
  12. Nutley S, Walter I, Davies HTO: From knowing to doing: a framework for understanding the evidence-into-practice agenda. Evaluation. 2003, 9: 125-48. 10.1177/1356389003009002002.View ArticleGoogle Scholar
  13. Sauerborn R, Nitayarumphong S, Gerhardus A: Strategies to enhance the use of health systems research for health sector reform. Trop Med Int Health. 1999, 4: 827-35. 10.1046/j.1365-3156.1999.00497.x.View ArticlePubMedGoogle Scholar
  14. Lomas J: Connecting research and policy. Can J Policy Res. 2000, 1: 140-4.Google Scholar
  15. Dobbins M, Ciliska D, Cockerill R, Barnsley J, DiCenso A: A framework for the dissemination and utilization of research for health-care policy and practice. Online J Knowl Synth Nurs. 2002, 9: 7-PubMedGoogle Scholar
  16. Jacobson N, Butterill D, Goering P: Development of a framework for knowledge translation: understanding user context. J Health Serv Res Policy. 2003, 8: 94-9. 10.1258/135581903321466067.View ArticlePubMedGoogle Scholar
  17. Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J: How can research organizations more effectively transfer research knowledge to decision makers?. Milbank Q. 2003, 81: 221-2. 10.1111/1468-0009.t01-1-00052.PubMed CentralView ArticlePubMedGoogle Scholar
  18. Dobrow MJ, Goel V, Lemieux-Charles L, Black NA: The impact of context on evidence utilization: a framework for expert groups developing health policy recommendations. Soc Sci Med. 2006, 63: 1811-24. 10.1016/j.socscimed.2006.04.020.View ArticlePubMedGoogle Scholar
  19. Bowen S, Zwi AB: Pathways to “evidence-informed” policy and practice: a framework for action. PLoS Med. 2005, 2: e166-10.1371/journal.pmed.0020166.PubMed CentralView ArticlePubMedGoogle Scholar
  20. Lavis JN, Lomas J, Hamid M, Sewankambo NK: Assessing country-level efforts to link research to action. Bull World Health Organ. 2006, 84: 620-8. 10.2471/BLT.06.030312.PubMed CentralView ArticlePubMedGoogle Scholar
  21. Moynihan R, Oxman A, Lavis JN, Paulsen E: Evidence-Informed Health Policy: Using Research to Make Health Systems Healthier. 2008, Nasjonalt kunnskapssenter for helsetjenesten, Rapport nr. 1-2008, [http://www.kunnskapssenteret.no/binary?download=true%26id=765]Google Scholar
  22. Hamid M, Bustamante-Manaog T, Dung TV, Akkhavong K, Fu H, Ma Y, Zhong X, Salmela R, Panisset U, Pang T: EVIPNet: translating the spirit of Mexico. Lancet. 2005, 366: 1758-9. 10.1016/S0140-6736(05)67709-4.View ArticlePubMedGoogle Scholar
  23. The Canadian Health Services Research Foundation (CHSRF): Is research working for you? A self-assessment tool and discussion guide for health services management and policy organizations. 2008, Canadian health Services Research Foundation, [http://www.chsrf.ca/other_documents/working_e.php]Google Scholar
  24. Thornhill J, Judd M, Clements D: CHSRF knowledge transfer: (re)introducing the self-assessment tool that is helping decision-makers assess their organization's capacity to use research. Healthc Q. 2009, 12: 22-4.View ArticlePubMedGoogle Scholar
  25. Grol R, Wensing M, Eccles M: Improving patient care: implementing change in clinical practice. 2005, Oxford, ElsevierGoogle Scholar
  26. Oxman AD, Fretheim A, Schunemann HJ: Improving the use of research evidence in guideline development: introduction. Health Res Policy Syst. 2006, 4: 12-10.1186/1478-4505-4-12.PubMed CentralView ArticlePubMedGoogle Scholar
  27. Akl EA, Treweek S, Foy R, Francis J, Oxman AD: NorthStar, a support tool for the design and evaluation of quality improvement interventions in healthcare. Implement Sci. 2007, 2: 19-10.1186/1748-5908-2-19.PubMed CentralView ArticlePubMedGoogle Scholar
  28. Alliance for Health Policy and Systems Research: Strengthening health systems: the role and promise of policy and systems research. 2004, Alliance for Health Policy and Systems Research, [http://www.who.int/alliance-hpsr/resources/Strengthening_complet.pdf]Google Scholar
  29. Taylor R, Reeves B, Mears R, Keast J, Binns S, Ewings P, Khan K: Development and validation of a questionnaire to evaluate the effectiveness of evidence-based practice teaching. Med Educ. 2001, 35: 544-7. 10.1046/j.1365-2923.2001.00916.x.View ArticlePubMedGoogle Scholar
  30. Johnston JM, Leung GM, Fielding R, Tin KY, Ho LM: The development and validation of a knowledge, attitude and behaviour questionnaire to assess undergraduate evidence-based practice teaching and learning. Med Educ. 2003, 37: 992-1000. 10.1046/j.1365-2923.2003.01678.x.View ArticlePubMedGoogle Scholar
  31. Ramos KD, Schafer S, Tracz SM: Validation of the Fresno test of competence in evidence based medicine. BMJ. 2003, 326: 319-21. 10.1136/bmj.326.7384.319.PubMed CentralView ArticlePubMedGoogle Scholar
  32. Kothari A, Edwards N, Hamel N, Judd M: Is research working for you? Validating a tool to examine the capacity of health organizations to use research. Implement Sci. 2009, 4: 46-10.1186/1748-5908-4-46.PubMed CentralView ArticlePubMedGoogle Scholar
  33. Gonzalez-Block MA, Mercado FJ, Ochoa H, Rivera H, Idrovo JI: Utilización de investigación por gestores de salud. Desarrollo y validación de una herramienta de autodiagnóstico para países de habla hispana. Salud Publica Mex. 2008, 50: 490-7.PubMedGoogle Scholar
  34. Lomas J: The in-between world of knowledge brokering. BMJ. 2007, 334: 129-32. 10.1136/bmj.39038.593380.AE.PubMed CentralView ArticlePubMedGoogle Scholar
  35. Lavis JN, Permanand G, Oxman AD, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP). 13. Preparing and using policy briefs to support evidence-informed policymaking. Health Res Policy Syst in press.Google Scholar
  36. Lavis JN, Boyko J, Oxman AD, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP). 14. Organising and using policy dialogues to support evidence-informed policymaking. Health Res Policy Syst. 7 (Suppl 1): S14-10.1186/1478-4505-7-S1-S14.Google Scholar
  37. The Canadian Health Services Research Foundation (CHSRF): Executive Training for Research Application (EXTRA). 2009, Canadian Health Services Research Foundation, [http://www.chsrf.ca/extra]Google Scholar
  38. Fox DM, Greenfield L: Helping public officials use research evaluating healthcare. J Law Policy. 2006, 14: 531-50.Google Scholar
  39. Jewell CJ, Bero LA: “Developing good taste in evidence”: Facilitators of and hindrances to evidence-informed health policymaking in state government. Milbank Quarterly. 2008, 86: 177-208. 10.1111/j.1468-0009.2008.00519.x.PubMed CentralView ArticlePubMedGoogle Scholar
  40. Oxman AD, Lewin S, Lavis JN, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP). 15. Engaging the public in evidence-informed policymaking. Health Res Policy Syst. 7 (Suppl 1): S15-10.1186/1478-4505-7-S1-S15.Google Scholar
  41. Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD: Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database Syst Rev. 2006, 3: CD004563-PubMedGoogle Scholar
  42. Milne R, Oliver S: Evidence-based consumer health information: developing teaching in critical appraisal skills. Int J Qual Health Care. 1996, 8: 439-45.PubMedGoogle Scholar
  43. Dickersin K, Braun L, Mead M, Millikan R, Wu AM, Pietenpol J, Troyan S, Anderson B, Visco F: Development and implementation of a science training course for breast cancer activists: Project LEAD (leadership, education and advocacy development). Health Expect. 2001, 4: 213-20. 10.1046/j.1369-6513.2001.00153.x.View ArticlePubMedGoogle Scholar
  44. United States Cochrane Center: Understanding evidence-based healthcare: a foundation for action. 2009, United States Cochrane Centre, [http://apps1.jhsph.edu/cochrane/CUEwebcourse.htm]Google Scholar
  45. Grilli R, Ramsay C, Minozzi S: Mass media interventions: effects on health services utilisation. Cochrane Database Syst Rev. 2002, 1: CD000389-PubMedGoogle Scholar
  46. Boyd EA, Bero LA: Improving the use of research evidence in guideline development: 4. Managing conflicts of interests. Health Res Policy Syst. 2006, 4: 16-10.1186/1478-4505-4-16.PubMed CentralView ArticlePubMedGoogle Scholar
  47. Oxman AD, Schunemann HJ, Fretheim A: Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence. Health Res Policy Syst. 2006, 4: 20-10.1186/1478-4505-4-20.PubMed CentralView ArticlePubMedGoogle Scholar
  48. Schunemann HJ, Fretheim A, Oxman AD: Improving the use of research evidence in guideline development: 1. Guidelines for guidelines. Health Res Policy Syst. 2006, 4: 13-10.1186/1478-4505-4-13.PubMed CentralView ArticlePubMedGoogle Scholar
  49. Green A, Bennett S: Sound Choices: Enhancing Capacity for Evidence-Informed Health Policy. 2007, World Health Organization, [http://www.who.int/alliance-hpsr/resources/Alliance_BR.pdf]Google Scholar
  50. Culyer AJ: Involving stakeholders in healthcare decisions--the experience of the National Institute for Health and Clinical Excellence (NICE) in England and Wales. Healthc Q. 2005, 8: 56-60.View ArticlePubMedGoogle Scholar
  51. de Joncheere K, Hill SR, Klazinga N, Makela M, Oxman A: The Clinical Guideline Programme of the National Institute for Health and Clinical Excellence. A review by the WHO. 2007, NICE, [http://www.nice.org.uk/niceMedia/pdf/boardmeeting/brdjan07item2b.pdf]Google Scholar
  52. National Institute for Health and Clinical Excellence: About the Patient and Public Involvement Programme (PPIP). 2009, National Institute for Health and Clinical Excellence, [http://www.nice.org.uk/media/D44/49/AboutPPIP.pdf]Google Scholar
  53. Oxman AD, Lavis JN, Fretheim A: The use of evidence in WHO recommendations. Lancet. 2007, 369: 1883-9. 10.1016/S0140-6736(07)60675-8.View ArticlePubMedGoogle Scholar
  54. Hill S, Pang T: Leading by example: a culture change at WHO. Lancet. 2007, 369: 1842-4. 10.1016/S0140-6736(07)60676-X.View ArticlePubMedGoogle Scholar
  55. Lancet editorial: WHO signals strong commitment to evidence. Lancet. 2007, 369 (9574): 1666-Google Scholar
  56. EVIPNet Americas Secretariat: EVIPNet Americas: informing policies with evidence. Lancet. 2008, 372: 1130-1. 10.1016/S0140-6736(08)61459-2.View ArticleGoogle Scholar
  57. van Kammen J, de Savigny D, Sewankambo N: Using knowledge brokering to promote evidence-based policy-making: The need for support structures. Bull World Health Organ. 2006, 84: 608-12. 10.2471/BLT.05.028308.PubMed CentralView ArticlePubMedGoogle Scholar
  58. Andersen KS: The reforming states group and the promotion of federalism. Milbank Q. 1998, 76: 103-20. 10.1111/1468-0009.00080.PubMed CentralView ArticlePubMedGoogle Scholar
  59. Center for Evidence-based Policy: Addressing public policy challenges by identifying and applying best available evidence through self-governing communities of interest. 2009, Center for Evidence-based Policy, [http://www.ohsu.edu/ohsuedu/research/policycenter/index.cfm]Google Scholar

Copyright

© Oxman et al; licensee BioMed Central Ltd. 2009

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement