Skip to main content

Box 1 Past work on assessing research demand and use in ministries of health

From: Assessing the capacity of ministries of health to use research in decision-making: conceptual framework and tool

Is Research Working for You? A Self-Assessment Tool and Discussion Guide for Health Services Management and Policy Organisation

Developed by the Canadian Health Services Research Foundation (CHSRF) in 1999 to examine and facilitate discussion around the capacity of health service management and policy organisations to use research evidence in making decisions [6, 9]. Focuses on four areas of assessment: acquire, assess, adapt and apply, with questions related to these ‘four As’ and a discussion guide for the participants to discuss research utilisation in the organisation in a group setting. The tool was validated with respondents in Canada [10] and in Colombia, Argentina, Mexico and Georgia [4], and results suggest that it was useful to understand organisational capacity; however, there were questions about how to best apply the discussion portion given how useful it could be but also subject to imbalanced power dynamics among participants in hierarchical settings.

Key limitations include (1) strict focus at the organisation level, with little to no assessment of individual or systems capacities and how they influence the uptake of research evidence, and (2) despite application across country settings, government participation appears to be limited.

SUPPORT Tools

The Supporting Policy Relevant Reviews and Trials (SUPPORT) project developed a set of tools for increasing well-informed and evidence-informed decision-making targeting primarily policymakers, non-governmental organisations and civil society groups, both in low- and middle-income and high-income country settings [11]. The tools address four broad areas related to policymaking: “1) supporting evidence-informed policymaking, 2) identifying needs for research evidence, 3) finding and assessing evidence, 4) going from research evidence to decisions” [11]. As part of the SUPPORT Tools, a specific tool for organisational capacity to support the use of research evidence to inform decisions was also developed [8]. The tool consists of seven sections that assess:

• Organisational culture and values to support the use of research evidence to inform decisions

• Setting priorities for obtaining research evidence

• Obtaining research evidence

• Assessing the quality and applicability of research evidence and interpreting the results

• Using research evidence to inform recommendations and decisions

• Monitoring and evaluating policies and programmes

• Supporting continuing professional development

It is not clear whether the SUPPORT Tools have been formally tested for reliability and validity, or whether there is a standard method for involving particular types of participants to obtain more standard measures and benchmarks. For the specific tool on organisational capacity, the focus is on the organisation with no specific assessment of individual or systems capacities.

Data Demand and Information Use (DDIU) in the Health Sector

Developed by MEASURE Evaluation focusing on generating and collecting health information data with a view towards informing policymaking [7]. The ‘Checklist for DDIU Assessment’ is not a fixed instrument but provides guidance on assessing technical, organisational and behavioural/individual constraints that can affect the demand and supply of data. The approach is not prescriptive and specific, but asks broader questions about potential barriers to collecting, sharing and using data. While DDIU takes on individual, organisational and systems capacities, its focus is on generation and use of data rather than research evidence.

Other Instrument

Boyko et al. [5] developed an instrument based on the theory of planned behaviour to evaluate the intention of policymakers to use research. The instrument was tested with Canadian policymakers and stakeholders who had participated in deliberative dialogues about relevant topics, and was found to be reliable. However, due to the small sample size, the instrument’s validity could not be assessed. This instrument is focused almost exclusively at the individual level, with assessments of individual attitudes and expectations towards use research. Additionally, it is not clear whether this instrument would be applicable in other settings, especially in low- and middle-income countries.