Table 2 Findings from formative evaluations of activities and outputs*
Domain | Themes | Factors influencing weight given to themes | ||
|---|---|---|---|---|
Larger number of studies | Higher-quality studies | Studies in different contexts | ||
Building demand | Awareness raising  • One study found that increasing awareness among policy-makers, stakeholders and researchers about the importance of initiatives to support evidence-informed policy-making was regarded as an organisational strength across seven KT platforms [33] – 2/5 (formative score 1/3) | No (1) | No | Yes (7) |
Prioritisation and co-production | Priority-setting exercises for activities and outputs  • One study found that prioritising operational research was consistently regarded as an organisational strength across three KT platforms, with one each operating in South Africa, Thailand and Uganda [37] – 2.5/3 (formative score 2/3) | No (1) | Yes (1) | No (3) |
Packaging, push, and support to implementation | Evidence briefs  • Six studies examined EVIPNet-style evidence briefs [33, 47, 52, 55, 57, 59] – 2/5 (formative score 1/3), 3/5 (formative score 2/3), 1.5/3, 1/5, 2/3, 2.5/5, respectively   ◦ Four studies found that they are highly regarded by policy-makers and stakeholders in Bangladesh, Nigeria and Zambia [33, 47, 55, 57], with the highest-quality study finding that EVIPNet-style evidence briefs were highly rated by policy-makers and stakeholders in Burkina Faso, Cameroon, Ethiopia, Nigeria, Uganda and Zambia – both in terms of whether they achieved their objective and in terms of their key design features – regardless of country, group or issue involved [47]   ◦ Two studies found that some readers struggled with them not concluding with recommendations [47, 52] while one study found that respondents’ self-reported professional roles being other than ‘policy-maker’ or ‘stakeholder’ was a significant predictor of giving a lower helpfulness score to evidence briefs not concluding with recommendations [47]   ◦ One study found that the graded-entry format of briefs is viewed a very favourable element [59] | Yes (6) | Yes (2) | No (3) |
Facilitating pull | Rapid evidence services  • Four studies examined the rapid evidence service in Uganda [44,45,46, 60] – 1/5, 1.5/3, 1/3, 1/3 (formative score 0/3), respectively   ◦ One study found that key success factors for such services include awareness of user needs (i.e. consultation with policy-makers), the opportunity for feedback from users (i.e. being a personalised service) and working within current norms and behaviours of users [46]   ◦ A second study identified regular contact between policy-makers and researchers (i.e. service staff) as a key factor in the uptake of, and response to, the service [45]   ◦ Two studies found that the rapid syntheses produced by these services are perceived as a desirable and user-friendly output by policy-makers and stakeholders [44, 46]   ◦ One study found identified as favourable aspects of rapid syntheses their policy relevance and right time frame for production [60]   ◦ Another study identified as aspects of rapid syntheses not always meeting expectations, the speed by which they were produced/delivered, their quality, the degree of contextualisation [60] and the absence of recommendations [44] | Yes (4) | No | No (1) |
Building capacity to use (and support the use of) research evidence  • Two studies examined training programmes for health policy advisory committee members to improve their use of research evidence in policy-making but neither identified explicitly the key findings from a formative evaluation [27, 58] – 1.5/3 (formative score 1/3), 0/3, respectively   ◦ One study described two 5-day training workshops that included sessions focused on the role of knowledge brokers, research methodology and writing, and impact evaluation [27]   ◦ A second study described both a 1-day training workshop on evidence briefs, deliberative dialogues and priority-setting, and a 3-month training programme focused on enhancing capacity for research, evidence-informed policy-making and health policy advocacy, leadership development in resource-limited areas, and health policy monitoring, evaluation and performance assessment [58] | No (2) | No | No (1 and 1) | |
Exchange | Deliberative dialogues  • Six studies examined deliberative dialogues informed by evidence briefs and found them to be highly regarded as a tool for enhancing evidence-informed policy-making [33, 47, 52, 55, 57, 59] – 2/5 (formative score 1/3), 3/5 (formative score 2/3), 1.5/3, 1/5, 2/3, 2.5/5, respectively   ◦ The highest-quality study found that EVIPNet-style deliberative dialogues were highly rated by policy-makers and stakeholders in Burkina Faso, Cameroon, Ethiopia, Nigeria, Uganda and Zambia – both in terms of whether they achieved their objective and in terms of their key design features – regardless of country, group or issue involved [47]   ◦ The same study found that participants without past research experience were more likely to associate ‘not aiming for consensus’ with a lower rating of the helpfulness of the dialogues [47] | Yes (6) | Yes (2) | No (3) |
Research-to-policy workshops  • Two studies examined research-to-policy workshops and found that they were perceived to have helped promote improved/new opportunities for collaboration and networks, increased/new knowledge, policy brief writing skills, and an enhanced understanding of the importance of research and evidence-based decision-making [43, 50] – 1/3, 1/5, respectively   ◦ One study held four successive workshops focused on research evidence and its policy implications, hands-on skills and policy brief writing, and presentation of policy briefs [43]   ◦ Another study found that participants in a 3-day international forum on evidence-informed policy-making (1) identified four areas for improvement – smaller programme to accommodate more time for discussions; clearer meeting objectives; further exploration of evidence-informed policy-making initiative sustainability; and inclusion of training on writing policy briefs; and (2) highlighted presentations on country experiences and impact evaluation/analysis sessions as the most enjoyable [50] | No (2) | No | No (2) | |