Table 3 Findings from summative evaluations of outcomes and impact*
Domain | Themes | Factors influencing weight given to themes | ||
|---|---|---|---|---|
Larger number of studies | Higher-quality studies | Studies in different contexts | ||
Impacts on policy-making processes | KT platforms  • Ten studies reported a total of 23 KT platforms conducting activities and outputs that collectively led to some direct impacts on select policy-making processes, although the number of policy-making processes influenced and the nature of these impacts were often not described [24, 33, 36, 46, 47, 51, 53,54,55, 59] – 1.5/5, 2/5, 1.5/5, 1/5, 3/5, 2/5, 1/4, 2.5/5, 1/5, 2.5/5, respectively | Yes (10) | Yes (1) | Yes (23) |
Evidence briefs and deliberative dialogues  • Four studies examined EVIPNet-style evidence briefs and deliberative dialogues [33, 47, 55, 59], – 2/5, 3/5, 1/5, 2.5/5, respectively, and found that they:   ◦ led to strong intentions to act among dialogue participants in Burkina Faso, Cameroon, Ethiopia, Nigeria, Uganda and Zambia [47]   ◦ had direct impacts on select policy-making processes in Bangladesh and Cameroon (both for evidence briefs alone and the combination of briefs and dialogues) and in nine countries with active KT platforms (for the combination of briefs and dialogues) [33, 47, 55, 59]  • One study found that policy briefs different from EVIPNet-style evidence briefs led to direct impacts on select policy-making processes in South Africa [54] – 2.5/5 | Yes (4) | Yes (1) | Yes (12) | |
Other activities and outputs  • Other studies that examined activities and outputs that led to direct impacts on select policy-making processes focused on rapid evidence services/rapid syntheses [46] – 1/5, research publications [24, 54] – 1.5/5, 2.5/5, respectively, participation in government meetings [24] – 1.5/5, translation and appraisal of research findings [36] – 1.5/5, and research-to-policy meetings [50, 51] – 1/5, 2/5, respectively | No for any given activity/output | No | No for any given activity/output | |
Outcomes – More policy-relevant research evidence available | Project-outcomes evaluation (as an activity)  • One study examined the influence of an activity on this outcome and found that 73% of individuals involved in project outcome evaluation in Bangladesh believed that the project increased access to research evidence [55] – 1/5 KT platforms  • One study examined the influence of two KT platforms on a range of outcomes related to policy-relevant research evidence being available (e.g. more funding for monitoring and evaluation) [53] – 1/4 | No (1) | No | No (1) |
Outcomes – Stronger relationships between policy-makers and researchers | KT platforms  • Three studies examined the influence of KT platforms on this outcome:   ◦ eight KT platforms reported strengthened relationships among policy-makers, stakeholders, researchers [33] – 2/5   ◦ one KT platform reported new spaces for deliberations on priority health policy issues having been created through a network of local and global factors and agents [52] – 1.5/3   ◦ another KT platform reported that relationships among policy-makers, stakeholders and researchers were strengthened as a result of a deliberative dialogue organised by the KT platform, with future meetings and workshops held independently to discuss implementation as an example of such strengthening [59] – 2.5/5 Project outcome evaluation (as an activity)  • The same study from Bangladesh reported above found that 73% of individuals involved in project outcome evaluation believed that the project cemented relationships between policy-makers and researchers [55] – 1/5 | No (3 or 1) | No | Yes (8+ for KT platforms) |
Outcomes – Greater policy-maker capacity to use research evidence | Workshops and other forms of training  • Four studies examined the influence of workshops and other forms of training on this outcome:   ◦ one study found that workshops honed policy brief writing skills, increased scientific knowledge and networks with researchers, and increased awareness of the importance of research and evidence-based decision-making [43] – 1/3   ◦ a second study found that an international forum with a partial focus on capacity-building led less than half of participants to report new skills as a benefit overall (46%) but one-fifth (19%) reported an intent to utilise new skills [50] – 1/5   ◦ a third study found that a workshop improved participants’ knowledge, understanding of policy-making and use of evidence [56] – 2.5/3   ◦ a fourth study found that training future policy-makers was a key contributor to their policy influence success [54] – 2.5/5 ‘Buddy’ programme that pairs policy-makers and researchers  • One study found that Policy BUDDIES (Policy BUilding Demand for evidence in Decision-making through Interaction and Enhancing Skills) enhanced the capacity of subnational policy-makers to ask for, demand and use systematic review evidence (and other products of evidence syntheses) to inform policy-making [36] – 1.5/5 Advisory committee  • One study found that a Health Policy Advisory Committee improved knowledge about evidence-to-policy links, KT and operationalisation of KT amongst Health Policy Advisory Committee members as well as their capacity to find and use evidence [58] – 0/3 | No for any given form | No | No for any given form |
Outcomes – Other (awareness and demand) | KT platforms  • Two studies examined the influence of KT platforms on other outcomes:   ◦ seven KT platforms reported they have increased awareness of the importance of initiatives supporting evidence-informed policy-making [33] – 2/5   ◦ six KT platforms reported higher policy-maker demand for KT products [33] – 2/5   ◦ one KT platform reported greater awareness of and demand for KT tools amongst policy-makers, stakeholders and researchers as a result of a deliberative dialogue organised by the KT platform [59] – 2.5/5 | No (2) | No | Yes (7) |