A literature review of knowledge translation and partnership research training programs for health researchers

Background Researchers and policy-makers are increasingly working together with the goal of creating research that is focused on solving real-world problems; however, knowledge translation (KT) activities, and the partnerships they often require, can be challenging. The aim of this review is to determine the extent of the literature on training programs designed to improve researcher competency in KT and to describe existing training methods that may be used by those hoping to build capacity for partnership research. Methods MEDLINE, EMBASE, PsycINFO and CINAHL were searched for peer review articles published between January 2000 and July 2019. Studies were eligible for inclusion in the review if they described the development of, curriculum for, or evaluation of KT and/or partnership research training programs. Data extraction included information on evaluation methods, outcomes and implications as well as the format, aims and themes of each capacity-building program. Results The review identified nine published articles that met inclusion criteria – four papers described training events, two papers described participant experiences of specific learning sessions within a larger training course, two papers described part time secondments for KT capacity-building and one paper described a plan for KT training embedded within an existing research training course. All programs were delivered face-to-face, all included practical skills-building opportunities, and all employed multiple learning modalities such as seminars and small group discussions. Evaluation of the training programs was primarily conducted through qualitative interviews or feedback surveys. Conclusion To date, few KT training initiatives have been described in the literature and none of these have been rigorously evaluated. The present review offers insights into the planning, development and participant experiences associated with the small number of training initiatives that have been described. There is insufficient evidence available at present to identify the most effective models for training researchers in KT and partnership skills.


Background
Funding bodies and researchers internationally are increasingly interested in ensuring that research has realworld impact, informing policy and/or practice [1][2][3][4]. The field of knowledge translation (KT) is concerned with how to bridge the research/research-user divide [5] and has been defined by the Canadian Institutes of Health Research as the "exchange, synthesis and ethically-sound application of knowledge -within a complex system of interactions among researchers and usersto accelerate the capture of the benefits of research" [2]. Common KT activities undertaken by researchers include disseminating research findings to policy-makers and creating evidence summary materials that are accessible and relevant to a policy audience [6,7]. Research has also been undertaken to determine how best to build the capacity of policymakers to engage with research and to understand the barriers they face in doing so [8][9][10].
More recently, those advocating for reductions in research wastage [11] have asserted that communicating the end products of research clearly for a policy audience is not enough to enhance translation if the evidence in and of itself is not relevant to the interests and needs of policy-makers and practitioners. Partnering with potential research users throughout the research process has been posited as a potentially effective method of improving the policy and practice relevance of research [12,13]. This style of research, where there is ongoing engagement between decision-makers (who bring with them contextual and tacit knowledge about what evidence is needed in practice) and researchers (who bring skills in research methodology), is becoming increasingly popular. Such collaborations are thought to increase the likelihood of evidence translation because policy-makers are involved in determining the research question (increasing relevance) and in the evidence-generation process (increasing the extent to which they are aware of and understand the findings) [13][14][15][16][17]. This mode of research, which places a strong emphasis on partnership, is often termed integrated KT (IKT) [14,15,18].
Improving the use of evidence in policy through increased collaboration with researchers is a deceptively simple idea; however, partnership research activities are often met with challenges resulting from the different needs, expectations and cultures in each professional context [3,[19][20][21]. In recognition of these challenges, interest in building capacity for effective KT and IKT work has been steadily growing [22]. A key contribution in this area was Straus et al.'s [22] mapping of four core competencies in KT, namely understanding models and theories of KT and KT research, capacity to conduct synthesis, capacity in KT research methods, and ability to evaluate the impact, effectiveness and sustainability of KT strategies. More recently, Mallidou et al. [23] built on these competencies and, after completing a scoping review that included grey literature and KT job descriptions, identified 19 competencies. They grouped these competencies into three categories as 'knowledge', 'skills' and 'attitudes'. Some themes identified under 'knowledge' and 'skills' included understanding the context, understanding the research process, understanding translation and dissemination activities, dissemination of research findings, knowledge brokering, fostering innovation, and use of research findings.
Understanding the competencies required to succeed in KT work provides a framework to understand KT training activities. The focus of this paper is published literature about training courses designed to build capacity in KT, IKT or partnership research that involved health researchers.
The aims of this study are two-fold. Firstly, to describe any KT and/or IKT partnership training directed at health researchers with respect to the methods of delivery, course content and training themes. Secondly, to examine the evaluation approaches used to establish the effectiveness of the training in achieving KT knowledge and skills.

Search strategy
A broad initial literature search was performed to establish major themes and to develop the final search strategy. MEDLINE, EMBASE, PsycINFO and CINAHL were searched for peer review articles published between 2000 and 2017 in October 2017 and articles published between 2017 and 2019 on 1 July 2019. A Boolean search of abstracts was performed using the terms 'knowledge translation' OR 'integrated knowledge translation' OR 'knowledge transfer' OR 'co-produc* research' OR 'partner* research' AND training using truncation (indicated by *) to allow for multiple word endings (e.g. co-produced research or co-production). The results were limited to humans and articles published in English between 2000 and 2019. The initial search returned 1041 and 516 results, respectively. After duplicates were removed in Endnote, there were 683 titles and abstracts to review in 2017 and 349 titles and abstracts to review in 2019 (Fig. 1).

Process for study selection
Initially, 30 papers were identified as potentially relevant (22 in 2017 and a further 8 in 2019), and reference lists of selected studies were scanned to identify relevant articles that may not have been captured in the initial search. Studies included in the review were peer reviewed articles describing the development, curriculum for, or evaluation of KT and partnership research training programs (using the most inclusive definition of these terms possible, as per the search terms listed above). The inclusion criteria were kept broad so as to include as many papers as possible, which might have relevant information pertaining to training of researchers in KT. Studies were excluded if they described a KT intervention or a training course focused on improving evidence-based practice among health professionals. Studies assessing KT training needs (to inform future training programs) or mentorship programs were beyond the scope of this review [24][25][26]. Studies describing training programs for dissemination and implementation science [27,28] were also excluded.

Data extraction
In order to describe the training initiatives about which published literature was available, we extracted information pertaining to the country in which the training was conducted, training participant type (e.g. researchers, policy-makers), the duration and modality of the training activity, and the aims and focus areas addressed by each initiative. In addition, we generated a brief description of each training activity.
In order to assess both the outcomes of the reported evaluations and their quality, we extracted information on the number of participants, the evaluation methods used, the outcomes of the evaluation and the reported implications of the evaluation, and assessed what level of evidence each evaluation reported according to National Health and Medical Research Council (NHMRC) level of evidence standards [29]. The Australian NHMRC's level of evidence hierarchy is commonly used in systematic reviews and clinical guidelines. It consists of four levels of evidence (I, II, III-1, III-2, III-3 and IV) with evidence from a systematic review of randomised controlled studies having a rating of I and cohort studies having a rating of III-2.

Results
The review identified nine published reports describing KT capacity-building activities for researchers. The studies came from Canada (n = 3), Nigeria (n = 3), America (n = 1), Australia (n = 1) and England (n = 1). The aims and training approaches of each program are outlined in Table 1. Three papers describe different aspects of the annual Knowledge Translation Summer Institute held in Canada [36][37][38], Kho et al. [36] outline the development and curriculum of the 2008 program, and two other papers provide informal descriptions of single training sessions presented in 2009 [37,38].

Participants, duration and training modality
The identified capacity-building initiatives all included researchers as participants; however, a range of other professionals were also included. Gerrish and Piercy's [30] secondment program included clinical nurses, while Uneke et al. [33][34][35] included policy-makers, healthcare managers, directors of NGOs and other stakeholders alongside researchers in their initiatives. Participants in the Canadian Summer Institute, from which three papers emerged [36][37][38], were all researchers or graduate students interested in KT research. The duration of the   [32] program, which was embedded into existing training programs for nurse scientists over several years (pre-and post-doctoral coursework). The other programs [31,33,35,36] were brief (between 1 and 4 days) face-to-face courses or were delivered during a single session [37,38].

Aims of the training programs
All programs aimed to build the capacity of trainees in KT; however, how this was operationalised differed. The program reported by Gerrish and Piercy [30]  Training program activities/curricula (practical skillsbuilding, relationship-building and theoretical knowledge) All the training initiatives identified included skillsbuilding elements in their programs. The most common focus of skills development was KT intervention planning, included in Jones et al. [31], Kho et al. [36], Gerrish and Piercy [30], Uneke et al. [35], and Santacroce et al. [32].
Relationship building with policy-makers was another common theme, addressed in five of the nine programs [30][31][32][33][34]. For example, Jones et al. [31] explored "developing relationships for engagement and exchange with decision-makers" in lecture form. In the most recent program described by Uneke et al. [34], a society for "health policy research and knowledge translation" was established to facilitate ongoing collaboration between researchers and policy-makers. All training programs offered the opportunity for professional networking during the training event.
Two additional foci that emerged as capacity-building strategies in this review were providing information about the theoretical background of KT and specific training in KT research. Specific sessions on KT theory were delivered in three out of nine initiatives [31,35,36]. The Knowledge Translation Summer Institute described by Kho et al. [36] had a strong focus on developing researchers' skills in KT research methods and theory, with sessions such as 'Evaluating knowledge translation interventions' and 'Ethics of knowledge translation and knowledge translation research'. This 4day course also offered participants mentoring and practical small group exercises to assist their learning. In these sessions, practise-based learning exercises were used to assist students in thinking through the challenges of developing an end-of-grant KT plan [37] or a KT intervention for a clinical scenario [38] while having supervision, peer support and access to relevant resources. Table 2 outlines the evaluation approaches, outcomes and implications reported in each of the papers that met selection criteria. NHMRC grade of evidence is also reported alongside this synthesis of findings. The programs from the United Kingdom [30], Australia [31] and Nigeria [33][34][35] all reported varying degrees of formal evaluation. The paper from the United States reports planned evaluation measures [32]. The reports coming out of Canada [36][37][38] do not report evaluation of the programs but do discuss participant experiences.

Evaluation methods and measures
Gerrish and Piercy [30] reported a qualitative evaluation of their secondment KT capacity-building program in the United Kingdom, involving focus groups and semi-structured interviews with secondees, KT project managers, KT teams and healthcare managers to assess the success of the program from a variety of perspectives. Pre-post surveys were used in four of the programs [31,[33][34][35] to assess confidence in KT knowledge and skills gained. For the two-way secondment program described by Uneke et al. [34], the pre-post survey was implemented during a policy dialogue event held at the completion of the secondments; qualitative and cross-sectional data were collected to assess participant experiences of the secondment.
The papers included in this review present preliminary indications of acceptability and appropriateness of the capacity-building initiatives described but do not provide rigorous evidence to support their effectiveness. The overall quality of evidence included in this review is rated as 'very low quality' according to GRADE criteria [39], meaning that the evidence is very uncertain. Five of the papers under the NHMRC rating scale would individually be ranked as 'level IV' evidence (the lowest level included in this rating scale, where systematic reviews of randomised controlled trials are given a level 'I' rating). This is due to the low-quality quantitative data reported in these five studies. The remaining four papers are considered qualitative reflections or reports and thus are not assessable by the NHMRC scale.

Research outcomes
All of the training initiatives included in this review were described as beneficial, although, as noted above, the level of evidence was low. From the qualitative evaluation reported in Gerrish and Piercy [30], participants, hosting teams and KT managers described the secondments as a success due to the reportedly improved performance of trainees in KT facilitation and evaluation skills. This improvement was believed to be due to the practical skills they developed while embedded in the KT team as well as the mentorship they received from senior team members. In the secondment paper by Uneke et al. [34], researchers (qualitatively) reported having a deeper understanding of the need for researchers and policy-makers to work together and that the secondments offered the opportunity for personal and professional development. Jones et al. [31] and Uneke et al. [33,35] found that participant selfreported knowledge of key outcomes increased postworkshop. Santacroce et al. [32] did not report outcomes in their report.
Participant reports emerging from the Knowledge Translation Summer Institute [36] suggest that the authors enjoyed the different learning formats used at the Institute, the brief mentoring sessions they had with faculty members and the chance to network with other early career researchers. The reports by Leung et al. [37], Bhogal et al. [38] and Kho et al. [36] all contribute to demonstrating the importance of a supportive learning environment when participants are first learning about KT.

Discussion
The current study is, to the best of our knowledge, the first systematic review of published reports on capacitybuilding initiatives for KT and/or partnership research in the field of health research. While internationally there is growing interest in this area, we found the published literature on capacity-building programs for health researchers to be sparse, with only nine papers identified covering seven separate training initiatives. A variety of approaches were used for training in each of the programs; this heterogeneity combined with the minimal level of evaluation undertaken to date means it is not yet clear which training strategies in this area are likely to be the most effective.
The identified training initiatives all utilised a combination of different delivery styles to expose trainees to the multidisciplinary nature of KT, an approach that has previously been recommended in the literature [24].

Training in practical KT skills
All the included training programs covered some of the practical skills needed for KT, including skills in KT planning and evaluation, relationship-building, and communication and teamwork; these skills have all been identified in the literature as important for KT or partnership research [19,[21][22][23]. All programs were face-to-face allowing participants the opportunity to network with other KT trainees or team members. Another key feature of many of the programs was opportunities to engage in practice-based learning through secondments [30,34], working alongside mentors to assist with KT activities [32], or classroom-based practical activities [37,38]. Practical learning exercises as well as instruction on relevant concepts and theory have been identified as important components of KT training by researchers in Canada, as well as more broadly in the adult-education literature [40,41]. Brief mentoring sessions between participants and course faculty during the Knowledge Translation Summer Institutes appear to have been well received by participants [36], and may be an additional method of training researchers in KT that merits further inquiry [42,43].
Skills-based competencies related to collaboration and partnership, particularly with policy-makers, were also identified in the programs described by Jones et al. [31] and Uneke et al. [33,34]. This type of training aligns with skills described by Mallidou et al. [23] such as 'sharing knowledge', 'collaboration and teamwork' and 'knowledge brokering'. This may be an important theme for building researcher capacity for KT, as research suggests that these relationship-building skills are important in creating policy-focused research and in contributing to greater research impact [13,44,45].

Training in KT theory
All of the training initiatives that utilised a traditional course structure (workshop over one or more days) included content on the theoretical background of KT [31,33,35,36]. Such training has been highlighted as critical in Mallidou et al.'s [23] KT key competencies framework, in particular, understanding of context, research processes, evidence resources and dissemination. In addition to teaching researchers about the theory of KT, many of the programs included in this review also provided information about health policymaking processes to increase participants' knowledge of the contextual aspects of KT work. The inclusion of both theoretical aspects of KT and practical skillsbased aspects of KT in all training programs indicates a shared view that participants need to be exposed to a broad range of competencies to be able to effectively engage in KT. Course attendees (number not stated) Before and after evaluation (self-report) (not described in the report) No data presented Authors reported that all course components were rated highly for quality relevance and usefulness; increases in participants' self-reported understanding and confidence in KT theory, planning and communications were noted The course appears to be relevant and useful and may be able to build researcher skills and confidence in KT and exchange  Pre-post evaluation and a focus group were conducted with all participants; additionally, a survey was conducted with the senior researchers who participated in the workshop (n = 6) Pre-post self-report questionnaire rating knowledge of health policy-making processes, own capacity to use evidence and knowledge of evidence-informed policymaking The focus group focused on participants thoughts regarding 'bridging the gap between health policymakers and researchers' Pre-post survey data was reported as means, medians and ranges Increases in self-reported knowledge and understanding of the health policymaking process were observed in all questionnaire items when compared to participant pre-workshop scores; items that showed the greatest change between the pre-and postworkshop condition were items related to knowledge of terms such as 'policy brief' or 'types of evidence used for policy-making' Thematic analysis of focus group data indicated a need for researchers to be more aligned to health systems and policy challenges, and to consider policy-maker perspectives in their work; furthermore, participants suggested increased collaboration between researchers and policy-makers could facilitate researchers informing policy-makers of relevant research as it becomes available; suggested models of partnership were either involving policy-makers in the planning of the research or including researchers in the implementation of policy programs

Evaluation of the KT programs and courses
As training initiatives in KT or partnership research are still in their infancy, it is unsurprising that the publications we found reported exploratory evaluations of, or reflections on, single rounds of training initiatives rather than large scale evaluations. Participant experience, self-report surveys and qualitative evaluation methods were used to describe the successes and failures of most programs.
Broadly speaking, all studies reported that their approach to capacity-building in KT was well received and (where measured) that the program improved participant confidence in key skills related to KT. While there is not yet strong evidence on which to base decisions about which training strategies might be most effective in this field, the papers we located have provided useful insights into the types of approaches interested parties might take to developing skills-building initiatives in this area.
As with any review, we may not have captured every relevant paper, especially due to the diversity of terms used in the field of KT [46]. In addition to these limitations, the authors acknowledge that the training initiatives identified in published articles are likely to represent a very small sample of the body of work that is being carried out in this field. Nevertheless, to the best of our knowledge, this review represents the first systematic review of available peer-reviewed literature on the subject of KT/partnership research capacity-building programs for researchers.