- Open Access
How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field
Health Research Policy and Systems volume 18, Article number: 35 (2020)
Research funders in Canada and abroad have made substantial investments in supporting collaborative research approaches to generating and translating knowledge as it is believed to increase knowledge use. Canadian health research funders have advocated for the use of integrated knowledge translation (IKT) in health research, however, there is limited research around how IKT compares to other collaborative research approaches. Our objective was to better understand how IKT compares with engaged scholarship, Mode 2 research, co-production and participatory research by identifying the differences and similarities among them in order to provide conceptual clarity and reduce researcher and knowledge user confusion about these common approaches.
We employed a qualitative descriptive method using interview data to better understand experts’ perspectives and experiences on collaborative research approaches. Participants’ responses were analysed through thematic analysis to elicit core themes. The analysis was centred around the concept of IKT, as it is the most recent approach; IKT was then compared and contrasted with engaged scholarship, Mode 2 research, co-production and participatory research. As this was an iterative process, data triangulation and member-checking were conducted with participants to ensure accuracy of the emergent themes and analysis process.
Differences were noted in the orientation (i.e. original purpose), historical roots (i.e. disciplinary origin) and partnership/engagement (i.e. role of partners etc.). Similarities among the approaches included (1) true partnerships rather than simple engagement, (2) focus on essential components and processes rather than labels, (3) collaborative research orientations rather than research methods, (4) core values and principles, and (5) extensive time and financial investment. Core values and principles among the approaches included co-creation, reciprocity, trust, fostering relationships, respect, co-learning, active participation, and shared decision-making in the generation and application of knowledge. All approaches require extensive time and financial investment to develop and maintain true partnerships.
This qualitative study is the first to systematically synthesise experts’ perspectives and experiences in a comparison of collaborative research approaches. This work contributes to developing a shared understanding of collaborative research approaches to facilitate conceptual clarity in use, reporting, indexing and communication among researchers, trainees, knowledge users and stakeholders to advance IKT and implementation science.
Collaborative research with knowledge users is believed to be one of the best ways to support the rapid application of research evidence and generate greater impact on practice, policy, health systems and societal outcomes (i.e. effective knowledge translation (KT) and implementation practices) [1, 2]. Scholars use multiple terms, concepts, traditions, models and frameworks when describing collaborative research and practice [3, 4]. Table 1 provides a list of terms commonly employed in Canada and recognised by many funders, governments and research teams to facilitate shared understanding. Over the past two decades in Canada, health research funding agencies have promoted the funding of research partnerships as a means to promote greater uptake of research findings and thereby produce the desired research impact. This logic relates to bridging the ‘know-do’ gap  by focusing research on the needs of those who would use it in the real world (i.e. knowledge users) to increase knowledge use and impact. The logic also implies that, through working together on the research, researchers and knowledge users learn from each other, breaking down some of the barriers that typically divide the two communities [29, 30].
Research funders in Canada , Australia , the United Kingdom , the Netherlands  and the United States  have created funding opportunities to promote and support collaborative research approaches to generating and translating knowledge. Specifically, in Canada, integrated knowledge translation (IKT) has been widely promoted and used in health research . Kothari et al. define IKT as “a model of collaborative research, where researchers work with knowledge users who identify a problem and have the authority to implement the research recommendations” . Original conceptions of IKT date back to the late 1990s with work at the former Canadian Health Service Research Foundation (CHSRF) under the leadership of Jonathan Lomas, who developed the first Canadian IKT funding opportunities (then referred to as Knowledge Exchange and Linkage and Exchange funding programmes) that required health system decision-makers to be co-principle applicants on grants submitted to the CHSRF ; this was followed by the Canadian Institutes for Health Research (CIHR) referring to it as IKT in 2007 . The CHSRF and the CIHR recognised and promoted the novelty of IKT compared to other existing collaborative research approaches through the engagement of decision-makers/policy-makers in research, who were positioned to use the outcomes to influence change, thus increasing knowledge uptake (a mandate of the CIHR). IKT and other collaborative research approaches contribute to “the practice and science of implementation research as they provide opportunities to advance understandings of processes and factors that facilitate and hinder the development and sharing of knowledge in health systems” . However, there is a limited understanding and formal research of how IKT compares with other collaborative research approaches that have a focus on the co-creation of knowledge and its application in the real world.
To ensure the feasibility of completing this study in a timely manner, we conducted a review and preliminary synthesis of the literature  to identify approaches that were prevalent in the published (Medline, CINAHL, PsycINFO, Embase) and grey literatures (government reports/websites) under the following terms: (1) engaged scholarship, (2) Mode 2 research, (3) co-production and (4) participatory research. We tracked back temporally from the most recent approach (IKT, engaged scholarship, Mode 2 research, co-production, then participatory research). We selected these approaches as they are widely used globally, they are prevalent in the literature, and the research team collectively represents these comparative approaches. Definitions for these approaches are presented in Table 1. The objective of this study is to better understand how IKT compares with engaged scholarship, Mode 2 research, co-production and participatory research by identifying the differences and similarities among them. It is important to note that our goal was not about refuting or supporting IKT or any single approach but rather to bring conceptual clarity around the concepts and intents of the different approaches/traditions. Trainees, in particular, report being unclear about whether they are the same or different and, if different, in what ways. We hope to establish a shared understanding and promote a clearer description about these approaches in order to advance knowledge about the use, application, measurement and evaluation of the outcomes/impacts arising from these different varieties of ‘working together’. We recognise that there is also growing interest in ‘patient engagement’ (PE) and ‘patient and public involvement’ (PPI) in health services and, to a lesser extent, in research generation [38, 39]; however, these concepts were excluded. Our study focus was entirely on research partnerships, the last or penultimate level of engagement in most engagement frameworks, while PE and PPI typically include lesser levels of engagement such as communications/sharing, consultation/listening, deliberation, and involvement/engagement, which fall short of partnership/shared decision-making. Modes of knowledge production other than Mode 2 research were also not considered because of their less frequent use.
We used a qualitative descriptive method  with interview data to explore experts’ perspectives and experiences on the differences and similarities among IKT, engaged scholarship, Mode 2 research, co-production and participatory research. We used the COnsolidated criteria for REporting Qualitative studies (COREQ) 32-item checklist  to guide the reporting of this study, including the methods (data collection, data analysis) as well as the structure of results.
TN conducted the majority of the interviews by telephone and a few face-to-face at a convenient location for participants (home, café, workplace, office).
Our primary data source were participant interviews. Interviews were semi-structured and averaged 45 min in duration (Additional file 1). The aim of the interview was to gain novel insights from participants about perceived differences and similarities among IKT, engaged scholarship, Mode 2 research, co-production and participatory research. The topics for discussion across the approaches included theoretical underpinnings (i.e. purpose and motivation) and historical development (i.e. key developers). We asked participants to identify key literature (Additional file 2) related to the various approaches, which served as our secondary data source. TN emailed the interview guide to participants 2 weeks prior to conducting the telephone/face-to-face interviews. Participants were asked to complete the interview questions independently prior to their scheduled interview. TN validated participants’ responses during the interview and took detailed notes including any additional comments. Importantly, TN utilised her learning to iterate and explore concepts of the approaches with participants, which added to the richness of the dialogue in a way that has not previously occurred. TN and IDG continued to recruit additional participants until data saturation was achieved, defined as the point at which no new themes emerged among the interviews.
Research team and reflexivity
TN is a postdoctoral fellow with extensive experience in qualitative interviewing who conducted all of the interviews with interested participants. TN is supervised by IDG and has a keen interest in collaborative research, in particular IKT, as it is the focus of her postdoctoral work. Our multidisciplinary research team (n = 15) consisted of experienced experts (researchers (PhD) and clinicians (MD)) recognised internationally for their contributions to the field of KT and implementation science (see Table 2 for research team characteristics). The majority of our team members (n = 14) also contributed as study participants (n = 17) and authors (n = 14) on this paper and they may prefer their own perspective over others. However, the team collectively represented the five approaches under study, and by inviting members/experts in each realm, the research team was encouraged to take a critical stance when reviewing what was reported about each approach. We feel that, by creating a safe and collaborative exchange, experts were more likely to critically review the results and interpretations. The team views this commitment to the embedded process both as a key strength and a scientific necessity of the project and paper. In this way, we both studied and approached the work using a collaborative/IKT approach (e.g. we were ‘walking the talk’).
Team members were aware of each other’s work and some have collaborated in the past. From the qualitative viewpoint of this study, involving diverse team members in the co-creation of the research is an optimal strategy to capture the richness of knowledge, experience and practice context, and enriches rather than detracts from the research. A primary rationale for undertaking this study was that these various approaches are typically siloed in the literature and not discussed or considered in relation to each other; this study provided an opportunity for team members to consider, analyse and describe their own approaches and view other approaches in a systematic, comparative way that had previously not been undertaken or described. This mode of dialogue offered the team a rich opportunity to see their own practice in light of others’ approaches and led to fruitful and in-depth dialogue among members and was a key design feature of the approach. Many insights were subsequently gleaned during the course of interpreting the findings and drafting the paper. Given the qualitative nature of the study, the intent was not to structure our efforts to achieve objectivity but, rather, we intended to capture and explore the depth and diversity of the approaches among those experts in conceiving/using them. The best informants for each approach were, in fact, the experts who developed them, and it is for this reason that they were recruited to participate and share their knowledge and experiences.
Participant selection (n = 17)
TN and IDG originally identified and compiled a list of 13 participants derived from the published and grey literature; an additional 10 were recommended through networking and respondent-driven sampling . TN and IDG contacted participants through email (face-to-face where possible) with detailed information about the study. Contact information for participants was sought through public domains (primarily university websites, faculty directories and author contact information from key literature). Participation was voluntary and informed consent sought prior to study enrolment. Participants included renowned experts (researchers and clinicians) who have published extensively or who are recognised by their peers to have extensive expertise in the various approaches. Ultimately, a sample of 17 participants (see Table 3 for participant characteristics), primarily researchers, were enrolled in the study. We used an IKT approach to conduct the research and involved participants in every phase of the study to increase knowledge uptake. We made efforts throughout the research to create a cross-walk for participants to consider, discuss and debate the differences and similarities among approaches. All participants were offered the opportunity to interpret findings as a team and contribute as scholarly co-authors in disseminating the research.
Relationship with participants
TN informed all participants of her background, interests, personal goals and reasons for conducting this study to ensure transparency. TN also kept an audit trail to ensure her bias, assumptions, personal goals, reasons and interests in conducting this study were made transparent. TN worked collaboratively with all participants on every aspect of this study, thus interested participants turned into research team members as the study evolved.
Analysis and findings
We centred the analysis around the most recent approach, IKT, given its germinal relationship within Canadian health research funding agencies. TN and IDG conducted thematic analysis  of participant interviews to derive core themes from the data, which were validated by the team. An iterative process of comparing and contrasting the phrasing and meanings inherent within different perspectives was conducted. Member-checking was carried out with participants to ensure the accuracy of the emergent themes and the analysis process in an iterative manner. Our review and synthesis of the key literature within and across approaches provided additional insights and validation of the emergent themes that arose from analysis of participant interviews. No theory guided the analysis. The analysis was inductive with the following analysis categories (with related sub-categories) developed from the data by the research team: orientation (original purpose/intent, primary motivation, epistemological stance, theoretical underpinnings and theory implicit/explicit), historical roots (geographic origin, disciplinary origin, health research versus other research, and disciplinary background of early developers), and partnership/engagement (unique features, what partners are called, role of partners and power sharing). We created Table 4, which builds on the work of Bowen  as a comparative summary of approach characteristics. In an effort to establish face validity, we presented Table 4 at a conference workshop composed of international KT scholars .
Overall, both participants and international KT scholars agreed that there is a lack of clarity among the approaches, and that differences between concepts in development, intent and context should be identified. Findings were divided into two major themes with corresponding sub-themes – (1) differences among the approaches and (2) similarities among the approaches.
Differences among the approaches
Most participants emphasised the similarities among the approaches as the differences are minimal. However, participants identified subtle but important differences in the orientation and historical roots of the approaches as well as in partnership/engagement (see Table 4 for comparison of differences among approaches). Importantly, all participants acknowledged that the initial intent of these approaches has matured over time. Some participants suggested there may be greater variations in interpretations within, rather than between, approaches, which blurs the boundaries among the approaches. Here, we provide a brief description of the development of each approach below in order to establish context and shared understanding of the concepts. We focused on early developers of each approach as identified by participants and prevalent among the key literature.
IKT was first coined in 2007 by Graham , and subsequently advanced by the CIHR funding agency, as reflected by a participant,
“We use this term [IKT] because Ian [Graham] so effectively popularized it in Canada.” (P2)
However, the conceptual origins of IKT trace back to the late 1990s health policy-orientated writings of Lomas . At that time, Lomas was CEO of the former Canadian Health Services Research Foundation (CHSRF), an organisation established to “facilitate evidence-based decision making in Canada’s health sector” . His philosophy of ‘linkage and exchange’ [36, 59] set a promising course for changing the way in which research was applied to decision-making. The concept of ‘linkage and exchange’ was introduced as the Canadian government sought to systematically improve the use of sound scientific evidence by healthcare sector leaders and decision-makers/policy-makers  through applied research funding within health services. Through the CHSRF, conceptual awareness of linkage and exchange entered the policy realm among leaders/managers and decision-makers/policy-makers through the early 2000s. However, awareness and influence of the concept was somewhat limited, in that research addressing provider–patient or provider–client interactions lay outside the CHSRF funding mandate (the organisation did note a specific interest in building research capacity with nursing decision-makers/policy-makers) .
Graham and colleagues at the CIHR took up and advanced the linkage and exchange concept within the context of the uptake and use of research findings [21, 60, 61] through engagement of knowledge users (initially referred to as decision-makers) during the research process as the primary intent . By the mid-2000s the CIHR had four standing funding opportunities requiring inclusion of knowledge users as co-investigators. During this time, the CIHR also refined its merit review process for these funding opportunities, which required the Review Panels to be comprised of roughly equal numbers of researchers and knowledge users. Both researchers and knowledge users assessed each proposal for (1) scientific merit and (2) potential relevance and impact. The CIHR promoted IKT as an approach rather than a study design and indicated that a collaborative approach could work with any study methodology.
While participants identified IKT as close kin to all four approaches through similar underpinnings, in the literature we reviewed, IKT is either considered epistemologically neutral or linked with a social constructivist epistemology . This apparent contradiction may be explained by the focus of the relevant authors. When IKT is viewed as an approach to research as opposed to a study design, it can be considered epistemologically neutral – it can be used with any philosophy of science (meaning pragmatism, positivism, realism, etc.). When focus is on the nature of the researcher–knowledge user research partnerships, the underlying epistemological stance is social constructivism related to co-creation, mutual learning, etc., within the research partnership (as opposed to within the actual study). Building on and reformulating the linkage and exchange concept, IKT became an integral, underlying process inherent to the then-emergent Knowledge to Action Process [21, 60, 61].
IKT makes a unique contribution by adding the term ‘knowledge users’ , broadening inclusivity to multi-sectoral stakeholders. Formal and deliberate consideration of knowledge users and their needs forces the research team to proactively think through how change is made. This includes who (besides interested and affected parties) needs to act and who is critical to the planning of the research. IKT invites the potential and meaningful involvement of knowledge users throughout the research process, representing a significant departure from traditional research approaches. This reformulation extended well past the conceptual scope of linkage and exchange into the realm of research co-creation. Further, the conceptual evolution of IKT within the context of implementation has orientated the concept to focus on specific ‘ends’ (e.g. the improvement of health, provision of more effective health services and products, strengthening of the healthcare system, etc.).
Since the early 2010s, scholars have further explained the use of IKT and offered practical advice about how to ‘do’ IKT. IKT encourages researchers to integrate knowledge users into shaping the research and its execution/interpretation as well as the use of its products, in practice [64, 65]. With time, the articulation of IKT has also become more explicit with respect to outcomes/impact, return on investment, and benefits to society as a form of public/social accountability for the investment of public funds in health research [66,67,68]. For example, IKT explicitly emphasises the inclusion of decision-makers/policy-makers who are positioned to use the generated knowledge to impact change. Examples of participant responses included,
“For me IKT must include policy makers/managers who are in positions to use the results to change policies or make decisions. Effective IKT requires inclusion of policy makers and decision makers from the beginning to help shape the question(s) and to recognize they have a very important role to play in using results to make a difference. Participatory and community-based participatory research etc. do not state this explicitly. Ian Graham with IKT shifted my insight to realize the importance of getting policy/decision makers to be involved (small or big policy people) if you want to make an impact (consciously aware for IKT).” (P9).
“IKT involves decision makers in many stages of the research process (generation of research). Decision makers who are partners have some kind of power to help implement the findings (chosen specially because of their position/power not because they are disempowered).” (P13)
Interestingly, since the early 2010s, documented use of IKT has moved outside the health domain towards a conceptual guide for the conduct of research [69,70,71,72,73] in non-health research domains (e.g. engineering, environmental science, stem cell research, technology development, public transportation, among others).
The concept of engaged scholarship was articulated by the educator Ernest Boyer in the 1990s in the United States . Boyer drew attention to the growing disconnect between academics and societal needs through the promotion of public engagement in research [75, 76]. During this time, there was a major movement and urgent call for American universities to ‘return to their roots’, and to conduct research that meets societal needs rather than the needs of individuals (researchers, students, faculties within the university) or institutions [77,78,79,80,81]. The roots of engaged scholarship stem from a re-evaluation of what we think about scholarship (how do we engage people) and the social issues that we care about (how do we make academia more relevant to social issues). Thus, there are two levels of engaged scholarship – (1) transformation of academia and re-invigorating existing academic systems and structures and (2) encouraging researchers to reconnect research with social issues through engagement.
The initial intent of engaged scholarship was education for democracy, civic responsibility/engagement and public scholarship facilitated by university researchers [81,82,83,84]. Importantly, Boyer posits that discovery (‘pure’ or ‘basic’ research) is only one form of scholarship and offers integration, application and teaching as three other complementary forms [76, 85, 86]. A participant stated:
“Engaged scholarship is unique as it engages all forms of scholarship - not only ‘discovery’ research, which is how research is often defined in the academic setting. It also recognizes and urges valuing of integration of research, finding applications for research (as well as teaching).” (P10)
The embracing of all forms of scholarship (discovery, integration, application, teaching) is a unique contribution. This framing also cuts across the traditional academic evaluation criteria of teaching, research and service. Van de Ven later introduced the concept of ‘arbitrage’ in engaged scholarship as “a strategy of exploiting differences in the kinds of knowledge that scholars and practitioners can contribute on a problem of interest” . As a way of bridging the gap between theory and practice, Van de Ven sought to leverage the expertise of scholars and practitioners to enhance knowledge use and implementation practices [45, 87].
Mode 2 research
Gibbons et al. [88, 89] (with further developments by Nowotny et al. [51, 90]) proposed a new form of discovery they termed Mode 2 research in the 1990s in the United Kingdom/Europe. A Mode 2 approach to discovery promotes research that is reflexive, transdisciplinary, relevant, scientifically valid, issue-driven, context-specific and socially robust through the engagement of end-users (including industry to consider for-profit partnerships) in the research process [23, 44, 81]. The initial intent of Mode 2 research was to create knowledge based on the needs of end-users and increase researcher and social accountability. It stands in stark contrast to Mode 1 research, the traditional view of ‘objective science’ that canonises the autonomy of researchers and institutions (and is characterised in a system of academic tenure, peer-reviewed publications and criteria for validity, scientific expertise) and is frequently carried out in disciplinary silos . Mode 1 research is driven by a more purely discovery model . More recently, scholars have introduced a Mode 3 form of discovery, in which teams work simultaneously cross Modes 1 and 2, are adaptive to current problem contexts, co-evolve different knowledge and innovations, participate in civic engagements (beyond university–business–government) and link systems and system theory [91, 92]. However, we have restricted our work to Mode 2, which clearly underpins an IKT approach and more commonly appears in the health literature as being a foundation for partnered forms of research.
Co-production is a term first coined by the economist Elinor Ostrom in the late 1970s in the United States [93, 94]. The initial intent of co-production was to ensure that consumers or citizens were involved in the production as well as consumption of public services [95,96,97,98,99]. Explicit inclusion of patients (as consumers of health services, who can be considered ‘temporarily marginalised’) is a unique feature of co-production. Edgar Cahn, a civil rights law professor, further developed co-production to incorporate principles of social justice and equity . Key contributors to the development of co-production include Jasanoff [101, 102] (science and technology) and Coot  (health services). Co-production combines academic insight and scientific excellence with public benefit, blurring the boundaries between ‘pure’ and ‘applied’ research, and aligning closely and building upon participatory research .
Participatory research predates all other approaches we are considering; it has the largest body of literature and longest lineage among the approaches. Participatory research has its origins in social action research and an emancipatory philosophy. Historically, there are two traditions of participatory research. First, there is the Northern Tradition [104, 105] or ‘action research’ developed by Lewin in the 1940s in the United Kingdom and United States. The aim of the Northern Tradition was to promote cyclical practices of involving community members and workers in collaborative efforts towards organisational and social change, rather than on, for or about them. Second, there is the Southern Tradition  or ‘emancipatory research’, inspired by Freire in the 1970s in South America. The aim of the Southern Tradition was to promote practices and strategies to engage oppressed, marginalised, disadvantaged or disempowered communities within a dominant society, primarily Latin America, Africa and Asia, to affect policy change. The initial intent of the Southern arms of participatory research was on social justice (generating knowledge that can be utilised for social change), power and emancipation (this also includes knowledge utilisation and self-determination) for interested or affected patients, individuals or communities. In the United States, this emancipatory approach has been most frequently adopted by community-based participatory research, or in Canada, by those involved in community-based research. Community-based participatory research has been equally invested in achieving research implementation and impact (as seen through National Institutes of Health, Centres for Disease and Control and Prevention, and Patient-Centered Outcomes Research Institute funding). Recent writers from Indigenous perspectives  and from the global south [108, 109] have called for knowledge democracy and cognitive justice that honours community expertise as equal to academic expertise in co-generating research knowledge, and that recognises the authority of communities to direct the research design and methodologies. Capacity-building is an outcome of all approaches but is intentional in participatory research.
Well-established in public health and community-based health initiatives, participatory research is also an emerging approach in other disciplines beyond health and extends to policy and health services research . The three key motivations of participatory research include social and environmental justice and a desire for impactful change (particularly to be accountable towards communities and to benefit underserved/vulnerable citizens and communities), translating knowledge into action (knowledge utilisation towards equity and social justice), and self-determination . Participants also pointed out that the humility of the researchers, including cultural humility, is an integral aspect of the participatory research process. This involves reflection on one’s positionality of power and privilege so that issues of power sharing can be directly addressed in partnering processes . Reflection about one’s power and privilege was not made explicit in other approaches, although the emphasis on equal but different roles found in engaged scholarship promotes a form of humility. It is important to recognise that there are many contributors who have informed the development of participatory research that are not discussed here (Allen , Argyris , Arnstein , Borda , Cargo , Dewey [115,116,117], Duran , Green , Habermas , Hall [121, 122], Israel [123,124,125], Jagosh [126, 127], Kemmis , Kuhn [129, 130], LeMaster , Macaulay [131, 132], Mercer , Minkler [134,135,136,137,138,139], Ramsden , Salsberg [141,142,143], Schon , Selener , McTaggart , Viswanathan , Wallerstein [146,147,148,149], Wright , Westfall , Foote Whyte ).
Similarities among the approaches
Analysis of participant responses and the key literature revealed several similarities among the approaches, for example, (1) true partnerships rather than simple engagement; (2) a focus on essential components and process rather than labels; (3) collaborative research orientations rather than research methods; (4) core values and principles, and (5) extensive time and financial investment. These are discussed below.
True partnerships rather than simple engagement
Most participants expressed the perspective that all approaches shared a commonality in facilitating true partnerships, meaning a deep relationship between researchers and knowledge users during the entire research process to ensure the research is beneficial for all parties involved. The knowledge and contributions of researchers and knowledge users are different but equally valued. This understanding of bi- or multi-directional learning has been strengthened by Indigenous theorists, who do not use the term knowledge users as much as co-knowledge creators and producers. In this perspective, knowledge users are not passive subjects but active participants and experts in their own right (lived experience), while researchers are experts in research design, coordinating research activities and securing funding.
“In a true partnership (and this reflects an engaged scholarship perspective), all perspectives/expertise are considered equally valuable – but different. We do not expect knowledge users to be researchers or bring research expertise, for example. All partners are involved from the earliest stage of the research (e.g. determining priority topics and research questions) in meaningful ways. Importantly, power is shared. Partnership involves creating an environment where diverse perspectives, including differences, are discussed in depth, resulting in a higher level of understanding.” (P10)
This quote highlights a power shift towards shared decision-making, equality and co-learning, recognising the expertise and unique contributions of all those involved in the co-creation of knowledge. Some participants noted that not all partnerships are authentic. Often, knowledge user engagement relates solely to consultation or feedback at one point in time where knowledge users do not have an active role in the decision-making process.
“Engagement refers more to consulting, whereas partnership implies a stronger and tighter relationship.” (P13)
However, another participant stated that engagement occurs along a spectrum or continuum ranging from passive to active participation as the term ‘engagement’ has many different meanings for different people.
Focus on essential components and process rather than labels
Aligning with the previous theme of true partnerships, another theme that emerged was the focus on essential components (core values and principles) and process (team interactions and dynamics) for working together in a meaningful way rather than labelling an approach. A participant summarised the essence of all the approaches as:
“It’s all about working together to solve problems that matter to people and making a difference. It’s about making sure the benefits of the research are applied and the outcomes realized to make things better.” (P16)
Another participant put it this way,
“[It’s] not so much what we call it but it’s what we do.” (P8)
While participants recognised the need for labelling an approach (particularly among trainees) in order to provide guidance for methods, scientific credibility and rigor, many participants voiced their lack of enthusiasm for labels. Participants expressed concern with the focus on labels with the changing terminology:
“I have found, over the years, that there are all sorts of new methods or approach that purport to be different, but often are marginally so. They may be coined as something different in order for someone to be seen to create something new, or to be able to sell an approach. Often it is repackaging with a new name. People want to label things and then they promote it.” (P4)
Many participants found it difficult to align themselves with a single approach. Thus, they focused on the collaborative process of working with knowledge users in meaningful partnership, using whatever approach (or combination of elements from various approaches) they deemed suitable to achieve the objectives of a particular study [152, 153].
“My overall comment is that the similarities outweigh the differences. In my opinion the KEY item is the level of true partnership throughout all phases of the research” and “[I’m] not excited by all this academic terminology, [I] feel strongly that the key issue is equity of team members; it may vary from project to project. How the team decides to work … it depends on all your other factors: research question; the funding source that drives a lot of this; expertise of the team; who you’ve got on the team; it depends on length of the projects; who owns the research and who can access it; every team is different, it is not a cookie cutter approach.” (P9)
Collaborative research approaches rather than research methods
All participants described all of the approaches as collaborative research orientations (plans and procedures for research that consist of philosophical underpinnings, broad assumptions/worldviews and integrity), rather than research methods (the forms of data collection, analysis and interpretation) or research methodologies (theoretical study or analysis of the methods) [154, 155]. All of the approaches challenge the traditional academic practice of purely researcher-driven research in favour of engaging knowledge users in the research process to improve research relevance, implementation practices and impact. A participant stated:
“I use the same theory/principles – i.e. the research end users and researchers come together to develop the question, develop the protocol and complete the study, interpret results, disseminate strategies. All team members are equal partners.” (P12)
Participants emphasised the importance of recognising distinctions between research orientations, methods and methodologies as these terms are often used interchangeably, which can cause confusion.
Core values and principles
Core values and principles common among the approaches include co-creation, reciprocity, trust, fostering relationships, collaboration, respect, co-learning, active participation, democratisation of knowledge and shared decision-making in the generation and application of knowledge.
“Inclusion of knowledge users is one of the core principles or values. People want to be heard and a healthcare system that is responsive to their needs and not the needs of researchers.” (P16)
“Involving patients and communities; principles are the same involving patients in shared decisions making for clinical care and for research.” (P9)
“The goal of answering questions of mutual concern, engagement with the ideas and the work, and motivation to collectively enable change are closely linked to the co-creation/co-production model. This work model may not be ideal for all projects, but in health care it is ideal for most.” (P3)
Extensive time and financial investment
All participants agreed that extensive time and financial investment must be recognised to establish and sustain true partnerships as it is often overlooked by funders.
“Research that is co-created takes time, patience, energy and commitment. The engagement of individuals/patients, organizations and/or communities has become increasingly important in all aspects of the research process. Research that is co-created with individuals/patients, organizations and/or communities is designed to improve health and well-being and to minimize health disparities” (P15)
“It took a long time for me to be ready to undertake such work [collaborative work] due to time and resources.” (P11)
Participants felt that extensive time, effort, commitment and resources are required in building rapport and trust with knowledge users prior to, during and beyond the completion of the study. Many participants also discussed the challenges of securing funding because of limited funding opportunities and limited rewards for undertaking collaborative work, particularly in the early stages of a researcher’s career.
In summary, most participants emphasised the similarities among the approaches, as the differences seemed minimal. Some participants also recognised that, while the approaches may share common principles, the way they are enacted is contextual, and is related to the epistemological stance of those involved and the particular project. Overall, the key contribution of this paper is to advance the practice and science of IKT by making concrete and explicit its differences and similarities compared to other collaborative research approaches. Identifying the similarities, with differences in origin and emphasis, among collaborative research approaches is novel and has added value in informing future research and practice as this is the first study to bring experts together to reflect on these approaches. Future research should build on this work and systematically review the advantages as well as the disadvantages of each of these approaches.
There are four key distinguishing features of IKT worth noting. First, IKT is the only approach originally developed in a health research context (Medicine and Nursing) by health research funders in Canada. Second, IKT is either considered epistemologically neutral or linked with a social constructivist epistemology  in the literature we reviewed. Third, IKT has typically not been about emancipation of knowledge users but rather focused on increasing knowledge use, implementation and impact. Fourth, IKT makes explicit the engagement of knowledge users throughout the research process, which forces the research team to proactively think through how change is made, and therefore who needs to be involved in the planning of the research to enact change. Further, the conceptual evolution of IKT within the context of implementation has orientated the concept to specific ‘ends’ (e.g. the improvement of health, provision of more effective health services and products, strengthening of the healthcare system, etc.).
Similarities among the approaches included true partnerships rather than simple engagement, a focus on essential components and process rather than labels, collaborative research orientations rather than fixed research methods, and core values and principles. Core values and principles common among the approaches include co-creation, reciprocity, trust, fostering relationships, collaboration, respect, co-learning, active participation, democratisation of knowledge, and shared decision-making in the generation and application of knowledge. All approaches were considered by participants to require extensive time and financial investment to develop and maintain true partnerships.
There was consensus in some areas (e.g. IKT is the only approach originally developed in a health context by health research funders in Canada) and negotiated agreement in others among the participants (e.g. IKT is either considered epistemologically neutral or linked with a social constructivist epistemology). Many participants found it challenging to align themselves with a single approach as they used a combination of principles from multiple approaches (embracing the diversity among the approaches and taking the best from whichever approach or approaches and make it work with the funding available). Instead of focusing on labelling an approach, participants emphasised the need to focus on the collaborative process (i.e. objectives, context, roles, who to engage, when to engage, power sharing, and how to initiate and continue the engagement) [152, 153]. Participants agreed that there is no recipe for collaborative research as it is a dynamic process and we need to learn through hands-on experience. Participants stated that engagement occurred along a continuum of participation  and researchers should provide descriptions of teams and shared decision-making processes when reporting collaborative research projects (i.e. project governance and decision-making process, team functioning, roles, contributions). We encourage researchers and trainees to use the key dimensions proposed in Table 4 as a guide for reporting.
Participants raised other important questions, including “What constitutes true partnership or minimum engagement?”, “What is participation?”, “Is true partnership necessary for all approaches?”, “What do ‘core’ or ‘valuable’ principles mean in relation to the marginal differences which could be important to acknowledge? Do we ask users what is valuable? Does it depend on the type of question?” Addressing these questions will begin to provide added clarity around the differences among the approaches. Participants expressed that, in order to promote the development of true partnerships in research, additional effort and revisions to current university standards for productivity and academic reward is needed to acknowledge the extensive time, effort and resource investment required for establishing true partnerships .
Potential use of findings for researchers, trainees and other knowledge users/stakeholders
This qualitative descriptive study is the first to systematically synthesise experts’ perspectives and experiences in a comparison of various approaches to collaborative research, using an IKT lens. It is also the first to bring experts in the field together to synthesise the similarities and differences among the approaches. For researchers and trainees, this work offers novel insights and foundational knowledge as a starting point for an inter-disciplinary discussion/dialogue about the nature, characteristics, measurement and evaluation of various collaborative research approaches in a more refined way. For example, the results of this study provide novice researchers and trainees with a common working document to help familiarise themselves with the terms and concepts used in collaborative research to inform future research that will advance IKT science and practice.
Knowledge users and stakeholders may find the explicit description of principles and values of collaborative research helpful in making decisions around how and when they would like to be engaged to facilitate meaningful engagement. The themes identified in this study may provide a foundation and entry point to assist knowledge users and stakeholders in initiating discussions about what constitutes engagement, collaborative research or co-creation with researchers prior to agreeing to partner with them. For funding agencies and decision-makers/policy-makers, the description of the approaches and the similarities and differences may be of use when designing partnered funding calls.
It is important to note that these results are primarily based on the views of the participants and a review of the key literature. The process of synthesising the differences and similarities between IKT and other collaborative research approaches is complex and multi-layered. It was challenging to integrate all of the participants’ views and feedback to establish negotiated agreement. Readers should be mindful when interpreting the results of this study and resist oversimplification and generalisation, as exemplified by the level of detail provided in Table 4. We acknowledge that the participants are predominately Canadian researchers, but this is not surprising given our team’s focus on IKT as well as the broader Canadian funder-linked orientation to IKT. We migrated this limitation by conducting a face validity check with the international community of KT scholars who identified similar concerns as our participants regarding a lack of clarity among the approaches. We also attempted to recruit international experts, yet many were unable to enrol in the study. Those who declined participation stated scheduling conflicts and other previous commitments as the hindrance. Furthermore, for several of the historically mature approaches, source experts were deceased.
Additionally, this paper is focused on researchers’ perspectives and we would have likely gained different insights had we included community members and other knowledge users with experiences with the different approaches. Thus, we recognise that our results and discussion may be skewed in addressing only the needs of researchers and not those of partners. We also acknowledge that the selection of approaches is not inclusive of all collaborative research (i.e. PE, PPI and other modes of knowledge production beyond Mode 2). We selected approaches most prevalent in the literature as a starting point and to keep the study manageable. Nevertheless, we suggest that the comparison of IKT with the included four other dominant approaches remains a useful starting point for researchers, trainees, and other knowledge users and stakeholders. Lastly, we did not take a systematic approach to the literature review because the focus was on experts and their identification of relevant literature.
Directions for future research
Primary data collection from community members, knowledge users and stakeholders who engage in collaborative research should also be undertaken to understand the issues and challenges from their perspectives, which may differ from those of theorists in the field. There is a need for additional comparative effectiveness research to examine outcomes and impact across [158,159,160] these approaches to reveal whether and how outcomes differ by approach. There is also a need for research into how current university standards for productivity and promotion influences researchers’ intent in undertaking collaborative research and how institutional disincentives to engage in collaborative research can be overcome .
Overall, the results of this study suggest that IKT, engaged scholarship, Mode 2 research, co-production and participatory research approaches are more similar than different to each other. All facilitate true partnerships (shifting to shared decision-making that values contributions from all partners equally), require extensive time and financial investment, and share core values and principles (inclusion of knowledge users throughout the research process, co-creation of knowledge, reciprocity, shared decision-making, active participation and democratisation of knowledge production). Important differences were noted in the orientation and historical roots as well as in partnership/engagement among the approaches. This study is the first to bring experts together to share insights/experiences and reflect on what they think about multiple collaborative research approaches. This work contributes to developing a shared understanding of collaborative approaches to facilitate clarity in use, reporting, indexing and communication among researchers, trainees, knowledge users and stakeholders to advance IKT and implementation science as well as knowledge around collaborative research.
Availability of data and materials
Not applicable to protect participants’ anonymity/confidentiality.
Canadian Health Services Research Foundation
Canadian Institutes of Health Research
Integrated knowledge translation
Patient and public involvement
Gagliardi AR, Berta W, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (IKT) in health care: a scoping review. Implement Sci. 2015;71(2):105–6. https://doi.org/10.1186/s13012-016-0399-1.
Jull J, Giles A, Graham ID. Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge. Implement Sci. 2017;12:150. https://doi.org/10.1186/s13012-017-0696-3.
Straus SE, Tetroe J, Graham ID. Knowledge translation in healthcare: moving from evidence to practice. 2nd ed. West Sussex: Wiley; 2013.
Drahota A, Meza RD, Brikho B, et al. Community-academic partnerships: a systematic review of the state of the literature and recommendations for future research. Milbank Q. 2016;94(1):163–214. https://doi.org/10.1111/1468-0009.12184.
Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q. 2016;94(2):392–429.
Voorberg WH, Bekkers VJ, Tummers LG. A systematic review of co-creation and co-production: Embarking on the social innovation journey. Public Manag Rev. 2015;17(9):1333–57.
Vargo SL, Lusch RF. Evolving to a new dominant logic for marketing. J Mark. 2004;68(1):1–17.
Ostrom E. Crossing the great divide: coproduction, synergy, and development. World Dev. 1996;24(6):1073–87.
Boyle D, Harris M. The challenge of co-production–How equal partnership between professionals and the public are crucial to improving public services. London: NESTA; 2009.
Canadian Foundation for Healthcare Improvement (formerly the Canadian Health Services Research Foundation). Glossary of knowledge exchange terms. 2018. https://www.cfhifcass.ca/PublicationsAndResources/ResourcesAndTools/GlossaryKnowledgeExchange.aspx. Accessed 24 May 2018.
Canadian Institutes of Health Research (CIHR). Glossary of funding-related terms. 2019. http://www.cihr-irsc.gc.ca/e/34190.html#d. Accessed 9 Feb 2019.
Graham ID, Tetroe J, Pearson A. Turning knowledge into action: practical guidance on how to do integrated knowledge translation research. 1st ed. Philadelphia: Lippincott Williams & Wilkins; 2014.
Boyer EL. The scholarship of engagement. J Public Serv Outreach. 1996;1(1):11–21.
Van de Ven AH, Johnson P. Knowledge for theory and practice. Acad Manag Rev. 2006;31(4):802–21.
Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1:1. https://doi.org/10.1186/1748-5908-1-1.
Graham ID, Tetroe J. CIHR research: how to translate health research knowledge into effective healthcare action. Healthc Q. 2007;10(3):20–2.
Graham ID, Tetroe J, MacLean RK. Chapter 1: some basics of integrated knowledge translation research. In: Graham ID, Tetroe J, Pearson A, editors. Turning knowledge into action: practical guidance on how to do integrated knowledge translation research. Philadelphia: Lippincott Williams & Wilkins; 2014. p. 196.
Kothari A, McCutcheon C, Graham ID, for the IKT Research Network. Defining integrated knowledge translation and moving forward: a response to recent commentaries. Int J Health Policy Manag. 2017;6:1–2.
Canadian Institutes of Health Research (CIHR). Knowledge translation at CIHR. 2016. http://www.cihr-irsc.gc.ca/e/29418.html#2. Accessed 9 Feb 2019.
Graham ID, Tetroe J. Getting Evidence into Policy and Practice: Perspective of a Health Research Funder. J Can Acad Child Adolesc Psychiatry. 2009;18(1):46–50.
Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N. Lost in knowledge translation: time for a map? J Contin Educ Heal Prof. 2006;26:13–24.
Estabrooks CA, Floyd JA, Scott-Findlay S, O’leary KA, Gushta M. Individual determinants of research utilization: a systematic review. J Adv Nurs. 2003;43(5):506–20.
Gibbons M, Limoges C, Nowotny H, Schwartzman S, Scott P, Trow M. The new production of knowledge: the dynamics of science and research in contemporary societies. London: SAGE Publications; 1994.
Estabrooks CA, Norton P, Birdsell JM, Newton MS, Adewale AJ, Thornley R. Knowledge translation and research careers: Mode I and Mode II activity among health researchers. Res Policy. 2008;37(6–7):1066–78.
Green LW, George A, Daniel M, Frankish J, Herbert CJ, et al. Participatory research in health promotion. Ottawa: The Royal Society of Canada; 1995.
Cargo M, Mercer SL. The value and challenges of participatory research: strengthening its practice. Annu Rev Public Health. 2008;29:325–50.
Mrklas KJ. A scoping review of available tools for assessing integrated knowledge translation research or health research partnership impact. Calgary: Department of Community Health Sciences, Cumming School of Medicine, University of Calgary; 2017.
Bucknall T. Bridging the know-do gap in health care through integrated knowledge translation. Worldviews Evid-Based Nurs. 2012;9(4):193–4.
Wehrens R. Beyond two communities -from research utilization and knowledge translation to co-production? Public Health. 2014;128(6):545–51.
Green LW, Mercer SL. Can public health researchers and agencies reconcile the push from funding bodies and the pull from communities? Am J Public Health. 2001;91:1926–9.
Canadian Institutes of Health Research (CIHR). A guide to knowledge translation at CIHR: integrated and end of grant approaches. 2015. http://www.cihr-irsc.gc.ca/e/45321.html. Accessed 9 Feb 2018.
Fisk NM, Wesselingh SL, Beilby JJ, Glasgow NJ, Puddey IB, Robinson BG, Angus JA, Smith PJ. Academic health science centres in Australia: let’s get competitive. Med J Aust. 2011;194(2):59–60.
National Institute for Health Research. About the Collaborations for Leadership in Applied Health Research and Care (CLAHRCs); 2019. https://clahrcprojects.co.uk/about. Accessed 9 Feb 2019.
Jansen MW, van Oers HA, Middelweerd MD, van de Goor IA, Ruwaard D. Conditions for sustainability of Academic Collaborative Centres for Public Health in the Netherlands: a mixed methods design. Health Res Policy Sys. 2015;13:36. https://doi.org/10.1186/s12961-015-0026-7.
Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, Konopka K, Daugherty S. Conceptual and practical foundations of patient engagement in research at the Patient-Centered Outcomes Research Institute. Qual Life Res. 2015;24:1033–41.
Lomas J. Using linkage and exchange to move research into policy at a Canadian Foundation. Health Aff. 2000;19:236–40.
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.
Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, Brito JP, Boehmer K, Hasan R, Firwana B, Erwin P. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89. https://doi.org/10.1186/1472-6963-14-89.
Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, Brito JP, Boehmer K, Hasan R, Firwana B, Erwin PJ. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66.
Patton M. Qualitative research and evaluation methods. 3rd ed. Thousand Oaks: SAGE Publications; 2002.
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.
Salganik MJ, Heckathorn DD. Sampling and estimation in hidden populations using respondent-driven sampling. Sociol Methodol. 2004;34(1):193–240.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.
Bowen S. The relationship between engaged scholarship, knowledge translation and participatory research. In: Higginbottom G, Liamputtong P, editors. Participatory qualitative research methodologies in health. Los Angeles: Sage Publications; 2015. p. 183–99.
Van de Ven AH. Engaged scholarship: a guide for organizational and social research. Oxford: Oxford University Press; 2007.
Voorberg WH, Bekkers VJ, Tummers LG. A systematic review of co-creation and co-production: embarking on the social innovation journey. Public Manag Rev. 2015;17(9):1333–57.
Piggot-Irvine E, Zornes D. Developing a framework for research evaluation in complex contexts such as action research. SAGE Open. 2016;6(3):1–15.
Chevalier JM, Buckles DJ. Chapter 1: action research history, in participatory action research: theory and methods for engaged inquiry. New York: Routledge; 2013. p. 459.
Thomas A, Menon A, Boruff J, Rodriguez AM, Ahmed S. Applications of social constructivist learning theories in knowledge translation for healthcare professionals: a scoping review. Implement Sci. 2014;9(1):54.
Bechara JP, Van de Ven AH. Chapter 2: philosophy of science underlying engaged scholarship. In: Van de Ven AH, editor. Engaged scholarship: a guide for organizational and social research. Minneapolis: Oxford University Press; 2007.
Nowotny H, Scott P, Gibbons M. Mode 2 revisited: the new production of knowledge. Minerva. 2003;41:179–94.
Kuutti K. Design research, disciplines, and new production of knowledge. Hong Kong: International Association of Socieities of Design Research (IASDR07); 2007. p. 1–13.
Kincheloe JL. Describing the bricolage: conceptualizing a new rigor in qualitative research. Qual Inq. 2001;7(6):679–92.
Latour B. We have never been modern. Cambridge: Harvard University Press; 1993.
Coghlan D. Retrieving a philosophy of practical knowing for action research. Int J Action Res. 2016;12(1):84–107.
Adelman C. Kurt Lewin and the origins of action research. Educ Action Res. 1993;1(1):7–24.
Rimmer TC, Johnson LL. Planned change theories for nursing. Thousand Oaks: Sage; 1998.
Nguyen T, Graham ID. What’s in a name? Exploring the differences and similarities among knowledge translation approaches. Knowledge Utilization Colloquium November 8-10, 2017 (Oral Presentation). Melbourne, Australia.
Lomas J. Improving research dissemination and uptake in the health sector: beyond the sound of one hand clapping. Hamilton: McMaster Univeristy Centre for Health Economics and Policy Analysis; 1997. p. 1–45.
Graham ID, Tetroe J, The KT Theories Research Group. Some theoretical underpinnings of knowledge translation. Acad Emerg Med. 2007;14:936–41.
Graham ID. Knowledge translation at CIHR: under new administration. Ottawa: Canadian Institutes of Health Research; 2007.
Graham ID, Bernstein A. Health research: an excuse not to act as a catalyst for change. Hill Times. 2007;30:1.
Tetroe J. Knowledge translation at the Canadian Institutes of Health Research: a primer. Focus Tech Brief. 2007;18:1–8.
Salsberg J, Macaulay AC, Parry D. Chapter 2: guide to integrated knowledge translation research: researcher and knowledge-user collaboration in health research. In: Graham ID, Tetroe JM, Pearson A, editors. Turning knowledge into action: practical guidance on how to do integrated knowledge translation research. Philadelphia: Lippincott Williams & Wilkins; 2014.
Kerner J. Knowledge translation in cancer surveillance: what do we know, how should we share it, and who cares? Toronto: Canadian Partnership Against Cancer; 2009.
Graham ID, Kothari A, McCutcheon C, Angus D, Lukaris-Banner D, Botting I, Bucknall T, Dunn S, Gagnon M, Gifford W, Godfrey C, Holmes B, Horsley T, Hutchinson A, Jull J, Law S, McLean R, Mrklas K, Nguyen T, Plamodon K, Rycroft-Malone J, Sibald S, Stacey D, Wright D, Yeung E, for the Integrated Translation Research Network. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation. Implement Sci. 2018;13:22. https://doi.org/10.1186/s13012-017-0700-y.
Kothari A, Wathen NC. A critical second look at integrated knowledge translation. Health Policy. 2013;109(2):187–91.
Gagliardi AR, Kothari A, Graham ID. Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know. J Epidemiol Community Health. 2016;71(2):105–6.
Flagg JL, Lane JP, Lockett MM. Need to Knowledge (NtK) Model: an evidence based framework for generating technological innovations wtih socio-economic impacts. Implement Sci. 2013;8:21.
Lane JP, Flagg JL. Traslating three states of knowledge-discovery, invention and innovation. Implement Sci. 2010;5:9.
Holmes C, McDonald F, Jones M, Graham J. Knowledge translation: moving proteomics science to innovation in society. OMICS. 2016;20(6):352–61.
Barber T, Sharif B, Teare S, Miller J, Shewchuk B, Green LA, Marlett N, Cibere J, Mrklas K, Wasylak T, Li LC, Campbell-Scherer D, Marshall DA. A qualitative study to elicit patients’ and primary care physicians’ perspectives on the use of a self-management mobile health application for knee osteoarthritis. BMJ Open. 2019;9(1):e024016.
The Service and Market Oriented Transport Research Group (SAMOT). Developing public transport through multidisciplinary and benefit-oriented research – Final Report VINN Excellence Center Stage 1. https://www5.kau.se/sites/default/files/Dokument/subpage/2011/08/samot_stage_1_pdf_10169.pdf. Accessed 6 Feb 2019.
Boyer EL. Scholarship reconsidered: the priorities of the professoriate. special report. Princeton: Carnegie Foundation for the Advancement of Teaching; 1990.
Bowen S, Graham ID. From knowledge translation to ‘engaged scholarship:’ promoting research relevance and utilization. Arch Phys Med Rehabil. 2013;94(Suppl 1):S3–8.
Campbell HJ, Vanderhoven D. Knowledge that matters: realising the potential of co-production. 2016. http://eprints.whiterose.ac.uk. Accessed 9 Feb 2018.
Kellogg Commission on the Future of State and Land-Grant Universities. Returning to our Roots: Executive summaries of the reports of the Kellogg Commission on the future of state and land grant universities. 2001. http://www.aplu.org. Accessed 9 Feb 2018.
Anderson JA. Engaged learning, engaged scholarship: a struggle for the soul of higher education. Northwest J Comm. 2014;42(1):143–66.
Harkavy I. The role of universities in advancing citizenship and social justice in the 21st century. Educ Citizenship Soc Justice. 2006;1(1):5–37.
Hartley M, Saltmarsh J, Clayton P. Is the civic engagement movement changing higher education? Br J Educ Stud. 2010;58(4):391–406.
Bowen S. A school of epidemiology and public health for the 21st Century: the role of engaged scholarship. Ottawa: Department of Epidemiology and Community Medicine University of Ottawa; 2014. p. 1–38.
Cox D. History of the scholarship of engagement movement. In: Fitzgerald HE, Burack C, Siefer S, editors. Handbook on engaged scholarship: contemporary landscapes, future directions. East Lansing: Michigan State University Press; 2010.
Glass CR, Fitzgerald HE. Engaged scholarship: historical roots, contemporary challenges. In: Fitzgerald HE, Burack C, Siefer S, editors. Handbook on engaged scholarship: contemporary landscapes, future directions. East Lansing: Michigan State University Press; 2010.
Stanton TE. New times demand new scholarship. Educ Citizenship Soc Justice. 2008;3:19.
Maxwell N. How universities can help to create a wiser world: the urgent need for an academic revolution. Exeter: Imprint Academic; 2014.
MacIntyre A. Three rival versions of moral enquiry: encyclopedia, genealogy, and tradition. Notre Dame: University of Notre Dame Press; 1990.
Van de Ven AH, Zlotkowski E. Toward a scholarship of engagement: a dialogue between Andy Van de Ven and Edward Zlotkowski. Acad Manag Learn Educ. 2005;4:355–62.
Gibbons M. Science’s new social contract with society. Nature. 1999;402:C81–4.
Hessels L, van Lente H. Re-thinking new knowledge production: a literature review and research agenda. Res Policy. 2008;37:740–60.
Nowotny H, Peter S, Michael G, Scott P. Re-thinking science: knowledge and the public in an age of uncertainty. Cambridge: Polity; 2001.
Carayannis EG, Campbell DFJ, Rehman SS. Mode 3 knowledge production: systems and systems theory, clusters and networks. J Innov Entrep. 2016;5(1):17.
Schoonmaker MG, Carayannis EG. Mode 3: a proposed classification scheme for the knowledge economy and society. J Knowl Econ. 2013;4(4):556–77.
Ostrom E, Baugh WH. Community organization and the provision of police services. Beverly Hills: Sage Publications; 1973.
Ostrom E, Parks RB, Whitaker GP, Percy SL. The public service production process: a framework for analyzing police services. Policy Stud J. 1978;7(Suppl 1):381–9.
Parks RB, Baker PC, Kiser L, Oakerson R, Ostrom E, Ostrom V, Percy SL, Vandivort MB, Whitaker GP, Wilson R. Consumers as coproducers of public services: some economic and institutional considerations. Policy Stud J. 1981;9(7):1001–11.
Powell K, Kitson A, Hoon E, Newbury J, Wilson A, Beilby J. A study protocol for applying the co-creating knowledge translation framework to a population health study. Implement Sci. 2013;8:98.
Ramirez R. Value co-production: intellectual origins and implications for practice and research. Strat Manage J. 1999;20:49–65.
Rycroft-Malone J, Burton CR, Bucknall T, Graham ID, Hutchinson AM, Stacey D. Collaboration and co-production of knowledge in healthcare: opportunities and challenges. Int J Health Policy Manag. 2016;5(4):221.
Filipe A, Renedo A, Marston C. The co-production of what? Knowledge values, and social relations in health care. PLoS Biol. 2017;15(5):e2001403.
Cahn E. No more throwaway people: the co-production imperative. Washington DC: Essential Books; 2001.
Jasanoff S. States of knowledge: the co-production of science and social order. 1st ed. London: Routledge; 2004.
Jasanoff S. Science and citizenship: a new synergy. Sci Public Policy. 2004;31(2):90–4.
Realpe A, Wallace LM. What is co-production. London: The Health Foundation; 2010. p. 1–1.
Lewin K. Action research and minority problems. J Soc Issues. 1946;2(4):34–46.
Lewin K. In: Lewin GW, editor. Resolving social conflicts; selected papers on group dynamics. New York: Harper & Row, 1948.
Freire P. The pedagogy of the oppressed. New York: Herder and Herder; 1970.
Tuhiwai Smith L. Decolonizing methodologies: research and Indigenous peoples. 2nd ed. New York: Zed Books; 2012.
Boaventura de Sousa Santos. Epistemologies of the South: justice against epistemicide. New York: Routledge; 2016.
Hall B, Tandon R, Tremblay C, editors. Strengthening community-university research partnerships: global perspectives. Victoria: University of Victoria; 2015.
Muhammad M, Wallerstein N, Sussman AL, Avila M, Belone L, Duran B. Reflections on researcher identity and power: the impact of positionality on community-based participatory research (CBPR) processes and outcomes. Crit Sociol. 2015;41(7–8):1045–63.
Allen ML, Salsberg J, Knot M, LeMaster JW, Felzien M, Westfall JM, Herbert CP, Vickery K, Culhane-Pera KA, Ramsden VR, Zittleman L, Martin RE, Macaulay AC. Engaging with communities, engaging with patients: amendment to the NAPCRG 1998 policy statement on responsible research with communities. Fam Pract. 2017;34(3):313–21. https://doi.org/10.1093/fampra/cmw074.
Argyris C, Schön DA. Organizational learning: a theory of action perspective. Reading: Addison-Wesley Publishing Company; 1978.
Arnstein SR. A ladder of citizen participation. J Am Inst Plann. 1969;35(4):216–24.
Fals-Borda O, Rahman MA. Action and knowledge: breaking the monopoly with participatory action- research. New York, London: Apex Press, Intermediate Technology Publications; 1991.
Dewey J. The public and its problems. 2nd ed. Athens: Swallow Press / Ohio University Press; 1954/1927.
Dewey J. Reconstruction in philosophy. Enlarged ed. Boston: Beacon Press; 1957.
Dewey J. The need for a recovery of philosophy. In: Morris D, Shapiro I, Dewey J, editors. The Political Writings. Indianapolis: Hackett Publishing Company; 1993.
Duran E, Duran B. Native American postcolonial psychology. Albany: SUNY Press; 1995.
Green LW. Making research relevant: if it is an evidence-based practice, where’s the practice-based evidence? Fam Pract. 2008;25:i20–4.
Habermas J. Knowledge and Human Interest. Boston: Beacon Press; 1971.
Hall B. The democratization of research in adult and non-formal education. In: Reason P, Rowan J, editors. Human inquiry: a source of new paradigm research. New York: Wiley; 1981. p. 447–56.
Hall B. Introduction. In: Park P, Brydon-Miller M, Hall B, Jackson T, editors. Voices of change: participatory research in the United States and Canada. Toronto: OISE Press; 1993. p. xiii–xii.
Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health. 1998;19:173–202.
Israel BA, Eng E, Schulz AJ, Parker EA. Methods in community-based participatory research for health. San Francisco: Wiley; 2005.
Israel BA, Eng E, Schulz AJ, Parker EA. Methods in community-based participatory research for health. 2nd ed. San Francisco: Wiley; 2012.
Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, Sirett E, Wong G, Cargo M, Herbert CP, Seifer SD, Green LW, Greenhalgh T. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Millbank Q. 2012;90(2):311–46.
Jagosh J, Bush PL, Salsberg J, Macaulay AC, Greenhalgh T, Wong G, Cargo M, Green LW, Herbert CP, Pluye P. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects. BMC Public Health. 2015;15:725.
Kemmis S, McTaggart R, Nixon R. The action research planner: doing critical participatory action research. Berlin: Springer Science & Business Media; 2013.
Kuhn TS. The structure of scientific revolutions. 1st ed. Chicago: University of Chicago Press; 1962.
Kuhn TS. The structure of scientific revolutions. 2nd ed. Chicago: University of Chicago Press; 1970.
Macaulay AC, Commanda LE, Freeman WL, Gibson N, McCabe ML, Robbins CM, Twohig PL. Participatory research maximises community and lay involvement. BMJ. 1999;319(7212):774–8.
Macaulay AC. Participatory research: what is the history? Has the purpose changed? Fam Pract. 2017;34(3):256–8.
Mercer SL, Green LW, Cargo M, Potter MA, Daniel M, Olds S, Reed-Gross E. Reliability-tested guidelines for assessing participatory research projects. In: Minkler M, Wallerstein N, editors. Community based participatory research for health: from process to outcomes. Hoboken: Wiley; 2008. p. 407–18.
Minkler M, Garcia A, Rubin V, Wallerstein N. Community-Based Participatory Research: A Strategy for Building Healthy Communities and Promoting Health through Policy Change. 2012. http://www.policylink.org/sites/default/files/CBPR.pdf. Accessed 26 Feb 2020.
Minkler M, Salvatore A. 10 Participatory Approaches for Study Design and Analysis in Dissemination and Implementation Research. In: Ross C. Brownson, Graham A. Colditz, and Enola K. Proctor, editors. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford University Press Online; 2012.
Minkler M, Wallerstein N. Community-based participatory research for health: from processes to outcomes. 2nd ed. San Francisco: Jossey-Bass; 2008.
Minkler M. Community-based research partnerships: challenges and opportunities. J Urban Health. 2005;82:ii3–12.
Minkler M, Vasquez VB, Warner JR, Steussey H, Facente S. Sowing the seeds for sustainable change: a community-based participatory research partnership for health promotion in Indiana, USA and its aftermath. Health Promot Int. 2006;21:293–300.
Minkler M, Freudenberg N. From community-based participatory research to policy change. In: Fitzgerald HS, Burack C, Siefer S, editors. Handbook on engaged scholarship: contemporary landscapes, future directions. East Lansing: Michigan State University Press; 2010.
Ramsden V, Salsberg J, Herbert C, Westfall J, LeMaster J, Macaulay AC. Patient and community oriented research: how is authentic engagement identified in grant applications? Can Fam Phys. 2017;63(1):74–6.
Salsberg J, Parry D, Pluye P, Macridis S, Herbert CP, Macaulay AC. Successful strategies to engage research partners for translating evidence into action in community health: a critical review. J Environ Public Health. 2015;2015:191856. https://doi.org/10.1155/2015/191856.
Salsberg J, Macridis S, Garcia Bengoechea E, Macaulay AC, Moore S. The shifting dynamics of social roles and project ownership over the lifecycle of a community-based participatory research project. Fam Pract. 2017;34(3):305–12.
Salsberg J, Merati N. Participatory health research in North America: from community engagement to evidence-informed practice, in participatory health research: voices from around the world. San Fransisco: Springer; 2018.
Selener D. Participatory action research and social change. Ithaca: Cornell University; 1997.
Viswanathan M, Ammerman A, Eng E, Garlehner G, Lohr KN, Griffith D, Rhodes S, Samuel-Hodge C, Maty S, Lux L, Webb L. Community-based participatory research: assessing the evidence. Evid Rep Technol Assess. 2004;99:1–8.
Wallerstein NB, Duran B. The conceptual, historical and practice roots of community based participatory research and related participatory traditions. In: Minkler M, Wallerstein N, editors. Community-based participatory research for health. San Francisco: Jossey-Bass; 2003. p. 25–47.
Wallerstein NB, Duran B. Using community-based participatory research to address health disparities. Health Promot Pract. 2006;7:312–23.
Wallerstein NB, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. Am J Public Health. 2010;100:S40–6.
Wallerstein NB, Duran B, Oetzel J, Minkler M. Community-based participatory research for health: advancing social and health equity. 3rd ed. San Francisco: Jossey-Bass; 2018.
Wright MT, Cook T, Springett J, Roche B. The international collaboration for participatory health research: legitimating the science and ensuring quality. In: Stern T, Townsend A, Rauch F, Schuster A, editors. Action research, innovation and change: International perspectives across disciplines. Abingdon: Routledge; 2013. p. 57–72.
Foote Whyte W. Participatory action research. Thousand Oaks: SAGE Publications; 1991.
Chambers R. Rural apprasial: rapid, relaxed and participatory. London: Institute of Development Studies; 1992.
Cornwall A, Jewkes R. What is participatory research? Soc Sci Med. 1995;41(12):1667–76.
Hakala J, Ylijoki OH. Research for whom? Research orientations in three academic cultures. Organization. 2001;8(2):373–80.
Crotty M. Introduction: the research process. In: Crotty M, editor. The foundations of social research: meaning and perspective in the research process. London: SAGE Publications; 1998. p. 1–17.
Oetzel JG, Wallerstein N, Duran B, Sanchez-Youngman S, Nguyen T, Woo K, Wang J, Schulz A, Kaholokula KA, Israel B, Alegria M. Impact of participatory health research: a test of the community-based participatory research conceptual model. Biomed Res Int. 2018;2018:7281405. https://doi.org/10.1155/2018/7281405.
Canadian Academy of Health Sciences. Academic recognition of team science: how to optimize the canadian academic system. 2018. https://www.cahs-acss.ca/academic-recognition-of-team-science-how-to-optimize-the-canadian-academic-system/. Accessed 1 Sep 2018.
Boivin A, L’Espérance A, Gauvin F-P, Dumez V, Macaulay AC, Lehoux P, Abelson J. Patient and public engagement in research and health system decision-making: a systematic review of evaluation tools. Health Expect. 2018;21(6):1075–84. https://doi.org/10.1111/hex.12804.
Greenhalgh T, Fahy N. Research impact in the community-based health sciences: an analysis of 162 case studies from the 2014 UK Research Excellence Framework. BMC Med. 2015;13:232.
Camden C, Shikako-Thomas K, Nguyen T, Graham E, Thomas A, Sprung J, Morris C, Russell DJ. Engaging stakeholders in rehabilitation research: a scoping review of strategies used in partnerships and evaluation of impacts. Disabil Rehabil. 2015;37(15):1390–400.
We would like to thank and acknowledge Jeremy Grimshaw, Andreas Laupacis, Janet Squires, Sharon Straus, Jacqueline Tetroe, Hans van Oers and the international KT experts at the 2017 Knowledge Utilization Colloquium in Melbourne for their support in the development and member-checking of this study.
This paper was written in response to a call by the IKTRN for papers on Integrated Knowledge Translation. The IKTRN (IDG Programme Leader) is funded by a Canadian Institutes of Health Research Foundation Grant (FDN #143247). TN holds a Postdoctoral Fellowship from the Canadian Institutes of Health Research (2016–2019).
Ethics approval and consent to participate
The study received ethics approval from the Ottawa Health Sciences Network Research Ethics Board (OHSN- REB #2017038-01H).
Consent for publication
Written informed consent for publication was obtained.
Several of the authors are members of the Integrated Knowledge Translation Research Network (IKTRN) (IDG, TN, AK, JL, ACM, MM, KJM, JS). The IKTRN brings together knowledge users and researchers to advance the science and practice of IKT and train the next generation of IKT researchers.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file 1.
Additional file 2.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Nguyen, T., Graham, I.D., Mrklas, K.J. et al. How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field. Health Res Policy Sys 18, 35 (2020). https://doi.org/10.1186/s12961-020-0539-6
- Integrated knowledge translation
- Engaged scholarship
- Mode 2 research
- Participatory research
- Collaborative research
- Implementation science