- Open Access
- Open Peer Review
How are evidence generation partnerships between researchers and policy-makers enacted in practice? A qualitative interview study
© The Author(s). 2019
- Received: 4 November 2018
- Accepted: 19 March 2019
- Published: 15 April 2019
Evidence generation partnerships between researchers and policy-makers are a potential method for producing more relevant research with greater potential to impact on policy and practice. Little is known about how such partnerships are enacted in practice, however, or how to increase their effectiveness. We aimed to determine why researchers and policy-makers choose to work together, how they work together, which partnership models are most common, and what the key (1) relationship-based and (2) practical components of successful research partnerships are.
Semi-structured qualitative interviews were conducted with 18 key informants largely based in New South Wales, Australia, who were (1) researchers experienced in working in partnership with policy in health or health-related areas or (2) policy and programme developers and health system decision-makers experienced in working in partnership with researchers. Data was analysed thematically by two researchers.
Researcher-initiated and policy agency-initiated evidence generation partnerships were common. While policy-initiated partnerships were thought to be the most likely to result in impact, researcher-initiated projects were considered important in advancing the science and were favoured by researchers due to greater perceived opportunities to achieve key academic career metrics. Participants acknowledged that levels of collaboration varied widely in research/policy partnerships from minimal to co-production. Co-production was considered a worthy goal by all, conferring a range of benefits, but one that was difficult to achieve in practice. Some participants asserted that the increased time and resources required for effective co-production meant it was best suited to evaluation and implementation projects where the tacit, experiential knowledge of policy-makers provided critical nuance to underpin study design, implementation and analysis. Partnerships that were mutually considered to have produced the desired outcomes were seen to be underpinned by a range of both relationship-based (such as shared aims and goals and trust) and practical factors (such as sound governance and processes).
Our findings highlight the important role of policy-makers in New South Wales in ensuring the relevance of research. There is still much to understand about how to initiate and sustain successful research/policy partnerships, particularly at the highly collaborative end.
Bridging the evidence–practice gap in health services has the potential to make a substantial contribution to improving health outcomes and service efficiencies [1, 2] and to reducing research waste . Proposed solutions to bridging this gap originally focussed heavily on strategies to enhance the dissemination of research findings to research users [2, 3] or on building the capacity of policy-makers and clinicians to engage with and use research evidence . More recently, focus has turned to the potential for evidence generation partnerships between researchers and knowledge users, such as policy-makers, community groups, clinicians and consumers, to improve the availability of relevant, timely evidence to help inform decision-making [5, 6]. There has been a proliferation of research funding initiatives internationally that require researcher/knowledge user collaboration  and increasing demands on researchers to demonstrate that their work has had an ‘impact’ on policy or practice [8–10].
A number of research approaches that centre around evidence generation partnerships have been outlined in recent years, including integrated knowledge translation [5, 11], participatory action research [12, 13] and engaged scholarship [14, 15]. Most of these approaches emphasise the co-production of knowledge , whereby researchers and those most likely to use or be affected by the evidence produced, work together to produce evidence. By combining the varied skills and expertise of these groups, it is hypothesised that the resultant research will have greater relevance to knowledge users and will more likely be used in decision-making [16, 17].
Much has been published about many of these approaches from a conceptual standpoint [18, 19], and there has similarly been a considerable amount of literature examining how evidence generation partnerships function [13, 20, 21]. Most of this research, however, does not explore evidence generation partnerships specifically between researchers and policy-makers [22, 23], focusing instead on partnerships with community groups , clinicians  and others. All of these partnership combinations are likely to share some common features such as a requirement to take into account the diverse perspectives and needs of all partners; however, the policy-maker/researcher combination is likely to present some unique partnership challenges and opportunities due to the vastly different work environment and pressures that these groups face [25–27]. Further, while rarely the case with most other common partnership combinations, evidence generation partnerships between policy agencies and researchers may be initiated and funded by the policy agency themselves or be funded through external research grants (such projects may be more likely to be researcher-initiated). These different modes of funding and initiation may have important implications for how a partnership functions ; however, this has rarely been explored. Thus, there remains much to be learnt about how research/policy partnerships are enacted in practice  and how best to bring the diffuse skills, needs and priorities of researchers together effectively [30, 31].
Why do researchers and policy-makers choose to work together? What are the perceived benefits?
How do they work together? Which partnership models are most common?
What are the key (1) relationship-based and (2) practical components of successful research partnerships?
Participants and setting
Data was collected between October 2017 and April 2018. We sought to interview participants from two groups; namely (1) researchers experienced in working in partnership with policy in health or health-related areas and (2) policy and programme developers and health system decision-makers (hereafter policy-makers, defined as “someone employed in a policy agency who drafts or writes health policy documents or develops health programs, or who makes or contributes significantly to policy decisions about health services, programs or resourcing” ) experienced in working in partnership with researchers. Participants were purposively identified by an Advisory Committee guiding this project. The role of the Advisory Committee was to provide expert advice on issues relating to the scope, implementation and dissemination of a programme of work centred on partnership research of which the current paper is a part. The Advisory Committee included an approximately equal mix of researchers and senior policy-makers, selected for their expertise in partnership research, and their extensive knowledge of researchers and policy-makers who work in partnered research. Advisory Committee members asked the permission of potential participants before passing on their contact details to the study team.
Participants were eligible to participate in the current study if they were nominated by a member of the Advisory Committee based on the criteria that they (1) were a researcher or policy-maker; (2) worked primarily in health or health-related areas (e.g. social care); (3) had extensive experience in partnering with researchers (for policy-makers) or policy-makers (for researchers) on research projects (partnership on at least five projects); and (4) were employed by a university, policy or programme agency or Local Health District. All participants provided written, informed consent to participate.
A semi-structured approach was used to elicit participants’ opinions and experiences regarding the barriers and facilitators to successful evidence generation partnerships amongst researchers and research users in accordance with best practice guidelines (Additional file 1: Appendix 1) . Interviews were conducted by AW, a researcher with extensive expertise in conducting research in partnership with policy-makers. Each interview sought information on why the interviewee engaged in partnership research and the perceived risks and benefits, how the interviewee would characterise the types of partnership models they have engaged in (and the advantages and disadvantages of each), characteristics of successful and unsuccessful partnerships (and the factors associated with each), and indicators and predictors of impact. Interviews were conducted in person or by phone depending on the preference of the interviewee and ranged from half to one hour in length. Data was audio recorded before being transcribed by a person with no personal or professional connection to the participants involved in the study.
Data was analysed thematically. Two researchers (AW and HT) independently read all of the transcripts and coded the data to discern themes inductively. No predetermined framework was used to guide analysis. The researchers met regularly to review the draft codes and themes throughout the process until agreement was reached regarding the final version. Synthesised data and emerging themes were reviewed by the Advisory Committee who participated in sense making of emerging data.
A total of 18 key informants participated in interviews, of whom 7 were primarily researchers and 11 were currently primarily policy-makers. All participating researchers were employed by universities at the Associate Professor or Professor level and had more than 15 years research experience in public health research. All of the policy-makers who participated were employed at a Manager level or above by government agencies whose work focussed on health or health-related issues (such as social care). Six of the policy-makers had PhDs and/or had previously been employed as researchers and were thus able to bring both a researcher and a policy-maker lens to their analysis of research partnerships. All participants, except for one researcher, were based in New South Wales, Australia. The themes which emerged from the interview with the participant from outside New South Wales were consistent with those which emerged more broadly.
Why do researchers and policy-makers choose to work together?
Three major themes emerged regarding why researchers choose to work with policy, namely (1) increasing the likelihood of research impact; (2) gaining access to sought after resources; and (3) obtaining funding. The reason most commonly cited by researchers for wanting to partner with policy-makers to conducting research was the belief that the resultant research would more likely have an impact on policy and thus contribute to improving health.
“I do research so that I can improve health… the main way in which research impacts health is through policy... Research without any sort of policy engagement will often sit on the shelf and do nothing…” Researcher
Many researchers were also motivated to partner with policy-makers due to the access to otherwise unavailable resources this could facilitate (such as routinely collected data held by agencies or the data required to evaluate large-scale government policies or programmes). Researchers also reported seeking collaborations with policy-makers due to the funds attached to the work, either through completing a tendered project or, preferably, receiving funds to carry out a project the researchers had initiated or co-produced in collaboration with a relevant agency.
Five major themes emerged regarding what policy-makers perceived they gained by partnering with researchers, namely (1) access to additional skills; (2) links to researchers who can be called on for timely, informal advice; (3) access to the networks of their research collaborators; (4) the creation of high quality, relevant evidence; and (5) public, evidence-based support for government decisions. Policy-makers most commonly reported seeking to collaborate with researchers in order to gain access to skills and capacity over and above that already available within their agency.
“…the reason you go to a researcher is that they’ve got the expertise or the resources or the time or whatever in order to be able to collect that information that you need and therefore that’s helpful to you.” Policy-maker
Having established collaborations with researchers was also said to create a situation in which policy-makers felt able to contact these researchers when they required fast expert advice on a relevant issue. Policy-makers also noted that their collaborators often assisted them in making links with researchers outside of the collaboration who possessed additional specific skills or knowledge they needed. Policy-makers were also driven to collaborate with researchers in order to create high quality evidence that was seen to have credibility to support their work.
Collaborations with researchers were also seen to be helpful when the researcher elected to support government decisions in the public domain, for example, by speaking about the evidence base underpinning policy, programme or health service delivery decisions. While research users did not report requesting assistance of this nature, they appreciated it when provided.
How do researchers and policy-makers collaborate on research projects?
Four major themes emerged in relation to researcher/policy-maker collaboration, namely (1) wide variation in extent of collaboration; (2) policy agency-initiated partnerships; (3) researcher-initiated partnerships; and (4) co-produced partnerships. Most participants agreed that “someone has to have the initial idea” for a partnership; as such, participants agreed that most partnerships could be categorised as either policy agency initiated, including projects put out for tender by agencies and commissioned work, or researcher initiated, either through a partnered funding application to an external granting body or by a direct approach to an agency to fund a particular piece of work. Once these partnerships began, there was said to be a considerable level of variation in the extent to which the researchers and agency staff involved collaborated from almost no collaboration through to high levels of collaboration throughout all stages of the research process. This sustained, high level collaboration was considered by participants to characterise co-production. Policy-makers and researchers tended to nominate research projects centred around evidence reviews and analysis of existing datasets as requiring minimal collaboration, while closer collaboration was considered important when conducting interventions and evaluations.
“So, I see partnerships as a sort of continuum. At one end is that transactional type interaction, through to co-production at the other end.” Policy-maker
However, these partnership categories (policy- or researcher-initiated and extent of collaboration) were not seen to be entirely clear cut, with some researchers and policy-makers explaining that, as some policy agencies move towards embedding researchers within their teams and sometimes funding research centres, the distinction between policy-makers and researchers can sometimes be blurry.
A few policy-makers felt that highly collaborative, co-produced research partnerships were always preferable; however, most participants reported that various levels of collaboration could be effective depending on the situation.
“I guess again we don’t come at it from which model works best. It’s more like what are the requirements of the project.” Policy-maker
Policy agency-initiated partnerships
Illustrative quotes regarding policy agency-initiated partnerships
Benefit to researchers
Risk to researchers
Risk to policy agencies
Agency best placed to identify critical real-world issues
“I feel like the agency’s good at identifying the problem and that’s often unknown to researcher. They’re just not in the trenches. They just kind of can’t see it.” Researcher
Stepping stone to future collaborations
“…it’s a long game, these relationships. Doing the work that they’re commissioning but also with a view to potentially collaborating with them on bigger things that we might potentially have an interest in, in the future.” Researcher
“I have a research team of people who are ambitious… In an ideal world they’ll be doing projects which are targeting high impact, international journals. The sort of work that we do for government here isn’t that sort of work. So, it does come at that cost as well.” Researcher
Researchers not engaging with complexity
“Sometimes they get very narrow in terms of what they get interested in or they need to define things in a very narrow way that means that they can measure it, so it makes it easier for them, but it doesn’t necessarily answer the questions that you want. When from our point of view we’ll think, they’re just not even trying to get how complex, they just want it to be simple because that’s their area of expertise.” Research user
Evidence generated more likely to be used in policy or practice
“I think if they’ve got a specific evidence need and know the evidence they need to gather or synthesise to answer it, I think that’s quite an acceptable process that will probably lead to really quick translation because they need the evidence and as soon as they get it, they’ll make the decision on its basis.” Researcher
Changing political environment
“So honestly my best work was dead in the water within six months because of politics.” Researcher
Small, short term, policy-initiated projects were rarely seen by researchers as offering opportunities to conduct ‘cutting edge research’, and reportedly often did not generate publications or turn a profit. Researchers reported willingness to engage in these and other types of agency-initiated projects anyway as they were seen as an effective way to build a relationship with a policy agency and gain a better understanding of the policy context in the hopes of developing more advantageous collaborations.
On the other hand, most researchers outlined a number of potential risks or costs that were sometimes associated with agency-initiated work. Chief among these was a perceived opportunity cost, with time spent working on agency-initiated research sometimes resulting in less traditionally valued research outputs (such as peer-reviewed publications) than time spent on researcher-initiated work. Indeed, policy-makers and researchers alike reported that, as policy-makers often needed evidence around a specific issue, within a specific timeframe and at a specific cost, research methods needed to be determined pragmatically, rather than striving for the most scientifically excellent design. In addition, the evidence produced was sometimes rendered irrelevant by changes in the political environment.
Time was again a concern amongst researchers when the amount of time spent to complete a project to an agency’s satisfaction was seen to ‘blow out’. Researchers and some policy-makers reported that this was often due to agencies not being “clear [about] what they want”, resulting in a more iterative process than was anticipated. This challenge was also sometimes seen to be related to agencies not disclosing key information to researchers, limiting their understanding of the context and what was required.
Policy-makers also reported potential risks related to engaging with researchers in agency-initiated research, including not receiving the information or evidence they required, or it being of poor quality, or delivered well after the agreed deadline. Another commonly reported barrier was researchers overpromising to win a tender but not being able to deliver on these promises. Frustrations were sometimes said to arise due to researchers ignoring the complexity of the issue being investigated.
Finally, policy-makers reported that, in policy-initiated projects, researchers often failed to adequately synthesise the reported evidence and/or provide recommendations for action. As recommendations for action were reportedly often one of the key outcomes sought by agencies in these partnerships, this was particularly disappointing.
Illustrative quotes about researcher-initiated partnerships
Benefit for researchers
Risk for research user agencies
“Investigator driven research is fine, we actually encourage it all the time. That’s where ideas come up, where things that we haven’t thought of - we don’t have the capacity to think of everything and have every partner and be at every meeting. So certainly, there’s a very important role for investigator driven research.” Research user
Increased opportunity for cutting edge research
“…if the researcher is initiating it…it’s more likely to be characterised by sort of something quite new in terms of maybe methods or thinking outside the square.” Policy-maker
“…they [researcher-initiated partnership projects] can take a long time, longer than you need or have available. So you don’t get the answer you want when you want it, you get it eventually, but it may be too late by that point, in which case it’s something of a waste of time.” Policy-maker
“...they [policy-makers] weren’t acknowledged. They weren’t told that a peer review paper had been produced, it just came out…There wasn’t  the respect to even ask to produce the paper, and  to even ask whether someone could be an author on it. Someone who put a lot of intellectual property into the project.” Policy-maker
Lack of practical outcomes
“[A risk is that researchers were not]…making recommendations in reports that speak to those issues of policy and practice implications, rather than being airy fairy high falutin, more research needs to be in A, B, C. It’s not very helpful.” Research user
This risk was seen to be exacerbated by another commonly noted risk of this partnership type, namely that the researchers take over and offer little opportunity for policy-makers to help shape the research agenda. Relatedly, many policy-makers reported previous dissatisfactions related to researchers not sharing credit for the work the partnership produced, for example, through co-authorship or shared publicity.
Researchers cited significant benefits associated with researcher-initiated partnerships, including the opportunity to focus on their specific research interests and employ more ambitious study designs, increasing their chances of obtaining peer-reviewed grants and high impact journal articles. The chief risks researchers reported in researcher-initiated research partnerships were that the policy-maker partners did not engage (attend meetings, provide information and advice) or did not deliver on promised resources such as access to particular datasets.
Illustrative quotes regarding co-production
Research more likely to be used
“…what increases the likelihood of research being used is if it’s coproduced, because then we’re actually interacting all the time. We’re talking, we know what the priorities are, and we’re actually setting an agenda for research and we’re collaborating on that…if we’re involved in the process from day dot, involved in its implementation, of course we’re going to use it, more likely to use it aren’t we?” Policy-maker
Co-production difficult to achieve
“I think the agency-led or researcher-led in my experience is common. But we have pursued that co-led idea, because I think that could have so many benefits. We haven’t - I don’t think we’ve had a lot of success with it, but I think it’s a really great model.” Policy-maker
Merging of experiential and scientific knowledge
“…there is a lot of evidence that is really more implicit, part of the tacit knowledge, part of the [experiential] knowledge that stakeholders have, and that engagement process and coproducing evidence of what works and what does not work is really quite promising. I tell you that in many cases, even systematic reviews…sometimes you might have strong evidence, but it doesn’t mean that at the level of implementation in governing contract it would work. This is where the knowledge coproduction piece is important.” Researcher
Co-production may require shared risk
“… you do need shared risk. Whether it’s financial shared risk, outcome shared risk, reputational shared risk, and that’s what kind of drives a lot of things, and sort of shared input in some ways, whether it’s in kind, or not in kind, or whatever it is. So, I think shared input, and shared risk, I think, is something that, probably, people might not want to say, but it’s very true, because that’s where you’ll get a lot of input.” Policy-maker
Keeps research relevant
“I think it makes the research better….you can check in with the people who are going to be most impacted by the research as you go along. Make sure you don’t go off on an academic tangent and lose relevance. Because real-world questions are messy and crowded, and often not exactly as they would be in the perfect academically articulated research questions.” Researcher
The benefits of co-production were seen to derive in part from the greater levels of engagement between all members of the team seen to be implicit in this model, driven by mutual aims and interests. This high level of engagement was in turn seen to allow the complementary skills and knowledge of all partners to be optimised. Resulting from this, a core strength of co-production was thought to be its ability to integrate tacit, experiential knowledge with traditional ‘evidence’, resulting in particularly nuanced and relevant outputs. Co-production was also seen to provide opportunities for frequent ‘relevance checks’ to ensure that the research being undertaken remains in line with ‘real world’ priorities.
Nonetheless, while co-production was the goal for many participants, some noted that, thus far, they had not been able to achieve it, with either the policy or the researcher partners dominating partnership projects in practice. Multiple reasons were given for the reported tendency for one group to dominate research partnerships, ranging from lower expectation amongst one group that the project might benefit themselves or their agency resulting in reduced engagement, to the dominating group offering little real opportunity for their partners to contribute to shaping the research agenda. The most frequently mentioned facilitators of co-production were things that allowed long-term relationships and trust to develop between researchers and policy-makers, namely stability of staff at policy agencies and policy agency-funded research centres.
What does ‘success’ in research partnerships look like?
Seven subthemes were identified in relation to the characteristics of successful partnerships. Key informants most commonly described a successful partnership as one where “all stakeholders are happy with the outcomes”. Expanding further, common perceptions of successful partnerships included all parties having a shared understanding of why they were partnering and what this would involve, the planned deliverables being delivered to a high standard and relevant, usable evidence being generated. Many participants noted that a successful partnership produces more than any partner could have alone and that it results in mutual gain. Creating evidence that impacts on policy and practice was considered an indicator of success, but as participants recognised that impact is influenced by a complex array of factors, they considered that partnerships could be considered successful even in the absence of this.
What are the components of successful research partnerships?
Major themes regarding relationship-based and practical facilitators of successful partnerships
Shared aims and goals
Taking time at the outset to ensure there is a shared understanding of the project
Understanding each other’s needs and drivers
Agreed governance structures and processes (including publication policy)
Clearly specified timelines and deliverables
Taking time to build a relationship OR a pre-existing relationship
All delivering as promised
Cross-cutting facilitators of successful partnerships
Sharing wins and credit
“So that’s my little tip there. All the energy at the start has to be about where you want to finish and why a partnership is important, because it will also help you during the times when the partnership’s tested, to remind yourself why working together was important in the first place.” Researcher
“… it’s the relationship building that’s critical, being prepared to put the time and effort into the relationship building and having the ability to empathise with the perspective of someone who’s trying to run a health service or someone who’s having to make a policy decision…” Researcher
Practical components of successful partnerships
“The best partnerships are clearly win/win…Where it becomes one-sided, all the gains being on one side at the expense of the other, that’s where things fall apart because the side that’s not gaining is thinking well, what am I doing here?” Researcher
Our findings suggest that both researcher-initiated and policy agency-initiated evidence generation partnerships were common among researchers and policy-makers in New South Wales. While policy-initiated partnerships were thought to be the most likely to result in impact , researcher-initiated projects were seen to play an important role in advancing the science and often provided researchers with more opportunities to achieve the outputs valued in their profession such as high impact publications. In keeping with the literature, participants acknowledged that levels of collaboration varied widely between research/policy partnerships [16, 20, 28, 35, 36].
Co-production, or collaboration across all stages of the research process, was seen by some participants as the ideal model for collaboration, increasing the relevance of the research produced and the likelihood of impact [37–39]. Others considered that this time and resource-intensive way of working was not always warranted . While agreeing aims, goals and deliverables collaboratively was always viewed as essential , these informants suggested that, once this was accomplished, some types of projects, such as analyses of large datasets and evidence reviews, could be completed largely independently by researchers with no loss of quality or relevance. In contrast, co-production was seen to be particularly worthwhile in relation to evaluation and implementation projects where the tacit, experiential knowledge of policy-makers provided critical nuance to underpin study design, implementation and analysis. Of note, several policy-makers suggested that, while ideally they would like to co-produce research, in practice they had not found this to be possible. Participants suggested that trusting, long-term relationships were generally a precursor to co-production. These were seen to be facilitated by policy agencies having a stable workforce; frequent staff changes in policy agencies are frequently cited as a barrier to successful evidence generation partnerships in the literature [31, 42] and policy agency-funded research centres .
Key informants in the current study highlighted a clear distinction between researcher and policy agency-initiated partnerships. In keeping with the literature, policy-makers were thought by both groups to be best placed to identify what evidence is needed to guide real world decision-making [5, 44, 45]. The relevance of the ensuing research and the ability of policy agencies to utilise it meant that policy agency-initiated work was reported to be particularly likely to lead to real-world impact [22, 34]. Achieving such impact was the primary motivation reported by researchers for engaging in evidence generation partnerships . Potential opportunity costs were also noted, including time taken away from producing ‘world class’ research [28, 46], challenges agreeing on final outputs [28, 29], unexpectedly iterative processes  and the potential for the evidence produced to remain unused due to changes in the policy landscape .
Many of the key frustrations expressed by policy-makers around policy agency-initiated partnerships in the current study have been less commonly reported in the literature. These include researchers overstating what is possible in order to win tenders (leading to disappointment with the final product), choosing only to focus on the elements of a topic that interest them, or oversimplifying things in order to make them ‘doable’ and failing to fully engage with the complexity of an issue. As systems thinking approaches gain momentum in research [48, 49], it will be interesting to observe whether this last challenge becomes less common. A key frustration noted by policy-makers was the failure of researchers to provide recommendations for action, one of the main outcomes reportedly sought by agencies when they initiated partnerships with researchers [28, 50, 51]. Despite the noted risks, researchers stated that even small agency-initiated projects were worthwhile as relationship-building and learning exercises [27, 52], which may later help to underpin partnerships they felt to be more favourable to themselves. Policy-makers for their part continued to initiate evidence generation partnerships in order to access additional capacity and expertise to produce high quality, credible evidence to inform their work.
Researchers in the current study placed particular value on researcher-initiated partnerships. These were considered to offer the opportunity to conduct research that was both ‘scientifically excellent’, and likely to have impact. Policy-makers, on the other hand, noted that, while sometimes providing useful evidence, these projects were less often of immediate relevance or importance to them. Unsurprisingly then, the major risk researchers reported in researcher-initiated partnerships was that policy-makers did not engage [29, 53]. This suggests that even researchers who aim to conduct research with real world impact, and who are experienced in working partnership with policy, are often not in alignment with policy agencies in their opinions as to research priorities. This finding is consistent with the large body of literature describing the widespread differences between the research and policy worlds [26, 27, 37, 54], whereby researchers most often attempt to ‘push’ their research to policy agencies rather than ascertaining when there may be a policy ‘pull’ for evidence and building at least some of their research agenda to align with this [55, 56]. One mechanism governments in New South Wales  and others internationally [58–61] employ to help develop long-term collaborations with researchers and increase the production of policy-relevant research is government-funded research centres. While as yet there have been few evaluations of such initiatives, it seems a potentially promising method of facilitating productive, lasting partnerships.
Many of the key barriers and facilitators reported to underpin partnership success or failure noted in the current study are consistent with those reported in the literature. Many of these appeared to speak primarily of a clear need to ensure that the different needs and drivers of all parties were understood, valued and accommodated as much as possible. Thus, participants underlined the foundational need for shared aims and goals, ensuring that all parties were benefiting from the collaboration and maintaining open communication, trust and mutual respect [16, 20, 21, 27, 42].
Amongst our key informants, practical tools were also thought to be of great value in supporting a well-functioning partnership; these included having clearly documented and agreed timelines, deliverables, workplans, publication policies and governance structures [31, 53]. While some groups appear to have well-developed methods in this area, this appears to be a partnership capacity gap for others. As noted elsewhere [29, 31], ongoing engagement, expressed through things such as attendance at meetings, participation in discussions and provision of feedback and presenting findings as they emerge, was described as critical to keeping the project on track and maintaining relationships.
A strength of the current study is the participation of both researchers and policy-makers who are highly experienced in working in partnership on evidence generation activities as we expected this group to be able to provide considerable insight into the complexities of research/policy partnerships. By exploring their views broadly, rather than in relation to a particular project, we were able to draw out their key learnings from working in partnership on evidence generation activities. We acknowledge the limitations inherent in the small sample size utilised here, the fact that most of our participants were from New South Wales and that our sample was not randomly selected. Indeed, the experiences of these highly practiced individuals may diverge from those of researchers and policy-makers who have less experience of participating in research/policy partnerships, or who work in settings where such partnerships are less common. Although many of our findings align with the existing literature, these limitations mean that our findings may not be broadly generalisable.
There is still much to understand about how to initiate and sustain successful research/policy partnerships, particularly at the highly collaborative co-production end. Our findings highlight the important role of policy-makers in ensuring the relevance of research, mostly in evaluation and implementation projects where co-production appears to be particularly valuable. The next phase of our work will seek to provide a more detailed exploration of the specific practical strategies, including governance approaches, that can be put in place to facilitate strong and effective evidence generation partnerships between researchers and policy-makers.
Our sincere thanks to the key informants who participated in this study and generously shared their experiences and learnings.
This work was funded through a New South Wales Health EMC Fellowship held by AW. LW is supported by a NHMRC CDF II Fellowship (APP1128348).
Availability of data and materials
The datasets generated and/or analysed during the current study are not publicly available due to privacy concerns but may be available from the corresponding author on reasonable request.
AW led this work, including contributing to the study design, conducting and analysing the key informant interviews and drafting the paper. HT analysed the interviews and contributed to drafting the paper. All authors made substantial contributions to the design of the study and the analysis and interpretation of data and were involved in critically revising the manuscript for important intellectual content. All authors read and approved the final manuscript.
Ethics approval and consent to participate
The current study was approved by the University of New South Wales human Ethics Committee (HC17843).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Macleod M, Michie S, Roberts I, Dirnagl U, Chalmers I, John PA, Ioannidis J, et al. Biomedical research: increasing value, reducing waste. Lancet. 2014;383(9912):101–4.View ArticleGoogle Scholar
- Dobbins M, Hanna SE, Ciliska D, Manske S, Cameron R, Mercer SL, et al. A randomized controlled trial evaluating the impact of knowledge translation and exchange strategies. Implement Sci. 2009;4:61.View ArticleGoogle Scholar
- Lavis JN, Wilson MG, Grimshaw JM, Haynes RB, Hanna S, Raina P, et al. Effects of an evidence service on health-system policy makers’ use of research evidence: a protocol for a randomised controlled trial. Implementation Sci. 2011;6:51.View ArticleGoogle Scholar
- CIPHER Investigators. Supporting Policy In health with Research: an Intervention Trial (SPIRIT) – protocol for a stepped wedge trial. BMJ Open. 2014;4(7):e005293.View ArticleGoogle Scholar
- Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Heal Prof. 2006;26:13–24.View ArticleGoogle Scholar
- Heaton J, Day J, Britten N. Collaborative research and the co-production of knowledge for practice: an illustrative case study. Implement Sci. 2016;11:20.View ArticleGoogle Scholar
- McLean RKD, Graham ID, Tetroe JM, Volmink JA. Translating research into action: an international study of the role of research funders. Health Res Policy Syst. 2018;16:44.View ArticleGoogle Scholar
- Greenhalgh T, Fahy N. Research impact in the community-based health sciences: an analysis of 162 case studies from the 2014 UK Research Excellence Framework. BMC Med. 2015;13:232.View ArticleGoogle Scholar
- Hanney S, Greenhalgh T, Blatch-Jones A, Glover M, Raftery J. The impact on healthcare, policy and practice from 36 multi-project research programmes: findings from two reviews. Health Res Policy Syst. 2017;15:26.View ArticleGoogle Scholar
- Newson R, King L, Rychetnik L, Bauman AE, Redman S, Milat AJ, et al. A mixed methods study of the factors that influence whether intervention research has policy and practice impacts: perceptions of Australian researchers. BMJ Open. 2015;5(7):e008153.View ArticleGoogle Scholar
- Graham ID, Kothari A, McCutcheon C, Integrated Knowledge Translation Research Network Project Leads. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation. Implementation Sci. 2017;13:22.View ArticleGoogle Scholar
- Baum F, MacDougall C, Smith D. Participatory action research. J Epidemiol Community Health. 2006;60(10):854.View ArticleGoogle Scholar
- Jagosh J, Biush PL, Slalsberg J, Macaulay MC, Greenhalgh T, Wong G, et al. A realist evaluation of community participatory research: partnership synergy, trust building and ripple effects. BMC Public Health. 2015;15:725.View ArticleGoogle Scholar
- Bowen SJ, Graham ID. From knowledge translation to engaged scholarship: promoting research relevance and utilization. Arch Phys Med Rehabil. 2013;94:S3–8.View ArticleGoogle Scholar
- Van de Ven AH. Engaged Scholarship: A guide to Organizational and Social Research. New York: Oxford University Press; 2007.Google Scholar
- Sibbald SL, Tetroe J, Graham ID. Research funder required research partnerships: a qualitative inquiry. Implementation Sci. 2014;9:176.View ArticleGoogle Scholar
- Solberb LI, Glasgow KE, Unutzer J, Jaeckels N, Oftedahl G, Beck A et al. Partnership Implementation Research. Medical Care. 2010; 48(7): 576-82.View ArticleGoogle Scholar
- Graham I, Tetroe J. Some theoretical underpinnings of knowledge translation. Acad Emerg Med. 2008;14(11):936–41.View ArticleGoogle Scholar
- Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q. 2016;94(2):392–429.View ArticleGoogle Scholar
- Corbin JH, Jones J, Barry MM. What makes intersectoral partnerships for health promotion work? A review of the international literature. Health Promot Int. 2018;33(1):4–26.PubMedGoogle Scholar
- Rycroft-Malone J, Burton C, Wilkinson J, Harvey G, McCormack B, Baker R, et al. Collective action for knowledge mobilisation: a realist evaluation of the Collaborations for Leadership in Applied Health Research and Care. Health Services and Delivery Research. Southampton: NIHR Journals Library; 2015.Google Scholar
- Bowen S, Botting I, Graham ID, Huebner LA. Beyond “two cultures”: guidance for establishing effective researcher/health system partnerships. Int J Health Policy Manag. 2017;6(1):27–42.View ArticleGoogle Scholar
- Voorberg WH, Bekkers VJJM, Tummers LG. A systematic review of co-creation and co-production: embarking on the social innovation journey. Public Manag Rev. 2015;17(9):1333–57.View ArticleGoogle Scholar
- Sahs JA, Nicasio AV, Storey JE, Guarnaccia PJ, Lewis-Fernandez R. Developing research collaborations in an academic clinical setting: challenges and lessons learned. Community Ment Health J. 2017;53(6):647–60.View ArticleGoogle Scholar
- Caplan N. The two-communities theory and knowledge utilization. Am Behav Sci. 1979;22(3):459–70.View ArticleGoogle Scholar
- Brownson RC, Royer C, Ewing R, McBride TD. Researchers and policymakers: travelers in parallel universes. Am J Prev Med. 2006;30(2):164–72.View ArticleGoogle Scholar
- Kothari A, MacLean L, Edwards N, Hobbs A. Indicators at the interface: managing policymaker-researcher collaboration. Knowledge Manage Res Pract. 2011;9:203–14.View ArticleGoogle Scholar
- Martin S. Co-production of social research: strategies for engaged scholarship. Public Money Management. 2010;30:211–8.View ArticleGoogle Scholar
- Nystrom ME, Karltun J, Keller C, Gare A. Collaborative and partnership research for improvement of health and social services: researchers experiences from 20 projects. Health Res Policy Syst. 2018;16:46.View ArticleGoogle Scholar
- Kothari A, Wathen CN. Integrated knowledge translation: digging deeper, moving forward. J Epidemiol Community Health. 2017;71:619–23.View ArticleGoogle Scholar
- Gagliardi AR, Berta W, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (IKT) in health care: a scoping review. Implementation Sci. 2016;11:38.View ArticleGoogle Scholar
- Haynes A, Turner T, Redman S, Milat AJ, Moore G. Developing definitions for a knowledge exchange intervention in health policy and program agencies: Reflections on process and value. Int J Soc Res Methodol. 2015;18(2):145–59.View ArticleGoogle Scholar
- Tracy SJ. Qualitative quality: eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry. 2010;16(10):837–51.View ArticleGoogle Scholar
- Kok MO, Gyapong JO, Wolffers I, Ofori-Adjei D, Ruitenberg J. Which health research gets used and why? An empirical analysis of 30 cases. Health Res Policy Syst. 2016;14:36.View ArticleGoogle Scholar
- Murray L. Deliberative research for deliberative policy making: creating and recreating evidence in transport policy. Social Policy Society. 2011;10(4):459–70.View ArticleGoogle Scholar
- Ross S, Lavis J, Rodriguez C, Woodside J, Denis J. Partnership experiences: involving decision-makers in the research process. J Health Serv Res Policy. 2003;8(Suppl 2):26–34.View ArticleGoogle Scholar
- Campbell D, Redman S, Jorm L, Cooke M, Zwi AB, Rychetnik L. Increasing the use of evidence in health policy: Practice and views of policy makers and researchers. Aust New Zealand Health Policy. 2009;6:21.View ArticleGoogle Scholar
- Innvaer S, Vist G, Trommald M, Oxman A. Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002;7(4):239–44.View ArticleGoogle Scholar
- Lomas J. Improving Research Dissemination and Uptake in the Health Sector: Beyond the Sound of One Hand Clapping. In: Policy Commentary. Hamilton, ON: McMaster University Centre for Health Economics and Policy Analysis; 1997. Report No.: C97–1.Google Scholar
- Gajda R. Utilizing collaboration theory to evaluate strategic alliances. Am J Eval. 2004;25:66–77.View ArticleGoogle Scholar
- Gagliardi AR, Kothari A, Graham ID. Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know. J Epidemiol Community Health. 2017;71(2):105–6.View ArticleGoogle Scholar
- Hofmeyer A, Scott C, Lagendyk L. Researcher-decision-maker partnerships in health services research: Practical challenges, guiding principles. BMC Health Serv Res. 2012;12:280.View ArticleGoogle Scholar
- Thackway S, Campbell D, Loppacher T. A long-term, strategic approach to evidence generation and knowledge translation in NSW. Australia Public Health Res Pract. 2017;27(1):e2711702.Google Scholar
- Hegger I, Marks LK, Janssen SWJ, Schuit AJ, Keijsers JFM, van Oers HAM. Research for Policy (R4P): development of a reflection tool for researchers to improve knowledge utilization. Implementation Sci. 2016;11:133.View ArticleGoogle Scholar
- Keown K, Van Eerd D, Irvin E. Stakeholder engagement opportunities in systematic reviews: Knowledge transfer for policy and practice. J Contin Educ Health Prof. 2008;28(2):67–72.View ArticleGoogle Scholar
- Golden-Biddle K, Reay T, Petz S, Witt C, Casebeer A, Pablo A, et al. Toward a communicative perspective of collaborating in research: the case of the researcher-decision-maker partnership. J Health Serv Res Policy. 2003;8(Suppl 2):20–5.View ArticleGoogle Scholar
- Johnson R, Grove A, Clarke A. It’s hard to play ball: a qualitative study of knowledge exchange and silo effects in public health. BMC Health Serv Res. 2018;18:1.View ArticleGoogle Scholar
- Cherney A, Head B. Supporting the knowledge-to-action process: a systems-thinking approach. Evid Policy. 2011;7(4):471–88.View ArticleGoogle Scholar
- Huang TTK, Grimm B, Hammond RA. A systems-based typological framework for understanding the sustainability, scalability, and reach of childhood obesity interventions. Children's Health Care. 2011;40(3):253–66.View ArticleGoogle Scholar
- Lavis J, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E. Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy. 2010;10(Supp 1):35–48.Google Scholar
- Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q. 2003;81(2):221–48 171–2.View ArticleGoogle Scholar
- Mirzoev TN, Omar MA, Green AT, Bird PK, Lund C, Ofori-Atta A, et al. Policy partnerships-experiences of the Mental Health and Poverty Project in Ghana, South Africa, Uganda and Zambia. Health Res Policy Syst. 2012;10:30.View ArticleGoogle Scholar
- Jose K, Venn A, Jarman L, Seal J, Teale B, Scott J, et al. Partnering Healthy@Work: an Australian university-government partnership facilitating policy-relevant research. Health Promot Int. 2017;32(6):964–76.PubMedGoogle Scholar
- Choi BCK, Pang T, Lin V, Puska P, Sherman G, Goddard M, et al. Can scientists and policy makers work together? J Epidemiol Community Health. 2005;59(8):632.View ArticleGoogle Scholar
- Kerr EA, Riba M, Udow-Phillips M. Helping health service researchers and policy makers speak the same language. Health Serv Res. 2015;50(1):1–11.View ArticleGoogle Scholar
- Oxman AD, Vandvik PO, Lavis JN, Fretheim A, Lewin S. SUPPORT Tools for evidence-informed health Policymaking (STP) 2: Improving how your organisation supports the use of research evidence to inform policymaking. Health Res Policy Syst. 2009;7(Suppl 1):S2.View ArticleGoogle Scholar
- Thackway S, Campbell D, Loppacher T. A long-term, strategic approach to evidence generation and knowledge translation in NSW, Australia. Public Health Res Practice. 2017;27(1):2711702.View ArticleGoogle Scholar
- Soper B, Hinrichs S, Drabble S, Yaqub O, Marjanovic S, Hanney S, Nolte E. Delivering the Aims of the Collaborations for Leadership in Applied Health Research and Care: Understanding their Strategies and Contributions. Health Services and Delivery Research. Southampton: NIHR Journals Library; 2015.Google Scholar
- Stetler CB, Mittman BS, Francis J. Overview of the VA Quality Enhancement Research Initiative (QUERI) and QUERI theme articles: QUERI Series. Implementation Sci. 2008;3:8.View ArticleGoogle Scholar
- Heller DJ, Hoffman C, Bindman AB. Supporting the needs of state health policy makers through university partnerships. J Health Polit Policy Law. 2014;39(3):667–77.View ArticleGoogle Scholar
- Molleman G, Fransen G. Academic collaborative centres for health promotion in the Netherlands: building bridges between research, policy and practice. Fam Pract. 2012;29(Suppl 1):157–62.View ArticleGoogle Scholar