Co-production practice and future research priorities in United Kingdom-funded applied health research: a scoping review

Abstract

Background

Interest in and use of co-production in healthcare services and research is growing. Previous reviews have summarized co-production approaches in use, collated outcomes and effects of co-production, and focused on replicability and reporting, but none have critically reflected on how co-production in applied health research might be evolving and the implications of this for future research. We conducted this scoping review to systematically map recent literature on co-production in applied health research in the United Kingdom to inform co-production practice and guide future methodological research.

Methods

This scoping review was performed using established methods. We created an evidence map to show the extent and nature of the literature on co-production and applied health research, based on which we described the characteristics of the articles and scope of the literature and summarized conceptualizations of co-production and how it was implemented. We extracted implications for co-production practice or future research and conducted a content analysis of this information to identify lessons for the practice of co-production and themes for future methodological research.

Results

Nineteen articles reporting co-produced complex interventions and 64 reporting co-production in applied health research met the inclusion criteria. Lessons for the practice of co-production and requirements for co-production to become more embedded in organizational structures included (1) the capacity to implement co-produced interventions, (2) the skill set needed for co-production, (3) multiple levels of engagement and negotiation, and (4) funding and institutional arrangements for meaningful co-production. Themes for future research on co-production included (1) who to involve in co-production and how, (2) evaluating outcomes of co-production, (3) the language and practice of co-production, (4) documenting costs and challenges, and (5) vital components or best practice for co-production.

Conclusion

Researchers are operationalizing co-production in various ways, often without the necessary financial and organizational support required and the right conditions for success. We argue for accepting the diversity in approaches to co-production, call on researchers to be clearer in their reporting of these approaches, and make suggestions for what researchers should record. To support co-production of research, changes to entrenched academic and scientific practices are needed.

Protocol registration details: The protocol for the scoping review was registered with protocols.io on 19 October 2021: https://dx.doi.org/10.17504/protocols.io.by7epzje.

Background

Despite the lack of clarity around the definition, what it means in practice and what it comprises, enthusiasm for co-production in healthcare services and research is growing. The lack of clarity is evident in the plethora of terms in use. For example, within healthcare we witness services, programmes and interventions being “co-created”, “co-designed”, “co-evaluated” or “co-implemented”. This can involve stakeholder and public engagement through participation or involvement in any or all steps of the applied research cycle [1, 2]. All are regarded as processes of co-production, but the way they are enacted and operationalized varies depending on the purpose, what is being co-produced and by whom [3, 4]. Some of the ambiguity in co-production also comes from its unclear relationship with patient and public involvement/and engagement (PPI/E). For some, co-production represents enhanced PPI/E, a way to improve on its shortcomings by re-engaging with the principles of power-sharing, equality and social justice, and reinforcing the democratic right of citizens to influence healthcare [3, 5]. For others, co-production simply represents another way of consulting the public and service users to provide instrumental inputs into health and social care services and research, demonstrating a more technocratic rationale [6]. New experimental perspectives on co-production, which frame it as a generative process and a social space within which new interactions, insights and knowledge are produced, challenge conventional notions of engagement and involvement [4]. However, whilst new conceptualizations and discussion can help the approach and foundational principles to further develop and evolve, and more and different forms of co-production to emerge, this also adds to the uncertainty around its use.

The United Kingdom National Institute for Health Research (NIHR) recently embraced co-production as a means of improving public involvement in research, framing it as a more collaborative and egalitarian mode of involvement with values and principles for greater equality [7]. Unlike other funders of health research globally, NIHR insists on community involvement in research proposals, and it is a key criterion for funding [8]. Other funders have started to encourage co-production by providing flexible funding to cover costs of user-led research design and engagement [9] and funding research into best practice for community engagement [10]. In the United Kingdom context, some argue that the architecture of the new NIHR Applied Research Collaboration funding model enables authentic and visible co-production [11]. Others are more cautious, arguing that co-production can only be as successful as the system allows, and that traditional research structures often fail to facilitate effective public involvement, leading to co-opting of the term co-production without making a tangible difference [12, 13]. However, there are anecdotal stories of successful collaborative working from the previous NIHR funding model, Collaborations for Leadership in Applied Health Research and Care (CLAHRC), where co-production projects added value and led to the implementation of novel services and interventions [14]. Success stories like these are not always published or reported on or described in a way that explicates how best to support researchers to co-produce applied health research or complex health interventions.

Recent systematic reviews of co-production have summarized the different co-production approaches in use and collated outcomes and effects of co-production, and some have focused specifically on replicability and reporting. Slattery et al. conducted a rapid overview of reviews, specifically of research co-design (defined as involvement of research users at the study planning phase only) and its effectiveness, and found that co-design is widely used but rarely reported or evaluated in detail [15]. Another review examining the use of experience-based co-design (EBCD) in health service improvement also found inconsistent reporting and variation in the use of the approach, leading the authors to argue for reporting guidelines to encourage consistency and to improve the potential of the approach [13]. Halvorsrud pooled effects data from co-creation projects in international health research and found moderate to small effects on a range of outcomes from different study designs and interventions, yet little evidence of longer-term effects of co-creation [16]. Acknowledging the lack of evidence of the impact of co-produced or co-created interventions in healthcare settings, some authors have reviewed the evidence on outcomes and factors influencing the quality and level of co-production and co-creation [17, 18]. These reviews found that studies of processes and factors influencing co-production dominated, and identified fewer studies evaluating clinical, service or cost outcomes.

While various aspects of co-production have been subject to more or less rigorous systematic reviews in the last 5 years, no reviews have targeted co-produced applied health research or the co-production of complex interventions (which is often the focus of applied research). Nor have previous reviews critically reflected on how co-production is conceptualized in applied health research, or how the principles are enacted, to draw out implications for the practice of co-production and for future research. Applied health research is becoming more collaborative, with patient and public groups increasingly engaged in research projects alongside academics and practitioners, and funders are gradually mandating the use of co-production principles. It is therefore timely to reflect on what has been learned about the practice of co-production in applied health research and help forecast the direction of future research.

We conducted a scoping review to systematically map recent literature on co-production in applied health research in the United Kingdom to inform co-production practice and guide future methodological research. The review was designed to answer the following questions:

• (1) What is the type and scope of literature on co-production in applied health research?

• (2) How is co-production conceptualized and understood?

• (3) How is co-production implemented in applied health research?

• (4) What lessons are there for co-production practice and future research, based on the current knowledge base?

Methods

We used established scoping review methods to systematically map the nature of the evidence, summarize practice, and identify gaps in the literature on co-production in applied health research [19, 20]. We had to streamline our approach to the study screening and selection process because of time and resource availability, and therefore followed accepted rapid review methods for single screening of titles and abstracts and independent verification of a sample of full-text articles [20]. We intentionally kept the review questions broad and open to generate breadth of coverage, and once we had a sense of the volume of literature, we set parameters to limit the number of studies to a manageable level. The protocol is published on protocols.io.

We define co-production as a way for academics, practitioners, and patients and the public to work together, sharing power and responsibility across the whole research cycle [7]. For the purpose of this scoping review, we have assumed that co-production happens at any or all stages of the research cycle, and so included reports using any of the plethora of terms in use including co-design, co-production, co-implementation, co-evaluation and co-creation.

Search strategy

We followed a standard approach to locate published literature in scoping reviews [21]. First, we listed key terms and synonyms relevant to each of the inclusion criteria (Table 1) and performed an initial high-level search of one relevant multidisciplinary database (ProQuest) using main keywords in the title. We analysed the text words used in the retrieved article titles and abstracts, then conducted a comprehensive search of five other relevant databases (CINAHL, Google Scholar, MEDLINE, Scopus, Web of Science) using all identified keywords and index terms. We conducted a separate search to ensure we identified co-production of complex health interventions as well as the broader applied health research literature. The third step involved searching all reference lists of retrieved articles to identify additional literature. An example search strategy can be found in Additional file 1. We downloaded all retrieved articles and managed the screening process in Mendeley.

Table 1 Scoping review inclusion criteria

Study selection

We included any type of published literature (empirical research, reviews, guidelines, opinion pieces or commentaries) relevant to co-production in applied health research or complex intervention development that reported on a range of outcomes including conceptual, methodological, impact or health. We were interested in literature that included definitions or conceptualizations of co-production, as well as implications for future research. We intentionally included only papers reporting applied health research conducted in the United Kingdom—to keep the focus on learning within a specific context. Following the initial searches and familiarity with the extent of the literature, we refined our inclusion criteria. Our initial database searches included papers published from 2010 onwards, when “co-production” began to appear in the health literature and as a requirement of some funding schemes in the United Kingdom; we subsequently limited the date range to 2018–2020 due to the large number of hits and to keep the charting and summarizing steps manageable.

Based on established rapid review methods [20], one author (HS) applied the inclusion criteria to all titles and abstracts retrieved in the search. After excluding articles that did not meet the criteria, we retrieved full text copies of all remaining articles. One author screened these for inclusion (HS), and another author (LB) independently screened 25% of articles; discrepancies in include or exclude decisions were resolved by discussion.

Data extraction

We used a Microsoft Excel worksheet to chart the characteristics and record key information from the articles included in the review (e.g. author, year of publication, study design, health speciality, aim, intervention type, outcomes reported, implications for practice and research). The items and information to be collected from each article were piloted by two team members, and adjustments made to ensure it was fit for purpose and standard information could be extracted in the same way for each article. Charting was completed by three authors (CG, IH, AH) and an independent check of 25% of the articles was done by another author (HS).

Summarizing and reporting the findings

We used a descriptive-analytical method using the charted information as an overall framework for reporting across all included articles [19]. The resulting chart or evidence map shows the extent and nature of the literature on co-production and applied health research. Based on this map we developed a narrative summary, first describing the characteristics of the articles and scope of the literature (type, study design, health speciality, key outcomes reported), followed by a summary of conceptualizations of co-production and how co-production was implemented, as described in the articles. We extracted from the discussion section of each study any mention of implications for co-production practice or future research and conducted a content analysis of this information to identify lessons for the practice of co-production and themes for future methodological research. Reporting of the findings follows the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) format [22].

Findings

Description of included studies

Database searching identified 793 records on co-production and applied health research and 225 on co-produced complex interventions (after limiting the search to 2018–2020). After removal of duplicates, there were 576 records on co-production and applied health research and 93 on complex interventions, of which we reviewed the full texts of 74 and 27, respectively. We excluded articles if they did not report on co-production, were not conducted in or relevant to the United Kingdom context, or were unpublished reports (Fig. 1). After including additional relevant articles identified from reference lists, n = 19 articles reporting co-produced complex interventions and n = 64 reporting co-production in applied health research met the inclusion criteria and were included in the scoping review.

Fig. 1.

Adapted Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram of the search strategy

Scope of literature on co-production in applied health research

Table 2 summarizes the key characteristics of studies included in the scoping review. Nineteen reported co-produced complex interventions (N = 19) including intervention development or evaluation studies (n = 10), systematic reviews or evidence reviews (n = 3) and critical reflections or opinions (n = 6). The intervention studies mainly used descriptive study designs, including mixed-method observational studies that described the development of co-produced interventions or qualitative research that reported the process of co-producing an intervention and/or stakeholder views on the process. The systematic reviews or rapid evidence reviews synthesized empirical evaluations, processes and outcomes of co-production, and the critical reflections or opinion pieces described author experiences of co-producing interventions, or provided interpretations and conceptualizations of co-production.

Table 2 Characteristics of included studies

Papers reporting co-production in applied health research (N = 64) included intervention development or evaluation studies (n = 34), systematic, scoping or rapid evidence reviews or literature reviews (n = 10) and critical reflection or opinion pieces (n = 20). Most studies describing intervention development were qualitative and concerned co-designing or co-producing research methods or tools, or exploring the feasibility or acceptability of co-produced knowledge or service improvements. Evaluations reported on the mechanisms, approaches and forms of co-produced research projects, or measured impact or effects of co-produced interventions or projects. The systematic, scoping and rapid evidence reviews summarized best practice, definitions, implementation and sustainability, reporting and effects of co-produced research. The opinion and reflection papers tended to summarize historical or theoretical perspectives on co-production or user involvement in research, as well as outlining current thinking, literature and debates relating to co-designed or co-produced research, while others offered opinion on how to realize co-production and tips for effective co-production of services and research.

The included studies represent a broad spectrum of health specialities or disciplines. For those reporting co-produced complex interventions, many of the reviews and opinion pieces related to health services research or biomedical research, while the intervention development studies were situated in public health (n = 2), acute and intensive care (n = 1), addiction and substance misuse (n = 1), older people (n = 1), chronic illness (n = 1), primary care (n = 1), social care (n = 1), maternal health (n = 1) and mental health (n = 1). The studies reporting co-production in applied health research were related to health services research (n = 21), or were conducted within specific specialities such as mental health (n = 19), public health (n = 7), health and social care (n = 4), older people (n = 3), critical or acute care (n = 2), health inequalities (n = 2), oral health (n = 2), primacy care (n = 1), HIV/AIDS (n = 1), chronic illness (n = 1) or adolescent health (n = 1).

A range of outcomes were reported across all studies including conceptual (e.g. defining or explaining some aspect of co-production), methodological (e.g. focused on the process of designing or carry out co-production), impact (e.g. challenges, barriers and facilitators of co-production, acceptability, cost or effectiveness of co-produced research) and health (e.g. impact of co-produced interventions on health outcomes). Many studies resulted in tangible outputs or products including toolkits, models, frameworks or principles (see Table 2). Five studies concerned with applied health research described co-production as a means for “knowledge mobilization” or “knowledge transfer”, including co-produced dissemination activities [53], public engagement for better understanding of health topics [59], co-production for facilitating research implementation [87], use of co-design for knowledge mobilization [89] and co-creation of public health evidence [97].

Overall, nine studies (47%) reporting on co-produced complex interventions and 12 (19%) of those reporting co-production in applied health research were funded or supported by the NIHR. Other funding sources for studies of co-produced complex interventions included PhD studentships or fellowships (n = 3), National Health Service (NHS) Trusts (n = 1), Medical Research Council (n = 2) or National Science Foundation (n = 1), or the funding source was not reported (n = 3). Funders of co-production in applied health research included Wellcome (n = 4), charities (n = 4), Scotland or Ireland health boards (n = 4), European Union or Erasmus+ (n = 4), United Kingdom Research Councils (n = 3), university/Grand Challenges (n = 2), other government funding (n = 2) and single-study funding by an NHS Trust, an Academic Health Science Network (AHSN), Public Health England or private/commercial funding. Five applied health research studies did not receive funding and 23 did not report the funding source.

In most studies reporting on the development or evaluation of co-produced interventions, the lead organizations were universities (8/10 complex interventions and 27/34 co-produced applied interventions). Very few were led or co-led by NHS Trusts (2/10 complex interventions and 4/34 applied interventions) [23, 29, 43, 46, 47, 55], one study was led by a community organization [26], and two co-produced applied interventions were led by independent service users or service user charities [48, 50].

Conceptualization and implementation of co-production

Fifty-five papers referred to co-production either independently or in conjunction with other terms such as PPI/E, co-creation or co-design (see Additional file 2). Twenty-three papers were concerned with either co-design or EBCD, 12 used the term co-creation, and 10 mentioned PPI/E. Sixty-eight papers reported their research as a single methodology (e.g. co-production, co-design, EBCD or co-creation), with the remaining 16 using a combination of these terms to describe their work (e.g. co-production/co-creation, co-production/PPI, co-production/co-design or co-creation/co-design).

Some papers were very explicit in the definition of their chosen term, whereas others opted to describe the term using references from pre-existing literature. A commonly referred to definition was that of PPI as defined by INVOLVE, a national advisory group for PPI: research being carried out “with” or “by” members of the public rather than “to”, “about” or “for” them” [100]. In some instances a distinction was made between PPI/E and other co-activities based on the level of “active involvement” or the presence of a shared-power dynamic, with PPI/E being seen as a more passive or advisory role with a lower share of power and control [34, 88, 95]. A number of authors, however, seemed to use the two terms interchangeably [53, 75, 81].

Co-production was the most widely used term, referring to both the co-production of research and the co-production of services. The concept of shared power was widely used when describing co-production and what it means to health research or service development [42, 52, 101]. In these definitions, and indeed many others, co-production implied the involvement of a variety of stakeholder groups (e.g. services users, charity representatives, healthcare professionals and academics) in multiple stages of the research process. Others, however, used co-production as an umbrella term encompassing all aspects of additional stakeholder involvement whether that be throughout the process or in a single stage of the research cycle. In their rapid review evaluating hospital tools and services that had been co-produced with patients, Lim et al. included “co-production (e.g. co-production, co-design… [and] co-creation)” in their search terms [32], which highlights its use as a catch-all term.

Co-design was usually used to refer to stakeholder involvement in the design process of user-friendly tools, interventions or initiatives. Emphasis was placed on the value that “experts by experience” (e.g. patients, services users or clinicians) can bring to the design process as equal partners, beyond user involvement or consultation [102]. Stakeholder groups involved in these co-design projects included patients, carers, healthcare professionals, service users, local people and software or technology developers [13, 43, 46, 49, 51, 54, 61, 103]. Another frequently mentioned term was EBCD, which was defined by Chisholm as “a service design strategy that facilitates collaborative work between professional staff and service users toward common goals” in every stage of the design process [46]. EBCD appears to be more often applied to service development, while co-design is more often referred to in research.

Where it was possible to discern how the concepts were enacted, the type of methods reported in papers describing co-production or co-design included individual interviews, group workshops, reflection and discussion meetings, focus group discussions, social media forums, surveys, or a mix of these activities [47, 48, 52, 55, 62, 63, 84] (see Table 2). While some papers described specific activities and participatory approaches used in co-design or co-production workshops or meetings [25, 30, 70, 72, 104], most did not elaborate on their methods. In studies describing intervention development or evaluation, we looked for reference to principles of co-production or co-design (defined by the NIHR¹) and how they were enacted. Of the studies reporting development of complex interventions (n = 10) and studies reporting development of applied interventions (n = 34), the principles described most frequently as key features of the projects were “including all perspectives”, “respecting and valuing all contributions”, “joint decision-making” and “involvement of stakeholders at all project stages”. Very few papers referred explicitly to “sharing of power” among stakeholders, or the principles of “reciprocity” and “building and maintaining relationships”. Fourteen of the applied research intervention studies did not mention co-production principles at all, and seven stated that co-production or co-design principles or approaches were used or agreed on, but specific features were not described (see Table 2).

Most studies reported on the stakeholder groups involved in co-producing interventions (Table 2), and most often these were combinations of academics/researchers, patients/carers/service users and health professionals/practitioners. Where project specialties or specific focus dictated, family members, friends, community members and representatives of other organizations were included as stakeholders. The studies did not report on stakeholder criteria, how stakeholders were chosen or the qualifications required to participate in co-production. A number of papers described having involved stakeholders as early as possible in the research process, with some even initiating patient involvement before drafting their funding application [36].

Co-creation and co-production appear to have considerable crossover in the literature, with similar definitions being used for both terms. In one report, Connelly et al. agreed that their review of the literature found that “co-production and co-creation are largely very similar”, but thought that co-creation represented a more prolonged involvement of stakeholders “throughout the process of programme design, development, implementation and evaluation (not just at the programme development and design stages)”. A similar idea was shared by Hannigan, who used co-creation to describe the involvement of additional stakeholders in all aspects of the project, namely “co-design of the research protocol, project governance, collaborative data interpretation and disseminating findings” [58]. As with co-production, co-creation has also been used as an umbrella term to describe the involvement of wider stakeholders in healthcare research [16].

Lessons for the practice of co-production

Content analysis of information extracted from the papers revealed several recommendations for the practice of co-production and more strategic requirements for co-production to become more established and embedded in organizational structures: (1) capacity to implement co-produced interventions, (2) skill set needed for co-production, (3) multiple levels of engagement and negotiation, and (4) funding and institutional arrangements for meaningful co-production.

Capacity to implement co-produced interventions

Papers concerned with co-production of applied health research emphasized building capacity to adapt and absorb the changes brought about through co-production and co-creation, and fostering a cadre of implementation-savvy researchers who can “do implementation” was considered vital [80, 86, 87]. Others highlighted middle managers as “critical catalysts” for strategic and operational impact of co-produced interventions, and frontline staff as key enablers or “active agents” of change [29].

Skill set needed for co-production

The papers also emphasized the skills required of researchers for co-production, which authors felt lay outside the typical academic or researcher skill set [29]—for example, collaborating with diverse stakeholders, negotiation skills [90], good persuasive communication [29, 46], managing expectations [29], prolonged involvement with service users and other stakeholders, and flexibility in maintaining relationships [30, 104] and showcasing outcomes [46]. One paper recognized the difficulty of doing co-production as a mandated activity and acknowledged that not everything has to be co-designed and not everyone will want to occupy this space [35].

Multiple levels of engagement and group negotiation

Many papers reporting co-produced complex interventions highlighted the importance of multiple levels of engagement (patient or user, practitioner or provider, and policy-makers) as well as multiple levels of experience and values (individual, family, organizational, cultural, political) for a truly participatory process [24, 25, 40, 84, 102]. This was thought to require leadership and careful negotiation of group politics for meaningful and productive discussions [24, 25], a balance of experienced and new co-production contributors and clear boundaries for involvement [84], as well as the ability to navigate the different types of knowledge, experience, research literacy, priorities and perspectives of diverse stakeholder groups involved [40].

With various levels and types of engagement, the need to resolve (inevitable) disagreements was mentioned in several papers. Some highlighted that when disagreement arises between stakeholders and researchers, or when there is conflict between service users, practitioners and organizational perspectives [49], discussion of differences should be encouraged and not regarded as a threat [24]. Similarly, allowing stakeholders to challenge researchers’ intentions and assumptions was encouraged and regarded as beneficial [52]. However, it was recognized that patients and other groups rarely feel able to challenge the hegemony, and this has implications for carrying out co-production and collaborative work with patient groups [61]. Others advocated for early consultation between all stakeholders to mitigate disagreements in relation to the aim, direction and outcomes of co-produced research [26].

Funding and institutional arrangements for meaningful co-production

Several papers mentioned that meaningful co-production required certain funding and institutional arrangements [36, 65, 76, 81, 84]. The main concerns were the need for sufficient funding to cover planned co-production activities [84] and to adequately resource co-production to ensure inclusivity, diversity and equality [81]. Others suggested that current funding systems prevent meaningful co-production in the planning stages of research [36] and that there is often insufficient time to establish relationships with patients and other groups [65]. Some papers questioned whether existing institutional arrangements could support the ideals of co-production and manage the tensions that arise [36]. It was also recognized that effective co-production requires changes in academic institutions and scientific practice, specifically to embrace more equal power distribution in the research process and ensure proper governance for co-production [60]. Others called for leadership for co-production to be more embedded in local health and social care organizations [76].

Themes for future research on co-production

Most of the papers reporting co-produced complex health interventions (16/19), and those relating to co-produced applied health research (50/64), reported research implications of their work, or suggested future directions for research on co-production. Content analysis of information extracted from included studies led to the following themes for future research: (1) who to involve in co-production and how; (2) evaluating outcomes of co-production; (3) the language and practice of co-production; (4) documenting costs and challenges; (5) vital components for co-production.

Who to involve in co-production and how

Many of the papers concerned with co-produced applied health research highlighted the need for future research to focus on better ways of involving more diverse groups of service users and stakeholders [31, 63, 68,69,70,71,72, 80, 95, 98]. There were suggestions for research to focus on understanding how to involve different groups and facilitate effective involvement [54, 77]. For example, it remains unknown whether planned activities [99] or less structure [72] allows for better involvement and more successful adoption of co-produced interventions. One questioned whether more extensive input from multiple stakeholders equates to more effective interventions [31].

Other papers relating to co-produced applied research suggested that future research should focus on identifying how best to recruit and involve people in co-production or co-creation [75, 91], specifically to identify which engagement strategies work best and whether different levels or types of engagement suited particular stakeholder groups. Research on how to build trust was also thought to be a priority, since it is the foundation for successful partnerships and co-produced interventions that are more likely to meet needs [91].

Evaluating outcomes of co-production

There was considerable discussion in the included papers on complex interventions about evaluating the impact of co-production and co-produced interventions on patient and provider outcomes, and the dominance of certain study designs [30, 32,33,34, 37, 104]. Concerns included the dominance of qualitative, case study and mixed-method research for evaluation of co-production [33, 34], known for small sample sizes, recruitment bias and weak designs yet strong claims about the effects of co-production [34]. Several authors stated that understanding the causal effects of co-production and disentangling the effects of participant involvement from the effects of co-produced interventions required quantitative techniques including randomized evaluations to promote confidence in causal relationships [30, 34]. Authors also highlighted that outcome evaluations of co-production tend to report positive impacts [34] and moderate to high acceptability, usability and uptake [32], yet the evidence for the effect of co-produced interventions on organizational, patient and health service provider outcomes is limited and the value added uncertain [32]. There is an abundance of research on factors affecting the success of co-production and participation in and experience of co-production, but some suggest not enough focus on broader outcomes (for example, whether co-production is empowering to individuals, increases acceptance of co-production among professionals and policy-makers or increases demand among service users for co-production and co-creation approaches) that really matter to patients and providers [34]. Indeed, some of the reported positive impacts may reflect possible negative outcomes for individuals involved in terms of efficiency and other costs that are rarely reported, for example feelings of pressure and frustration among those taking part in co-production (despite it being empowering) and a lack of time to implement co-production (even though there may be increased appetite for it) [34]. The real costs and benefits and how co-production could be used to produce better outcomes, more efficiently and at less cost are under-researched [34, 37].

Papers reporting applied health research contained much the same discussion, centred on the need for more rigorous evaluation of co-production and its impact on quality, implementation and outcomes of research [31, 45, 59, 66, 85, 90, 91, 95], including longer-term effects on health outcomes derived from co-produced interventions [15, 16, 55] and more “quantitative” research, especially inclusion of comparators as a minimum requirement [45]. One paper specifically mentioned the lack of evidence that co-produced services have led to improved satisfaction or resulted in better quality of care for end users, mainly due to the use of non-comparative study designs [85].

The language and practice of co-production

A few papers reporting complex health interventions questioned whether the increased traction of co-production in academic and policy debates had changed attitudes and practice around PPI to produce more authentic collaboration, or whether tokenism persisted [35]. Adaptations to “traditional” co-production approaches were advocated in the complex intervention studies—for example, feasible and acceptable enhancements to the EBCD approach [38]. Others commented that research is needed to compare different co-production approaches, to identify which ways of working for complex health intervention development are likely to maximize creativity and lead to health gains [39].

Many papers reporting applied health research similarly commented on the discrepancy between the language and practice of co-production. Some highlighted the rhetoric and concerns about tokenism of co-production [52, 80]. There were calls to improve and deepen “true” co-production [52] and to develop measures of co-creation in research to indicate the extent to which researchers use the methods they claim to [78]. Comments were also made about the variety of co-production approaches or strategies in use and the adaptation of co-production to different people and contexts [15, 85]. There were also concerns about the diversity of practice and adaptation of co-production and a feeling that more research is needed to understand what conditions of co-production contribute to evidence use and improved outcomes [91], what constitutes effective co-production in which circumstances [80, 85], and at which stages of research co-production is appropriate and useful [15, 90].

Documenting costs, challenges, barriers and facilitators

Studies reporting complex interventions reflected on challenges and barriers to co-production [25, 26, 34, 36, 37] and the need for further research to focus on facilitators as well as challenge in co-production. Two studies highlighted facilitators including the use of a “needs analysis” at the first meeting, open questions and subgroups, multidisciplinary debate and a problem-solving approach [25] and a need to identify champions and allies to gain entry and engage stakeholders [29]. There was a view that future research needs to be open to “what doesn’t work” and should be developed based on the learning from previous work [29, 36].

The applied research papers also suggested the need for more open identification, discussion and resolution of challenges in co-produced research [52, 61, 74, 87]. Challenges included inevitable disagreements between researchers and service users [52], and situational and organizational barriers [61] and tensions specifically related to co-production and policy-making (e.g. resolving power issues, high involvement costs and providing incentives to stakeholders to secure buy-in) [87].

Vital components for co-production

The included studies concerning co-produced complex interventions pointed towards the need for future work to document and facilitate sharing of best practice. Some suggested that future research should help to understand how co-creation and co-production work in practice [33], and should clarify the concepts and processes to better operationalize co-creation and co-production [34]. Others suggested research to compare different approaches to intervention design to uncover which work best for co-production of complex intervention development and which are most likely to lead to health gains. They also called for future research to clearly and consistently report the methods used [39]. Two studies suggested it was important for future research to document and share best practices in co-production, detailing the components that are vital to the process [26, 37]. One suggested that best practice may only be determined by studying “live examples” of effective co-production strategies [32].

Similar narratives were evident in applied health research papers, with calls for future research to report contextual details that inform the selection of co-production approaches as well as better reporting of activities and processes involved in co-production [15]. Other papers suggested research to explore the mechanisms for optimum success in co-creation, and to determine what factors affect success [16] and which co-production activities are best suited for which research projects or health and social care programmes [85]. One paper reported the development of a framework of principles to facilitate co-creation of local public health interventions [60] and another outlined procedural steps for implementing co-production in mental health services [92].

Discussion

This scoping review has mapped out the recent literature on co-production in applied health research in the United Kingdom and offers an interpretation of how co-production is being practised and what methodological research questions remain. Co-produced complex interventions were evident across a range of health specialities, from acute and intensive care to public health and surgery. In applied health research, co-production was most apparent in mental health research. The majority of the empirical research we found used observational methods to describe co-production processes and mechanisms or qualitative research to explore stakeholder experiences and perspectives on how co-production can be applied in research or service improvements. The literature also appears to be dominated by commentary or opinion pieces that describe author experiences of co-producing interventions or offer historical or theoretical perspectives of user involvement in research. We found very few empirical studies of the impact or effect of co-produced complex interventions or knowledge within applied health research.

Implications for co-production practice

The included studies make an important point about co-production ideally involving multiple stakeholders with multiple levels of experience and understanding and of differing values. Because of this rich and diverse participation, highly developed leadership and negotiation skills are often cited as requirements for meaningful co-production. However, this should not imply that one group holds power to facilitate productive discussion or resolve disagreements: power-sharing and managing conflict in co-production is a joint responsibility. Safe space for stakeholders to challenge each other and where all groups feel able to discuss their differences is important. Yet this space should not become so comfortable that it breeds homogeneity, because it is the very diversity of views, experiences, skills and knowledge and the equal importance of all contributions that co-production strives to harness.

That co-produced research requires adequate funding and certain institutional arrangements is an unsurprising finding. There is no doubt, as others have also suggested, that more and reliable funding could overcome some of the barriers [13], such as resourcing activities in the planning stages of research [15] and having enough resources to ensure inclusivity and reciprocity throughout the process [15, 16]. However, our review identified entrenched academic and scientific practices as a potentially greater impediment to progress in co-produced applied health research. Changes required at the individual researcher level such as embracing more equal power-sharing, refining negotiation and communication skills, and managing stakeholder relationships can be developed, but organizational changes such as proper governance and research policies that enable co-production take longer.

Research implications and gaps in knowledge

The included studies highlighted a need to identify better ways to recruit stakeholders and to facilitate more effective involvement in research, including ways to involve more diverse groups. In applied health research, there are undoubtedly various modes of engagement being tried and tested but probably not reported on. There is clearly a desire to learn from successful projects and teams about specific methods for engagement and flexibility in approaches including whether structured activities or more extensive input from multiple stakeholders makes for better involvement and ultimately better interventions [31, 72, 99]. The research community could do better in terms of reporting this learning. Being explicit about who or what constitutes a stakeholder group is key to identifying how best to involve and collaborate with different groups [3].

The need for more rigorous research on the benefits of co-produced research, the added value of co-produced interventions and their effects on quality of care and satisfaction with services was a dominant finding. Producing this kind of evidence is difficult to do; co-produced processes and outcomes are often context-specific. However, a recently published review that pooled evidence from reviews and primary studies of co-creation from the international healthcare literature found moderate positive effects on immediate health-related outcomes including health service access and health-promoting behaviour, but less evidence on long-term effects [16]. The rather technocratic focus on “more evidence” of effects and impact ignores the democratic rationale for co-production—that it is the right thing to do in principle regardless of the outcomes. Many argue that the endeavour of co-production itself is sufficient to achieve the end goal of collaboration to realize outcomes that would not happen otherwise [3], and quantitative or experimental research to justify its value is unnecessary. Perhaps there is value in regarding co-production as an exploratory “social space” and a generative process rather than a means to deliver impact and outputs [4]. We agree, though, that capturing evidence of the “value” of co-production to participants—in relation to broader outcomes such as equality in power in the research process, empowerment and new skills developed—would be useful, not least to research funders who now expect research to be “co-produced”, as would involving health economists in more creative ways of estimating the cost and cost-effectiveness of co-produced research [34, 37].

Findings relating to the language and practice of co-production in applied health research highlighted the variety in the application of co-production and deviation from “traditional” approaches. Concerns were raised not only about these adaptations, but also about the persistence of tokenistic co-production. We do not think a race to demonstrate what constitutes “true” co-production and authentic co-produced research is what is needed. Rather, we think it is important to be mindful of how divisive this narrative can be. While it is good to aspire to the ideal and hold up the “gold standard” in co-production, the commitment in terms of time, resources or perceived expertise can make this feel unachievable and can put people off [12]. We argue for acceptance of a diversity of approaches to co-production that allows more researchers and others to “give it a go” and learn by doing. We would suggest that “pragmatic” decisions made to tailor co-production to specific project circumstances are transparently reported, acknowledging where compromises to ideal co-production are made, and why.

The included studies reported many challenges to co-production—for service users, researchers and organizations, and in relation to the practical “doing” of co-production. To resolve these challenges, one could argue that the publication of unsuccessful cases and reflective pieces that tease out lessons learned would be most helpful; but this is generally not an ambition of researchers or publishers, so these examples remain hidden. Failure drives learning and the greatest learning happens when things go wrong, and some of the studies seemed to support this view [29, 36, 52, 61, 87]. Perhaps new journals coming online such as Research Involvement and Engagement, which is co-produced by patients, academics, policy-makers and service users, will lead the way.

Perhaps the most important finding was the evident call for practitioners to share experiences of co-production in practice, to help others better operationalize the principles. There was demand for reports of co-produced research to elaborate context and help determine which co-production activities are best suited to which projects or which projects and interventions are likely to best suit co-production. Some authors argued for procedural steps or principles for co-production. However, we believe there may already be too much guidance and prescription, and instead the applied health research community needs practical and financial support to enact and operationalize co-production. On a practical level, co-production needs to match the context, actors and purpose of new projects, and researchers need to organize structural, personal and organizational factors to set up the right conditions from the start. Financial support is required to fund people with the skills to carry out co-production and time for people to accomplish co-production over the long term. Most current funding models support the practical conduct of research but fall short of investing in relationship- and network-building over time. A democratically driven vision of producing research with patients, the public and other stakeholders across projects and over time will only become a reality with a commitment to fund and support it.

Limitations of the review

We intentionally included only papers reporting applied health research conducted in the United Kingdom—partly to keep the review manageable (the literature on co-production is extensive) and partly to keep the focus on learning within a specific context. A large proportion of the applied research conducted in the United Kingdom is funded by the NIHR, which directly reflects United Kingdom government and policy priorities; research groups often secure repeated funding for programmes, some of which have now spanned more than a decade. We felt it was timely to learn about co-production practice within these groups and identify priorities for future research funded within similar schemes and infrastructure. We acknowledge that the themes identified in the scoping review may not be generalizable to applied research conducted in other countries or under other funding arrangements. Our initial database searches included papers published from 2010 onwards, when “co-production” began to appear in the health literature and as a requirement of some funding schemes in the United Kingdom; we subsequently limited the date range to 2018–2020 due to the large number of hits and to keep the data extraction and synthesis manageable. We recognize that we may have missed important work that could contribute to our findings. However, this is a scoping review, conducted to rapidly map the recent literature, and not an exhaustive systematic review. We only had one author screening titles and abstracts, and independent screening of 25% of full texts due to researcher time and resource availability; we received no additional funding to conduct the review. We acknowledge that there is a small risk of selection bias through exclusion of eligible studies.

Conclusion

This scoping review provides ample evidence that complex health interventions, service improvements and applied research are being co-designed and co-produced with patients, the public and other stakeholders, and supports current knowledge about the diverse processes and formats of co-production. However, what is clear from this review is that researchers are operationalizing co-production in various ways, often without the necessary financial and organizational support required and the right conditions for success.

Instead of trying to define a gold standard in co-production, we argue for accepting the diversity in approaches to co-production and call on researchers to be clearer in their reporting. Different approaches are needed to tailor co-production to context, different stakeholder groups and various stages of the research and implementation process. To assess which approaches are best suited in which context, for which groups and at what stage, researchers should be more reflective on the use of their chosen approaches in practice and be more systematic in reporting their learning (including failures) to allow for better operationalization of co-production principles and guard against tokenistic use of the term “co-production”.

As a minimum, researchers should record:

• a description of activities they undertake as part of co-production;

• which stakeholders were involved in this process and in what way ways, with a particular emphasis on how power is shared between stakeholders;

• the stages of the research and implementation process these stakeholders were involved in;

• skills that were developed by participants (including researchers); and

• the desired and achieved outcomes of these activities and the methods used to assess these outcomes.

Instead of being overly prescriptive about these different reporting elements, we argue that there is value in regarding co-production as an exploratory “social space” and a generative process, rather than a means to deliver impact and outputs, in order to encourage people to “give it a go” and learn by doing. However, without adequate resources and institutional support for people to work co-productively across projects and over time, the key principles become harder to enact, and innovation and creativity in collaboration and involvement in research is likely to be stifled. Entrenched academic and scientific practices are an apparent impediment to progress in co-producing applied health research. Changes are required at the individual researcher level, such as embracing more equal power-sharing, accepting different forms of evidence production, refining negotiation and communication skills and managing stakeholder relationships. Organizational changes such as proper governance and research policies that enable co-production are likely to take longer. As a result, the democratically driven vision for co-production—that it is the right thing to do and the endeavour in itself is sufficient to realize outcomes—which so many funders and commissioners of applied health research aspire to, and many researchers believe in, remains out of reach. In spite of these challenges, our scoping review has highlighted different ways of realizing co-production in applied health research, which can be further refined and researched within current research infrastructures such as the NIHR Applied Research Collaborations.