- Open Access
Public involvement in health research systems: a governance framework
Health Research Policy and Systems volume 16, Article number: 79 (2018)
Growing interest in public involvement in health research has led to organisational and policy change. Additionally, an emerging body of policy-oriented scholarship has begun to identify the organisational and network arrangements that shape public involvement activity. Such developments suggest the need to clearly conceptualise and characterise public involvement in health research in terms of governance.
We drew on an established health research system framework to analyse governance functions related to public involvement, adapting scoping review methods to identify evidence from a corpus of journal papers and policy reports. We drew on the logics of aggregation and top down configuration, using a qualitative interpretive approach to combine and link findings from different studies into framework categories.
We identified a total of 32 scholarly papers and 13 policy reports (n = 45 included papers) with relevance to governance for public involvement. Included papers were broadly consonant in identifying the need for activity to specify and support public involvement across all four governance functions of stewardship, financing, creating and sustaining resources, and research production and use. However, different visions for public involvement, and the activity required to implement it and achieve impact, were particularly evident with respect to the stewardship function, which seeks to set overall directions for research while addressing the potentially competing demands of a system’s many constituents.
A governance perspective has considerable value for public involvement in health research systems, supporting efforts to coordinate and institutionalise the burgeoning public involvement enterprise. Furthermore, it highlights challenges for what is, ultimately, a highly political intervention, suggesting that diverse publics must be both involved within health research systems and enrolled as governors of them.
Public involvement in health research is increasingly seen as essential to the legitimacy, relevance and quality of the research enterprise, enabling research to better account for the needs of service users and caregivers, to respond to the demands of the lay communities affected by research practices and results, and to respect the imperatives of democratic accountability in service of the public interest [1,2,3,4]. Accordingly, there has been a marked growth of public involvement activity, with increased reliance on representatives of diverse publics as advisors to, or investigators within, individual research projects [3,4,5,6], and growing interest in involving public members in advising on and setting priorities for research funding . This has been accompanied by an explosive growth of scholarship reporting on the rationales, methods and impacts of public involvement activity [8,9,10,11,12].
While much of this effort has been motivated by individual researchers and members of the public, it has also been conditioned by policy effort. Public sector authorities in many countries, including funding agencies and oversight bodies, are increasingly seeking to encourage, coordinate or evaluate such initiatives [13, 14]. Organisations with the mandate to facilitate public engagement have been established and public involvement policies increasingly inform the efforts of research producers, including individual researchers as well as producer organisations such as universities, hospitals or research institutes [13,14,15,16,17,18]. Such policy developments have also attracted scholarly interest, and an emerging body of policy-oriented work has begun to explore organisational and jurisdictional efforts related to public involvement as well as the arrangements and institutions that direct the health research enterprise and condition the potential for the successful implementation of the public involvement agenda [2, 19].
Policy efforts to direct public involvement in health research, alongside an emerging body of policy-oriented scholarship, suggest the need to clearly conceptualise and characterise public involvement in health research in terms of governance; that is, the way rules, norms and actions are structured, sustained and regulated to condition the operation and impact of public involvement activity. The term ‘governance’ differs from ‘government’ in highlighting the distributed nature of authority and the many ways that individuals and groups “organize themselves to achieve agreed goals” . Governments are looked to as essential actors from this perspective, but so too are authorities in the para-public or private sectors, creating opportunities for “good governance”  alongside challenges for democratic and formal accountability .
Governance efforts in the health sector are increasingly informed by ‘systems thinking’. For example, WHO has issued guidance on Systems Thinking for Health System Strengthening  as well as the WHO framework for conceptualising ‘health research systems’ (HRS) to ensure “knowledge for better health”, which has particular salience for research policy . Emerging from the international Commission on Health Research for Development, the HRS literature aims to support countries, and the global community, to build and sustain systems of research “involving people, institutions and processes” that serve health systems and support population health and health equity . The WHO World Health Report of 2013 reinforced the value of the ‘systems’ perspective on health research, noting that, “To make the best use of limited resources, systems are needed to develop national research agendas, to raise funds, to strengthen research capacity, and to make appropriate and effective use of research findings” .
The HRS framework is highly relevant to governance for public involvement. The framework was designed in recognition of the multiplicity of arrangements, expectations, obligations and incentives in research contexts that are often “fragmented, competitive, [and] highly specialized” , which is salient to public involvement, given the challenges it may pose to dominant interests and usual practice . In a related vein, the framework is attentive to the diversity of stakeholders within a HRS, and the navigation of stakeholders’ multiple, and not always compatible, interests . Further, the HRS framework expresses an openness to public involvement, including where communities lead their own research, as with emancipatory research designs [25, 28]. Finally, the HRS framework is clear regarding the imperative to attend to health research beyond healthcare in order to address the social determinants of health and the demands of health equity [25, 27].
Thus, to inform efforts to foster public involvement across the provincial health research enterprise in Ontario, Canada, we sought to explore the relevance of the HRS framework to governance for public involvement. We drew specifically on the conceptual framework developed by Pang et al.  to consider the principal functions and associated operational components of a HRS. The framework provided the basic tools for conceptualising governance and for operationalising our review of relevant evidence.
We drew on scoping review methodology to identify a corpus of scholarly papers and policy reports with relevance to governance for public involvement within HRS [29, 30]. We included conceptual and empirical papers and did not assess quality or exclude papers on that basis. Given time and resource constraints, we iteratively reviewed only one database, and supplemented the database search with a targeted environmental scan of policy reports relevant to public involvement from major public sector research organisations that fund health research or support public involvement in health research across selected jurisdictions. Finally, the project team was advised in the review and targeted scan by a committee of local experts from the health sector, who participated in team meetings and advised on project execution .
Database search and selection strategy
We began with a series of targeted searches to understand the state of the literature and refine our research questions. Then, in consultation with a research librarian, we developed a strategy for searching electronic databases, translating search concepts into keywords and medical subject heading (MeSH) terms using common indexing practices. From December 2015 through February 2016, we searched the biomedical electronic database, Ovid MEDLINE, from its inception through December 2015. Search terms were compiled and tested several times to capture potentially relevant articles. Due to time and resource constraints, no additional database searches were conducted. However, given the rapidly evolving nature of the field, we updated our search in 2017, using the same search terms in the same database, limited to articles published between December 2015 and June 2017. We report here on these combined searches to June 2017, inclusive (Additional file 1: Search terms).
Two reviewers independently screened the titles and abstracts for potential inclusion, retaining all papers that were included by either reviewer. The same two reviewers then independently assessed the full text of potentially relevant papers for eligibility for final inclusion. Final decisions on inclusion were made through discussion with a third reviewer.
We included all original studies of any design, including all non-empirical papers (commentaries, editorials) published in the English language that discussed how to organise, systematise or oversee the involvement of patients, members of the lay public or communities in health research within organisations or jurisdictions. Specifically, we included papers that were (1) concerned with ‘publics’ – patients (clients, consumers, informal caregivers organised groups), members of the lay public, lay communities; (2) about ‘engagement’/‘involvement’ of publics, including community-based participatory action and related research; (3) about health research of any sort (basic, health services, health systems, etc.); and (4) relevant to health research governance and ‘systems’, by addressing considerations or efforts at organisational or inter-organisational (i.e. meso) or jurisdictional (i.e. macro) levels. Articles specific to involvement in healthcare systems or health service delivery were excluded, as were articles concerned with public involvement in science that lacked a clear focus on health research. To identify additional potential papers, we reviewed reference lists from included articles and conducted further searches using the Google Scholar ‘cited by’ and the MEDLINE PubMed ‘related articles’ features. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria were used to help guide the conduct and reporting of the review .
Targeted environmental scan of policy reports
Informed by findings from the MEDLINE database search and suggestions by Advisory Committee members, we identified major organisations with leadership roles with respect to funding health research or supporting public (patient, community, lay public) involvement in health research in selected jurisdictions (United Kingdom, United States, Australia and Canada), and searched online using Google for relevant policy reports (Additional file 2: list of reviewed organisations). Only key framework, guidance or evaluative documents were included.
Data extraction, analysis and synthesis
We aimed to use the HRS framework developed by Pang et al.  as a data extraction template for the corpus of collected papers, seeking evidence across the framework’s four domains and nine embedded operational components. Given the utility of the existing framework for an international community of practice, we aimed to make few modifications. However, some adaptations to ensure that the framework was relevant to public involvement were necessary. Adaptations were based mainly on an a priori logic but also arose from reflection on the data captured in each category.
We drew on the ‘logics’ for mixed methods-mixed research synthesis identified by Sandelowski et al.  to analyse the dataset. Specifically, we adopted the logic of research synthesis by aggregation, which entails the assimilation of findings considered to address the same relationship or connection between two or more aspects of a target phenomenon. Additionally, we adopted the logic of research synthesis by configuration, which involves a more theoretical rendering of the data reviewed, arranging diverse findings to forge a new interpretation. Our approach to configuration was ‘top down’, as we sought to interpret studies in light of a prior conceptual framework . We analysed the data thematically, using a qualitative interpretive approach and drawing on the traditions of constructivist grounded theory to merge and link findings from different studies into common categories [34,35,36].
Overview of included papers
The database search identified 3121 unique papers, of which 56 papers were retained after title and abstract screening. Of these, 14 were retained after full text screening. A further 18 papers were added after reviewing reference lists of included papers and conducting supplementary citation searches and from investigators’ files (Additional file 3: full list of 32 scholarly papers). The targeted environmental scan identified 13 policy reports, for a total of 45 included papers (Table 1; Additional file 4: list of 13 policy reports; Fig. 1 for PRISMA diagram).
Most included papers reported on, recommended or described the engagement efforts of various research organisations such as funding agencies, academic organisations or organisations that represent publics. Only a small proportion explicitly aimed to explore public engagement as a policy effort or within health research networks, structures or systems that spanned organisations [2, 19, 37], and only one referenced a HRS conceptual framework . Commentary was more common than empirical research.
Governing public involvement for HRS
We identified activities relevant to public involvement across all functions in the HRS framework. Further, we identified roles related to public involvement for both governments as well as research organisations that act as public or para-public stewards such as research funding agencies, research producer organisations (e.g. universities, hospitals) and organisations or organisational units that aim to support public involvement  (Table 2).
According to Pang et al. , the first core function of an effective HRS is stewardship.
Define and mobilise a vision for public involvement
As anticipated by the Pang et al. framework , ‘vision’ was identified as important for advancing public involvement in health research, including at the level of the jurisdiction or research organisation. Yet, the substance of such visions and their implications varied (Table 3). On the one hand, there was a general endorsement of efforts to formalise and implement clear visions, which was particularly emphasised in policy reports [15,16,17], though some scholarly papers also endorsed the value of well-defined visions that were supportive of public involvement.
Several scholarly papers reflected critically on the source and implications of organisational or policy visions for public involvement. At the national level, government attention to public involvement in health research could be seen as a response to public scepticism or concern [39, 40], but also as a resource for political platforms such as patient choice in healthcare or democratic accountability in general [19, 40]. Importantly, such political visions did not empower all publics equally [41, 42]. Additionally, some authors cautioned that visions for public involvement could be used to support the self-interest of research organisations more than organisational or system change .
Further, visions for public involvement differed substantively in the way publics and public involvement were conceived of. Visions differed, first, with respect to the type of publics to engage – whether patients or communities, with attention to communities more present in scholarly literature than policy reports. These different constituents offered various forms of expertise and implied different types of research. Patients were understood to provide expertise derived from personal experience with health conditions, such that patient involvement encouraged attention to the various types of health research that could inform improved health outcomes and high-quality care [1, 4, 15, 16, 43]. Community expertise was understood to emerge from collective conditions and opportunities relevant to population health, and scholarly papers drawing on traditions of community-based participatory action research encouraged attention to health inequities and the social determinants of health [38, 41, 42, 44,45,46]. In addition to these two core publics, lay persons were occasionally referenced as general or disinterested publics to involve . Finally, other research stakeholders (e.g. health professionals) were sometimes included in discussions of public involvement [47, 48], without necessarily clarifying the distinctive significance of patient, community or lay public expertise (Additional file 5).
Visions also differed in terms of the types of involvement to pursue. While the scholarly literature generally anticipated high levels of public involvement [49, 50] and the literature on community involvement envisaged a specifically community-based, participatory and action-oriented approach [46, 51], policy reports were agnostic and typically referenced a spectrum of involvement levels, including activities such as communication about research, fundraising for research and participation in research [13, 17, 52].
Finally, some scholars considered the question of vision implementation and impact (Table 4). Several papers pointed to the importance of mechanisms to encourage visions to be implemented, including policy instruments at the national level that recommended, incentivised or obliged action [4, 37, 40, 48], as well as the policies or strategies of individual research producer organisations [4, 53,54,55]. Further, some scholars highlighted the limited degree of organisational or system change realised to date, identifying the partial ways in which publics have been embedded structurally with influence on outcomes, operationalised as partners in processes of ‘user led’ or ‘co-produced’ research, or able to successfully challenge dominant epistemological models of science that privilege professional scientific expertise [19, 37, 39]. These works also offered insight into the organisational actors and network processes that enable and condition public involvement, with key roles for ‘intermediary organisations’ , such as research funding bodies, research producing organisations and public involvement support organisations, in channelling and structuring public involvement activity .
Defining the role of public involvement in identifying appropriate health research priorities and coordinating adherence to them
The included papers endorsed Pang et al.’s  view of priority-setting as a key stewardship function and highlighted the importance of public involvement in such processes (Table 5). For most, discussion of the issue centred on the need for public involvement in priority-setting and the expectation that involvement would make a difference with respect to the priorities selected [1, 7]. These discussions were typically focused at the level of the research organisation, with recommendations to increase public involvement in priority-setting [15, 16, 56], as well as descriptions of the ways different funding organisations had done so [6, 48, 57,58,59]. Less commonly, public involvement in priority-setting was discussed at the national level, and with respect to a specific moral purpose – that of health equity . Discussions of approaches to involving publics in priority-setting were typically descriptive [57,58,59]. However, some authors cautioned about the need for more attention to the implications of different structures and practices of involvement for research priority-setting , while Pratt et al.  offered a particularly detailed discussion of the priority-setting processes required to enable what they characterised as “deep inclusion”.
Strategies for coordinating adherence to research priorities – ensuring and sustaining the impact of public involvement in research priority-setting – were less commonly discussed. Several scholars argued that, with rare exceptions, priority-setting across the research system had been relatively unaffected by public involvement [4, 7, 19, 37], and pointed to the limited capacity for public involvement strategies and processes to redirect “value-laden and political” research priorities . Involvement initiatives were often partial , and even well-developed efforts might not prove sufficient to redirect research priorities away from dominant concerns to, for example, “consumer control” research [4, 37] or action-oriented and intervention research concerned with social determinants [38, 42, 45].
The role of public involvement in setting and monitoring ethical standards for health research
Ethical questions are central to the third operational component of the stewardship function given the ethical, legal, economic and social challenges arising with many scientific advances, as well as the persistent challenges in how the risks and rewards of conducting research or allocating its benefits are to be distributed among and between private and public sector actors .
The role of public involvement with respect to ethical standards evoked two argument strands (Table 6). First and most clearly, some papers identified a role for publics as ethical arbiters in the conduct of specific projects or research organisations [38, 41, 44, 61]. Second, some papers conceived of a higher order ethical balancing act – one which was provoked by public involvement but did not clearly identify a role for public involvement in its resolution. These authors conceived of public involvement as a powerful challenge to existing authorities and epistemic assumptions about the practice and purpose of health research – a challenge that necessitated a better balance between traditional and newer approaches and normative assumptions [2, 4, 19, 39].
Monitoring and evaluating public involvement in the HRS
The final operational component of the stewardship function relates to monitoring and evaluation. Included papers attended to this activity at two levels, namely at the level of the individual project and at the level of the organisation or jurisdiction. Many of the papers called for, described or recommended approaches to monitoring and evaluating public involvement in health research, focusing primarily on public involvement in research projects [4, 5, 47, 59, 62]. A few policy reports advocated monitoring and evaluation of organisations and systems, with recommendations for organisations to monitor and evaluate the public involvement activity that they fostered or pursued, as well as approaches, such as regular review and data collection, to monitor the uptake of public involvement across sectors or systems [17, 63].
Additionally, a number of scholarly papers offered critical reflections on the challenges of evaluating public involvement (Table 7). A few engaged debates about the extent to which an activity motivated by ethical commitments should be judged for its instrumental effectiveness . Others reflected on the challenges of methodology, measurement and intent, given the complexity of public involvement as an intervention, as well as concerns about the evaluative interpretations, methods and processes that might minimise or misunderstand public involvement’s value or impact [4, 19, 40, 61].
The second major function of a HRS with relevance to governance for public involvement concerns its financing, both in securing funds to support public involvement and in accountably allocating these funds .
Secure funds to support public involvement in research
The issue of securing funds to support public involvement in research was not discussed in great detail (Table 8). Some scholarly papers identified specific needs for funds for public involvement, advancing the argument that robust public involvement adds to the cost of the research enterprise [46, 57, 58], but that these added costs may be neither anticipated nor appropriately valued [4, 5, 60]. Furthermore, some authors considered how research funds to support public involvement should be secured and managed. Governments and other research organisations (e.g. universities, NGOs) were seen as having a role in securing or coordinating funding to enable public involvement [38, 46, 60]. Finally, funding policy was identified as a relevant factor given the potential significance of stable as compared to competitive funding and the risk of conflicts of interest where funders of public involvement had an interest in specific research outcomes .
Define the role of public involvement in accountably allocating funds
In contradistinction to the issue of securing funds, included papers offered considerable insight on how monies should be allocated to support public involvement in research (Table 9). Additionally, unlike Pang et al. , who noted the importance of “[a]n efficient, transparent, and peer-review-based process … at the core of this function”, these papers emphasised a role for publics – not peers – in fund allocation. Publics could be involved in funding allocation in three ways, namely (1) through participation in grant review processes, (2) through the criteria and calculus used to assess the adequacy of projects and of the public involvement activity embedded within them, and (3) through the types of financial flows and reimbursement mechanisms that could support public involvement in practice.
Among the papers that discussed allocation processes, many were highly descriptive, identifying different ways in which publics could be involved in research review [13, 17, 46, 47]. Others took a more explicitly evaluative approach, analysing approaches to involving publics within specific organisations or across whole research systems [6, 19, 57,58,59, 64]. These papers were more critical, pointing to the significance of different types of involvement, at different points in the process, for shaping research agendas  and the limited extent to which publics were involved in such processes across research funding agencies generally [19, 64].
A number of papers considered the issue of allocation criteria and calculus, identifying different ways in which public involvement might be relevant to the adjudication of research projects. Several papers were concerned with how projects that involved publics would be assessed and called for attention to consumer-identified values and guidelines [46, 47], or the use of criteria that were specifically relevant to community-based participatory research where such research was under review [46, 51]. Other papers were concerned with how evidence of involvement with publics might be considered in research funding decisions. Several papers identified criteria for judging public involvement, such as evidence of the relevance of the specific partnership and the depth of extant involvement, or the adequacy of the time and financial resources allocated to further public involvement activities [6, 44, 48]. Others discussed the calculus to be used in weighing and balancing public involvement as a criterion relative to other criteria. Papers offered descriptions of the varied ways in which funding agencies specified expectations for how involvement was to be described in applications and standards for how these descriptions were to be factored into decisions . Further, some scholars criticised prevalent evaluative logics, which were seen to minimise the importance of public involvement in the adjudication of research projects .
Finally, several papers discussed the allocation of funds in terms of the mechanisms through which funds were disbursed, which could be more or less supportive of public involvement in practice. Some of this literature focused on the funding mechanisms that could encourage researchers to pursue public involvement in research. This included funding requirements, such as where public involvement was a condition of funding , or funding opportunities (e.g. leaves, fellowships) that might encourage students or junior researchers to pursue public involvement activities [61, 65]. Most of the literature discussed the limitations of existing funding arrangements for enabling publics to engage in research.
That publics face funding challenges that typically impede involvement was highlighted in many papers [38, 41, 42, 51, 66]. To address these challenges, several policy reports discussed the need for researchers to proactively plan for involvement activities in the development of research budgets [17, 43, 52, 56]. Additionally, challenges included mechanisms that could limit the allocation of funds, specifically rules regarding eligibility for receipt of funds, and administrative practices that could significantly delay payment or reimbursement to community partners [17, 44, 45]. Finally, several scholars took issue with the usual approaches to funding public involvement. This included the tendency to fund public involvement where it was embedded within specific research projects rather than as a standalone activity, and to flow funds through academic rather than public participants. Some scholars argued for direct funding for publics to enable them to partner in health research, including by permitting a community partner to be the grant recipient, and by providing pre-application support to increase the odds of success [46, 65].
Creating and sustaining resources to conduct, absorb and utilise health research that involves publics
Pang et al.  argue that “creating and sustaining resources remains as a central issue” for HRS, with a need to produce and sustain human capacity, including through training and the availability of reliable career paths, as well as through physical infrastructure to conduct and use health research. While the creation of capacity clearly requires financial resources, as noted above, we herein review the non-financial dimensions of the resources needed to support public involvement in HRS, distinguishing between human resources on the one hand and organisational arrangements and infrastructure on the other.
Concern with human resources to support public involvement was prominent among included papers (Table 10). A commonly identified strategy to build human resources for public involvement concerned the provision of information or short-term training, for both researchers and publics [4, 41, 46, 47, 52, 57,58,59, 63, 64, 67]. This was needed to provide basic knowledge about research and partnerships working and necessitated that individuals be provided with access and that research organisations ensure availability [41, 43, 47, 56].
Yet, the need for support was not only initial or temporary but included longer-term mentorship or support and necessitated committed effort by research organisations. Ongoing mentorship and feedback was needed to enable publics to participate actively and contribute substantively [54, 55, 59, 62, 68], and was especially important to permit meaningful inclusion by members of more disadvantaged communities . To make such support available, research organisations needed to commit resources, including staff time and space [54, 55, 59, 62, 68]. Finally, there was a need for specific efforts to encourage researchers to involve publics in research. Research capacity could be created through graduate programmes, which could provide training in the principles and practice of public involvement in research and might recruit students from ‘studied’ communities to become future researchers [46, 51]. There was also a need to sustain research capacity through opportunities for mentorship from senior researchers with expertise in public involvement and access to supportive networks and a ‘critical mass’ of committed colleagues [44, 61, 65, 69]. Finally, support for researchers necessitated the appropriate valuation of public involvement in professional advancement metrics and processes, given concerns about the extent to which usual tenure and promotion processes and metrics for ‘impact’ devalued the types of applied, often-local and time-consuming work that meaningful public involvement required [44, 46, 61, 63, 65].
The resources required to ensure that individual researchers and members of the public were informed and empowered to collaborate necessarily implicated organisational practices, as already suggested. However, capacity-building needs were also expressly organisational in nature, including consideration of how commitments to public involvement could be organisationally instantiated and shared across the research system (Table 11). One area of organisational practice that was regularly identified related to the mobilisation of advocates for public involvement, including through roles for managers and other public involvement professionals, or through the organisational arrangements that gave regular voice to publics in research organisations [54, 56, 63, 68]. A second area of organisational practice was related to research infrastructure to enable the conduct of research that involved publics. In part, this related to organisational reform to alter the operation of research-producing organisations through restructured academic committees, as well as support for novel research positions, organisations, networks or nodal capacity to serve as the coordinators, conveners, doers, resource brokers and all-round capacity-builders for public involvement in health research [61, 62, 65, 68].
Organisational capacity was also needed beyond the walls of individual research-producing organisations to serve the research system as a whole. This involved mobilising advocates of public involvement in structural and strategic ways, with formal positions for publics within key research organisations [2, 37, 39, 51]; it also involved the creation of infrastructure that could be jurisdiction-wide in its relevance and impact. Such infrastructure could help to coordinate the involvement of publics in priority-setting or provide coordinated training and support to publics and public involvement. Furthermore, system-wide capacity was discussed as a means of overcoming fragmentation of effort, for example, through the creation of single access points or ‘matchmaking’ services for publics and researchers [55, 68]. Such infrastructure might be developed by states or major research organisations or emerge through collaboration [4, 40, 57]. Against this, some authors cautioned against the risk that an overly professionalised public involvement infrastructure might side-line publics, and the more radical demands that diverse publics might articulate .
Producing, synthesising and utilising research
In general, attention to research use was limited in the documents we consulted, though the literature is clear in perceiving involvement as a key force in ensuring that the right research, asking the right questions, is done in the right way, so that it can be used. Publics provide expertise, judgment and capacity in knowledge production and are seen as key beneficiaries and users of – as well as advocates for the use of – health research [38, 60]. Indeed, the public commitment to research use may be more developed than is the case for other participants in the research process, including academics, universities or funding organisations.
Increased involvement of publics in health research has been accompanied by increased effort by governments and research organisations to foster, direct and evaluate the public involvement enterprise. This raises critical questions about how and with what effect public involvement has been embedded within organisations and jurisdictions – questions that policy-oriented scholars have begun to pose.
We approached these policy questions from the perspective of governance – asking how the rules, norms and actions that structure and sustain public involvement in health research are coordinated and held accountable within jurisdictions. Central to our effort was an established framework designed to support countries to strengthen their HRS . This framework identified four core ‘functions’ of a HRS, and nine embedded ‘operational components’, which served as the data collection architecture for our search, selection and analysis of scholarly papers and policy reports. Additionally, it provided a centrally important concept – that of the HRS itself – to define the space within which governance effort is embedded.
Though newer and less well known, the concept of the HRS is as important as the complementary concept of the ‘health system’. Both terms call attention to the interconnected sets of organisational and individual actors and the web of rules, expectations and structures that produce health research or health services and population health outcomes, namely research, services or outcomes that are more or less robust, relevant and responsive. Where system theories that emphasise complexity and adaptation are drawn on, these concepts also highlight the dynamic and emergent nature of many system properties and the potential for both intended and unintended effects [23, 70,71,72], and suggest the value of longitudinal research that queries not only the achievement of the desired visions and intended outcomes of public involvement initiatives, but also the unintended effects of these initiatives on the health and healthcare of actors and the subsystems in which they are embedded.
A governance perspective has considerable value for public involvement in HRS, supporting efforts to coordinate and institutionalise the burgeoning public involvement enterprise. The perspective points, first, to the range of ways in which public involvement has to be enacted by organisations and within jurisdictions to support and give full effect to the efforts of countless individuals. This involves stewardship to articulate a compelling vision for public involvement, to clarify the ways in which publics will set research priorities and ethical standards, and to foster clarity and consistency in the monitoring and evaluation of public involvement. This also involves financing to ensure the adequacy of funds to support public involvement and to address issues in funding policy, including the structure of funding flows and the potential for conflicted expectations by funding sources. This further involves capacity-building, in both human and organisational terms, to empower individual researchers and publics to partner and build commitment to, and enable, public involvement within research organisations and across research systems. Finally, this involves attention to the cycle of research production, synthesis and use – in recognition of the public commitment to research relevance and impact , and the role of public involvement in ensuring such outcomes.
The evidence collected and organised here might be used by public involvement leaders and research system stewards to catalyse reflection and action. It is relevant also to public involvement researchers to encourage existing efforts to forge system-wide approaches on critical issues such as evaluation [74, 75], and to identify the need for more policy-oriented work to compare, assess and theorise about the operation and effects of varied organisational and jurisdictional efforts across all HRS functions.
Yet, a governance perspective does not simply identify domains of activity and opportunities for research. It also highlights challenges for what is, ultimately, a highly political intervention. As many scholars have noted, the aim of public involvement is contested – whether to enable to ‘do better’ what is already done, or to challenge existing assumptions and arrangements, including about the nature and epistemic authority of science [37, 39]. Contestation over the ends of public involvement was clearly evident in our review, particularly with respect to the stewardship function. Importantly, visions for public involvement differ with respect to both ‘who’ to involve and ‘how’. Further, differences extend beyond goal-setting to encompass the question of implementation and the ultimate impact of public involvement in health research.
The differences in vision that our review highlighted have important implications for HRS and the health systems they seek to serve. In particular, the scholarly literature is largely split between discussions of communities on the one hand, and patients on the other, while policy reports are primarily concerned with patients. Relatedly, scholarly literature on patient involvement emphasises high levels of involvement and scholarship on community involvement emphasises a specifically participatory and action-oriented approach, while policy reports are typically agnostic, allowing the notion that involvement can extend from research partnership through to research participation. Terminological differences are relevant here; in some contexts, the term ‘involvement’ implies more than ‘engagement’, while in others the reverse is the case [4, 76, 77]. Additionally, the term ‘public’ is not necessarily recognised or welcomed by the individuals to be involved, who may instead identify as patients, clients, service users, family members, community members or activists .
Nevertheless, the differences we identified extend beyond terminology to the politics of public involvement itself. Public involvement that aims towards partnership and action is potentially transformative, whereas public involvement that aims at public participation is not. Further, which ‘publics’ are to be involved has implications for what expertise will be mobilised and thus what research will be prioritised and pursued. If patients are only asked to represent themselves as individuals, and not as and with diverse communities nor as and with lay publics, then the aims of HRS to support health equity and to enable health beyond the delivery of care, including through public health efforts and ‘health in all’ policies and practice, will be abridged.
In addition to contestation over visions, our review identified concerns about the particular and partial implementation of the public involvement agenda and its limited impact. Visions for public involvement may be implemented to advance the self-interest of the research organisation rather than foster public empowerment , and even the impact of transformative visions may be conditioned by resistant organisational arrangements and network processes . Furthermore, public involvement has seen partial uptake across funding organisations and its impact on redirecting research priorities remains limited [41, 60].
Issues of implementation and impact become highly visible from a health research ‘systems’ perspective but raise profound challenges for governance and accountability. Both in theory and in practice, the governance of research systems is a distributed function that mobilises many authoritative actors. Indeed, as Caron-Flinterman et al.  highlighted in an early example of policy scholarship on public involvement, multiple ‘intermediary organisations’, such as research funding agencies, research producer organisations (i.e. universities, hospitals) and public involvement support organisations (e.g. INVOLVE), are positioned between science and society and structure opportunities for public involvement effort and impact. As more recent research on regulation and science policy makes clear, such critical ‘intermediaries’ are not simply passive channels for transferring knowledge and objects between science, policy and economy , but rather “they mobilize, reframe and structure expertise and policy imperatives” .
Governance challenges within networked systems are not unique to public involvement, though they assume a specific form. Given the important role of powerful intermediary organisations in HRS, questions arise about how accountability expectations for public involvement are to be established, how they are to be exercised (specifically what policy instruments or tools can be used) and how and with what transparency and traceability they can be enforced. Scholars who have addressed the challenges of governance and accountability for health systems highlight the need to emphasise the distinctive legitimacy of governments as authoritative actors and the importance of financial instruments in driving coordinated action and outcomes . These remedies may have relevance to the governance of HRS in general and to public involvement for such systems in particular.
The political nature of the public involvement agenda and the governance challenges that arise for HRS highlight an additional dimension of public involvement – one that we did not anticipate. The initial aim of our review was to explore how to govern for public involvement ‘within’ a HRS, which we understood as ensuring that support for public involvement is embedded ‘system wide’ and that ‘systemic effort’ is made to ensure its effectiveness. Yet, our review also highlighted a slightly different but equally important governance question – that is, what is the ‘role’ of public involvement in ‘governing’ HRS. Here, the question is less about how to embed public involvement through policy, process or infrastructure, and more about the importance of public involvement for navigating tensions within, and legitimising the efforts of, a HRS. Contests over vision and issues of accountability in the implementation and impact of public involvement cannot be resolved by researchers and the traditional governors of the research enterprise. Judgments of this sort, as Milewa et al.  have argued in a related context, “can only rest upon discursive processes within the social realm”.
The limitations of this review are several. Of note, time and resource constraints significantly limited our evidence review. We systematically searched only one database and supplemented our collection through backward and forward citation searches of included papers, and the collection of relevant papers from the investigators’ files. The limitations of our restricted search were likely compounded by the use of MEDLINE, which does not reference all relevant social science literature, the weakness of MeSH terms for identifying public involvement scholarship, and the importance of grey literature for reports on public involvement. Our targeted environmental scan aimed to address some of the grey literature limitations, but it too was limited by the restricted number of jurisdictions and organisations targeted and our focus on specific types of policy reports.
While searches across additional databases would be a partial remedy, we would suggest that a greater limitation may arise from our focus on policy-relevant scholarship, specifically the requirement that papers be focused on research systems, networks, organisations or jurisdictions to be eligible for inclusion. While such a selection criterion was necessary to make our search tractable, it is likely that a much broader literature set would be relevant to the issues of governance that were the focus of our review. This is perhaps especially notable in relation to the fourth function of the HRS – producing/using health research – for which we identified very little research. A much more expansive review or, more feasibly, a series of reviews focused on specific research system functions, would add significantly to the evidence collected and might highlight governance issues not seen through this review.
Despite limitations, the suggested framework for governance for public involvement in HRS provides both insights and guidance. One conclusion must certainly be the need for more policy-oriented research on public involvement; specifically, research that compares, contrasts and evaluates the varied processes, arrangements and initiatives that mobilise public involvement within and across organisations and research systems. Additionally, we offer insight into the role of publics in HRS, not simply as components of a HRS, but as a central plank in its good governance.
health research systems
Medical subject heading
Caron-Flinterman JF, Broerse JE, Bunders JF. The experiential knowledge of patients: a new resource for biomedical research? Soc Sci Med. 2005;60(11):2575–84.
Caron-Flinterman JF, Broerse JE, Bunders JF. Patient partnership in decision-making on biomedical research: changing the network. Sci Technol Human Values. 2007;32(3):339–68.
Staley K, Buckland SA, Hayes H, Tarpey M. ‘The missing links’: understanding how context and mechanism influence the impact of public involvement in research. Health Expect. 2014;17(6):755–64.
Boote J, Telford R, Cooper C. Consumer involvement in health research: a review and research agenda. Health Policy. 2002;61(2):213–36.
O’Donnell M, Entwistle V. Consumer involvement in research projects: the activities of research funders. Health Policy. 2004;69(2):229–38.
O’Donnell M, Entwistle V. Consumer involvement in decisions about what health-related research is funded. Health Policy. 2004;70(3):281–90.
Cowan K. The James Lind alliance: tackling treatment uncertainties together. J Ambul Care Manage. 2010;33(3):241–8.
Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient. 2014;7(4):387–95.
Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2014;17(5):637–50.
Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14(1):89.
Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66.
Rowe G, Rawsthorne D, Scarpello T, Dainty JR. Public engagement in research funding: a study of public capabilities and engagement methodology. Public Underst Sci. 2010;19(2):225–39.
Canadian Institutes of Health Research. CIHR’s Framework for Citizen Engagement. 2012. http://www.cihr-irsc.gc.ca/e/documents/ce_framework_e.pdf. Accessed 25 Jul 2018.
United Kingdom Department of Health. Research and Development for a First Class Service: R&D Funding in the New NHS. 2000. http://webarchive.nationalarchives.gov.uk/+/http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4005946. Accessed 25 Jul 2018.
Canadian Institutes of Health Research. Strategy for Patient-Oriented Research (SPOR). 2011. http://www.cihr-irsc.gc.ca/e/documents/P-O_Research_Strategy-eng.pdf. Accessed 25 Jul 2018.
Canadian Institutes of Health Research. Strategy for Patient-oriented Research (SPOR) Patient Engagement Framework. 2014. http://www.cihr-irsc.gc.ca/e/documents/spor_framework-en.pdf. Accessed 25 Jul 2018.
National Institute for Health Research. Going the Extra Mile: Improving the Nation’s Health and Wellbeing through Public Involvement in Research. 2015. https://www.nihr.ac.uk/patients-and-public/documents/Going-the-Extra-Mile.pdf. Accessed 25 Jul 2018.
United Kingdom Department of Health. Best research for Best Health: A New National Health Research Strategy. 2005. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/568772/dh_4127152_v2.pdf. Accessed 25 Jul 2018.
Van Bekkum JE, Fergie GM, Hilton S. Health and medical research funding agencies’ promotion of public engagement within research: a qualitative interview study exploring the United Kingdom context. Health Res Policy Syst. 2016;14:23.
Dodgson R, Lee K, Drager N. Global Health Governance: A Conceptual Review. Centre on Global Change and Health. London School of Hygiene & Tropical Medicine/World Health Organization. http://apps.who.int/iris/bitstream/handle/10665/68934/a85727_eng.pdf?sequence=1&isAllowed=y. Accessed 25 Jul 2018.
Saltman RB, Ferroussier-Davis O. The concept of stewardship in health policy. Bull World Health Organ. 2000;78(6):732–9.
Tuohy CH. Agency, contract and governance: shifting shapes of accountability in the health care arena. J Health Polit Policy Law. 2003;28(2–3):195–215.
De Savigny D, Adam T. Systems Thinking for Health Systems Strengthening. World Health Organization. 2009. http://apps.who.int/iris/bitstream/10665/44204/1/9789241563895_eng.pdf. Accessed 25 Jul 2018.
Hanney SR, González-Block MA. Organising health research systems as a key to improving health: the world health report 2013 and how to make further progress. Health Res Policy Syst. 2013;11:47.
Commission on Health Research for Development. Health Research: Essential Link to Equity in Development. New York: Oxford University Press; 1990.
World Health Organization. The World Health Report 2013: Research for Universal Health Coverage. 2013. http://apps.who.int/iris/bitstream/10665/85761/2/9789240690837_eng.pdf?ua=1. Accessed 25 Jul 2018.
Pang T, Sadana R, Hanney S, Bhutta ZA, Hyder AA, Simon J. Knowledge for better health: a conceptual framework and foundation for health research systems. Bull World Health Organ. 2003;81(11):815–20.
Hanney S, Kuruvilla S, Soper B, Mays N. Who needs what from a national health research system: lessons from reforms to the English Department of Health's R&D system. Health Res Policy Syst. 2010;8:11.
Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.
Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69.
Daudt HM, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. BMC Med Res Methodol. 2013;13:48.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
Sandelowski M, Voils CI, Leeman J, Crandell JL. Mapping the mixed methods–mixed research synthesis terrain. J Mix Methods Res. 2012;6(4):317–31.
Thorne S. Data analysis in qualitative research. Evid Based Nurs. 2000;3(3):68–70.
Charmaz K. Constructing Grounded Theory: A Practical Guide through Qualitative Analysis. Thousand Oaks: Pine Forge Press; 2006.
Clarke A. Situational Analysis: Grounded Theory after the Postmodern Turn. Thousand Oaks: Sage Publications; 2005.
Green G. Power to the people: to what extent has public involvement in applied health research achieved this? Res Involv Engagem. 2016;2(1):28.
Delisle H, Roberts JH, Munro M, Jones L, Gyorkos TW. The role of NGOs in global health research for development. Health Res Policy Syst. 2005;3:3.
Milewa T, Buxton M, Hanney S. Lay involvement in the public funding of medical research: expertise and counter-expertise in empirical and analytical perspective. Crit Public Health. 2008;18(3):357–66.
Evans D. Patient and public involvement in research in the English NHS: a documentary analysis of the complex interplay of evidence and policy. Evid Policy. 2014;10(3):361–77.
Beinare D, McCarthy M. Civil society organisations, social innovation and health research in Europe. Eur J Pub Health. 2011;22(6):889–93.
Knabe A, McCarthy M. Civil society organisations and public health research-evidence from eight European Union new member states. Cent Eur J Public Health. 2012;20(4):287.
National Health and Medical Research Council. Statement on Consumer and Community Participation in Health and Medical Research. 2016. https://www.nhmrc.gov.au/_files_nhmrc/file/publications/16298_nhmrc_-_statement_on_consumer_and_community_involvement_in_health_and_medical_research-accessible.pdf. Accessed 25 Jul 2018.
Michener L, Cook J, Ahmed SM, Yonas MA, Coyne-Beasley T, Aguilar-Gaxiola S. Aligning the goals of community-engaged research: why and how academic health centers can successfully engage with communities to improve health. Acad Med. 2012;87(3):285.
Sanders D, Labonte R, Baum F, Chopra M. Making research matter: a civil society perspective on health research. Bull World Health Organ. 2004;82(10):757–63.
Israel BA, Schulz AJ, Parker EA, Becker AB. Community-based participatory research: policy recommendations for promoting a partnership approach in health research. Educ Health (Abingdon). 2001;14(2):182–97.
Petit-Zeman S, Philpots E, Denegri S. “Natural ground” for medical research charities: public and patient involvement in research funding. J Ambul Care Manage. 2010;33(3):249–56.
Fleurence R, Selby JV, Odom-Walker K, Hunt G, Meltzer D, Slutsky JR, et al. How the Patient-Centered Outcomes Research Institute is engaging patients and others in shaping its research agenda. Health Aff. 2013;32(2):393–400.
Arnstein SR. A ladder of citizen participation. J Am Inst Plann. 1969;35(4):216–24.
Bovaird T. Beyond engagement and participation: user and community coproduction of public services. Public Adm Rev. 2007;67(5):846–60.
Ahmed SM, A-GS P. Community engagement in research: frameworks for education and peer review. Am J Public Health. 2010;100(8):1380–7.
INVOLVE. Public Involvement in Research: Values and Principles Framework. 2015. http://www.invo.org.uk/wp-content/uploads/2017/08/Values-Principles-framework-Jan2016.pdf. Accessed 25 Jul 2018.
Saunders C, Crossing S, Girgis A, Butow P, Penman A. Operationalising a model framework for consumer and community participation in health and medical research. Aust New Zealand Health Policy. 2007;4(1):13.
Saunders C, Girgis A. Status, challenges and facilitators of consumer involvement in Australian health and medical research. Health Res Policy Syst. 2010;8:34.
Minogue V, Girdlestone J. Building capacity for service user and carer involvement in research: the implications and impact of best research for best health. Int J Health Care Qual Assur Inc. 2010;23(4):422–35.
National Health and Medical Research Council. A Model Framework for Consumer and Community Participation in Health and Medical Research. 2004. https://www.nhmrc.gov.au/_files_nhmrc/file/publications/r33.pdf. Accessed 25 Jul 2018.
Oliver S, Milne R, Bradburn J, Buchanan P, Kerridge L, Walley T, et al. Involving consumers in a needs-led research programme: a pilot project. Health Expect. 2001;4(1):18–28.
Royle J, Oliver S. Consumer involvement in the health technology assessment program. Int J Technol Assess Health Care. 2004;20(4):493–7.
Moran R, Davidson P. An uneven spread: a review of public involvement in the National Institute of Health Research's health technology assessment program. Int J Technol Assess Health Care. 2011;27(4):343–7.
Pratt B, Merritt M, Hyder AA. Towards deep inclusion for equity-oriented health research priority-setting: a working model. Soc Sci Med. 2016;151:215–24.
Nyden P. Academic incentives for faculty participation in community-based participatory research. J Gen Intern Med. 2003;18(7):576–85.
Venuta R, Graham ID. Involving citizens and patients in health research. J Ambul Care Manage. 2010;33(3):215–22.
Research Councils UK. Concordat for Engaging the Public with Research. 2010. https://re.ukri.org/documents/hefce-documents/concordat-for-engaging-the-public-with-research/. Accessed 25 Jul 2018.
Van Bekkum JE, Hilton S. UK research funding bodies’ views towards public participation in health-related research decisions: an exploratory study. BMC Health Serv Res. 2014;14(1):318.
Ahmed SM, Beck B, Maurana CA, Newton G. Overcoming barriers to effective community-based participatory research in US medical schools. Educ Health (Abingdon). 2004;17(2):141–51.
Pinto RM. Community perspectives on factors that influence collaboration in public health research. Health Educ Behav. 2009;36(5):930–47.
National Institutes of Health. Historical Documents Related to the Council of Public Representatives. 2015. https://www.nih.gov/institutes-nih/nih-office-director/office-communications-public-liaison/freedom-information-act-office/historical-documents-related-council-public-representatives. Accessed 25 Jul 2018.
Saunders C, Girgis A. Enriching health research through consumer involvement? Learning through atypical exemplars. Health Promot J Austr. 2011;22(3):196–202.
Buys N, Bursnall S. Establishing university–community partnerships: processes and benefits. J High Educ Policy Manag. 2007;29(1):73–86.
Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q. 2016;94(2):392–429.
Jordon M, Lanham HJ, Anderson RA, RR MD Jr. Implications of complex adaptive systems theory for interpreting research about health care organizations. J Eval Clin Pract. 2010;16(1):228–31.
Peters D. The application of systems thinking in health: why use systems thinking? Health Res Policy Syst. 2014;12:51. https://doi.org/10.1186/1478-4505-12-51.
Dobrow MJ, Miller FA, Frank C, Brown AD. Understanding relevance of health research: considerations in the context of research impact assessment. Health Res Policy Syst. 2017;15:31.
Staniszewska S, Brett J, Mockford C, Barber R. The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care. 2011;27(4):391–9.
Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017;358:j3453.
INVOLVE. What is Public Involvement in Research? 2017. http://www.invo.org.uk/find-out-more/what-is-public-involvement-in-research-2/. Accessed 25 Jul 2018.
Canadian Institutes of Health Research. Patient Engagement. 2014. http://www.cihr-irsc.gc.ca/e/45851.html. Accessed 25 Jul 2018.
Sophie S, Ade A, Rosemary B, Peter B, Louca-Mai B, Jo B, et al. Developing the evidence base of patient and public involvement in health and social care research: the case for measuring impact. Int J Consum Stud. 2011;35(6):628–32.
Abbott KW, Levi-faur D, Snidal D. Theorizing regulatory intermediaries: the RIT model. Ann Am Acad Polit Soc Sci. 2017;670(1):14–35.
Meyer M, Kearnes M. Introduction to special section: intermediaries between science, policy and the market. Sci Public Policy. 2013;40(4):423–9.
National Coordinating Centre for Public Engagement. Concordat for Engaging the Public with Research: Briefing(s) for Senior Managers, Managers of Researchers, Supporters of Researchers, and Researchers. 2014. https://www.publicengagement.ac.uk/sites/default/files/publication/concordat_-_briefing_for_senior_managers.pdf. Accessed 25 Jul 2018.
We owe sincere thanks to Jeremy Grimshaw for helping to identify a productive way of narrowing the scope of our original ambition for this paper, to our expert Advisors – Emily Nicholas-Angl, Wayne Warry, Maureen O’Neil, Joshua Tepper and Paula Rochon – who informed and supported this work, to Ruhi Kiflen for her diligent assistance, and to the expert participants in a Round Table dialogue who provided insightful critiques of an earlier version (Lauren Ettin, John Lavis, Alies Maybee, Eddy Nason, Emily Nicholas-Angl, Kirk Nylen, Maureen O’Neil, Laura Williams). Thanks are also due to Adalsteinn Brown for having the foresight to anticipate the paper’s value and the energy to catalyse its production. Finally, we thank the Ontario SPOR Support Unit for funding this paper and acknowledge the support of Fiona Miller’s Chair in Health Management Strategies. Notwithstanding these debts, the thoughts presented here are our own.
This study was funded by the Ontario SPOR (Strategy for Patient Oriented Research) Support Unit, as one component of the Partnership for Health System Strengthening, involving the Institute of Health Policy, Management and Evaluation at the University of Toronto, the Health Forum at McMaster University, and the Ottawa Hospital Research Institute. The sponsor had no role in study design or execution, or in decisions about publication.
Availability of data and materials
The datasets used and analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Search terms. (DOCX 32 kb)
List of key organisations included in environmental scan by jurisdiction. (DOCX 21 kb)
List of included journal papers. (DOCX 20 kb)
List of included policy reports by jurisdiction. (DOCX 16 kb)
About this article
Cite this article
Miller, F.A., Patton, S.J., Dobrow, M. et al. Public involvement in health research systems: a governance framework. Health Res Policy Sys 16, 79 (2018). https://doi.org/10.1186/s12961-018-0352-7
- Public involvement
- Community-based research
- Health research systems
- Research governance
- Research policy