Skip to main content

Applying systems thinking to knowledge mobilisation in public health

Abstract

Context

Knowledge mobilisation (KM) is a vital strategy in efforts to improve public health policy and practice. Linear models describing knowledge transfer and translation have moved towards multi-directional and complexity-attuned approaches where knowledge is produced and becomes meaningful through social processes. There are calls for systems approaches to KM but little guidance on how this can be operationalised. This paper describes the contribution that systems thinking can make to KM and provides guidance about how to put it into action.

Methods

We apply a model of systems thinking (which focuses on leveraging change in complex systems) to eight KM practices empirically identified by others. We describe how these models interact and draw out some key learnings for applying systems thinking practically to KM in public health policy and practice. Examples of empirical studies, tools and targeted strategies are provided.

Findings

Systems thinking can enhance and fundamentally transform KM. It upholds a pluralistic view of knowledge as informed by multiple parts of the system and reconstituted through use. Mobilisation is conceived as a situated, non-prescriptive and potentially destabilising practice, no longer conceptualised as a discrete piece of work within wider efforts to strengthen public health but as integral to and in continual dialogue with those efforts. A systems approach to KM relies on contextual understanding, collaborative practices, addressing power imbalances and adaptive learning that responds to changing interactions between mobilisation activities and context.

Conclusion

Systems thinking offers valuable perspectives, tools and strategies to better understand complex problems in their settings and for strengthening KM practice. We make four suggestions for further developing empirical evidence and debate about how systems thinking can enhance our capacity to mobilise knowledge for solving complex problems – (1) be specific about what is meant by ‘systems thinking’, (2) describe counterfactual KM scenarios so the added value of systems thinking is clearer, (3) widen conceptualisations of impact when evaluating KM, and (4) use methods that can track how and where knowledge is mobilised in complex systems.

Peer Review reports

Introduction

Knowledge mobilisation is concerned with generating robust and useful knowledge and facilitating its movement and use in arenas where it can do most good [1]. This knowledge is often informed by diverse sources and is shared multi-directionally [2]. While knowledge mobilisation is well recognised as a vital strategy in efforts to improve public health, the ways in which it is theorised and operationalised vary widely, reflecting influences from different disciplines and paradigms [3, 4]. We make the case that systems thinking should be one of these influences. Systems thinking adds value because it offers productive ways of understanding and working with the multiple complexities that knowledge mobilisation strategies face. These include: 1. “Wicked” intractable public health problems that have multiple interacting causes and are characterised by uncertainty and conflicting values and views [5,6,7]; 2. Messy policy and practice environments which are constrained by competing demands and expectations, politicised decision-making and uncertainty [8, 9]; and 3. The dynamic nature of the open systems within which policies and programs are implemented [10,11,12,13,14]. Health is a property of many such systems including education, transport, the environment, housing, food systems, welfare systems and the economy [15]. Efforts to mobilise knowledge in public health policy and practice must take account of this complexity [5, 16,17,18,19].

Systems thinking has gained traction as a valuable approach to tackling complex problems as it offers concepts, tools and frameworks that have the potential to strengthen and complement knowledge mobilisation practice and research [20,21,22]. However, there is a long way to go in exploiting its full potential [23]. This commentary seeks to investigate how systems thinking changes the ways in which knowledge mobilisation is conceptualised and operationalised. We argue that systems thinking can do much more than enhance knowledge mobilisation: it can transform perceptions of what knowledge is, how it is created and valued, and how it is (and could be) used. Our aim is to move beyond description and theory to identify practical strategies, including examples of how systems approaches have been applied. We lay the groundwork by first offering an overview of the relationship between knowledge mobilisation and systems thinking, and then exploring three questions:

  1. 1.

    How can systems thinking advance knowledge mobilisation?

  2. 2.

    What does systems-informed knowledge mobilisation look like in practice?

  3. 3.

    What’s next for advancing systems-informed knowledge mobilisation?

We introduce key concepts, including frameworks for (a) understanding knowledge mobilisation (KM), based on eight empirically-derived archetypes of KM practice [3], and (b) using systems thinking to leverage change in complex systems (Fig. 1). We then describe how these frameworks interact (Table 1)—providing guidance for operationalising the ideas—and draw out some learnings.

Fig. 1
figure1

Leverage points for changing complex systems Adapted from Finegood, Malhi and colleagues [47, 48, 148] and Kania et al. [45]

Table 1 Overview of how systems-focused strategies and intervention points intersect with knowledge mobilisation practicesa

The relationship between KM and systems thinking

What is knowledge mobilisation for public health?

Knowledge mobilisation refers to a broad set of activities that,

“… help move research results into society, as well as bring new ideas into the world of research. From knowledge-brokering and outreach, to more effective dissemination through new technologies, to the ‘co-creation’ of knowledge, these processes help ensure that public investments in ... research have the greatest possible impact—intellectually, socially and economically.” ([1]:12)

In public health, knowledge mobilisation (KM) focuses on impacting public policy as well as health service management and practice. There is widespread agreement that research-infused knowledge should inform policy and practice but systems and structures are often very poor at facilitating this. As Boaz et al argue, “KM is in everyone’s interests and no-one’s job description, and everyone blames everyone else for its absence” [124].

There is also debate about what forms of knowledge should be mobilised, how and with what aims. Echoing the changing discourse about evidence-based policy, the dominant conceptualisation of how knowledge is shared has moved from a linear model towards multi-directional and complexity-attuned approaches where knowledge is produced and becomes meaningful through social processes [3, 27]. The literature on Mode 2 knowledge production [125] and work by Graham, Ward, Nutley, Holmes and others (e.g. [2, 4, 20, 38, 126, 127]) emphasises that knowledge is not independent of systems, processes or people and that context, co-design, power relations and different ways of knowing are key constructs. The evolution of terminology (e.g. transfer, translation, exchange, diffusion, integration and mobilisation) illustrates this well [2] (although the proliferation and inconsistency of terminology has also been noted [128]). Similarly, what is being mobilised is increasingly conceptualised more broadly as knowledge rather than research or evidence. This recognises that policy and practice are necessarily informed by a fusion of locally interpreted information, ideas and values from multiple sources, only some of which stem from academia [129, 130]. We use the term knowledge mobilisation here because it seems the most fluid and inclusive option for capturing what occurs in real-world settings. The influence of, or overlap with, systems thinking can be seen in this evolution but, as we will show, there are further contributions that systems approaches can make to inform or reconceptualise contemporary understandings of KM practice.

What is systems thinking?

Systems thinking in public health is a broad conceptual lens informed by a multidisciplinary body of theories, tools and methods [32, 49, 97, 131]. It posits that the world is comprised of complex systems which are dynamic, interconnected and evolving, and cannot be controlled, so they must be better understood if we are to effect desired change [10, 16, 132, 133].

A system is a perceived collection of interrelated but independent parts that are linked by a common purpose and, through their interactions, function as a whole (see Box 1.). It includes the range of actors, activities and settings that appear to have direct or indirect influence on (or be affected by) a given situation [112, 134]. Systems that impact public health may include communities; coalitions and networks; school, hospitals and other government services; policy agencies that work at local, state and national levels; and broader economic and political systems that impact on how wealth and health are created and distributed.

Systems thinking makes a crucial distinction between simple, complicated and complex phenomena. Where systems (and the problems they produce) are simple or complicated they are relatively linear, deterministic and stable with few interdependencies. In these relatively ‘knowable’ situations, planned KM strategies with fixed outcomes may be most appropriate. Where they are complex, however, they are non-linear and dynamic, adapting unpredictably in response to feedback and thus often undermining ‘logical’ efforts to bring about change [64, 119]. In these situations, systems approaches to KM are likely to be most valuable because they invite a multi-perspective, emergent view of the phenomena.

Systems thinking has an increasingly strong presence in the public health literature with near-exponential growth in recent years [145]. For many, it offers valuable perspectives, tools and strategies for better understanding problems in their settings and, potentially, for strengthening policies so that they are more inclusive, effective and resilient [14,15,16,17, 24, 29, 32, 146,147,148]. However some authors note that the contribution of systems thinking to improving public health quality or intervention effectiveness is not yet clear or convincing, suggesting that it may be a passing fad offering little more than a compelling metaphor [138, 149,150,151]. This may be, in part, because it is currently underutilised and under-studied [133].

Using systems thinking to leverage change in complex systems: a framework

Systems thinking encompasses socio-ecological models within which health problems and solutions are conceptualised as multi-level phenomena involving individuals, groups, organisations and wider systems at sector and societal levels [4]. But it also draws attention to the structures, interactions and forces that operate within and across social layers and shape causality [148]. Systems approaches focus on patterns of inter-relationships, rather than independent forces, as the leverage points for intervention [76, 132, 152].

Malhi and colleagues [47] (who adapted work by Meadows [32, 132]) and Kania et al.,[45] suggest six key areas or leverage points to target when applying a systems approach to organisational change. Each of these areas raises questions for practicing KM:

Paradigm

What are dominant mental models in this system? What assumptions are held about how the system works and should work? How can these values and beliefs best be leveraged (or challenged) in efforts to mobilise knowledge?

Goals

What do people believe are the key aims of this system, its overall purpose? Are the espoused goals congruent with what really drives people? In what ways do current goals and past achievements align with the goals of KM? How can ‘goals of goodness’ (aiming for positive practice) be enhanced in this context?

Structure and rules

What are this system’s organising foundations (including infrastructure, and rules such as policies and guidelines)? How do they shape knowledge practices and opportunities? How do past experiences impact the present and possible futures? What changes in structure or rules might best support the system to self-organise productively in relation to KM?

Relationship and power

What types (and quality) of connections are there within and between people and institutions? How do roles and decision-making powers influence change trajectories? What opportunities are there to create new roles and relationships, leverage existing relationships, and address power imbalances?

Feedback

What forms and channels of information help patterns of change to remain stable in this system? Which cause patterns of change? How can existing feedback channels be used (or new ones created) to generate engagement with KM efforts, and to sustain desired change?

Actors and elements

What people, practices, resources, and physical elements comprise this system? Interventions that target discrete parts of a system are often ineffective in creating system-wide change (or require many actions at this level) so is this the most effective strategy? If so, what else needs to be done?

As Fig. 1 indicates, these areas are conceptualised as a hierarchical framework in which change is harder to achieve at the upper levels but, if successful, is likely to be more transformational. Systems change will usually require action at multiple levels, often simultaneously, but this framework can guide decisions about where best to focus efforts and what sort of strategies might be most successful.

In the next sections we explore what systems-informed KM might look like and offer suggestions as to how that could be operationalised in public health policy and practice.

What is the current relationship between systems thinking and knowledge mobilisation in public health?

Systems thinking has a growing presence in KM discourses [3, 20]. Commentaries recognise that not only are public health problems and solutions shaped by the forces of complex systems, but so is the practice of KM itself. From this perspective, KM is also emergent, highly relational and profoundly context dependent [62]. A systems approach to KM may include identifying who (or what) has the power to influence which forms of knowledge are valued and used, and recognising that KM efforts can create ‘push back’ from a system, resulting in unexpected and counterintuitive outcomes [87]. As we argue below, systems thinking can significantly recalibrate the scope of strategies and targets associated with mobilising knowledge to inform public health policy and practice.

Systems thinking yields strategies that improve the value, reach and impact of knowledge [20, 62, 134, 153]. This includes knowledge about the co-benefits of policies and programs that address the complexity of entangled and syndemic public health problems [10, 32, 33]. Empirical studies indicate that intervention outcomes are positively associated with strategies that address the complexity of processes and conditions they are trying to improve. For example, studies of 124 interventions tackling congestive heart failure, type II diabetes and hospital readmission rates found greater effectiveness when the interventions targeted two or more characteristics of complex systems (including interdependencies between people, resources, processes and structures), and when they allowed the implementation to evolve responsively over time [141, 154, 155]. These intervention designs were congruent with systems principles but not explicitly informed by them. This suggests that systems ideas may be having an ‘enlightenment’ effect [156]: seeping into policies and programs by influencing underlying mental models rather than directing conscious choices.

System thinking has inspired whole-school level co-production approaches to primary prevention. Cluster-randomised controlled trial methods have shown these to have greater effect sizes in reducing adolescent smoking and drug and alcohol use than conventional (health curriculum) methods [157, 158].

Other empirical studies indicate that systems thinking can significantly improve leadership performance and organisational efficiency [159], and is integral to effective project management [160] and organisational crisis responses [161]. Systems approaches can also optimise specific practices such as improving the use of organisational level knowledge [17], co-producing research, network formation, fostering ‘bottom-up’ innovation, implementation and scaling-up [142]. Single studies have found that local solutions implemented flexibly by frontline staff, and supported by reflexive learning and discussion, improved patient safety in five hospitals [94]. Willis et al. describe how systems approaches underpinned a large-scale healthcare transformation tackling improvements in surgical redesign, primary care and quality improvement [98]. Systems thinking has also been used in process evaluation to develop explanatory understandings of complex relationships between individual interactions and health organisation structure and functions [162].

Despite this growing body of evidence about the value of systems approaches for addressing complex problems, and calls for applying system thinking to KM in public health, the actual practice of KM has not necessarily changed [71]. For example, a recent global survey of 106 research organisations found that most of their KM strategies focused on ‘pushing’ their own research (a strategy the authors call “disseminate and hope”) with very little use of strategic ideas from the literature that “take account of competing definitions of knowledge, the internal and external contexts, the parties involved, the organisational factors and the political dynamics” [153]. This may be in part because dominant KM and implementation frameworks do not currently take account of core concepts in systems thinking such as unpredictability and emergence, dynamism and the need for local autonomy [142, 163]. Further, the literature on applying systems approaches to KM in public health policy and practice has few operational models or in-depth case studies, and minimal practical guidance on how it should be done [3, 134]. We seek to address this gap.

1. How can systems thinking advance knowledge mobilisation?

Davies and colleagues [3] describe eight archetypes of KM practice that were derived inductively from an extensive literature review and empirical investigation of practices used in organisations that aim to mobilise knowledge for policy and/or practice (including website review of 186 international agencies, in-depth interviews with agency leaders, a web survey and deliberative workshops with stakeholders). These archetypes integrate the patterns of KM assumptions, activities, configurations and rationales observed across the agencies and can be used to map the breadth of KM and its basic architecture [3]. Impacting policy is considered an outcome rather than a practice in its own right.

We now explore the contribution that systems thinking makes to each of these archetypal practices, and reflecting on how a systems approach might further extend, strengthen or transform them.

Archetype 1: Producing knowledge

This archetype includes the production and dissemination of research-based products such as empirical research papers, systematic reviews, research summaries and syntheses and web portals. Its focus is on instrumental, knowledge-driven problem solving.

Traditionally, knowledge production has been viewed as an exclusively scientific endeavour, separate to the knowledge transfer processes that followed it. The dominant scientific mode (which privileges internal validity, objectivity, reliability and generalisability) has produced robust knowledge that has advanced many areas of public health, but it has some important limitations for addressing policy and practice problems where complexity is a defining characteristic [30, 130, 164,165,166].

With its focus on interaction and dynamism, systems thinking demands that knowledge production grapples with real world contexts [144]. It is methodologically pluralistic and creative, designing new methods to address complexity [23]. It also integrates theories, ideas and views from multiple sources so is usually transdisciplinary and collaborative [24, 62, 71]. For example, the emphasis on ‘What works?’ is shifting to take account of much broader questions exploring social impact, causality and equity [167], and how best to include the voices of people whom this knowledge is intended to benefit [124]. Systems thinking (in common with some other approaches, e.g. participative action research [168]) makes this explicit by asking how knowledge production and use reflects community needs and attends to local and institutionalised values and power inequities [45, 50, 89]. This expands and changes what we focus on, how we conduct the research, how it is integrated with values and other forms of knowledge, how we try to implement it, and who is involved. Ultimately, it reconceptualises what we mean by knowledge and by “good evidence ”[167].

Archetypes 2 and 3: Brokering and intermediation

These two archetypes describe relational models for promoting research flow and use. Key activities include tailored dissemination, education, and the creation of interactive spaces for researchers and policymakers/practitioners. Archetype 2 focuses on brokering new local research, while archetype 3 focuses on brokering wider bodies of existing research.

Intermediaries have a vital role to play in brokering and championing new knowledge and the strategies used to mobilise it within local contexts [17]. This is well established in the wider KM literature (e.g. [169, 170]). Systems thinking adds to this by emphasising that public health problems are the products of complex social, economic, political and institutional forces [10]. This encourages mobilisation efforts to break down silos, integrate different forms of knowledge, and facilitate its use synergistically across different layers and sectors of government [15, 24]. So, from a systems perspective, intermediary roles are less about ‘bridging gaps’ or packaging and ‘transferring’ research findings, and more about ‘blurring the boundaries’ between groups and enabling a more continuous process of knowledge exchange as part of everyday practice [69, 171]. This can involve building capacity, creating hybrid roles, harnessing local champions and communication channels, bringing people together, and developing shared vocabularies, knowledge and objectives. Several studies illustrate the benefits of collaboratively generated visual models and maps as ‘boundary objects’ that provide a focal point for such activities [58, 62, 79].

Archetype 4: Advocating evidence

This archetype describes proselytising for evidence-informed action in which interaction is central. Activities include education, social influence, forging alliances, and the use of incentives and reinforcements. Advocacy recognises that knowledge use is socially situated and so aims to reduce structural, organisational and cultural barriers to desired health outcomes.

Public health advocates tend to be systems thinkers whether or not they use that term. They analyse systems to identify power structures, vested interests and leverage points; mobilise important stakeholders and the media to put pressure on these points; and ‘frame’ their communications to target and change prevailing mental models [90]. For example, the advocacy coalition framework recognises that political change is complex and uncertain, thus the need for “policy learning” and cross-system efforts to influence feedback loops, values and beliefs [81]. The anti-tobacco movement is notable for its success in denormalising smoking using complementary strategies to target legislation, public perceptions and industry misinformation [172]. Such approaches deliberately engage with multiple levels of systems as illustrated in Fig. 1.

Archetypes 5 and 6: Getting research into practice and researching in practice

Archetypes 5 and 6 describe hands-on support for local implementation, developing networks and building local knowledge capacities. Archetype 5 focuses on improving practice through the application of research from outside the organisation where change is sought. This may emphasise ‘transfer’ of explicit knowledge, but its adoption can also include local adaption and contingencies. Archetype 6 focuses on producing knowledge within the organisation. Key activities may include local learning, capacity development and co-production.

KM has tended to be conceptualised as a staged activity with models that focus on rational cognitive processes and linear ‘transmission’ routes (e.g. pipelines, ladders [173, 174]). Increasingly however, the KM literature recognises that the use of knowledge is contingent, dependent on perceptions of credibility, legitimacy and political acceptability which are shaped by epistemological beliefs, ideology and changeable local conditions [175, 176]. As Nutley et al. argue,

… there is no simple answer to the question of what counts as good evidence. It depends on what we want to know, for what purposes, and in what contexts we envisage that evidence being used [130].

Systems thinking builds on these developments by introducing the concepts in Box 1 which have profound implications for development, implementation and evaluation of policies and programs. Interdependence, self-organisation and emergence means that leveraging change in one part of the system will lead to desired outcomes only if the right concurrent shifts happen in the system’s wider relationships and structures [112], and thus mobilisation strategies cannot always be planned and implemented with predictable effects [14, 136]. For example, from a systems perceptive, interventions will scale-up more effectively if they are conceptualised as provisional plans for action that work with variation across local contexts, striving to harness the self-organising and sensemaking capacities within intervention settings [11, 144]. This involves a ‘bottom up’ approach of working closely with stakeholders, fostering local champions and providing strategic feedback for ongoing problem-solving and adaption [104, 144].

A systems lens on archetypes 5 and 6 also changes how we understand and study implementation fidelity. Hawe argues that interventions are not replicated in the same forms across complex systems, rather, what gets transferred are the intervention’s core principles [29] or what some call its “powerful ideas” [177]. This means rejecting the traditional doctrine of precisely implementing intervention components (its form or surface structure) and instead focusing on fostering and evaluating local adaption that is consistent with the intervention’s theory of change (its function or deep structure )[29, 64, 101, 178, 179]. The contribution of systems thinking to this burgeoning “science of intervention adaptation” [178] can be to distil the essence of an intervention, design more fit-for-purpose methods to embed it, enhance its local effects, and help evaluate it [29]. This highlights a need for qualitative methods that can capture and use emergent knowledge generated in the adaptation process (e.g., what is the new knowledge about, for what purpose, in what form, used by whom and how?).

When intervention design and implementation are poorly integrated, interventions may be ill-suited to local populations and settings, resulting in implementers having no sense of investment or ownership, and a poor understanding of how to deliver their core functions [94, 180]. Collaborative problem-identification and intervention design (which includes those implementing and receiving the intervention), together with fostering local adaptation, are systems-informed responses to this enduring challenge [14, 105, 180].

Archetype 7: Fostering networks

This archetype describes the creation, development or facilitation of linkage and collaboration to shape and share expertise and increase the role of research-based knowledge. Knowledge from within, and external to, the organisation/network is used, and attention is paid to tacit and local knowledge. Knowledge production and/or dissemination is combined with social influence and facilitated interaction.

The wider KM literature draws attention to the significant epistemic and cultural differences between the worlds of research, policy and practice [181,182,183], and to the detrimental effects of silos within health research itself [184]. Thus many agree that cross-sector and transdisciplinary networks, including communities of practice, are key to effective KM [26, 42].

Systems thinking adds to this by exploring how networks of people or organisations exchange information to work together, and how network structures change across time and new structures emerge. It highlights the benefits of operating both vertically (connecting layers of government and health systems) and horizontally (connecting sectors) [15, 59]. Strategies for fostering networks build on and ‘nudge’ existing connections and dynamics, and strive to develop feedback mechanisms for engagement and exchange [39, 56]. Viewing formal collaborations as networks also invites boundary critique: asking who and what will be included in the enterprise, and what this means for equity and the values (and ‘facts’) that are represented [49, 50, 185]. This goes well beyond a focus on disciplinary boundaries to consider the institutional, epistemological and political boundaries that impact on knowledge sharing and use [22]. Boundary critique requires explicit justification of choices and fostering participation by stakeholders who may be marginalised by existing structures and processes [50].

Archetype 8: Advancing knowledge mobilisation

This last archetype describes attempts to develop a coherent theoretical foundation for KM and to evidence it with robust empirical evaluation. The aim is to refine the field, build shared understandings, generate commitment to further study and reflexively apply “knowledge about knowing”.

Systems thinking advances KM as a field by shining a lens on the complex, real-world contexts in which knowledge is produced, set in motion and used. For example, it tells us that KM contexts are interdependent and dynamic so that effective KM is likely to be multifaceted, iterative and adaptive. Consequently, these archetypes have fuzzy boundaries and should be employed complementarily depending on circumstances and goals. Systems thinking also goes beyond these archetypal practices. It promotes a paradigm that questions what knowledge is, how it should be generated, the contexts and processes in which it is used, methods for mobilisation, what impacts we can hope to achieve through KM efforts and how they should be evaluated [14, 56, 59, 144, 148, 163]. KM is no longer conceptualised as a discrete piece of work within wider efforts to strengthen public health, but as integral to and in continual dialogue with them [31].

The influence of systems thinking becomes more prominent as these archetypes progress. In the first three (producing knowledge, brokering local research and brokering bodies of research) research knowledge tends to be treated as an exemplary product that has some independent value. This construes KM as a complicated practice in which reduced barriers or enhanced enablers will facilitate knowledge use [186]. In the latter four archetypes, knowledge is positioned as more dynamic and imbued with value (or not) through a social process of negotiated sensemaking and local adaption. Here, context is foregrounded and knowledge is both generated and used by those with very different forms of expertise [187]. Values, rhetoric and tacit knowledge gleaned from personal and professional experiences are part of the mix [11, 144, 188]. These archetypes, therefore, tend to construe KM as complex. This highlights the existing influence of systems ideas but also suggests the need for a coherent systems-informed approach if KM is to achieve its potential.

2. What does systems-informed knowledge mobilisation look like in practice?

Table 1 presents an overview of how systems thinking can inform strategies and intervention points for KM. It explores the archetypal KM practices described above in relation to the systems change framework depicted in Figure 1 and the questions raised by KM in each area of the framework. The purpose is to provide practical guidance about the range of considerations that may be useful when planning, developing and implementing KM strategies for complex problems.

The ideas and recommendations in this table are not new; many of them are currently used by individuals and organisations, albeit variably and often without a systems label. The content is also not definitive; others will have different ideas and may challenge some of our assertions. Many of the concepts and strategies in the table have roots in other traditions and are thus not the sole property of systems thinking, but are potentially repositioned or given new force when KM is viewed through a systems lens [187]. It may be considered somewhat paradoxical to use a table to describe complex interdependent phenomena, but tables are familiar communication devices that enable information to be distilled efficiently. In this table, dotted lines between rows and columns indicate permeable boundaries and potential fuzziness in the distinction between concepts or even in which cell they should sit.

The strategies to support systems-informed KM as outlined in Table 1 should also be underpinned by a healthy humility that recognises the complexity and unpredictability of the systems within which public health problems are created and solved, and how the perspectives of different actors, including ourselves, are invariably limited [97]. This suggests that collaboration/co-production, empowering stakeholders and continual learning about the interactions between context and mobilisation activities are cornerstones of this work. As noted previously, such ideas are not exclusive to systems thinking, but they are elevated by it. We expand on this below with some examples of practical tools and application:

Contextual understanding

Systems thinking moves the context into the foreground [187]. It demands that we understand what local needs look like, the meanings attached to different forms of knowledge, how it moves around the systems, and how knowledge is constrained or gains legitimacy, including key agencies, actors, systems structures and dynamics. This helps the development of contextualised theories of change, indicates possible leverage points and strategies, indicates where best to draw the system’s boundaries and aligns expectations with local realities [14, 59, 64, 66, 78, 112, 116]. It also raises questions about how knowledge claims can be established, e.g. what forms of interrogation about accuracy and authority are appropriate? This nudges the focus of KM strategies towards the more interactive and participatory activities associated with the latter archetypes.

In many cases, contextual understanding will require explicit systems analysis. Examples of systems analysis methods and tools include concept (or systems) mapping [60, 61], dynamic systems modelling [62], and social network analysis [24, 25, 64]. These approaches are often used as the starting point for stakeholders to come together for collaborative KM. When visual representations of systems are co-produced (e.g. behaviour-over-time graphs, causal loops, stock–flow diagrams and simulations) they support joint problem-framing and diagnosis, help identify solutions and motivate those involved to take action on proposed solutions [189].

Collaboration and co-production

A central feature of the systems approach across all the archetypes is cross-sector deliberative collaboration (or co-production) with key stakeholders. This is demanding terrain. Challenges include the need for additional time and resources, and management of power inequities and conflicting values and priorities [190]. Collaboration is not always appropriate or feasible so other forms of participation may be preferable in some circumstances [41]. Some argue that collaborative research has the potential to harm reputations, relationships and research integrity [190]. Empirical studies have found that genuine co-production is hard to achieve in practice [191] and that partnerships can be plagued by distrust, inertia and antagonism [192]. There is little guidance about how best to constitute and support collaborations as (and within) dynamic systems.

Yet others report that collaboration (and closer relationships in general) can combat distrust between epistemic groups [193]. Collaborative relations provide critical nuance by eliciting stakeholders’ tacit, experiential knowledg e[191], increase the perceived relevance, applicability and legitimacy of knowledge thereby facilitating its use in policy and practice, and can increase accountability and investment by those who are best positioned to champion or facilitate strategies at the local level [16, 25, 28, 94, 143, 193,194,195]. The mechanisms for collaboration are relatively well understood [122, 123, 196] and there are many examples of where it has been used effectively for knowledge mobilisation. These include cross-sector partnerships to empower first nations people [51,52,53]; intra-organisational collaboration that generates ‘frontline ownership’ by healthcare providers [94]; co-developed equity indicators used by public health agencies to address social determinants of health [28]; cross-sector social learning platforms that have tackled health promotion, food security, campus health systems and tobacco control [39]; stakeholder co-production of a framework for natural experimental evaluation of a levy on soft drinks [197]; and a national partnership using systems science to prevent chronic disease [40, 123]. Bammer [41] describes tools for identifying levels of collaboration, for stakeholder analysis and knowledge synthesis, and for understanding value conflict and compatibility.

Addressing power imbalances in how knowledge is constituted and legitimised

Systems thinking across all the KM archetypes also invites consideration of the insidious detriments of formal and informal power inequities, including the influence of vested interests [45, 50, 85]. We need to ask questions about how knowledge is constructed, whose knowledge is being mobilised for what purpose and whose gain, and what our own role is within that dynamic [2, 86]. Collaboration with those who are disempowered is essential; especially with people who have been structurally disadvantaged [51,52,53]. Examples at the micro level that illustrate how different perspectives (knowledge) can be brought to the table include paying for first class disability train tickets so that stakeholders with disabilities can participate in workshops [195], using empowerment techniques to support young homeless people to generate service design proposals [50], and linking community members with researchers in a buddy system [54]. At the meso levels, strategies may involve modifying roles or procedures within an organisation [198], establishing advisory bodies comprising members of disenfranchised communities [45], or using citizen juries, deliberative polls or boundary organisations and objects [167]. Beyond this, participatory action research methods are designed to retain the voice and control of those typically marginalised [199].

Adaptive learning

To mobilise knowledge within self-organising, dynamic and unpredictable systems, we need to engage in continuous learning and adaptation to identify and nurture emergent local responses. This is exemplified in archetype 6 (researching in practice) but also applies to brokering knowledge, advocacy, implementation and fostering networks. The value of reflexive learning-in-practice is emphasised by Brown and Duguid [200] who differentiate between modus operandi (the way a task looks to someone working on it as it unfolds over time when “many of the options and dilemmas remain unresolved”) as opposed to opus operatum (the way a finished task appears in hindsight). Opus operatum, they argue, focuses on the task and glosses over the process of doing which is structured by changing contextual conditions. This means that as a work process becomes more complex, our post-hoc depiction of it increasingly obscures what actually needs to be done. Close attention to modus operandi is needed to understand how knowledge is (and can best be) generated, shared and used.

Adaptive learning requires observation and understanding of the flow-on effects and feedback loops of intervention. This incorporates ‘trial and error’ learning, using fit-for-purpose methods to understand the subtleties of the system’s dynamic behaviours [104], and reflexive practices founded on organisational learning principles [76, 89]. Implementation strategies should be loosely specified while attached to clearly defined goals so they can be locally adapted for maximum effect and incorporate ‘bottom up’ expertise and innovatio n[98, 99]. Intervention ‘fidelity’ is thus indicated by the presence of underlying principles, or through the achievement of goals, rather than by faithful adherence to standardised strategie s[64, 201]. Evaluation that informs on-going mobilisation efforts (e.g. developmental evaluation) is warranted [109]. Peters calls this commitment to testing and continuously revising strategies a “scientific habit of mind” [97].

3. What’s next for advancing systems-informed knowledge mobilisation?

There are several challenges in applying systems thinking to real-world KM practice. First, the evidence-base for the effectiveness of systems thinking in KM is nascent and scattered, in part because systems approaches are diverse and applied very differently [202, 203]. There are calls for further empirical research [204] including in-depth case studies and cross-case analyses, realist research and meta-ethnographie s[3, 126, 134, 138, 205]. Most pressingly, we need to answer the question, ‘How do we know when systems thinking has enhanced KM?’ But generating high quality systems evidence may require new methodologies and new ways of thinking about health and its interconnections with society and the environment [33, 197, 206]. As Rutter at a l[197]. argue, “Existing approaches to the generation and use of evidence remain necessary, but are not sufficient”. The turn to realist, narrative and ethnographic methods in public health reflects this shifting terrain, but we are yet to realise the impacts of this next phase of engaging with complexity [207].

Second, system thinking is conceptually and operationally challenging [62]. It counters many familiar approaches [16] and much of the literature is abstract and theoretically dense [131, 137, 150]. Organisational and individual capacities for using systems theory may be limited [55, 208] and impeded by the often confusing terminology and definitions [46, 204]. Policy partners can experience political pressure to identify simple, easy-to-explain and quick-to-implement solutions [46] using familiar methods [175]. Such challenges may be exacerbated by a lack of professional incentives for cross-sector collaboration [142], perceptions that systems approaches are more resource intensive than traditional methods, and the often longer timeframes they require [187, 195].

Third, there are no universal indicators or standards for good systems-informed KM practice. The KM literature often fails to identify tangible or measurable outcomes, or even to explain the nature of the knowledge that is to be mobilised [2]. There is emerging agreement about the scope and tasks of KM (as illustrated by the archetypes described above [3]) but systems thinking indicates that they are not fixed: KM practices need to be deployed in myriad configurations depending on the local context and goals (see also [31, 142] who identify core skills for transdisciplinary KM). Given the degree to which KM is influenced by power relationships and disciplinary traditions—each sensitive to different indicators and outcomes, and focused on different process paths—what counts as success, for whom and why, is invariably a contentious value-laden issue [17, 85, 209]. Some systems thinkers refute the notion of ‘best practice’ within complex, dynamic contexts – an assertion that might equally apply to KM practices [92, 114]. Thus, while many principles for systems-informed KM may be transferable, a suite of tightly specified practices is unlikely to be applicable in all contexts and circumstances [56, 187].

We make four suggestions for advancing systems-informed knowledge mobilisation:

  1. i.

    Be specific about what is meant when using the term ‘systems thinking’. What key ideas or methods are included? For what KM purposes? How are they (to be) operationalised? For example, developing systems maps or models is not the same as co-production - they might be done together or separately but with different likely impacts.

  2. ii.

    Describe counterfactual scenarios in KM so the added value of systems thinking becomes clearer. For example, is a systems approach providing an alternative to a specific model of ‘business-as-usual’ (e.g. one using a linear logic model)? Or is it supplementing (or replacing) another form of collaborative problem solving such as community development?

  3. iii.

    When evaluating KM, cast the net widely to capture impact. Any intervention which activates the agents in a system will have different effect sizes in different contexts, and may also impact different associated problems [29, 210]. This means that effects and the knowledge employed or generated in their production will take time, care and resources to track thoroughly.

  4. iv.

    The uncertainty created by complexity is exacerbated when there is poor understanding of how and why change occurred. Use methods that can track KM effectively in complex systems. A key strategy is to make more visible the mechanisms of KM practice, and the short- and long-term impacts of interventions. Careful monitoring and reporting, theory-driven evaluation and process evaluation that utilises qualitative methods can make important contributions to understanding and explaining what occurred [46, 66, 144, 157, 211].

Conclusion

This paper explores the contribution that systems thinking makes to the planning and implementation of KM in public health policy and practice and suggests areas where there is further potential for value gains. By applying a systems thinking framework to a suite of KM practices, we develop an argument that systems thinking not only enhances but fundamentally transforms KM. It changes what we mean by knowledge, identifying it as a process as much as a transferable product. Knowledge here is pluralistic, informed by multiple parts of the system and reconstituted through use. This contrasts with traditional conceptualisations of exemplary knowledge remaining unchanged through use (fidelity). Systems thinking also changes what we mean by mobilisation – repositioning it as a socially situated, non-prescriptive and potentially destabilising practice that is contingent on addressing power dynamics and on continual adaptive learning about interactions between mobilisation activities and context. KM is no longer conceptualised as a discrete piece of work within wider efforts to strengthen public health, but as integral to and in continual dialogue with those efforts. The challenges presented by systems-informed KM are also very real but, as Braithwaite and colleagues argue, “we must grapple with the world we actually inhabit, not the one we wish we did” [144]. We make four suggestions to facilitate constructive debate and further develop empirical evidence about how systems thinking can enhance our capacity to mobilise knowledge for solving complex problems. These include being specific about what is meant by systems thinking and how it has been operationalised; including counterfactual KM scenarios so the added value of systems thinking is clearer; widening the conceptualisations of impact when evaluating KM to capture a range of systems effects; and using evaluation methods that can effectively track KM in complex systems.

Availability of data and materials

Not applicable.

Abbreviations

KM:

Knowledge mobilisation

References

  1. 1.

    Social Sciences and Humanities Research Council, Framing our direction 2010-2012. Strategic plan., 2008, Government of Canada: Ottowa. Available from: http://www.sshrc-crsh.gc.ca/about-au_sujet/publications/FramingOurDirection_2010-12_final_e.pdf.

  2. 2.

    Ward V. Why, whose, what and how? A framework for knowledge mobilisers. Evid Policy. 2016;13:477–97.

    Article  Google Scholar 

  3. 3.

    Davies HT, Powell AE, Nutley SM. Mobilising knowledge to improve UK health care: learning from other countries and other sectors–a multimethod mapping study. Health Serv Deliv Res. 2015;3.

  4. 4.

    Nutley S, Davies H. Knowledge mobilisation: creating, sharing and using knowledge. In: Orr K, Nutley S, Russell S, Bain R, Hacking B, Moran C, editors. Knowledge and Practice in Business and Organisations. London: Routledge; 2016.

    Google Scholar 

  5. 5.

    Head B, Alford J: Wicked problems: the implications for public management. In Presentation to panel on public management in practice, International Research Society for Public Management 12th Annual Conference; 26-28 March; Brisbane 2008: 26-28.

  6. 6.

    Lindblom CE. The science of "muddling through". Public Adm Rev. 1959:79–88.

  7. 7.

    Jones H. Taking responsibility for complexity: How implementation can achieve results in the face of complex problems. In: Working Paper 330. London: Overseas Development Insititute; 2011.

    Google Scholar 

  8. 8.

    Begun JW, Zimmerman B, Dooley K. In: Mick S, Wyttenbach M, editors. Health care organizations as complex adaptive systems. In Advances in health care organization theory, vol. 253. San Francisco: Jossey-Bass; 2003. p. 253–88.

    Google Scholar 

  9. 9.

    Sweeney K, Griffiths F. Complexity and healthcare: an introduction. Oxford: Radcliffe; 2002.

    Google Scholar 

  10. 10.

    Sterman JD. Learning from evidence in a complex world. Am J Public Health. 2006;96:505–14.

    PubMed  PubMed Central  Article  Google Scholar 

  11. 11.

    Lanham HJ, Leykum LK, Taylor BS, McCannon CJ, Lindberg C, Lester RT. How complexity science can inform scale-up and spread in health care: understanding the role of self-organization in variation across local contexts. Soc Sci Med. 2013;93:194–202.

    PubMed  Article  Google Scholar 

  12. 12.

    Garside R, Pearson M, Hunt H, Moxham T, Anderson R: Preventing obesity using a ‘whole system’ approach at local and community level: PDG1. A report commissioned by NICE Centre for Public Health Excellence. Peninsula Technology Assessment Group (PenTAG), Peninsula Medical School, Universities of Exeter and Plymouth NICE Centre for Public Health Excellence 2010.

  13. 13.

    Plsek PE, Greenhalgh T. The challenge of complexity in health care. BMJ. 2001;323:625.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  14. 14.

    de Savigny D, Taghreed A. Systems thinking for health systems strengthening. Alliance for Health Policy and Systems Research: World Health Organization; 2009.

    Google Scholar 

  15. 15.

    Kickbusch I, Gleicher D. Governance for health in the 21st century: World Health Organization; 2012.

  16. 16.

    Best A, Holmes B. Systems thinking, knowledge and action: towards better models and methods. Evid Policy. 2010;6:145–59.

    Article  Google Scholar 

  17. 17.

    Cherney A, Head B. Supporting the knowledge-to-action process: a systems-thinking approach. Evid Policy. 2011;7:471–88.

    Article  Google Scholar 

  18. 18.

    OECD, Applications of Complexity Science for Public Policy: New tools for finding unanticipated consequences and unrealized opportunities, Global Science Forum Workshop 5-7 Oct 2008, 2009: Ettore Majorana International Centre for Scientific Culture, Erice, Sicily Available from: http://www.oecd.org/science/sci-tech/43891980.pdf.

  19. 19.

    Sanderson I. Complexity, 'practical rationality' and evidence-based policy making. Policy Polit. 2006;34:115–32.

    Article  Google Scholar 

  20. 20.

    Holmes BJ, Best A, Davies H, Hunter D, Kelly MP, Marshall M, Rycroft-Malone J. Mobilising knowledge in complex health systems: a call to action. Evid Policy. 2017;13:539–60.

    Article  Google Scholar 

  21. 21.

    Finegood D, Holmes B. Practical strategies to mobilise knowledge in complex systems. Sydney: The Australian Prevention Partnership Centre; 2017. Available from: https://www.youtube.com/watch?v=klkZBkw7eiA&feature=youtu.be. Accessed: 5 Jun 2019.

    Google Scholar 

  22. 22.

    Smith KE, Joyce KE. Capturing complex realities: understanding efforts to achieve evidence-based policy and practice in public health. Evid Policy. 2012;8:57–78.

    Article  Google Scholar 

  23. 23.

    Luke DA, Stamatakis KA. Systems science methods in public health: dynamics, networks, and agents. Annu Rev Public Health. 2012;33:357–76.

    PubMed  PubMed Central  Article  Google Scholar 

  24. 24.

    Leischow SJ, Best A, Trochim WM, Clark PI, Gallagher RS, Marcus SE, Matthews E. Systems thinking to improve the public’s health. Am J Prev Med. 2008;35:S196–203.

    PubMed  PubMed Central  Article  Google Scholar 

  25. 25.

    Willis CD, Mitton C, Gordon J, Best A. System tools for system change. BMJ Qual Saf. 2012;21:250–62.

    PubMed  Article  PubMed Central  Google Scholar 

  26. 26.

    Mabry PL, Olster DH, Morgan GD, Abrams DB. Interdisciplinarity and systems science to improve population health: a view from the NIH Office of Behavioral and Social Sciences Research. Am J Prev Med. 2008;35:S211–24.

    PubMed  PubMed Central  Article  Google Scholar 

  27. 27.

    Weber EP, Khademian AM. Wicked Problems, Knowledge Challenges, and Collaborative Capacity Builders in Network Settings. Public Adm Rev. 2008;68:334–49.

    Article  Google Scholar 

  28. 28.

    Beckett K, Farr M, Kothari A, Wye L, le May A. Embracing complexity and uncertainty to create impact: exploring the processes and transformative potential of co-produced research through development of a social impact model. Health Res Policy Syst. 2018;16:118.

    PubMed  PubMed Central  Article  Google Scholar 

  29. 29.

    Hawe P. Lessons from complex interventions to improve health. Annu Rev Public Health. 2015;36:307–23.

    PubMed  Article  PubMed Central  Google Scholar 

  30. 30.

    Green LW, Ottoson JM, Garcia C, Hiatt RA. Diffusion Theory and Knowledge Dissemination, Utilization, and Integration in Public Health. Annu Rev Public Health. 2009;30:151–74.

    PubMed  Article  Google Scholar 

  31. 31.

    Murphy K, Wolfus B, Lofters A. From complex problems to complex problem-solving: transdisciplinary practice as knowledge translation. In: Kirst M, Schaefer-McDaniel N, Hwang S, O'Campo P, editors. Converging disciplines: a transdisciplinary research approach to urban health problems: Springer; 2011. p. 111–29.

  32. 32.

    Meadows D. Thinking in systems: a primer. London: Chelsea Green; 2008.

    Google Scholar 

  33. 33.

    Swinburn BA, Kraak VI, Allender S, Atkins VJ, Baker PI, Bogard JR, Brinsden H, Calvillo A, De Schutter O, Devarajan R, et al. The Global Syndemic of Obesity, Undernutrition, and Climate Change: The Lancet Commission report. Lancet. 2019; Available from.

  34. 34.

    Sawin E. The magic of 'multisolving': six principles and practices to unlock cross-sectoral collaboration. In: Stanford Social Innovation Review; 2018.

    Google Scholar 

  35. 35.

    Sawin E, McCauley S, Edberg S, Mwaura G, Gutierrez MJ, Multisolving at the intersection of health and climate: lessons from success stories, 2018, Climate Interactive. Available from: https://img.climateinteractive.org/wp-content/uploads/2018/02/Multisolving-at-the-Intersection-of-Health-and-Climate-1.pdf.

  36. 36.

    Markides C. Crossing the Chasm: How to Convert Relevant Research Into Managerially Useful Research. J Appl Behav Sci. 2010;47:121–34.

    Article  Google Scholar 

  37. 37.

    Cairney P, Oliver K. If scientists want to influence policymaking, they need to understand it: The Guardian Guardian Media Group; 2016.

  38. 38.

    Bowen SJ, Graham ID. From Knowledge Translation to Engaged Scholarship: Promoting Research Relevance and Utilization. Arch Phys Med Rehabil. 2013;94:S3–8.

    PubMed  Article  Google Scholar 

  39. 39.

    Norman CD, Charnaw-Burger J, Yip AL, Saad S, Lombardo C. Designing health innovation networks using complexity science and systems thinking: the CoNEKTR model. J Eval Clin Pract. 2010;16:1016–23.

    PubMed  Article  Google Scholar 

  40. 40.

    Wilson A, Wutzke S, Overs M. The Australian Prevention Partnership Centre: systems thinking to prevent lifestyle-related chronic illness. Public Health Res Pract. 2014;25:e2511401.

    PubMed  Article  Google Scholar 

  41. 41.

    Bammer G. Key issues in co-creation with stakeholders when research problems are complex. Evid Policy. 2019.

  42. 42.

    Fitzgerald L, Harvey G. Translational networks in healthcare? Evidence on the design and initiation of organizational networks for knowledge mobilization. Soc Sci Med. 2015;138:192–200.

    PubMed  Article  Google Scholar 

  43. 43.

    Equal Measure | Harder+Company, Cultivating systems leadership in cross-sector partnerships: lessons from the linked learning regional hubs of excellence, 2017, James Irvine Foundation. Available from: http://www.equalmeasure.org/wp-content/uploads/2017/08/Systems-Leadership-Issue-Brief-081017-FINAL.pdf.

  44. 44.

    Davidson Knight A, Lowe T, Brossard M, Wilson J. A whole new world: funding in complexity: Collaborate, Newcastle University; 2017. Available from: https://marcusjenal.wordpress.com/2017/05/17/a-whole-new-world-funding-in-complexity/.

  45. 45.

    Kania J, Kramer M, Senge P, The water of systems change, 2018, FSG. Available from: www.fsg.org/publications/water_of_systems_change?utm_source=newsletter&utm_medium=email&utm_content=Read%20the%20article&utm_campaign=20180604waterofsystemschangeall.

  46. 46.

    Walton M. Expert views on applying complexity theory in evaluation: opportunities and barriers. Evaluation. 2016;22:410–23.

    Article  Google Scholar 

  47. 47.

    Malhi L, Karanfil Ö, Merth T, Acheson M, Palmer A, Finegood DT. Places to intervene to make complex food systems more healthy, green, fair, and affordable. J Hunger Environ Nutr. 2009;4:466–76.

    PubMed  PubMed Central  Article  Google Scholar 

  48. 48.

    Johnston LM, Matteson CL, Finegood DT. Systems science and obesity policy: a novel framework for analyzing and rethinking population-level planning. Am J Public Health. 2014;104:1270–8.

    PubMed  PubMed Central  Article  Google Scholar 

  49. 49.

    Reynolds M, Holwell S: Introducing systems approaches. In Systems approaches to managing change: a practical guide. Edited by Reynolds M, Holwell S. Milton Keynes: Springer; 2010: 1-23.

  50. 50.

    Midgley G. Systemic intervention for public health. Am J Public Health. 2006;96:466–72.

    PubMed  PubMed Central  Article  Google Scholar 

  51. 51.

    Armitage D, Berkes F, Dale A, Kocho-Schellenberg E, Patton E. Co-management and the co-production of knowledge: Learning to adapt in Canada's Arctic. Glob Environ Chang. 2011;21:995–1004.

    Article  Google Scholar 

  52. 52.

    Sherriff SL, Miller H, Williamson A, Tong A, Muthayya S, Redman S, Bailey S, Eades S, Haynes A. Building trust and sharing power for co-creation in Aboriginal health research: a stakeholder interview study. Evid Policy. 2019.

  53. 53.

    Hernandez A, Ruano AL, Marchal B, San Sebastian M, Flores W. Engaging with complexity to improve the health of indigenous people: a call for the use of systems thinking to tackle health inequity. Int J Equity Health. 2017;16.

  54. 54.

    Abercrombie R, Boswell K, Thomasoo R. Thinking big: how to use theory of change for systems change. UK: Lankelly Chase Foundation; 2018.

    Google Scholar 

  55. 55.

    Holmes BJ, Noel K. Time to shift from systems thinking-talking to systems thinking-action: Comment on" Constraints to applying systems thinking concepts in health systems: A regional perspective from surveying stakeholders in Eastern Mediterranean countries". Int J Health Policy Manag. 2015;4:245.

    PubMed  PubMed Central  Article  Google Scholar 

  56. 56.

    Best A, Terpstra JL, Moor G, Riley B, Norman CD, Glasgow RE. Building knowledge integration systems for evidence-informed decisions. J Health Organ Manag. 2009;23:627–41.

    PubMed  Article  PubMed Central  Google Scholar 

  57. 57.

    Datta A, Shaxson L, Pellini A. Capacity, complexity and consulting. In: Lessons from managing capacity development projects. London: Overseas Development Institute; 2012.

    Google Scholar 

  58. 58.

    Zurcher KA, Jensen J, Mansfield A. Using a Systems Approach to Achieve Impact and Sustain Results. Health Promot Pract. 2018;19:15S–23S.

    PubMed  Article  PubMed Central  Google Scholar 

  59. 59.

    Gates EF. Making sense of the emerging conversation in evaluation about systems thinking and complexity science. Eval Program Plann. 2016;59:62–73.

    PubMed  Article  PubMed Central  Google Scholar 

  60. 60.

    Egan M, McGill E, Penney T, de Cuevas RA, Er V, Orton L, White M, Lock K, Cummins S, Savona N. What to consider when planning a systems evaluation. Guidance on Systems Approaches to Local Public Health Evaluation. Part 2: National Institute for Health Research School for Public Health Research; 2019. Available from.

  61. 61.

    Sturgiss E, Luig T, Campbell-Scherer DL, Lewanczuk R, Green LA. Using Concept Maps to compare obesity knowledge between policy makers and primary care researchers in Canada. BMC Res Notes. 2019;12:23.

    PubMed  PubMed Central  Article  Google Scholar 

  62. 62.

    Freebairn L, Rychetnik L, Atkinson J-A, Kelly P, McDonnell G, Roberts N, Whittall C, Redman S. Knowledge mobilisation for policy development: implementing systems approaches through participatory dynamic simulation modelling. Health Res Policy Syst. 2017;15:83.

    PubMed  PubMed Central  Article  Google Scholar 

  63. 63.

    Freebairn L, Atkinson J-A, Osgood ND, Kelly PM, McDonnell G, Rychetnik L. Turning conceptual systems maps into dynamic simulation models: An Australian case study for diabetes in pregnancy. PLoS One. 2019;14:e0218875.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  64. 64.

    Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43:267–76.

    PubMed  Article  Google Scholar 

  65. 65.

    Langeveld K, Stronks K, Harting J. Use of a knowledge broker to establish healthy public policies in a city district: a developmental evaluation. BMC Public Health. 2016;16:271.

    PubMed  PubMed Central  Article  Google Scholar 

  66. 66.

    Langlois EV, Becerril Montekio V, Young T, Song K, Alcalde-Rabanal J, Tran N. Enhancing evidence informed policymaking in complex health systems: lessons from multi-site collaborative approaches. Health Res Policy Syst. 2016;14:20.

    PubMed  PubMed Central  Article  Google Scholar 

  67. 67.

    Harvey G. The many meanings of evidence: implications for the translational science agenda in healthcare. Int J Health Policy Manag. 2013;1:187–8.

    PubMed  PubMed Central  Article  Google Scholar 

  68. 68.

    McCormack B, Rycroft-Malone J, DeCorby K, Hutchinson AM, Bucknall T, Kent B, Schultz A, Snelgrove-Clarke E, Stetler C, Titler M, et al. A realist review of interventions and strategies to promote evidence-informed healthcare: a focus on change agency. Implement Sci. 2013;8:107.

    PubMed  PubMed Central  Article  Google Scholar 

  69. 69.

    Evans S, Scarbrough H. Supporting knowledge translation through collaborative translational research initiatives: ‘Bridging’ versus ‘blurring’ boundary-spanning approaches in the UK CLAHRC initiative. Soc Sci Med. 2014;106:119–27.

    PubMed  PubMed Central  Article  Google Scholar 

  70. 70.

    Bruce A, O'Callaghan K. Inside out: knowledge brokering by short-term policy placements. Evid Policy. 2016;12:363–80.

    Article  Google Scholar 

  71. 71.

    Kitson AL. The need for systems change: reflections on knowledge translation and organizational change. J Adv Nurs. 2009;65:217–28.

    PubMed  Article  PubMed Central  Google Scholar 

  72. 72.

    Cassidy CE, Burgess S, Graham ID. It’s All About the IKT Approach: Three Perspectives on an Embedded Research Fellowship; Comment on “CIHR Health System Impact Fellows: Reflections on ‘Driving Change’ Within the Health System”. Int J Health Policy Manag. 2019;8:455–8.

    PubMed  PubMed Central  Article  Google Scholar 

  73. 73.

    Kislov R, Hodgson D, Boaden R. Professionals as knowledge brokers: The limits of authority in healthcare collaboration. Public Adm. 2016;94:472–89.

    Article  Google Scholar 

  74. 74.

    Lockett A, El Enany N, Currie G, Oborn E, Barrett M, Racko G, Bishop S, Waring J. A formative evaluation of Collaboration for Leadership in Applied Health Research and Care (CLAHRC): institutional entrepreneurship for service innovation. Health Serv Deliv Res. 2014;2.

  75. 75.

    Frost H, Geddes R, Haw S, Jackson CA, Jepson R, Mooney JD, Frank J. Experiences of knowledge brokering for evidence-informed public health policy and practice: three years of the Scottish Collaboration for Public Health Research and Policy. Evid Policy. 2012;8:347–59.

    Article  Google Scholar 

  76. 76.

    Senge PM: The fifth discipline: The art and practice of the learning organization. Broadway Business; 2006.

    Google Scholar 

  77. 77.

    Bornbaum CC, Kornas K, Peirson L, Rosella LC. Exploring the function and effectiveness of knowledge brokers as facilitators of knowledge translation in health-related settings: a systematic review and thematic analysis. Implement Sci. 2015;10:162.

    PubMed  PubMed Central  Article  Google Scholar 

  78. 78.

    Best A, Hiatt RA, Norman CD. Knowledge integration: conceptualizing communications in cancer control systems. Patient Educ Couns. 2008;71:319–27.

    PubMed  Article  Google Scholar 

  79. 79.

    Northridge ME, Metcalf SS. Enhancing implementation science by applying best principles of systems science. Health Res Policy Syst. 2016;14:74.

    PubMed  PubMed Central  Article  Google Scholar 

  80. 80.

    Farrer L, Marinetti C, Cavaco YK, Costongs C. Advocacy for Health Equity: A Synthesis Review. Milbank Q. 2015;93:392–437.

    PubMed  PubMed Central  Article  Google Scholar 

  81. 81.

    Sabatier PA, Weible CM. In: Sabatier PA, editor. The advocacy coalition framework. In Theories of the Policy Process, vol. 2: Westview Press; 2007. p. 189–220.

  82. 82.

    Kingdon JW. Agendas, alternatives, and public policies (2nd Edition). New York: Longman; 2003.

    Google Scholar 

  83. 83.

    Cairney P, Kwiatkowski R. How to communicate effectively with policymakers: combine insights from psychology and policy studies. Palgrave Commun. 2017;3:37.

    Article  Google Scholar 

  84. 84.

    Herbert C, Best A. It's a matter of values: partnership for innovative change. HealthcarePapers. 2011;11:31–7.

    PubMed  Article  PubMed Central  Google Scholar 

  85. 85.

    Murphy K, Fafard P. Taking Power, Politics, and Policy Problems Seriously. J Urban Health. 2012;89:723–32.

    PubMed  PubMed Central  Article  Google Scholar 

  86. 86.

    Weyrauch V, Echt L, Suliman S. Knowledge into policy: going beyond ‘context matters’: International Network for the Availability of Scientific Publications; 2016. Available from: http://www.politicsandideas.org/wp-content/uploads/2016/07/Going-beyond-context-matters-Framework_PI.compressed.pdf.

  87. 87.

    Mayne R, Green D, Guijt I, Walsh M, English R, Cairney P. Using evidence to influence policy: Oxfam’s experience. Palgrave Commun. 2018;4:122.

    Article  Google Scholar 

  88. 88.

    Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J. Knowledge Transfer Study G: How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q. 2003;81:221–172.

    PubMed  PubMed Central  Article  Google Scholar 

  89. 89.

    Hummelbrunner R. Learning, systems concepts and values in evaluation: proposal for an exploratory framework to improve coherence. Inst Dev Stud (IDS) Bull. 2015;46:17–29.

    Article  Google Scholar 

  90. 90.

    Chapman S. Advocacy for public health: a primer. J Epidemiol Community Health. 2004;58:361–5.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  91. 91.

    Boswell C, Smith K. Rethinking policy ‘impact’: four models of research-policy relations. Palgrave Commun. 2017;3:44.

    Article  Google Scholar 

  92. 92.

    Snowden DJ, Boone ME. A leader's framework for decision making. Harv Bus Rev. 2007;85:68.

    PubMed  PubMed Central  Google Scholar 

  93. 93.

    Riley BL, Robinson KL, Gamble J, Finegood DT, Sheppard D, Penney TL, Best A. Knowledge to action for solving complex problems: insights from a review of nine international cases. Health Promot Chronic Dis Prev Can. 2015;35:47–53.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  94. 94.

    Zimmerman B, Reason P, Rykert L, Gitterman L, Christian J, Gardam M. Front-line ownership: generating a cure mindset for patient safety. HealthcarePapers. 2013;13:6.

    PubMed  Article  PubMed Central  Google Scholar 

  95. 95.

    Head BW, Alford J. Wicked Problems: Implications for Public Policy and Management. Adm Soc. 2015;47:711–39.

    Article  Google Scholar 

  96. 96.

    Malakellis M, Hoare E, Sanigorski A, Crooks N, Allender S, Nichols M, Swinburn B, Chikwendu C, Kelly PM, Petersen S. School-based systems change for obesity prevention in adolescents: outcomes of the Australian Capital Territory ‘It's Your Move!’. Aust N Z J Public Health. 2017;41:490–6.

    PubMed  Article  Google Scholar 

  97. 97.

    Peters DH. The application of systems thinking in health: why use systems thinking? Health Res Policy Syst. 2014;12:51.

    PubMed  PubMed Central  Article  Google Scholar 

  98. 98.

    Willis CD, Best A, Riley B, Herbert CP, Millar J, Howland D. Systems thinking for transformational change in health. Evid Policy. 2014;10:113–26.

    Article  Google Scholar 

  99. 99.

    Plsek PE, Wilson T. Complexity, leadership, and management in healthcare organisations. BMJ. 2001;323:746.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  100. 100.

    Pieper S. Good to great in healthcare. Healthc Exec. 2004;19:21–6.

    Google Scholar 

  101. 101.

    Joyce A, Ollis D, Kearney S, Leung L, Foenander E. The influence of contextual factors on implementation fidelity in a whole school approach to prevention of violence against women. Health Promot J Aust. 2019;30:238–45.

    Article  Google Scholar 

  102. 102.

    Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581–629.

    PubMed  PubMed Central  Article  Google Scholar 

  103. 103.

    Jenal M. An alternative to a Theory of Change approach. In: Rumination on systemic economic and social change, vol. 2019; 2016.

    Google Scholar 

  104. 104.

    Paina L, Peters DH. Understanding pathways for scaling up health services through the lens of complex adaptive systems. Health Policy Plan. 2011;27:365–73.

    PubMed  Article  PubMed Central  Google Scholar 

  105. 105.

    Best A, Greenhalgh T, Lewis S, Saul JE, Carroll S, Bitz J. Large-System Transformation in Health Care: A Realist Review. Milbank Q. 2012;90:421–56.

    PubMed  PubMed Central  Article  Google Scholar 

  106. 106.

    Rycroft-Malone J, Burton RC, Wilkinson J, Harvey G, McCormack B, Baker R, Dopson S, Graham ID, Staniszewska S, Thompson C, et al. Collective action for implementation: a realist evaluation of organisational collaboration in healthcare. Implement Sci. 2016;11:1–17.

    Google Scholar 

  107. 107.

    Holmes B. Co-producing health research: saying what we mean, meaning what we say, and learning as we go. Michael Smith Foundation for Health Research: Michael Smith Foundation for Health Research; 2017. Available from: http://www.msfhr.org/news/blog-posts/co-producing-health-research?platform=hootsuite. Accessed: 27 Jul 2017.

  108. 108.

    Laihonen H. A managerial view of the knowledge flows of a health-care system. Knowl Manag Res Pract. 2015;13:475–85.

    Article  Google Scholar 

  109. 109.

    Patton MQ. Developmental evaluation: applying complexity concepts to enhance innovation and use. New York: Guilford Press; 2011.

    Google Scholar 

  110. 110.

    Marshall M, Pagel C, French C, Utley M, Allwood D, Fulop N, Pope C, Banks V, Goldmann A. Moving improvement research closer to practice: the Researcher-in-Residence model. BMJ Qual Saf. 2014;23:801–5.

    PubMed  PubMed Central  Article  Google Scholar 

  111. 111.

    Joyce A, Green C, Kearney S, Leung L, Ollis D. Alignment and political will: upscaling an Australian respectful relationships program. Health Promot Int. 2018;34:892–901.

    Article  Google Scholar 

  112. 112.

    Foster-Fishman PG, Nowell B, Yang H. Putting the system back into systems change: a framework for understanding and changing organizational and community systems. Am J Community Psychol. 2007;39:197–215.

    PubMed  Article  PubMed Central  Google Scholar 

  113. 113.

    Snowden D, Cynefin framework introduction. CognitiveEdge, 2010, CognitiveEdge. Available from: https://cognitive-edge.com/videos/cynefin-framework-introduction/. Accessed: 18 Jun 2019.

  114. 114.

    Ramalingam B, Laric M, Primrose J. From best practice to best fit: understanding and navigating wicked problems in international development. London: Overseas Development Institute; 2014. p. 1–44.

    Google Scholar 

  115. 115.

    Fraser SW, Greenhalgh T. Coping with complexity: educating for capability. BMJ. 2001;323:799–803.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  116. 116.

    Alford C. How systems mapping can help you build a better theory of change. In: In too deep: tools for tackling tough problems, vol. 2019; 2017.

    Google Scholar 

  117. 117.

    Funnell SC, Rogers PJ. Purposeful program theory: effective use of theories of change and logic models: Wiley; 2011.

  118. 118.

    Sniehotta FF, Araújo-Soares V, Brown J, Kelly MP, Michie S, West R. Complex systems and individual-level approaches to population health: a false dichotomy? Lancet Public Health. 2017;2:e396–7.

    PubMed  Article  PubMed Central  Google Scholar 

  119. 119.

    Riley B, Willis C, Holmes B, Finegood D, Best A, McIsaac J. Systems thinking and dissemination and implementation research. In: Brownson R, Coldiitz A, Protor E, editors. Dissemination and Implementation Research in Health: Translating Science to Practice. New York: Oxford University Press; 2018.

    Google Scholar 

  120. 120.

    Brandes U, Robins G, McCranie ANN, Wasserman S. What is network science? Netw Sci. 2013;1:1–15.

    Article  Google Scholar 

  121. 121.

    Bailie R, Matthews V, Brands J, Schierhout G. A systems-based partnership learning model for strengthening primary healthcare. Implement Sci. 2013;8:143.

    PubMed  PubMed Central  Article  Google Scholar 

  122. 122.

    Bryson JM, Crosby BC, Stone MM. Designing and Implementing Cross-Sector Collaborations: Needed and Challenging. Public Adm Rev. 2015;75:647–63.

    Article  Google Scholar 

  123. 123.

    Wutzke S, Rowbotham S, Haynes A, Hawe P, Kelly P, Redman S, Davidson S, Stephenson J, Overs M, Wilson A. Knowledge mobilisation for chronic disease prevention: the case of the Australian Prevention Partnership Centre. Health Res Policy Syst. 2018;16:109.

    PubMed  PubMed Central  Article  Google Scholar 

  124. 124.

    Boaz A, Locock L, Ward V. Whose evidence is it anyway? Evid Policy. 2015;11:145–8.

    Article  Google Scholar 

  125. 125.

    Nowotny H, Scott P, Gibbons M. Introduction: 'Mode 2' revisited: the new production of knowledge. Minerva. 2003;xli:179–94.

    Article  Google Scholar 

  126. 126.

    Rycroft-Malone J. It's more complicated than that: Comment on "Translating evidence into healthcare policy and practice: single versus multi-faceted implementation strategies - is there a simple answer to a complex question?". Int J Health Policy Manag. 2015;4:481–2.

    PubMed  PubMed Central  Article  Google Scholar 

  127. 127.

    Graham ID, Kothari A, McCutcheon C. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation. Implement Sci. 2018;13.

  128. 128.

    McKibbon KA, Lokker C, Wilczynski NL, Ciliska D, Dobbins M, Davis DA, Haynes RB, Straus SE. A cross-sectional study of the number and frequency of terms used to refer to knowledge translation in a body of health literature in 2006: a Tower of Babel? Implement Sci. 2010;5:16.

    PubMed  PubMed Central  Article  Google Scholar 

  129. 129.

    Greenhalgh T, Wieringa S. Is it time to drop the ‘knowledge translation’ metaphor? A critical literature review. J R Soc Med. 2011;104:501–9.

    PubMed  PubMed Central  Article  Google Scholar 

  130. 130.

    Nutley S, Powell A, Davies H. What counts as good evidence? Provocation paper for the Alliance for Useful Evidence: Alliance for Useful Evidence; 2013.

  131. 131.

    Rusoja E, Haynie D, Sievers J, Mustafee N, Nelson F, Reynolds M, Sarriot E, Swanson RC, Williams B. Thinking about complexity in health: A systematic review of the key systems thinking and complexity ideas in health. J Eval Clin Pract. 2018;24:600–6.

    PubMed  Article  PubMed Central  Google Scholar 

  132. 132.

    Meadows D. Leverage points: places to intervene in a system: The Sustainability Institute, Vermont; 1999. Available from: http://drbalcom.pbworks.com/w/file/fetch/35173014/Leverage_Points.pdf.

  133. 133.

    Lich KH, Ginexi EM, Osgood ND, Mabry PL. A call to address complexity in prevention science research. Prev Sci. 2013;14:279–89.

    PubMed  Article  PubMed Central  Google Scholar 

  134. 134.

    Riley B, Norman CD, Best A. Knowledge integration in public health: a rapid review using systems thinking. Evid Policy. 2012;8:417–31.

    Article  Google Scholar 

  135. 135.

    Eppel E, Matheson A, Walton M. Applying Complexity theory to New Zealand public policy: Principles for Practice. Policy Q. 2011;7:48–55.

    Article  Google Scholar 

  136. 136.

    Khan S, Vandermorris A, Shepherd J, Begun JW, Lanham HJ, Uhl-Bien M, Berta W. Embracing uncertainty, managing complexity: applying complexity thinking principles to transformation efforts in healthcare systems. BMC Health Serv Res. 2018;18.

  137. 137.

    Lissack MR. Complexity: the science, its vocabulary, and its relation to organizations. Emergence. 1999;1:110–26.

    Article  Google Scholar 

  138. 138.

    Cairney P. Complexity Theory in Political Science and Public Policy. Polit Stud Rev. 2012;10:346–58.

    Article  Google Scholar 

  139. 139.

    DeCoste S, Puri J. Complexity, climate change and evaluation: IEU Working Paper No. 02. Songdo: Green Climate Fund; 2019. Available from: https://ieu.greenclimate.fund/documents/977793/985626/Working_paper_-_Complexity__climate_change_and_evaluation.pdf/62551804-67f0-0f38-862e-4b185a8c1165.

    Google Scholar 

  140. 140.

    Trochim WM, Cabrera DA, Milstein B, Gallagher RS, Leischow SJ. Practical challenges of systems thinking and modeling in public health. Am J Public Health. 2006;96.

  141. 141.

    Penney LS, Nahid M, Leykum LK, Lanham HJ, Noël PH, Finley EP, Pugh J. Interventions to reduce readmissions: can complex adaptive system theory explain the heterogeneity in effectiveness? A systematic review. BMC Health Serv Res. 2018;18:894.

    PubMed  PubMed Central  Article  Google Scholar 

  142. 142.

    Kitson A, Brook A, Harvey G, Jordan Z, Marshall R, O’Shea R, Wilson D. Using complexity and network concepts to inform healthcare knowledge translation. Int J Health Policy Manag. 2018;7:231.

    PubMed  Article  Google Scholar 

  143. 143.

    Reed JE, Howe C, Doyle C, Bell D. Simple rules for evidence translation in complex systems: a qualitative study. BMC Med. 2018;16.

  144. 144.

    Braithwaite J, Churruca K, Long JC, Ellis LA, Herkes J. When complexity science meets implementation science: a theoretical and empirical analysis of systems change. BMC Med. 2018;16.

  145. 145.

    Chughtai S, Blanchet K. Systems thinking in public health: a bibliographic contribution to a meta-narrative review. Health Policy Plan. 2017;32:585–94.

    PubMed  Google Scholar 

  146. 146.

    OECD. Systems approaches to public sector challenges: working with change: OECD Observatory of Public Sector Innovation; 2017. Available from: http://www.oecd.org/gov/systems-approaches-to-public-sector-challenges-9789264279865-en.htm.

  147. 147.

    Allender S, Brown AD, Bolton KA, Fraser P, Lowe J, Hovmand P. Translating systems thinking into practice for community action on childhood obesity. Obes Rev. 2019:1–6.

  148. 148.

    Holmes B, Finegood D, Riley B, Best A. Systems thinking in dissemination and implementation research. In: Brownson R, Colditz G, Proctor E, editors. Dissemination and Implementation Research in Health: Translating Science To Practice. Oxford: Oxford University Press; 2012.

    Google Scholar 

  149. 149.

    Carey G, Malbon E, Carey N, Joyce A, Crammond B, Carey A. Systems science and systems thinking for public health: a systematic review of the field. BMJ Open. 2015;5:e009002.

    PubMed  PubMed Central  Article  Google Scholar 

  150. 150.

    Burnes B. Complexity theories and organizational change. Int J Manag Rev. 2005;7:73–90.

    Article  Google Scholar 

  151. 151.

    Martin CM, Félix-Bortolotti M. W(h)ither complexity? The emperor's new toolkit? Or elucidating the evolution of health systems knowledge? J Eval Clin Pract. 2010;16:415–20.

    PubMed  Article  PubMed Central  Google Scholar 

  152. 152.

    Shankardass K, O'Campo P, Muntaner C, Bayoumi AM, Kokkinen L. Ideas for Extending the Approach to Evaluating Health in All Policies in South Australia Comment on "Developing a Framework for a Program Theory-Based Approach to Evaluating Policy Processes and Outcomes: Health in All Policies in South Australia". Int J Health Policy Manag. 2018;7:755–7.

    PubMed  PubMed Central  Article  Google Scholar 

  153. 153.

    Powell A, Davies H, Nutley S. Missing in action? The role of the knowledge mobilisation literature in developing knowledge mobilisation practices. Evid Policy. 2017;13:201–23.

    Article  Google Scholar 

  154. 154.

    Leykum LK, Parchman M, Pugh J, Lawrence V, Noël PH, McDaniel RR. The importance of organizational characteristics for improving outcomes in patients with chronic disease: a systematic review of congestive heart failure. Implement Sci. 2010;5:66.

    PubMed  PubMed Central  Article  Google Scholar 

  155. 155.

    Leykum LK, Pugh J, Lawrence V, Parchman M, Noël PH, Cornell J, McDaniel RR. Organizational interventions employing principles of complexity science have improved outcomes for patients with Type II diabetes. Implement Sci. 2007;2:28.

    PubMed  PubMed Central  Article  Google Scholar 

  156. 156.

    Weiss CH. The circuitry of enlightenment: Diffusion of social science research to policymakers. Knowledge. 1986;8:274–81.

    Article  Google Scholar 

  157. 157.

    Bond L, Glover S, Godfrey C, Butler H, Patton GC. Building capacity for system-level change in schools: lessons from the Gatehouse Project. Health Educ Behav. 2001;28:368–83.

    CAS  PubMed  Article  Google Scholar 

  158. 158.

    Patton GC, Bond L, Carlin JB, Thomas L, Butler H, Glover S, Catalano R, Bowes G. Promoting social inclusion in schools: a group-randomized trial of effects on student health risk behavior and well-being. Am J Public Health. 2006;96:1582–7.

    PubMed  PubMed Central  Article  Google Scholar 

  159. 159.

    Skarzauskiene A. Managing complexity: systems thinking as a catalyst of the organization performance. Meas Bus Excell. 2010;14:49–64.

    Article  Google Scholar 

  160. 160.

    Kerzner H. Project management: a systems approach to planning, scheduling, and controlling: Wiley; 2017.

  161. 161.

    Goldberg KI. Crisis decision-making: Understanding the decision-making process during emergencies. CRISIS. 2013;25.

  162. 162.

    Chandler J, Rycroft-Malone J, Hawkes C, Noyes J. Application of simplified Complexity Theory concepts for healthcare social systems to explain the implementation of evidence into practice. J Adv Nurs. 2016;72:461–80.

    PubMed  Article  Google Scholar 

  163. 163.

    Reed JE, Green S, Howe C. Translating evidence in complex systems: a comparative review of implementation and improvement frameworks. Int J Qual Health Care. 2018.

  164. 164.

    Nowotny H. Re-thinking science: from reliable knowledge to socially robust knowledge. In: Lepenies W, editor. In Entangled Histories and Negotiated Universals. Frankfurt: Campus; 2003. p. 14–31.

    Google Scholar 

  165. 165.

    Nutley SM, Walter I, Davies HT. Using evidence: How research can inform public services: Policy press; 2007.

  166. 166.

    Bristow D, Carter L, Martin S. Using evidence to improve policy and practice: the UK What Works Centres. Contemp Soc Sci. 2015;10:126–37.

    Article  Google Scholar 

  167. 167.

    Nutley S, Boaz A, Davies H, Fraser A. What works now? Continuity and change in the use of evidence to improve public policy and service delivery. Public Money Manag. 2019;39:310–6.

    Article  Google Scholar 

  168. 168.

    Mullett J. Issues of equity and empowerment in knowledge democracy: Three community based research examples. Action Res. 2015;13:248–61.

    Article  Google Scholar 

  169. 169.

    Lomas J. The in-between world of knowledge brokering. BMJ. 2007;334:129–32.

    PubMed  PubMed Central  Article  Google Scholar 

  170. 170.

    Ward V, House A, Hamer S. Knowledge brokering: the missing link in the evidence to action chain? Evid Policy. 2009;5:267.

    PubMed  PubMed Central  Article  Google Scholar 

  171. 171.

    King L, Hawe P, Wise M. Making dissemination a two-way process. Health Promot Int. 1998;13:237–44.

    Article  Google Scholar 

  172. 172.

    Chapman S. Public health advocacy and tobacco control: making smoking history: Wiley-Blackwell; 2008.

  173. 173.

    Glasziou P, Haynes B. The paths from research to improved health outcomes. BMJ Evid Based Med. 2005;10:4–7.

    Google Scholar 

  174. 174.

    Landry R, Amara N, Lamari M. Climbing the ladder of research utilization: Evidence from social science research. Sci Commun. 2001;22:396–422.

    Article  Google Scholar 

  175. 175.

    Brown C. The policy agora: how the epistemological and ideological preferences of policy-makers affect the development of government policy. Human Welfare. 2012;1:57–70.

    Google Scholar 

  176. 176.

    Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences on the use of evidence in public health policy. A systematic review. PLoS One. 2013;8:e77404.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  177. 177.

    Miller RL, Shinn M. Learning from communities: overcoming difficulties in dissemination of prevention and promotion efforts. Am J Community Psychol. 2005;35:169–83.

    PubMed  Article  PubMed Central  Google Scholar 

  178. 178.

    Castro FG, Yasui M. Advances in EBI development for diverse populations: towards a science of intervention adaptation. Prev Sci. 2017;18:623–9.

    PubMed  PubMed Central  Article  Google Scholar 

  179. 179.

    Haynes A, Brennan S, Redman S, Williamson A, Gallego G, Butow P. Figuring out fidelity: a worked example of the methods used to identify, critique and revise the essential elements of a contextualised intervention in health policy agencies. Implement Sci. 2016;11:1–18.

    Google Scholar 

  180. 180.

    Spiel C, Schober B, Strohmeier D. Implementing Intervention Research into Public Policy—the “I3-Approach”. Prev Sci. 2018;19:337–46.

    PubMed  Article  Google Scholar 

  181. 181.

    Caplan N. 2-Communities theory and knowledge utilization. Am Behav Sci. 1979;22:459–70.

    Article  Google Scholar 

  182. 182.

    Choi BCK, Pang T, Lin V, Puska P, Sherman G, Goddard M, Ackland MJ, Sainsbury P, Stachenko S, Morrison H, Clottey C. Can scientists and policy makers work together? J Epidemiol Community Health. 2005;59:632.

    PubMed  PubMed Central  Article  Google Scholar 

  183. 183.

    Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, Blachman M, Dunville R, Saul J. Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. Am J Community Psychol. 2008;41:171–81.

    PubMed  Article  Google Scholar 

  184. 184.

    Butler D. Translational research: crossing the valley of death. Nature. 2008;453:840–2.

    CAS  PubMed  Article  Google Scholar 

  185. 185.

    Shiell A, Hawe P, Kavanagh S. Evidence suggests a need to rethink social capital and social capital interventions. Soc Sci Med. 2018; Available online 8 September 2018.

  186. 186.

    Oliver K, Lorenc T, Innvær S. New directions in evidence-based policy research: a critical analysis of the literature. Health Res Policy Syst. 2014;12:34.

    PubMed  PubMed Central  Article  Google Scholar 

  187. 187.

    Hawe P, Bond L, Butler H. Knowledge theories can inform evaluation practice: What can a complexity lens add? N Dir Eval. 2009;2009:89–100.

    Article  Google Scholar 

  188. 188.

    Ferlie E, Crilly T, Jashapara A, Peckham A. Knowledge mobilisation in healthcare: A critical review of health sector and generic management literature. Soc Sci Med. 2012;74:1297–304.

    PubMed  Article  PubMed Central  Google Scholar 

  189. 189.

    Black LJ. When visuals are boundary objects in system dynamics work. Syst Dyn Rev. 2013;29:70–86.

    Article  Google Scholar 

  190. 190.

    Oliver K, Kothari A, Mays N. The dark side of coproduction: do the costs outweigh the benefits for health research? Health Res Policy Syst. 2019;17:33.

    PubMed  PubMed Central  Article  Google Scholar 

  191. 191.

    Williamson A, Tait H, El Jardali F, Wolfenden L, Thackway S, Stewart J, O’Leary L, Dixon J: How are evidence generation partnerships between researchers and policy-makers enacted in practice? A qualitative interview study. Health Research Policy and Systems 2019, 17:41.

  192. 192.

    McCabe KE, Wallace A, Crosland A. A model for collaborative working to facilitate knowledge mobilisation in public health. Evid Policy. 2015;11:559–76.

    Article  Google Scholar 

  193. 193.

    Gollust SE, Seymour JW, Pany MJ, Goss A, Meisel ZF, Grande D. Mutual Distrust: Perspectives From Researchers and Policy Makers on the Research to Policy Gap in 2013 and Recommendations for the Future. INQUIRY: J Health Care Org Provision Financ. 2017;54:0046958017705465.

    Google Scholar 

  194. 194.

    Kislov R, Wilson PM, Knowles S, Boaden R. Learning from the emergence of NIHR Collaborations for Leadership in Applied Health Research and Care (CLAHRCs): a systematic review of evaluations. Implement Sci. 2018;13:111.

    PubMed  PubMed Central  Article  Google Scholar 

  195. 195.

    Filipe A, Renedo A, Marston C. The co-production of what? Knowledge, values, and social relations in health care. PLoS Biol. 2017;15:e2001403.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  196. 196.

    Heaton J, Day J, Britten N. Collaborative research and the co-production of knowledge for practice: an illustrative case study. Implement Sci. 2016;11:1.

    Google Scholar 

  197. 197.

    Rutter H, Savona N, Glonti K, Bibby J, Cummins S, Finegood DT, Greaves F, Harper L, Hawe P, Moore L, et al. The need for a complex systems model of evidence for public health. Lancet. 2017;390:2602–4.

    PubMed  Article  PubMed Central  Google Scholar 

  198. 198.

    March JG, Olsen JP. The uncertainty of the past: organizational learning under ambiguity. Eur J Polit Res. 1975;3:147–71.

    Article  Google Scholar 

  199. 199.

    Israel BA, Schulz AJ, Parker EA, Becker AB. Critical issues in developing and following community-based participatory research principles. In: Community-based participatory research for health: Jossey-Bass; 2008. p. 47–62.

  200. 200.

    Brown JS, Duguid P. Organizational learning and communities-of-practice: toward a unified view of working, learning, and innovation. Organ Sci. 1991;2:40–57.

    Article  Google Scholar 

  201. 201.

    Hawe P, Shiell A, Riley T. Complex interventions: how "out of control" can a randomised controlled trial be? BMJ. 2004;328:1561–3.

    PubMed  PubMed Central  Article  Google Scholar 

  202. 202.

    Hawe P. Minimal, negligible and negligent interventions. Soc Sci Med. 2015;138:265–8.

    PubMed  Article  Google Scholar 

  203. 203.

    Haynes A, Garvey K, Davidson S, Milat A: What can policymakers get out of systems thinking? Policy partners' experiences of a systems-focused research collaboration in preventive health International Journal of Health Policy and Management In Press.

  204. 204.

    Wutzke S, Morrice E, Benton M, Wilson A. Systems approaches for chronic disease prevention: sound logic and empirical evidence, but is this view shared outside of academia? Public Health Res Pract. 2016;26:e2631632.

    Article  Google Scholar 

  205. 205.

    Sautkina E, Goodwin D, Jones A, Ogilvie D, Petticrew M, White M, Cummins S. Lost in translation? Theory, policy and practice in systems-based environmental approaches to obesity prevention in the Healthy Towns programme in England. Health Place. 2014;29:60–6.

    PubMed  Article  PubMed Central  Google Scholar 

  206. 206.

    El-Sayed AM, Galea S. Systems science and population health: Oxford University Press; 2017.

  207. 207.

    Hanlon P, Carlisle S, Hannah M, Reilly D, Lyon A. Making the case for a ‘fifth wave’ in public health. Public Health. 2011;125:30–6.

    CAS  PubMed  Article  Google Scholar 

  208. 208.

    Agyepong IA. “Wood already touched by fire is not hard to set alight”; comment on “Constraints to applying systems thinking concepts in health systems: a regional perspective from surveying stakeholders in Eastern Mediterranean countries”. Int J Health Policy Manag. 2015;4:191–3.

    PubMed  PubMed Central  Article  Google Scholar 

  209. 209.

    Ottoson JM. Knowledge-for-action theories in evaluation: Knowledge utilization, diffusion, implementation, transfer, and translation. N Dir Eval. 2009;2009:7–20.

    Article  Google Scholar 

  210. 210.

    Hawe P, Bond L, Ghali LM, Perry R, Davison CM, Casey DM, Butler H, Webster CM, Scholz B. Replication of a whole school ethos-changing intervention: different context, similar effects, additional insights. BMC Public Health. 2015;15:265.

    PubMed  PubMed Central  Article  Google Scholar 

  211. 211.

    Martin GP, Ward V, Hendy J, Rowley E, Nancarrow S, Heaton J, Britten N, Fielden S, Ariss S. The challenges of evaluating large-scale, multi-partner programmes: the case of NIHR CLAHRCs. Evid Policy. 2011;7:489–509.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This research was supported by The Australian Prevention Partnership Centre through the National Health and Medical Research Council (NHMRC) partnership centre grant scheme (Grant ID: GNT9100003) with the NHMRC, Australian Government Department of Health, ACT Health, Cancer Council Australia, NSW Ministry of Health, South Australian Department for Health and Wellbeing, Tasmanian Department of Health, and VicHealth.

Author information

Affiliations

Authors

Contributions

LR conceived this commentary. AH led the writing in continual collaboration with LR, PH, DF, LF and MI. All authors contributed to the developing manuscript and reviewed and approved the final version.

Corresponding author

Correspondence to Abby Haynes.

Ethics declarations

Ethics approval and consent to participate

Not applicable – no data was collected.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Haynes, A., Rychetnik, L., Finegood, D. et al. Applying systems thinking to knowledge mobilisation in public health. Health Res Policy Sys 18, 134 (2020). https://doi.org/10.1186/s12961-020-00600-1

Download citation

Keywords

  • Systems thinking
  • Knowledge mobilisation
  • Public health
  • Policy-making