What do we think we are doing? How might a clinical information network be promoting implementation of recommended paediatric care practices in Kenyan hospitals?

Evaluations of district hospitals in low- and middle-income countries have shown that the care provided in many clinical areas is of inadequate quality [1–3]. This is often linked to poor infrastructure and limited material and human resources. However, it is also observed that the technical process of care is often inconsistent with evidence-based recommendations that are designed with resource constraints in mind [4]. There are clearly many potential explanations for failing to provide recommended forms of care even when resources are available; these range from inadequate knowledge to poor motivation to embedded practice norms that conflict with recommendations [5–8]. Overcoming such an array of challenges is implicit to calls for improved translation of knowledge into practice or implementation of essential interventions at scale. Many tests of implementation tend to focus on a specific intervention or tackle a single illness or a carefully delineated small set of challenges. Building on earlier work [9, 10], we designed [11], and have now implemented, a possible broader strategy aimed at improving adoption of multiple evidence-based guidelines for paediatric hospital care in Kenya. Here, we summarise our original intentions and outline the activities that have comprised this strategy for a period of over 2 years. We then re-interpret our intervention strategy by reflecting on how its components align with a recently published typology [12] before exploring how these components may leverage change using a second comprehensive framework, the Behaviour Change Wheel [13]. Finally, we consider how we might reframe our strategy in realist terms as a mid-range proposition or theory [14]. As implementers of the strategy ourselves, we aim to make our assumptions more transparent so that these can be contrasted with an independent evaluation of the strategy through the eyes of its targets (work in progress). We hope this presentation of our activities and thinking will also facilitate interpretation of reports more fully describing changes in care within the established Clinical Information Network (CIN), as well as contributing to wider thinking on the nature of clinical networks and their potential role in improvement in low-income countries.

We (ME, PA and GI) first utilised research records, prospectively maintained administrative logs capturing the dates and names of activities, and field diaries kept by key implementers to construct a temporal sequence of key activities that together have constituted network processes over 2.5 years.

Using this characterization of the CIN’s activities we specifically wished to describe the implementation effort as understood by the implementing team itself drawing on literature and our experience of being part of the intervention. In this sense, our efforts are analogous to describing from an ‘insider’ perspective our emergent theory of change based on the reality of implementation. Our specific intention to present an insider perspective was based on a desire to answer the question ‘What do we think we are doing?’ In separate work (ongoing), investigators that are not part of the implementing team are examining the perceptions of intervention recipients. Our rationale for keeping these investigations separate is to explore coherence or divergence in the pictures that emerge that may shed light on the relative importance of specific intervention strategies (for example, whether our categorisation of major and minor is meaningful) and provide some insight into the fidelity of our characterisation of the intervention approach. Therefore, the process of elucidating our understanding of the intervention included only the authors and three further members of the implementation team as participants in an iterative process of review and discussion. During this process, academic colleagues were consulted to help inform our understanding of key literature, to clarify ideas and to help build consensus where necessary. The implementing team were aware of the emerging quantitative data spanning a number of indicators used to measure effects of the CIN that were the subjects of feedback to hospitals. However, this report is not aimed at trying to present or explain these quantitative findings formally. Instead, it is aimed at better defining the intervention so that more formal mixed methods analyses examining intervention effects are possible, analyses that will include insights gained from intervention recipients.

The approach proceeded in stages. Our first step was to critically analyse the CIN activities to distil their central active ingredients using a recently published typology of implementation components as a reference to give greater granularity to this description [12]. To achieve this, the reference material was circulated to the group (n = 9) 2 weeks before one author (ME) circulated an initial proposed listing of CIN intervention strategies. Over a subsequent period of 8 weeks, group members met and made comments on successive drafts of a final listing until consensus was reached. As part of this process we classified intervention strategies employed as either major or minor. We believed major strategies to be a central component of the intervention specifically aligned with CIN activities. Minor strategies were those that the group felt were employed but only partially executed or that were implemented as a by-product of the overarching network approach without being specifically planned. As part of this process, we identified strategies that were clearly not relevant to the CIN as an intervention. In some cases, strategies were discussed as potential minor intervention components but were not felt to meet the criteria of being actively and sufficiently employed within the network. A final category of strategies were those that were historically related to the CIN intervention but not a current active component. In response to peer review comments, this stage of the process was revisited and refined in two ways in a new round of consultation. First, specific reference was made to supplementary material provided by Powell et al. (Appendix 6 linked to [12]) that offers more detail on definitions of the 73 intervention strategies. Second, we adopted the conceptual groupings of all 73 implementation strategies defined by Waltz et al. [20] to organise our final list of the strategies we believe are encompassed in the CIN intervention.

In a next step, we used the Behaviour Change Wheel [13] to propose how the network activities and linked intervention strategies we had articulated may operate to achieve change in frontline health workers behaviour. This exercise was again based on review of the primary reference, reflection and discussion in meetings until consensus was reached on the policy categories, intervention functions and sources of behaviour (as defined by Michie et al. [13]) that we believe are influencing participants in the CIN.

To illustrate from our insider perspective how intervention strategies and behavioural influences may produce effects within the CIN we present results from two specific indicators of practices the CIN was aiming to influence that have been tracked since the inception of the network as simple time-series graphical plots. It is important to note that these are just two measures of many indicators being measured across the illnesses targeted for improved care within the network and are used for illustration only. A fuller account of how quantitative measures of care changed over time will be provided elsewhere. In a final step in the process, presented in the discussion, we (ME and GI) attempted to synthesise in more summary form our understanding of the way our network may be achieving change by formulating a realist context–mechanism–outcome configuration [14].

We have used work on a recently published typology of intervention strategies [12, 20] to characterise a broadly based network intervention approach aimed at improving hospital care for children in Kenya. In doing this, we have illustrated some of the decisions we made, decisions some might contest. Of nine conceptual domains that usefully categorise 73 intervention strategies we feel the CIN approach employs components from across six domains. We were unable to utilise financial strategies and did not attempt to engage consumers in this low-income setting. We felt efforts to tailor the intervention approach were not an active part of the CIN intervention but that this was very much informed by earlier work. In general, the CIN intervention seems to draw most heavily on the conceptual domains ‘Use evaluative and iterative strategies’ (three major components), ‘Develop stakeholder interrelationships’ (four major and four minor components), and ‘Train and educate stakeholders’ (two major and three minor components) from across the 12 major and 10 minor components we identified. In conducting this work a potential limitation of the typology used appeared. It seems that the intervention strategy encompassing altered incentives is defined to specifically include financial incentives (Alter incentive/allowance structures) and is conceptually mapped to the ‘Utilise financial strategies’ domain [12, 20]. We slightly amended the name of this intervention strategy to ‘Alter incentive structure’ to encompass non-financial incentives we and others believe may be important [24]. In the specific case of the CIN we aim to leverage the recognition of senior colleagues and peers linked to reaffirming professional values as incentives.

To help create a bridge between this effort to identify the network intervention’s constituent parts and proposed mechanisms of action we have used the Behaviour Change Wheel framework [13]. This provides one with means of articulating how interventions may influence individuals’ capability, opportunity and motivation to change their behaviour, ultimately the purpose of the network. Our rationale for explaining what we consider to be what we are doing is to enable others to critique our assumptions. As the authors are a core part of the network and implementation team, such external views may be illuminating. We also provide a link between our original efforts to explicate an intervention design to meet the challenges felt to be present in the Kenyan context [11] and the practical consequences of implementing this design thinking. In essence, we are articulating an emerging theory of change from an insider perspective. As we move beyond the examples of change illustrated here to report more formally what changes the network may have enabled across multiple clinical conditions we also hope this articulation of the network processes will help people understand both what was done and consider whether any effects reported in the future are plausibly related to the approach. Additionally, we anticipate that an independent, qualitative evaluation of the network will examine its effects from the perspective of key hospital participants. We wished to state our expectations of how the network may be operating in advance of this evaluation to allow for a more fruitful comparison.

This report is not aimed at presenting or claiming that the network is an effective mechanism for change, although we do present examples of the types of change we are interested in to illustrate the application of intervention strategies. There are clearly many difficulties in making claims of cause and effect convincing; we do intend to present a more specific evaluation of network effects in due course drawing more fully on mixed methods approaches to evaluation. Such mixed methods approaches may help overcome, to some extent, the considerable challenges in meeting current (and appropriate) standards for evidence of cause and effect when examining complex interventions. In the case of a network as intervention it is of course hard to provide an appropriate counterfactual in a temporally parallel experimental design. In essence, we only have one unit of intervention and analysis – the entire network – although this spans multiple individual hospitals. It is not entirely clear that a set of hospitals outside a network are a grouping that provide an appropriate comparison. Clearly, the intervention itself is highly complex and this provides further challenges in teasing out cause and effect relationships [29, 30]. Alternative evaluation designs might include a stepped-wedge strategy to implement the network. However, as one key feature of a network is establishing partnerships, sharing learning and developing a peer-to-peer group, any experimental advantage of this design would likely be undermined by its potentially detrimental effect on the intervention process and thus effect. Time series approaches offer perhaps the best prospect of exploring change and its association with network participation [31], although as networks such as ours tackle a number of diverse areas measured by a multitude of indicators, conducting such studies is also not without challenge. However, rather than evaluating the effect of a network compared with its absence opportunities exist for examining change within the network and spread of innovation over time. Thus, we hope to test the value of alternative feedback strategies, recognised as a current research need [32], by randomising hospitals already within the network to different feedback interventions.

There are, of course, many additional evaluation strategies one can use to explore the effects of our network. Gaining popularity as an approach for examining the success of complex health system change efforts is realistic evaluation [14]. In the United Kingdom, Collaborations for Leadership in Applied Health Research and Care (CLAHRCs) are efforts to create research and practice partnerships to improve services. These efforts have been the subject of research to explore the effectiveness of partners’ relationships and their ability to collaborate and ultimately change care [33]. Arguably, in contrast to the effort to deconstruct a network intervention, as we have done, the focus of realistic evaluation is to synthesise understanding and offer more unifying explanations articulated as context, mechanism and outcome statements. Commonly used to explain what has occurred context, mechanism and outcome statements can be used to propose an explanatory framework or starting theory. Here, adopting the slight modification recently suggested by Dalkin et al. [34], we offer such an explanatory proposition for the network intervention we have initiated (Fig. 2). Our proposition could be evaluated by others interested in an independent exploration of our network’s functions or effects, something we would welcome. Research on networks and their value in health and other areas has been undertaken in high-income settings [17, 33, 35–37] and may be a fruitful area of research in low- and middle-income country settings as we seek solutions to complex healthcare problems.

There are a number of limitations to the work we report. We used only a small sample of people who are all part of the CIN intervention delivery partnership to identify specific intervention strategies. We used an iterative, reflective but largely discursive approach to selecting intervention strategies and the domains of the Behaviour Change Wheel that we felt best mapped to the CIN intervention as enacted. Including a wider range of stakeholders and using a more structured consensus process, such as a Delphi approach, may have been more rigorous and could have produced somewhat different results. However, our aim was to address the question ‘What do we think we are doing’, to provide an insider’s view of an emerging theory of change. Despite these limitations, we hope the report is useful to those considering similar exercises and helps others more fully understand the CIN as a package of interventions.

A network intervention was initially considered a suitable strategy for improving hospital care for children in a low-income setting based on insights from theory and an understanding of context. This intervention has been initiated and we describe here the practical form it is taking and how this can be deconstructed into a set of intervention components. These components we propose combine to help change behaviours especially those of clinical team leaders who, it is argued, are hybrid managers and should be much more engaged in improving services in general and in promoting adoption of evidence-based clinical guidelines in particular in low-income settings such as Kenya. Evaluating the effectiveness in quantitative terms of a network as an intervention is challenging. Although it spans multiple facilities, it should perhaps be considered a single entity. We intend that it does result in measurable change in delivery of recommended forms of care at a meaningful scale. Simultaneously, it should help reorient key actors in the health system to take greater responsibility for service improvement. Determining its value and the potential role of similar networks to help support adoption of recommended practices and health system change would benefit from further independent examination of the intervention approach and its successes and failures that we would welcome.