- Review
- Open access
- Published:
Making change last? Exploring the value of sustainability approaches in healthcare: a scoping review
Health Research Policy and Systems volume 18, Article number: 120 (2020)
Abstract
Background
Numerous models, tools and frameworks have been produced to improve the sustainability of evidence-based interventions. Due to the vast number available, choosing the most appropriate one is increasingly difficult for researchers and practitioners. To understand the value of such approaches, evidence warranting their use is needed. However, there is limited understanding of how sustainability approaches have been used and how they have impacted research or practice. This review aims to consolidate evidence on the application and impact of sustainability approaches in healthcare settings.
Methods
A systematic scoping review was designed to search for peer-reviewed publications detailing the use of sustainability approaches in practice. A 5-stage framework for scoping reviews directed the search strategy, and quality assessment was performed using the Mixed Method Appraisal Tool. Searches were performed through electronic citation tracking and snowballing of references. Articles were obtained through Web of Science, PubMed and Google Scholar. Six outcome variables for sustainability were explored to ascertain impact of approaches.
Results
This review includes 68 articles demonstrating the application of sustainability approaches in practice. Results show an increase in the use of sustainability approaches in peer-reviewed studies. Approaches have been applied across a range of healthcare settings, including primary, secondary, tertiary and community healthcare. Approaches are used for five main purposes, namely analysis, evaluation, guidance, assessment and planning. Results outline benefits (e.g. improved conceptualisation of sustainability constructs and improved ability to interpret sustainability data) and challenges (e.g. issues with approach constructs and difficulty in application) associated with using a sustainability approach in practice. Few articles (14/68) reported the sustainability outcome variables explored; therefore, the impact of approaches on sustainability remains unclear. Additional sustainability outcome variables reported in retrieved articles are discussed.
Conclusions
This review provides practitioners and researchers with a consolidated evidence base on sustainability approaches. Findings highlight the remaining gaps in the literature and emphasise the need for improved rigour and reporting of sustainability approaches in research studies. To guide future assessment and study of sustainability in healthcare settings an updated list of sustainability outcome variables is proposed.
Trial Registration
This review was registered on the PROSPERO database CRD 42016040081 in June 2016.
Background
Healthcare research continuously produces new and innovative findings but there is evidence that these improvements are often forgotten, changed or replaced in practice long before they can reach wider populations [1,2,3,4,5,6]. Therefore, the sustainability of evidence-based interventions has been identified as one of the most critical gaps in implementation science [3, 7, 8].
In healthcare, the term sustainability often refers to broad notions of continuation or permanence over time but the concept of what will be sustained is diverse [1, 9, 10]. While there is currently no consensus in the field on which definition of sustainability to operationalise and advance, for this paper, the comprehensive definition developed by Moore et al. will be used, namely that: “after a defined period of time, a program, clinical intervention, and/or implementation strategies continue to be delivered and/or individual behaviour change is maintained; the program and individual behaviour change may evolve or adapt while continuing to produce benefits for individuals/systems” [11]. This definition allows sustainability to be viewed as both an ‘outcome’ where health benefits and intervention activities are maintained, as well as an ongoing ‘process’, recognising the need to respond and adapt to promote continuation of improved practices, benefits or outcomes [12].
Rising demands and competition for resources have highlighted the need to understand if and how the sustainability of implemented initiatives can be influenced to support the long-term impact of investments [3, 4]. Sustaining improved outcomes and processes is a recognised challenge to healthcare staff and stakeholders. Numerous definitions and determinants of sustainability make it exceedingly complicated for researchers and practitioners to understand or influence sustainability in practice [3, 5, 13,14,15,16]. To address the challenge of conceptualising, measuring and influencing sustainability, many researchers and healthcare practitioners have developed sustainability frameworks, models and tools to assess sustainability in their initiatives or settings [17,18,19,20,21,22,23,24,25,26]. (The term ‘sustainability approaches’ will be used throughout this paper to refer to sustainability models, frameworks, tools, checklists, processes, strategies and conceptualisations [27]).
A 2018 systematic review identified 62 sustainability approaches designed for use in healthcare settings [27]. The review found that sustainability approaches may offer a means of connecting findings across diverse sustainability studies by providing explicit definitions and foundations for measurement but it also concluded that further research was needed to understand the application of these approaches in practice to assess their potential impact on sustainability processes and outcomes [17, 19, 27]. Although there is evidence of mounting interest in using sustainability approaches, little work has been conducted to understand how approaches have been used and how they have impacted research or practice [28]. This has resulted in seemingly random selection of approaches across research studies driven by ‘convenience or prior exposure’ [29,30,31]. This issue highlights an ongoing gap in knowledge between the development of frameworks, models or tools and the understanding of how they are used in and impact research and practice [29].
In order to understand the value of approaches such as models and frameworks, researchers and practitioners require supporting evidence to warrant their use [32]. Clear descriptions of how approaches inform planning and conduct of sustainability research, along with the potential challenges of use, would provide insight into which approaches have demonstrated positive impact for different programmes and settings [3, 7, 19, 27, 33, 34]. Combining this evidence across sustainability approaches may also support further theory development to create improved frameworks and models for sustainability [19, 35, 36].
In the available literature, it is also not currently known if or how sustainability approaches are contributing to sustainability outcomes or improving sustainability reporting. Proctor et al. defined ‘sustainability outcomes’ as the “subsequent impact (healthcare improvement or public health outcomes) of sustained intervention use” [37]. To understand the impact of the approaches, more rigorous measurement and conceptualisation of sustainability outcomes is needed for the ‘methodological objectivity and interpretability’ of findings [3, 17]. Scheirer and Dearing suggest six sustainability outcome variables [17], namely (1) continuation of benefits or outcomes for consumers, clients or patients; (2) continuation of programme activities or components of the original intervention; (3) maintenance of partnerships developed during the funded programme; (4) maintenance of new organisational practices, procedures and policies; (5) sustaining attention to the issue or problem; and (6) programme diffusion and replication in other sites.
These variables provide a quantifiable way to understand the success of a project or intervention [17, 36]. Assessing the impact of sustainability approaches on these variables will support a nuanced understanding of how sustainability approaches contribute to sustainability outcomes, aiding researchers and practitioners to critically appraise their value to research and implementation efforts.
Aims and research questions
This review aims to consolidate the current evidence on the application and impact of sustainability approaches in healthcare settings, including the benefits and challenges that arise through their use, and explore the sustainability outcome variables they have supported. Two research questions will be explored – (1) What sustainability approaches have demonstrated application in healthcare practice within peer-reviewed publications? (2) What is the value of using a sustainability approach? What are the documented benefits and challenges associated with the use of sustainability approaches? What impact have approaches shown on sustainability outcomes variables?
Methods
A previous systematic review identified 62 papers that described sustainability approaches created for use in healthcare settings [27]. The sustainability approaches come in a variety of forms, including 32 frameworks, 16 models, 8 tools, 4 guidance strategies, 1 checklist and 1 process [27]. Full details on the sustainability approaches, including their constructs, scoring mechanisms and development, can be found here (https://doi.org/10.1186/s13012-017-0707-4) [27]. This scoping review builds on these findings to explore the application and impact of these sustainability approaches in healthcare settings.
A systematic scoping review was designed to explore the available evidence on the application and impact of sustainability approaches in healthcare. To ensure overall quality and coverage of the literature, authors developed the search strategy in collaboration with a medical librarian at their academic institution. A scoping review was deemed to be the appropriate method as it allowed for examination of the “extent, range and nature of research activity” in the use and impact of sustainability approaches [38]. This method specifically supported authors to track the application of the 62 sustainability approaches through citation tracking and snowballing (the 62 approach articles which were tracked through the literature are listed in Additional file 1) [27].
To ensure rigorous documentation of methods, Arksey and O’Malley’s five-stage framework for scoping reviews was followed and is described below [38]. PRISMA guidelines were also used to ensure clarity of reporting [39, 40]. The PRISMA extension for Scoping Reviews Checklist can be found in Additional file 2.
Research question(s)
The research questions were established through discussion between authors after consolidating the results from the previous systematic review, which demonstrated gaps in the current literature regarding the application and impact of sustainability approaches in healthcare settings.
Relevant studies
Citation tracking and snowballing for each of the 62 approaches was used to determine their use in peer-reviewed publications [41]. The search was carried out between May and August 2018, with citation tracking and snowballing taking place on Web of Science and Google Scholar. This was done by putting the title and author of the sustainability approach paper into the database and selecting the ‘times cited’ or ‘cited by’ icon. Once citations were tracked for all 62 papers, we moved onto the next stage of the methodology consisting of screening the titles and abstracts.
Data collection process and study selection
We sought peer-reviewed publications detailing the application of a sustainability approach within healthcare settings. The level of use and rationale for application were not specified but could include assessment, planning, evaluation, monitoring, prediction or testing of sustainability. Papers published in peer-reviewed journals introducing a description of the approach used as well as clear information about application were included. Papers with vague or unclear descriptions on the approach used or application were excluded. For example, papers mentioning or introducing a sustainability approach but not applying it within their studies were excluded. Papers published in languages other than English were excluded. Commentaries, research protocols, conference proceedings, editorials and perspectives were excluded. Two screening steps were undertaken. First, the selected papers went through a rapid title and abstract screening; if a sustainability approach was not mentioned in the abstract, a rapid full-text screening was conducted to confirm whether the approach was used or not within the study. Full-text screening was then conducted by two authors (LL and AL) for all eligible papers.
Data extraction, data items and data synthesis
A data extraction table was designed to record the key components and information from the retrieved papers. This method allowed for the data to be sorted into themes and categories promoting structured analysis and interpretation [38]. Data extraction included author(s), journal, year of publication, country of study, study methodology, sustainability approach used, purpose of use, description of use, healthcare setting, timeframe, measurement, documented benefits and challenges, and reported sustainability outcome variables.
Data was independently extracted by one author (LL) on an excel spreadsheet. A second author (AL) then read through and verified the extracted information from the original full text articles.
Quality appraisal
Although scoping reviews do not consider appraisal tools mandatory, the lack of clarity and rigour of much of the sustainability research prompted the use of a quality assessment tool within this study [17, 38, 42]. To establish study quality, the Mixed Method Appraisal Tool was used [43], which is divided into two sections. The first screening step is to ensure non-empirical studies such as reviews and theoretical papers are discounted for the analysis [44]. The screening questions are then followed by specific questions based on study methodology [43].
Collating, summarising and reporting the results
Following the quality assessment, data extraction categories and sustainability outcomes were compared and consolidated across the included studies. Results have been summarised using descriptive statistics and narrative summaries to show the studies’ characteristics, purpose of use, the benefits and challenges of use as well as impact on sustainability outcome variables.
Results
The electronic citation tracking of the previously identified sustainability approaches identified 3119 publications for potential inclusion. Additional file 1 includes full information on how each of the 62 approaches were tracked through the literature. It lists the number of citations for each sustainability approach along with the number of full-text articles retrieved and included in this review. This process identified 109 eligible articles which underwent full-text screening. Following quality assessment, 41 articles were excluded based on inclusion criteria. Details on inclusions and exclusion of full text papers are included in Additional file 3. This resulted in 68 articles that were included in this review (Fig. 1). Table 1 summarises the included papers. Full data extraction (including study design, sustainability approach applied, documented benefits and challenges of use, and reported sustainability outcome variables) for the 68 articles can be found in Additional file 4.
Scoping review flow diagram
Results are presented in three sections. The first describes the application of the sustainability approaches in healthcare settings, the second discussed the reported benefits and challenges of use, and the third describes the impact of approaches on sustainability outcome variables.
Sustainability approach application
Results demonstrate an increase in the use of sustainability approaches in peer-reviewed studies over time (Fig. 2). Approaches have been applied in a range of healthcare settings (Fig. 3). Most (53%, 36/68) were based in primary care settings (e.g. primary care trusts, GP surgeries, clinics, etc.). Fewer approaches were used within tertiary care (13%, 9/68) or secondary care settings (10%, 7/68). Five approaches specified that they were used in community healthcare initiatives and three approaches were used in nursing education initiatives.
Number of articles describing the application of a sustainability approach by year
Percentage of publications by healthcare setting
The approaches have been applied in over 19 countries (Table 2). The highest proportion were conducted in the United Kingdom (34%, 23/68), followed by 15% (10/68) in the United States. Commonly, studies were undertaken within one country, but a small proportion (4%, 3/68) took place across multiple countries. While multiple study designs were found, qualitative methods were the most common, with 59% (40/68) of the studies employing methods such as interviews, focus groups and participant observation; this included four case studies [52, 54, 110, 112], two grounded theory approaches [59, 105] and one ethnography [85]. In total 28% (19/68) were mixed-method studies and 13% (9/68) were quantitative study designs using questionnaires, regression analyses or weighted scales. Most studies (79%, 54/68) explored the application of the sustainability approach, across multiple sites or organisations. For example, Campbell et al. applied Gruen’s Model of health-programme sustainability to describe the sustainability of a hospital-based smoking cessation program across six Canadian hospitals [53]. Finally, 21% (14/68) were used within single sites (e.g. intensive care units, care homes, GP clinics, etc.).
Sustainability approaches applied in reviewed studies
Within the 68 reviewed articles, 17 different sustainability approaches were applied. Table 3 summarises the sustainability approaches demonstrating use within the reviewed publications along with the purpose and constructs included within each approach. The most commonly applied approach was the Normalisation Process Theory (NPT), which was used to in 30 articles. The Normalisation Process Model, Level of Institutionalisation (LoIn) Scale, Program Sustainability Assessment Tool, Program Sustainability Index and the NHS III Sustainability Model were all used within 3 or more studies. The remaining 11 approaches were used in 2 or fewer articles.
Purpose of application
The purpose of using a sustainability approach fell into five categories – analysis (to examine and interpret data in relation to sustainability), evaluation (to appraise sustainability of an existing programme), guidance (to direct study design, data collection, discussion, etc.), assessment (to measure constructs for sustainability) and planning (to design a programme to achieve sustainability). Twenty-six (38%) articles used a sustainability approach for a combination of these purposes such as planning and evaluation or guidance and analysis (Fig. 4). For example, Johnson et al. used NPT to both guide and assess sustainability of an intervention to improve the nutritional care of preterm infants [79]. Nineteen of the articles (28%) used a sustainability approach for assessment. For example, Cramm et al. used Slaghuis’s Framework and Instrument for Sustainability to assess connections between partnership functioning and sustainability of innovative programmes in community care [57]. Fifteen studies (22%) used an approach to inform data analysis for the study. This was demonstrated in Toledo Romanib et al.’s study where the LoIn scale was used to guide content analysis [102]. No study used a sustainability approach exclusively for planning for sustainability, but planning was reported in combination with other purposes.
Purpose of use described in articles
Benefits and challenges
Details on and documentation of the application of the sustainability approaches varied across the studies. While 65% (44/68) of articles identified benefits associated with the use of approaches, only 34% (23/68) reported challenges. Table 4 reports findings on the benefits and challenges for the individual approaches. This is followed by discussion of six common themes related to the benefits and challenges found across sustainability approaches.
Benefits of use
For the 65% (44/68) of articles describing benefits of use, three main benefits were found, namely (1) improved conceptualisation of sustainability, sustainability constructs and their impact on systems, (2) improved ability to organise, analyse and interpret sustainably data, and (3) increased focus on areas for improvement.
Improved conceptualisation of sustainability, sustainability constructs and their impact on systems: 37% (25/68)
The most common benefit reported from the use of sustainability approaches was the improved understanding of sustainability and sustainability constructs. Twenty-five articles stated they were able to improve conceptualisation of sustainability of interventions [47, 48, 50, 53, 54, 58, 61, 71,72,73, 75, 78,79,80, 86, 88, 92, 95, 96, 100, 101, 103, 104, 110]. For example, Deconinck et al. applied Atun’s Conceptual Framework for Analysing Integration of Targeted Health Interventions, and found that the approach provided a systems lens that increased understanding of pathways for integration to help sustain coverage of effective interventions [58, 120]. Some articles also explicitly reported gaining a better understanding of barriers and facilitators to sustainability of their initiatives. The Program Sustainability Assessment Tool was used to help predict an initiatives’ integration in Scudder et al.’s study which found barriers in financial support and implementation funding [96, 116].
Improved ability to organise, analyse and interpret sustainably data: 26% (18/68)
Many of the approaches were used to guide study design and analysis. In this process, authors noted that the approaches aided analysis and interpretation of data [52, 55, 60, 63, 64, 68, 70, 76, 77, 81, 89, 94, 98, 105, 107, 108, 111]. For example, within Sanders et al.’s article, the NPT was used to organise their semi-structured interview data [94]. Authors noted that the NPT not only aided in the design of the interview but also guided thematic analysis on improving uptake of the new system for general practitioners. Similarly, Zakumumpa et al. found that the LoIn Scale aided them to analyse antiretroviral therapy institutionalisation scores across multiple cases [111].
Increased focus on areas for improvement: 9% (6/68)
Finally, the use of sustainability approaches was beneficial to improve focus and highlight areas needing further work [56, 62, 66, 69, 82, 85]. This was highlighted within Ford et al.’s study using the NHS III Sustainability Model. This study found that the model provided a measurement indicating where a component in the organisation needs improvement for sustainability [66]. Similarly, Blanchet et al. noted that the Sustainability Analysis Process aided them to better understand the need for participation of major stakeholders to promote sustainability within the rehabilitation sector [50].
Challenges of use
Key challenges were reported in 23 articles. These included (1) issues with approach constructs, (2) need for adaptation or improvement and (3) difficulty in application.
Issues with approach constructs: 16% (11/68)
The most common challenge associated with the use of sustainability approaches was associated with approach constructs. Issues such as applicability, relevance, overlap or missing constructs were encountered across a number of studies [47, 62, 70, 71, 82, 95, 96, 100, 104, 107, 109]. For example, users found difficulty in discerning the differences between the constructs within the NPT, which were often reported to overlap in content and concept. Both Drew et al. and Gillespie et al. found that the overlapping nature of the four constructs within the NPT meant the data could be coded into more than one construct [62, 71]. Another noted issue was the inability of constructs within approaches to cover all the relevant areas needed for studies [82, 104, 109]. This was identified within Underwood et al.’s study, which found that Leffer’s Conceptual Framework for Partnership and Sustainability did not adequately explore resources or partner factors [104].
Need for adaptation and improvement: 10% (7/68)
Seven studies noted that the approaches were a challenge to be used as originally designed and suggested areas for improvement [24, 46, 48, 68, 79, 85, 93]. For example, the Program Sustainability Index was used by Ahmad and Abu Talib to assess community capacity. During the application, authors stated that the index had to be adapted to suit the study needs [46]. A number of studies also recommended improvements for approaches to work more effectively. Redman and Barab suggested the LoIn scale required further testing with larger samples and different health programmes to improve its applicability [93]. Bamford et al. suggested that the NPT may require further development to explore specific behaviour change techniques to increase the practical value of the theory [48].
Difficulty in application: 9% (6/68)
It was noted within six papers that authors encountered some difficulty in applying the approaches [24, 52, 77, 91, 97, 110]. In Winterton and Chambers’ study, they found that Shediac-Rizkallah and Bone’s conceptual framework was difficult to apply to initiatives in certain ethnic populations [1, 110]. Doyle et al. reported similar findings with the NHS III Sustainability Model. In this study, users reported concerns over aspects of the model design and only 12 of the 19 teams found it acceptable for routine use [24, 125].
Examining impact on sustainability outcome variables
To understand potential links between the use of a sustainability approach and enhanced reporting of or impact on sustainability, six sustainability outcome variables were investigated. The majority (79%, 54/68) did not report sustainability outcome variables in their studies. Low reporting was attributed to how the approaches were used in practice as many articles applied approaches to aid conceptualisation of sustainability constructs; therefore, sustainability outcomes were not explicitly assessed. Only 21% (14/68) reported one or more sustainability outcome variable. Table 5 summarises the articles that reported sustainability outcomes. Below each outcome variable is discussed with specific examples of how each was reported within the studies.
Continued programme activities or components of the original intervention (9/14)
The most common outcome reported across the studies was continuation of programme activities or components, with nine articles exploring the continuation of initiative activities [76, 78, 82, 84, 93, 96, 99, 102, 106]. ‘Continuation of activities’ was demonstrated through multiple measures, including continuation of services (e.g. continued delivery of the HIV testing service [82]), tool use (e.g. continued use of a shared decision-making tool [84]) and continued delivery educational programmes for staff and patients (e.g. diabetes education programmes [93]). Three of these detailed partial continuation of activities [78, 99, 106]. For example, Redman et al.’s study using the LoIn scale found ‘low to moderate levels’ of routinisation were achieved, which suggested that the programmes had embedded into the host organisations [93]. Similarly, Ibrahim et al. discussed partial continuation of a grip strength measurement programme across five acute medical wards with weekly coverage of the programme ranging between 65% and 100% in one site and between 0% and 93% in another [78].
Maintenance of new organisational practices, procedures and policies that were started during programme implementation (7/14)
Maintenance of new practices and policies was highlighted in six articles [50, 66, 79, 82, 93, 99, 102]. This variable was demonstrated through intervention integration within clinical guidelines [79, 82], electronic records [66], through ongoing discussion at team meetings [66] and monitoring/evaluating activities [99]. For example, Toledo Romani et al.’s study using the LoIn scale found that elements of the intervention (community working groups and provincial/municipal coordination groups) had lost their separate identity and had become part of the programme’s regular procedures [102]. Blanchet et al. followed two rehabilitation sectors using the Sustainability Analysis Process for 2 years. At follow-up, they noted that, although there were differences in the structure of social networks, the programme policies continued to be used [50].
Whether benefits or outcomes for consumers, clients or patients are continued (6/14)
Six articles reported continued benefits to patients [66, 76, 79, 84, 102, 106]. Detail on these benefits was variable across the studies. Some reported specific measures of continuation of benefits such as continued behaviour change or continuation of improved patient outcomes [102]. For example, Toledo Romani et al. used the LoIn scale and demonstrated maintenance of behavioural changes related to a dengue control intervention [102]. Johnson et al. used the NPT to study guideline compliance and found that the intervention resulted in maintenance of improvements in health outcomes for infants, specifically protein intake and weight gain [79]. Others reported continuation of benefits in terms of continued support of patients and families (e.g. a shared decision-making tool helping patients to choose treatment options [84] or family care maps supporting families and patients on the continuum of rehabilitation process [66]). Higuchi et al. used the NHS Sustainability Model and reported vague continuation of benefits. They described how sustained organisational perceptions of benefits existed 5 years after their initial initiative to introduce multiple nursing guidelines [76]. A number of studies also described the benefits to other stakeholders affected by the programme, such as their staff, communities and carers [65, 66, 79].
Maintenance of community-level partnerships or coalitions developed during the funded programme (5/14)
Five articles reported the maintenance of partnerships and described how it was beneficial or challenging for the continuation of the programme [50, 60, 87, 102]. This variable included the active continuation of stakeholder networks, champion engagement, leadership groups, steering committees and building further collaborative relationships. The maintenance of partnerships was highlighted in Diaz del Castillo et al.’s study where champions were a core part of the improvement delivery. The programme was grounded in the work of champions, who would continuously advocate, garner further resources, recommend strategies and implement activities [60]. Scudder et al. also discussed the importance of partnerships in sustaining a parent-child interaction therapy programme. They noted that the leadership groups and steering committees developed during the maintenance period were key sustainability outputs from the programme. Blanchet et al. found similar importance of developing strong partnerships and networks but noted that these relationships can also be a challenge to sustain. Within this study, one site documented the breakdown of their stakeholder network, which impacted their ability to continue to collect sustainability indicators [50].
Sustained attention to the issue or problem (3/14)
Three studies highlighted that attention to the issues or problem was sustained [60, 102]. This was demonstrated through the use of national campaigns and goals [60], mass media and TV campaigns [102], and community meetings [102]. Diaz del Castillo reported that their programme was able to sustain attention through involvement in national campaigns where their ‘Healthy Habits and Lifestyles Program’ was included in the Ten-Year National goals [60]. Scudder et al.’s study also reported maintaining attention to their programme through a number of strategies, including ‘word of mouth’ and TV campaigns.
Programme diffusion and replication in other sites (2/14)
Two studies reported programme diffusion or replication [60]. Scudder et al.’s study demonstrated diffusion of their programme across the United States. The authors noted that the original programme was replicated but the strategies for integration were adapted to meet the needs of other systems, organisations or populations involved. Diaz Del Castillo et al. reported that their programme grew from 12 locations in 1996 and to 31 locations in 2005. They stated, “programs have lasted more than 10 years, scaled-up locally/nationally, and exist in similar forms elsewhere” [60].
Additional sustainability outcome variables
Throughout data extraction, it was noted that several articles discussed and reported sustainability outcome variables that did not fit into the six variables explored [17]. Three emerging outcomes reported across various studies are discussed below.
Capacity built (5/14)
This outcome was discussed within five studies [60, 66, 76, 99, 102] in relation to how the skills and capabilities developed throughout an initiative could continue to be utilised and developed after the end of initiative funding. This variable refers to the development of strengths, skills and abilities of staff, stakeholders and communities to maintain new organisational practices that were started, through their involvement with an improvement initiative [126]. This was demonstrated through integration of intervention into training programmes and orientations [66, 99, 102], and attending knowledge exchange events [76]. Diaz del Castillo et al.’s study specifically commented on the continuation of capacity built and skills gained in the initiative [60]. Authors reported that programmes had expanded and been maintained due to the quality of the instructors, which was seen to impact both people’s continued attendance and perceived benefits. The programme maintained training for staff with a ‘Teachers’ Academy’ created to select and train instructors for the programme [60].
Innovation and Adaptation (5/14)
This outcome relates to the fidelity and flexibility of initiatives and the ability to respond and adapt to challenges and changing needs. While most studies that were reviewed did not describe adaptations or examine their impact on health-related outcomes, this variable was discussed in select studies [50, 60, 87, 106]. Specifically, adaptations to system characteristics [50], funding changes [60], internal culture [87], existing needs and challenges and products (e.g. changing campaign images [106]) were noted. For example, Scudder et al.’s study found that, while initiatives were actively sustaining, no clear strategies were linked to their success but rather, “the common theme across initiatives was the importance of being responsive to existing needs and challenges”. Diaz del Castillo also found that, for both programmes in their study, there was “a need and willingness to be flexible” [60]. Flexibility, in this case, meant adapting programmes to people’s preferences, funding changes and diverse scenarios/conditions [60]. Blanchet et al. also noted how adaptation and resilience played a part in the sustainability of improvements with two rehabilitation sectors. They noted that “the structure of the rehabilitation system characteristics evolved over time and helped understand the changing nature of relationships between actors and their capacity to work as a system. This aided in the creation of consensus on a common vision of sustainability” [50]. To facilitate a greater understanding through future research, some clarity regarding adaptation and fidelity is necessary [3].
Ability to garner further funding (4/14)
An initiatives’ ability to garner further funds to continue the initiative and maintain gains was also reported as a sustainability outcome across several studies [50, 60, 99, 102]. Toledo Romani et al. reported how funds for the activities of the community dengue intervention were included in the annual budget of the wider dengue control programme as an action towards sustainability [102]. This was also highlighted in the Diaz Del Castillo study where they described how 19 of 32 departments had co-funded programmes and six programmes operated using their own resources. They highlighted that “the creation of a specific PA (Physical Activity) unit in 2011 led to a dedicated budget, resources, goals and a capacity to translate policy into action” [60]. Blanchet et al. also identified gaining further resources as a mechanism indicating sustainability and noted that “financial support by an international organization facilitated advancement toward self-identified sustainability goals” [50].
Discussion
While early implementation research was criticised for a lack of attention to the theoretical underpinnings, in the last two decades, numerous theories, models and frameworks have been produced [19]. Due to the vast number of approaches available, choosing the most appropriate one is increasingly difficult [30, 31]. To understand the value of models and frameworks, we require supporting evidence to warrant their use [32]. This review provides healthcare professionals and researchers with the current evidence on sustainability approaches, including the benefits and challenges that arise through their use and the outcomes they have supported.
This review tracked sustainability approaches in the literature and found 68 articles detailing their use in peer-reviewed publications. It found that, while many sustainability approaches are available, few have demonstrated their practical application or benefits, with only 17 of the 62 available approaches showing use in peer-reviewed publications [27]. Sustainability approaches were used to assess, evaluate, analyse and guide sustainability measurement and research. Benefits of use include improved understanding of the barriers and risks to sustainability, improved conceptualisation of sustainability and its constructs and improved ability to organise, analyse and interpret sustainably data. The literature has also documented challenges, including issues with approach constructs, difficulty in application and need for improvement in design [4, 54, 62, 82, 127]. These results support previous findings reporting that tools, frameworks and models may improve team knowledge and enhance action but they can also be complicated to use, hinder teamwork and take substantial amounts of time [21, 46, 48, 54, 62, 82, 127,128,129,130,131,132]. The findings presented in this study, along with these previous reports, demonstrate the need for practitioners and researchers to have clear and realistic expectations when using frameworks, models or tools for sustainability. It is important to not only understand the potential benefits but also the challenges to realistically consider application and weigh potential issues with expected impact [3, 35, 36].
Implications for future measurement
This study has shown that, while there is increasing interest in the use of sustainability approaches, there is little reported evidence of their impact on sustainability outcome variables [21, 26, 36, 116, 133, 134]. Although most sustainability approaches are intended to support or influence long-term success, only 21% (14/68) of studies reported any information on sustainability outcomes. This is similar to the results reported by Stirman et al. in their systematic review, which found only 22% of studies reporting information about sustained outcomes [3].
The low reporting of sustainability outcome variables is often attributed to poorly defined concepts of what sustainability means within individual studies [36]. This issue is compounded by the frequent reporting of sustainability as a dichotomous measure [135]. For example, in Leon et al.’s, study they concluded that their intervention had become sustained and was “embedded in practice (normalised) during a two-year period” but information on what was embedded, what specific activities were continued, and what other variables were sustained were not discussed [82]. These findings highlight that, although the use of multiple outcome variables to report sustainability outcomes is recommended, it is not common practice [17].
Future research should endeavour to clearly describe and outline expected sustainability outcomes and consider the use of multiple outcome variables to capture comprehensive sustainability narratives [17]. Reporting dichotomous or unclear outcomes contributes to underreporting of sustainability failures and successes, making it increasingly difficult to understand the true scale of the problem, compare outcomes across studies or provide generalised learning on sustainability measurement.
While this study examined six sustainability outcome variables, three additional variables were discussed in our results, namely Capacity built, Innovation and adaptation, and Garnering funding. Although not part of Scheirer and Dearing’s original six variables, these outcomes have been suggested elsewhere as being potential variables that are sustained following healthcare initiatives [1, 21, 22, 116, 121, 136,137,138]. These additional outcomes may lend a new understanding of how sustainability outcomes can be viewed, assessed and reported in practice.
The role of continued innovation and adaptation in assessing sustainability is of particular interest to those adopting a ‘process’ view of sustainability. Much of the previous research considered deviations from intervention protocols to be implementation failure but, due to growing recognition that institutionalisation of activities may impede organisations from adopting more effective practices, there is now greater acceptance of the role of adaptation and capacity-building in sustainability [3, 4, 22, 36, 139]. Unfortunately, few studies explore what adaptations are made to interventions, why such changes are necessary, and how decisions to change intervention components are made [3, 17, 140]. Future research should prioritise studying this process to provide insight into how adaptations may impact outcomes and sustained services [3, 141, 142].
Implications for practitioners and researchers
It is recommended “that researchers specify the unit of data collection and analysis for both sustainability determinants and outcomes” [36]. This review seeks to support this further by incorporating emerging findings to guide measurement and study of sustainability in healthcare settings by providing an updated list of sustainability outcome variables (Table 6). It is hoped that having a clear list of potential outcomes to assess will support practitioners and researchers to clearly define the sustainability outcome variables of interest and consider how evidence for each outcome may be collected and analysed. It also aims to support the consideration of a wide breadth of sustainability outcomes to discourage unclear, over simplistic or dichotomous reporting of outcomes. While all variables may not apply to all settings, this list can serve as a baseline for sustainability outcome planning and measurement. To understand its potential value in achieving these aims, this list requires further testing and validation within future studies. Specifically, there is a need for future work to provide insight into how data on each of these variables can be collected and analysed.
Given this expanded list of outcome variables, planning for their collection and analysis may be a challenge. The most appropriate methods for collecting evidence related to each of these variables has not been determined but it is increasingly recognised that sustainability research should draw on mixed methods [3, 13, 17, 143]. A mixed methods approach supports both quantitative methods key to measuring ongoing performance of initiatives (e.g. continuation of benefits or activities) as well as the qualitative methods to provide the necessary ‘texture and detail’ to describe intervention sustainability (e.g. continued partnerships or training) [143].
Future research
While this review presented the available evidence on the application and impact of sustainability approaches in healthcare, it also highlighted the remaining gaps in the literature. First, there is a need for future research to more rigorously report and explicitly describe how frameworks, tools and models inform development, organisation and implementation of research [7, 19, 27, 34]. This evidence will help to support improved sustainability approach development and improve the reliability of sustainability studies [3, 19, 35, 36]. Further work is also needed to build an understanding of under-use and misuse of frameworks, models and tools [29]. This study found that sustainability approaches were largely used for analysis and conceptualisation purposes to aid in interpretation and organisation of data. This does not reduce the value of these studies but highlights the challenge of understanding the impact of these approaches if they are not used as fully intended. For example, the aim of the NPT is to help understand how complex interventions can embed change and become ‘normalised’ in practice [113]. Although this approach has the potential to explain how complex interventions become routinely embedded, the evidence shows that it is often used to assess implementation processes only. Future studies may benefit from reporting on all aspects of the approach, including embedding and integration. There is also a need to empirically test the updated sustainability outcomes list to assess if it clarifies what continued, discontinued or changed for interventions and if it provides better insight into sustaining initiatives. This will be critical in understanding the process and potential difficulty of studying these diverse sustainability outcomes in practice.
Strengths and limitations
This is the first review to examine the application and impact of sustainability approaches in healthcare and the first review attempting to understand the impact of sustainability approaches on sustainability outcomes.
There are a number of limitations associated with this systematic scoping review that are must be acknowledged. The first limitation is the potential for publication bias which may have manifested in a number of ways. First, the search strategy may have missed published articles if they had not referenced the use of an approach within their study. Second, this review only drew on the 62 approaches identified in a previous systematic review, which meant that approaches published after 2018 were not included in this work. This was largely a pragmatic decision as we believed the existing 62 approaches would provide sufficient evidence of the scope of application and impact of approaches. It was also believed that newer approaches would not have had sufficient time to build evidence of their impact within this short timeframe. Therefore, future work should continue to assess and collect information on emerging approaches and the evidence of their application. Finally, the review did not search the grey literature, which means that findings may be more representative of academic and research application and analysis.
While there is value in consolidating insight into the perceived experience of implementing such approaches, another limitation to this work is the reliance on reported benefits and challenges. Many papers did not discuss benefits or challenges and those that did may not have been comprehensive in their descriptions as this was not the primary aim of their work. This may have resulted in over or under reporting of benefits and challenges.
Another limitation is the disproportionate inclusion of studies using the NPT. As this review sought to give an overview of the available approaches and their use, attempts were made to balance results where possible by providing narrative examples and descriptions for all sustainability approaches throughout the manuscript.
Finally, it is also important to note that this work does not seek to critique the included approaches but instead to consolidate the reported evidence on their use. Many of the sustainability approaches have been developed to encourage more systematic work and guide empirical studies; therefore, it should be acknowledged that this area of research is still in its infancy. Many of the conceptual frameworks and models aim to instigate knowledge generation on the determinants that influence sustainability processes and outcomes. In order to do so, they need to be applied and tested across real-world settings [4, 21, 116, 122]. Therefore, full insight into the impact of sustainability approaches will continue to be developed over time.
Conclusion
This review provides healthcare practitioners and researchers with a consolidated summary of the current evidence base on sustainability approaches, including the benefits and challenges that arise through their use and the outcomes they have contributed to. The use of a sustainability approach can improve the knowledge of sustainability constructs, aid in analysis and increase focus to promote sustainability. They can also create challenges associated with their content, relevance and applicability. These findings highlight the need for thoughtful consideration of the potential impact prior to use. They also emphasise the remaining gaps in the literature, with a substantial amount of work still needed to ascertain the value and contributions of many sustainability approaches. In order to understand if these approaches are providing the foundations for rigorous inquiry and advancing sustainability research, well-planned studies recognising comprehensive sustainability outcome variables are needed.
Availability of data and materials
All data generated or analysed during this study are included in this published article and its supplementary information files.
Abbreviations
- LoIn:
-
Level of Institutionalisation Scale
- NPT:
-
Normalisation Process Theory
References
Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13:87–108 http://www.ncbi.nlm.nih.gov/pubmed/10178339.
Sirkin HL, Keenan P, Jackson A. The hard side of change mangement. Harv Bus Rev. 2005:109–18.
Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations : a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17 https://doi.org/10.1186/1748-5908-7-17.
Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117 https://doi.org/10.1186/1748-5908-8-117.
Williams L, Daggett V, Slaven JE, Yu Z, Sager D, Myers J, et al. A cluster-randomised quality improvement study to improve two inpatient stroke quality indicators. BMJ Qual Saf. 2015; https://doi.org/10.1136/bmjqs-2015-004188.
Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Obstet Gynecol. 2009;114:1341–5 https://doi.org/10.1097/AOG.0b013e3181c3020d.
Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S, et al. Sustainability of evidence-based healthcare : research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10:88.
Johnson AM, Moore JE, Chambers DA, Rup J, Dinyarian C, Straus SE. How do researchers conceptualize and plan for the sustainability of their NIH R01 implementation projects? Implement Sci. 2019;14:1–9.
Altman DG, Endres J, Linzer J, Lorig K, Howard-Pitney B, Rogers T. Obstacles to and future goals of ten comprehensive community health promotion projects. J Community Health. 1991;16:299–314.
Martin GP, Weaver S, Currie G, Finn R, Mcdonald R. Innovation sustainability in challenging health-care contexts: embedding clinically led change in routine practice. Health Serv Manag Res. 2012;25:190–9 https://doi.org/10.1177/0951484812474246.
Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implement Sci. 2017;12:110 https://doi.org/10.1186/s13012-017-0637-1.
NHS Modernisation Agency, Agency M, NHS Modernisation Agency. Complexity of sustaining healthcare improvements: what have we learned so far? 2004.
Bowman CC, Sobo EJ, Asch SM, Gifford AL. Measuring persistence of implementation: QUERI series. Implement Sci. 2008;3:21 https://doi.org/10.1186/1748-5908-3-21.
Virani T, Lemieux-charles L, Davis DA, Berta W. Sustaining change: once evidence-based practices are transferred, what then? Healthc Q. 2009;12:89–96.
Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26:320–47 https://doi.org/10.1177/1098214005278752.
Ham C. Evaluation of the projects within the National Booking Program. Birmingham; 2004.
Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101:2059–67 https://doi.org/10.2105/AJPH.2011.300193.
Greenhalgh T, Robert G, Bate P, Kyriakidou O, Macfarlane F, Peacock R. How to spread good ideas: a systematic review of the literature on diffusion, dissemination and sustainability of innovations in health service delivery and organisation. London; 2004.
Damschroder LJ. Clarity out of chaos: use of theory in implementation research. Psychiatry Res. 2019; https://doi.org/10.1016/j.psychres.2019.06.036.
Maher L, Gustafson D, Evans A. Sustainability model and guide. Coventry: NHS Institute for Innovation and Improvement; 2010.
Lennox L, Doyle C, Reed J, Bell D. What makes a sustainability tool valuable, practical, and useful in real world healthcare practice? A qualitative study on the development of the long term success tool in Northwest London. BMJ Open. 2017;7:1–13.
Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, et al. Sustainability science: an integrated approach for health-programme planning. Lancet. 2008;372:1579–89 https://doi.org/10.1016/S0140-6736(08)61659-1.
Mancini JA, Marek LI. Sustaining community-based programs for families: conceptualization and measurement. Fam Relat. 2004;53:339–47.
Doyle C, Howe C, Woodcock T, Myron R, Phekoo K, McNicholas C, et al. Making change last: applying the NHS institute for innovation and improvement sustainability model to healthcare improvement. Implement Sci. 2013;8:127 https://doi.org/10.1186/1748-5908-8-127.
Sarriot EG, Winch PJ, Ryan LJ, Bowie J, Kouletio M, Swedberg E, et al. A methodological approach and framework for sustainability assessment in NGO-implemented primary health care programs. Int J Health Plann Manag. 2004;19:23–41.
Calhoun A, Mainor A, Moreland-Russell S, Maier RC, Brossart L, Luke DA, et al. Using the program sustainability assessment tool to assess and plan for sustainability. Prev Chronic Dis. 2014;11 Table 1:130185 https://doi.org/10.5888/pcd11.130185.
Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci. 2018;13:1–17.
Lennox L, French C, Reed J. A qualitative exploration of sustainability processes for improvement: the role of structured sustainability tools. In: Nugus P, Denis J-L, editors. Transitions and boundaries in the coordination and reform of health services. Montreal: Palgrave Macmillan; 2020. p. 271–94.
Birken SA, Powell BJ, Shea CM, Haines ER, Kirk MA, Leeman J, et al. Criteria for selecting implementation science theories and frameworks: results from an international survey. Implement Sci. 2017;12:1–9.
Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:1–13.
Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implement Sci. 2014;9:1–9.
Jaccard J, Jacoby J. Theory construction and model-building skills: a practical guide for social scientists. London: The Guildford Press; 2010.
Schouten LMT, Hulscher MEJL, van Everdingen JJE, Huijsman R, Grol RPTM. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ. 2008;336:1491–4 http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2440907&tool=pmcentrez&rendertype=abstract. Accessed 6 May 2014.
Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–50.
Tricco AC, Ashoor HM, Cardoso R, MacDonald H, Cogo E, Kastner M, et al. Sustainability of knowledge translation interventions in healthcare decision-making: a scoping review. Implement Sci. 2016;11:1–10 https://doi.org/10.1186/s13012-016-0421-7.
Shelton R, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care: Ssrn; 2018.
Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10:1–13.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2004;8:19–32.
PRISMA STATEMENT. PRISMA. Transport reporting of systematic reviews and meta- analyses. 2009.
Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–73.
Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005;331:1064–5.
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26:91–108.
Hong QN, Pluye P, Sergi F, Gillian B, Felicity B, Margaret C, et al. Mixed methods appraisal tool ( MMAT ) version 2018. 2018.
Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the mixed methods appraisal tool (MMAT). J Eval Clin Pract. 2018;24:459–67.
Ahmad MS, Talib NBA. Empirical investigation of community empowerment and sustainable development: quantitatively improving qualitative model. Qual Quant. 2015;29:1–19.
Ahmad MS, Abu Talib NB. Analysis of community empowerment on projects sustainability: moderating role of sense of community. Soc Indic Res. 2016;129:1039–56 https://doi.org/10.1007/s11205-014-0781-9.
Atkins S, Lewin S, Ringsberg KC, Thorson A. Provider experiences of the implementation of a new tuberculosis treatment programme: a qualitative study using the normalisation process model. BMC Health Serv Res. 2011;11:1–12 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=med5&NEWS=N&AN=22004533.
Bamford C, Heaven B, May C, Moynihan P. Implementing nutrition guidelines for older people in residential care homes: a qualitative study using normalization process theory. Implement Sci. 2012;7:106 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=medl&NEWS=N&AN=23110857.
Blakeman T, Protheroe J, Chew-graham C, Rogers A, Kennedy A. Understanding the management of early-stage chronic kidney disease in primary care. Br J Gen Pract. 2012;62:233–42.
Blanchet K, Palmer J, Palanchowke R, Boggs D, Jama A, Girois S. Advancing the application of systems thinking in health: analysing the contextual and social network factors influencing the use of sustainability indicators in a health system--a comparative study in Nepal and Somaliland. Health Res Policy Syst. 2014;12:46 https://doi.org/10.1186/1478-4505-12-46.
Bocoum FY, Tarnagda G, Bationo F, Savadogo JR, Nacro S, Kouanda S, et al. Introducing onsite antenatal syphilis screening in Burkina Faso: implementation and evaluation of a feasibility intervention tailored to a local context. BMC Health Serv Res. 2017;17:1–10.
Burau V, Carstensen K, Fredens M, Kousgaard MB. Exploring drivers and challenges in implementation of health promotion in community mental health services: a qualitative multi-site case study using normalization process theory. BMC Health Serv Res. 2018;18:1–12.
Campbell S, Pieters K, Mullen K-A, Reece R, Reid RD. Examining sustainability in a hospital setting: case of smoking cessation. Implement Sci. 2011;6:108 https://doi.org/10.1186/1748-5908-6-108.
Chilundo BGM, Cliff JL, Mariano ARE, George A. Relaunch of the official community health worker programme in Mozambique : is there a sustainable basis for iCCM policy ? Health Policy Plan. 2015;30:54–64.
Coupe N, Anderson E, Gask L, Sykes P, Richards DA, Chew-Graham C, et al. Facilitating professional liaison in collaborative care for depression in UK primary care; a qualitative study utilising normalisation process theory. BMC Fam Pract. 2014;15:78 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=emed12&NEWS=N&AN=24885746.
Cramm JM, Nieboer AP. Short and long term improvements in quality of chronic care delivery predict program sustainability. Soc Sci Med. 2014;101:148–54 https://doi.org/10.1016/j.socscimed.2013.11.035.
Cramm JM, Phaff S, Nieboer AP. The role of partnership functioning and synergy in achieving sustainability of innovative programmes in community care. Health Soc Care Community. 2013;21:209–15.
Deconinck H, Hallarou ME, Pesonen A, Gerard JC, Criel B, Donnen P, et al. Understanding factors that influence the integration of acute malnutrition interventions into the national health system in Niger. Health Policy Plan. 2016;31:1364–73.
Desveaux L, Mitchell JI, Shaw J, Ivers NM. Understanding the impact of accreditation on quality in healthcare: a grounded theory approach. Int J Qual Health Care. 2017;29:941–7.
Díaz del Castillo A, González SA, Ríos AP, Páez DC, Torres A, Díaz MP, et al. Start small, dream big: experiences of physical activity in public spaces in Colombia. Prev Med. 2017;103:S41–50.
Dickinson C, Gibson G, Gotts Z, Stobbart L, Robinson L. Cognitive stimulation therapy in dementia care: exploring the views and experiences of service providers on the barriers and facilitators to implementation in practice using normalization process theory. Int Psychogeriatr. 2017;29:1869–78.
Drew S, Judge A, May C, Farmer A, Cooper C, Javaid MK, et al. Implementation of secondary fracture prevention services after hip fracture: a qualitative study using extended Normalization Process Theory. Implement Sci. 2015;10:57 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=medl&NEWS=N&AN=25903563.
Dugdale S, Elison S, Davies G, Ward J, Dalton M. A qualitative study investigating the continued adoption of breaking free online across a national substance misuse organisation: theoretical conceptualisation of staff perceptions. J Behav Health Serv Res. 2017;44(1):89–101.
Farr M, Banks J, Edwards HB, Northstone K, Bernard E, Salisbury C, et al. Implementing online consultations in primary care: a mixed-method evaluation extending normalisation process theory through service co-production. BMJ Open. 2018;8:1–11.
Fleiszer AR, Semenic SE, Ritchie JA, Richer M, Denis J. A unit-level perspective on the long-term sustainability of a nursing best practice guidelines program : an embedded multiple case study. Int J Nurs Stud. 2016;53:204–18 https://doi.org/10.1016/j.ijnurstu.2015.09.004.
Ford JH, Wise M, Krahn D, Oliver KA, Hall C, Sayer N. Family care map: sustaining family-centered care in Polytrauma Rehabilitation Centers. J Rehabil Res Dev. 2014;51:1311–24 https://doi.org/10.1682/JRRD.2014.03.0066.
Fox A, Gardner G, Osborne S. Nursing service innovation: a case study examining emergency nurse practitioner service sustainability. J Adv Nurs. 2018;74(2):454–64.
Franx G, Oud M, de Lange J, Wensing M, Grol R. Implementing a stepped-care approach in primary care: results of a qualitative study. Implement Sci. 2012;7:1–13.
Furler J, Spitzer O, Young D, Best J. Insulin in general practice: barriers and enablers for timely initiation. Aust Fam Physician. 2011;40:617–21.
Gask L, Bower P, Lovell K, Escott D, Archer J, Gilbody S, et al. What work has to be done to implement collaborative care for depression? Process evaluation of a trial utilizing the normalization process model. Implement Sci. 2010;5:1–11.
Gillespie BM, Harbeck E, Lavin J, Gardiner T, Withers TK, Marshall AP. Using normalisation process theory to evaluate the implementation of a complex intervention to embed the surgical safety checklist. BMC Health Serv Res. 2018;18:170.
Glynn LG, Glynn F, Casey M, Wilkinson LG, Hayes PS, Heaney D, et al. Implementation of the SMART MOVE intervention in primary care: a qualitative study using normalisation process theory. BMC Fam Pract. 2018;19:1–10.
Godden DJ, King G. Rational development of telehealth to support primary care respiratory medicine: patient distribution and organisational factors. Prim Care Respir J. 2011;20:415–20 https://doi.org/10.4104/pcrj.2011.00063.
Green AE, Trott E, Willging CE, Finn NK, Ehrhart MG, Aarons GA. Child abuse & neglect. The role of collaborations in sustaining an evidence-based intervention to reduce child neglect. Child Abuse Negl. 2016;53:4–16 https://doi.org/10.1016/j.chiabu.2015.11.013.
Herbert G, Sutton E, Burden S, Lewis S, Thomas S, Ness A, et al. Healthcare professionals’ views of the enhanced recovery after surgery programme: a qualitative investigation. BMC Health Serv Res. 2017;17:1–14.
Higuchi K, Downey A, Davies B, Bajnok I, Waggott M. Using the NHS sustainability framework to understand the activities and resource implications of Canadian nursing guideline early adopters. J Clin Nurs. 2013;22:1707–16 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=ovftn&NEWS=N&AN=00019038-201306000-00024.
Hooker L, Small R, Humphreys C, Hegarty K, Taft A. Applying normalization process theory to understand implementation of a family violence screening and care model in maternal and child health nursing practice: a mixed method process evaluation of a randomised controlled trial. Implement Sci. 2015;10:39 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=medl&NEWS=N&AN=25890352.
Ibrahim K, May CR, Patel HP, Baxter M, Sayer AA, Roberts HC. Implementation of grip strength measurement in medicine for older people wards as part of routine admission assessment: Identifying facilitators and barriers using a theory-led intervention. BMC Geriatr. 2018;18(1):79.
Johnson MJ, Leaf AA, Pearson F, Clark HW, Dimitrov BD, Pope C, et al. Successfully implementing and embedding guidelines to improve the nutrition and growth of preterm infants in neonatal intensive care: a prospective interventional study. BMJ Open. 2017;7(12):e017727.
Kennedy A, Chew-Graham C, Blakeman T, Bowen A, Gardner C, Protheroe J, et al. Delivering the WISE (Whole Systems Informing Self-Management Engagement) training package in primary care: learning from formative evaluation. Implement Sci. 2010;5:1–15.
Latter S, Hopkinson JB, Lowson E, Hughes JA, Hughes J, Duke S, et al. Supporting carers to manage pain medication in cancer patients at the end of life: a feasibility trial. Palliat Med. 2017;32:246–56.
Leon N, Lewin S, Mathews C. Implementing a provider-initiated testing and counselling (PITC) intervention in Cape town, South Africa: a process evaluation using the normalisation process model. Implement Sci. 2013;8:97.
Levin RF, Fineout-Overholt E, Melnyk BM, Barnes M, Vetter MJ. Fostering evidence-based practice to improve nurse and cost outcomes in a community health setting. Nurs Adm Q. 2011;35:21–33 https://doi.org/10.1097/NAQ.0b013e31820320ff.
Lloyd A, Joseph-Williams N, Edwards A, Rix A, Elwyn G. Patchy “coherence”: using normalization process theory to evaluate a multi-faceted shared decision making implementation program (MAGIC). Implement Sci. 2013;8:102 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=medl&NEWS=N&AN=24006959.
Mair FS, Hiscock J, Beaton SC. Understanding factors that inhibit or promote the utilization of telecare in chronic lung disease. Chronic Illn. 2008;4:110–7.
May C, Finch T, Cornford J, Exley C, Gately C, Kirk S, et al. Integrating telecare for chronic disease management in the community: what needs to be done? BMC Health Serv Res. 2011;11:1–11 https://doi.org/10.1186/1472-6963-11-131.
Moreland-Russell S, Combs T, Polk L, Dexter S, Warren G. Assessment of the sustainability capacity of a coordinated approach to chronic disease prevention. 2018;24:E17–24.
Murray E, Burns J, May C, Finch T, O’Donnell C, Wallace P, et al. Why is it difficult to implement e-health initiatives? A qualitative study. Implement Sci. 2011;6:1–11 https://doi.org/10.1186/1748-5908-6-6.
Naldemirci Ö, Wolf A, Elam M, Lydahl D, Moore L, Britten N. Deliberate and emergent strategies for implementing person-centred care: a qualitative interview study with researchers, professionals and patients. BMC Health Serv Res. 2017;17:1–10.
O’Donnell A, Kaner E. Are brief alcohol interventions adequately embedded in UK primary care? A qualitative study utilising normalisation process theory. Int J Environ Res Public Health. 2017;14:350.
O’Donnell CA, Mair FS, Dowrick C, de Brún MOR, de Brún T, Burns N, et al. Supporting the use of theory in cross-country health services research: a participatory qualitative approach using normalisation process theory as an example. BMJ Open. 2017;7:e014289.
Pentecost C, Richards DA, Frost J. Amalgamation of Marginal Gains (AMG) as a potential system to deliver high-quality fundamental nursing care: A qualitative analysis of interviews from high-performance AMG sports and healthcare practitioners. J Clin Nurs. 2018;27:2387–402.
Redman BK, Barab S. Diabetes education infrastructure and capacity in hospitals and home health agencies in Maryland and Pennsylvania. Diabetes Educ. 1997;23:449–55.
Sanders T, Foster NE, Ong BN. Perceptions of general practitioners towards the use of a new system for treating back pain: a qualitative interview study. BMC Med. 2011;9:49.
Scott NA, Vian T, Kaiser JL, Ngoma T, Mataka K, Henry EG, et al. Listening to the community: using formative research to strengthen maternity waiting homes in Zambia. PLoS One. 2018;13(3):e0194535.
Scudder AT, Taber-Thomas SM, Schaffner K, Pemberton JR, Hunter L, Herschell AD. A mixed-methods study of system-level sustainability of evidence-based practices in 12 large-scale implementation initiatives. Health Res Policy Syst. 2017;15:102.
Smith ML, Durrett NK, Schneider EC, Byers IN, Shubert TE, Wilson AD, et al. Examination of sustainability indicators for fall prevention strategies in three states. Eval Program Plann. 2017;2018(68):194–201.
Stoll S, Janevic M, Lara M, Ramos-Valencia G, Stephens TB, Persky V, et al. A mixed-method application of the program sustainability assessment tool to evaluate the sustainability of 4 pediatric asthma care coordination programs. Prev Chronic Dis. 2015;12:150133 https://doi.org/10.5888/pcd12.150133.
Stolldorf DP, Mion LC, Jones CB. A survey of hospitals that participated in a statewide collaborative to implement and sustain rapid response teams. J Healthc Qual. 2016;38:202–12.
Sutton E, Herbert G, Burden S, Lewis S, Thomas S, Ness A, et al. Using the normalization process theory to qualitatively explore sense-making in implementation of the enhanced recovery after surgery programme: “it’s not rocket science”. PLoS One. 2018;13:1–14.
Thomas LH, French B, Burton CR, Sutton C, Forshaw D, Dickinson H, et al. Evaluating a systematic voiding programme for patients with urinary incontinence after stroke in secondary care using soft systems analysis and normalisation process theory: findings from the ICONS case study phase. Int J Nurs Stud. 2014;51:1308–20 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=medl&NEWS=N&AN=24656435.
Toledo Romani ME, Vanlerberghe V, Perez D, Lefevre P, Ceballos E, Bandera D, et al. Achieving sustainability of community-based dengue control in Santiago de Cuba. Soc Sci Med. 2007;64:976–88.
Trietsch J, van Steenkiste B, Hobma S, Frericks A, Grol R, Metsemakers J, et al. The challenge of transferring an implementation strategy from academia to the field: a process evaluation of local quality improvement collaboratives in Dutch primary care using the normalization process theory. J Eval Clin Pract. 2014;20:1162–71 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=ovftq&NEWS=N&AN=00066683-900000000-99910.
Underwood M, Gleeson J, Konnert C, Wong K, Valerio B. Global host partner perspectives : utilizing a conceptual model to strengthen collaboration with host partners for international nursing student placements. Public Health Nurs. 2016;33:351–9.
Upvall MJ, Leffers JM. Revising a conceptual model of partnership and sustainability in global health. Public Health Nurs. 2018;35:228–37.
Van Acker R, De Bourdeaudhuij I, De Cocker K, Klesges LM, Willem A, Cardon G. Sustainability of the whole-community project “10,000 Steps”: a longitudinal study. BMC Public Health. 2012;12:155 https://doi.org/10.1186/1471-2458-12-155.
Volker N, Williams LT, Davey RC, Cochrane T, Clancy T. Implementation of cardiovascular disease prevention in primary health care: enhancing understanding using normalisation process theory. BMC Fam Pract. 2017;18:1–9.
Walker JG, Bickerstaffe A, Hewabandu N, Maddumarachchi S, Dowty JG, Jenkins M, et al. The CRISP colorectal cancer risk prediction tool: an exploratory study using simulated consultations in Australian primary care. BMC Med Inform Decis Mak. 2017;17:1–11 https://doi.org/10.1186/s12911-017-0407-7.
Wallen GR, Mitchell SA, Melnyk B, Fineout-Overholt E, Miller-Davis C, Yates J, et al. Implementing evidence-based practice: effectiveness of a structured multifaceted mentorship programme. J Adv Nurs. 2010;66:2761–71.
Winterton R, Hulme CA. Developing sustainable social programmes for rural ethnic seniors: perspectives of community stakeholders. Health Soc Care Community. 2017;25:868–77.
Zakumumpa H, Bennett S, Ssengooba F. Accounting for variations in ART program sustainability outcomes in health facilities in Uganda: a comparative case study analysis. BMC Health Serv Res. 2016;16:1–13 https://doi.org/10.1186/s12913-016-1833-4.
Murray E, Treweek S, Pope C, Macfarlane A, Ballini L, Dowrick C, et al. Normalisation process theory: a framework for developing evaluating and implementing complex interventions. BMC Med. 2010;6:1–11.
May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43:535–54 https://doi.org/10.1177/0038038509103208.
May C. A rational model for assessing and evaluating complex interventions in health care. Health Serv Res. 2006;6:11 https://doi.org/10.1186/1472-6963-6-86.
Goodman RM, McLeroy KR, Steckler AB, Hoyle RH. Development of level of institutionalization scales for health promotion programs. Health Educ Q. 1993;20:161–78.
Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The program sustainability assessment tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:130184 https://doi.org/10.5888/pcd11.130184.
Slaghuis SS, Strating MMH, Bal RA, Nieboer AP. A framework and a measurement instrument for sustainability of work practices in long-term care. BMC Health Serv Res. 2011;11:314 https://doi.org/10.1186/1472-6963-11-314.
Melnyk BM, Fineout-Overholt E, Gallagher-Ford L, Stillwell S. Evidence-based practice, step by step: sustaining evidence-based practice through organizational policies and an innovative model. Am J Nurs. 2011;111:57–60 https://doi.org/10.1097/01.NAJ.0000405063.97774.0e.
Leffers J, Mitchell E. Conceptual model for partnership and sustainability in global health. Public Health Nurs. 2011;28:91–102 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=ovftl&NEWS=N&AN=00006620-201101000-00012.
Atun R, de Jongh T, Secci F, Ohiri K, Adeyi O. Integration of targeted health interventions into health systems: a conceptual framework for analysis. Health Policy Plan. 2010;25:104–11 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=ovftk&NEWS=N&AN=00003845-201002000-00002.
Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH, Mueller NB, et al. Public health program capacity for sustainability: a new framework. Implement Sci. 2013;8:15 https://doi.org/10.1186/1748-5908-8-15.
Fox A, Gardner G, Osborne S. A theoretical framework to support research of health service innovation. Aust Health Rev. 2014;39:70–5.
Blanchet K, Girois S. Selection of sustainability indicators for health services in challenging environments: balancing scientific approach with political engagement. Eval Program Plann. 2013;38:28–32 https://doi.org/10.1016/j.evalprogplan.2012.11.003.
Fleiszer AR, Semenic SE, Ritchie JA, Richer MC, Denis JL. An organizational perspective on the long- term sustainability of a nursing best practice guidelines program: a case study. BMC Health Serv Res. 2015;15:204–18 https://doi.org/10.1016/j.ijnurstu.2015.09.004.
Howe C, Randall K, Chalkley S, Bell D. Supporting improvement in a quality collaborative. Br J Healthc Manag. 2013;19:434–42.
Dearing JW. The state of the art and the state of the science of community organizing. In: Thompson TL, Dorsey AM, Miller KI, Parrott R, editors. Handbook of health communication. Mahwah: Lawrence Erlbaum Associates Inc.; 2003. p. 207–20.
Ullrich PM, Sahay A, Stetler CB. Use of implementation theory: a focus on PARIHS. Worldviews Evid-Based Nurs. 2014;11:26–34.
Leape LL, Rogers G, Hanna D, Griswold P, Federico F, Fenn CA, et al. Developing and implementing new safe practices: voluntary adoption through statewide collaboratives. Qual Saf Health Care. 2006;15:289–95.
Esmail R, Cummings C, Dersch D, Duchscherer G, Glowa J, Liggett G, et al. Using healthcare failure mode and effect analysis tool to review the process of ordering and administrating potassium chloride and potassium phosphate. Healthc Q. 2005;8:73–80 http://www.ncbi.nlm.nih.gov/pubmed/16334076.
Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. 2014;23:290–8 https://doi.org/10.1136/bmjqs-2013-001862.
Reed JE, Card AJ. The problem with plan-do-study-act cycles. BMJ Qual Saf. 2015;25:1–6.
Hughes R. Tools for quality improvement and patient safety. In: Patient and safety and quality: an evidence-based handbook for nurses. Rockville: Agency for Healthcare Research and Quality; 2008. p. 1–42.
Johnson K, Collins DWA. Sustaining innovations in community prevention systems: a data-informed sustainability strategy. J Community Psychol. 2013;41:322–44.
Hodge LM, Turner KMT, Sanders MR, Filus A. Sustained implementation support scale: validation of a measure of program characteristics and workplace functioning for sustained program implementation. J Behav Health Serv Res. 2017;44:442–64.
Shelton R, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annu Rev Public Health. 2018;39:55–76.
Elshaug AG, Hiller JE, Moss JR. Exploring policy-makers’ perspectives on disinvestment from ineffective healthcare practices. Int J Technol Assess Health Care. 2008;24:1–9.
Nelson DE, Reynolds JH, Luke DA, Mueller NB, Eischen MH, Jordan J, et al. Successfully maintaining program funding during trying times: lessons from tobacco control programs in five states. J Public Health Manag Pract. 2007;13:612–20.
Stefanini A, Ruck N. Managing externally-assisted health projects for sustainability in developing countries. Int J Health Plann Manag. 1992;7:199–210.
Johnson K, Hays C, Center H, Daley C. Building capacity and sustainable prevention innovations: a sustainability planning model. Eval Program Plann. 2004;27:135–49 https://doi.org/10.1016/j.evalprogplan.2004.01.002.
Dusenbury L, Patterson M, Wood S, Booth A, Rick J, Balain S, et al. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18:237–56 https://doi.org/10.1093/her/18.2.237.
Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci. 2013;8:65 https://doi.org/10.1186/1748-5908-8-65.
Tyre M, Orlikowski W. Windows of opportunity: temporal patterns in technical adaptation in organizations. Organ Sci. 1994;5:98–105.
May C, Harrison R, Finch T, MacFarlane A, Mair F, Wallace P. Understanding the normalization of telemedicine services through qualitative evaluation. J Am Med Inform Assoc. 2003;10:596–604.
Acknowledgements
The authors would like to thank the Imperial College Medical Librarians for their assistance and support in creating the search strategy.
Funding
This research was funded by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care Northwest London (NIHR CLAHRC NWL), now recommissioned as NIHR Applied Research Collaboration Northwest London. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.
Author information
Authors and Affiliations
Contributions
LL conceived of the study. LL developed the review protocol and undertook the search strategy with AL. LL and AL conducted the data extraction and analysis and performed the quality assessment. LL summarises the findings in collaboration with LM and JR who refined and developed the emerging themes and contributed to data interpretation. LL drafted the first version of the paper with all authors making contributions to final content of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Additional file 1.
Sustainability approaches tracked in the literature.docx. Summarises the number of citations and full-text articles retrieved per approach.
Additional file 2.
PRISMA screening fillable checklist for scoping reviews.Docx. The Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist details the location of specific scoping review requirements.
Additional file 3.
Quality assessment inclusion and exclusion of full text.docx Details on inclusions and exclusion of full text papers.
Additional file 4.
Data extraction form with full data extraction for each included study
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Lennox, L., Linwood-Amor, A., Maher, L. et al. Making change last? Exploring the value of sustainability approaches in healthcare: a scoping review. Health Res Policy Sys 18, 120 (2020). https://doi.org/10.1186/s12961-020-00601-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12961-020-00601-0