Table 4 Overview of outcome data including patient, provider, population and health system cost outcomes and implementation outcomes
LHS name | Author, Year | Allen’s list of LHS outcomes (11) | Implementation measures (13) | Impact outcomes | Outcome data | Key finding(s) |
|---|---|---|---|---|---|---|
Quantitative studies | ||||||
 Peds-CHOIR | Bhandari 2016 | Population health | Adoption Penetration | Patient Population | Peds-CHOIR allowed for faster decision-making and trialing of interventions based on patient and caregiver feedback | Peds-CHOIR is an example of a platform that highlights predictors of chronic pain and enables individually tailored interventions/treatment |
 VA-ESP | Floyd, 2019 | Knowledge to action latency; Systematic adoption of EBP | Acceptability Adoption Appropriateness Feasibility | Provider | Evidence synthesis reports most often requested to inform clinical guidance (58%), identify future research needs (58%), and determine implementation strategies (47%) 91% of end-users used the evidence reports within 3 months of completion (82%) Evidence reports most often used to inform policy or guidance (26%) and inform procurement decisions (21%) | VA-ESP evidence products can inform clinical practice and policy and are often used within 3 months of completion by decision makers |
 LFEP | Lowes 2017 | Knowledge to action latency; Systematic adoption of EBP; Care experience; Programmatic return on investment | Feasibility Implementation cost | Patient Cost | 43% reduction in inpatient days 27% reduction in inpatient admissions 29% reduction in urgent care visits 176%-210% reduction in healthcare costs | LHS can be implemented to rapidly integrate evidence into clinical practice in a cost-effective way |
Noritz, 2018 | Systematic adoption of EMP; Systematic elimination of wasteful and ineffective practices | Implementation cost | Patient Cost | Patient radiation exposure was reduced Annual costs were reduced to an average of $66 per x-ray per child | Implementation of a local LHS allowed for integration of evidence into practice leading to improved patient care and reduced costs | |
 SCD | Miller, 2020 | Knowledge to action latency Systematic adoption of EBP | Adoption Feasibility | Provider Population | SCD providers entered dates for clinical visits correctly 99% of the time The LHS allowed for the collection of population health data to inform clinical knowledge | The SCD LHS allowed for data collection at the bedside and timely integration of data into clinical records |
 MS PATHS | Mowry, 2020 | Systematic adoption of EBP; Population health | Feasibility Intervention complexity | Patient | The MS PATHS LHS enrolled over 16 500 participants, with 88.4% providing data for at least one time point and the average contribution of 15.6 person-months | MS PATHS is an example of how an MS practice can collect and integrate patient data to inform clinical decisions and continuous learning |
Descriptive case studies | ||||||
 Baylor Scott & White Health | AHRQ, 2019 | Knowledge to action latency; Systematic adoption of EBP; Programmatic return on investment | Penetration Reach | Patient Cost | Over 5 years, the model has led to $280 million in savings while improving patient outcomes | Baylor Scott & White Health have prioritized learning & knowledge generation by focusing on data infrastructure, organizational culture and supporting a continuous cycle of improvement |
 Denver Health | AHRQ, 2019 | Knowledge to action latency; Systematic adoption of EBP | Adoption Penetration Reach | Patient | Reduced surgical infection rates due to Denver Health’s culture of learning; Improved cancer screening rates due to digital infrastructure interventions | Denver health provides a model for how a LHS can provide higher quality, safer and more efficient care |
 HCA | AHRQ, 2019 | Knowledge to action latency; Systematic adoption of EBP | Adoption Penetration | Patient | Improved time to biopsy results for patients; Improved SPOT system led to quicker identification and survival rates of sepsis patients; | HCA's LHS exemplifies how a large for-profit health system can use its resources to support health system transformation using a strong foundation of data and continuous learning |
 University of Utah Health | AHRQ, 2019 | Care experience; Programmatic return on investment | Intervention complexity Penetration | Patient Cost | Health system leaders created a Resident Value Council to support resident training in quality, safety, efficiency and workflow The University of Utah Health Care Partners program is an initiative that led to the approval of 65 projects and a cost savings of $8.6 million | University of Utah Health provides a strong model for how an LHS can function by investing in value-based care, having sophisticated data operation and a culture and workforce dedicated to continuous improvement |
 Learning Networks | Britto, 2018 | Systematic adoption of EBP; Population health | Intervention complexity Penetration | Patient Cost | The ICN LHS resulted in an 80% clinical remission rate The NPCQIC LHS led to 40% reduced mortality among patients The SPS resulted in a reduction of several hospital acquired conditions by 5%-79% The OPQC LHS led to improved outcomes across multiple areas of patient care Several LHSs invest in the program due to the observed financial benefits to the system and patients | The LHS networks described in this paper are examples of replicable LHSs and have led to improved patient outcomes across multiple diseases and patient populations |
 Geisinger Health System | Foley, 2015 | Systematic elimination of wasteful & ineffective practices | Intervention complexity | Patient | Reduction in no-shows from 47 to 24% as a result of using predictive data modeling | GHS developed innovative analytic techniques to capture data and recognizes that patient engagement with information is key in improving patient experiences and outcomes. Challenges identified in fully realizing the LHS |
 PEDSnet & ICN | Forrest, 2014 | Systematic adoption of EBP Population health | Adoption Penetration Reach | Patient Population | Since its inception in 2007, ICN has grown from 8 to 66 GI care centres across the USA ICN increased remission rates from 55 to 77% | Based on the success of ICN for pediatric patients with inflammatory bowel disease, a national LHS, PEDSnet, will be scaled up and spread across health care organizations and patient populations |
Porcaro, 2022 | Programmatic return on investment | Intervention complexity | Population | n/a | LHS networks can grow with supportive governance that is flexible to changing technology and stakeholder needs | |
 The Ottawa Hospital | Fung-Kee-Fung, 2018 | Systematic adoption of EBP Population health | Adoption Feasibility Penetration Reach Sustainability | Patient Population | Time to diagnosis within 14 days was improved & above provincial target timeline Diagnosis made to 80% of referrals within 28 days, versus 57% for the province Time from referral to 1st treatment decreased from 92 to 47 days (by 48%) Diagnosed patients receiving no treatment decreased from 22 to 16% over 2 years | The implementation of an LHS for cancer patients in Ottawa led to improved time from referral to initial treatment and was sustained over time |
 Neotree | Heys 2022 | Systematic adoption of EBP; Population health; Equity | Acceptability Adoption Appropriateness Feasibility Intervention complexity Penetration Reach | Patient Provider Population | Reduction from 79 to 38% admission rate of hypothermic babies Unnecessary antibiotic prescribing fell from 97 to 2% Improved health care provider confidence & ability to provide newborn care | Initial development of a digital QI system for newborn care shows potential for a sustainable LHS in low resource settings. Neotree is an ongoing project with additional implementation, evaluation and outcome data to be published in future |
 SHOnet | Koscielniak, 2022 | Knowledge to action latency | Feasibility | Population | The SHOnet LHS includes data from over 2 million patient encounters over 10 years | SHOnet provides an example of an LHS for pediatric rehabilitation settings, where data are extracted from EHRs and integrated into clinical care |
 MSQC | Krapohl, 2020 | Knowledge to action latency; Systematic adoption of EBP; Care experience | Adoption Sustainability | Patient Population | Reduced opioid prescribing Less variability between providers across the collaborative | LHS principles can accelerate the translation of evidence into practice and improve patient outcomes |
 PC-ICCN | Levin, 2022 | Systematic elimination of wasteful and ineffective practices; Population health | Appropriateness Penetration | Population | To date, 5364 patients have been referred and 2354 patients have visited the clinic at least once | PC-ICCN is an LHS that allows for the integration of data into clinical care across a provincial program |
 TRANSFoRm | Lim, 2015 | Population health | Intervention complexity | Population | n/a | TRANSFoRm is an example of an LHS that integrates research into clinical practice by working with the EHR as a data collection system |
 IDEA4PS | Moffat-Bruce, 2018 | Systematic adoption of EBP | Acceptability Intervention complexity | Provider | Three pilot studies implemented as part of IDEA4PS led to improvements: Implementation of a falls wheel was effective in engaging patients, clinicians and researchers & improved falls safety; Patient safety indicators study demonstrated how research could improve processes & practices; A telemetry and alarms study led to reduced telemetry days and increased ED throughput | By reframing the role of research in improving outcomes, IDEA4PS has allowed for capacity building and the development of a learning culture |
 ATN/AIR-P | Murray, 2019 | Care experience | Acceptability Adoption Sustainability | Patient | As of 2018, 731 patients were enrolled in the network Parents identified priorities for clinical care | The ATN/AIR-P is an example of how infrastructure enabled improvement and research and allowed for systematic collection of clinical data to inform practice |
 Alliance for Healthier Communities | Nash, 2022b | Systematic adoption of EBP | Acceptability | Population | Preliminary outcomes indicate improved cancer screening and equity across marginalized groups, for a sub-group of the LHS, indicating future potential impact of the broader AHC Over 70 stakeholders actively involved in co-creating the LHS | The LHS success can be attributed to a positive organizational culture, supportive leadership, an EHR that allows for digital data capture, motivated providers and staff, and resources for data support |
 SNEPT | Perito, 2021 | Knowledge to action latency; Population health | Appropriateness Feasibility Sustainability | Patient Population | In 2 years, SNEPT built a network that integrates family and stakeholder input, supports transparency and data-sharing efforts, and includes multicenter collaboration for improved pediatric liver transplantation | Pediatric liver transplantation care can be advanced through implementing a LHS and leveraging patient engagement, big data, technology and thought leaders to address most challenging issues |
 myAva | Satveit, 2017 | Systematic adoption of EBP | Acceptability Adoption | Patient | The myAva platform highlighted a need for better PCOS care; MyAva allowed for more patient empowerment and engagement, but revealed challenges in payment structure and provider knowledge | Significant improvements noted after implementing the myAva platform with expectations for continued improvement in PCOS care overtime |
 ClaudicatioNet | Sinnige, 2022 | Knowledge to action latency; Population health; Care experience | Intervention complexity Penetration | Patient Population | ClaudicoNet is a national network of over 2100 physical therapists and includes a registry of routinely collected patient data, which is used to inform care | The Claudicatio Net LHS is an example of how physical therapists can facilitate continuous learning and integrate clinical data into patient care |
 CORE | Taylor, 2021 | Knowledge to action latency | Intervention complexity | Population | CHOSEN antibiotic stewardship case study led to 10% decrease in inappropriate prescribing & a new process for tracking inappropriate prescribing | Case studies exhibit how practice informs research and research informs practice change in an LHS model |
 EQUIPPED | Vandenberg, 2020 | Systematic adoption of EBP | Adoption Feasibility Sustainability | Population | Significant reduction in prescribing potentially inappropriate medications at one site and benzodiazepine prescriptions reduced at all sites, 12-months after implementation | EQUIPPED is feasible to implement and shows promise in reducing inappropriate medication prescribing |
Vaughan 2021 | Knowledge to action latency; Population health | Adoption Intervention complexity Sustainability | Patient | Following implementation of EQUIPPED, one site showed a statistically significant decrease in PIMs prescribing rates, while two sites showed no difference | EQUIPPED is a model for addressing medication safety through sequential implementation, with opportunities for scale and spread in other community-based settings | |
 n/a | Varnell, 2022 | Population health | Intervention complexity | Patient | Cholesterol level checks increased from 84 to 95% Number of dyslipidemia patients on statins increased from 52 to 88% Number of patients with healthy LDL level went from 65 to 83% Improved rates of rejection-free transplants from 80 to 90% Additional improvements in biochemical markers were observed | This is an example of how a learning health network can be implemented and improve outcomes for a pediatric nephrology population |
Qualitative studies | ||||||
 RCLS-CF | Dixon-Woods, 2020 | Work life for care teams | Appropriateness Intervention complexity | Patient Provider | Although all stakeholders shared the same vision and values for the LHS, there are challenges in implementation related to the social and technical aspects of LHSs | |
Themes: | T1: Design stakeholders’ views of the foundations of RCLS-CF (co-production and its transformational potential)—every clinical interaction and patient input could generate meaningful knowledge to improve patient care and decision making T2: Design stakeholders’ views of the technical prerequisites for the learning system—to facilitate more co-produced care plans, need a dashboard that could be viewed by both patients and clinicians T3: Design stakeholders’ views of the social conditions necessary for program implementation—social, cultural and practical barriers identified in gaining universal LHS buy-in from clinicians and patients T4: Design stakeholders’ views of tensions and challenges in implementing the LHS—possible tensions identified in inhibiting the implementation of the LHS, however they can be managed | |||||
 n/a | Enticott, 2020 | Work life for care teams | Intervention complexity Sustainability | Patient | Structure, governance, trust, culture, vision, leadership and quality data access seen as crucial to implementing and sustaining a LHS | |
Themes: | T1: Systematic approaches and iterative, continuous learning with implementation into healthcare contributing to new best-practice care T2: Broad stakeholder, clinician and academic engagement, with collective vision, leadership, governance and a culture of trust, transparency and co-design T3: Skilled workforce, capability and capacity building T4: Resources with sustained investment over time T5: Data access, systems and processes being integral to a sustainable LHS | |||||
 n/a | Jeffries, 2018 | Work life for care teams | Acceptability Feasibility | Provider | There is value in integrating information technology and pharmacists in the general practice setting to optimize safe medication administration | |
Themes: | T1: Coherence – dashboard was perceived as easy, valuable and able to integrate by working with other staff T2: Cognitive participation – mixed engagement from different stakeholders but by leading the work, pharmacists were able to show value of the intervention T3: Collective action – communication & collaboration between practitioners was key to success T4: Reflexive monitoring – pharmacists helped improve features of the dashboard, which was a tool seen to enhance patient care & changes in work | |||||
 Alliance for Healthier Communities | Nash, 2022 (a) | Work life for care teams | Appropriateness Intervention complexity | Provider | Key elements needed to establish a LHS in primary care include having a positive organizational culture and supportive leadership, an integrated data entry system, motivated providers and staff with capacity to engage with the LHS and access to resources to support LHS initiatives and data collection | |
Themes: | T1: Shared organizational goals and culture—viewed as important for a functioning LHS and this was already in place in some community health centres T2: Data quality—good data quality was identified as necessary for a LHS T3: Resources—limited time for data entry and quality improvement was a barrier T4: People—having leadership who supports LHS is important, but some centres face resistance from staff T5: Motivation—different reasons for motivation among staff to adopt a LHS model, such as improving patient care and making work load more efficient | |||||
 n/a | Steels, 2021 | Work life for care teams | Intervention complexity | Patient Provider | This paper outlines the challenges in implementing a LHS in Northern England, while also highlighting that this work led to building IT and health informatics infrastructure across NHS organizations | |
Themes: | T1: Challenges in the implementation of LHS pathways—Several challenges identified as sub-themes including time constraints, data access, long-term sustainability and commitment, different working cultures and priorities and communication T2: Benefits to the CHC approach for both staff and patients—Several benefits identified as sub-themes including benefits for staff involved in the CHC program and patients of the CHC program activities | |||||