Skip to main content

The pragmatic, rapid, and iterative dissemination and implementation (PRIDI) cycle: adapting to the dynamic nature of public health emergencies (and beyond)

A Correction to this article was published on 19 August 2021

This article has been updated

Abstract

Background

Public health emergencies—such as the 2020 COVID-19 pandemic—accelerate the need for both evidence generation and rapid dissemination and implementation (D&I) of evidence where it is most needed. In this paper, we reflect on how D&I frameworks and methods can be pragmatic (i.e., relevant to real-world context) tools for rapid and iterative planning, implementation, evaluation, and dissemination of evidence to address public health emergencies.

The pragmatic, rapid, and iterative D&I (PRIDI) cycle

The PRIDI cycle is based on a “double-loop” learning process that recognizes the need for responsiveness and iterative adaptation of implementation cycle (inner loop) to the moving landscapes, presented by the outer loops of emerging goals and desired outcomes, emerging interventions and D&I strategies, evolving evidence, and emerging characteristics and needs of individuals and contexts. Stakeholders iteratively evaluate these surrounding landscapes of implementation, and reconsider implementation plans and activities.

Conclusion

Even when the health system priority is provision of the best care to the individuals in need, and scientists are focused on development of effective diagnostic and therapeutic technologies, planning for D&I is critical. Without a flexible and adaptive process of D&I, which is responsive to emerging evidence generation cycles, and closely connected to the needs and priorities of stakeholders and target users through engagement and feedback, the interventions to mitigate public health emergencies (e.g., COVID-19 pandemic), and other emerging issues, will have limited reach and impact on populations that would most benefit. The PRIDI cycle is intended to provide a pragmatic approach to support planning for D&I throughout the evidence generation and usage processes.

Peer Review reports

Background

Public health emergencies—such as the 2020 COVID-19 pandemic—dramatically accelerate the need for evidence generation and synthesis, as well as the rapid dissemination and implementation (D&I) of evidence-based practices and interventions [1,2,3]. In a matter of weeks in late winter 2020, the scientific enterprise in clinical and translational research in public health and medicine was nearly universally reoriented to pressing and emergent COVID-19-related concerns. In addition to research on tests and treatments, there is a need for studying emerging healthcare system-level interventions. From the rapid adoption of telehealth services across nearly every health discipline [4,5,6], to the development and implementation of procedures for risk stratification and delaying elective procedures during the pandemic [7], to strategies for reopening and revamping healthcare and messaging considering physical distancing principles [8, 9], the pandemic became a driving force for rapid change in healthcare and public health systems.

D&I science has emerged as an evolving field to address the well-documented gap between research and practice [10]. Dissemination specifically relates to the active or planned communication of best practices and evidence-based interventions to encourage their widespread adoption among key decision-makers across a range of settings, whereas implementation focuses on factors and strategies to support the adoption and the routine use and delivery of the recommended practices or evidence-based interventions in real-world clinical and community settings [11]. The D&I of organizational- and system-level interventions, practices, or policies often involve modifying existing structures (physical or technological), redesigning processes of work and clinical workflow, and redefining roles, operating within a broader complex and dynamic organizational or healthcare system context. In emergencies, these already challenging modifications can become even more burdened by strained resources, competing demands, and overextended or strained systems and healthcare workers. Decision-makers may think of systematic planning for D&I as expendable or irrelevant in emergencies—perhaps perceived as an academic exercise or too time-consuming with limited added value. However, unsuccessful or inequitable implementation of resource-intensive system-level interventions can result in treatment delays, inequities in access to and delivery of care, and poor population health outcomes—including death. For example, the inequitable delivery of care may partially explain the racial and ethnic disparities in COVID-19 mortality [12, 13], or delays in provision of diagnostic tests may reduce the effectiveness of contact-tracing strategies [14].

However, classical D&I frameworks and approaches may need rapid adaptations to be pragmatic and of use in rapidly evolving emergency situations. An important feature of emergencies is the quick, dynamic, and unpredictable course of events and evolving nature of science [15], which makes planning for D&I challenging. The following are some examples:

  • The health or healthcare problem itself may be dynamic and rapidly changing. In February 2020, the main concern of many health systems was implementing case finding and quarantine strategies; in March 2020, it was allocating intensive care unit (ICU) beds and ventilators; in May 2020, safe strategies to gradually lifting lockdowns [16]; in early 2021, how to administer mass vaccination; and in summer 2021, how to address the significant vaccine hesitancy [17]. This dynamic evolution of the problem affects the contexts of the study, selection of interventions and implementation strategies, and evaluation frameworks.

  • The evidence and associated interventions or solutions and strategies to support delivery of evidence are not fixed, as the evidence for effectiveness of cloth masks, hydroxychloroquine, antibody tests, and various diagnostic approaches has evolved rapidly [18]. The COVID-19 “infodemic” [19] resulted in the outpouring of misinformation, which complicated the separation of fact from fiction and contributed to confusion in messaging among the public, as well as erosion of public trust in the information provided.

  • The contexts/settings in which COVID-19 is being transmitted and in which testing, vaccination, and treatment occur are dynamic. Both inner settings (hospital resources, hospital policies, capacity, exhaustion) [20, 21] and outer settings (effective social distancing, vaccination rates, economic constraints, state/national policies) [22, 23] are changing by the day and over time, and require continuous monitoring and reconsideration of plans.

  • Key stakeholders (e.g., healthcare workers, patients, community members, leadership) within systems and broader communities have evolving concerns, needs, and values. Their readiness, knowledge, and capabilities are evolving based on changing circumstances and contexts; and these stakeholders’ trust in medical institutions and perceptions of the importance of scientific evidence varies.

  • There is usually redundancy and parallelism within systems, which positively and negatively affects the implementation of evidence-based processes and practices. On the positive side, we can learn from the experience of other health systems who deal with similar situations and challenges (e.g., in allocating ventilators and ICU beds) [24]. On the negative side, redundancy and parallelism and lack of communication may result in confusion, conflicts, dilution of resources, burn-out, and lack of monitoring and evaluation of what practices are both feasible and have impact.

  • Additional complexities and considerations that need to be addressed include the striking racial/ethnic inequities that have been apparent with respect to COVID-19 morbidity and mortality, related in part to embedded systems that create and reinforce structural and interpersonal forms of inequity, discrimination, and racism [13, 25].

To align the science of D&I with the practice of D&I in real-world settings, it is important to explicate how health systems can apply D&I frameworks and methods rapidly, effectively, equitably, and with few resources to guide local adoption of evidence-based interventions or emerging best practices/protocols (informed by the best available evidence at the time). In this paper, we reflect on widely adopted D&I frameworks and tools and how they can be adapted to address dynamic trajectories of public health emergencies.

The pragmatic, rapid, and iterative D&I (PRIDI) cycle

Figure 1 shows the PRIDI model for D&I. It depicts the dynamic connection between the cyclical process of executing and evaluating D&I (centre), the interventions and strategies (left side), the evolving nature of evidence (bottom), the multilevel nature of the context (upper side), and goals and outcomes of D&I (right side). Consistent with recent emphasis on the iterative and pragmatic nature of D&I [26, 27], the implementation journey is not a linear process, particularly in the fluid and dynamic contexts of emergencies. This cyclical process of Assess > Plan > Do > Evaluate > Report should be done rapidly and iteratively as an intervention and strategies to support its implementation are rolled out [28], a process that highlights the overlap between D&I and quality improvement approaches [29].

Fig. 1
figure1

The PRIDI cycle

As shown in Fig. 1, the cyclical process resides at the centre of the PRIDI model. The inner circle involves the cycle of D&I, which activates and is influenced by the outer circle that involves revisiting the mental models, goals and outcomes, interventions and D&I strategies, and individuals and contexts through the course of D&I. It resembles a double-loop learning model [30, 31]. While single-loop learning involves incremental corrective actions aiming to improve current processes, and are most suitable for more stable conditions and contexts, we argue that the dynamic and evolving nature of emergencies calls for more complex learning processes and rapid refinements. If we apply Plan > Do > Study > Act (PDSA) cycles using existing models (i.e., single-loop learning), we might fail to learn from the higher-order feedback loops that require more than incremental improvements in efficiency and time [32]. Second-order learning might inform entirely different approaches based on different assumptions and different mental models. This mindset could even be extended to triple-loop learning (meta-learning, i.e., learning about learning) through which the process of reflection and learning is adapted in response to emerging complexities [33], considering the scarcity of time and resources.

To the extent possible, monitoring and iterative evaluation should be prioritized, and results should be regularly communicated and interpreted in partnership with stakeholders, and meaningfully and consistently incorporated in any redesign or planned adaptations/modifications within the system [2]. If an intervention or a D&I strategy is ineffective (or worse, proves harmful), it should be modified or abandoned (de-implemented) in a timely manner. Evaluations and monitoring may include information that changes the nature of the evidence supporting the effectiveness of the intervention itself or strategies to support its use (see cyclical path from the implementation to intervention and strategies).

The engagement of stakeholders within these dynamic contexts is critical throughout this process to understand what is working or not and why, where inequities are emerging, and the feasibility and acceptability of the programmes and practices. The double-loop nature of the process also has implications for engagement of diverse stakeholders and providing opportunities where people feel free to express contrarian views, thus challenge conventional assumptions. For example, suppose we assume that African Americans by virtue of higher SARS-CoV-2 infection rates and worse COVID-19 outcomes should be prioritized for testing, vaccination, and treatments. This would be a reasonable assumption from which we could develop cyclical PDSA strategies for messaging regarding testing, vaccination, and treatment. This assumption suggests that finding ways to promote awareness and access in the African American community regarding where to get tested and where to receive vaccines and treatments will reduce disparities in infection. Yet, if the African American community were at the table and divergent views were encouraged based on recognition of second-order learning, members might express reservations about COVID-19 testing, vaccination, and treatment, including risks related to family separation, forced quarantine without pay, and greater stigma. Similarly, members might voice deep scepticism towards receiving vaccines, including mistrust of government statements, concerns about a vaccine that has been rushed to market, and/or concerns about the vaccine safety, or treatments perceived as expensive, inaccessible, or unsafe. This second-order learning might suggest a fundamentally different approach from that of incremental changes in content, dose, or frequency of messages.

In Table 1, we summarize the suggested information that should be collected, discussed, and re-evaluated at each round of PRIDI cycle

Table 1 A template for recording progress in PRIDI cycles

Interventions and strategies

Consistent with the RE-AIM [reach, effectiveness, adoption, implementation, and maintenance]/PRISM [Practical, Robust Implementation and Sustainability Model] framework [34], the interventions and strategies to facilitate their dissemination, adoption, and use are the key elements of D&I efforts, which are shown on the left side of Fig. 1:

  • The intervention (e.g., evidence-based practice, policy, programme, treatment) to be disseminated and implemented (e.g., COVID-19 vaccines, diagnostic tests, and potential treatments, setting up an online meeting model for grand rounds, best practices for mental health screening among COVID-19 patients/healthcare workers, safety protocols and policies for birthing mothers) (e.g., the “what”)

  • D&I strategies involve the processes, approaches, or interventions that facilitate and enhance the proactive D&I of the interventions. Examples include tailored email/online communication for the self-assessment platform, literacy-appropriate instructional packages for patients about the COVID-19, staff training/education to learn about the new workflow, and motivational incentives to enhance staff participation in grand rounds. See Powell et al. (2015) for a taxonomy of implementation strategies based on the Expert Recommendations for Implementing Change (ERIC) project [35].

Interventions generally include a core (the essence or function of the intervention that is responsible for its impact) and an adaptable periphery (that could be modified to adapt to various contexts and situations) [36]. Ignoring the distinction of these two components may result in rigid interventions that are not sufficiently flexible to survive varying and unprecedented contextual variations and barriers, or that are too complex or costly to be implemented. As such, it is important that a flexible approach is taken during the design of D&I activities, and the local implementers are trusted to adapt the intervention to fit into their own local contexts, resources, needs, and policies. Consequently, we added adaptation as an important phase in the PRIDI cycle (centre of Fig. 1).

D&I adaptation models may be useful to help guide planned adaptations (e.g., ADAPT-ITT [assessment, decision, administration, production, topical experts, integration, training, testing]) [37], to help balance considerations of fit and fidelity. Ideally, the core component of the intervention should be defined, dynamically updated (as changes are made over time), and communicated; relevant data could be collected through iterative evaluations to understand the impact of both the core elements of the programme and any planned adaptations made, as well as the evolution of the programme across its life course [38]. For example, preventive health messages delivered through health organizations such as the Centers for Disease Control and Prevention (CDC) [39] and local and state health authorities typically target broad audiences and are not always adapted to the needs, values, or expectations of vulnerable individuals and communities. The messages may not address the limited behavioural control of the target audience (e.g., in practising social distancing or staying at home), may not include information about local services and resources, and may not be adapted to the literacy levels of individuals who may be at greatest risk for COVID-19 [40, 41]. For example, an individual living in a dense, multigenerational household may have difficulty adhering to isolation and physical distancing guidelines or may lack digital technology to access electronic health literacy resources [42, 43]. Communities of colour, including Black Americans who have experienced striking COVID-19 inequities, are more likely to be exposed to multiple layers of structural racism, including living in buildings and neighbourhoods that are more crowded and have poorer infrastructures, irrespective of income, and may feel unsafe using face masks in public. Asian or Asian American groups may face stigma, discrimination, and violence related to the disease due to misinformation about its origins and spread. An individual living in unstable economic conditions who needs to work may not be able to self-isolate for the recommended period while symptomatic. Therefore, standard messaging should be adapted to the needs, expectations, and capacities of diverse subgroups and populations to be able to educate or motivate and improve the understanding of COVID-19 and both individual and community responses to it.

Evidence base

COVID-19 is a great example of the importance of implementing solutions as their evidence base is continuously evolving over time. This includes the evidence supporting diagnostic tests and therapeutic interventions, systemic interventions (such as lockdowns and public mask use, vaccination strategies, and awareness of and access to evidence-based treatments), and social issues and their corresponding interventions (such as approaches to address vaccine hesitancy). Such interventions are being implemented while their evidence base is limited and evolving [44]. Under time constraints and public pressure, decision-makers feel an urgent need to make prompt and clear decisions that are acceptable to the public and based in science. But assessing the quality of evidence and crafting careful recommendations are as critical in emergencies as in usual practice. For example, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for assessing the quality of evidence and formulating recommendations, which has been widely used in guideline and policy development [45], has been adapted to emergencies and shorter time frames to address the need for evidence for effectiveness of interventions to address COVID-19 [46].

The experience of COVID-19 boosted efforts to revise the traditional evidence pipeline to be more responsive to dynamic changes [47], and to attend to social aspects of evidence generation such as equity, acceptability, and feasibility [48]. Depending on the specific health or healthcare issue and its corresponding interventions, the nature of the evidence and indicators of evidence quality may differ. While high-quality academic research is the broadly accepted source of evidence, many public health and social issues require a broader definition of evidence, including local evaluations, policy documents, population-based data, community-defined evidence, and professional experience [49]. For example, the challenges to mask use and vaccine hesitancy, and locally acceptable solutions to address those challenges, could best be addressed using localized and culturally tailored surveys, qualitative interviews, and focus groups, as well as surveying comparable subgroups in regularly collected surveillance data and polls. Recognizing the shortcomings of existing surveillance systems in response to emergencies such as COVID-19, and the development of dynamic, adaptable, and responsive data infrastructure and mechanisms have been recognized as a health system priority.

Goals and outcomes

Evaluation is not a one-time post-intervention process in D&I; it is an iterative, ongoing process that can enhance and inform the evolvability of evidence-based interventions and strategies, including their design, adaptation, refinement, and delivery throughout the process of D&I. Consequently, intended goals and outcomes of D&I should ideally be incorporated from the beginning (right side of Fig. 1). In emergency planning, the value of learning from continuous evaluation is even more essential, as the path forward can be more uncertain, the interventions are more experimental and their evidence-base evolving, and the clinical situation and healthcare contexts can change quickly. As such, it may be useful for decision-makers to have a compass to guide them as to whether they are moving in the right direction or need to reassess and redesign and challenge existing models that might not fit with such a dynamic context.

RE-AIM provides a systematic conceptual framework to guide the planning, adaptation, and evaluation of the D&I activities, programmes, practices, or policies [26, 50]. An intervention should reach the target populations equitably (Did we reach the those who needed the intervention or would benefit the most from it?); be effective (Did the intervention achieve its goals and impact on health behaviours/outcomes?); be widely adopted (Did the settings and stakeholders/decision-makers adopt the intervention?); be implemented (Did the target users or implementers actually use it as it was intended? How was it adapted?); and be maintained/sustained (Did the target users continue using it over time and did it continue to have long-term impact?). Importantly, in light of dynamic contexts [51], RE-AIM can be iteratively applied to track these D&I indicators to help document where inequities and challenges in each of these areas are arising and to inform refinements of adaptations to respond to changing system challenges (e.g., costs, resources), population needs/values, and evolving evidence [26, 52].

Glasgow et al. (2020) applied RE-AIM iteratively in a participatory process to support prospective adjustments during implementation projects [27]. Through this cyclic process, it may be useful for implementing agents/teams to receive practical and customized feedback about their performance, so they can understand progress in comparison to the original goals or in comparison to other implementers in their setting, and correct their path if needed [53]. RE-AIM dimensions may differ in terms of importance and feasibility of assessment. At each round of the cycle, stakeholders can decide which RE-AIM dimensions are more important, more in need of improvement, and are potentially more feasible to assess [27, 52].

Individuals and contexts

The upper side of Fig. 1 shows the multilayered and complex nature of contextual factors and their role in determining the success or failure of D&I efforts. It is critical to consciously consider the complexity of personal, interpersonal, organizational, social, economic, policy, community, and cultural contexts at the design phase, and across the continuous process of re-evaluation and adaptations throughout implementation phases. A seemingly useful intervention may fail to be realized, since patients may find it irrelevant to their needs and characteristics, or may face certain financial and structural/logistical barriers to accessing and using it, or may not trust the source of the intervention; staff or administrators may find it burdensome (since many staff who are running these programmes are delivering them in addition to their normal workload, they may be overwhelmed or have many competing demands under limited resources); and at the organizational level, infrastructure needed to deliver the programme may have geographical, demographic, and structural limitations. External environment factors such as policies, economic challenges, and cultural and social norms are also rapidly changing. For example, adherence to long-term physical/social distancing may vary based on demographics and cultural backgrounds [54]; country-level and state-level disease mitigation policies may affect the implementation and sustainment of interventions [55]; and wider economic impact of the lockdowns and current mitigation strategies may affect the effectiveness and sustainment implementation of those mitigation strategies (through activation of feedback loops) [56]. Many of these barriers are difficult to overcome in emergency situations; however, having the tools to recognize and address them may facilitate development of innovative alternative solutions and enhance the reach and impact of evidence-based intervention, particularly with prioritization on health equity.

Stakeholder engagement

It may seem like an inconvenient time to engage stakeholders in the context of emergency situations. However, even brief engagement of stakeholders has immense benefits that make it worth prioritizing, at the design phase and through the cyclical process of re-evaluation and redesign [57]. Stakeholders that are actively involved and engaged in the processes of D&I may [58]:

  • feel more invested to help disseminate, implement, and sustain an intervention or public health practice;

  • be prepared cognitively and operationally and be more committed to execute plans for adoption of an intervention or public health practice;

  • identify setting- or cultural-specific barriers that may have been have missed;

  • provide real-time feedback on whether strategies are working and inform important refinements or adaptations of interventions and strategies; and

  • enhance relevance and fit, and may propose innovative solutions.

Stakeholder engagement may be applied at different degrees along the spectrum of implementation, depending on the availability of time and resources and the nature of the intervention and D&I strategies [59]. Even at the most basic levels of engagement (i.e., information provision and consultation), involving stakeholders in planning, dissemination, and interpretation and sense-making can facilitate preparedness and elicitation of feedback critical in the success of D&I efforts. Given its importance in informing and guiding the process of D&I, “stakeholder engagement” is shown as a circle surrounding all phases of the D&I cycle in the PRIDI model (Fig. 1).

Leadership

All mentioned processes are only possible under the context of strong organizational commitment [60], as well as transformational (inspiring and motivating) and transactional (providing contingent rewards) leadership [61], that have shown to predict implementation success [62]. Organizational leaders can help maximize the fit between all aspects of D&I activities [34], make and effectively communicate strategic decisions, and are nimble and ready to change course midway if the iterative evaluations suggest the need for modification of goals and strategies. A successful crisis leader should be well-versed with the subject matter (e.g., public health) or consult team members with expertise in the specific area; should make evidence-based and timely decisions, while continuously collecting data from the environment; should inspire trust and confidence; and should feel responsible for the safety and welfare of the team members [63]. In emergency situations, it is very likely that multiple groups try independently to develop solutions, which may result in fragmented efforts and confusion. The leader should develop an effective project management structure as well as an atmosphere in which teams and individuals have the means and feel free to express criticisms and suggest alternative solutions. Finally, the leader should highlight the importance of and provide resources necessary to apply the processes and principles central to the PRIDI framework.

Conclusions

In this paper, we reflected on the cyclical model of Assess > Plan > Do > Evaluate > Report [28], the RE-AIM/PRISM framework [34], and recent advancement of RE-AIM to incorporate equity [26] and to inform rapid implementation [27]. We proposed the PRIDI model that takes the dynamic nature of problems, interventions, evidence, contexts, and stakeholders into account. D&I in the context of emergency should be a continuous and iterative process. RE-AIM provides a framework for the evaluation of D&I activities, that includes reach, effectiveness, adoption, implementation, maintenance. Recent extensions of this model can also inform more explicit consideration of understanding and addressing health equity and equitable implementation over time and in dynamic contexts [26]. Interventions are disseminated and implemented in complex and multilayered contexts. Overlooking these complexities will hamper the success of the adoption, use, and impact of the intervention.

The cyclical process of D&I informs double-loop learning processes that may result in revisiting mental models, goals and outcomes, interventions and D&I strategies, and individuals and contexts. The results of cyclical evaluations should also be communicated with local implementers and stakeholders through customized and actionable feedback. Stakeholder engagement is a key solution to understand and address contextual variations and barriers. It is a continuum ranging from informing the stakeholders to co-ownership, and will be critical to addressing some of the striking racial/ethnic and setting inequities evidenced for COVID-19, including redistribution of decision-making and resources with the community. Learning from and with communities is broadly recognized as an important source of evidence to guide learning organizations and health systems [64].

During an epidemic, the priority of the health system is provision of evidence-based prevention and treatment, while the priority of the research community is rapid development of effective diagnostic and therapeutic technologies. Even though the health system priority at this moment is the provision of the best care to the individuals in need and the development of effective diagnostic and therapeutic technologies [2], prospective, flexible D&I planning is also critical [2, 65]. Without planning and tailoring, meaningful partnerships, and engagement of local stakeholders, D&I strategies will never reach target populations that would most benefit, but rather will be primarily accessed and used by sociodemographic groups that face fewer structural barriers to care (hence deepening the equity gap); and will not sustain as intended. While limited organizational readiness and lack of time and resources are challenges to effective D&I plans, emergency response interventions may fail to meet their objectives and waste limited resources if critical D&I considerations are ignored.

Key to preparing for national emergencies such as COVID-19 are the development of infrastructures, organizational cultures, trainings, and establishment of processes towards a rapid-learning health system (LHS) [66, 67] that is grounded in D&I as its key component [67]. These steps will prepare healthcare systems and organizations to effectively respond to future emergencies. An LHS, as a type of learning organization, develops capacities for both single-loop and double-loop learning at the individual and organizational levels [64]. An LHS paradigm facilitates the processes of evidence generation and synthesis through the development of interoperable data platforms and infrastructure to provide real-time and adaptable data to continuously inform policies and practices [68]. The iterative process of data aggregation, analysis, interpretation, feedback, and change is responsive to the emergent nature of evidence and the need for learning from and with stakeholders, including communities and frontline practitioners. COVID-19 underscores the importance of accelerating progress towards creation of genuine LHS [68, 69].

This paper calls for dynamic and adaptive D&I models that are responsive to the rapid and unpredictable nature of emergencies through a double-loop process (or triple-loop, considering time, resources, and complexity) involving rapid and iterative cycles of implementation through continuous engagement of stakeholders that are embedded in and adapted for the emergent and evolving nature of goals, interventions, evidence base, and contexts. Establishing these models is essential to preparing for the next national health crisis.

Availability of data and materials

Data sharing is not applicable to this article as no data sets were generated or analysed during the current study.

Change history

Abbreviations

CDC:

Centers for Disease Control and Prevention

D&I:

Dissemination and implementation

ERIC:

Expert recommendations for implementing change

PDSA:

Plan, do, study, act

PRIDI:

Pragmatic, rapid, and iterative dissemination and implementation

PRISM:

Practical, Robust Implementation and Sustainability Model

RE-AIM:

Reach, effectiveness, adoption, implementation, and maintenance

References

  1. 1.

    Yang L, Yin J, Wang D, Rahman A, Li X. Urgent need to develop evidence-based self-help interventions for mental health of healthcare workers in COVID-19 pandemic. Psychol Med. 2020. https://doi.org/10.1017/S0033291720001385.

    Article  PubMed  PubMed Central  Google Scholar 

  2. 2.

    Chambers DA. Considering the intersection between implementation science and COVID-19. Implement Res Pract. 2020;1:0020764020925994. https://doi.org/10.1177/0020764020925994.

    Article  Google Scholar 

  3. 3.

    Alwan NA, Bhopal R, Burgess RA, et al. Evidence informing the UK’s COVID-19 public health response must be transparent. Lancet (London, England). 2020;395(10229):1036–7.

    CAS  Article  Google Scholar 

  4. 4.

    Hollander JE, Carr BG. Virtually perfect? Telemedicine for COVID-19. N Engl J Med. 2020;382(18):1679–81. https://doi.org/10.1056/NEJMp2003539.

    CAS  Article  PubMed  Google Scholar 

  5. 5.

    Keesara S, Jonas A, Schulman K. COVID-19 and health care’s digital revolution. N Engl J Med. 2020;382(23):e82. https://doi.org/10.1056/NEJMp2005835.

    CAS  Article  PubMed  Google Scholar 

  6. 6.

    Ohannessian R, Duong T, Odone A. Global telemedicine implementation and integration within health systems to fight the COVID-19 pandemic: a call to action. JMIR Public Health Surveill. 2020;6: e18810.

    Article  Google Scholar 

  7. 7.

    Stahel P. How to risk-stratify elective surgery during the COVID-19 pandemic? Patient Saf Surg. 2020;14:8.

    Article  Google Scholar 

  8. 8.

    Bailey JP, Hess FM. A Blueprint for Back to School. Washington, DC: American Enterprise Institute; 2020.

  9. 9.

    Carlo F, Ichino A, Rustichini A. Restarting the economy while saving lives under COVID-19 (November 8, 2020). SSRN. 2020. https://doi.org/10.2139/ssrn.3580626.

    Article  Google Scholar 

  10. 10.

    Brownson R, Colditz G, Proctor E. Dissemination and implementation research in health: translating science to practice. Oxford: Oxford University Press; 2018.

    Google Scholar 

  11. 11.

    Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3:32. https://doi.org/10.1186/s40359-015-0089-9.

    Article  PubMed  PubMed Central  Google Scholar 

  12. 12.

    Dyer O. COVID-19: Black people and other minorities are hardest hit in US. BMJ. 2020;369: m1483. https://doi.org/10.1136/bmj.m1483.

    Article  PubMed  Google Scholar 

  13. 13.

    Gross CP, Essien UR, Pasha S, Gross JR, Wang S, Nunez-Smith M. Racial and ethnic disparities in population-level COVID-19 mortality. J Gen Intern Med. 2020;35(10):3097–9. https://doi.org/10.1007/s11606-020-06081-w.

    Article  PubMed  PubMed Central  Google Scholar 

  14. 14.

    Kretzschmar ME, Rozhnova G, Bootsma MCJ, van Boven M, van de Wijgert JHHM, Bonten MJM. Impact of delays on effectiveness of contact tracing strategies for COVID-19: a modelling study. Lancet Public Health. 2020;5(8):e452–9. https://doi.org/10.1016/S2468-2667(20)30157-2.

    Article  PubMed  PubMed Central  Google Scholar 

  15. 15.

    Uitdewilligen S, Waller M. Adaptation in multiteam systems: the role of temporal semistructures. Multiteam systems. An organization form for dynamic and complex environments. In: Zaccaro S, Marks M, DeChurch L, editors. Multiteam systems: an organization form for dynamic and complex environments. Routledge: Milton Park; 2012. p. 365–94.

    Google Scholar 

  16. 16.

    Hauck, G.; Gelles, K.; Bravo, V.; Thorson, M. Five months in: A timeline of how COVID-19 has unfolded in the US. USA Today, 23 June 2020.

  17. 17.

    Dror AA, Eisenbach N, Taiber S, et al. Vaccine hesitancy: the next challenge in the fight against COVID-19. Eur J Epidemiol. 2020;35(8):775–9.

    CAS  Article  Google Scholar 

  18. 18.

    The Centre for Evidence-Based Medicine. Oxford COVID-19 Evidence Service. https://www.cebm.net/oxford-covid-19-evidence-service/. Accessed Jun 2021.

  19. 19.

    Orso D, Federici N, Copetti R, Vetrugno L, Bove T. Infodemic and the spread of fake news in the COVID-19-era. Eur J Emerg Med. 2020;27(5):327–8. https://doi.org/10.1097/MEJ.0000000000000713.

    Article  PubMed  Google Scholar 

  20. 20.

    Cavallo JJ, Donoho DA, Forman HP. Hospital capacity and operations in the coronavirus disease 2019 (COVID-19) pandemic—planning for the Nth patient. JAMA Health Forum. 2020;1(3):e200345–e200345. https://doi.org/10.1001/jamahealthforum.2020.0345.

    Article  Google Scholar 

  21. 21.

    Kumar P, Kattan O, Broome B, Singhal S. Reassessing COVID-19 needs: how providers can reexamine their surge capacity, supply availability, workforcereadiness, and financial resiliency. NEJM Catal. 2020. https://doi.org/10.1056/CAT.20.0112.

    Article  Google Scholar 

  22. 22.

    Gupta S, Nguyen TD, Rojas FL, Raman S, Lee B, Bento A, et al. Tracking Public and Private Responses to the COVID-19 Epidemic: Evidence from State and Local Government Actions [Internet]. Cambridge (MA): National Bureau of Economic Research; 2020 Apr [cited 2021 July 30]. (NBER Working Paper No. 27027). Available from: https://www.nber.org/papers/w27027.

  23. 23.

    Colenda CC, Applegate WB, Reifler BV, Blazer DG 2nd. COVID-19: financial stress test for academic medical centers. Acad Med. 2020;95(8):1143–5. https://doi.org/10.1097/ACM.0000000000003418.

    Article  PubMed  Google Scholar 

  24. 24.

    Chokshi DA, Katz MH. Emerging lessons from COVID-19 response in New York City. JAMA. 2020;323(20):1996–7. https://doi.org/10.1001/jama.2020.7310.

    Article  PubMed  Google Scholar 

  25. 25.

    Craven J. How racial health disparities will play out in the pandemic. Available at: https://slate.com/news-and-politics/2020/03/how-racial-health-disparities-will-play-out-in-the-coronavirus-pandemic.html. Retrieved 2021 July 30.

  26. 26.

    Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health. 2020;8:134.

    Article  Google Scholar 

  27. 27.

    Glasgow RE, Battaglia C, McCreight M, Ayele RA, Rabin BA. Making implementation science more rapid: use of the RE-AIM framework for mid-course adaptations across five health services research projects in the veterans health administration. Original research. Front Public Health. 2020. https://doi.org/10.3389/fpubh.2020.00194.

    Article  PubMed  PubMed Central  Google Scholar 

  28. 28.

    Harden SM, Smith ML, Ory MG, Smith-Ray RL, Estabrooks PA, Glasgow RE. RE-AIM in clinical, community, and corporate settings: perspectives, strategies, and recommendations to enhance public health impact. Perspective. Front Public Health. 2018. https://doi.org/10.3389/fpubh.2018.00071.

    Article  PubMed  PubMed Central  Google Scholar 

  29. 29.

    Tyler A, Glasgow RE. Implementing improvements: opportunities to integrate quality improvement and implementation science. Hosp Pediatr. 2021;11(5):536–45. https://doi.org/10.1542/hpeds.2020-002246.

    Article  PubMed  Google Scholar 

  30. 30.

    Kim H, MacDonald RH, Andersen DF. Simulation and managerial decision making: a double-loop learning framework. Public Adm Rev. 2013;73(2):291–300. https://doi.org/10.1111/j.1540-6210.2012.02656.x.

    Article  Google Scholar 

  31. 31.

    Bakacsi G. Managing crisis. Strateg Manag Int J Strateg Manag Decision Support Syst Strateg Manag. 2010;15(3):3–9.

    Google Scholar 

  32. 32.

    Northridge ME, Metcalf SS. Enhancing implementation science by applying best principles of systems science. Health Res Policy Syst. 2016;14(1):74. https://doi.org/10.1186/s12961-016-0146-8.

    Article  PubMed  PubMed Central  Google Scholar 

  33. 33.

    Berta W, Cranley L, Dearing JW, Dogherty EJ, Squires JE, Estabrooks CA. Why (we think) facilitation works: insights from organizational learning theory. Implement Sci. 2015;10(1):141. https://doi.org/10.1186/s13012-015-0323-0.

    Article  PubMed  PubMed Central  Google Scholar 

  34. 34.

    Glasgow RE, Harden SM, Gaglio B, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year. Review mini review. Front Public Health. 2019. https://doi.org/10.3389/fpubh.2019.00064.

    Article  PubMed  PubMed Central  Google Scholar 

  35. 35.

    Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21. https://doi.org/10.1186/s13012-015-0209-1.

    Article  PubMed  PubMed Central  Google Scholar 

  36. 36.

    Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. https://doi.org/10.1186/1748-5908-4-50.

    Article  PubMed  PubMed Central  Google Scholar 

  37. 37.

    Escoffery C, Lebow-Skelley E, Haardoerfer R, et al. A systematic review of adaptations of evidence-based public health interventions globally. Implement Sci. 2018;13(1):125. https://doi.org/10.1186/s13012-018-0815-9.

    Article  PubMed  PubMed Central  Google Scholar 

  38. 38.

    Carvalho ML, Honeycutt S, Escoffery C, Glanz K, Sabbs D, Kegler MC. Balancing fidelity and adaptation: implementing evidence-based chronic disease prevention programs. J Public Health Manag Pract. 2013;19(4):348–56. https://doi.org/10.1097/PHH.0b013e31826d80eb.

    Article  PubMed  Google Scholar 

  39. 39.

    Centers for Disease Control and Prevention. How to protect yourself & others. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html. Accessed June 2021.

  40. 40.

    Sentell T, Vamos S, Okan O. Interdisciplinary perspectives on health literacy research around the world: more important than ever in a time of COVID-19. Int J Environ Res Public Health. 2020;17(9):3010.

    CAS  Article  Google Scholar 

  41. 41.

    Paakkari L, Okan O. COVID-19: health literacy is an underestimated problem. Lancet Public Health. 2020;5(5):e249–50. https://doi.org/10.1016/S2468-2667(20)30086-4.

    Article  PubMed  PubMed Central  Google Scholar 

  42. 42.

    Seng JJB, Yeam CT, Huang CW, Tan NC, Low LL. Pandemic related health literacy—a systematic review of literature in COVID-19, SARS and MERS pandemics. medRxiv. 2020. https://doi.org/10.1101/2020.05.07.20094227.

    Article  Google Scholar 

  43. 43.

    Beaunoyer E, Dupéré S, Guitton MJ. COVID-19 and digital inequalities: reciprocal impacts and mitigation strategies. Comput Hum Behav. 2020;111: 106424. https://doi.org/10.1016/j.chb.2020.106424.

    Article  Google Scholar 

  44. 44.

    Lancaster K, Rhodes T, Rosengarten M. Making evidence and policy in public health emergencies: lessons from COVID-19 for adaptive evidence-making and intervention. Evid Policy. 2020;16(3):477–90.

    Article  Google Scholar 

  45. 45.

    Alonso-Coello P, Schünemann HJ, Moberg J, et al. GRADE evidence to decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353: i2016. https://doi.org/10.1136/bmj.i2016.

    Article  PubMed  Google Scholar 

  46. 46.

    Schünemann HJ, Santesso N, Vist GE, et al. Using GRADE in situations of emergencies and urgencies: certainty in evidence and recommendations matters during the COVID-19 pandemic, now more than ever and no matter what. J Clin Epidemiol. 2020;127:202–7. https://doi.org/10.1016/j.jclinepi.2020.05.030.

    Article  PubMed  PubMed Central  Google Scholar 

  47. 47.

    Clark J, Glasziou P, Del Mar C, Bannach-Brown A, Stehlik P, Scott AM. A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol. 2020;121:81–90.

    Article  Google Scholar 

  48. 48.

    Niang M, Dupéré S, Alami H, Gagnon M-P. Why is repositioning public health innovation towards a social paradigm necessary? A reflection on the field of public health through the examples of Ebola and COVID-19. Glob Health. 2021;17(1):46. https://doi.org/10.1186/s12992-021-00695-3.

    Article  Google Scholar 

  49. 49.

    Armstrong R, Waters E, Moore L, et al. Understanding evidence: a statewide survey to explore evidence-informed public health decision-making in a local government setting. Implement Sci. 2014;9(1):188. https://doi.org/10.1186/s13012-014-0188-7.

    Article  PubMed  PubMed Central  Google Scholar 

  50. 50.

    Jilcott S, Ammerman A, Sommers J, Glasgow RE. Applying the RE-AIM framework to assess the public health impact of policy change. Ann Behav Med. 2007;34(2):105–14.

    Article  Google Scholar 

  51. 51.

    Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8(1):117. https://doi.org/10.1186/1748-5908-8-117.

    Article  PubMed  PubMed Central  Google Scholar 

  52. 52.

    Holtrop JS, Estabrooks PA, Gaglio B, et al. Understanding and applying the RE-AIM framework: clarifications and resources. J Clin Transl Sci. 2021;5:1–31. https://doi.org/10.1017/cts.2021.789.

    Article  Google Scholar 

  53. 53.

    Brehaut JC, Colquhoun HL, Eva KW, et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med. 2016;164(6):435–41.

    Article  Google Scholar 

  54. 54.

    Pedersen MJ, Favero N. Social distancing during the COVID-19 pandemic: who are the present and future noncompliers? Public Adm Rev. 2020;80(5):805–14. https://doi.org/10.1111/puar.13240.

    Article  Google Scholar 

  55. 55.

    Anderson RM, Heesterbeek H, Klinkenberg D, Hollingsworth TD. How will country-based mitigation measures influence the course of the COVID-19 epidemic? Lancet. 2020;395(10228):931–4.

    CAS  Article  Google Scholar 

  56. 56.

    Douglas M, Katikireddi SV, Taulbut M, McKee M, McCartney G. Mitigating the wider health effects of COVID-19 pandemic response. BMJ. 2020;369: m1557. https://doi.org/10.1136/bmj.m1557.

    Article  PubMed  PubMed Central  Google Scholar 

  57. 57.

    Neta G, Glasgow RE, Carpenter CR, et al. A framework for enhancing the value of research for dissemination and implementation. Am J Public Health. 2015;105(1):49–57.

    Article  Google Scholar 

  58. 58.

    European Joint Programme for the Integration of Radiation Protection Research. D9.90—report on venues, challenges, opportunities and recommendations for stakeholder engagement in emergency and recovery preparedness and response. Accessed June 2021, https://www.engage-h2020.eu/-/media/Files/ENGAGE/publications/deliverables/_Lists_Deliverables_Attachments_156_D9_90.pdf

  59. 59.

    Brown D, Donini A, Knox Clarke P. Engagement of crisis-affected people in humanitarian action. In Background paper of ALNAP's 29th annual meeting, 11–12 March 2014, Addis Ababa. London: ALNAP/ODI; 2014.

  60. 60.

    Birken SA, Bunger AC, Powell BJ, et al. Organizational theory for dissemination and implementation research. Implement Sci. 2017;12(1):62. https://doi.org/10.1186/s13012-017-0592-x.

    Article  PubMed  PubMed Central  Google Scholar 

  61. 61.

    Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45. https://doi.org/10.1186/1748-5908-9-45.

    Article  PubMed  PubMed Central  Google Scholar 

  62. 62.

    Michaelis B, Stegmaier R, Sonntag K. Shedding light on followers’ innovation implementation behavior: the role of transformational leadership, commitment to change, and climate for initiative. J Manag Psychol. 2010;25:408–29.

    Article  Google Scholar 

  63. 63.

    Deitchman S. Enhancing crisis leadership in public health emergencies. Disaster Med Public Health Prep. 2013;7(5):534–40.

    Article  Google Scholar 

  64. 64.

    Crystal M, Dorothy W, Whitney B. Reimagining community relationships for organizational learning: a scoping review with implications for a learning health system. PREPRINT (Version 1) available at Research Square. 2020. https://doi.org/10.21203/rs.3.rs-135727/v1.

  65. 65.

    Taylor SP, Kowalkowski MA, Beidas RS. Where is the implementation science? An opportunity to apply principles during the COVID-19 pandemic. Clin Infect Dis. 2020;71(11):2993–5.

    CAS  Article  Google Scholar 

  66. 66.

    Greene SM, Reid RJ, Larson EB. Implementing the learning health system: from concept to action. Ann Intern Med. 2012;157(3):207–10.

    Article  Google Scholar 

  67. 67.

    Chambers DA, Feero WG, Khoury MJ. Convergence of implementation science, precision medicine, and the learning health care system: a new model for biomedical research. JAMA. 2016;315(18):1941–2.

    CAS  Article  Google Scholar 

  68. 68.

    Romanelli RJ, Azar KMJ, Sudat S, Hung D, Frosch DL, Pressman AR. Learning health system in crisis: lessons from the COVID-19 pandemic. Mayo Clin Proc Innov Qual Outcomes. 2021;5(1):171–6. https://doi.org/10.1016/j.mayocpiqo.2020.10.004.

    Article  PubMed  Google Scholar 

  69. 69.

    McMahon M, Nadigel J, Thompson E, Glazier RH. Informing Canada’s health system response to COVID-19: priorities for health services and policy research. Healthc Policy. 2020;16(1):112–24. https://doi.org/10.12927/hcpol.2020.26249.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors thank Professor Ann Dozier, University of Rochester, who commented on an earlier version of the manuscript.

Funding

RYN, KF, JMM are supported by the University of Rochester CTSA award numbers UL1 TR002001 and U54 PAR-18-940 from the National Center for Advancing Translational Sciences of the National Institutes of Health. RCS was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant Number UL1TR001873, and American Cancer Society, Research Scholar Grant (RSG-17-156-01-CPPB (Shelton)). BMK was supported by NIH/NCATS Colorado CTSA Grant Number UL1 TR002535.

Author information

Affiliations

Authors

Contributions

RYN developed the initial draft. RCS, KF, BMK, and JMM significantly contributed in conception and development of the manuscript and added sections and revisions. All authors reviewed, read, and approved the final version.

Corresponding author

Correspondence to Reza Yousefi Nooraie.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: Following the publication of the original article, we were notified that an out-of-place circle had been mistakenly included in Figure 1, without any impact on the scientific content of the figure.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yousefi Nooraie, R., Shelton, R.C., Fiscella, K. et al. The pragmatic, rapid, and iterative dissemination and implementation (PRIDI) cycle: adapting to the dynamic nature of public health emergencies (and beyond). Health Res Policy Sys 19, 110 (2021). https://doi.org/10.1186/s12961-021-00764-4

Download citation

Keywords

  • Rapid cycle
  • Public health emergencies
  • D&I models
  • COVID-19