Skip to main content

Policy implementation and outcome evaluation: establishing a framework and expanding capacity for advocacy organizations to assess the impact of their work in public policy


Advocacy organizations can play a crucial role in evaluating whether legislation or regulation has had its intended effect by supporting robust public policy implementation and outcome evaluation. The American Heart Association, working with expert advisors, has developed a framework for effective evaluation that can be used by advocacy organizations, in partnership with researchers, public health agencies, funders, and policy makers to assess the health and equity impact of legislation and regulation over time. Advocacy organizations can use parts of this framework to evaluate the impact of policies relevant to their own advocacy and public policy efforts and inform policy development and guide their organizational resource allocation. Ultimately, working in partnership, advocacy organizations can help bring capacity, commitment and funding to this important implementation and outcome evaluation work that informs impactful public policy for equitable population health and well-being.

Peer Review reports


Policy implementation and outcome evaluation, which assesses whether a particular public policy has had its intended impact when implemented, is an integral component of the policy process. It provides an overall assessment of the policy effects and can guide responsible decision making for ongoing policy development. Effective evaluation of how policies are implemented, scaled and funded can also inform decision making and prioritization by policy makers, funders, advocacy organizations and public health agencies [1]. Yet, it is significantly underutilized [2,3,4].

Many advocacy organizations, including the American Heart Association, are interested in an evaluation framework that helps assess the impact of the public policies they work to pass and implement. A framework that examines whether their resources and investments are optimally focussed on the most effective policies that can impact population health and whether there is equity impact would help guide organizational decision making. Evaluating the impact and outcome of public policy implementation allows advocacy organizations to engage with the research community to bring additional capacity for the kind of evaluation that can lead to evidence-based, equity-focussed policy making.

Advocacy organizations support social, public health or other causes that require changes in government, public policy or systems, often related to their mission. Advocacy organizations vary significantly in size, causes, structure/function and funding. The American Heart Association has a mission of being a relentless force for a world of longer, healthier lives and has done advocacy work for more than 40 years across issues related to prevention (tobacco, nutrition security, physical activity), acute systems of care, access to care, heart and brain research, digital health, public health infrastructure and appropriations. The organization has previously described how it is structured and functions in doing its advocacy work [5, 6].

Evidence for impact—public policy implementation and outcome evaluation

Fundamentally, for advocacy organizations to optimize their work in public policy, they need to understand whether the policies they work so hard to get into place are implemented as intended. This includes whether the policies are associated with specific population impacts, whether they increase equity or disparities, what they cost to implementers and priority populations, the degree and scale of their penetration and uptake, whether they are associated with unintended consequences, and whether they contribute to creating longer, healthier lives. Effective evaluation assesses policy adoption, acceptability, penetration, feasibility, fidelity, implementation cost, cost-effectiveness, unintended consequences and sustainability [7, 8].

Table 1 provides definitions for each of these aspects of evaluation with equity considerations. A need exists to include quantitative metrics from relevant data monitoring systems to assess objective change or progress in health and equity over time [9]. Quantitative analysis has historically been under-utilized in policy implementation and outcome evaluation and is necessary to objectively assess population impact. [9] Relevant surveillance systems must be matched with the objective measures that will be tracked over time and the lag between data collection and public reporting of the surveillance data must be taken into account. Table 2 provides an example of a cascade approach to assessing health and equity impact longitudinally with quantitative measures. Equity domains may include income, educational attainment, race/ethnicity, geography, rurality, sexual orientation, gender identity, physical and mental disability, and other considerations specific to the issue area. While these metrics will primarily demonstrate association rather than causality with the policy change, the analysis will be important for assessing any impact on population health over time.

Table 1 Areas of policy implementation and outcome evaluation assessment [11, 19,20,21]
Table 2 Examples of longitudinal approach to quantitative health and equity assessment in policy implementation and outcome evaluation

The framework

Although there are a range of existing frameworks, [10,11,12], most focussed on program implementation. There is a need for a more robust public policy implementation and outcome evaluation framework that can be used by non-profit advocacy organizations, researchers, government agencies and other key collaborators. In 2019, the American Heart Association’s policy research team convened an expert advisory group to help develop such a framework. The policy research team translates science and evidence base into impactful public policy in the areas of cardiovascular disease and stroke prevention and health promotion. The association’s policy research team provides policy development and the foundation for AHA’s advocacy work at the global, national, state and local levels. The expert advisory group it convened was comprised of individuals representing the funding community that has invested in community-led programmes and leading researchers and clinicians. Each of these individuals are recognized in the acknowledgements. The group met for about a year with the objective of informing a draft framework that was completed at the end of the convenings. The policy research team gathered further internal and external feedback, including from key staff at the Centers for Disease Control and Prevention, and finalized the framework. The Fig. 1 provides a visual representation of the conceptual framework that can be applied across public policy issues. This evaluation framework calls for longitudinal assessment and blends qualitative and quantitative analysis.

Fig. 1
figure 1

Research Framework

As we move forward into implementing parts of the framework, we will convene additional expert advisory groups to inform our evaluation and we will purposefully include people with lived experience.

Equity considerations

Implementation and outcome evaluation is essential to optimize public policy efforts that aim to address inequities. Long-standing health disparities stemming from the historical, inequitable distribution of wealth, power and privilege and exist across almost every health indicator and outcome [13]. There is a need for equitable policy, systems and environment changes that are rooted in an understanding of the historical arc of structural racism and the influence of structural inequities on the proliferation of health-compromising conditions and the American Heart Association has issued a call to action identifying structural racism as a fundamental driver of health disparities [14]. Advocacy organizations can help catalyse public policy making at all levels of government to cultivate environments that support population health and well-being [13]. Policies are frequently passed with exaggerated claims of reducing inequities. Without implementation and outcome evaluation, there is little or no accountability measure to determine if those claims are met. Although there may be more attention and discussion of heath equity recently, research shows that this has not necessarily been translated into increased equity as policies are implemented. The overall health equity in mortality, for example, has slowed in recent decades, not accelerated [15].

The importance of evaluation and accountability is highlighted in What is Health Equity brief developed by the Robert Wood Johnson Foundation [16]. It outlined three essential elements to achieve health equity as follows:

  • The approach addresses the underlying, cross-sector social inequities that are needed to be healthy.

  • The approach is focussed to produce the greatest benefit for groups that live in or are members of disadvantaged or under-invested communities.

  • The impact is evaluated, not by measuring the average impact on the whole population, but by examining the impact in under-invested communities, as well as comparing the absolute and relative differences with advantaged communities.

Engagement of under-invested communities is needed from the beginning of policy advocacy work. At each stage of the policy implementation and outcome evaluation process there are important questions to examine the potential impact on locations and community groups most burdened by inequities.

The review by Brownson et al. provides the following 10 recommendations as a pathway for advancing health equity through implementation science: ‘(1) link social determinants with health outcomes, (2) build equity into all policies, (3) use equity-relevant metrics, (4) study what is already happening, (5) integrate equity into implementation models, (6) design and tailor implementation strategies, (7) connect to systems and sectors outside of health, (8) engage organizations in internal and external equity efforts, (9) build capacity for equity in implementation science, and (10) focus on equity in dissemination efforts’[17].

Partnership engagement and collaboration

Advocacy organizations cannot do policy implementation and outcome evaluation unilaterally. They must do it in partnership with academia, public health departments, funders, coalition partners, those with lived experience and other key collaborators. Only through partnering and collaboration can organizations bring the necessary capacity to this work. There are several key steps that advocacy organizations can take to facilitate greater commitment and collaboration around public policy evaluation. Depending on their capacity, mission-focused priorities, partner engagement, and community context, advocacy organizations may focus on some or all of these key steps:

  • Make it a standard of practice to advocate for and ensure appropriations for the monitoring and evaluation of a law or regulation.

  • Convene the research community and key collaborators, including public health departments, to develop an evergreen research agenda. Bring feedback from field and federal advocacy to the research community and funders to help support policy-relevant research.

  • Develop relationships with career professionals in the regulatory agencies across all levels of government to help support and monitor policy implementation.

  • Commit to some level of evaluation at the organizational level to assess the impact of public policies the organization has worked to pass.

  • Conduct and/or secure funding for message testing research funded to better make the case to policy makers for the importance of policy implementation and outcome evaluation and why it needs to be appropriated.

  • Support public/private resources and partnerships to support technical assistance in implementation and engage lived experience into policy outcome evaluation.

  • Enable, through partnerships with citizen groups/organizations, the ability to study public policy implementation and outcomes in real time and monitor ongoing refinement [18].


If advocacy organizations can commit to facilitating, initiating and advocating for policy implementation and outcome evaluation in public policy, there is the opportunity to develop more impactful policy, understand the health and equity impact of legislation and regulation across the population, and guide organizational decision making and resource commitment. Engagement with funders, key collaborators, the research community, policy makers, government agencies, those with lived experience and public health is essential to bring the necessary capacity, resources and commitment to this work. Partnering with public health agencies is especially important to optimize the impact of funding to states and communities, improve population health monitoring and inform public health frameworks. The American Heart Association will use the framework to guide efforts to better capture the reach and impact of the public policies that we have worked to pass. Capturing the reach and impact of enacted policies will help us understand how our advocacy efforts have contributed to our organizational impact goal and strategic priorities. We will also disseminate this framework to our key partners and collaborators and to the research community. Together, with significant collaboration and coordination to achieve robust public policy implementation and outcome evaluation, the American Heart Association and other advocacy organizations can play an important role in informing the most effective public policy strategies to support population health and well-being.

Availability of data and materials

Not applicable.


  1. Grembowski D. The practice of health program evaluation. California: Sage Publications; 2015.

    Google Scholar 

  2. Asada Y, Smith NR, Kroll-Desrosiers A, Chriqui J. Policy implementation science: NIH-funded case examples and lessons learned. Front Health Serv. 2022.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Witter S, Kardan A, Scott M, Moore L, Shaxson L. Generating demand for and use of evaluation evidence in government health ministries: lessons from a pilot programme in Uganda and Zambia. Health Res Policy Syst. 2017;15(1):86.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Improving governance with policy evaluation: Lessons from country experiences. 2023. Accessed 12 Dec 2023.

  5. Churchwell K, Antman E, Birnbaum J, et al. Nonprofit advocacy at the American heart association: creating a visionary way forward at our 40th anniversary: an American heart association policy statement. Circulation. 2021;143(19):e947–58.

    Article  PubMed  Google Scholar 

  6. Labarthe DR, Goldstein LB, Antman EM, et al. Evidence-based policy making: assessment of the american heart association’s strategic policy portfolio: a policy statement from the American heart association. Circulation. 2016;133(18):e615–53.

    Article  CAS  PubMed  Google Scholar 

  7. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.

    Article  PubMed  Google Scholar 

  8. Jilcott S, Ammerman A, Sommers J, Glasgow RE. Applying the RE-AIM framework to assess the public health impact of policy change. Ann Behav Med. 2007;34(2):105–14.

    Article  PubMed  Google Scholar 

  9. Allen P, Pilar M, Walsh-Bailey C, et al. Quantitative measures of health policy implementation determinants and outcomes: a systematic review. Implement Sci. 2020;15(1):47.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Harris-Roxas BF, Harris PJ, Harris E, Kemp LA. A rapid equity focused health impact assessment of a policy implementation plan: an Australian case study and impact evaluation. Int J Equity Health. 2011;10(1):1–12.

    Article  Google Scholar 

  11. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Policy Mental Health Mental Health Serv Res. 2011;38:65–76.

    Article  Google Scholar 

  12. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8(1):1–11.

    Article  Google Scholar 

  13. Whitsel LP, Johnson JC. Addressing social and racial justice in public policy for healthy living. Prog Cardiovasc Dis Mar-Apr. 2022;71:37–42.

    Article  Google Scholar 

  14. Churchwell K, Elkind MSV, Benjamin RM, et al. Call to action: structural racism as a fundamental driver of health disparities: a presidential advisory from the American heart association. Circulation. 2020;142(24):e454–68.

    Article  PubMed  Google Scholar 

  15. Anderson NW, Zimmerman FJ. Trends in health equity in mortality in the United States, 1969–2019. SSM Population Health. 2021;16: 100966.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Braveman P, Arkin E, Orleans T, Proctor D, Plough A. What is health equity? And what difference does a definition make? Behav Sci Policy. 2017.

    Article  Google Scholar 

  17. Brownson RC, Kumanyika SK, Kreuter MW, Haire-Joshu D. Implementation science should give higher priority to health equity. Implement Sci. 2021;16:1–16.

    Article  Google Scholar 

  18. Emmons KM, Chambers DA. Policy implementation science–an unexplored strategy to address social determinants of health. Ethn Dis. 2021;31(1):133.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Jilcott S, Ammerman A, Sommers J, Glasgow RE. Applying the RE-AIM framework to assess the public health impact of policy change. Ann Behav Med. 2007;34(2):105–14.

    Article  PubMed  Google Scholar 

  20. Mathison S. Encyclopedia of evaluation. California: Sage publications; 2004.

    Google Scholar 

  21. Oliver K, Lorenc T, Tinkler J. Evaluating unintended consequences: new insights into solving practical, ethical and political challenges of evaluation. Evaluation. 2020;26(1):61–75.

    Article  Google Scholar 

Download references


We would like to acknowledge the thoughtful, impactful contributions of time and expertise from an expert advisory group we convened to start this work: Jamie Chriqui, University of Illinois—Chicago; Carlos Rodriguez, Wake Forest University; Darwin Labarthe, Northwestern University; Donald Lloyd-Jones, Northwestern University; Mary Story, Duke University/Healthy Eating Research; Keith Churchwell, AHA Past President; Amy Yarock, Gretchen Swanson Center for Nutrition; Laura Leviton, Special Advisor; Whitney Garney, Texas A & M University; and Angie Cradock, Harvard University. We also want to acknowledge the many experts at the Centers for Disease Control and Prevention who provided their review and input, especially Michael Schooley, Christopher Jones, Heidi Blanck, Erika Fulmer, and Carrie Dooyema. A special thanks to Yu Wang for her thoughtful feedback.



Author information

Authors and Affiliations



All authors have contributed significantly to the writing and conceptual development of this manuscript. All authors have approved the manuscript for submission and the content of the manuscript has not been published or submitted for publication elsewhere. The authors also provide consent for publication.

Corresponding author

Correspondence to Laurie P. Whitsel.

Ethics declarations

Competing interests

All of the authors are employed by the American Heart Association but otherwise have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Whitsel, L.P., Honeycutt, S., Radcliffe, R. et al. Policy implementation and outcome evaluation: establishing a framework and expanding capacity for advocacy organizations to assess the impact of their work in public policy. Health Res Policy Sys 22, 27 (2024).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: