This article has Open Peer Review reports available.
Evidence for Health III: Making evidence-informed decisions that integrate values and context
© Andermann et al. 2016
Received: 25 February 2015
Accepted: 16 February 2016
Published: 14 March 2016
Making evidence-informed decisions with the aim of improving the health of individuals or populations can be facilitated by using a systematic approach. While a number of algorithms already exist, and while there is no single ‘right’ way of summarizing or ordering the various elements that should be involved in making such health-related decisions, an algorithm is presented here that lays out many of the key issues that should be considered, and which adds a special emphasis on balancing the values of individual patients and entire populations, as well as the importance of incorporating contextual considerations. Indeed many different types of evidence and value judgements are needed during the decision-making process to answer a wide range of questions, including (1) What is the priority health problem? (2) What causes this health problem? (3) What are the different strategies or interventions that can be used to address this health problem? (4) Which of these options, as compared to the status quo, has an added benefit that outweighs the harms? (5) Which options would be acceptable to the individuals or populations involved? (6) What are the costs and opportunity costs? (7) Would these options be feasible and sustainable in this specific context? (8) What are the ethical, legal and social implications of choosing one option over another? (9) What do different stakeholders stand to gain or lose from each option? and (10) Taking into account the multiple perspectives and considerations involved, which option is most likely to improve health while minimizing harms? This third and final article in the ‘Evidence for Health’ series will go through each of the steps in the algorithm in greater detail to promote more evidence-informed decisions that aim to improve health and reduce inequities.
KeywordsDecision-making Evidence-based medicine Health equity Health outcomes Health policy Public health Research
Define the priority health problem
At the outset, it is helpful to try and identify the priority health problem (or problems) that, if improved, will have the greatest impact on health overall . All too often, a new drug or other intervention is discovered and people lobby for its implementation and scale-up when the need for this intervention has not even been demonstrated and it is unclear whether it will truly have a widespread impact on the health of a large segment of the population. Therefore, it is important, in the first instance, that we understand what the true health needs are in order to be able to then address them in an evidence-informed way. An objective needs assessment based on surveillance data of the common causes of morbidity and premature mortality, combined with a subjective needs assessment based on what the community may consider the most important priorities, is a good way to start. For example, the Tanzania Essential Health Intervention Project involved a community-based research approach that identified the true causes of death in two districts in Tanzania (where most deaths occurred at home, thus rendering hospital data unreliable). The evidence was then used to adjust the allocation of the district health budget to be more in line with the real disease burden. Within 5 years, a 52% reduction in child mortality was noted and the two districts reached the related Millennium Development Goal targets well before 2015 .
Understand the underlying causes
“What’s the cause of diabetes? Many would say diet and exercise. The cause of that is poverty and lack of choice regarding diet and exercise. And the cause of that is colonization and lack of economic opportunity” .
While increasing access to and quality of clinical services is clearly important, it cannot be the only approach to improving health. Rather, a broad range of strategies are needed, including engaging community leaders, end-users/consumers and other key stakeholders to create healthier physical and social environments, as well as incorporating culturally-adapted and contextualized solutions to local health problems: “Diabetes is a complicated disease that is nested in the experience of rapid social and cultural change; thus, its prevention and control may need new ideas that go beyond an individual approach in a clinic or hospital ward” . Indeed, changing these distal determinants will require strong advocacy for larger social change and legal reforms, which will only occur if there is buy-in and support from the highest political levels.
List the options to improve health
Once we have defined the health priorities and have understood the underlying causes of these problems at various levels, the next question to ask is “what can we do about it?” There is always more than one option available, including the status quo, but how to choose? The first step is simply identifying what the options are before attempting to weigh the pros and cons of each.
Do the added benefits outweigh the harms?
To answer this question, it can be helpful to consider a number of sub-questions: What is the option being considered and what is it being compared to (i.e. the current gold standard or, if there is no gold standard, then the status quo)? Is there an added benefit at all? And, if so, what are the added benefits? Is there data on more distal benefits such as improvements in health outcomes or only intermediate and proximal measures, such as changes in patient or health provider behaviours, leaving one to extrapolate as to the actual impacts of such an intervention? Are there any harms? And, if so, what are the harms? Can one say with confidence that there are no or few harms, or was this something that was not measured, and therefore an area of uncertainty? How likely are the benefits and harms and for whom?
To answer these questions, syntheses of the existing evidence are often used, including systematic reviews and meta-analyses, which provide an overview of the existing evidence base. These studies are generally known as ‘secondary research’ since they do not involve primary data collection. Instead, the current body of evidence is used as the starting point, and the study involves asking a clearly formulated research question, systematically collecting research evidence around this question, critically appraising the research studies to determine which should be included, and finally integrating the results from a wide range of studies, to thereby present a clearer picture than any individual study alone can provide . This often serves as an important anchor to decision-making, particularly in areas which are highly value-driven and open to debate. However, it can be difficult to assess the risk-benefit balance when the harms are not being measured. For instance, in the case of breast cancer screening, the literature on mortality benefit came first. As time went on and national breast screening programs were being implemented, only then did a new literature start to emerge on the potential harms of screening, ranging from psychological side effects and cancer worries  to the effects of radiation related to mammography itself . As a result, this new evidence provided an opportunity for rethinking the balance of benefits and harms .
Indeed, determining whether there is an added benefit from the intervention which outweighs the harms often involves a value judgement and may vary according to one’s risk tolerance – how much a person wants to avoid getting the disease being targeted (e.g. breast cancer) and how much they want to avoid the potential negative consequences inherent to the intervention itself (e.g. false positive test result, unnecessary-treatment). Where there is widespread agreement in a given context that the added benefits outweigh the harms, we can continue on with our algorithm; however, if not, then it is back to the drawing board to look at other options from the list or to maintain the status quo while working on developing better options for the future.
Are the options acceptable to those involved?
Before implementing any treatment, program or policy, it is important to think about those who will be ‘on the receiving end’. Since we cannot know people’s preferences, opinions or concerns unless we ask, it is therefore important that they be involved. Understanding individual and population preferences, as well as the local context of implementation, is critical to choosing the best option under the circumstances. No point thinking about costs if the option is not acceptable in the first place.
For example, some genetic screening programs entail providing information to be able to make more informed reproductive choices, and during pregnancy, that could involve the delicate issue of abortion. In some communities, due to religious beliefs or cultural values, having a screening program which includes the possibility of aborting an unborn child would be quite unacceptable, even if certain individuals within that community may want to have that avenue open to them if it turns out that their child is affected. Thus, the issue of acceptability can become quite tricky, especially when it involves balancing the preferences of individuals with those of the target population and society overall. Another example of where issues of acceptability are particularly sensitive is when the intervention is delivered to the entire population and there is no real possibility to ‘opt out’. For instance, fluoridation of water is widely recognized as a powerful means of improving oral health . Yet, in spite of strong evidence in support of such programs , outcries from vocal opponents have resulted in millions of people not being protected against dental decay, and even the halting of certain community-based water fluoridation programs . Thus, if none of the options are acceptable, then either further information can be provided, which may succeed in changing attitudes and generating support for these options, or else it may be a sign to go back to the drawing board and try to think up more options.
Calculate costs and opportunity costs
Generally, we find ourselves in a situation where resources are scarce and, importantly, there is competition with priorities beyond the health sector. So, how do we determine which option to choose and which options need to be left behind (i.e. the opportunity cost)? The Disease Control Priorities Project  and the Choosing Wisely campaign  use effectiveness and cost-effectiveness data to determine which interventions to scale-up to improve global health and which interventions to scale back. The cost effectiveness ratios are generally much lower for preventive interventions as opposed to curative interventions. For instance, childhood immunization, insecticide-treated bed nets for malaria prevention and road traffic safety interventions (such as speeding laws and speed bumps at busy intersections) cost only a few dollars per DALY averted, whereas treating a stroke or heart attack with expensive ‘clot-busting’ medications can cost thousands of dollars per DALY averted.
Thus, action on the social determinants of health to prevent problems at the source (i.e. by improving housing, education, job quality, and so forth) are likely to be much less expensive as compared to treating people once they are ill, with all the healthcare costs, missed work and human suffering that this entails. That being said, it is often difficult to make such calculations for interventions that address the social determinants of health as these tend to involve more complex ‘clusters’ of interventions with less clarity in terms of the specific costs and outcomes . Indeed, the evidence base relating to the costs of working more upstream are often lacking, even if evidence of effectiveness is growing. That does not mean that we should not be working more upstream, simply that it is not always easy to estimate the costs involved. Moreover, when engaging in intersectoral action and whole-of-government approaches, it also remains to be determined which budgetary envelope these costs should be withdrawn from (i.e. the health budget or the education budget, the national budget or the local budget, and so forth). Often, within government, ministries of health tend to have less bargaining power, especially in low- and middle-income countries, which poses a challenge when it comes to making health improvement a priority. Further, there can be “inter-governmental downloading of responsibilities” between national, district and municipal levels . Nonetheless, while cost issues are clearly important, they are not the only factor in determining the feasibility of one option over another.
Which options are feasible in this context?
Even if an option is affordable and can be funded using existing budgets, this does not necessarily mean that it is feasible since there may be many different kinds of logistical barriers involved, which can often be difficult to assess in advance. Indeed, just because an intervention works well in one context or setting does not mean that it will be applicable and transferable to a different one. There are many factors that can influence whether it is likely to expect the same benefits in this new implementation context ; for instance, if the health issue is just as prevalent and important as in the original study population, if there is strong local capacity for implementing the intervention, if the intervention can be suitably adapted to the new social and cultural context, and so forth.
Across many parts of Africa and South East Asia, while it may be possible to afford the cost of vaccines against childhood diseases – especially as much of the cost is subsidized by the GAVI alliance through the Expanded Programme on Immunization – there are nonetheless various logistical challenges that may remain. Vaccination attempts may be thwarted by social or political unrest, by difficult access to remote areas, and by lapses in the cold chain . Even if a program is successfully launched, ongoing sustainability remains a concern once initial funding is depleted, governments change and attention turns to other priorities. Piloting would therefore be needed, sometimes also termed a ‘feasibility study’, to first provide evidence of local effectiveness and potential for long-term health gains.
Ethical, legal and social considerations
In some cases, such as end-of-life issues or genetic screening, the ethical, legal and social issues are quite apparent . However, it may be even more important to proceed with caution in situations where such issues are less obvious and could potentially be overlooked. For instance, an economic development project that proposes to create more jobs in a rural area may seem harmless enough. Yet, what if only a small minority benefit from such an initiative, or even worse, what if those who benefit least are burdened with most of the harms? Kass, who developed an ethics framework for public health, emphasized the importance of ensuring that interventions not only lead to greater benefit than harm, but also ensuring that the distribution of benefits and harms is equitable .
Thus, while there may be high quality evidence of efficacy and low cost, this in itself is not sufficient to move ahead with one option over another. It is important to carefully consider the ethical, legal and social implications and to systematically examine who is affected and in what way, to ensure that everyone is able to share in the benefits while especially protecting those who are most vulnerable from the harms. This brings us to our penultimate step on understanding what the various stakeholders stand to gain or lose.
What do stakeholders stand to gain or lose?
When considering the potential impacts of any given option, it is important to look at the different groups who could be affected – including various lobby groups, target populations, innocent bystanders and the decision-makers themselves – to better appreciate what each group would gain or lose from implementing this option and what would be the distribution of the benefits and harms.
Many policies outside the health sector (e.g. tax laws, parental leave policies, employee disability insurance schemes, etc.) can have tremendous potential to greatly benefit and/or harm the health of various groups in society. This is the impetus behind the “Health in All Policies” approach championed by Ilona Kickbush et al. , and also the growing interest in conducting Health Impact Assessments to help create more evidence-informed policy .
Returning once again to the example of an economic development project being proposed in a remote community, it would certainly be unfair if the company shareholders retained the bulk of the profits while the workers exposed to potentially hazardous conditions are paid low wages. Similarly, if community leaders do not redistribute the wealth and other benefits from deals signed with the company or at least use the funds to benefit the local population in some way, that would also be unfair. Indeed, lack of transparency and profit sharing, lack of equal opportunities for employment, lack of inclusion and empowerment of those affected, as well as failure to reinvest profits into health and social services and tackling the upstream determinants of health (e.g. through improving education, access to child care, building social cohesion, etc.) could all lead to worsening health inequities – in spite of an influx of money and jobs in the region. Knowing all this, prior to the last step in the algorithm, there is an opportunity for refining or even redesigning the proposed option such that the benefits are maximized, the harms are minimized, and the distribution of benefits and harms is more equitable. Thus, once these additional safeguards and adjustments are incorporated, only then is it possible to determine which options have the greatest potential for improving health and reducing inequities.
Which options improve health most while minimizing harms?
At the end of the day, a decision must be made that integrates all of the evidence from the various steps in the algorithm and makes a final judgement regarding whether or not to proceed. The ultimate question is “which option(s) maximize potential health and social benefits, minimize potential harms, and ensure that the distribution of benefits and harms within and between populations is fair?” Many different types of evidence and expertise are called upon to make this overarching decision. Clearly, there will be differences in how one goes about this depending on whether these are clinical decisions, population level decisions or global policy decisions.
Integrating the evidence, values and contextual factors to come to a final decision is indeed a difficult challenge; even with all the information at hand, the challenge is how to take the plunge and transform all of these different considerations into a binary choice – “will I choose this option or that option?” There are examples, especially in clinical settings, of decision-aids with explicit value clarification exercises which have been shown to lead to more informed value-based choices  and to decrease decisional conflict . However, especially in policy settings, information cannot simply be plugged into a formula or equation to come up with the best answer. Indeed, there needs to be a structure that involves the various stakeholders in a deliberative process and, by following the algorithm outlined in this chapter, they will be better able to have a more logical and structured discussion that teases out the complexity involved and avoids recourse to polemics and entrenched views, which are often counter-productive. Often, the process itself is part of the product – by helping to ensure that different voices are heard, that people feel respected, and that a fair and transparent process is used to come to a decision , it will be possible to make more reflected judgements about improving health and reducing inequities based on the best available evidence and taking into account a multitude of complex considerations.
As Michael Marmot once said, “Scientific findings do not fall on blank minds that get made up as a result. Science engages with busy minds that have strong views about how things are and ought to be” . While no decision is ever free from value judgements, at least using a systematic approach ensures that the decision is nonetheless grounded in the best available evidence, in all its facets. Additionally, by identifying and making explicit the various trade-offs and opportunity costs, and by providing reasons for choosing one option rather than another, at least there will be the possibility of understanding why certain value judgements were made, as well as being able to challenge the underlying reasons in future should the context or evidence-base evolve.
It is important to strike a balance between making decisions that are not at all thought through versus overthinking every step. The systematic approach can therefore be helpful to avoid becoming too bogged-down in the details to see the big picture, or even worse, being paralyzed by indecision when faced with a large number of perspectives and trade-offs. In the end, decisions must be made, since failing to choose is simply choosing the status quo. Being transparent and explicit about the basis for making the decision, including the quality of evidence, is an important requirement. There are also the very helpful support tools developed by Lavis et al.  which aim to help in bridging the ‘know-do’ gap.
The goal is therefore to foster a dialogue among stakeholders that will promote decisions that are more nuanced, more transparent and, ultimately, more likely to have an impact on improving health. Nonetheless, decision-making remains an inherently iterative and often somewhat disorganized process, especially as we move towards population-based and global-level decisions. According to Anderson, a policy analyst, “Public policy is messy. If you hold on too tightly to your policy formulation you will wither in this environment. Policy is rarely final and usually changes with every new administration” .
Therefore, all the more reason to ensure that decisions are informed by evidence and that the key stakeholders are involved in a meaningful way.
AA is funded through a Clinician Scholar Award by the Quebec Health Research Fund (Fonds de Recherche du Québec – Sante) and the Quebec Federation of Medical Specialists (Fédération des Médecins Spécialistes du Québec).
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Andermann A. Evidence for Health: From Patient Choice to Global Policy. Cambridge: Cambridge University Press; 2013.Google Scholar
- Harper PR. A review and comparison of classification algorithms for medical decision making. Health Policy. 2005;71(3):315–31.View ArticlePubMedGoogle Scholar
- Brownson RC, Chriqui JF, Stamatakis KA. Understanding evidence-based public health policy. Am J Pub Health. 2009;99(9):1576–83.View ArticleGoogle Scholar
- Hutton J, McGrath C, Frybourg JM, Tremblay M, Bramley-Harker E, Henshall C. Framework for describing and classifying decision-making systems using technology assessment to determine the reimbursement of health technologies (fourth hurdle systems). Int J Technol Assess Health Care. 2006;22(1):10–8.View ArticlePubMedGoogle Scholar
- Green LW, Kreuter MW. Health Program Planning: An Educational and Ecological Approach. 4th ed. New York: McGraw-Hill; 2005.Google Scholar
- de Savigny D, Kasale H, Mbuya C, Reid G. Fixing Health Systems. 2nd ed. Ottawa: International Development Research Centre; 2008.Google Scholar
- Marmot M, on behalf of the Commission on Social Determinants of Health. Achieving health equity: from root causes to fair outcomes. Lancet. 2007;370(9593):1153–63.View ArticlePubMedGoogle Scholar
- Evans R, Barer M, Marmor T. Why Are Some People Healthy and Others Not? The Determinants of Health of Populations. New York: Aldine de Gruyter; 1994.Google Scholar
- Reading C, Wien F. Health inequalities and social determinants of Aboriginal people’s health. National Collaborating Centre for Aboriginal Health. 2009. http://www.nccah-ccnsa.ca/docs/social%20determinates/NCCAH-Loppie-Wien_Report.pdf. Accessed 11 February 2016.
- Reading J. Keynote address. International Union for Health Promotion and Education (IUHPE). Vancouver: World Conference; 2007.Google Scholar
- Reading J. The quest to improve aboriginal health. Can Med Assoc Journal. 2006;174(9):1233. http://www.cmaj.ca/cgi/reprint/174/9/1233. Accessed 11 February 2016.View ArticleGoogle Scholar
- Vineis P, Elliott P. Why is epidemiology necessary to policy-making? J Epidemiol Commun Health. 2009;63(3):186–7.View ArticleGoogle Scholar
- Mackenbach J. Politics is nothing but medicine at a larger scale: reflections on public health’s biggest idea. J Epidemiol Commun Health. 2009;63(3):181–4.View ArticleGoogle Scholar
- Glossary. The Cochrane Collaboration. 2012. http://www.cochrane.org/glossary/. Accessed 11 February 2016.
- Lerman C, Trock B, Rimer B, Jepson C, Brody D, Boyce A. Psychological side effects of breast cancer screening. Health Psychol. 1991;10(4):259–67.View ArticlePubMedGoogle Scholar
- Beemsterboer PM, Warmerdam PG, Boer R, de Koning HJ. Radiation risk of mammography related to benefit in screening programmes: a favourable balance? J Med Screen. 1998;5(2):81–7.View ArticlePubMedGoogle Scholar
- Roberts M. Breast screening: Time for a rethink? BMJ. 1989;299:1153–5.View ArticlePubMedPubMed CentralGoogle Scholar
- Fluoride Recommendations Work Group. Recommendations for using fluoride to prevent and control dental caries in the United States. MMWR. 2001;50(RR14):1–42.Google Scholar
- Federal-Provincial-Territorial Committee on Drinking Water. Fluoride in Drinking Water. Ottawa: Health Canada; 2009. http://www.hc-sc.gc.ca/ewh-semt/alt_formats/hecs-sesc/pdf/consult/_2009/fluoride-fluorure/consult_fluor_water-eau-eng.pdf. Accessed 11 February 2016.
- Attwood D, Blinkhorn A. Dental health in schoolchildren 5 years after water fluoridation ceased in south-west Scotland. Int Dent J. 1991;41(1):43–8.PubMedGoogle Scholar
- Jamison D, Breman J, Measham A, Alleyne G, Claeson M, Evans D, et al. Disease Control Priorities in Developing Countries. 2nd ed. Washington, DC, and Oxford: The World Bank and Oxford University Press; 2006. http://www.dcp2.org/pubs/DCP. Accessed 11 February 2016.Google Scholar
- Feldman LS. Choosing wisely. Things we do for no reason. J Hosp Med. 2015;10(10):696.View ArticlePubMedGoogle Scholar
- Petticrew M, Whitehead M, Macintyre SJ, Graham H, Egan M. Evidence for public health policy on inequalities: 1: the reality according to policymakers. J Epidemiol Commun Health. 2004;58(10):811–6.View ArticleGoogle Scholar
- Collins PA, Hayes MV. Examining the capacities of municipal governments to reduce health inequities: a survey of municipal actors’ perceptions in Metro Vancouver. Can J Public Health. 2013;104(4):e304–10.View ArticlePubMedGoogle Scholar
- Wang S, Moss JR, Hiller JE. Applicability and transferability of interventions in evidence-based public health. Health Promot Int. 2006;21(1):76–83.View ArticlePubMedGoogle Scholar
- Schoub BD, Cameron NA. Problems encountered in the delivery and storage of OPV in an African country. Dev Biol Stand. 1996;87:27–32.PubMedGoogle Scholar
- Screening G. Ethical Issues. London: Nuffield Council on Bioethics; 1993. http://www.nuffieldbioethics.org/sites/default/files/Genetic_screening_report.pdf. Accessed 11 February 2016.Google Scholar
- Kass N. An ethics framework for public health. Am J Public Health. 2001;91(11):1776–82.View ArticlePubMedPubMed CentralGoogle Scholar
- Kickbush I, McCann W, Sherbon T. Adelaide Revisited: from healthy public policy to Health in All Policies. Health Promot Int. 2008;23(1):1–4.View ArticleGoogle Scholar
- HIA and policy-making. Geneva: World Health Organization; 2012. http://www.who.int/hia/policy/en/. Accessed 11 February 2016.
- Stacey D, Bennett C, Barry M, Col N, Eden K, Holmes-Rovner M, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2011;10:CD001431.Google Scholar
- Shorten A, Shorten B, Keogh J, West S, Morris J. Making choices for childbirth: a randomized controlled trial of a decision-aid for informed birth after cesarean. Birth. 2005;32(4):252–61.View ArticlePubMedGoogle Scholar
- Gruskin S, Daniels N. Process is the point: justice and human rights: priority setting and fair deliberative process. Am J of Pub Health. 2008;98(9):1573–7.View ArticleGoogle Scholar
- Marmot M. Evidence based policy or policy based evidence? Willingness to take action influences the view of the evidence—look at alcohol. BMJ. 2004;328(7445):906–7.View ArticlePubMedPubMed CentralGoogle Scholar
- Lavis JN, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP). Health Res Policy Syst. 2009;7 Suppl 1:I1.View ArticlePubMedPubMed CentralGoogle Scholar
- Anderson J. Sausage and the art of public policy-making. The Examiner. November 27, 2010. http://www.examiner.com/environmental-policy-in-national/sausage-and-the-art-of-public-policy-making#ixzz1qcwBaGil. Accessed 11 February 2016.