Table 1 Main results of the strategies for communicating scientific evidence that were implemented and evaluated

Communication of health risks and benefits under different numerical or nominal formats

[Akl, 2011b] [7]

This systematic review included 35 studies comparing the communication of health risks and benefits by natural frequency, percentages, relative risk reduction (RRR), absolute risk reduction (ARR) or number needed to treat (NNT). The main results are as follows

Natural frequencies versus percentages:

• comprehension: greater with natural frequencies than percentages (SMD 0.69; 95% CI 0.45–0.93; 642 participants; 7 comparisons; moderate-certainty evidence)

RRR versus ARR:

• comprehension: little or no difference between presentation formats (SMD 0.02; 95% CI −0.39 to 0.43; I [2] = 80%; 2 studies; moderate certainty evidence)

• persuasiveness: greater with RRR (SMD 0.62; 95% CI 0.42–0.83; 15 studies; moderate-certainty evidence)

RRR versus NNT:

• comprehension: greater with RRR (SMD 0.73; 95% CI 0.43–1.04; 1 study; evidence certainty not evaluated)

• persuasiveness: greater with RRR (SMD 0.65; 95% CI 0.51–0.80; 10 studies; moderate-certainty evidence)

ARR versus NNT:

• comprehension: greater with ARR (SMD 0.42; 95% CI 0.12–0.71; 1 study; moderate-certainty evidence)

• persuasiveness: little or no difference between presentation formats (SMD 0.05; 95% CI −0.04 to 0.14; 10 studies; moderate-certainty evidence)

[Büchter, 2014] [8]

This systematic review included ten studies, and the main results (measured by a six-point Likert scale, suggesting small to moderate effects) were:

• nominal presentation resulted in an overestimation of event risk

• numerical presentation resulted in more accurate estimates, increased satisfaction with the information (MD 0.48; 95% CI 0.32–0.63; p < 0.00001; I [2] = 0%) and the probability of medication use

[Chapman, 2020] [2]

This overview included 44 systematic reviews on strategies for health knowledge dissemination, including strategies for communicating health risks and benefits. The main results with sufficient evidence to be implemented were:

Natural frequencies versus percentages:

• comprehension (about outcomes of health intervention effects): greater with the use of natural frequencies than percentages

RRR versus ARR:

• comprehension: no difference between the formats

• persuasiveness: higher with RRR

RRR versus NNT:

• comprehension: higher with RRR

• persuasiveness: higher with RRR

ARR versus NNT:

• comprehension: higher with ARR

• persuasiveness: little or no difference between the formats

Numerical versus nominal (textual, printed) communication:

• satisfaction: for reporting adverse event risk in printed materials, satisfaction was significantly higher with numerical communication

[Fortin, 2001] [85]

This survey collected the opinion of 15 women about different ways of presenting risks related to hormone replacement therapy. The main results were:

• 83% of the participants preferred bar to line graphs, survival curves and visual scales with facial expressions;

• mortality estimates were preferred over 10 or 20 year survival

• there was a preference for absolute risks over relative risks and NNT

[Ghosh, 2005] [88]

This narrative review presented the following results about strategies for communication of risk for the population:

• there was a preference for ARR over NNT

• the ability to interpret graphs was limited

• for the population aged 75 years and over, there was a preference for graphs over percentages

[Knapp, 2004] [96]

This randomized controlled trial (RCT) included 120 participants taking simvastatin or atorvastatin after cardiac surgery or heart attack: 60 received a text communicating the risk of adverse events (constipation or pancreatitis) in nominal format, and 60 received the same text but with numerical reporting of the risks (for constipation, ‘common’ or 2.5%; for pancreatitis, ‘rare’ or 0.04%). The main results were:

• estimated mean probability of constipation: 34.2% in the nominal communication group and 8.1% in the numerical communication group; for pancreatitis: 18% in the nominal communication group and 2.1% in the numerical communication group

• nominal communication was associated with more negative perceptions of medications than equivalent numerical communication

• nominal risk communication overrides the harm level and could lead patients to make inappropriate decisions about whether to use the medication

[Kristiansen, 2012] [98]

This survey interviewed participants (general population) who were communicated about the risk of heart attack with the use of a hypothetical medication using the NNT. The results showed that:

• the NNT did not communicate information about the proportion of patients who benefited from an intervention or the extent to which an adverse event was being prevented

• 80% of participants agreed with the benefit of the medication, regardless of NNT

• some people who disagreed with the benefit of the medication misinterpreted the NNT

• the population may have difficulty understanding the meaning of NNT, and its use should be avoided for this target audience

[Lipkus, 2007] [100]

The main results of this narrative review are presented below:

Numerical risk communication:

• people generally preferred numerical information over other formats (for example, nominal categories for probability: unlikely or very likely)

• among numerical formats, natural frequencies were easier to understand

• being consistent with using numerical formats, for example, not comparing percentages with probabilities or frequencies, facilitated understanding

• using the same numerical denominator (for example, comparing 5 out of 100 with 15 out of 100) facilitated comparisons and reduced cognitive effort

• in general, individuals more easily understood base-10 denominators (for example, 10, 100, 1000)

• rounding numbers and avoiding decimals made it easier to comprehend (for example, it was easier to comprehend 30 than 29.6)

• expressing a ratio as small numbers (for example, 1 in 10) led to fewer perceptions of the probability of events than the same ratio incorporating larger numbers (for example, 10 in 100)

• specifying the relative risk and including the absolute risk were more comprehensive (for example, the risk of non-smokers getting the disease is 1%, while the risk of smokers is 10%, so smokers have a ten times higher risk of getting the disease than non-smokers)

Nominal risk communication:

• when nominal risk communication was the chosen format, using the main term and its variations added some objectivity and allowed comparisons (for example: likely, unlikely, very likely)

Visual risk communication:

• bar charts (histograms) were best suited for making comparisons, especially for subgroups (for example, comparing the magnitude of risk by ethnic group or sex)

• line graphs (survival curves) were best suited to show trends over time and perhaps interactions between risk factors

[McCormack, 2013] [1]

This systematic review included 61 studies about communication strategies. The main results are presented below

Communication about benefits – effectiveness:

• people who received non-numerical or factual communication about medication with a higher probability of benefit for myocardial infarction chose this medication more often than people who did not receive this communication (one study, low strength of evidence)

• receiving additional non-numerical information about benefits had little effect on refusals of cancer screening tests, but receiving non-numerical information about harms significantly increased refusals to screening tests and significantly decreased satisfaction with the decision (one study, low strength of evidence)

Communication about precision/imprecision:

• risk communication with numerical point estimates versus reporting with 95% CI: studies showed varying results depending on the outcome, the range of the 95% CI and the presence or absence of comparative information on the population mean risk

• numerical versus graphical communication of 95% CI regarding risk perception: uncertainty evidence (one study, insufficient strength of evidence)

Communication about direct evidence:

• choosing a cholesterol medication for which there was direct evidence of benefit was more frequent among people who received non-numerical communication or factual information with direct evidence encouraging the choice of the medication than those who did not receive this communication (one study, insufficient strength of evidence)

[Sheridan, 2003] [128]

This RCT included 407 participants who were randomized to receive one of four formats of risk communication about the comparative effects of two medications: RRR (n = 97), ARR (n = 108), NNT (n = 100) or a combination of all three formats (n = 98). The main results were:

• comprehension of the comparative effectiveness: higher in the RRR group (60% with RRR, 42% with ARR, 30% with NNT and 43% with the combination group; p = 0.001)

• ability to calculate treatment effect from baseline disease risk: higher in the RRR group (21% with RRR, 17% with ARR, 6% with NNT and 7% with the combination; p = 0.004)

• response on the effect calculation: 26% with RRR, 32% with ARR, 39% with NNT and 42% with the combination

• greater difficulty with comparisons and estimates was observed in the subgroups of non-white, with some college education, females, persons with health problems or who had not previously discussed quantitative data with their physicians

[Trevena, 2006] [130]

This systematic review included 10 systematic reviews of RCTs and 30 additional RCTs. The main results were:

• more modern communication strategies (verbal, textual, visual and electronic offered by the provider) increased patient comprehension but were more likely to do so if they were structured, adapted and/or interactive

• probabilistic information was best communicated as event rates (natural frequency) rather than nominal terms (for example, much, little) and measures of effect size (such as relative risk reduction)

• figures such as cartoons or graphs (for example, vertical bar graphs) aided comprehension

• value clarification exercises helped in individual decision-making

Health communication with positive (benefits, gains) or negative (losses) words/terms

[Akl, 2011a] [6]

This systematic review included 35 studies (16 342 participants from the general population) that compared communication of health attributes or effects of health interventions/exposures with positive (benefits, gains) or negative (losses) words/terms. The main results are below

For communicating attributes:

• comprehension (Likert scale): higher with negative words/terms; SMD −0.51; 95% CI −0.94 to −0, 22; 1 study; moderate effect size; low-certainty evidence

• persuasiveness (measured as a hypothetical decision or intention or willingness to adopt an intervention, Likert scale): little or no difference with positive or negative words/terms; SMD 0.07; 95% CI −0.23 to 0.37; 11 studies; low-certainty evidence

• behaviour (Likert scale): little or no difference with positive or negative words/terms; SMD 0.09; 95% CI −0.14 to 0.31; 1 study; moderate-certainty evidence

For communicating the effects of interventions/exposition:

• comprehension: no study evaluated the effect of communication as losses or gains on comprehension

• persuasiveness (measured as a hypothetical decision or intention or willingness to adopt an intervention, Likert scale): for communication about treatment effects, persuasiveness was higher with words/terms signifying losses (SMD −0.50; 95% CI −1.04 to 0.04; 3 studies; moderate effect size; very low-certainty evidence)

• behaviour (Likert scale): little or no difference with words/terms meaning gains or losses; SMD −0.06; 95% CI −0.15 to 0.03; 16 studies; low-certainty evidence

[Edwards, 2001] [82]

This narrative review concluded that there is a greater comprehension with the use of words/terms meaning losses versus words/terms meaning gains when communicating health risks and benefits (OR 1.18, 95% CI 1.01–1.38)

[Gallagher, 2013] [37]

This systematic review included 94 studies comparing the communication of health outcome results with words/terms meaning gains or losses. In total, 189 measures of effect size were evaluated, and the main results were:

• behaviour: reporting the results of effect size measures as gains was more effective for encouraging desirable behaviour than reporting the results as losses (p = 0.002), particularly for skin cancer prevention, smoking cessation and physical activity

[McCormack, 2013] [1]

This systematic review included 61 studies on health communication strategies, and the main results on positive (benefits, gains) or negative terms were:

• loss content communications associated with dichotomous (yes/no) narratives were more persuasive than (i) loss content communications associated with statistical information or (ii) gain content communications associated with narratives or statistical information (one study; insufficient strength of evidence)

Verbal versus visual communication of the effects of interventions

[Lopez, 2008] [102]

This systematic review included five RCTs on contraceptive efficacy communication strategies. The main results were:

• comprehension: higher in communication with sound slides than with the physician’s explanation (MD −19.00, 95% CI −27.52 to −10.48; one RCT)

• correct answers: more frequent with efficacy communication using a category table compared to a numerical table (OR 2.42, 95% CI 1.43–4.12) and when compared to a category/numeric table (OR 2.58. 95% CI 1.5–4.42; one RCT)

Communicating health risks and benefits with bar charts or bar charts and histograms

[Ghosh, 2008] [89]

This RCT included 150 women: 74 were randomized to receive communication via bar charts (categoric bars) and 76 via bar charts and histograms (frequency diagrams, bars with a range of values). The main results were:

• 72% of the women overestimated their risk of breast cancer before the interventions

• the frequency of women who had improved comprehension of this risk was not different between the different communication strategies (42% versus 54%; p = 0.1)

• for the subgroup that overestimated the risk before the interventions, improvements in the estimate accuracy were more frequent in those receiving communication with bar charts and frequency diagrams (19% versus> 9%, p = 0.004)

Strategies for communication of health evidence

[Epstein, 2004] [83]

This systematic review included eight studies, and the main results were:

• the order of communicating the information and the outcomes may distort the comprehension of the population

• when evidence was limited, using graphs or figures with human faces representing probabilities and vertical bar charts for comparative information were helpful

• less-educated and older people preferred proportions to percentages and did not comprehend confidence intervals

• the absolute risk was better comprehended than RRR

• review authors suggested five aspects that should be considered when communicating scientific evidence to the population: comprehension of the experience and expectations of the patient (and their family members), building partnerships, providing evidence (including a balanced discussion of uncertainties), presenting recommendations informed by clinical judgment and patient preferences, and checking for comprehension and agreement

[Grimshaw, 2012] [11]

This narrative review presented some strategies for communicating health evidence divided into three groups

Decision support strategies (to assist choices about health treatment options):

• when compared with no strategy, decision support improved knowledge and risk accuracy perceptions, reduced the proportion of people who were passive in decision-making, resulted in a higher proportion of patients reaching informed decisions consistent with their values, reduced the number of people who remained undecided, reduced decision-making conflict and reduced choice for elective major surgery options favouring conservative options. Decision support did not impact satisfaction; however, further research is needed to clarify its effects on adherence to the chosen option, patient–professional communication, its cost-effectiveness and its impact on low-literate or developing populations (86 RCTs, 20 209 participants)

Personalized risk communication (information focusing on a personal interest using, for example, epidemiological calculation methods for risk calculations):

• personalized risk communication (textual, verbal or visual): increased uptake of screening tests for health conditions (weak evidence, consistent with a small effect; 22 RCT

Communication before consultations (any intervention delivered before a medical visit to help the patient to clarify his/her doubts during consultations):

• compared with control, communication before consultations increased questions asked during these consultations. Both verbal counselling and textual intervention produced similar effects on questions, but counselling increased patient satisfaction (33 RCT, 8244 participants)

Strategies for communicating risks and benefits in health with different animated graphical presentations

[Zikmund-Fisher, 2012] [137]

This RCT included 4198 participants who were randomized to receive risk–benefit outcomes of health interventions based on ten different graphical presentations. Following this, participants were asked to choose the most effective and safest treatment. The probability of the participant choosing the ‘correct’ treatment with each type of graph was:

• static grouped: OR 1

• static scattered: OR 0.59; 95% CI 0.38–0.91

• scatter, settles: OR 0.67; 95% CI 0.43–1.03

• grouped, built: OR 1.01; 95% CI 0.64–1.60

• scatter, built: OR 0.75; 95% CI 0.48–1.18

• scatter, built, settles: OR 0.80; 95% CI 0.51–1.26

• scatter, auto shuffles: OR 0.64; 95% CI 0.40–1.00

• scatter, auto shuffles, settles: OR 0.94; 95% CI 0.59–1.49

• scatter, user shuffles: OR 0.52; 95% CI 0.34–0.80

• scatter, user shuffles, settles: OR 0.81; 95% CI 0.52–1.27

After interpreting the results, the authors concluded that the proposed animated risk graphics showed no benefits over the traditional charts. In some cases, the animated graphs worsened the communication of risks and benefits of the intervention

Communication of uncertainties about the effects of interventions on health

[Büchter, 2020] [73]

In this RCT, eight versions of a summary about the effects of medication for tinnitus were compared. The versions varied in degree, type and magnitude (number of reasons) of uncertainty. Overall, 1727 participants were randomized to receive one of these versions, and the following results were reported:

• perception of treatment efficacy: no difference between the methods of presenting the degree and type of uncertainty; as to the method of presenting the magnitude of uncertainty, there was greater perception when two reasons were presented compared to three (p = 0.04)

• certainty for judging the efficacy of treatment: no difference between the variation of showing the degree, type and magnitude of uncertainty

• perception about the final body of evidence: the description of imprecision was associated with a greater perception of the limitations of the evidence than the general statement that more research is needed (p = 0.01)

• quality of the text: no difference between the methods for presenting the degree, type and magnitude of uncertainty

• the decision to use the medication: no difference between the methods for presenting the degree, type and magnitude of uncertainty

Blogshots to communicate the results of systematic reviews

[Arienti, 2018] [69]

This case study reported the experience of implementing five blogshots (infographics) to communicate the results of Cochrane reviews on rehabilitation in plain language. The results of accessing each blogshot were:

• yoga: 2633 views, 67 interactions, 49 access to review

• vocational rehabilitation: 2697 views, 67 interactions, 23 accesses to the full review

• fatigue treatment: 1712 views, 76 interactions, 39 access to review

• cardiovascular rehabilitation: 3419 views, 120 interactions, 51 accesses to full review (p = 0.12)

Evidence synthesis summary template

[Hartling, 2018] [92]

This survey collected information and opinions from managers on evidence synthesis summaries. The qualitative analysis of the results showed that:

• decision-makers suggested a three-page summary with key messages, details on results, meaningful numbers/tables, and strength of evidence

Detailed methods and contextual information were considered less important

Plain language abstract

[Kerwer, 2021] [95]

This descriptive study evaluated, through a survey, the opinion of 166 people about 12 original (scientific) abstracts and their respective plain language abstracts reporting the results of 12 different study designs. The main results showed that the plain language versions:

• presented greater readability and allowed a correct comprehension of the corresponding information

• were perceived as more reliable

• were able to make the reader more confident about their ability to decide based on the content learned

Cochrane plain language summaries

[Santesso, 2015] [124]

This RCT included 143 participants from five countries (Canada, Norway, Argentina, Spain and Italy): 97 were randomized to receive the plain language summaries and 96 to receive the original abstract (scientific) of a Cochrane systematic review about the effects of vitamin C for the common cold. The main results were:

• more participants in the plain language group comprehended the benefits and harms of treatment and the certainty of the evidence (53% versus 18%, p < 0.001). Comprehension occurred regardless of education level

• more participants in the plain language group answered the requested questions correctly (p < 0.001)

• reliability, accessibility, comprehensiveness and utility for helping decisions were more frequent among those who received plain language summaries

Templates for plain language abstracts of systematic reviews

[Marquez, 2018] [103]

In this survey, managers evaluated two new templates and one traditional template for summaries of systematic reviews through the System Usability Scale (SUS, a score < 68 is below average usability). The SUS score (standard deviation) was 55.7 (17.2) for the traditional template, 85.5 (8.0) for the new template 1 and 86.4 (11.5) for the new template 2

Printed newsletters for communicating health evidence

[Murthy, 2012] [108]

This literature review identified interrupted time series assessing the dissemination of printed newsletters based on evidence from systematic reviews. The main results were:

• reduced surgery rates for prominent ear correction in children younger than 10 years (mean annual decline: −10.1%, 95% CI −7.9 to −12.3)

• reduced surgery rates for prominent ear correction in children younger than 15 years (mean quarterly decline: −0.044, 95% CI −0.080 to −0.011)

Systematic review summaries of evidence templates for policymakers and health system managers

[Petkovic, 2016] [116]

This systematic review included six studies assessing the use of evidence summaries by managers, and the main results showed that:

• evidence summaries were more effortless to comprehend than full systematic reviews

• it is unclear whether using abstracts increased the use of evidence derived from systematic reviews in decision-making

• targeted and personalized messages have increased the number of evidence-based public health policies and programmes

• there was little or no difference in the use of evidence summaries for decision-making regarding knowledge, comprehension or beliefs, perceived usefulness or usability

• summary tables of findings with the certainty of evidence rating were considered easier to comprehend compared with full systematic reviews

• reporting of study event rates and absolute differences were considered comprehensible

Guidelines for designing and evaluating health evidence communication products (CDC Clear Communication Index)

[Baur, 2014] [71]

In this survey, 870 participants (general population) blindly assessed the quality of two versions of the same health evidence communication material: the original version and the version adjusted according to the CDC clear checklist items. Communication index for designing and evaluating health communication products (http://www.cdc.gov/healthcommunication/ClearCommunicationIndex/). The three assessed materials were: questions and answers about using thimerosal (preservative) in vaccines, a fact sheet on heart disease and a fact sheet on cell phone use and health. The results of the evaluations showed that:

• on average, the original versions of the three materials scored less than 30% on the checklist, and the adjusted versions scored 90% or more

• the adoption of the checklist increased the quality of health communication and evidence products for the population

Tool for evaluating the quality of health texts in plain language

[Logullo, 2019] [101]

In this study, the DISCERN tool was translated, culturally adapted and had its psychometric properties evaluated in Portuguese. The tool was applied in a validation study that used the plain language summary of a Cochrane systematic review. The main results were:

• the Brazilian Portuguese version had excellent internal consistency and good reproducibility

• age, sex and health literacy did not interfere with the score resulting from the application of the tool

Communication/learning of key concepts related to the effects of health interventions

[Cusack, 2018] [9]

This systematic review included 24 studies (most at high or moderate risk of bias) on communication/learning strategies key concept effects of health interventions

Strategies implemented inside and outside the school environment, single or multiple, were identified using different approaches such as discussion groups, printed material, online classes, and short- or long-term courses. The outcomes evaluated included: knowledge, skills, behaviour, confidence, perception of knowledge and/or skill, attitude and satisfaction. The main results observed were:

• the effects of strategies on trust, attitude and behaviour were uncertain

• improvements in the quality of studies, consistency of outcome measures, and longer-term evaluation of strategies are needed to improve reliability in estimating the effects of the strategies evaluated

Communication/learning resources from the IHC initiative on key concepts of evidence for health

[Ikirezi, 2016] [94]

This case study evaluated the feasibility of implementing communication/learning resources to support the comprehension and application of key concepts in the critical assessment of the evidence in health in a preschool in Rwanda. The main results observed in the qualitative analysis were:

• the user experience was positive, as implementing the IHC features was considered beneficial, contextualized, reliable, feasible and desirable

• the restricted time to use the resources was considered a barrier, while curiosity and a positive attitude towards the resources were facilitators

• students and faculty suggested that IHC resources be distributed to other students at other schools so they could also benefit from the teachings and importance of making evidence-informed health choices

[Mugisha, 2016] [107]

This case study reported the experience of implementing communication/learning resources to support the comprehension and application of key concepts in the critical appraisal of the evidence in health in a primary school in Rwanda. The main results observed in the qualitative analysis were:

• the use of IHC resources translated into Kinyarwanda was considered viable in Rwanda

• it was essential to collect suggestions and ideas from participants to contextualize the IHC resources in the local scenario

• children and teachers can be helpful in evaluating and reviewing primary school resources and contribute significantly to improving educational resources that would benefit ministries of education

• the resources were considered useful, feasible, reliable and comprehensible by users

[Nsangi, 2017] [110]

This cluster clinical trial included 120 schools that were randomized to receive learning resources (for example, teacher’s guides, exercise and textbooks, posters, songs and activities cards; intervention group, n = 60, 76 teachers and 6383 children) or not (control group, n = 60, 67 teachers and 4430 children). The main results were:

• children's mean score on a test with two multiple-choice questions for each of the 12 key concepts in the material: 62.4% (SD 18.8) in the intervention group versus 43.1% (SD 15.2) in the control group (adjusted mean difference 20%; 95% CI 17.3–22.7; p < 0.00001)

• the proportion of children with sufficient scores to pass the same test (≥ 13 of 24 correct answers): 69% (3967/5753) in the intervention group versus 27% (1186/4430) in the control group (adjusted difference of 50%; 95% CI 44–55)

• the intervention was effective for children with different reading skills but was more effective in the subgroup of children with better reading skills

Communication/learning of key concepts of health evidence

[Nordheim, 2016] [109]

This systematic review identified RCT and non-randomized studies comparing different educational strategies to acquire skills for the critical assessment of health evidence. The main results were:

• active versus traditional teaching methodologies: capacity for basic knowledge about causality and association, randomization, epidemiology concepts and evidence-based medicine was 71% higher with active methods

• educational strategy versus control: the ability to recognize that multiple outcomes can influence cancer research results were twice as high in the group that received an educational strategy

• educational strategy with active methodology versus control: the comprehension of the need for comparative studies to make inferences about causality was 51% higher with the use of the educational strategy

• fictional evidence reasoning simulation versus control group: reduction in the number of inappropriate responses, including personal beliefs and unsupported opinions, with the use of simulation

Educational podcasts from the IHC initiative on key health evidence concepts

[Ringle, 2020] [118]

This RCT included 250 American parents randomized to listen to the podcasts with evidence-based health content developed by the IHC initiative (intervention group, n = 128) or podcasts with general information (control group, n = 122). The main results were:

• critical thinking skill: 53% of parents in the intervention group achieved a satisfactory score on the applied skill test (> 18 and 21) versus 26.2% in the control group

• satisfaction with the podcast (scale 1–5): similar between groups (4.16 ± 0.93 versus 4.20 ± 0.85)

• listening to the IHC podcast improved parents’ behaviour towards evidence-based practice and preference for evidence-based health information

• podcasts are available at: https://www.informedhealthchoices.org/podcast-for-parents/

[Semakula, 2017; Semakula, 2020] [125, 126]

This RCT included 675 parents of Ugandan elementary school students who were randomized to listen to podcasts with evidence-based health content (intervention group, n = 334) developed by the IHC initiative or podcasts with general health information used by the public service (control group, n = 341). The main initial and post-1 year results were:

• mean critical thinking skill test score (two multiple-choice questions for each of nine key critical thinking concepts, 18 questions total): 67.8% (SD 19.6%) in the intervention group versus 52 .4% (SD 17.6%) in the control group (adjusted DM 15.5%; 95% CI 12.5–18.6%; p < 0.0001); after 1 year: 58.9% in the intervention group versus 52.6% in the control group (adjusted DM 6.7%; 95% CI 3.3–10.1%)

• frequency of parents who achieved the minimum passing test score (at least 11 out of 18): 71% (203/288) in the intervention group versus 38% (103/273) in the control group (adjusted SD 34%; 95% CI 26–41%; p < 0·0001); after 1 year: 47.2% in the intervention group versus 39.5% in the control group (adjusted SD 9.8%; 95% CI 0.9–18.9%; p < 0.0001)

• listening to the IHC initiative podcast improved parents’ ability to critically assess the information at baseline, but this ability declined substantially after 1 year

[Semakula, 2019b] [127]

This descriptive study used design thinking methodology to develop podcasts on health evidence and presented users’ assessments of the podcasts. The main results were:

• usefulness: IHC podcasts were considered useful tools that could help encourage critical thinking when publicized in the general media and specific contexts (for example, schools)

• usability and comprehension: were considered satisfactory

• credibility: considered satisfactory

• desire to use: some participants asked if they could have access to all episodes. A non-governmental health communication organization and producers from the Uganda Broadcasting Corporation expressed their interest in broadcasting the podcasts on radio as part of their health communication programming

Training for parliamentarians on scientific health evidence

[Cockcroft, 2014] [78]

In this case study, the authors reported the experience of training on scientific evidence in health for 36 of Botswana’s 54 elected parliamentarians. The training took place in two sessions (one theoretical and one practical workshop). It addressed the following topics: (i) initial concepts about scientific evidence (clinical trials, randomization, statistical significance, number needed to treat and bias) and the importance of control or comparators when evaluating the effects of interventions, (ii) how biases can distort results and reports, (iii) importance of evaluating the impact on public health and not just on individual health. The short-term qualitative assessment showed that:

• feedback from Botswana parliamentarians were very favourable: they requested additional sessions to address the topics in more detail, and that training be offered to other decision-makers

• after the training, one of the parliamentarians reported that the debate on updating the national human immunodeficiency virus (HIV) policy was more detailed and focused on evidence

Inclusion of stakeholders in the working group for preparing comparative effectiveness summaries

[Balshem, 2011] [70]

In this case study, the authors presented the process and results of including stakeholders (including managers) in preparing the summary ‘Medications to reduce the risk of primary breast cancer in women’. Stakeholders suggested that the conclusions of the summaries go beyond just saying that the evidence is insufficient and that further studies are needed. Instead, stakeholders indicated that the following issues be addressed in the summaries:

• what evidence is available, and what can be learned from it?

• what evidence tells us when and if an intervention is safe/harmful and effective/ineffective for relevant clinical outcomes?

• what can we learn from the evidence from a study inferior to a randomized clinical trial?

• what can patient records tell us about the safety of a treatment?

• what evidence identifies the subpopulations most likely to benefit from its use?

• what kind of evidence is needed to assess short-term and long-term effectiveness?

• when is short-term evidence appropriate to be implemented?