Skip to main content

Table 7 How confident are we in the estimated impacts? Case example: The licensing of tobacco retailers

From: SUPPORT Tools for evidence-informed health Policymaking (STP) 16: Using research evidence in balancing the pros and cons of policies

The expert report commissioned by the government concerned concluded that the empirical basis for the licensing of tobacco retailers was "robust" but the basis for this judgement was unclear. The experts did not conduct, or cite, the systematic review that is referenced in Table 3, or any other systematic review as the basis for their estimates, even though a systematic review was available [24]. In contrast to the experts' unexplained judgement, an assessment of the evidence summarised in the systematic review using the GRADE approach, suggests that the quality of the evidence was very low for all the important outcomes (see Table 8 for further information related to the GRADE assessment system). Table 1 summarises the findings of the experts' report in the form of a balance sheet for this policy decision and shows an assessment of the quality of the evidence for the three estimates using the GRADE approach.
The authors of the systematic review (which included a broader range of interventions and study designs) concluded: "Interventions with retailers can lead to large decreases in the number of outlets selling tobacco to youths. However, few of the communities studied in this review achieved sustained levels of high compliance. This may explain why there is limited evidence for an effect of the intervention on youth perceptions about ease of access to tobacco, and on smoking behaviour." The 'pessimistic' estimates of the benefits in Table 1 are consistent with the findings of the systematic review and were not considered in the expert report.