Table 5 What should policymakers do when different systematic reviews that address the same question have different results?
When looking for evidence to inform a particular policy decision, it is not uncommon to identify more than one relevant systematic review. Sometimes the results of these reviews may be different, and this may result in review authors drawing different conclusions about the effects of an intervention. This scenario differs from one in which the findings of two or more reviews agree but in which researchers or others disagree on the interpretation of these findings [19]. There are many reasons why the results of different systematic reviews may differ. These include differences in: the questions addressed by the reviews, the inclusion and exclusion criteria used, which data were extracted from the studies, how the quality of the studies was assessed, and decisions regarding (and methods for) statistical analysis of the data [19]. |
|---|
The following series of questions designed by Jadad and colleagues can be used to assist with identifying and addressing the causes of discordance [19]: |
• Do the reviews address the same question? If not, the review that is chosen should be the one which addresses a question closest to that of the policy question for which evidence is needed. Alternatively, it should assess outcomes most relevant to the policy question |
• If the reviews address the same question, do they include the same trials or primary studies? If they do not include the same trials, the review that includes studies most relevant to the policy question being considered should be selected |
• If the reviews include the same studies, are the reviews of the same quality? If not, the higher quality review should be used |
Where both reviews are relevant, for example where they address different aspects of the same question, it may be useful to draw evidence from both. |