Skip to main content

Table 3 Assessing how much confidence can be placed in the findings of systematic reviews of qualitative studies and systematic reviews of economic studies

From: SUPPORT Tools for evidence-informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review

An increasing number of systematic reviews of qualitative studies are being undertaken. These use a wide range of approaches, including narrative synthesis, meta-ethnography and realist review. As well as providing important information in their own right, reviews of qualitative studies can also inform and supplement systematic reviews of effects [51, 52]. However, it is important for the reader to assess the reliability of these reviews. To date, few tools have been designed for this specific purpose. Many of the questions used to guide policy makers when assessing the reliability of systematic reviews of effects, however, are also useful for reviews of qualitative studies. These include:

1. Did the review address an appropriate policy or management question? The review question should be amenable to being addressed using qualitative data and should be relevant to policymaking. Reviews of qualitative studies can provide insights about stakeholders' views and experiences regarding health and healthcare and thus help to clarify a problem [39]. Reviews of qualitative studies can also provide information on how or why options work (for example, through examining process evaluations conducted alongside the implementation of a policy or programme) and about stakeholders' views about the options and their relevant experiences [40, 53]

2. Were the criteria used to select studies appropriate? The description of how studies were selected should be appropriate in relation to the research question

3. Was a clear and appropriate explanation provided for the search approach used? Some reviews of qualitative studies undertake comprehensive literature searches while others may use sampling approaches. The chosen approach should be clearly described and justified

4. Was the approach used to assess the reliability of the included studies appropriate? The review should describe how the reliability of the included studies was taken into account

5. Was an appropriate approach used to analyse the findings of the included studies? The review should use an accepted approach to synthesis and should describe the rationale for the approach chosen

Questions to consider when assessing the reliability of reviews of economic studies include (from[54]):

1. Is it unlikely that important relevant studies were missed?

2. Were the inclusion criteria used to select articles appropriate?

3. Was the assessment of studies reproducible?

4. Were the design and/or methods and/or topic of included studies broadly comparable?

5. How reproducible are the overall results?

6. Will the results help resource allocation in healthcare?