Table 2 Common concerns identified on fidelity assessment
From: The use of GRADE-CERQual in qualitative evidence synthesis: an evaluation of fidelity and reporting
Fidelity assessment questions Common concerns identified in bullet points |
|---|
1. The authors demonstrate an accurate conceptualisation of GRADE-CERQual (that is, an approach for assessing confidence in the findings of a qualitative evidence synthesis) • Appears under quality appraisal section • Sometimes referred to as a tool to assess quality of findings or evidence • Conceptualised as an assessment of contributing studies |
2.The authors have made an overall assessment of confidence based on the assessment of all four components • No mention of the 4 components at all • Some components not assessed • Applied own scoring rules for determining level of assessment |
3.The authors applied GRADE-CERQual to individual review findings • Applied GRADE-CERQual at the study level not finding level • Applied it to short theme or category titles |
4.Authors conceptualise methodological limitations in line with the guidance • Applied the levels of concern to individual studies rather than review findings • Conceptualised the assessment as a count of appraisal categories, not specific limitations in relation to the finding • Component not defined and no Evidence Profile or SoQF tables from which to infer • Not conceptualised in terms of identifying concerns • Problems with how critical appraisals were done (e.g., only yes or no, no explanation) • Specific methodological limitations mentioned but not how important they are in relation to the finding |
5. Authors conceptualise coherence in line with the guidance • Component not defined and no Evidence Profile or SoQF tables from which to infer • No demonstration of thinking of it in terms of the fit between review finding and data from primary studies, only focus on primary studies • Not conceptualised in terms of identifying concerns • Using wrong definition (“Consistent within and across studies”) • Assessment was quantified |
6. Authors conceptualise adequacy of data in line with the guidance • Component not defined and no Evidence Profile or SoQF table from which to infer • Not assessed in terms of concerns • Not assessing both quantity and richness, emphasising one or the other • Confounding with other components • Quantify the assessment of the component |
7. Authors conceptualise relevance in line with the guidance • Component not defined and no Evidence Profile or SoQF tables from which to infer • Language of concerns not used, or not used correctly • Not all elements of ‘context’ were considered in the assessment • Quantify the assessment by counting how many primary studies are indirect or partial, rather than identifying concerns |
8.The GRADE-CERQual assessments are presented in-line with the guidance for SoQF tables and or Evidence Profiles • No SoQF or Evidence Profile tables included • Key elements missing or left out (such as references or explanations) • Way of writing explanations for component or overall assessments not aligned with guidance |
9. Summarised review findings were produced in line with the guidance • Summaries of findings either too detailed or too brief • Just theme or category names, not summarised review findings |