Table 4 Pragmatic tool evaluation consolidated scores (n = 58 tools)
From: Tools for assessing health research partnership outcomes and impacts: a systematic review
Tool evaluation criteria | Present (n) | Absent (n) | Cannot answer (n) | Score (%) | |
|---|---|---|---|---|---|
Scientific rigour | |||||
 SR1 | Based on systematic literature review? | 10 | 41 | 7 | 17% |
 SR2 | Based on the experiences or expertise of partners? | 32 | 25 | 1 | 55% |
 SR3 | Based on a conceptual or theoretical framework? | 52 | 6 | · | 90% |
 SR4 | Is there evidence of tool validity? (any source) | 52 | 6 | · | 90% |
 SR5 | Is there evidence of tool reliability? (any source) | 54 | 4 | · | 93% |
Scientific rigour domain score | Mean 3.58 | Std dev 0.87 | Range 1–5 | ||
Partner perspective | |||||
 PP1 | Partners involved as co-designers? | 34 | 23 | 1 | 59% |
 PP2 | Tool designed to be self-administered by partners? | 56 | 2 | · | 97% |
 PP3 | Assessment results must be reported back to partners? | 16 | 41 | 1 | 28% |
 PP4 | Tool assesses level of partner involvement? | 16 | 42 | · | 28% |
 PP5 | Tool captures the influence of partners? | 44 | 14 | · | 76% |
Partner perspective domain score | Mean 2.84 | Std dev 1.04 | Range 1–5 | ||
Comprehensiveness | |||||
 C1 | Tool documents partnership context? | 56 | 2 | · | 97% |
 C2 | Tool assesses partnership process? | 55 | 3 | · | 95% |
 C3 | Tool documents partnership outcome(s) and/or impact(s)? | 58 | 0 | · | 100% |
 C4 | Tool monitors the partnering process at multiple moments? | 19 | 37 | 2 | 33% |
 C5 | Tool consists of open- and closed-ended questions? | 32 | 25 | 1 | 55% |
Comprehensiveness domain score | Mean 3.79 | Std dev 0.75 | Range 2–5 | ||
Usability | |||||
 U1 | Tool purpose stated? | 58 | 0 | · | 100% |
 U2 | Tool freely accessible? | 29 | 29 | · | 50% |
 U3 | Tool available in a readily usable format? | 36 | 22 | · | 62% |
 U4 | Tool easy to read and understand? | 31 | 26 | 1 | 53% |
 U5 | Tool accompanied by instructions? | 33 | 25 | · | 57% |
Usability domain score | Mean 3.19 | Std dev 1.38 | Range 1–5 | ||
Overall D1–D4 total score | Mean 66.64 | Std dev 15.54 | Range 35–90 | ||