Table 1 Use of reviews by type of agency
From: Do policy-makers find commissioned rapid reviews useful?
| Â | Frontline government agencies | Central government agencies | Government funded agencies | Non-government organisations | Total | |||||
|---|---|---|---|---|---|---|---|---|---|---|
Did you use the review? | n | % | n | % | n | % | n | % | n | % |
  Yes | 80 | 95.2 | 25 | 100 | 25 | 96.2 | 4 | 100 | 134 | 89.3 |
  No | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
  No, but use is planned | 3 | 3.6 | 0 | 0 | 1 | 3.8 | 0 | 0 | 4 | 2.7 |
  Other | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
  Not stated | 1 | 1.2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0.7 |
  Not interviewed | n/a | 0 | n/a | n/a | n/a | n/a | n/a | n/a | 11 | 7.3 |
  Total | 84 | 100 | 25 | 100 | 26 | 100 | 4 | 100 | 150 | 100 |
Was it used in one or more policy processes? | ||||||||||
  Used in a single policy process | 15 | 17.9 | 3 | 12 | 4 | 15.4 | 1 | 25 | 23 | 15.3 |
  Used in multiple policy processes | 68 | 81 | 22 | 88 | 22 | 84.6 | 3 | 75 | 115 | 76.7 |
  Other | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
  Not stated | 1 | 1.2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0.7 |
  Not interviewed | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 11 | 7.3 |
  Total | 84 | 100 | 25 | 100 | 26 | 100 | 4 | 100 | 150 | 100 |
In what policy process was it used? | ||||||||||
  In policy agenda-setting | 31 | 36.9 | 23 | 92.0 | 6 | 23.1 | 1 | 25.0 | 61 | 40.7 |
  In research agenda-setting | 1 | 1.2 | 0 | 0.0 | 2 | 7.7 | 2 | 50.0 | 5 | 3.3 |
  In policy or programme development | 43 | 51.2 | 2 | 8.0 | 16 | 61.5 | 1 | 25.0 | 62 | 41.3 |
  In policy or programme implementation | 2 | 2.4 | 0 | 0.0 | 1 | 3.8 | 0 | 0.0 | 3 | 2.0 |
  In policy or programme evaluation | 2 | 2.4 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 2 | 1.3 |
  In a research process | 4 | 4.8 | 0 | 0.0 | 1 | 3.8 | 0 | 0.0 | 5 | 3.4 |
  Other | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
  Not stated | 1 | 1.2 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 0.7 |
  Not interviewed | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 11 | 7.3 |
  Total (all instances) | 84 | 100.0 | 25 | 100.0 | 26 | 100.0 | 4 | 100.0 | 150 | 100.1 |
How was it used? | ||||||||||
 Instrumental use | ||||||||||
  To determine the details of a policy or programme | 52 | 26.7 | 3 | 4.8 | 19 | 26.4 | 2 | 20 | 76 | 22.5 |
  To identify or evaluate alternative solutions for a policy or programme | 12 | 6.2 | 17 | 27.4 | 5 | 6.9 | 1 | 10 | 35 | 10.4 |
  To communicate information to stakeholders or the general public | 14 | 7.2 | 0 | 0 | 12 | 16.7 | 1 | 10 | 27 | 8 |
  To develop a clinical guideline, protocol or resource | 9 | 4.6 | 1 | 1.6 | 7 | 9.7 | 0 | 0 | 17 | 5 |
  To determine the details of a research programme or process | 12 | 6.2 | 0 | 0 | 2 | 2.8 | 1 | 10 | 15 | 4.4 |
  To design or inform data collection, data linkage or data analysis | 3 | 1.5 | 0 | 0 | 0 | 0 | 0 | 0 | 3 | 0.9 |
  To determine the details of an evaluation programme or framework | 1 | 0.5 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0.3 |
   Subtotal 1: Instrumental use | 103 |  | 21 |  | 45 |  | 5 |  | 174 | 51.50 |
 Conceptual use | ||||||||||
  To determine priorities for future action or investment | 36 | 18.5 | 18 | 29 | 11 | 15.3 | 1 | 11.1 | 66 | 19.5 |
  To prepare for or negotiate a decision across agencies or jurisdictions | 25 | 12.8 | 19 | 30.6 | 2 | 2.8 | 1 | 10 | 47 | 13.9 |
  To understand the nature or extent of a problem | 13 | 6.7 | 3 | 4.8 | 4 | 5.6 | 2 | 20 | 22 | 6.5 |
  To confirm thinking or verify ideas | 5 | 2.6 | 1 | 1.6 | 7 | 9.7 | 0 | 0 | 13 | 3.8 |
  To create impetus for change | 1 | 0.5 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0.3 |
   Subtotal 2: Conceptual use | 80 |  | 41 |  | 24 |  | 4 |  | 149 | 44.08 |
 Symbolic use | ||||||||||
  To consult with stakeholders or to seek consensus | 8 | 4.1 | 0 | 0 | 2 | 2.8 | 0 | 0 | 10 | 3 |
  To justify or strengthen an existing policy position | 4 | 2.1 | 0 | 0 | 1 | 1.4 | 0 | 0 | 5 | 1.5 |
   Subtotal 3: Symbolic use | 12 |  | 0 |  | 3 |  | 0 |  | 15 | 4.44 |
  Subtotal 1: Instrumental use | 103 | 52.8 | 21 | 33.9 | 45 | 62.5 | 5 | 55.6 | 174 | 51.5 |
  Subtotal 2: Conceptual use | 80 | 41 | 41 | 66.1 | 24 | 33.3 | 4 | 44.4 | 149 | 44.1 |
  Subtotal 3: Symbolic use | 12 | 6.2 | 0 | 0 | 3 | 4.2 | 0 | 0 | 15 | 4.4 |
  Total (all instances) | 195 | 100 | 62 | 100 | 72 | 100 | 9 | 100 | 338a | 100.0 |
What evidence of use was provided? | ||||||||||
 Details specified during interviews | ||||||||||
  The ministries, agencies or people who commissioned the review | 84 | 37.5 | 25 | 39.7 | 26 | 34.2 | 4 | 57.1 | 139 | 100.0 |
  The policy, programme or guideline to which the findings contributed | 73 | 32.6 | 23 | 36.5 | 22 | 28.9 | 2 | 28.6 | 120 | 86.3 |
  The target audience(s) (if additional to those commissioning the review) | 39 | 17.4 | 11 | 17.5 | 13 | 17.1 | 1 | 14.3 | 64 | 46.0 |
  The forum or workshop where the findings were presented | 13 | 5.8 | 1 | 1.6 | 6 | 7.9 | 0 | 0 | 20 | 14.4 |
  Stakeholders groups attending the forum or workshop | 10 | 4.5 | 0 | 0 | 4 | 5.3 | 0 | 0 | 14 | 10.1 |
  People or agencies who requested copies of the findings | 3 | 1.3 | 2 | 3.2 | 3 | 3.9 | 0 | 0 | 8 | 5.8 |
  People or agencies who cited or reported the findings in a document | 2 | 0.9 | 1 | 1.6 | 2 | 2.6 | 0 | 0 | 5 | 3.6 |
  Other | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 |
  Total (all instances) | 224 | 100 | 63 | 100 | 76 | 100 | 7 | 100 | 370a | 100.0 |
 Additional details identified | ||||||||||
  On a website, or in an online media report, or in social media | 20 | 47.6 | 4 | 66.7 | 15 | 51.7 | 3 | 60 | 42 | 51.2 |
  In a white paper, other consultation or discussion document | 6 | 14.3 | 1 | 16.7 | 0 | 0 | 0 | 0 | 7 | 8.5 |
  In a new/revised policy or programme document | 4 | 9.5 | 1 | 16.7 | 2 | 6.9 | 0 | 0 | 7 | 8.5 |
  In a ministerial, policy brief or summary | 4 | 9.5 | 0 | 0 | 3 | 10.3 | 0 | 0 | 7 | 8.5 |
  In an email, e-bulletin or newsletter | 0 | 0 | 0 | 0 | 4 | 13.8 | 2 | 40 | 6 | 7.3 |
  In a clinical guideline, manual or other clinical resource | 4 | 9.5 | 0 | 0 | 1 | 3.4 | 0 | 0 | 5 | 6.1 |
  In the agenda or records of a meeting, forum or workshop | 3 | 7.1 | 0 | 0 | 1 | 3.4 | 0 | 0 | 4 | 4.9 |
  In an evaluation plan, protocol or document | 0 | 0 | 0 | 0 | 1 | 3.4 | 0 | 0 | 1 | 1.2 |
  In a grant application, research protocol or research report | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 |
  Other (media release) | 1 | 2.4 | 0 | 0 | 2 | 6.9 | 0 | 0 | 3 | 3.7 |
  Total (all instances) | 42 | 100 | 6 | 100 | 29 | 100 | 5 | 100 | 82b | 100.0 |
Why were reviews not used? | ||||||||||
  The findings disagreed with an existing policy position | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
  The findings told us nothing new about the issue | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
  The findings were not presented in a useful way | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
  The findings gave us insufficient information to support action | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
  There was a change in the policy environment | 2 | 2.4 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 1.3 |
  There was no momentum for change in the agency or sector | 1 | 1.2 | 0 | 0 | 1 | 3.8 | 0 | 0 | 2 | 1.3 |
  It was difficult to integrate the findings in a policy or programme | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
  Not stated | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0.0 |
  Not applicable (reviews were used) | 81 | 96.4 | 25 | 100 | 25 | 96.2 | 4 | 100 | 134 | 90.0 |
  Not interviewed | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 11 | 7.3 |
  Total (all instances) | 84 | 100 | 25 | 0 | 26 | 0 | 4 | 0 | 150 | 100 |
What barriers were mentioned when reviews were used? | ||||||||||
  The findings disagreed with an existing policy position | 1 | 1.2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0.7 |
  The findings told us nothing new about the issue | 0 | 0 | 0 | 0 | 1 | 3.8 | 0 | 0 | 1 | 0.7 |
  The findings were not presented in a useful way | 2 | 2.4 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 1.3 |
  The findings gave us insufficient information to support action | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
  There was a change in the policy environment | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 25 | 1 | 0.7 |
  There was no momentum for change in the agency or sector | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 |
  It was difficult to integrate the findings in a policy or programme | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
  Other (review was not completed in a timely way) | 1 | 1.2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0.7 |
  Not stated (no barriers were mentioned) | 80 | 95.2 | 25 | 100 | 25 | 96.2 | 3 | 75 | 133 | 88 |
  Not interviewed | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 11 | 7.3 |
  Total (all instances) | 84 | 100 | 25 | 0 | 26 | 100 | 4 | 100 | 150 | 100 |