Skip to main content

Table 1 Use of reviews by type of agency

From: Do policy-makers find commissioned rapid reviews useful?

  Frontline government agencies Central government agencies Government funded agencies Non-government organisations Total
Did you use the review? n % n % n % n % n %
  Yes 80 95.2 25 100 25 96.2 4 100 134 89.3
  No 0 0 0 0 0 0 0 0 0 0
  No, but use is planned 3 3.6 0 0 1 3.8 0 0 4 2.7
  Other 0 0 0 0 0 0 0 0 0 0
  Not stated 1 1.2 0 0 0 0 0 0 1 0.7
  Not interviewed n/a 0 n/a n/a n/a n/a n/a n/a 11 7.3
  Total 84 100 25 100 26 100 4 100 150 100
Was it used in one or more policy processes?
  Used in a single policy process 15 17.9 3 12 4 15.4 1 25 23 15.3
  Used in multiple policy processes 68 81 22 88 22 84.6 3 75 115 76.7
  Other 0 0 0 0 0 0 0 0 0 0
  Not stated 1 1.2 0 0 0 0 0 0 1 0.7
  Not interviewed n/a n/a n/a n/a n/a n/a n/a n/a 11 7.3
  Total 84 100 25 100 26 100 4 100 150 100
In what policy process was it used?
  In policy agenda-setting 31 36.9 23 92.0 6 23.1 1 25.0 61 40.7
  In research agenda-setting 1 1.2 0 0.0 2 7.7 2 50.0 5 3.3
  In policy or programme development 43 51.2 2 8.0 16 61.5 1 25.0 62 41.3
  In policy or programme implementation 2 2.4 0 0.0 1 3.8 0 0.0 3 2.0
  In policy or programme evaluation 2 2.4 0 0.0 0 0.0 0 0.0 2 1.3
  In a research process 4 4.8 0 0.0 1 3.8 0 0.0 5 3.4
  Other 0 0.0 0 0.0 0 0.0 0 0.0 0 0.0
  Not stated 1 1.2 0 0.0 0 0.0 0 0.0 1 0.7
  Not interviewed n/a n/a n/a n/a n/a n/a n/a n/a 11 7.3
  Total (all instances) 84 100.0 25 100.0 26 100.0 4 100.0 150 100.1
How was it used?
 Instrumental use
  To determine the details of a policy or programme 52 26.7 3 4.8 19 26.4 2 20 76 22.5
  To identify or evaluate alternative solutions for a policy or programme 12 6.2 17 27.4 5 6.9 1 10 35 10.4
  To communicate information to stakeholders or the general public 14 7.2 0 0 12 16.7 1 10 27 8
  To develop a clinical guideline, protocol or resource 9 4.6 1 1.6 7 9.7 0 0 17 5
  To determine the details of a research programme or process 12 6.2 0 0 2 2.8 1 10 15 4.4
  To design or inform data collection, data linkage or data analysis 3 1.5 0 0 0 0 0 0 3 0.9
  To determine the details of an evaluation programme or framework 1 0.5 0 0 0 0 0 0 1 0.3
   Subtotal 1: Instrumental use 103   21   45   5   174 51.50
 Conceptual use
  To determine priorities for future action or investment 36 18.5 18 29 11 15.3 1 11.1 66 19.5
  To prepare for or negotiate a decision across agencies or jurisdictions 25 12.8 19 30.6 2 2.8 1 10 47 13.9
  To understand the nature or extent of a problem 13 6.7 3 4.8 4 5.6 2 20 22 6.5
  To confirm thinking or verify ideas 5 2.6 1 1.6 7 9.7 0 0 13 3.8
  To create impetus for change 1 0.5 0 0 0 0 0 0 1 0.3
   Subtotal 2: Conceptual use 80   41   24   4   149 44.08
 Symbolic use
  To consult with stakeholders or to seek consensus 8 4.1 0 0 2 2.8 0 0 10 3
  To justify or strengthen an existing policy position 4 2.1 0 0 1 1.4 0 0 5 1.5
   Subtotal 3: Symbolic use 12   0   3   0   15 4.44
  Subtotal 1: Instrumental use 103 52.8 21 33.9 45 62.5 5 55.6 174 51.5
  Subtotal 2: Conceptual use 80 41 41 66.1 24 33.3 4 44.4 149 44.1
  Subtotal 3: Symbolic use 12 6.2 0 0 3 4.2 0 0 15 4.4
  Total (all instances) 195 100 62 100 72 100 9 100 338a 100.0
What evidence of use was provided?
 Details specified during interviews
  The ministries, agencies or people who commissioned the review 84 37.5 25 39.7 26 34.2 4 57.1 139 100.0
  The policy, programme or guideline to which the findings contributed 73 32.6 23 36.5 22 28.9 2 28.6 120 86.3
  The target audience(s) (if additional to those commissioning the review) 39 17.4 11 17.5 13 17.1 1 14.3 64 46.0
  The forum or workshop where the findings were presented 13 5.8 1 1.6 6 7.9 0 0 20 14.4
  Stakeholders groups attending the forum or workshop 10 4.5 0 0 4 5.3 0 0 14 10.1
  People or agencies who requested copies of the findings 3 1.3 2 3.2 3 3.9 0 0 8 5.8
  People or agencies who cited or reported the findings in a document 2 0.9 1 1.6 2 2.6 0 0 5 3.6
  Other 0 0 0 0 0 0 0 0 0 0.0
  Total (all instances) 224 100 63 100 76 100 7 100 370a 100.0
 Additional details identified
  On a website, or in an online media report, or in social media 20 47.6 4 66.7 15 51.7 3 60 42 51.2
  In a white paper, other consultation or discussion document 6 14.3 1 16.7 0 0 0 0 7 8.5
  In a new/revised policy or programme document 4 9.5 1 16.7 2 6.9 0 0 7 8.5
  In a ministerial, policy brief or summary 4 9.5 0 0 3 10.3 0 0 7 8.5
  In an email, e-bulletin or newsletter 0 0 0 0 4 13.8 2 40 6 7.3
  In a clinical guideline, manual or other clinical resource 4 9.5 0 0 1 3.4 0 0 5 6.1
  In the agenda or records of a meeting, forum or workshop 3 7.1 0 0 1 3.4 0 0 4 4.9
  In an evaluation plan, protocol or document 0 0 0 0 1 3.4 0 0 1 1.2
  In a grant application, research protocol or research report 0 0 0 0 0 0 0 0 0 0.0
  Other (media release) 1 2.4 0 0 2 6.9 0 0 3 3.7
  Total (all instances) 42 100 6 100 29 100 5 100 82b 100.0
Why were reviews not used?
  The findings disagreed with an existing policy position 0 0 0 0 0 0 0 0 0 0
  The findings told us nothing new about the issue 0 0 0 0 0 0 0 0 0 0
  The findings were not presented in a useful way 0 0 0 0 0 0 0 0 0 0
  The findings gave us insufficient information to support action 0 0 0 0 0 0 0 0 0 0
  There was a change in the policy environment 2 2.4 0 0 0 0 0 0 2 1.3
  There was no momentum for change in the agency or sector 1 1.2 0 0 1 3.8 0 0 2 1.3
  It was difficult to integrate the findings in a policy or programme 0 0 0 0 0 0 0 0 0 0
  Not stated 0 0 0 0 0 0 0 0 1 0.0
  Not applicable (reviews were used) 81 96.4 25 100 25 96.2 4 100 134 90.0
  Not interviewed n/a n/a n/a n/a n/a n/a n/a n/a 11 7.3
  Total (all instances) 84 100 25 0 26 0 4 0 150 100
What barriers were mentioned when reviews were used?
  The findings disagreed with an existing policy position 1 1.2 0 0 0 0 0 0 1 0.7
  The findings told us nothing new about the issue 0 0 0 0 1 3.8 0 0 1 0.7
  The findings were not presented in a useful way 2 2.4 0 0 0 0 0 0 2 1.3
  The findings gave us insufficient information to support action 0 0 0 0 0 0 0 0 0 0
  There was a change in the policy environment 0 0 0 0 0 0 1 25 1 0.7
  There was no momentum for change in the agency or sector 0 0 0 0 0 0 0 0 0 0.0
  It was difficult to integrate the findings in a policy or programme 0 0 0 0 0 0 0 0 0 0
  Other (review was not completed in a timely way) 1 1.2 0 0 0 0 0 0 1 0.7
  Not stated (no barriers were mentioned) 80 95.2 25 100 25 96.2 3 75 133 88
  Not interviewed n/a n/a n/a n/a n/a n/a n/a n/a 11 7.3
  Total (all instances) 84 100 25 0 26 100 4 100 150 100
  1. aTotals > 150: details provided about use
  2. bTotals < 150: 82 uses separately identified