Table 1 Summary of SEER scales, items developed for each domain, and scoring
Domains and factors measured by SEER scales | What the scale measures | Source of items | No. of items | Response options and scoring |
|---|---|---|---|---|
Capacity – predisposing factors | ||||
1. Value individual places on using research | Individual policymakers’ views on the value of research for informing each stage of policy work (e.g. deciding on policy content, designing evaluation) | New items were written for this scale because no suitable scales or items were identified | 7 | Five-point adjectival scale ranging from “not at all valuable” (score = 1) to “very valuable” (score = 5); scores are summed across items to create a scale score (range 7 to 35) |
2. Confidence in using research | Individual policymakers’ confidence in their ability (knowledge and skills) to engage with research (by accessing, appraising, generating and applying research) and researchers; items from these instruments were not suitable for measuring individual knowledge or skills | New items were written for this scale, informed by the concepts covered in two measures of organisational capacity (‘Is research working for you?’ [24, 55] and SUPPORT [2]; for analysis, see Additional file 1) | 7 | Five-point adjectival scale ranging from “not at all confident” (score = 1) to “very confident” (score = 5); scores are summed across items to create a scale score (range 7 to 35) |
3. Value organisation places on research use | Individual policymakers’ perceptions of leaders’ beliefs and organisational expectations about the use of research | New items were written for this scale, informed by the concepts measured by the Canadian Institutes of Health Research (CIHR) and SUPPORT instruments | 5 | Five-point adjectival scale ranging from “never” (score = 1) to “always” (score = 5); scores are summed across items to create a scale score (range 5 to 25) |
4. Tools and systems organisation has to support research use | Individual policymakers’ perceptions of the supports their organisation has in place for training, accessing research, guiding policy evaluation and research commissioning, and engaging with researchers | New items were written for this scale, informed by the CIHR and SUPPORT instruments | 7 | Four response options: ‘no’ (organisation does not have this tool or system) (score = 1), ‘yes, but limited’ (score = 2), ‘yes, well developed’ (score = 3), or ‘I don’t know’ (recoded as ‘no’, reflecting the interpretation that lack of awareness of support suggests a support that is not functional) Scores are summed across items to create a scale score (range 7 to 21) |
Research engagement actions | ||||
5. Accessed synthesised research | Whether individual policymakers searched for or commissioned reviews of research over the last 6 months; responses were in relation to the policy on which most time had been spent | New items were written for this scale | 2 | Binary response to individual items (yes/no) A ‘yes’ response to either or both items attracts the maximum score doing both actions (commissioning or searching for syntheses) is unlikely to be necessary |
6. Accessed primary research | Whether individual policymakers searched for single studies or government websites over the last 6 months; responses were in relation to the policy on which most time had been spent | New items were written for this scale | 2 | Binary response to individual items (yes/no) Items are summed to create a scale score (ordinal scale score: 0, 1, 2) |
7. Appraised research | Whether individual policymakers assessed the methods, reliability of results, and generalisability of research used to inform a specific policy over the last 6 months; responses were in relation to the policy on which most time had been spent | New items were written for this scale | 3 | Binary response to individual items (yes/no) Items are summed to create a scale score (ordinal scale score: 0, 1, 2, 3) Items are administered only if respondents answer ‘yes’ to an item asking if they found research |
8. Generated research | Whether individual policymakers generated research or analyses to inform a specific policy through an internally conducted project, commissioning or partnering with researchers, or evaluation of a policy or program; responses were in relation to the last 6 months and the policy on which most time had been spent | One item was adapted from Campbell et al.’s [11] five item scale measuring links with researchers and two new items were written | 3 | Binary response to individual items (yes/no) A ‘yes’ response to one or more items attracts the maximum scale score because undertaking one of the three actions is sufficient |
9. Interacted with researchers | The extent to which individual policymakers contributed to academic research through collaboration, advisory roles or attending research fora; responses were in relation to the last 6 months | Items were based on Campbell et al.’s [11] seven item scale measuring involvement in research; items were collapsed (e.g. combining ‘collaboration on research write up’ with ‘authorship of a research publication’) with minor rewording; one item was adapted from Campbell et al.’s ‘links with researchers’ scale | 6 | Responses are on a 4-point adjectival scale ranging from ‘not at all’ (score = 1) to ‘more than twice’ (score = 4); items are summed to create a scale score (range 6 to 24) |
Research use – extent of use | ||||
10. Extent of research use | Use of research in each stage of the policy development process (agenda setting/scoping, development, implementation, evaluation) over the last 6 months | New items were written for this scale | 4 | Responses are on a 6-point adjectival scale ranging from ‘none’ (score = 1) to ‘extensive’ (score = 6); a ‘not applicable’ option is provided for stages not yet addressed (e.g. for a policy at the scoping stage, items about extent of use of research in policy evaluation are not applicable) The highest score across the four items is taken as the measure of the extent of research use (range 1 to 6) |
Research use – type of use | ||||
11. Conceptual research use | Use of research to understand an issue over the last 6 months | A new item was written for this measure | 1 | Binary response to individual items (yes/no) |
12. Instrumental research use | Use of research to decide about content or direction of a policy or programme over the last 6 months | A new item was written for this measure | 1 | Binary response to individual items (yes/no) |
13. Tactical research use | Use of research to persuade others to a point of view or course of action over the last 6 months | A new item was written for this measure | 1 | Binary response to individual items (yes/no) |
14. Imposed research use | Use of research to meet organisational requirements over the last 6 months | A new item was written for this measure | 1 | Binary response to individual items (yes/no) |