Skip to main content
Download PDF
Download ePub

The SPARK Tool to prioritise questions for systematic reviews in health policy and systems research: development and initial validation

Health Research Policy and Systems volume 15, Article number: 77 (2017) Cite this article

Abstract

Background

Groups or institutions funding or conducting systematic reviews in health policy and systems research (HPSR) should prioritise topics according to the needs of policymakers and stakeholders. The aim of this study was to develop and validate a tool to prioritise questions for systematic reviews in HPSR.

Methods

We developed the tool following a four-step approach consisting of (1) the definition of the purpose and scope of tool, (2) item generation and reduction, (3) testing for content and face validity, (4) and pilot testing of the tool. The research team involved international experts in HPSR, systematic review methodology and tool development, led by the Center for Systematic Reviews on Health Policy and Systems Research (SPARK). We followed an inclusive approach in determining the final selection of items to allow customisation to the user’s needs.

Results

The purpose of the SPARK tool was to prioritise questions in HPSR in order to address them in systematic reviews. In the item generation and reduction phase, an extensive literature search yielded 40 relevant articles, which were reviewed by the research team to create a preliminary list of 19 candidate items for inclusion in the tool. As part of testing for content and face validity, input from international experts led to the refining, changing, merging and addition of new items, and to organisation of the tool into two modules. Following pilot testing, we finalised the tool, with 22 items organised in two modules – the first module including 13 items to be rated by policymakers and stakeholders, and the second including 9 items to be rated by systematic review teams. Users can customise the tool to their needs, by omitting items that may not be applicable to their settings. We also developed a user manual that provides guidance on how to use the SPARK tool, along with signaling questions.

Conclusion

We have developed and conducted initial validation of the SPARK tool to prioritise questions for systematic reviews in HPSR, along with a user manual. By aligning systematic review production to policy priorities, the tool will help support evidence-informed policymaking and reduce research waste. We invite others to contribute with additional real-life implementation of the tool.

Peer Review reports

Background

Health policy and systems research (HPSR) can strengthen health systems, drive progress towards universal health coverage and help deliver the promise of better health for all [1,2,3,4]. Evidence from HPSR can help inform critical health systems decisions, including who delivers health services and where and how these services are financed and organised [5,6,7]. It can also be used in the design and evaluation of innovative health system interventions that can help improve the quality of health services and reduce health inequities [8].

Systematic reviews of HPSR can be of great help to decision-makers as they constitute a more reliable and robust source of evidence than individual studies, particularly when the findings of the individual studies are complex or conflicting [9]. In addition to addressing the effectiveness of policy options under consideration, they can help clarify problems and their causes, and address implementation, resource use, acceptability, feasibility and impact on health equity [4, 10].

Groups or institutions funding or conducting systematic reviews in HPSR should prioritise topics according to the needs of policymakers and stakeholders [11, 12]. A prioritisation process can increase the likelihood that the best available evidence informs health policy decision-making [13, 14]. It can also promote optimal allocation of scarce resources in order to pursue the review questions that are likely to have a significant impact on knowledge, policy or practice [15]. In addition, a carefully-planned and inclusive priority setting process provides a platform for interaction and trust building among diverse stakeholders, both of which are important for the eventual uptake of research in decision-making [16, 17].

A number of tools and approaches have been published for the setting of research priorities [18, 19]. For example, Viergever et al. [20] developed a nine-item checklist that provides guidance on the planning of research prioritisation processes. However, these tools and approaches focus on setting priorities for health or clinical research in general, with none specific to systematic reviews or HPSR. Some of the limitations hindering their application to systematic reviews in HPSR include their disease-driven orientation, lack of transparency in the prioritisation process, inexplicit criteria for decision-making, and time-consuming nature due to involvement of multi-stage discussions or multiple iterations [18]. Importantly, when HPSR is considered through technical, disease-driven priority setting processes, it is systematically undervalued, thus contributing to fragmentation of health systems research [21].

A tool to prioritise review questions in HPSRs would address the abovementioned gap. In addition, it could help promote evidence-informed approaches to health system reforms which, in turn, could contribute to strengthened health systems and improved health outcomes [22]. Therefore, the aim of this study was to develop and validate a tool to prioritise questions for systematic reviews in HPSR.

Methods

General approach

We followed a standard approach for instrument development using the four steps described in the framework by Kirshner and Guyatt [23]:

  • ▪ Step 1: Definition of the purpose and scope;

  • ▪ Step 2: Item generation and reduction;

  • ▪ Step 3: Testing for content and face validity;

  • ▪ Step 4: Pilot-testing.

The project team included researchers with expertise in systematic review methodology, health policy and systems research, and research tool development. The project was led by the team of the Center for Systematic Reviews on Health Policy and Systems Research (SPARK) at the American University of Beirut. The Institutional Review Board at the American University of Beirut approved the project.

Specific steps

Step 1: Definition of the purpose and scope

The research team defined the purpose and scope of the tool based on internal discussions, and consultation with a purposive sample of policymakers and other stakeholders. The definition reflected the objective of the tool to prioritise questions for systematic reviews in HPSR.

Step 2: Item generation and reduction

For item generation, we conducted a literature review to capture any documents relevant to the objective of this project. We used the following combination of terms to search Medline and PubMed: (“priority setting” OR “priority-setting” OR “setting of priorit*”) AND (health). We initially ran the search in June 2014 followed by an updated search in March 2015. We also screened the reference lists of relevant articles identified through the search. The research team then abstracted from the identified literature all potentially relevant items for inclusion in the tool. For item reduction, the team members created a preliminary list of candidate items by removing obviously repetitive, redundant and irrelevant items. We followed an inclusive approach in determining the final selections of items to allow customisation to the user’s needs.

Step 3: Testing for content and face validity

In order to establish the content and face validity, we sought input from content experts on the clarity of the wording of items, the relevance of included items, the need to include additional items and the potential merging of items.

We sought input from three groups of content experts, as detailed below.

  • Group 1: International experts in the field. We shared the draft tool with six international experts in health policy and systems research and systematic review methodology. The draft tool contained the preliminary list of candidate items alongside an explanation for each item (Additional file 1). We asked participants to rate their agreement on a 5-point scale (1, strongly disagree, to 5, strongly agree) on whether or not each item should be retained in the tool. In addition, participants had the opportunity to suggest refinements and modifications to each of the items as well as nominate new items and suggest merging of items. We automatically retained items rated favourably by at least half of the participants. For the remaining items and for additional items nominated by participants, final decisions were made through consensus amongst the research team members.

  • Group 2: Participants in a workshop on prioritising questions for systematic reviews in health policy and systems research at the 22nd Cochrane Colloquium in Hyderabad, India. We grouped the revised items (generated from group 1) into four domains, namely problem, context, impact and technical, prior to administering the tool to participants. We divided participants into three focus groups, and asked each to pick two domains for discussion. Then, we asked participants to comment on the clarity and comprehensiveness of the items within the selected themes. Participants were then asked to reflect on the tool as a whole. Members of the research team took thorough notes of all the discussions.

  • Group 3: Participants in an interactive presentation on the tool at the Third Global Symposium on Health Systems Research held in Cape Town, South Africa. The same version of the tool used for group 2 was presented to this group. The presentation was followed by an open discussion about the tool and its components. The research team used the qualitative feedback from both groups 2 and 3 to refine some of the items.

Step 4: Pilot testing

As part of pilot testing, we pre-tested the revised tool through the interviewing of a purposive sample of three international experts in the field of evidence-informed policymaking, systematic review methodology and HPSR. We conducted semi-structured interviews following a brief guide developed by the team to elicit their input on the clarity, readability and comprehensiveness of the items and of the user manual. Then, we administered Module 1 of the revised tool to three policymakers (two from Lebanon and one from South Africa). We asked the policymakers to complete the module for two selected review questions (once for each review question). Finally, we asked them to reflect on the process.

We obtained final feedback on the general organisation of the tool and the wording of the items from two separate groups, (1) participants in a workshop on priority setting at the 2017 Cochrane Canada meeting and (2) participants in two consecutive webinars held by the Global Evidence Synthesis Initiative.

Results

In the next section, we present the findings of each of the four development steps as well as a description of the current version of the tool and the user manual.

Step 1: Definition of the purpose and scope

The tool is intended to prioritise questions of HPSR in order to address them in systematic reviews. HPSR is an multidisciplinary field of research that investigates issues such as how healthcare is financed, organised, delivered and used; how health policies are prioritised, developed and implemented; and how and why health systems do or do not achieve health and wider social goals [24].

Ideally, the tool is used during formal processes such as priority setting exercises. However, policymakers and stakeholders can also use it on an individual basis, e.g. when a formal process is not feasible. The tool needs to be used independently for each review question being considered for prioritisation.

Step 2: Item generation and reduction

We identified 40 relevant articles on previous priority setting exercises, priority setting approaches and guidelines on how to develop priority setting tools for research. Members of the research team with expertise in systematic review methodology, and in health policy and systems research, abstracted potentially relevant items from these 40 articles. Then, they reviewed these items and eliminated those that were obviously repetitive, redundant or unrelated to systematic reviews of HPSR. This created a preliminary list of 19 candidate items along with explanations of their meanings (Additional file 1).

Step 3: Testing for content and face validity

Group 1 involved 6 participants, group 2 involved 14 participants and group 3 involved more than 20 participants. Participants included academic health researchers, directors of research institutes/centres, systematic review methodologists, members of health professional associations and policymakers. Inputs from participants led to iterative refinements of the items and their wording.

Using the results of the quantitative and qualitative feedback from participants, the research team held a number of meetings and reached a consensus to:

  • Refine the wordings for some items, merge others and add new ones. This brought the number of items from 19 to 22. Additional file 2 shows the detailed changes made to the initial list of 19 items and to their meanings.

  • Split the tool into two modules. The first module includes items relevant to policymakers and stakeholders, while the second module includes items relevant to systematic review teams.

  • Convert the revised list of items into declarative statements. We opted for a 5-point scale with the following anchors: ‘strongly disagree’ (1), ‘disagree’ (2), ‘neither agree nor disagree’ (3), ‘agree’ (4), ‘strongly agree’ (5).

Step 4: Pilot testing

Based on the feedback from the three international experts and consultations among the research team, we refined and changed the wording for some of the items, merged two items into one and added one additional item, bringing the final number of items to 22 (Additional file 2). An average of 3 minutes was required to complete Module 1 of the tool for each review question.

The pilot testing confirmed the ease of use of the tool and its relevance in prioritising review questions. Participants in the pilot testing made suggestions for the rewording of a few items to enhance their clarity, but they did not suggest additional items. The pilot testing also revealed the need to assess the systematic review team’s available financial and human resources prior to the prioritisation process. This would subsequently inform the number of systematic reviews that the team can conduct, thus allowing them to establish a plan to translate the priorities to actual research.

Based on the final feedback on the tool, we developed signaling questions for each item in order to minimise variations in interpretation. We also reworded some of the items to improve clarity. The discussions highlighted the importance of keeping the use of the tool flexible in terms of what items to include or omit.

The SPARK tool

In the current version of the tool, the 22 items are organised in two modules. The first module includes 13 items relevant to policymakers and stakeholders, while the second module includes 9 items relevant to systematic review teams. The 22 items are presented in Box 1. The complete tool, along with the signaling questions, is presented in Additional file 3 as part of the user manual. Users can customise the tool to their needs by omitting items that may not be applicable to their settings.

Box 1 The 22 items included in the SPARK tool
Full size table

The user manual

The user manual is divided into five sections, namely (1) purpose of the SPARK tool, (2) components of the SPARK tool, (3) preparatory work, (4) using the SPARK tool, and (5) the SPARK tool (full version) (Additional file 3).

The recommended approach to administer the tool is for policymakers and stakeholders to complete Module 1 in order to rank questions according to their relevance. Module 2 is then applied to those relevant questions in order to rank them according to the feasibility and appropriateness of conducting a systematic review to address them. The order of administration can be reversed, for example, when there is a relatively large number of questions to prioritise and a time constraint for policymakers and stakeholders.

The use of the tool does not include assigning weights to each item or to each module. However, the technical team undertaking the prioritisation process may decide a priori on different weightings for different items or for the two respective modules. They may also define a threshold score in order to consider the review question a priority.

Discussion

In this article, we describe the development and initial validation of a tool to prioritise questions for systematic reviews in HPSR. The current version of the tool consists of 22 items, in two modules. The first module includes 13 items about question relevance (to be answered by policymakers and stakeholders). These items could also be framed around the problems when the questions have not been refined by the time of the priority setting exercise. The second module includes 9 items about the feasibility and appropriateness of conducting a review (to be answered by systematic review teams), typically only for those questions deemed relevant by policymakers and stakeholders. Users can customise the tool to their needs by omitting items that may not be applicable to their settings. We also developed a user manual that provides detailed guidance on how to use the SPARK tool, along with signaling questions. To our knowledge, this is the first tool designed for the prioritisation of questions for systematic reviews in HPSR.

Ideally, the use of Module 1 of the tool is performed in a group setting, where policymakers and stakeholders are physically together and can discuss the questions (with subsequent refinement/addition of new questions), rating them either individually or in a group. When it is not feasible to have all policymakers and stakeholders physically together, the rating can be performed individually (e.g. by email or using a web-based survey).

The use of the tool assumes the existence of a pool of potential questions (or problems) in need of prioritisation. Therefore, preparatory work might be needed to generate those questions (or problems). This can be in the form of a literature review, surveys and informal consultations with policymakers and stakeholders. In preparation for using Module 1, it would be useful to prepare brief vignettes containing background and contextual information on the problem being addressed by each question of interest and distribute these to policymakers [25]. Additionally, in preparation for using Module 2, it would be ideal to develop evidence maps of systematic reviews and of primary studies addressing the questions of interest [26]. The mapping of systematic reviews would help in avoiding duplication of efforts when a relevant, up to date, and of sufficiently high quality systematic review exists. The mapping of primary studies would help in avoiding questions that would result in empty systematic reviews.

As a key strength of this study, a multidisciplinary team developed and validated the tool following a standard methodology with the involvement of international experts in HPSR, systematic review methodology and tool development. We used a mix of surveys, qualitative interviews and feedback from international experts to enhance the validity of our findings. While some of the items may not be applicable to all settings, we attempted to address this by following an inclusive approach in determining the final selection of items to allow customisation to the user’s needs. Nonetheless, the tool could benefit from additional real-life testing in different contexts to enhance its generalisability. In fact, we are planning to use the tool in priority setting exercises to identify priority questions at both the national and regional level.

The SPARK tool will address the gap identified in the scientific literature on setting priorities for systematic reviews in the area of HPSR, as expressed by those involved in evidence synthesis in the field of HPSR [24]. In addition, the tool will support evidence-informed decision-making and practice by promoting the production of policy-relevant systematic reviews. It will also facilitate engaging policymakers and stakeholders in prioritising review questions [22].

Using this tool is particularly relevant in the context of low- and middle-income countries, where the capacity of production of systematic reviews is limited and often misaligned with policy needs and priorities [11, 27, 28]. The prioritisation can help channel limited resources to areas of highest priority [27, 29]. Furthermore, by assessing appropriateness of conducting systematic reviews, the tool contributes to global efforts to reduce research waste and avoid duplication of research efforts [30]. This could particularly resonate with funding organisations. For instance, as part of its efforts to minimise waste in research, the National Institute for Health Research requires systematic reviews of existing evidence as pre-requisite for any new research [31].

While using both modules of the tool is required to prioritise questions for systematic reviews, there are cases where one could use only one of the two modules. For example, one may opt to use Module 1 only to generate national research priorities regardless of the feasibility and appropriateness of conducting systematic reviews. Additionally, in the setting of guideline development, it could be used to inform the ‘priority setting’ domain in the guideline development checklist [32], and the ‘priority of the problem’ domain in the GRADE Evidence to Decision tables [33]. Similarly, Module 2 could be used to help decide on the feasibility of a systematic review, e.g. when deciding what questions to address in systematic review work based on the results of a mapping exercise [26].

Finally, it is worth noting that priority setting is just a first step in the knowledge framework [34]. Following a priority setting exercise, it is important to document the details of the prioritisation process to increase the credibility and thus the acceptability of the final products [20]. This should be followed up with evidence synthesis, knowledge translation activities and impact analysis [34], and will help with examining the degree to which the priorities have been addressed in research, as well as whether and how the research was used (or not) in decision-making [20, 34].

Conclusion

The SPARK tool for prioritising questions for systematic reviews in HPSR will address a gap in the scientific literature. We believe the tool will be useful for groups or institutions funding or conducting systematic reviews in HPSR. Additionally, it will help support evidence-informed policymaking and practice and reduce research waste by aligning systematic review production to policy priorities. We are currently experimenting with the tool at the SPARK Center. We encourage people involved in health systems and policy to use the tool and researchers in the field to conduct further testing within their own contexts as a contribution to refining the tool.

Abbreviations

GRADE:

Grading of Recommendations, Assessment, Development and Evaluations

HPSR:

Health policy and systems research

SPARK:

Center for Systematic Reviews in Health Policy and Systems Research

References

  1. World Health Organization. Changing Mindsets: Strategy on Health Policy and Systems Research. Geneva: WHO; 2012.

    Google Scholar 

  2. Hatt L, Johns B, Connor C, Meline M, Kukla M, Moat K. Impact of Health Systems Strengthening on Health; 2015. https://www.hfgproject.org/wp-content/uploads/2016/03/Impact-of-Health-Systems-Strengthening-on-Health-7-24-1.pdf. Accessed 31 Aug 2017.

  3. Chanda-Kapata P, Campbell S, Zarowsky C. Developing a national health research system: participatory approaches to legislative, institutional and networking dimensions in Zambia. Health Res Policy Syst. 2012;10:17.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA, Pielemeier NR, Mills A, Evans T. Overcoming health-systems constraints to achieve the Millennium Development Goals. Lancet. 2004;364(9437):900–6.

    Article  PubMed  Google Scholar 

  5. Alliance for Health Policy, Systems Research (AHPSR). Briefing Note Number 1: What Is Health Policy and Systems Research and Why Does It Matter? Geneva: WHO; 2007.

    Google Scholar 

  6. Gilson L, Hanson K, Sheikh K, Agyepong IA, Ssengooba F, Bennett S. Building the field of health policy and systems research: social science matters. PLoS Med. 2011;8(8):e1001079.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Koon AD, Rao KD, Tran NT, Ghaffar A. Embedding health policy and systems research into decision-making processes in low- and middle-income countries. Health Res Policy Syst. 2013;11:30.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Bennett S, Agyepong IA, Sheikh K, Hanson K, Ssengooba F, Gilson L. Building the field of health policy and systems research: an agenda for action. PLoS Med. 2011;8(8):e1001081.

    Article  PubMed  PubMed Central  Google Scholar 

  9. World Health Organization. Systematic Reviews in Health Policy and Systems Research; 2009. http://digicollection.org/hss/documents/s16867e/s16867e.pdf. Accessed 31 Aug 2017.

  10. Langlois EV, Ranson MK, Barnighausen T, Bosch-Capblanch X, Daniels K, El-Jardali F, Ghaffar A, Grimshaw J, Haines A, Lavis JN, et al. Advancing the field of health systems research synthesis. Syst Rev. 2015;4:90.

    Article  PubMed  PubMed Central  Google Scholar 

  11. El-Jardali F, Akl EA, Karroum LB, Kdouh O, Akik C, Fadlallah R, Hammoud R. Systematic reviews addressing identified health policy priorities in Eastern Mediterranean countries: a situational analysis. Health Res Policy Syst. 2014;12:48.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Fleurence RL, Torgerson DJ. Setting priorities for research. Health Policy. 2004;69(1):1–10.

    Article  PubMed  Google Scholar 

  13. Lavis JN, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP) 3: Setting priorities for supporting evidence-informed policymaking. Health Res Policy Syst. 2009;7 Suppl 1:S3.

    Article  PubMed  Google Scholar 

  14. Campbell S. Deliberative Priority Setting. A CIHR Knowledge Translation Module. Ottawa: Canadian Institutes for Health Research; 2010.

    Google Scholar 

  15. Kok MO, Gyapong JO, Wolffers I, Ofori-Adjei D, Ruitenberg J. Which health research gets used and why? An empirical analysis of 30 cases. Health Res Policy Syst. 2016;14:36.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Mador RL, Kornas K, Simard A, Haroun V. Using the Nine Common Themes of Good Practice checklist as a tool for evaluating the research priority setting process of a provincial research and program evaluation program. Health Res Policy Syst. 2016;14:22.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Cole DC, Nyirenda LJ, Fazal N, Bates I. Implementing a national health research for development platform in a low-income country - a review of Malawi’s Health Research Capacity Strengthening Initiative. Health Res Policy Syst. 2016;14:24.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21(st) century. J Glob Health. 2016;6(1):010507.

    PubMed  Google Scholar 

  19. The Collaborative Training Program. Health Research for Policy, Action and Practice. Resource Modules. Module III: Promoting the use of knowledge in policy and practice, Version 2. 2004. http://www.who.int/alliance-hpsr/resources/ModuleIII_U2_CommunityV2.pdf?ua=1. Accessed 31 Aug 2017.

  20. Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst. 2010;8:36.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Ranson MK, Bennett SC. Priority setting and health policy and systems research. Health Res Policy Syst. 2009;7:27.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Langlois EV, Becerril Montekio V, Young T, Song K, Alcalde-Rabanal J, Tran N. Enhancing evidence informed policymaking in complex health systems: lessons from multi-site collaborative approaches. Health Res Policy Syst. 2016;14:20.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chronic Dis. 1985;38(1):27–36.

    Article  CAS  PubMed  Google Scholar 

  24. Gilson L. Health Policy and System Research – A Methodology Reader: The Abridged Version. Geneva: Alliance for Health Policy and Systems Research, WHO; 2013.

  25. Bryant J, Sanson-Fisher R, Walsh J, Stewart J. Health research priority setting in selected high income countries: a narrative review of methods used and recommendations for future practice. Cost Eff Resour Alloc. 2014;12:23.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Schmucker C, Motschall E, Antes G, Meerpohl JJ. Methods of evidence mapping. A systematic review. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2013;56(10):1390–7.

    Article  CAS  PubMed  Google Scholar 

  27. Law T, Lavis J, Hamandi A, Cheung A, El-Jardali F. Climate for evidence-informed health systems: a profile of systematic review production in 41 low- and middle-income countries, 1996-2008. J Health Serv Res Policy. 2012;17(1):4–10.

    Article  PubMed  Google Scholar 

  28. Oliver S, Bangpan M, Stansfield C, Stewart R. Capacity for conducting systematic reviews in low- and middle-income countries: a rapid appraisal. Health Res Policy Syst. 2015;13:23.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Uneke CJ, Ezeoha AE, Ndukwe CD, Oyibo PG, Onwe F, Aulakh BK. Research priority setting for health policy and health systems strengthening in Nigeria: the policymakers and stakeholders perspective and involvement. Pan Afr Med J. 2013;16:10.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gulmezoglu AM, Howells DW, Ioannidis JP, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65.

    Article  PubMed  Google Scholar 

  31. Nasser M, Clarke M, Chalmers I, Brurberg KG, Nykvist H, Lund H, Glasziou P. What are funders doing to minimise waste in research? Lancet. 2017;389(10073):1006–7.

    Article  PubMed  Google Scholar 

  32. Oxman AD, Schunemann HJ, Fretheim A. Improving the use of research evidence in guideline development: 2. Priority setting Health Res Policy Syst. 2006;4:14.

    Article  PubMed  Google Scholar 

  33. Alonso-Coello P, Schunemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Rada G, Rosenbaum S, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016.

    Article  PubMed  Google Scholar 

  34. El-Jardali F, Fadlallah R. A call for a backward design to knowledge translation. Int J Health Policy Manag. 2015;4(1):1–5.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We would like to thank all the participants who provided input on our tool.

Funding

This study was supported by the Alliance for Health Policy and Systems Research, WHO, Geneva. Although one of the authors (EL) is employed by the funder, the funder was not involved in the design of the study and collection, analysis, and interpretation of data and in writing of the manuscript.

Availability of data and materials

All data generated or analysed during this study are included in this published article and its Additional files.

Author information

Authors and Affiliations

  1. Department of Internal Medicine, American University of Beirut, Beirut, Lebanon

    Elie A. Akl

  2. Center for Systematic Reviews of Health Policy and Systems Research (SPARK), American University of Beirut, Beirut, Lebanon

    Elie A. Akl, Racha Fadlallah & Fadi El-Jardali

  3. Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada

    Elie A. Akl, John N. Lavis, Holger Schünemann & Fadi El-Jardali

  4. Department of Health Management and Policy, Faculty of Health Sciences, American University of Beirut, Beirut, Lebanon

    Racha Fadlallah & Fadi El-Jardali

  5. Knowledge to Policy (K2P) Center, American University of Beirut, Beirut, Lebanon

    Racha Fadlallah & Fadi El-Jardali

  6. Department of Clinical Epidemiology and Biostatistics, Faculty of Health Sciences, American University of Beirut, Beirut, Lebanon

    Lilian Ghandour

  7. Primary Healthcare Department at the Ministry of Public Health, Beirut, Lebanon

    Ola Kdouh

  8. Alliance for Health Policy and Systems Research, World Health Organization, Avenue Appia 20, 1211, Geneva, Switzerland

    Etienne Langlois

  9. McMaster Health Forum, McMaster University, Hamilton, ON, Canada

    John N. Lavis

  10. Centre for Health Economics and Policy Analysis, McMaster University, Hamilton, ON, Canada

    John N. Lavis

  11. Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, MA, United States of America

    John N. Lavis

  12. McMaster GRADE Centre and Department of Medicine, McMaster University, Hamilton, ON, Canada

    Holger Schünemann

Authors
  1. Elie A. Akl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Racha Fadlallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lilian Ghandour
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ola Kdouh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Etienne Langlois
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. John N. Lavis
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Holger Schünemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Fadi El-Jardali
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

EAA and FE contributed to conception and design, acquisition of data, analysis and interpretation of data, and drafting of the manuscript. RF contributed to design, acquisition of data, analysis and interpretation of data, and drafting and finalising the manuscript. OK contributed to design, acquisition of data, and analysis of data. LG contributed to interpretation of data and initial drafting of the manuscript. EL, JL and HS contributed to interpretation of data, and critical revision of the manuscript for important intellectual content. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Fadi El-Jardali.

Ethics declarations

This project was approved by the Institutional Review Board at the American University of Beirut, Lebanon.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Preliminary list of 19 candidate items along with their meanings. (PDF 90 kb)

Additional file 2:

Iterative refinements of the items and their wording through the development and validation process. (PDF 196 kb)

Additional file 3:

User manual for the SPARK tool. (PDF 511 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Akl, E.A., Fadlallah, R., Ghandour, L. et al. The SPARK Tool to prioritise questions for systematic reviews in health policy and systems research: development and initial validation. Health Res Policy Sys 15, 77 (2017). https://doi.org/10.1186/s12961-017-0242-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12961-017-0242-4

Keywords

Health Research Policy and Systems

ISSN: 1478-4505

Contact us