Table 2 Framework for evaluating priority-setting
Domains | Parameters of successful priority-setting | Objectively verifiable indicators | Means of verification |
|---|---|---|---|
Contextual factors | Conducive political, economic, social and cultural context | Relevant contextual factors that may impact priority-setting | Follow-up intermittent interviews with local stakeholders, systematic longitudinal observations, relevant reports, Media |
Pre-requisites | Political will | Degree to which the politicians support the set priorities | Follow-up intermittent interviews with local stakeholders, systematic longitudinal observations, relevant reports, Media |
Resources | Budgetary and human resource allocation to the health research | National budget documents | |
Legitimate and credible institutions | Degree to which the priority-setting institutions can set priorities, public confidence in the institution | Stakeholder and public interviews | |
Incentives | Material and financial incentives | National budget documents | |
The priority-setting process | Stakeholder participation | Number of stakeholders participating, number of opportunities, each stakeholder gets to express opinion | Observations/minutes at meetings, media reports, special reports |
Use of clear priority-setting process/tool/methods | Documented priority-setting process and/or use of priority-setting framework | Observation/minutes at meetings, media reports, special reports | |
Use of explicit relevant priority-setting criteria | Documented/articulated criteria | Observations/minutes at meetings, media reports, special reports | |
Use of evidence | Number of times available data is resourced/number of studies commissioned/existing strategies to collect relevant data | Observations/minutes at meetings, media reports, special reports | |
Reflection of public values | Number and type of members from the general public represented, how they are selected, number of times they get to express their opinion, proportion of decisions reflecting public values, documented strategy to enlist public values, number of studies commissioned to elicit public values | Observations/minutes at meetings, study reports, meeting minutes and strategic plans | |
Publicity of priorities and criteria | Number of times decisions and rationales appear in public documents | Media reports | |
Functional mechanisms for appealing the decision | Number of decisions appealed, number of decisions revised | Observations/minutes at meetings, media reports, special reports | |
Functional mechanisms for enforcement | Number of cases of failure to adhere to priority-setting process reported | Observations/minutes at meetings, media reports, special reports | |
Efficiency of the priority-setting process | Proportion of meeting time spent on priority-setting, number of decisions made on time | Observations/minutes at meetings, annual budget documents, health system reports | |
Implementation of the set priorities | Decreased dissentions | Number of complaints from stakeholder | Meeting minutes, media reports |
Allocation of research resources according to priorities | Degree of alignment of resource allocation and agreed-upon priorities, times budget is re-allocated from less prioritised to highly prioritised areas, stakeholder satisfaction with the decisions | Annual budget reports, evaluation documents | |
Decreased resource wastage/misallocation | Proportion of budget unused or allocated to non- priority research | Budget documents, research and evaluation reports | |
Improved internal accountability/reduced corruption | Number of publicised resource allocation decisions | Evaluation reports, stakeholder interviews, media reports | |
Increased stakeholder understanding, satisfaction and compliance with the priority-setting process | Number of stakeholders attending meetings, number of complaints from stakeholder, percentage of stakeholder that can articulate the concepts used in research priority-setting and appreciate the need for priority-setting | Observations/minutes at meetings, special reports, stakeholder satisfaction survey, media reports, stakeholder interviews, evaluation reports | |
Improved internal accountability/reduced corruption | Number of publicised resource allocation decisions | Evaluation reports, stakeholder interviews, media reports | |
Strengthening of the PS institution | Indicators relating to increased efficiency, use of data, quality of decisions and appropriate resource allocation, percentage of stakeholders with the capacity to set priorities | Training reports, evaluation reports, budget documents | |
Impact on research institution goals and objectives | Percentage of of research institution objectives met that are attributed to the priority-setting process | Evaluation reports, special studies | |
Outcome and impact | Impact on health policy and practice | Changes in health policy to reflect identified priorities | Policy documents |
Achievement of health system goals | Research contribution to achievement of health system goals | Ministry of Health documents, Demographic and Health Surveys, commissioned studies | |
Improved financial and political accountability | Number of publicised financial resource allocation decisions, number of corruption instances reported, percentage of the public reporting satisfaction with the process | Reports, media reports, interviews with stakeholders | |
Increased investment in the health sector and strengthening of the healthcare system | Proportion increase in the health research budget, percentage of the public/researchers reporting satisfaction with the health research system | National budget allocation documents |