The quest for a framework for sustainable and institutionalised priority-setting for health research in a low-resource setting: the case of Zambia

Abstract

Background

Priority-setting for health research in low-income countries remains a major challenge. While there have been efforts to systematise and improve the processes, most of the initiatives have ended up being a one-off exercise and are yet to be institutionalised. This could, in part, be attributed to the limited capacity for the priority-setting institutions to identify and fund their own research priorities, since most of the priority-setting initiatives are driven by experts. This paper reports findings from a pilot project whose aim was to develop a systematic process to identify components of a locally desirable and feasible health research priority-setting approach and to contribute to capacity strengthening for the Zambia National Health Research Authority.

Methods

Synthesis of the current literature on the approaches to health research prioritisations. The results of the synthesis were presented and discussed with a sample of Zambian researchers and decision-makers who are involved in health research priority-setting. The ultimate aim was for them to explore the different approaches available for guiding health research priority-setting and to identify an approach that would be relevant and feasible to implement and sustain within the Zambian context.

Results

Based on the evidence that was presented, the participants were unable to identify one approach that met the criteria. They identified attributes from the different approaches that they thought would be most appropriate and proposed a process that they deemed feasible within the Zambian context.

Conclusion

While it is easier to implement prioritisation based on one approach that the initiator might be interested in, researchers interested in capacity-building for health research priority-setting organisations should expose the low-income country participants to all approaches. Researchers ought to be aware that sometimes one shoe may not fit all, as in the case of Zambia, instead of choosing one approach, the stakeholders may select desirable attributes from the different approaches and piece together an approach that would be feasible and acceptable within their context. An approach that builds on the decision-makers’ understanding of their contexts and their input to its development would foster local ownership and has a greater potential for sustainability.

Background

The limited resources available for global health research require that research priorities are set to identify the topics or issues that most urgently require study. Since setting priorities is ultimately about resource allocation, it often becomes a highly politicised process. This is more so in low-income countries (LICs), where the lack of resources to fund health research introduces non-state stakeholders whose research agendas may not necessarily be aligned with the national priorities. This makes it critical for LICs to strengthen capacity and increase funding for their national health research systems to enable them to set and fund national health priorities based on systematic processes using clear approaches with explicit criteria [1,2,3,4,5].

To facilitate priority-setting (PS) for health research, several frameworks have been developed. These include the Combined Approach Matrix (CAM) [6], the Essential National Health Research (ENHR) method [7], and the Child Health and Nutrition Research (CHNRI) approach [8, 9]. Further, two frameworks which have been used predominantly in high-income countries include Listening for Direction (L4D) [10] and the James Lind Alliance (JLA) method [11].

In addition to developing frameworks for PS in health research, there is an increasing interest in developing and strengthening the health research infrastructure by introducing national health research organisations/authorities and systems ((NHROs) in LICs; these should be responsible for, among other duties, setting the national health research priorities. This is because of the realisation that localised PS can contribute to ensuring that the research priorities are aligned with the local needs, increase the likelihood that the research findings will be used, facilitate dialogue between researchers, research funding organisations, policy-makers and implementers, and prevent duplicity of research projects and hence resource wastage. This also provides an opportunity for aligning PS to existing national planning systems and infrastructure [4, 5].

To date, several LICs have instituted NHROs [12]. However, not many of these have been sustainable, possibly due to lack of capacity and financial support [13]. Capacity-building for NHROs involves both the institutionalisation and the integration of the National Health Research Agenda in the national health policies, in addition to ensuring that resources are available for implementing the identified research priorities [14]. One way through which the NHROs’ capacities could be strengthened is by facilitating South–South collaboration, whereby LIC NHROs can harness, synthesise and share their experiences, identifying the limitations of the processes and devising improvement strategies. There are opportunities for countries currently developing their own NHRO to learn from the experiences of countries that have already established NHROs. For example, researchers from South Africa, a country with a long standing and as well as a well-functioning NHRO, could share their experiences with countries such as Zambia, which are in the process of developing their NHRO [15].

Zambia established the National Health Research Advisory Committee in 2005 to provide ad hoc support to the Zambian Ministry of Health on all matters relating to health research. In November 2008, the Committee began a participatory process of creating a new national institution designed to govern health research in the country [16]. In 2010, the Zambian government approved the National Health Research Policy; one of its recommendations was the formation of a national body to coordinate health research. By 2013, the government passed legislation providing a legal framework for the creation of the National Health Research Authority (NHRA), an entity designed to coordinate, fund and provide ethical oversight for health research, in addition to offering capacity strengthening to all stakeholders involved in research, advocating for the role of research in the health policy process and conducting PS exercises [17]. The acting co-ordinator of the temporary research authority identified the need for capacity strengthening within their institution; they thought it critical that there is robust support for this new entity to carry out its mandate. A critical step in ensuring the success of the NHRA is to ensure that the stakeholders involved have the necessary resources and the capacity to credibly set priorities for health research.

Capacity-building and availability of resources are critical to ensuring the sustainability and legitimacy of any priority-setting institution [18]. Capacity-building should involve members of an organisation that have the knowledge and ability to apply the available approaches/frameworks and evidence to their PS. However, while there are many resources devoted to facilitating health research PS (HRPS) in LICs, these resources are often not easily available or accessible to the people tasked with implementation. In Zambia, for example, where a systematic PS process was project based and led by consultants or international experts, resource constraints (including time) did not allow for sustained capacity strengthening for the NHROs [19]. In prioritisation processes that have not followed the COHRED process, only one PS approach is explained in detail and applied, for example, in the case of CHNRI applications [9]. This does not allow for the stakeholders involved to learn about other approaches. This lack of capacity-building makes it difficult for the NHRO to independently set health research priorities without the assistance of external experts. We argue that the tendency to depend on the external (and often expensive) experts to facilitate the process results in a vicious cycle of dependence, an inability to set research priorities if the resources are limited to hire the expert and a lack of sustainability of the activities of the NHROs.

There is a paucity of literature that addresses capacity-building for HRPS in LICs. While studies identify limited capacity as one of the main weaknesses of NHROs in LICs, existing capacity strengthening efforts have not involved empowering the NHROs with the necessary evidence enabling them to critically evaluate the available frameworks and assess what aspects of the approaches would be applicable, acceptable and feasible within their contexts. In this paper, we argue that the sustainability of NHROs in LICs will, in part, be facilitated by ensuring that the local technical staff in the NHRO in LICs have the necessary capacity to set health research priorities. Part of the capacity-building process should involve educating NHROs (and the relevant stakeholders) on all the available HRPS approaches, where they have been used and their strengths and limitations. This would enable NHROs to critically consider all the PS approaches for health research and to choose the most appropriate approach (or a hybrid of desired components from the different approaches).

Such an inclusive and comprehensive approach to capacity-building will contribute to ensuring ownership of the PS process, therefore facilitating a sustained application of the approach that emerges by stakeholders; such a method would also facilitate the institutionalisation of the chosen approach.

This paper reports findings from a pilot project in Zambia whose aim was to develop a systematic process for identifying components of a locally desirable and feasible HRPS approach and to contribute to capacity strengthening for the Zambia NHRA.

Methods

This was a qualitative study involving literature review and systematic facilitated workshop discussions.

Literature review

LK, a consultant and a research assistant participated in retrieving, reviewing and synthesising grey and published literature (at the global level) on the most commonly used HRPS approaches and their application from 1997 to 2014. The published literature was accessed using PubMed and Ovid databases (including Embase, Global Health, Medline, PsycINFO). The main search terms included different combinations of ‘priority setting’, ‘approaches’, ‘frameworks’ ‘health research’, ‘low income countries’ and ‘developing countries’.

Step 2

A total of 1005 hits were derived from the different combinations of our search terms; these were initially scanned based on their titles to exclude papers that did not fit our criteria, e.g. those that did not address health research. Once we had identified all papers corresponding to our search strings and our date range, we applied the following inclusion and exclusion criteria in order to arrive at a shorter list of abstracts to review.

Inclusion criteria

English language papers that appeared to describe a PS for health research framework¹ or a case-study application of a PS for health research framework in any context (e.g. in a high-income country, middle-income country or LIC) or specific elements (e.g. stakeholder engagement, criteria generation, evaluation) stemming from either a PS for health research framework or a case study application were included in the study.

Exclusion criteria

Papers that appeared to focus on PS for health interventions, describe a PS exercise that did not use an explicit PS framework for guidance or were in other languages were excluded.

Step 3

A total of 108 papers met the inclusion criteria and were included in our literature review (a full reference list is included as Additional file 1). We then devised a ‘literature review framework’ that assigns the literature into one of three different categories, posing specific questions about the literature to help identify important trends, strengths and weaknesses. Table 1 below shows the three categories, the numbers of papers selected for full review (by category) and then a selection of the questions used to explore each literature category.

Table 1 Number of papers included in the literature by category

This literature was supplemented with information from the websites of the organisation (such as COHRED, CHNRI, JLA) that developed the HRPS frameworks. PK and the research assistant in Zambia accessed the grey literature on HRPS within sub-Saharan Africa.

We summarised this information according to the above themes, as follows: What are the components of the framework? How was it developed and/or introduced? Where has the framework been used? What are the experiences with using the framework? What are the documented strengths and limitations of the framework?

Validation of the synthesis

To formally validate our interpretation of the literature, the literature synthesis reports were sent to the different experts who developed and applied the frameworks. For the purposes of this study, we contacted experts from CAM (n = 1), CHNRI (n = 2), ENHR (n = 1), JLA (n = 2), and L4D (n = 1), specifically requesting them to assess whether our presentation of their framework was appropriate and to assess if we had missed any key literature related to their framework. We received responses from all experts with the exception of L4D. Their feedback and comments were used to edit the literature review report that was used in facilitating the capacity-building workshops.

Capacity-building workshops

Two capacity-building workshops with different groups of stakeholders from Zambia were convened in July and November 2015. Both workshops involved researchers and policy-makers from the Zambia Ministry of Health. The second workshop also involved the board members of the Zambia NHRO and local researchers. The researcher participants are co-investigators on this research project. They identified policy-makers who were relevant to HRPS in Zambia.

Workshop I

The purposes of the workshop were to strengthen participants’ capacity for HRPS and to identify the desired PS attributes and processes that are best suited for implementation and sustainability within the Zambian context.

The process – capacity strengthening: participants included three Ministry of Health officials from Zambia (who were also researchers) and four Canadian researchers. The approach was participatory, with shared responsibilities from both Zambian and Canadian researchers. LK led the first part of the meeting by asking participants about their experience with HRPS and their knowledge of HRPS frameworks. The Zambian participants reported and described a previous PS process in which they had participated. However, since they were only introduced to the CHNRI methodology, their understanding of the other HRPS frameworks was still limited. Hence, the first step in the capacity-strengthening process was to ensure that all workshop participants understood the commonly used HRPS frameworks, and the available evidence on their application.

• Step 1:   LK presented descriptions of the five most common PS approaches in the literature – CAM, CHNRI, ENHR, JLA and L4D – and discussed the components and theoretical underpinnings of the HRPS approaches.

• Step 2:   Based on Step 1, and using flip charts with the names of the different approaches, each participants was asked to independently indicate, on the respective flip charts, the perceived strengths and limitations for each framework (as in Table 1).

• Step 3:   LK and PK presented and discussed the strengths and limitations of the approaches that were identified in the literature.

• Step 4:   The Zambian participants identified the desired attributes and processes for HRPS in the Zambian context, based on the discussion in steps 2 and 3. Each attribute of each framework was discussed based on the following questions: would this [attribute or process] be desirable in the Zambian context? Would it be feasible? If not, would it require modification? The output of this process was a desired approach to HRPS and the key related attributes (summarised in Table 3). The responses to the questions were summarised and presented in the Results section.

The meeting proceedings were audio recorded with permission from all participants.

Workshop II

The second workshop was convened in Zambia and facilitated by the Zambian investigators who had participated in the first workshop. It involved six senior officers from the Ministry of Health in Zambia (two of whom were also researchers), and six university researchers who were identified by the Zambian co-investigator by virtue of their involvement in the Zambian NHRO. Most had also participated in the CHNRI PS exercise and therefore had prior experience with participatory decision-making and were also familiar with HRPS. The objective of this meeting was to validate the PS approach that was developed during the first workshop.

Prior to the workshop, all participants were sent invitation letters with an information package containing the workshop objectives and the proposed framework. They were requested to independently reflect on the different components of the proposed framework, and if they were indeed desirable within the Zambian context.

During the 1-day face-to-face meeting, after self-introductions (where participants also explained their knowledge of HRPS), the facilitators presented the objectives of the workshop. This was followed by a facilitated review of the proposed PS process for Zambia. Next, the participants were split into three groups and were requested to (1) review the proposed framework, step by step; (2) propose improvements or changes; and (3) present the main findings of their peer-review to all participants. The recommendations from each group were reviewed and consensus was sought on the final recommendations to be included in the proposed PS process for Zambia.

This study received ethics clearance from both Zambia and McMaster Research ethics boards. We obtained consent from all the group participants, and all findings were anonymised in the recording and reporting.

Results

Strengths of the five PS approaches

Table 2 summarises the strengths of the commonly used approaches from the literature and as perceived by the workshop participants. The most commonly identified strengths (in the literature and by the workshop participants) included the clarity and simplicity of the approach, involvement of a wide range of stakeholders, and explicit processes and criteria.

Table 2 Summary of the strengths of the five priority-setting frameworks

According to the workshop participants, clear, simple, participatory HRPS approaches that involve vulnerable populations and facilitate participants’ understanding by providing them with evidence on which to base their decision-making would be the easiest to adopt within a LIC context. Participants reasoned that the people charged with setting priorities, within this context, often lack the necessary capacity and time to understand complex PS approaches. Therefore, the adoption of clear and simple PS approaches, with step-by-step instructions and clear criteria would reduce the need for experts, promoting local ownership.

Participants also strongly supported participatory PS approaches (a common theme in all the original five PS approaches). There was, however, more support for the method that fostered participation of vulnerable populations who would not normally participate in prioritisation. Hence, the method used in the JLA framework, which focuses on involving patient groups (who were perceived to be vulnerable) in the PS process was greatly desired, more so since they provide the participants with information for decision-making [20]. Most of the above strengths were consistent with those identified in the literature [21,22,23,24,25,26,27,28,29].

Another strength which was identified by participants as well as the literature was the ease with which a HRPS approach could be aligned with the existing national health system planning processes and infrastructure [5]; this would increase the potential for the process to be sustainable and institutionalised. In this regard, the CAM approach, which recommends multi-sectoral types of evidence at different levels of decision-making (e.g. community, regional, national) was deemed valuable and in line with the national development planning process in Zambia. Furthermore, approaches such as L4D, which facilitate decentralised decision-making by considering priorities from sub-national levels, were perceived favourably since they are aligned with already existing processes and infrastructure and would be easier to adopt and sustain.

Limitations of the five PS approaches

Table 3 summarises the limitations of the five approaches as identified in the literature review and by workshop participants. There were also similarities between the limitations identified in the literature and the limitations perceived by workshop participants. However, during the workshops, most of the discussion dwelt on the limitations related to the practical applicability (feasibility) and sustainability of using the HRPS approach within LICs such as Zambia.

Table 3 Summary of the limitations of the commonly used priority-setting approaches

According to workshop participants, from a practical perspective, the main limitation of the approaches was the potential difficultly in integrating them into the current health system’s planning cycle and infrastructure.

Workshop participants also discussed how approaches that tend to be resource intensive would not be sustainable, given the resource constraints in LICs. For example, while the collection of evidence from different sectors and at different levels (recommended by CAM) was perceived as a strength, the resources that are required to collect all this information may be prohibitive, especially in LICs that lack an established information system [6, 20]. This would limit the feasibility of the HRPS approach and the potential for it to be sustained and institutionalised.

Furthermore, in view of the decentralisation of the health sector, participants thought it was important that HRPS approaches should reflect this by also facilitating decentralised decision-making at sub-national levels. Therefore, approaches that did not provide case studies of sub-national level application were perceived less favourably by the participants.

Although desired, workshop participants observed that most of the HRPS approaches (with the exception of ENHR) lacked a systematic strategy for (1) evaluating the PS process (immediately after setting priorities to gauge if stakeholders were satisfied with the process); (2) monitoring and evaluating the implementation of the set priorities and; (3) enforcing the actual allocation of funds to the identified research priorities. These are key aspects of HRPS, especially in contexts where resources are limited; it is counterintuitive to fail to evaluate the impact and outcomes of HRPS when valuable resources are committed.

The HRPS approach proposed and validated by the participants

Based on our discussion of the strengths and limitations of the various HRPS approaches, we present a synthesis of the attributes and processes that were deemed desirable and could guide HRPS in Zambia (and potentially other similar LICs) (Table 4). This synthesis is presented in three phases, namely (1) the preparatory phase, including all the activities that must occur before the HRPS process begins; (2) identification of the priorities, namely how a group of stakeholders will arrive at a list of priorities, and the exact type of priorities to be determined (e.g. research questions, research themes, topics, options, or issues etc.); and (3) after PS, namely monitoring, evaluation, funding, enforcement, etc.

Table 4 A synthesis of the desired features and process

Discussion

Contrary to approaches that have already been captured in the literature, this paper presents a process that predominantly included stakeholders who are involved in HRPS in Zambia. Therefore, the discussions and findings focused on contextualising the attributes with the purpose of identifying those that would be sustainable and easily institutionalised within the Zambian context as well as, potentially, in other similar LIC contexts. This paper presents findings from a participatory capacity-building process to identify an acceptable approach to HRPS in Zambia (and other similar LICs). While many of the proposed activities are consistent with those identified in the literature [30, 31], our systematic approach deliberately included the ‘voices’ of the potential users of the approach in the process of its development. These nuances are discussed below.

For example, in the preparation phase, participants emphasised the importance of identifying the ‘right’ level of PS. This is especially relevant in contexts with decentralised systems, as is the case in most LICs. To date, most of the HRPS has been focused at the national level [5,6,7,8,9, 32,33,34,35]; however, capacity-building and equitable participation would require that stakeholders from the regions and districts be involved in this process, as was proposed by the workshop participants. This is especially important given that a great deal of health research is conducted within their jurisdictions. The other underlying feature of this process is the emphasis on capacity-building that would involve empowering the institution to ‘own’ the PS process. While COHRED has worked to strengthen NHROs, several of the PS processes for health research that have taken place in sub-Saharan Africa did not demonstrate clear linkages with the NHROs [36, 37]. This not only undermines the credibility of these organisations, it presents a missed opportunity for capacity-building. There is a strong need to support these organisations by ensuring that they take the lead in all PS exercises within their countries. In order to succeed, PS should be part of a broad research management strategy, which should be championed by the NHROS with support from the health and other related ministries.

In the implementation phase, there is no recommendation for a specific PS approach. This is a deliberate omission on our part because we think different approaches would work better for different PS situations. For example, the JLA approach could be the best option for determining treatment-related priorities [38], while CHNRI may prove superior in determining national-level priorities within various research domains, from research priorities within health systems to priorities relating to specific diseases [39]. However, since the current practice has been that the PS facilitators determine and recommend an approach, we propose that, in order to facilitate capacity-building and empower the NHROs, the decision on the framework/approach should be made collectively after all the current approaches have been presented and discussed by the stakeholders, as we demonstrated in the pilot workshops. However, we highlight the need to facilitate an understanding of key considerations when selecting an approach such as feasibility and the potential for integration into current health planning infrastructure during the capacity-building process. We also highlight the need to contextualise the criteria for ranking priorities in order to reflect local values. The HRPS approaches should be flexible so that they can accommodate any locally prioritised values, which may not be included in the standard approaches.

In line with some of the current literature, the proposed approach also emphasises the need for monitoring and evaluation [31]. However, since we also describe the need to build capacity for PS, we discuss the importance of participatory evaluation of the PS process. Preferably, evaluation should be integrated throughout the process and be conducted with the stakeholders involved in PS immediately after the process, as well as a year or two later. An evaluation of the PS process ensures that there is continuous learning about and improvement of the PS process, which will improve and sustain commitment, stakeholder participation and the ongoing institutionalisation of the process. Furthermore, the PS process should be evaluated since it is a waste of scarce resources to invest resources in PS but not to invest in ensuring that the set priorities are actually implemented [18]. Efforts should be devoted to ensuring that there are resources available to fund at least the top ten priorities. There should be active advocacy by the NHRO to ensure that research funders support the research priorities identified through the national PS process. Involvement of research funders in the PS process – from the outset – would greatly facilitate their support.

Evaluation necessitates a standardised form of reporting and explicit parameters. The modified version of the evaluation framework proposed by Kapiriri and Martin [18] could assist with this standardisation. This framework, although developed for evaluating PS for health interventions, has been modified and validated by LIC researchers, and could serve as a resource for stakeholders who want to comprehensively evaluate their HRPS.

Limitations

Almost all participants were from Zambia, which could limit the applicability of the proposed approach elsewhere, although the similarity of some of the countries in the region may make it applicable.

Conclusions

This pilot study, conducted in Zambia, provides a summary of the desired attributes for a PS process for health research in LICs. While some of the identified attributes of successful HRPS are consistent with the PS literature, we identified some unique attributes for further discussion. These include a focus on capacity-building by making sure that every PS process starts with introducing the research managers and stakeholders directly involved in the prioritisation process to all existing frameworks, thus ensuring understanding of all HRPS possibilities before choosing the approach to be used. Such an approach not only ensures capacity-building, but cultivates a sense of ownership of the process in LIC stakeholders. Furthermore, the emphasis on seeking a process that is well aligned with existing infrastructure is critical as it ensures sustainability and contributes to the institutionalisation of the PS process.

To date, most of the literature on PS for health research in LICs describes predominantly top-down, one-off, externally-driven exercises. Processes such as the one proposed in this paper, which facilitate local capacity-building, may contribute to ensuring sustainability and institutionalisation of the PS process. There is a need for case studies where such participatory capacity-building approaches are implemented and evaluated.