Identifying and characterising health policy and system-relevant documents in Uganda: a scoping review to develop a framework for the development of a one-stop shop

Abstract

Background

Health policymakers in low- and middle-income countries continue to face difficulties in accessing and using research evidence for decision-making. This study aimed to identify and provide a refined categorisation of the policy documents necessary for building the content of a one-stop shop for documents relevant to health policy and systems in Uganda. The on-line resource is to facilitate timely access to well-packaged evidence for decision-making.

Methods

We conducted a scoping review of Uganda-specific, health policy, and systems-relevant documents produced between 2000 and 2014. Our methods borrowed heavily from the 2005 Arksey and O’Malley approach for scoping reviews and involved five steps, which that include identification of the research question; identification of relevant documents; screening and selection of the documents; charting of the data; and collating, summarising and reporting results. We searched for the documents from websites of relevant government institutions, non-governmental organisations, health professional councils and associations, religious medical bureaus and research networks. We presented the review findings as numerical analyses of the volume and nature of documents and trends over time in the form of tables and charts.

Results

We identified a total of 265 documents including policies, strategies, plans, guidelines, rapid response summaries, evidence briefs for policy, and dialogue reports. The top three clusters of national priority areas addressed in the documents were governance, coordination, monitoring and evaluation (28%); disease prevention, mitigation, and control (23%); and health education, promotion, environmental health and nutrition (15%). The least addressed were curative, palliative care, rehabilitative services and health infrastructure, each addressed in three documents (1%), and early childhood development in one document. The volume of documents increased over the past 15 years; however, the distribution of the different document types over time has not been uniform.

Conclusion

The review findings are necessary for mobilising and packaging the local policy-relevant documents in Uganda in a one-stop shop; where policymakers could easily access them to address pressing questions about the health system and interventions. The different types of available documents and the national priority areas covered provide a good basis for building and organising the content in a meaningful way for the resource.

Background

Linking health research evidence to action is one of the many important components of national and global responses to contemporary public health challenges [1, 2]. It is important for both making evidence-informed policies¹ and decisions² on health services and improving the health systems within which the interventions and services are provided [3]. When policies and decisions are made without considering the best available evidence, it may waste resources and opportunities, and possibly do more harm than good [4]. In the recent past, there has been a strong emphasis worldwide on reflecting the best available evidence in health policies and decisions [2, 5]. However, more evidence may not necessarily mean better policies [6, 7]. Greenhalgh et al. [7] notes that a better policy is what is appropriate in the circumstances in agreement with the overall desirable goal. Research evidence is one of the necessary inputs into policymaking, which may also be influenced by context and other factors [8].

Linking research to action in low- and middle-income countries has remained a challenge, despite the international attention and significant efforts to address it [3, 9]. Health policymakers³ and stakeholders⁴ continue to face difficulties in accessing and using research evidence for policy and decision-making [10]. They are often unable to rapidly identify decision-relevant information when pressing issues emerge, partly due to the lack of one-stop shops with optimally packaged evidence [11]. By making evidence available, a one-stop shop becomes one of the necessary inputs for increasing access to evidence. Although it may not be a sufficient factor, its absence creates a clear gap and thus a barrier to use of evidence by policy and decision-makers [11]. The one-stop shop may facilitate timely access to well-packaged evidence by policy and decision-makers when faced with questions about health systems and interventions [12]. In a systematic review on health policymakers’ perceptions of their use of evidence, Innvaer et al. [10] identified timely access to evidence as one of the facilitators of use of evidence in policymaking. This is corroborated by Lavis et al. [11] and further supported by Oliver et al. [13] in an updated systematic review. The latter included perceptions of other stakeholder groups such as researchers, managers and research users other than policymakers [13].

In the recent past, there have been efforts to develop one-stop shops for both global research evidence and local policy-relevant documents to address questions about health interventions and health systems in high-income countries. Examples include The Cochrane Library, Health Systems Evidence, NHS Library and Knowledge Center and HTA Database Canada Search Interface [14–17]. On the other hand, resources focused on local policy-relevant documents are lacking in low- and middle-income countries and the feasibility of developing them has not been tested.

To address this challenge in Uganda, in 2011, the Supporting the Use of Research Evidence (SURE) in African health systems project [18] embarked on developing the Uganda Clearinghouse for Health Policy and Systems – a one-stop shop for health policy-relevant documents. The SURE project was a collaborative project that built on and supported Evidence-Informed Policy Networks (EVIPNets) in Africa and the Regional East Africa Community Health (REACH) Policy Initiative in Uganda specifically. EVIPNet Africa includes African partners such as Burkina Faso, Cameroon, Central Africa Republic, Ethiopia, Mozambique, and Zambia [19].

This Clearinghouse is intended to facilitate timely access to decision-relevant information required by policymakers, stakeholders and researchers about the Ugandan health system and interventions. However, this resource could be limited by the adequacy of its content and the way it is organised. There is a lack of documented evidence on the available Uganda-specific health policy and systems-relevant documents that would inform whether most of the important documents have been included in the clearinghouse. Further, there is no clear framework to guide the organisation of the documents in the clearinghouse.

This paper provides a scoping review of Uganda-specific health policy and systems-relevant documents produced in the last 15 years, up to December 2014. It is a step forward towards the mobilisation of documents for improving the Uganda Clearinghouse for Health Policy and Systems. The paper also identifies and provides a refined categorisation of the policy-relevant documents necessary for building the content of the clearinghouse to facilitate easy search by the users. The purpose of this paper is not just to indicate the ideal content of the Clearinghouse but to demonstrate that it is doable in a low-income setting. It aims to provide a framework which one can follow explicitly to generate an inventory of policy-relevant documents.

Methods

We reviewed published documents relevant for health policy and decision-making about the Uganda health system and interventions produced from January 1, 2000, to December 31, 2014. The year 2000 marked the beginning of implementation of key health sector reforms in Uganda [20]. Of interest was to identify and characterise documents produced since then to the beginning of this study. Our methods borrowed heavily from the 2005 Arksey and O’Malley methodological framework for scoping reviews [21].

Step 1: Identification of the research question

Since the research question guides the subsequent steps, including the search strategy, the Arksey and O’Malley methodological framework recommends considering all aspects of the research area to ensure a breadth of coverage and to define the relevant aspects of the research question [21]. In light of this, we developed our research questions as: What are the available types of documents relevant for health policy and systems that are specific to Uganda? What is the volume and nature (i.e. type, coverage of national priority areas, frequency of health-system topics) of these documents? From the onset, we were aware that such documents could be available as printed copies, published on websites of relevant non-governmental organisations (NGOs) and national institutions or just in the form of soft copies on personal computers that are not yet uploaded on websites. We focused on identifying and characterising the available documents relevant for health policy and systems published on websites.

Step 2: Identification of relevant documents

We conducted the search for Uganda-specific health policy and systems-relevant documents in January 2015. We selected the websites of relevant government institutions, international and national NGOs, health professional councils and associations, religious medical bureaus and research networks (Table 1). We used the search engine Google to locate such websites, which we then navigated by the tabs and menus available on the homepage (such as policy documents and guidelines, e-library, resources, publications, legislation). The fact that different websites are organised differently, we developed specific search strategies for each website depending on its individual navigability. In addition, we searched Google Scholar using the following keywords in various combinations with Boolean operators (and, or) [22]: Uganda, health policy, health system, guidelines, strategies, plans, and reports. We checked the reference lists of the documents found to expand our list of included documents. Importantly, we used the websites as an entry point to other repositories for national policy documents (Tables 2 and 3).

Table 1 Institutions/organisations whose websites were searched

Table 2 Type of documents mapped by the national health priority areas

Table 3 Type of the document by health systems domains, implementation strategy within the health systems and the crosscutting domain (10)

Step 3: Screening and selection of relevant documents

To minimise selection bias, two independent reviewers (BM and RB) screened all documents and selected those that were appropriate for our research question. Our selection involved the use of a pre-determined inclusion and exclusion criteria. We included all Uganda-specific published documents relevant to health policy and systems produced between 2000 and 2014. We borrowed from the Hoffman et al. [23] model shown in Fig. 1, which depicts the boundaries of health policy and systems research, to determine if documents were relevant to health policy and systems in order for us to include them. We therefore included all documents that addressed (1) issues related to health systems (i.e. on governance, financial, and delivery arrangements and implementation strategies); (2) policy about clinical issues that include essential medicines, diagnostics and medical supplies; and (3) policy about public/population issues such as policies on immunisation and family planning.

Fig. 1

Conceptual issues related to health systems research to inform a WHO Global Strategy on Health Systems Research. Adapted with permission from Hoffman et al. [20]

We excluded documents that (1) did not have national coverage (such as NGO project reports that covered only a few districts), (2) were at the draft stage, (3) covered less than a year (such as quarterly or semi-annual reports), or (4) described primary studies and systematic reviews.

Step 4: Charting of the data

The two independent reviewers used a specially developed data-charting form (Table 4) to extract data from each document on the title/topic, document type, coverage of national priority areas, coverage of health system topics, year published and the source of the document. We developed a tailored index of health policy documents based on the national priority issues, types of documents emerging from the search results and health system topics borrowed from the Health System Evidence [15, 24, 25]. We specifically categorised the documents as policies, strategies, plans, guidelines, rapid response summaries, and evidence briefs for policy, dialogue reports and other reports. We also coded the documents for national health priority areas as identified in the Second National Development Plan (NDPII 2015/16–2019/20) [24] and Second Health Policy (NHPII) [25]. The NDPII stipulates the Country’s medium term strategic direction, priorities (including health priorities) and implementation strategies up to the year 2020. The national health priority areas are disease prevention, mitigation and control; health education, promotion, environmental health and nutrition; governance, coordination, monitoring and evaluation; maternal and child health; reproductive health; human resources for health; health financing; health infrastructure; early childhood development; essential medicines and supplies; palliative care services; rehabilitation services; and curative services [24, 25].

Table 4 Documents reviewed

We further classified these documents as governance, financial and delivery arrangements, and as implementation strategies within the health systems [15]. The governance arrangements category includes documents on centralisation/decentralisation of health services, registration and accreditation of the services, consumer and stakeholder involvement in service delivery, stewardship of the non-state actors in health financing and delivery, among other topics [15]. The financial arrangements category includes documents on financing systems, funding organisations, remunerating providers, purchasing products and services, and incentives targeted at consumers [15]. The delivery arrangements category covers documents on how care is designed to meet consumer needs, human resources for health, and support systems for the provision of care, plus where care is provided [15]. The implementation strategies category includes documents on consumer-, provider- and organisation-targeted strategies [15]. The third reviewer (EO) arbitrated areas that BM and RB disagreed on.

Step 5: Collating, summarising and reporting results

After charting the information from the evidence documents, we presented the review findings as numerical analyses of the volume of documents, their nature (i.e. type, coverage of national priority areas, the frequency of health system topics) and trends over time in the form of tables and charts. The data were summarised using descriptive statistics, including for the type of the documents included, the national health priority areas/issues covered in the documents, the extent of coverage of the health priority for the different health policy and system domains, and the trends over time in the nature and distribution of the documents.

Results

The website hand-searches resulted in a total of 909 health policy and systems-relevant documents, including 10 duplicates. Of the remaining 899 documents, 265 met the selection criteria and were considered for analysis (Fig. 2).

Fig. 2

Results of the document search process and screening

Volume of the documents and trends over time

Most of the health policy and systems-relevant documents included in our review were rapid response summaries (18%) and guidelines (13%). The least were strategies (4%) and policy dialogue reports (3%) (Table 2). There was a consistent increase in the volume of the policy and system-relevant documents between 2000 and 2011 followed by a decline. The increase was from around six documents per year in 2000 to 49 per year in 2011 that later dropped to 27 in 2014 (Fig. 3).

Fig. 3

Number of policy-relevant documents produced by year

Coverage of the national health priority areas by the documents

In Table 2, the top three national priority areas (clusters) addressed in the documents are governance, coordination, monitoring and evaluation (74, 28%), disease prevention, mitigation and control (63, 24%), and health education, promotion, environmental health and nutrition (41, 16%). The least addressed national health priorities were curative services, health infrastructure, palliative care services, rehabilitation services, each addressed in three documents (1%), and early childhood development (in only one document). The biggest percentage of policies (46%), guidelines (31%), policy dialogue reports (29%) and evidence briefs for policy (18%) addressed the cluster of disease prevention, mitigation and control, while most strategies (90%) and rapid response summaries (17%) covered the health education, promotion, environmental health and nutrition cluster, and plans (36%) addressed the governance, coordination, monitoring and evaluation cluster (Table 2).

Coverage of health systems topics by the documents

Generally, the highest number of documents (n = 101, 38%) addressed the delivery arrangements domain, followed by the governance arrangements (68, 26%), implementation strategies (56, 21%), and lastly the financial arrangements (16, 6%) (Table 3). Notably, the delivery arrangements domain was addressed by most of “other reports” (93%), evidence briefs for policy (33%) and rapid response summaries (25%). Nearly, two-thirds (63%) of guidelines, 49% of policies and 30% of rapid response summaries addressed the governance arrangements domain. A small percentage of evidence briefs for policy (12%), rapid response summaries (17%) and other reports (5%) addressed the financial arrangement domain; this was not at all covered by guidelines, plans, policies, policy dialogue reports and strategies. Most of the plans (96%) covered the implementation strategy; there was no single policy or other report that addressed it. Other documents that were about public or clinical issues included 51% of policies, 29% of policy dialogue reports, 9% of guidelines and 4% of rapid response summaries.

Discussion

In this paper, we conducted a scoping review of policy and systems-relevant documents in Uganda to support the identification and characterisation of policy and systems-relevant documents for the content of an on-line repository. The first step of our framework, which is identification of documents, borrows from the Arksey and O’Malley methodological framework for scoping reviews. The second step involves the development of the tailored index of health policy documents based on the national priority issues, types of documents emerging from the search results and health system topics [15, 24, 25].

A number of key findings emerged. First, the review demonstrates the availability of the policy and system-relevant documents in the country that include policies, guidelines, plans, strategies, rapid response summaries, evidence briefs for policy, and policy dialogue reports. The available documents address several national health priority issues identified in the Second Health Policy and National Development Plan (2015/2016–2019/2020). Further, the review findings show varying coverage of the national health issues and health system topics by the documents, which is an important indication of areas of interest. Finally, it demonstrates that there has, until recently, been a progressive increase in the number of documents produced although the distribution of different document types has not been uniform.

Findings in relation to other studies

Our findings especially, on the availability of the policy and system-relevant documents are supported by literature from previous studies in low- and middle-income countries [26, 27]. We found out that apart from “other reports”, rapid response summaries were by far the most dominant type of documents produced, although most of them were produced in a particular period, from 2008 to 2013. The fact that their production was majorly by research networks with a local presence [18, 19] is suggestive of increased local capacity to produce summaries. This may also reflect a growing interest by policymakers and stakeholders to use the summaries to address urgent policy questions in the country. This was corroborated in a study by Mijumbi, which showed that a rapid response mechanism service in Uganda has been widely used by national policymakers at the Ministry of Health and development partners and stakeholders from NGOs [26]. Although we utilised a multifaceted search strategy, relatively few plans, strategies and policy dialogue reports meeting the specified inclusion criteria were identified. The lack of these documents may indicate their scarcity in the country. This may instead point to the fact that these documents were not readily on governmental and NGOs’ websites: an indication for lack of one-stop shops for knowledge sharing in the country. This is supported by findings from a study by Murphy [27], which demonstrated that information on training and deployment policies for health workers for maternal, newborn and child health in rural Africa was not available on governmental websites; it was instead readily located on the sites of institutions with a greater capacity for knowledge sharing. Generally, there was a noted increase in the number of documents from 6 documents per year in 2000 to 49 per year in 2011, dropping in the following years. The lack of documents in the earlier years may not be surprising because, the older they are the harder it is to find such documents online. However, it could also be due to the delay in posting the documents on the website. The increase in the volume of documents over time may reflect an increase in funding for health policy and systems in the country.

The national health priority areas were not equally tackled by the documents, some were more addressed than others. For example, the clusters of disease prevention, mitigation and control and that of governance, coordination, monitoring and evaluation were each covered by almost a quarter of the documents. The breadth of topics and types of documents available reflects what is considered most important by the government of Uganda or may be what the major funding sources perceive to be the most important health issues in Uganda. Besides funding and politics, the sector decision-making process is guided by a sector-wide approach, the compact and International Health Partnerships Plus frameworks involving all key stakeholders, including donors, private sector, civil society and the Government of Uganda. However, these findings may be an indication of the lack of a clear priority-setting mechanism for health policy and systems and of sufficient funding to address important areas.

The delivery arrangement was the most popular health system domain covered by the documents, followed by governance and implementation strategies. A paucity of documents on the financial arrangements domain and the crosscutting issues was noted. This is similar to findings from other studies in low- and middle-income countries [26, 28]. For example, a review by Rao et al. [28] on health systems research in the time of health system reform in India indicated that service delivery was the health system domain most covered by the publications reviewed in comparison to other domains. The neglect of the finance arrangements domains was also noted in Mijumbi’s study on the feasibility of a rapid response mechanism to meet policymakers’ urgent needs for research evidence about health systems in a low-income country [26]. The lack of documents in some health system domains, such as financial arrangements, may reflect many reasons such as lack of interest in the area by the authorities.

Strengths and limitations

To our knowledge, this is the first scoping review of local policy-relevant documents that address questions about health systems and interventions in the eastern African region; previous papers have focused on developed countries in general [14–17]. Our study utilised a rigorous methodological approach for scoping reviews that ensured the validity of the results [21]. We tried to identify all published documents by searching different websites of relevant NGOs and national institutions. The combination of heterogeneous sources of data adds value to the results. However, the review is not exhaustive because we were not able to include hard copies of the documents that were not yet uploaded on the websites. Further, given the wide range of terminology used to describe policy and systems-relevant documents, the study could have missed identifying some documents. We did not consider hard copies due to the limited resource setting. However, we used the website documents as tracers. We recommend future research to consider documents not on websites. Looking at national documents is just one of the inputs in decision-making. International learning can inform local policies. However, our study focused on the Uganda-specific documents as just one of the many inputs.

Implications for policy and research

Our study provides novel insights into the creation of one-stop shops for research evidence and policy-relevant documents. Specifically, it demonstrates the feasibility of identifying the content of the clearinghouse in a low- and middle-income country, provides an explicit mechanism for categorising the content, and shows that it is possible to adapt the index of health policy documents. Our approach provides academic and other research institutions involved in knowledge brokerage in low- and middle-income countries with a framework for identifying and organising the content of the on-line repositories for health policy and system information. To our knowledge, this is the first scoping review of local policy-relevant documents that address questions about health systems and interventions in the African region. Previous research has focused on developed countries in general.

It is anticipated that this framework may add to the ongoing research efforts in high-income countries that have focused on developing one-stop shops for both global research evidence and local policy-relevant documents. Such efforts include a study by Lavis et al. [15] on health system evidence that focused on developing and refining the methods for a ‘one-stop shop’ for synthesised research evidence about health systems. In this study, they developed a taxonomy of health system topics for categorising systematic reviews and systematic review protocols. This study demonstrated that policymakers and stakeholders could easily access and use a wide variety of types of research evidence about health systems to inform decision-making and advocacy. Rosenbaum et al. [14] also studied the user experiences of The Cochrane Library, providing a basis for building and improving on-line resources for evidence-based practices. In another study, Faith et al. [17] developed and tested a search tool for HTA Database Canadian Search interface for supporting the use of health technology assessments by decision-makers.

In particular, our findings can inform re-designing of the Uganda Clearinghouse for Health Policy and Systems. The documents reviewed tackle issues identified by the Second Health Policy and National Development Plan, which feed into the Vision 2040 and subsequently contribute to Sustainable Development Goal 3 (i.e. ensure healthy lives and promote well-being for all at all ages) [24, 25, 29, 30]. Thus, categorising content of the on-line repository according to the national health priority issues may increase the chances of using the resource by health policy and decision-makers. The study findings can also inform government and funders to support the production of policy and systems documents to address the coverage gaps in the national priority issues and health systems domains. In this study, it was not feasible to hold consultations with consumers and stakeholders, future scoping work should consider this for prioritisation, additional sources of information and perspective.

Conclusion

A one-stop shop for health policy-relevant information may increase the likelihood of using the resource to inform decisions about health systems and interventions if it consists of a wide variety of relevant document types. Thus, the demonstrated availability of health policy and systems documents that address a number of national priority health issues is important for facilitating efforts towards mobilising, building and organising the content of a one-stop shop for Uganda-specific documents. With the resource in place, policymakers, decision-makers and stakeholders will now easily access and use well-packaged policy-relevant documents for decision-making.