A systematic review of Health Technology Assessment tools in sub-Saharan Africa: methodological issues and implications

Background Health technology assessment (HTA) is mostly used in the context of high- and middle-income countries. Many “resource-poor” settings, which have the greatest need for critical assessment of health technology, have a limited basis for making evidence-based choices. This can lead to inappropriate use of technologies, a problem that could be addressed by HTA that enables the efficient use of resources, which is especially crucial in such settings. There is a lack of clarity about which HTA tools should be used in these settings. This research aims to provide an overview of proposed HTA tools for “resource-poor” settings with a specific focus on sub-Saharan Africa (SSA). Methodology A systematic review was conducted using basic steps from the PRISMA guidelines. Studies that described HTA tools applicable for “resource-limited” settings were identified and critically appraised. Only papers published between 2003 and 2013 were included. The identified tools were assessed according to a checklist with methodological criteria. Results Six appropriate tools that are applicable in the SSA setting and cover methodological robustness and ease of use were included in the review. Several tools fulfil these criteria, such as the KNOW ESSENTIALS tool, Mini-HTA tool, and Multi-Criteria Decision Analysis but their application in the SSA context remains limited. The WHO CHOICE method is a standardized decision making tool for choosing interventions but is limited to their cost-effectiveness. Most evaluation of health technology in SSA focuses on priority setting. There is a lack of HTA tools that can be used for the systematic assessment of technology in the SSA context. Conclusions An appropriate HTA tool for “resource-constrained” settings, and especially SSA, should address all important criteria of decision making. By combining the two most promising tools, KNOW ESSENTIALS and Multi-Criteria Decision Analysis, appropriate analysis of evidence with a robust and flexible methodology could be applied for the SSA setting.


Background
Health technology assessment (HTA) is a multidisciplinary field of policy analysis that examines the medical, economic, social, and ethical implications of the incremental value, diffusion, and use of a medical technology in health care [1]. Currently, HTA is mostly used in the context of developed countries. Several methodologies exist for "resource-poor" settings but implementation of HTA and transparent use in most African countries is still limited [2]. In addition, some methodological aspects of HTA do not fit into the setting of developing countries and need to be adapted appropriately according to specific needs [2]. Especially in "resource-constrained" settings, the need for the systematic evaluation of health technology and of the available alternative technologies has never been greater [3,4]. HTA is performed in order to improve the quality of health care and ensure good value for money investments in any setting. It is because of this that HTA should form the basis for health technology policies especially in "resource-poor" settings with limited health sector budgets.
The current lack of HTA in sub-Sahara Africa (SSA) can be attributed to the lack of capacity to undertake HTA. Even though countries like South Africa currently employ HTA to a small degree and have begun the process of forming a HTA mechanism, the lack of capacity coupled with a weak health system capacity to implement interventions are contributing to widespread implementation of HTA. Another problematic issue is the limitation of high-quality data availability and lack of research evidence, especially in the context of "resource-limited" health systems like the assessment of health states [5,6]. This reduces the ability for "resource-poor" settings to implement rigorous HTA practices. In addition, "resource-limited" countries have few resources to support HTA, which then undermines the ability of HTA to utilize appropriate incentives (that would promote more efficient management of resources.
Incentives for HTA in resource-limited settings: What are the incentives for resource-limited countries to develop HTA agencies?
1 -Providing international guidelines and model essential medicines lists as well guidance on good and ethical governance practices specifically in liaison with ethic review committee and bioethics committee Low-income countries with very limited capacity and countries that face major challenges of lack of transparency in decision-making are likely to face difficulty implementing HTA for medicines effectively. Use of international guidelines and model essential medicines lists can help such countries to incorporate HTA in their policies.

-Complement HTA with evidence-based guidelines
Policies to encourage doctors to prescribe formulary medicines and follow evidence-based guidelines are needed to complement HTA.
3 -Strengthening capacity in HTA before to set-up HTA agencies through national capacity building workshops specifically for decision makers and health professionals Capacity for HTA in resource limited settings should be established early and supported; prerequisites and barriers are extensive but not insurmountable and must be considered as HTA processes are developed.
Decisions in many "resource-poor" settings can easily be influenced by past experience without an evidence base and by preferences of donor agencies and lobbying pressure for new technologies, for example from commercial organisations or global funding and donor organizations [2]. This can lead to the use of technologies which do not address health needs and in effect contribute to the inefficient use of resources [7]. Decisions made in this context often do not reflect local values and evidence based local information on clinical and cost effectiveness [2]. As a result of this, many resources are disproportionately allocated or wasted. The need for HTA becomes more prominent when the need for collective decision making for the good of the whole is necessary. It also becomes apparent that HTA in "resource-poor" settings cannot be addressed the same as it is in high-income countries [8]. Equity and equality considerations are different from these in developed countries. These include more important social issues like poverty reduction [9].
There is a clear need for HTA use in "resource-limited" settings, and SSA countries in particular, as it is these countries especially that cannot afford to waste resources. Not only would HTA highlight health technologies that would be too costly relative to their benefits, but would also identify potentially harmful and ineffective technologies. In light of a higher percentage of insured patients stemming from advanced economic development and the potential to purchase more and more complex, and expensive, health technologies, the introduction of HTA into public health policies becomes more evident. The WHO resolution on "Health Intervention and Technology Assessment in Support of Universal Health Coverage" passed on 24 May 2014 by the World Health Assembly provides an important mandate for SSA countries to accelerate their HTA efforts.

Objectives
The research objectives of this systematic review are to provide an overview of HTA tools used in "resourceconstrained" settings, with a specific focus on the assessment on SSA. It was chosen as the focus setting because it is the region with the least covered area by HTA practices and also the most "resource-constrained" setting. Another objective is to determine how many of the appropriate HTA tools identified in this review were applied for the assessment of medical devices.

Search method
The PRISMA guidelines for conducting systematic reviews were followed [10]. Within the study the following databases were searched: PubMed, Science Direct, Scopus, Ebscoh, and EconLit alongside the journals Health Policy and Planning, Cost-Effectiveness and Resource Allocation, and the WHO Bulletin (see Appendix for detailed search string). To ensure optimal coverage, additional articles were found within the reference section of retrieved articles and through citation snowballing by undertaking wider searches by author name for those appearing as key publishers in the area. Additionally, the web pages of the WHO and the World Bank were searched manually. The search was limited to articles published within the 2003 and 2013 timeframe and excluded non-empirical studies or those that did not focus on a HTA tool.

Selection of manuscripts and data extraction
Articles that met the inclusion criteria of an evidencebased HTA tool appropriate for the use in the SSA context were retrieved and examined more closely. The quality of research papers was evaluated according to adequate description of the theoretical framework, background, and methodology [11].
For those papers that fulfilled the criteria for quality, data was extracted according to the following content: date published, study funding source, possible conflicts of interest, study objectives, target population, application of tool, site/setting, study focus, HTA tool proposed or approach used in the paper, description of tool or approach, stand alone or support tool, aspects of clinical effectiveness, costs and contextual issues addressed by tool or approach, all stakeholders involved, literature search incorporated, results of implementation, and focus on medical technology/intervention. Each study was described by addressing the criteria in the data extraction form ( Table 1).
The HTA tool or approach found in each study were critically appraised using a second data extraction form ( Table 2) based on proposed criteria for the assessment of HTA activities [12]. These criteria are formed by a series of 15 principles, as described by Drummond et al. [12], which cover the structure of HTA programs, methods of HTA, processes for conduct of HTA, and use of HTAs in decision making. This set of principles was utilized to ensure robustness of the included HTA approaches and tools. The extraction form prompted i) the context and applicability of the described tool (geographical focus, target setting, reasons for use, degree of needed training, and experts); ii) measures for sensitivity and effectiveness (clinical effectiveness, cost effectiveness, context sensitivity); iii) the approach to HTA assessment (type of tool, inclusion of full social perspective, use of available evidence, transparency, generalizability, result focus, ling to decision making), and iv) if the described tool was piloted for medical devices.
Criteria addressing areas particular to SSA were included in the evaluation to assess how adaptable the tool or approach would be for that setting. The criteria included the following: ease of training, flexibility of evidence requirement, economical standing, and local context consideration (Table 3). These criteria have been elaborated in consensus with our Research Consortium Members including representatives from SSA countries.

Results
The search retrieved 1,073 papers in total, of which six fulfilled the inclusion criteria (PRISMA flowchart, Figure 1). Data was extracted from the six papers published between 2003 and 2013.
HTA tools overview: WHO CHOICE cost-effectiveness analysis tool [13] KNOW ESSENTIALS tool using thirteen criteria related to context-specific HTA and prioritization of these criteria [8] Weighting of Multi-Criteria Decision Analysis criteria on the basis of corresponding HTA report [14] Adaptation of Mini-HTA or hospital-based HTA tool for the decision-making related to medical devices purchased in a hospital setting [15] Equity-oriented toolkit for HTA, containing four elements: burden of illness, community effectiveness, economic evaluation and knowledge translation [16] Mapping system using Analytic Hierarch Process (AHP) methods between different diseases and their matching technologies to minimize technology underutilization [17] None of the papers were funded or stated funding by other sources. Four studies proposed novel tools for HTA [8,[15][16][17] and the other two used current HTA approaches originally developed for high-income settings [13,14]. One study specified their target population to be in SSA countries [15], the others focused on "resourcepoor" settings [13,14,16,17] and settings without formal HTA [8]. However, all studies included in the review can be applied in the SSA setting. From the proposed tools, two were stand-alone [8,16] and the other two were support tools [15,17]. The support tools served as decision      Yes, involves stakeholders in the process which bring in a contextual perspective

Considerations
Does not include an explicit priority setting system. However, this tool is used to assess specific health technologies so it is further down in the HTA process. Evidence has the potential of not being very robust Economical standing of this tool is dependent on the price of the decision making assistance software Economical standing of this approach is dependent of amount of special training needed The mini-HTA tool is a fast tool; however, it is not very comprehensive and only vaguely addresses the principles that it proposes. In addition, one of its limitations is that it is only applied in the context of hospital settings support in the purchase and acquisition of cardiovascular disorder equipment [17] and the scaling up of training and education of health workers [16]. The current HTA approaches used in the studies were: multi-criteria decision analysis (MCDA) [14], the WHO-CHOICE project [13], and generalized cost-effectiveness analysis [13]. Four papers applied their tool or approach [8,14,15,17], two of those were pilot tests [8,17]. The other two approaches and tools were not applied and only described proposed HTA approaches or tools [13,16]. Two of the tools were focused on pharmaceuticals [8,13], whereas three HTA tools were focused on medical devices [14,15,17]. For the medical devices, varying contexts were chosen: one study focused on the introduction of a screening test [14], another on the underutilisation of medical devices [17] and a third study focused on the decision-making process for the purchasing of medical devices [15]. One tool was centred on a health education intervention [16].

Structure of HTA programs
All of the papers were found to explicitly address relevant goals and scopes of their HTA tool or approach. Five studies were unbiased and had transparent processes [8,13,14,16,17]. One tool required subjective assessment during its process [15]. Five methods included all relevant technologies by considering all alternatives [8,14,15,17] or by using an extensive database to do so [13], while the sixth study did not state if this was addressed [16].

Processes for conducting HTA
All of the approaches required for all available data to be sought [8,[13][14][15][16][17]. Two of the papers failed to actively include all key stakeholders [13,17], whereas all others included key stakeholders by including them in the process or considering them throughout [8,[14][15][16]. How findings would be monitored was not stated in any of the papers.

Use of HTA in decision-making
Only two studies mentioned a timeframe for completing the HTA, which also was timely [15,17]. Findings were communicated appropriately to different decision makers by four tools or approaches [8,[14][15][16]. Three of the papers clearly defined the link between HTA findings and decision-making processes [8,14,15], while the others did not state this [13,15,17].
Principles for HTA activities specific for SSA In the context of SSA, five papers presented tools or approaches that were easy to use [8,[14][15][16][17], whereas one required special training to carry out the generalized cost-effectiveness analysis [13]. The evidence requirement was flexible and included available data whether from a literature search or other less rigorous, yet justified, sources when it was not available for five of the studies [8,[13][14][15]17]. One was not as flexible by requiring the data to be collected in databases such as the Cochrane Library and no additional alternate sources were mentioned [16].
Only one of the papers failed to address the local context explicitly in their tool [17].

HTA advantages and areas of use
The advantages of using HTA are first and foremost the systematic evaluation of cost and effectiveness of medical technologies and allowing health systems to achieve the greatest good for the greatest number of patients. Chalkidou et al. highlight the importance of HTA for universal health coverage systems by efficiently and equitably allocating resources [18]. This focused resource use has an effect on better budgeting and long-term financial sustainability of the health systems in SSA countries. It is crucial that the increased use of HTA in these countries is complemented by capacity building and increased expertise in the HTA area in order to ensure a sustainable infrastructure [19]. International collaboration among HTA bodies can facilitate the development of methods and more efficient assessment processes, and facilitate knowledge transfer and capacity-building in less established HTA systems and programmes. Another area of need is emerging for relevant applications of HTA, namely its use in global funding organisations as a means for increased value for money. Teerawattananon et al. have argued for the use of HTA approaches for the Global Fund to Fight HIV/AIDS, Tuberculosis and Malaria in order to provide strategic directions for the prioritization of health care interventions currently funded by the organization [20]. Other global health initiatives and national donor countries may follow suit in conducting HTAs before, during, and after grant implementation in order to improve efficiency and identify areas of unmet need (in June 2014 the Gates Reference Case has been launched which involves more principled costeffectiveness analysis in health programme funding by the Bill and Melinda Gates Foundation). Therefore, the necessity for the thorough evaluation and fine-tuning of relevant HTA tools and approaches becomes more evident.

Lack of application
The review shows widespread methodological heterogeneity among the different studies included in the review. The HTA tools or approaches used varied a lot in their context and scope. There was a lack of application of some tools for a specific medical technology or intervention in the SSA setting. The tools analysed in this review would benefit from a wider application and pilot-testing as well as user friendliness. A direct comparison between applying the tools or approaches would also highlight the advantages and disadvantages of each tool or approach and guide decision makers in regards to which tools should be used, in which context, and for which tasks.

Robustness of HTA
While the need for HTA in SSA is evident, the robustness of HTA cannot be neglected when resources are limited. The majority of the approaches involved a well-rounded structure for their HTA that was accomplished with explicit goals and scopes, unbiased and transparent processes, and the inclusion of alternative technologies and some also had priority setting processes incorporated [14,16,17]. In this context, "priority-setting" focuses on identifying different health technologies for which an evaluation regarding their inclusion in the health system is warranted, while HTA relates to the actual evaluation of a specific technology.
The methods utilized in the approaches followed the principles for HTA activities in most of the studies. However, one approach that seemed to be limited by its purpose as a computerized decision-making aide [17] seemed to lack robustness in its methods by not assessing costs. The processes for conducting HTA were limited for some approaches by the exclusion of key stakeholders [13,17], an integral dimension of good practice in HTA methodology. In addition, the monitoring of HTA practices lacked in all of the studies. When assessing the use of HTA in decisionmaking, two approaches showed limitations [13,17]. One approach completely lacked the inclusion of the principles in this section, but this may be connected with the priority setting orientation of the approach [13]. All of the tools addressed the issue of economic resource deficiency; however, they lacked specifications explicitly considering the local context [17], requiring little training for their use [13] and being flexible with the data necessary for undertaking the HTA when there is limited data or access available [16].

Potential opportunities
In general, the review revealed that the majority of approaches were not applied in a stand-alone manner and were rather used as a support tool to existing decisionmaking processes. The analysis revealed that two approaches show particular promise for further investigation: the KNOW ESSENTIAL tool for its compact, yet comprehensive coverage [8], and the MCDA approach for the active involvement of stakeholders in its process [14]. These incorporated all aspects important for the SSA context into their evaluation such as contextual issues, flexible data collection practices, and economical and easy use. However, it is important to point out that the MCDA approach is, per se, focused on assessing available evidence and not generating new evidence as in the case with other HTA tools. Although several MCDA tools have been applied in the SSA setting, we only included one study in our review as it was explicitly connected with the assessing of evidence in the context of a HTA report.
Even though the mini-HTA tool [15] also incorporated the majority of the principles for HTA activities and all of the contextual SSA principles, it had a number of limitations related to comprehensive and detailed coverage due to its restriction to the hospital setting [15]. The critical appraisal also revealed that other tools did not meet all of the contextual criteria for SSA [13,16,17] or were limited to a position further down-stream in the decision making process. Although it is possible to weight the criteria applied in the KNOW ESSENTIALS tool, it would be beneficial and interesting to combine the tool with aspects of MCDA in order to allow for a more detailed stakeholder evaluation and prioritization of the evidence.
A thorough evaluation of the available tools is highly desirable, involving a wide range of academic and ministry of health partners in a SSA setting in order to ensure context-specific application of HTA tools. Specific emphasis should be made on the need for HTA evaluations to allow for differences in the evaluations of pharmaceuticals and medical devices. The majority of HTAs currently focus on the assessment of pharmaceuticals and tend to neglect medical devices. Due to the varying focus of the HTA tools that are centred on the evaluation of medical devices in our review, it is not possible to draw clear conclusions on the appropriate emphasis of the development of medical device HTA tools.

Study limitations
The methodology for the evaluation on each quality criteria point highlighted in Table 3 "Extraction Form for Principles for HTA Activities in SSA" is based on broad expert opinion elicitation in our research consortium, which represents different areas and institutions such as academia, WHO, HTA expertise, and political decision makers. Ultimately, the evaluation on the basis of expert opinion is a subjective assessment of the research consortium which may be subject to potential bias and as such has to be highlighted as a study limitation.

Conclusions
Our review has emphasised that there is a lack of HTA tools that can be used for systematically assessing technology in the SSA context. A clear gap in HTA methodology focused on "resource-limited" settings, and particularly the SSA context, calls for more research into further evaluating and developing relevant HTA methods and approaches, especially in the context of the WHO resolution on "Health Intervention and Technology Assessment in Support of Universal Health Coverage".
An appropriate HTA tool for "resource-constrained" settings, and especially SSA, should address all important criteria of decision making. By combining the two most promising tools, KNOW ESSENTIALS and MCDA, appropriate analysis of evidence with a robust and flexible methodology could be applied for the SSA setting. Although there are a range of arguments favouring the need for HTA in the SSA context, advocacy for the importance of HTA in these settings needs to emerge more clearly [18].