Research | Open | Open Peer Review | Published:
Health policy and systems research in access to medicines: a prioritized agenda for low- and middle-income countries
Health Research Policy and Systemsvolume 11, Article number: 37 (2013)
To identify priority policy issues in access to medicines (ATM) relevant for low- and middle-income countries, to identify research questions that would help address these policy issues, and to prioritize these research questions in a health policy and systems research (HPSR) agenda.
The study involved i) country- and regional-level priority-setting exercises performed in 17 countries across five regions, with a desk review of relevant grey and published literature combined with mapping and interviews of national and regional stakeholders; ii) interviews with global-level stakeholders; iii) a scoping of published literature; and iv) a consensus building exercise with global stakeholders which resulted in the formulation and ranking of HPSR questions in the field of ATM.
A list of 18 priority policy issues was established following analysis of country-, regional-, and global-level exercises. Eighteen research questions were formulated during the global stakeholders’ meeting and ranked according to four ranking criteria (innovation, impact on health and health systems, equity, and lack of research). The top three research questions were: i) In risk protection schemes, which innovations and policies improve equitable access to and appropriate use of medicines, sustainability of the insurance system, and financial impact on the insured? ii) How can stakeholders use the information available in the system, e.g., price, availability, quality, utilization, registration, procurement, in a transparent way towards improving access and use of medicines? and iii) How do policies and other interventions into private markets, such as information, subsidies, price controls, donation, regulatory mechanisms, promotion practices, etc., impact on access to and appropriate use of medicines?
Our HPSR agenda adopts a health systems perspective and will guide relevant, innovative research, likely to bear an impact on health, health systems and equity.
The provision of reliable access to affordable, quality medicines is a necessary component of functioning health systems . However, access to and use of essential medicinesa is often poor in low- and middle-income countries (LMICs), with particular challenges faced by the poor. In the formal sector of LMICs, average availability of medicines is 35% in public facilities and 66% in the private sector, although prices are often unaffordable in the latter . Up to 50% of medicines are inappropriately prescribed or dispensed and 50% are used incorrectly by patients . Health-seeking behaviour towards an unregulated private market is widespread [4, 5]; substandard and counterfeit medicines are also prominent [6–8]. Medicines typically account for 20% to 60% of health spending , and 50% to 90% of this amount is out-of-pocket .
There is a need to support Health Policy and Systems Research (HPSR) in the field of medicines in resource poor settings and the topic has been highlighted as an important area of research by the Taskforce on Health Systems Research . A bibliometric survey of publications in HPSR in LMICs showed 648 publications on medicines between 2003 and 2009 . This figure represents only 10% of global publications on medicines and, of these, only 50% are written by authors from LMICs.
Given limited resources available for HPSR in LMICs and the large number of issues to be researched, priority setting is an essential initial step to undertaking meaningful, policy-relevant research. The World Report on Knowledge for Better Health reports that policy-makers, researchers and users are usually unaware of how research priorities are set ; HPSR is often driven by donor support rather than by country needs [13, 14]. In relation to medicines, priority-setting exercises have usually aimed at defining lists of priority medicines to address global disease burden  or specific health problems faced by vulnerable groups such as mothers and children , but they have not defined a HPSR agenda. The proceedings of the International Conference on Improving the Use of Medicines present key recommendations for research . However, these are almost 100 research recommendations across 19 themes and are not translated into prioritized research questions. Moreover, as they emanate from the proceedings of a scientific conference, these recommendations reflect gaps in academic work presented, which may not adequately reflect the concerns of decisions-makers or patients and communities.
This paper presents a priority-setting exercise performed by the Alliance for Health Policy and Systems Research (AHPSR) between 2010 and 2012, with the objectives of: i) identifying priority policy issues in access to medicines (ATM) relevant for LMICs; ii) identifying research questions that would help address these policy issues; and iii) prioritizing these research questions in an actionable HPSR agenda.
The WHO framework “Equitable Access to Essential Medicines: A Framework for Collective Action”  was used as a guide. This framework presents four dimensions of ATM: rational selection and use, affordable prices, sustainable financing, and reliable health and supply systems. Each of these four dimensions is influenced by determinants of ATM rooted in local, national and international context .
The stepwise approach used to determine research priorities was adapted from Ranson et al. [20, 21] and is summarised in Figure 1. First, in 17 countries across five regions, research teams performed an exhaustive literature review, focusing on ATM in their countries and regions, and which covered published and grey literature in local as well as international databases. This was followed by key informant interviews (KIIs) and definition of priority policy issues. This resulted in a series of country and regional reports, which were supplemented by KIIs with global stakeholders. Second, a literature scoping review was conducted to identify existing published research, which was then mapped against the priorities identified at country-, regional- and global-level. The findings of the first two steps were shared with a group of experts during a consultative workshop aimed at formulating priority research questions and ranking these in a prioritized HPSR agenda.
Country-, regional- and global-level priority setting
Seventeen countries were selected by the AHPSR based on the following criteria: i) the capacity of in-country researchers and institutions; ii) their ability to approach a diverse group of stakeholders in a meaningful dialogue; and iii) their ability to adopt a health system perspective on ATM. Efforts were made to keep a balance between regions and to include both low- and middle-income countries. The analytical framework  was presented and discussed in detail with research teams at an inception meeting. Meeting participants also discussed how best to design and implement data collection tools that would capture all elements of the framework and determinants of access at all levels of the health system. Country teams were given general directions for steps to follow in collecting and analysing data. They were asked to: i) perform a search of grey and published literature related to their country and region, and identify existing information on priority policy issues and existing research in ATM; ii) map relevant stakeholders from various groups: researchers, policy makers, health service providers, civil society and NGOs including patient representatives, donor agencies, pharmaceutical companies and other relevant stakeholders; iii) select a sample of stakeholders with adequate representation of all groups; perform KIIs to identify priority policy issues and research questions; iv) derive from these priority policy issues on ATM and provide a list of research questions if possible. Each team conceived search and survey methods adapted to their contexts and in local languages. Table 1 presents the list of countries by region, the number of informants in each country and whether identification of priority policy issues was based on consensus meeting vs. ranking (either in ranking meetings or as a component of KIIs).
Institutions were requested to seek ethical approval in their own country or region. Data collection took place between November 2010 and December 2011 and country/regional reports were submitted to the AHPSR between May and December 2011b. The reports were read by two reviewers independently (MB and JH). Priority policy issues for each country were extracted and categorized according to the analytical framework. Results of independent data extraction were compared; discrepancies were discussed and resolved by returning to the original report. Cross-cutting priority policy issues appearing in at least three regions were identified and considered as being of high priority. Priority policy issues appearing in fewer than three regions were not considered as high priority.
During the data extraction process, some amendments were made to the original framework. The team observed that there was considerable overlap between issues related to domains “affordable prices” and “sustainable financing”. That is, when issues of affordability as a demand-side aspect were studied, naturally the supply-side, sustainable financing and implications, were also studied and vice versa. Thus, the two domains were merged into one, inclusive of both demand- and supply-side financing. Further, the domain “reliable health and supply system” covered a large number of policy issues and was therefore expanded and sub-domains defined. The final domains used were as follows:
Medicine selection and use
Sustainable financing and affordable prices of medicines
Leadership and governance (reliable health and supply systems)
Availability of medicines (reliable health and supply systems)
Human resources for health (reliable health and supply systems)
Quality of medicines and quality assurance systems (reliable health and supply systems)
Medicines information and information systems (reliable health and supply systems)
To collect data on priority policy issues from a global perspective, 23 stakeholders from headquarters and regional offices, of international multilateral and bilateral agencies, academia and NGOs were interviewed. These informants were selected for their expertise and knowledge of ATM issues at international level. Priority policy issues identified at country- and regional-levelc were used to design a semi-structured telephone interview guide. In relation to each of the seven domains of the revised framework and through a series of open-ended questions, informants were asked whether they agreed with priority issues defined at country- and regional-level, and whether any issue was missing and would deserve to be prioritized. Interviews were performed between June and November 2011 by researchers external to the AHPSR with 23 informants who responded in their professional capacity. Interviews were conducted in English following email agreement and oral consent at the start of the interview. Interviews were not recorded but transcribed; transcripts were sent back to the interviewees for final agreement. Transcripts were read by two independent readers (JH, who performed the interviews and DJ, who did not) and priority policy issues identified by global key informants were extracted in an Excel spread sheet, according to the modified analytical framework. Results of independent reviews were compared and discrepancies were discussed and resolved by returning to the original transcripts.
Policy issues were categorized according to the seven domains above. Topics were ordered firstly according to the number of regions in which the issue was raised and second according to the number of interviewees who raised the issue at global level.
Using the list of priorities identified through the KIIs and the country reports, a search strategy was devised to extract relevant literature. The search strategy was adapted from a review recently published by the AHPSR  and is presented in Additional file 1 and described below. As PubMed is limited in its inclusion of relevant LMIC literature, additional sources were considered. Country and regional teams conducted literature reviews on ATM in their local context, using international and local databases, resulting in a comprehensive reference list for each country and region. To supplement the global search, reference lists provided by country and regional teams were included to strengthen the inclusiveness of the literature scoping (see Additional file 2 for the list of databases searched by country, regional and global research teams). Finally, a report by Gray and Suleman as commissioned by the AHPSR was used as well to further broaden the search . These additional sources covered major databases for published and grey literature and databases specific to LMICs. From these, relevant publications and systematic reviews were included in the scoping exercise.
Literature search strategy
The main search theme included three concepts: medicines, LMICs, and access. These were combined with specific search terms related to each of the seven domains of the analytical framework. The global search was limited to PubMed 2003–2012 with additional references being drawn from country literature reviews through local and global databases highlighted in Additional file 2. Published papers were screened by two independent researchers and were included based on whether or not they were relevant to access to medicines, set in LMICs, and relevant to the priority areas. Results focusing on vaccines, research and development, legislative details of intellectual property rights, and pharmacoepidemiology were excluded as they were not outlined as priority ATM concerns at this time. Abstracts of included articles were also reviewed by two independent researchers and the following information was extracted for each publication: country; region; urban vs. rural setting focus; level at which the study was conducted, according to the same levels as previously outlined, namely i) patient, household, community, ii) health service delivery, health service provider, iii) health sector, iv) cross-cutting national policies, and v) international level; public vs. private sector focus; study methods: qualitative, quantitative, mixed, systematic review, other (including reviews and commentaries); whether the study presents impact or results of an intervention aimed at improving access to medicines; and which of the 18 policy issues was addressed in the article. Where studies were focused on more than one setting or more than one of the seven priority domains, this has been outlined.
Formulation of research questions and ranking
A global stakeholders meeting was organized in Bangkok on 6–8 March 2012. Participants from academia were selected based on their records of research and publications in the field of pharmaceuticals and health systems. Non-academia participants were selected to represent decision makers in the public and private sectors, which included multi-lateral agencies, government agencies, NGOs, pharmaceutical companies, and private foundations. The 20 participants included three who lead country-level exercises and five who were interviewed as part of the global stakeholders’ interviews. Participation was voluntary and based on availability. In preparation for the meeting, participants received summaries of individual country- and regional-level priority setting exercises, a background paper detailing the methods and results of country-, regional- and global-level priority setting as well as methods and results of literature scoping, and a spreadsheet presenting the full list of references including abstracts. During the first half of the meeting, 14 participants from academia reviewed the list of priority policy issues resulting from country-, regional-, and global-level research; and agreed on the categories as well as the wording. Once agreement was reached, they formulated research questions corresponding to each of the ATM policy issues identified in the first two steps. This exclusive time for formulation of research questions was designed to overcome the difficulty of translating policy issues into health policy and systems research questions, which required the expertise of researchers. During the second half of the meeting, non-academic participants joined the group, reviewed the list of policy issues, and drafted questions formulated by academia. Consensus was reached on a final set of 18 unranked research questions.
Participants also debated and reached consensus on four ranking criteria, to which they decided to attribute the same weight:
Innovation: Will the research stimulate innovation?
Impact: Is the resulting research likely to be effective in strengthening health systems and improving health?
Equity: Is the research likely to reduce disparities in health outcomes, perceived as unfair?
Lack of research: Is there a lack of research on this topic?
In addition, participants suggested and agreed to add an overall score, irrespective of ranking criteria, that would assess the overall importance of each research question in LMICs and proposed that each participant lists his/her top five priority research questions at the end of the questionnaire. The questionnaire was modified to account for these suggestions.
Participants ranked the 18 research questions through an individual anonymous questionnaire. A five-point Likert scale was used for each ranking criteria where one represented low priority and five represented high priority. Three different methods were used for combining scores across participants:
First, the “total score” involved: i) for each participant and question, summing the four ranking criteria (given the consensus decision to assign equal weight to each criterion); and ii) taking the flat (non-weighted) average of scores for each question across all participants. Questions were ranked from one (top rank, highest average total score) to 18 (lower rank, lowest average total score).
Second, the “overall score” was entered also using a five-point Likert scale where one represented a low overall priority and five represented a high overall priority. The overall score was also combined by taking a flat average for each question, across all participants. Questions were ranked from one (top rank, highest average overall score) to 18 (lower rank, lowest average overall score).
Third, the “top five” was a simple count of the number of times each research question was listed in the top five priorities of all participants and questions were ranked from one (highest number of counts) to 18 (lowest number of counts).
The results of the ranking exercise were analysed on site, presented and discussed in plenary, and consensus was reached on top priority questions.
Policy concerns raised at country-, regional-, and global-level
A total of 26 crosscutting high priority policy issues were identified at country- and regional-level. Nine policy issues were consistently raised in all five regions and related to: i) perception of generic medicines by patients, prescribers and providers, and their preference for branded medicines; ii) knowledge, awareness, and health seeking behaviour of patients; iii) development of central and essential policy documents such as standard treatment guidelines and national essential medicine lists; iv) universal coverage, social health protection and coverage of medicines under health financing arrangements; v) level of public and government funding for health and medicines; vi) inefficiencies in budget disbursement and implementation; vii) medicine pricing policies; viii) capacity of staff and systems for procurement, supply and stock management; and ix) fight against counterfeit medicines. An additional eleven issues were raised in four regions and six issues were raised in three regions. Policy issues raised by less than three regions are considered as low priority and are not presented here.
A total of 23 informants were interviewed at global level, of which six were women. Ten informants originated from a LMIC. Interviewees had expertise either in pharmaceuticals or health systems and worked at headquarters of international multilateral agencies (n = 8), at regional offices of multilateral agencies (n = 4), in a bilateral donor organization (n = 1), in international NGOs (n = 4) and six were researchers in academic institutions. The results of global KIIs were fairly consistent with country- and regional-level results. Global KIIs placed more emphasis on the role of human resources for health in medicine access, although for both groups, the specific policy focus was the impact of financial and non-financial providers’ incentives on access. The second most important domain for global key informants was the same as for countries and regions: sustainable financing and affordable prices. Consistent with data from countries and regions, quality issues came third with global key informants; however, here, a focus on substandard medicines was considered to be more critical than counterfeit medicines as these were seen as a symptom of other system failures outside of quality assurance. In general, all priority policy issues that were identified consistently in all five regions were presented as important concerns for global key informants.
A consolidation of results from country, regional, and global exercises allowed authors to define a final list of 18 crosscutting high-priority policy issues related to ATM in LMICs. Inclusion of priority policy issues in this list was based first on the number of regions where the issue was raised and second according to the number of informants who raised the issue (Table 2). As the results of Global KII were consistent with country- and regional-level results, no additional policy issues raised by Global Key Informants were added to the list of country and regional priorities. A few policy issues among the list of 26 priorities resulting from country- and regional-level exercises were combined as they appeared redundant or strongly linked (e.g., “provider payment methods and financial incentives to providers” was combined with “financial and non-financial incentives to providers”; “procurement and supply of generic medicines” was combined with “quality of generic medicines”).
A total of 1,279 articles were retrieved from PubMed; 619 were found relevant for our study. Results show a distribution of publications predominantly in Sub-Saharan Africa (32.1%), followed closely by the Asian continent (23.6%, with 12.6% in South Asia and 11% in East Asia and the Pacific), and Latin America and the Caribbean (18.6%). A majority of publications focused on the public sector (65.4%) followed by both public and private sector (8.7%); 15.8% of publications studied only the private sector. Half of the publications had a specific urban focus and 12% specifically targeted rural settings.
From reference lists of country and regional priority-setting reports, an additional 72 articles were identified as relevant to the search. Sixteen systematic reviews were also extracted from the review by Gray and Suleman . A total of 707 references were categorized, based on screening through each article, according to the list of priority policy issues in ATM (Table 3).
Priority policy issues for which there are fewer references in the literature include resources allocation for health and medicines; deployment and shortage of human resources for health; counterfeit medicines; procurement, supply and stock management; and crosscutting policies outside the health sector affecting health and medicines. Moreover, some of the priorities are only covered by less robust research methodologies. For example, for the following five priorities, no publication was identified that attempts to assess the effects or outcomes of specific interventions: resource allocation for health and medicines; health seeking behaviour of patients, households and communities; crosscutting policies outside the health sector affecting health and medicines; governance over the private sector and financial and non-financial incentives for providers.
Formulation of research questions and ranking
Fourteen academic researchers were present during the first half of the meeting, including one private foundation and three researchers who had performed country- and regional-level priority setting exercises. They came from institutions in Brazil, China, Ghana, Iran, Mexico, South Africa, Switzerland, Sweden, Thailand, and the United States of America. Seven stakeholders joined the academic group during the second half of the meeting; they came from international multilateral agencies (n = 2), a government agency (n = 1), NGOs (n = 2), a pharmaceutical company (n = 1), and a private foundation (n = 1). Two WHO officers were also present. Eight participants were female. Research questions addressing the 18 priority policy issues were formulated by the academic researchers, and then reviewed by the second group of stakeholders who introduced minor editorial revisions. While the experts did have the option of developing more than one research question per issue, they jointly decided on a single question corresponding to each policy issue. Table 4 presents the research questions formulated by the group and the results of the ranking exercise according to the three ranking methods. It also disaggregates the results according to the four ranking criteria (Innovation, Impact, Equity, and Lack of Research).
Participants in the ranking exercise were presented with preliminary quantitative analyses of all three ranking methods. They reached a consensus that priority should be given to research questions 1, 12 and 18.
Question 1 (“In risk protection schemes, which innovations and policies improve equitable access to and appropriate use of medicines, sustainability of the insurance system, and financial impact on insurance members?”) was overwhelmingly ranked as a top priority, irrespective of the ranking method used. This question also scored as a high priority according to all ranking criteria (top priority with regards to Impact and Lack of Research, second priority with regards to Innovation and Equity). This research topic corresponds to a priority policy issue raised consistently in all five regions and by 18 out of 23 global key informants (Table 2). As confirmed by our literature scoping, medicine publications have only recently shifted towards examining health-financing arrangements and investigating medicine prices in LMICs. Our priority setting exercise confirms an on-going interest in these topics, which are also consistent with the current trend of focussing on universal coverage in a post-Millennium Development Goal world.
The top five ranking method placed Question 18 (“How can stakeholders use the information available in the system, e.g., price, availability, quality, utilization, registration, and procurement, in a transparent way towards improving access and use of medicines?”) in second position while the results of the total score and overall score methods pointed to Question 12 (“How do policies and other interventions into private markets (such as information, subsidies, price controls, donation, regulatory mechanisms, promotion practices, etc.) impact on access to and appropriate use of medicines?) as a second priority. Question 12 was considered highly innovative and addressing a substantial research gap, as confirmed by our literature scoping. Question 18, related to medicine information, was regarded as potentially achieving high impact.
Consensus was reached among the meeting participants for the above top three ranking research questions that needed attention and investment in the future. Participants suggested that the results of the ranking, disaggregated by ranking criteria, be made publicly available to allow researchers, decision makers and funders to select research questions according to their expertise and their needs, depending on whether they wanted to focus on equity, innovation, impact on health and health systems or on filling existing research gaps.
A list of 18 research questions corresponding to policy issues in ATM raised at country-, regional-, and global-level were thoroughly formulated and ranked during our priority-setting exercise. The prioritized HPSR agenda in ATM corresponds to existing research gaps and sets the tone for future research that is likely to be innovative and to have a higher impact on health, health systems, and equity. The top ranking research questions relate to medicine financing (“In risk protection schemes, which innovations and policies improve equitable access to and appropriate use of medicines, sustainability of the insurance system, and financial impact on insurance members?”); medicine information and information systems (“How can stakeholders use the information available in the system, e.g., price, availability, quality, utilization, registration, and procurement, in a transparent way towards improving access and use of medicines?”); and to leadership and governance, and more specifically, stewardship over the private sector (“How do policies and other interventions into private markets (such as information, subsidies, price controls, donation, regulatory mechanisms, promotion practices, etc.) impact on access to and appropriate use of medicines?). This priority-setting approach has been successful in ensuring that a health systems lens is adopted in ATM research : the top priority research questions are tightly interlinked with three other building blocks of the health system, namely health financing, health information systems and governance . Moreover, all three research questions are related to the building block of service delivery. The other research questions highlight the importance of incentives in the health system, demand-side and equity aspects, and contextual factors outside the health sector.
Viergever et al. have established a checklist of nine common themes of good practice for health research priority setting . Our priority-setting exercise was assessed against this checklist as follows:
Context: This priority-setting exercise was the major activity of the inception phase for the ATM project launched by the AHPSR in 2010 (http://www.who.int/alliance-hpsr). The objectives of the project are to increase availability and use of HPSR evidence in the field of ATM, and thereby improve access for the most vulnerable populations in LMICs. Adequate level of funding and human resources were provided by the AHPSR since the start of the project and it was foreseen that a bottom-up exercise would be conducted in select countries to generate context-specific evidence on policy issues and priority research questions.
Stepwise comprehensive approach: We adopted a stepwise approach, similar to previous priority-setting exercises conducted by the AHPSR. This is consistent with the approach used by Ranson and Bennett , who conclude that more successful approaches for considering HPSR priority-setting are country-driven, interpretive and include a wide range of stakeholders. Consistent with this, the Taskforce on Health Systems Research  also puts forward approaches to setting priorities in HPSR, based on consensus views of informed participants.
Inclusiveness: Our study starts with a bottom-up, country-driven approach and all prioritised research questions respond to policy issues in ATM raised by countries and regions. We have also paid particular attention to the inclusion of typically neglected, but significant actors such as the pharmaceutical industry.
Information gathering: Data collection at country-, regional-, and global-level included KIIs on priority policy issues in ATM as well as a comprehensive literature scoping.
Plans to translate priorities into research: From the start, it was known that the AHPSR had available funding for a call for research proposals based on established HPSR agenda (http://www.who.int/alliance-hpsr/alliancehpsr_atmcall_for_eoi.pdf)d.
Criteria: In all priority-setting exercises, whether for health interventions, health research, or HPSR, authors insist on the importance of criteria used for ranking [24–29]. The four criteria we used for our priority-setting are considered as relevant in many of the above publications: equity is a common criterion to almost all priority-setting papers above, the expected impact of research on health and health systems is considered by several authors as well as lack of existing and previous research [20, 21, 24]. Innovation is seldom used by other authors, however, Ranson et al. observe that the research questions ranked in their study on health financing were usually not very innovative ; introducing innovation as a criteria helps to overcome this specific weakness of previous AHPSR priority-setting exercises. Ranson et al. [20, 21] concur with COHRED  that ranking criteria should be based on consensus among participants and this is especially relevant for HPSR . We have applied this by discussing and adjusting ranking criteria prior to conducting the ranking exercise; stakeholders suggested two alternate ranking methods (overall score and top five list) which were used to discuss the final prioritized agenda.
Methods: Viergever et al.  suggest that priority-setting methods may be consultative-based, metrics-based, or a combination of both, while the Taskforce on Health Systems Research  and Ranson and Bennett  support the use of qualitative methods to prioritize HPSR agendas. Our method was a mixed approach, combining qualitative data collected at country-, regional-, and global-level, a consensus-based approach to formulating the final research questions, a metrics-based approach to the ranking exercise, and finally, consensus-building towards the final agenda.
Transparency: The entire process has been transparent with each step having been documented starting with country- and regional-level reports and ending with a full research report following the ranking workshop.
Evaluation: The process has not yet undergone any formal evaluation, but is assessed against the checklist of nine common themes of good practice for health research priority-setting as established by Viegever et al. .
Particular attention has been devoted to mitigating some of the weaknesses of prior priority-setting exercises [20, 21]. The lack of standardized methodology for country- and regional-level exercises is one such limitation. To address this, some level of standardization was introduced by holding an inception meeting prior to field data collection (Cambodia, October 2010). Extensive discussions took place to understand the analytical framework, identify determinants of ATM at local, national and international levels that should guide the data collection process and map stakeholders. However, no further standardization was introduced beyond the conceptual level in order to provide sufficient flexibility for the development of context-specific methodologies, in particular with regard to survey tools. In previous exercises, it was documented that key informants had difficulty formulating research questions, and felt more comfortable discussing policy issues. In response to this limitation, exclusive time has been devoted to formulation of research questions by experienced academic researchers during the consultation and ranking workshop. On the other hand, using the expertise of academic experts gathered in a global meeting to formulate research questions may bias the questions towards global level research priorities; however, the presence of researchers from LMICs in the meeting, especially lead country and regional investigators, mitigated such biase. Another limitation of the formulation of research questions was that an in-depth triangulation of research priorities with existing research could not take place; instead academic experts referred to a scoping of the literature, with limited consideration of quality of published papers. This was combined with their own knowledge of available research in the field.
As in any other exercise of this kind, the choice of the frameworks guides the way in which policy concerns are voiced during KIIs. The WHO framework  is largely centred on health sector level issues. Such limitation was mitigated by the stepwise comprehensive and inclusive approach adopted for data collection, especially mapping and inclusion of stakeholders at country-, regional-, and global-level.
An intermediate result of our research consists of 17 country-level priority-setting exercises that can be used by national decision-makers to identify important gaps in their pharmaceutical policies and programmes. Summaries of country- and regional-level results are available on the AHPSR website and country research teams pursue their efforts of disseminating these results in national and international level, including in peer-reviewed publications [30–32]. Further, the proposed prioritized HPSR agenda in ATM comprises 18 priority research questions, of which only three have been selected as top priority for the AHPSR Call for Expressions of Interest. Obviously, this investment is limited. Academic institutions at national and international level as well as funding agencies may use this research agenda to orient their efforts and investments towards policy-relevant research in ATM. Being a step-wise priority-setting process, inclusive of a range of stakeholders, including donors, this exercise serves to support policy-relevant research by demonstrating the need and desire for it across stakeholders and by bringing together those interested in the field, leading to improved consolidation of both research and policy-making efforts. Several agencies present at the Global Stakeholders Meeting have already expressed willingness to support this research agenda.
The authors hope that this paper will contribute to the overall science – both in terms of methodological approaches and findings – of priority setting for investment in health research. It is clear, however, that much remains to be done in strengthening the field of priority-setting. First, many organizations funding health or development research require broader lists of priorities as they may need to allocate research across sectors or broadly within the health sector. A broad, cross-sectoral priority-setting exercise might be particularly relevant as the international community establishes – and identifies strategies for addressing – the post-2015 development goals . Second, in order for national-level policy-makers to decide how to spend scarce research resources, priority-setting exercises may need to be carried out at the country-level. This requires a commitment to both building up the skills for conducting priority-setting exercises, and using their results. Relatively few LMICs regularly conduct priority-setting exercises for health research (the most cited examples being Malaysia and Brazil). Third, priority-setting exercises at global level tend to look at demand for, and supply of, research evidence at a single point in time. In fact, the issues of greatest priority to policy-makers and the corresponding evidence base are fast changing. This calls for priority-setting processes that are dynamic (“real-time”), or at least revised at regular intervals.
The priority-setting process described in our paper established a HPSR agenda in ATM, with 18 prioritized and thoroughly formulated research questions. The top-priority questions focused on: i) medicine financing within risk protection schemes; ii) medicine information; and iii) interventions into private markets. The agenda adopts a health systems perspective and will guide innovative research likely to bear a higher impact on health, health systems, and equity.
aAccording to the WHO, “Essential medicines are those that satisfy the priority health care needs of the population. They are selected with due regard to public health relevance, evidence on efficacy and safety, and comparative cost-effectiveness.” [http://www.who.int/topics/essential_medicines/en/] Accessed 3 March 2013.
bSummaries of country- and regional-level priority setting exercises can be found at the following link [http://www.who.int/alliance-hpsr/projects/medicines/en/index.html].
cThis excluded Rwanda and Ghana which started later than other countries and had not submitted their draft report when the global-level interviews were designed.
dAccessed 18 January 2013.
eIt should be noted that lead country and regional investigators did not participate in the ranking exercise.
MB is responsible for the Alliance HPSR’s programme of work on Access to Medicines, including primary research in LMICs based on the top priority research questions identified in this study.
Alliance for Health Policy and Systems Research network of researchers on Access to Medicines (in alphabetical order): Daniel Arhinful (Noguchi Memorial Institute for Medical Research, University of Ghana); Samer Jabbour (American University of Beirut, Lebanon); Vera Lucia Luiza (National School of Public Health, Oswaldo Cruz Foundation, Brazil); Chean Men (Centre for Advanced Studies and Chean-Jaco Co, Cambodia); Chuc Nguyen Thi Kim (Hanoï Medical University, Vietnam); Joseph Ntaganira (School of Public Health, National University of Rwanda); Claudine Ntsama Essomba (University of Yaoundé I, Cameroon); Arash Rashidian (Tehran University of Medical Sciences, Iran); Sakhtivel Selvaraj (Public Health Foundation of India); Lamphone Syhakhang (Ministry of Health, Lao PDR) and Shehla Zaidi (Agha Khan University, Pakistan).
Alliance for Health Policy and Systems Research
Access to medicines
Health Policy and Systems Research
Key informant interviews
Low- and middle-income countries.
World Health Organization: Everybody’s Business: Strengthening Health Systems to Improve Health Outcomes. WHO’s Framework for Action. 2007, Geneva: World Health Organization
Cameron A, Ewen M, Ross-Degnan D, Ball D, Laing R: Medicine prices, availability, and affordability in 36 developing and middle-income countries: a secondary analysis. Lancet. 2009, 373: 240-249. 10.1016/S0140-6736(08)61762-6.
World Health Organization: Medicines Strategy: Countries at the Core. 2004–2007. 2004, Geneva: World Health Organization
Mills A, Brugha R, Hanson K, Mcpake B, Public Health Reviews: What can be done about the private health sector in low-income countries?. Bull World Health Organ. 2002, 80: 325-330.
Smith F: The quality of private pharmacy services in low and middle-income countries: a systematic review. Pharm World Sci. 2009, 31: 351-361. 10.1007/s11096-009-9294-z.
Cockburn R, Newton PN, Kyeremateng Agyarko E, Akunyili D, While NJ: The global threat of counterfeit drugs: why industry and governments must communicate the dangers. PLoS Med. 2005, 2 (4): e100-10.1371/journal.pmed.0020100.
Burki T: The real cost of counterfeit medicines. Lancet Infect Dis. 2010, 10: 585-586. 10.1016/S1473-3099(10)70173-0.
Newton PN, Amin AA, Bird C, Passmore P, Dukes G, Tomson G, Simons B, Bate R, Guerin PJ, White NJ: The primacy of public health considerations in defining poor quality medicines. PLoS Med. 2011, 12: e1001139-
World Health Organization: World Medicines Situation. 2004, Geneva: World Health Organization
Taskforce on Health Systems Research: Informed choices for attaining the millennium development goals: towards an international cooperative agenda for health-systems research. Lancet. 2004, 364: 997-1003.
Adam T, Ahmad S, Bigdeli M, Ghaffar A, Rottingen JA: Trends in health policy and systems research over the past decade: still too little capacity in low-income countries. PLOS One. 2012, 6: 1-10.
World Health Organization: World Report on Knowledge for Better Health. 2004, Geneva: World Health Organization
Ranson MK, Bennett SC: Priority setting and health policy and systems research. Health Res Policy Syst. 2009, 7: 27-10.1186/1478-4505-7-27.
Commission on Health Research for Development: Health Research: Essential Link to Equity in Development. 1990, Oxford: Oxford University Press
Kaplan W, Laing R: Priority Medicines for Europe and the World. 2004, Geneva: World Health Organization
World Health Organization: Priority Medicines for Mothers and Children. 2011, Geneva: World Health Organization
Proceedings of the International Conference on Improving the Use of Medicines: Topic Summaries. 2011, [http://www.inrud.org/icium2011-documents/upload/ICIUM2011-Theme-Summaries-16APRIL2012.pdf] Accessed 18 January 2013
World Health Organization: Equitable Access to Essential Medicines: A Framework for Collective Action, Policy Perspectives on Medicines. 2004, Geneva: World Health Organization, 1-6. 8
Bigdeli M, Jacobs B, Tomson G, Laing R, Ghaffar A, Dujardin B, Van Damme W: Access to medicines from a health system perspective. Health Policy Plan. 2012, [Epub ahead of print]
Ranson K, Law TJ, Bennett S: Establishing health systems financing research priorities in developing countries using a participatory methodology. Social Sci Med. 2010, 70: 1933-1942. 10.1016/j.socscimed.2010.01.051.
Ranson MK, Chopra M, Atkins S, Dal Poz MR, Bennett S: Priorities for research into human resources for health in low- and middle-income countries. Bull World Health Organ. 2010, 88: 435-443. 10.2471/BLT.09.066290.
Gray A, Suleman F: Overview of Systematic Reviews on Pharmaceutical Policy and their Relevance to Low- and Middle-Income Countries. Report prepared for the Alliance for Health Policy and Systems Research. 2009, Geneva: World Health Organization
Viergever RF, Olifson S, Ghaffar A, Terry RF: A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst. 2010, 8: 36-
The Working Group on Priority Setting; Council of Health Research for Development (COHRED): Priority setting for health research: lessons from developing countries. Health Policy Plan. 2000, 15: 130-136. 10.1093/heapol/15.2.130.
Baltussen R, Niessen L: Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Effectiveness Resour Allocation. 2006, 4: 1-9. 10.1186/1478-7547-4-1.
Kapiriri L, Norheim OF: Criteria for priority-setting in health care in Uganda: exploration of stakeholders’ values. Bull World Health Organ. 2004, 82: 172-179.
Lavis JN, Lomas J, Hamid M, Sewankambo NK: Assessing country-level efforts to link research to action. Bull World Health Organ. 2006, 84: 620-628. 10.2471/BLT.06.030312.
Oxman AD, Schünemann HJ, Fretheim A: Improving the use of research evidence in guideline development: 2. Priority setting. Health Res Policy Syst. 2006, 4: 14-10.1186/1478-4505-4-14.
Tromp N, Baltussen R: Mapping of multiple criteria for priority setting of health interventions: an aid for decision makers. Health Serv Res. 2012, 12: 454-10.1186/1472-6963-12-454.
Emmerick ICM, Oliveira MA, Luiza VL, Azeredo TB, Bigdeli M: Access to medicines in Latin America and the Caribbean (LAC): a scoping study. BMJ Open. 2013, 3 (5): e002224-10.1136/bmjopen-2012-002224.
Zaidi S, Bigdeli M, Aleem N, Rashidian A: Access to essential medicines in Pakistan: policy and health systems research concerns. PLoS One. 2013, 8 (5): e63515-10.1371/journal.pone.0063515.
Rashidian A, Jahanmehr N, Jabbour S, Zaidi S, Soleymani F, Bigdeli M: Bibliographic analysis of research publications on access to and use of medicines in low-income and middle-income countries in the Eastern Mediterranean Region: identifying the research gaps. BMJ Open. 2013, 3: e003332-10.1136/bmjopen-2013-003332.
United Nations Millennium Development Goals. [http://www.un.org/millenniumgoals/beyond2015.shtml] Accessed 3 March 2013
The authors sincerely thank all participants of the Global Stakeholders Meeting in Bangkok 6–8 March 2012. Three outstanding interns helped with various phases of the literature scoping and global stakeholders’ interviews, namely Ray Deng, Brian Saindon and Sandra Hwang.
The authors declare that they have no competing interests.
MB conceptualized the study; coordinated data collection and data analysis at country- and regional-level; conceived and coordinated data collection at global-level and analysed the results; conceived and facilitated the final stakeholders consensus meeting; analysed ranking results; and wrote the manuscript. DJ contributed to data analysis at country-, regional- and global-level; conceived and conducted the literature scoping; facilitated the final stakeholders’ consensus meeting; analysed ranking results; and contributed to writing the manuscript. JH extracted data from country and regional reports; conducted data collection and data analysis at global-level; and commented and approved the manuscript. RL provided advice on the study design, conceptual framework, data collection at country-, regional- and global-level; and facilitated the final stakeholders’ consensus meeting and analysis of ranking results. He has read and approved the manuscript. KR has contributed to conceiving the consensus building process; facilitated the final stakeholders’ consensus meeting; analysed the ranking results; and contributed to writing the manuscript. The AHPSR network of researchers on ATM conceived and conducted the country and regional-level priority-setting exercises and wrote the corresponding research reports. All authors read and approved the final manuscript.