Our approach combined a bibliometric analysis of health economics research outputs produced in the four target jurisdictions with an online survey of academic and non-academic health professionals. The bibliometric analysis was conducted first in order to provide an initial overview of health economics activity in terms of topics and active organisations; its main findings then informed the design and dissemination of the online survey.
Bibliometric analysis
The bibliometric analysis focused on the question – ‘What type of health economics research has been produced in Jordan, Lebanon, OPT and Turkey between 1 January 2014 and 31 December 2018?’ There were two specific questions:
-
Which health economics topics have been researched by organisations in the four jurisdictions, and to what extent?
-
Which organisations have published health economics research and what are their co-publishing patterns, both within and outside the Middle East and North Africa region?
Data sources
Seven electronic databases were searched on 28 January 2019 – two databases with an economic focus (Econlit–Ovid and NHSEED), three health-focused databases (MEDLINE, including MEDLINE In-Process, EMBASE Classic and Embase, and Global Health – all searched through Ovid), two generalist databases (Scopus and Web of Science) and one regional database (Index Medicus for the Eastern Mediterranean Region).
Search strategy
A custom literature search strategy was devised with the help of an information specialist at Imperial College London Medical School Library. The search strategy comprised two types of search terms – geographic descriptors (i.e. the names of the four jurisdictions searched in the ‘institution’ field of each database) and subject descriptors (i.e. keywords pertaining to health economics). We tested four search strategies for health economics research outputs, informed by previous bibliometric reviews [13, 14, 15] and reference textbooks (details in Additional file 1), and selected a search strategy that was then adapted and ran across seven databases (full search strategies in Additional file 1).
Eligibility criteria
The following inclusion criteria were applied: scope – titles or abstracts that explicitly listed any economic outcome related to health, disease or disability, including but not limited to cost, cost-effectiveness, patient-reported outcomes, health expenditure, provider or health system performance, and poverty associated with accessing healthcare; geography – studies with at least one author affiliated with an organisation in Jordan, Lebanon, OPT (West Bank and Gaza) or Turkey; language – studies in any languages; publication date – studies published between 1 January 2014 and 31 December 2018; publication type – studies published as journal articles (original articles and reviews), book chapters, conference proceedings and theses. Editorials were excluded.
Study selection
Search results from each database were downloaded in Microsoft Excel and duplicates were removed. The following fields were retained for each record: title, abstract, authors, authors’ affiliations, publication year and publication journal/source. Titles and abstracts were screened by the lead researcher (AG). Institutional affiliation data were cleaned manually for the records meeting the inclusion criteria. For the authors’ institutional affiliation, only the highest-level affiliation was retained (e.g. Department of Internal Medicine, Faculty of Medicine, University of Buenos Aires was recorded as University of Buenos Aires).
Data analysis
Descriptive analyses were conducted in Microsoft Excel and bibliometric analyses were conducted using VOSviewer software version 1.6.10 [16]. Two types of analyses were conducted using VOSviewer functionalities – keyword co-occurrence and co-publication networks. Keyword co-occurrence was analysed based on VOSviewer’s built-in natural language processing algorithms, which were applied to the titles and abstracts of included records [17]. The software allows a similar analysis of author and database-indexed keywords, but a deliberate decision was made to conduct the analysis only on titles and abstracts to ensure that the body of text was as uniform as possible across sub-disciplines, journals and indexing databases as well as to minimise any systematic differences in the authors’ choice of keywords. The binary counting method was used to assess the frequency of terms, whereby a given word (sequence) is counted only once per record irrespective of how many times it appears within it.
For co-publication networks, both organisations and jurisdictions were analysed based on the authors’ affiliations. When there were more authors from a single organisation, the organisation was counted only once. The fractional counting method was used to ascertain country and organisation authorship, whereby the strength of a co-authorship link between two organisations is determined not only by the number of records co-authored but also by the total number of organisations of each of record – this method reduces the influence of large collaborations. For example, if a study is authored by authors from five different organisations, each link between two co-authoring organisations gets a strength of 1/5 (versus a strength of 1 if the full counting method had been employed) [18].
Results are presented as network visualisations, where each circle depicts a keyword (or institution); the size of the circle is proportional with the number of records mentioning that keyword (or authored by someone at that institution). The closer two keywords (institutions) appear and the thicker the links between them and the larger the number of records in which they appear (co-author) together. The colours distinguish between clusters of keywords (institutions), i.e. collections of keywords (institutions) that tend to appear (publish) together. A keyword (institution) can only belong to one cluster.
Online survey
The survey focused on the following question: ‘What are the capacity development priorities for health economics among academic and non-academic professionals in Jordan, Lebanon, OPT and Turkey?’ There were three specific questions:
-
Which health economics topics have respondents received training in and applied in research or analysis, respectively, over the past 5 years?
-
Which health economics topics are respondents most interested to develop in the future?
-
What are the respondents’ preferred modalities to develop their health economics skills and knowledge?
Survey sample
Survey respondents were identified through several means. Firstly, the email addresses of the authors of the 566 records included in the bibliometric analysis were identified based on information publicly available online on their respective institutions’ and departments’ websites. Secondly, R4HC project partners were consulted on the relevant academic and non-academic institutions/departments to approach in each country. Thirdly, representatives of relevant regional organisations (e.g. WHO Office for the Eastern Mediterranean Region) and in-country professional associations (e.g. local chapters of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR)) were approached for additional relevant professionals/organisations and for support with disseminating the questionnaire within their networks. A consolidated list of potential respondents was created by combining these three sources. Only researchers affiliated with university departments of public health, pharmacy, health economics, health policy and economics were retained as these departments were judged to be likely hosts of institutional health economics expertise.
Recruitment
Individuals on the consolidated recruitment list were sent an email invitation to participate in the survey; the recruitment email explained the purpose of the study and invited recipients to participate by clicking the enclosed weblink, which lead to the online questionnaire. The recruitment email also encouraged recipients to forward the recruitment email to colleagues for whom health economics is a discipline relevant to their professional role. A total of 286 unique individuals across the four target jurisdictions and the region were sent the invitation email from 9 September 2019. A single reminder email was sent to all recipients around 20 September 2019.
Survey instrument
The survey instrument (Additional file 1) distinguished between two tracks based on the respondents’ professional role, as reported in the first question. The technical track applied to respondents self-identifying as academic researchers, technical analysts (non-academic) and clinicians); the 14 questions focused on detailed health economics methods. The managerial track applied to respondents self-identifying as policy managers or healthcare administrators; the 12 questions focused on the application of health economics methods to answer higher-level policy questions. In questions about past exposure to health economics topics, respondents were presented with multiple choice items, e.g. ‘no exposure to the topic’, ‘I have worked on this topic’ or ‘I am an expert in this topic’. In questions concerned with future priorities in terms of health economics capacity development, respondents were asked to rank the items (using the drag and drop functionality of the survey platform) from the most important to the least important.
This approach was preferred to the Likert-scale rating approach used by the WHO Hennessy-Hicks needs assessment tool [19], which compares, for a given skill, the self-reported achievement and the perceived importance of the skill for the professional role, for two reasons, namely health economics is a niche discipline in the public health space and preliminary discussions with R4HC consortium partners and other academics in Jordan, Lebanon and Palestine indicated that there were very few ‘pure’ health economists in these settings. As such, a deliberate decision was made to broaden the professional profiles of the response sample while acknowledging that their professional role specifications (where these are available) may not include explicit health economics competencies.
The draft survey (English version) was piloted with five researchers (including two health economists) in the Global Health Development group at Imperial College London and five health systems researchers with regional experience (including four health economists), following which modifications were made based on their feedback. The English version was then translated into Arabic and Turkish by native speaker health economists. Based on the pilot we anticipated that completing the survey would take between 10 and 15 minutes; this information was included in the participant information sheet linked to the recruitment email ‘[the survey] should take about 10–12 minutes to complete’. No incentives were offered to respondents for completing the survey.
Data collection
Survey responses were collected using the Qualtrics platform through the Principal Investigator’s Imperial College’s secure account. The online questionnaire was open for a period of 5 weeks (9 September to 14 October 2019), following which it was locked and responses were no longer accepted. Each page of the online questionnaire allowed responses to be collected in English, Turkish or Arabic based on the translated versions, as explained above.
Data analysis
Survey responses were downloaded in spreadsheet format for cleaning, following which they were transferred to R statistical software for analysis [20]. Partial responses were kept on the Qualtrics platform, but not downloaded for analysis. Responses to each question were summarised using appropriate descriptive statistics (e.g. proportions) and by subgroups (e.g. by country, by type of institution).