- Open Access
- Open Peer Review
The state of international collaboration for health systems research: what do publications tell?
Health Research Policy and Systemsvolume 4, Article number: 7 (2006)
International collaboration for health system development has been identified as a critical input to meet pressing global health needs. North-South collaboration has the potential to benefit both parties, while South-South collaboration offers promise to strengthen capacity rapidly and efficiently across developing countries. There is an emerging trend to analyze the fruits of such collaboration. This paper builds on this trend by applying an innovative concept-based bibliometric method to identify the international scope of collaboration within the field of health policy and systems research. Two key questions are addressed: to what extent are papers comparing developing countries as against reporting on single country studies? To what extent are papers in either case being produced by researchers within their respective countries or through North-South or South-South collaboration?
A total of 8,751 papers published in Medline between 1999 and 2003 with data on health systems and policies in developing countries were identified and content-analyzed using an innovative concept-based search technology. A sample of 13% of papers was used to identify the corresponding institution and countries covered. The sampled data was then analyzed by income group.
Papers with an international, cross-country focus account for only 10% of the total. Just over a third of all papers are led by upper middle income country authors, closely followed by authors from high income countries. Just under half of all papers target low income countries. Cross-country papers are led mostly by institutions in high income countries, with 74% of the total. Only seven countries concentrate 60% of the papers led by developing country institutions. Institutions in the United States and the United Kingdom concentrate between them as many as 68% of the papers led by high income countries. Only 11% of all single-country papers and 21% of multi-country studies are the product of South-South collaboration. Health Financing is the topic with the greatest international scope, with 26% of all papers in the topic. Topics such as Costing and Cost Effectiveness, Finance, Sector Analysis and Insurance, regardless of their national or international scope, are led in 38% to 54% of cases by high income authors.
While there is modest health systems research capacity in many developing countries for single country studies, capacity is severely limited for multi-country studies. While North-South collaboration is important, the number of international studies is still very limited to produce the kind of knowledge required to learn from experiences across countries. The fact that lead institutions as well as study countries are concentrated in a handful of mostly middle income countries attests to great disparities in research capacity. However, disparities in research capacity and interest are also evident in the North. It is urgent to build cross-country research capacity including appropriate forms of South-South and North-South collaboration.
Collaboration for health research between developed (North) and developing (South) countries holds the potential of generating knowledge and strengthening the capacity of all parties concerned. Most North-South collaboration has been driven by developed countries and purports to benefit partners in the South . However, collaborative health services research conducted in developing countries has led to significant practical and philosophical influences for health systems in the developed world [2, 3]. Cross-country analysis including North and South countries has been promoted as a means of benchmarking the performance of health systems internationally, identifying limitations in specific functions and spurring improvements . Comparative health systems research can also be a tool for shared learning from health sector reforms . International research is a means to learn across countries and to transfer and scale-up successful interventions. The ethical dilemmas in North-South collaboration have been explored [6, 7] and the best ways of approaching such collaboration have also been assessed.
International health systems research is today more necessary than ever. Global funding agencies and initiatives for disease control such as the Global Fund for AIDS, TB and Malaria and the Presidential Initiative PEPFAR are exerting increasing influence on health system strengthening and important questions are being asked as to their impact at country level . Comparative and international health systems research in developing countries can be an important tool to steer such global efforts. Strategic health systems research that aims both to solve problems and to generate knowledge of international significance can benefit from research designs involving multiple countries. Importantly, strategic research capacity should exist within developing countries to ensure that high priority problems are addressed and that recommendations are taken up in a timely manner [8, 9].
Yet, with few exceptions , no research has been undertaken to identify the situation of international, comparative health systems research. Two key questions are addressed: to what extent is the literature on health systems focusing on multiple country experiences as against reporting on single country studies? To what extent are papers in either case being produced by researchers within their respective countries or through North-South or South-South collaboration? These questions are addressed through analyzing the inter-relation of three dimensions. The first is the geographical scope of health systems papers by looking at whether they report on single or multiple countries. Multiple-country papers are in themselves evidence of international collaboration. The second dimension concerns the leadership of the studies, whether it is provided by foreign author to the study country, or in the case of multiple country papers, by an author foreign to all countries. The third dimension is the level of development of the countries studied as measured by per capita income. Through this dimension North-South or South-South collaboration can be identified. In the case of South-South collaboration, looking at the level of development allows to assess the collaboration of upper middle income, lower middle income or low income country institutions.
The study was undertaken using as the starting point Medline, a database of scientific publications in the health and medical fields published since 1991 through PubMed by the National Library of Medicine. Medline was accessed in three steps. The fist involved an advanced search in PubMed for journal articles (excluding letters, editorials and other kinds of citations) indexed by the National Library of Medicine experts under the major subject headings (MeSH) shown in figure 1. Citations were retrieved yearly from 1991 up to 2003 and for three income regions: Upper Middle Income (UMI), Lower Middle Income (LMI) and Low Income (LI). Income regions were identified by using the annual per capita income classification of the World Bank whereby LI regions have an annual per capita income of US$ 755 or less, LMIs between $756 and $2,995, and UMIs between $2,996 and $9,265.
In the second step citations where HPSR appeared as a secondary and minor topic were discarded and relevant papers were classified according to specific health systems research topics. For this step a modified version of the Evidence Base of the Alliance for Health Policy and Systems Research was used http://www.alliance-hpsr.org. The search engine powered by Collexis classified papers through analyzing the concepts in the abstract to produce an individualized fingerprint. Fingerprints consist of a mathematical algorithm describing the relation of concepts in the text according to frequency, proximity and concept interrelations. The set of concepts for the analysis is that of the thesaurus of the National Library of Medicine (MeSH).
The fingerprints for each paper were matched against the topic fingerprints developed for the field of health systems research, as follows:
A total of 19 health systems research topics were identified by analyzing 321 research projects undertaken by developing country researchers between 1999 and 2001 and reported to the Alliance-HPSR (Figure 2). These projects were being undertaken by 108 institutions in 39 countries and provided a good picture of developing country interests . Topics were identified by focusing on the main health system functions that were the subject of inquiry . Health system functions were identified at different scales of systems and by allowing cross-cutting themes. For example, a major topic "Finance" was identified, but also a minor topic "Insurance" that is conceptually a part of finance. The main criterion used to consider topics separately was that they should contain at least 2% of the total projects and that the sub-function in question was the main subject of research. Cross cutting themes such as "Community participation" included other functions or sub-functions, such as community finance and information systems, all subsumed under a different yet more prominent function within the research project.
A conceptual fingerprint was created for each of the 19 topics by submitting the glossary terms to the Collexis engine.
The topic fingerprints were then used to discard non-relevant citations and to classify the remainder. At this stage, topic fingerprints were refined through an iterative process by adding concepts found in fingerprints of highly relevant articles but absent in the topic fingerprint. This ensured that topic fingerprints captured in the end the highest number of articles with the highest relevance scores.
Citations were ranked according to relevance to the topic in a scale of 1% to 100%. Relevant articles were deemed to be those above 10%, as suggested by inspection of a sample of abstracts.
Table 1 shows the citations obtained at different steps in the analysis of Medline. Citations containing at least one MeSH term in the health systems research field total 87,300 for developing countries in the period 1991–2003) Of these, 9,066 or 10.4% were deemed to be relevant using the Alliance/Collexis search engine and could be classified with a fair degree of certainty within the various health systems research topics. Relevance was assessed statistically by the search engine as above 10%. The quality of selection was assessed through content analysis a selection of abstracts within each topic. Out of all papers selected, a total of 3.5% were classified in two or more topics. Once these repetitions were eliminated, a total of 8,751 papers were counted as published within health policy and systems between 1991 and 2003.
The concept of "Lead author" was operationalized by assuming that the author with greatest influence in the publication corresponded with the named person under the corresponding institution in the AD field in Medline. The countries studied in the paper were identified through Medline's MH field. The geographic scope of papers (single vs. multiple countries) and the income group of the countries studied and of the corresponding author were identified through quantifying the number of countries studied in each paper and through classifying them by income group. These variables were observed for a sample of papers within each of the 19 topics to enable inferences at this level. Given the exploratory nature of the study, a convenience sample of 10% of total papers was undertaken. In order to analyze proportions at the level of each topic oversampling at this level was allowed to obtain at least 40 citations per topic. This oversampling led to 242 additional papers, accounting for 24% of the total. In all, 13% of citations were sampled (Table 2). No significant changes were observed due to oversampling in the main variables at the aggregate level.
Only 69% of papers included the corresponding author's institutional data in the AD field, limiting the sample for this variable from 1,203 to 828 cases or 9% of the universe. Missing AD field data was more frequent for publications reporting on data for low income countries, represented in 46% of the total sample and in 39% of the citations with AD field data available. This drop is of consideration as the proportion of publications for low income countries in the universe amounts to 50%. This sampling and missing data bias implies that comparisons across income groups should be interpreted with caution.
While Medline is a very extensive database, it has well known limitations. Articles written in English account for 92% of the total while those in French and German for 2% each, Spanish for 1% and other languages for the remaining 3%. Furthermore, many journals published in developing countries and mostly of national circulation are not included. Medline also leaves out research published as internal reports or in the "grey" literature. However, the analysis of Medline is of value in itself as it reflects the knowledge that is widely available for shared learning internationally, whatever its limitations. Future studies of this type should analyze publications at the country level to include the full range of literature that can influence research as well as policy and systems development.
The corresponding author in the AD field is assumed to be the lead author. However, it may well be the case that several authors play a lead role, a fact that will not be reflected in the corresponding author. Furthermore, no consideration is taken here of the nationality of authors as this information is not available in Medline. As a result, no consideration is made of the participation of multiple authors that could be representing the different countries under study. The search was conducted to detect only papers with named developing countries in the MH field. Papers based on multiple country data not classified under a particular country will have been omitted.
Out of a total of 828 sampled papers analyzed, 744, (90%) make reference to single developing countries and 84 (10%) make reference to multiple countries. Papers led by authors from UMI and HI countries are most frequent, with 34% and 31% of the total, respectively (bottom row in Table 3). Lead authors account for 20% and 12% of total papers in the case of LI and LMI countries, respectively. International agencies author only 3% of papers. Considering authors from HI countries and international agencies together, they account for just over one third of total papers. Looking now at the distribution of papers by the income region of the country or countries studied (last column in Table 3), those for UMI countries account for the majority, with 42%, followed by those studying LI countries, with 38% and those for LM countries with 18%. Papers including results from countries in different income regions account for 2% of the total. The disparity between papers for LI countries and the proportion of corresponding authors from this same region is quite noticeable, with 38% of papers in the literature but only 20% of corresponding authors.
Looking now in more detail at Table 3, in the case of papers led by authors from HI countries, just under half or 49%, targeted LI countries while 29% targeted UMI countries and the remaining 19% LMI countries. International agencies focus more on LI countries, with 69% of papers led by authors from these agencies. Papers led by authors from all developing country regions focus almost exclusively on their own region (and country, as will be seen below), with between 95% and 99% of cases.
How is the authorship of single vs. multiple country papers distributed across regions? In the case of single country papers, UMI lead authors account for the most, with 37% of the total, followed about equally by HI and LI authors and then by LMI authors (bottom row of Table 4). In the case of multiple country papers (bottom row of Table 5), HI lead authors account for 74% of total, authors from international agencies for 5%, while developing country authors account for the remaining 21%, with UMI authors accounting for half of these.
To what extent are papers the fruit of foreign collaboration or leadership? This can be answered by analyzing whether the lead author's institution is foreign to the country or countries studied. In the case of multiple country studies led by HI authors, it is relevant to ask whether their country is included or not in the study.
The frequency of papers involving foreign collaboration is 34% in the case of single country studies (last column in Table 4) and 43% in the case of multi-country studies (last column in table 7). Who is leading this collaboration? In the case of single country papers, foreign collaboration came mostly from HI countries, with 77% of the total (second to last row in table 6). International agency authors led 9% of collaborations while developing country authors led the remaining 15%. In the case of multiple country papers, 83% of foreign collaboration came from HI authors, 11% came from international agency authors and only 6% from UMI authors. No multi-country papers led by LI or LMI were reported. Analyzing further the collaboration by HI authors in multi-country papers, it can be seen that in 52% of cases the author's institution was included. In these cases, an important reason to engage in multi-country work may have been to derive lessons for the HI country in question. In the remainder 48% of cases HI support may have been more directly related to support developing countries.
Regardless of the origin of expertise, a large proportion of publications demonstrate some form of international collaboration or interest. This is evidently the case of papers with a clear cross-country scope. However, international collaboration is also shown by single country papers with a foreign lead author, accounting for 41% of total papers. Papers led by authors from foreign institutions, whether single or multi-country, account for 35% of the total. However, international expertise for multi-country studies is heavily concentrated in HI countries and international agencies, where 79% of papers are lead authored.
What is the range of countries studied in multi-country papers? (Table 6). Out of all papers, as already stated, 90% are single-country studies. Of the remaining 10%, they are mostly two-country studies, with 7% of the total, while the remaining 3% include more than two. The one study sampled with most countries included 5. Most multi-country papers were lead by authors from HI countries and from international agencies.
What is the country distribution of papers? A total of 80 countries have institutions in the lead author position, of which 59 are in developing countries and 21 in high income countries (Table 7). There is a heavy concentration in a handful of countries. Only seven countries: India, Nigeria, China, Thailand, South Africa, Brazil and Mexico, concentrate 60% of the papers led by developing country authors. The United States and the United Kingdom concentrate between them as many as 68% of the papers led by high income countries and account for 21% of all papers (including those by international agencies, not shown in Table 5). This suggests there is a marked concentration of expertise in health systems research and in international health in a few countries and institutions.
The countries actually studied in the sampled papers total 106, of which 94 are developing countries and 12 are high income countries included as part of multi-country papers (Table 8). However, the concentration of countries studied is still very high, with the same seven developing countries mentioned above accounting for 47% of total papers and 19 developing countries, or 20% of the total, accounting for 70% of papers.
To what extent is leadership related to the specific topic of the papers? It is worth noting first, however, that topics with the greatest number of papers are those for Information, Education and Communication, Community Participation, Costing and Cost Effectiveness and Policy Process, with between 9.4% and 14.4% of all topics (figure 2). Topics with lowest frequency are Research Policy and Process, Decentralization and Economic and Social Policy and Health, with between 0.7% and 1.5% of total papers. The frequency of topics is analyzed in greater detail elsewhere .
Of all papers, Health Financing has the greatest international scope, with 26% of all papers within the topic (Table 9). This figure then drops from 14% to 18% for papers on Information, Education and Communication (IEC), Equity, Human Resources and Research Policy and Process. Topics with very few cross-country studies, at only 3% to 6%, are Community Participation, Costing and Cost Effectiveness and Program Evaluation. There is a negative association between the frequency of topics in the total sample and the frequency of cross-country papers within each topic (corr = -0.27).
Topics show important differences with respect to the income region of the lead author (Table 10). Topics such as Costing and Cost Effectiveness, Finance, Sector Analysis and Insurance are led in 38% to 54% of cases by high income authors. At the other end, topics such as Policy Process, Programme Evaluation and Quality of Care are only led by such authors in 21% to 22% of cases. This data suggest a concentration of expertise in the North for certain subjects, particularly health economics. There is a positive association between the frequency with which topics are treated internationally and the extent of leadership by authors from HI (corr = 0.29).
Looking now at the frequency of international papers by topic, the greatest number of them belongs to IEC, with 181 or 14 per year. This is followed by Policy Process, with 115 or 13 per year. The topic of Financing, in spite of having the greatest percentage of international publications, accounts for 91 total papers or 9 per year. Topics such as Decentralization, Economic and Social Policy, Community Participation and Accessibility account for less than 2 international papers per year on average.
The utilization of the concept-based search engine enabled the reliable analysis of a whole field of interdisciplinary research. This methodology proved more robust than the classification of papers through the Medline MeSH key-words, which tend to include papers with only marginal interest for the field. Furthermore, this methodology enabled the topic-specific analysis of the field, identifying the specific areas where international collaboration is stronger and where international papers are more frequent.
The fact that up to two thirds of the total health systems papers for developing countries are led by developing country authors indicates the existence of a minimum of research capacity in many developing countries. On the other hand, the one third of papers led by HI authors and to a lesser extent by international agencies points to the importance of North-South collaboration. This form of collaboration gives greater emphasis to LI countries, although close to one third of these papers involve UMI countries. Given marked disparities in research capacity across income regions in the developing world, a higher proportion of North-led papers would be expected to focus in LI countries. Given also the expertise in health systems within international agencies such as WHO, the fact that only 3% to 5% (for single and multiple country studies, respectively) of all publications are led by such agencies suggests a missed opportunity to disseminate in scientific journals and to make the most of available data.
South-South collaboration in health systems analysis and research is at a very low level. Only 11% of all single-country papers led by foreigners (which account for 34% of the total) are led by developing country institutions. Capacity to lead multi-country studies is also low in developing countries, with only 21% of such studies and virtually absent in the case of foreign-led studies.
Multi-country studies, with only 10% of the total and around 115 papers per year are scarce no matter who undertakes them, and may not be realizing the full potential to generate the needed knowledge for health systems development. While the figure of cross-country studies is encouraging for some topics such as Health Financing, most of these papers are two-country comparisons, perhaps not extensive enough to generate knowledge on trends and determinants that can be generalized beyond the study countries.
While minimum research capacity may exist in many developing countries, the fact that lead institutions as well as study countries are concentrated in a handful of mostly middle income countries attests to great disparities in research capacity. However, disparities are also evident in the North, where just two institutions, the United States of America and the United Kingdom, concentrate 60% of papers led by high income countries and produce one fifth of all the literature indexed in Medline on health system research. Paper leadership by high income country institutions has important differences across topics, with a marked emphasis in health economics. This finding suggests a particular lack of capacity in this area in the developing world, although it could also be the result of a marked interest in health economics on the part of institutions in high income countries.
This paper attests to the small part played by comparative health systems literature and by papers with an international focus in the context of Medline. Furthermore, the analysis demonstrates the large role played in this respect by Northern authors. However, this analysis says nothing on the actual influence of the literature on the research agendas or on policy. Further research is required to answer these questions, in the light of what is known about how research projects actually influence on the research agendas and on health policy .
Regional research networks have been encouraged to strengthen capacity and to promote comparative research. However, evidence shows that the limiting factor in this kind of research is funding. Indeed, current funding averaging around US$ 20,000 is too low even to ensure successful single country research . It is likely that successful multi-country research projects will require funding in the order of US$500,000 per project, to involve three to four countries. This is a small amount considering the current funding levels for health system strengthening at the global level.
It is urgent to build capacity to generate knowledge based on cross-country research. North-South collaboration can be strengthened by promoting further involvement of countries such as Germany, France, Belgium and Spain in funding, supporting and leading research in developing countries. However, South-South collaboration has to be strengthened through innovative strategies. Institutions in upper middle income countries that are now concentrating most comparative research capacity can be encouraged and supported to lead research involving low income countries. This involvement can ensure not only technical capacity, but also a more appropriate research process leading to effective policy impact.
KFPE , Enhancing research capacity in developing and transition countries: Experience, discussions, strategies and tools for building research capacity and strengthening institutions in view of promoting research for sustainable development. Geographica Bernensia, Berne. 2001
Mari , Lozano , Duley , Mari JJ, Lozano JM, Duley L, Erasing the global divide in health research: Collaboration provides answers relevant to developing and developed countries. BMJ. 1997, 314: 390-
Johnstone P, McConnan I, Primary health care led NHS: learning from developing countries. BMJ. 1995, 311: 891-892.
WHO : The world health report 2000–health systems improving performance. 2000, Geneva, WHO
González-Block MA: "Comparative research and analysis methods for shared learning from health system reforms". Health Policy. 1997, 42: 187-209. 10.1016/S0168-8510(97)00072-9.
Tan-Torres Edejer T: North-South research partnerships: the ethics of carrying out research in developing countries. BMJ. 1999, 319: 438-441.
SDC, Research policy of the Swiss Agency for Development Cooperation (SDC). Berne, SDC. 2002
Alliance for Health Policy and Systems Research, Strengthening health systems: the role and promise of health policy and systems research. Geneva, Alliance HPSR. 2004
ChitrSitthi-amorn C, Somrongthong R: Strengthening health research capacity in developing countries: a critical element for achieving health equity. BMJ. 2000, 321: 813-817. 10.1136/bmj.321.7264.813.
Maselli D, Lys JA, Schmid J: Improving impacts of research partnerships, Swiss Commission for Research Partnerships with Developing Countries. KFPE Geographica Bernensia, Berne. 2004
Gonzalez-Block MA: Health policy and systems research agendas in developing countries. Health Research Policy and Systems. 2004, 2: 6-10.1186/1478-4505-2-6. [http://www.health-policy-systems.com/content/2/1/6]
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.