Open Access

Four centuries on from Bacon: progress in building health research systems to improve health systems?

Health Research Policy and Systems201412:56

https://doi.org/10.1186/1478-4505-12-56

Received: 8 September 2014

Accepted: 8 September 2014

Published: 23 September 2014

Abstract

In 1627, Francis Bacon’s New Atlantis described a utopian society in which an embryonic research system contributed to meeting the needs of the society. In this editorial, we use some of the aspirations described in New Atlantis to provide a context within which to consider recent progress in building health research systems to improve health systems and population health. In particular, we reflect on efforts to build research capacity, link research to policy, identify the wider impacts made by the science, and generally build fully functioning research systems to address the needs identified.

In 2014, Health Research Policy and Systems has continued to publish one-off papers and article collections covering a range of these issues in both high income countries and low- and middle-income countries. Analysis of these contributions, in the context of some earlier ones, is brought together to identify achievements, challenges and possible ways forward. We show how 2014 is likely to be a pivotal year in the development of ways to assess the impact of health research on policies, practice, health systems, population health, and economic benefits.

We demonstrate how the increasing focus on health research systems will contribute to realising the hopes expressed in the World Health Report, 2013, namely that all nations would take a systematic approach to evaluating the outputs and applications resulting from their research investment.

Keywords

Collaborative approach Health research system Health systems Payback framework Research agenda setting Research capacity Research impacts Research to policy Value of medical research World Health Report

Editorial

In 1627, Francis Bacon’s New Atlantis, an unfinished book published a year after his death, described a utopian society that included an embryonic research system which contributed to meeting society’s needs, although it did so on its own terms [1]. Some of the aspirations highlighted in New Atlantis are still extant and provide a useful context within which this editorial explores recent progress in building health research systems to improve health systems and population health. In particular, we reflect on progress, and consider prospects for efforts to build research capacity, link research to policy, identify the wider impacts made by the science, and, generally, build fully functioning research systems addressing the needs identified.

In New Atlantis, the narrator of the tale describes what he was told by the head of the fictional society’s scientific college. The numbers of people involved were not large, but worked together and in a diversity of roles set in a framework that prefigured features of a modern research system. Some of the team of scientists collected experiments by travelling to other countries, some went through books to collect experiments, some collected the experiments of ‘all mechanical arts’, i.e., practical fields, and some ‘try new experiments, such as themselves think good.’ It seems fair to ask whether Bacon also possibly foresaw the role for systematic reviewers in the description given of a fifth group in the team who ‘draw the experiments of the former four into titles and tables, to give the better light for the drawing of observations and axioms out of them. These we call compilers.’ [1].

The use of research was important in the utopian society; there was another group who would contribute by: ‘looking into the experiments of their fellows, and cast about how to draw out of them things of use and practice for man’s life and knowledge’. And, as would be recommended in any properly functioning health research system, yet another team of scientists would build on the findings to inform further research, but in this case would do so only after a team effort to identify the best way forward: ‘Then after divers meetings and consults of our whole number, to consider the former labours and collections, we have three that take care out of them to direct new experiments, of a higher light, more penetrating into nature than the former. These we call lamps.’ [1].

Building health research capacity

Nearly 400 years on, the capacity to undertake a similar, and expanding, range of functions is required in any health research system, but is still quite limited in some countries. However, there is a range of initiatives underway to increase health research capacity at system, institution, and individual levels, and many of these are in low- and middle-income countries (LMICs). In June 2014, Health Research Policy and Systems (HARPS) published a mini-series entitled Strengthening Institutional Health Systems Research Capacity for Seven Schools of Public Health in East and Central Africa[25]. This series describes how, in a programme funded by the UK’s Department for International Development under the auspices of the Future Health Systems initiative, seven schools of Public Health and selected health policy institutions across six countries in East and Central Africa embarked on a five-year project to strengthen their capacity to undertake high quality, policy-relevant health systems research. The project provides examples of how such activities can continue to be taken forward. For example, Jessani et al. developed a tool for the self-assessment of health systems research capacity which can be used by schools of public health to produce institutional capacity development plans [3].

The increased attention on building health research capacity is also illustrated by the initiative called Enhancing Support for Strengthening the Effectiveness of National Capacity Efforts (ESSENCE on Health Research), a collaborative framework between funding agencies to scale up research capacity and hosted by the Special Programme for Research and Training in Tropical Diseases at the World Health Organisation (WHO). A Canadian/UK team working with ESSENCE on Health Research have published several recent papers in HARPS, including ones on indicators for tracking programmes to strengthen health research capacity [6] and on frameworks for evaluating health capacity strengthening [7]. The Canadian and UK members of the team also used peer-reviewed and grey literature to develop a five-step pathway for designing and evaluating health research capacity strengthening programmes, and tested it in a variety of contexts in Africa [8]. Recent research capacity initiatives are not, of course, limited to LMICs. Canadian examples include one led by the Canadian Association for Health Services and Policy Research [9], and another that demonstrated the role networking can play in researchers’ careers and more broadly in research capacity strengthening [10]. The latter did so through examining the opportunity for networking among new global health researchers provided by the annual Summer Institute of the Canadian Coalition for Global Health Research.

Despite progress, the need for the existing and continuing efforts to increase health research capacity are underlined by a recent assessment of publications involving health policy and systems research from the 71 LMICs that had worked with WHO to produce Country Cooperation Strategies. This analysis indicates a continuing need for WHO and other global health agencies to work to build health policy and systems research capacity [11]. A further recent analysis published in HARPS shows that even where there has been progress in a country such as India, it can be very patchy and research capacity still needs to be strengthened in many parts of the country [12].

Using research to inform health policy

As Bacon emphasised, it is also important to give attention to how research can be ‘of use and practice for man’s life’ [1]. Almost since its formation in 2003, HARPS has had a focus on the role of research in health policymaking, with an early paper [13] building on the work of Kogan and Henkel in the 1970s and 1980s [14]. Kogan and Henkel were pioneers, developing the concept of the collaborative approach between researchers and policymakers as a way of addressing some of the difficulties of promoting research use in policymaking, and also recognising that there is a diversity of situations in which there might potentially be scope for various types of research to have impact on policymaking. HARPS subsequently published the influential SUPPORT Tools for Evidence-Informed Health Policymaking (STP) series led by Lavis, Oxman, and colleagues [15].

There have been many recent initiatives to strengthen the use of health research for policy, some described in HARPS, and the scope of these varies considerably. One example focussed on lessons recently learnt about the knowledge translation platforms (the partnerships between policymakers, stakeholders, and researchers) that are being established in LMICs to enhance evidence-informed health policymaking [16]. In a paper from Australia, but with an international focus, Milat et al. concluded that for ‘scaling-up’ decisions, ‘Research evidence formed a component of the overall set of information used in decision-making, but its contribution was limited by the paucity of relevant intervention effectiveness research, and data on costs and cost effectiveness.’ [17]. At a much more specific level, a study from the Netherlands showed how a research contribution mapping approach, developed by Kok and Schuit and first described in HARPS[18], could be applied to examine how far a specific project commissioned by the Health Care Inspectorate had contributed to the Inspectorate’s work [19].

The relationship between research and policymaking remains complex, and has still not been fully explored, as Oliver et al. recently described in HARPS[20]. In this paper, the authors critically analysed the body of work included in the systematic reviews of studies of the use of evidence by policymakers that were published in 2014 by Oliver et al. [21] and, in 2003, by Innvær et al. [22]. They suggested there should be a greater focus on studying the processes of policymaking, the needs of policymakers, the information they do use, and the diversity of circumstances in which knowledge might be used in policymaking. They also claimed that more needs to be done to evaluate the impact of research on populations.

Assessing the wider impacts of research

There are, nevertheless, already some approaches that use wider frameworks to explore the impact of research use not only on policymaking but also (sometimes through policymaking) in terms of improved health services, population health, and benefits to the economy. In the UK, the Higher Education Funding Council for England (HEFCE) has developed the Research Excellence Framework [23], in which 20% of the evaluation of all university research will depend on the assessment of the wider impact made by the research on non-academic audiences. In a memorandum to the UK Parliament, HEFCE described how it had developed proposals to assess the wider impact, and in doing so explained that the evidence it drew on came from methods to assess the impact of health research. HEFCE stated: ‘In developing our proposal we drew heavily on the existing evidence. There have been a number of studies that have estimated the impact of research’. The two examples provided to support this claim were: ‘work using the “payback framework” … [and] Medical Research: What’s it Worth … these studies have helped us to identify common methodological challenges and have shown how they can be overcome.’ [23].

Building on the same body of evidence, the World Health Report 2013: Health Research for Universal Coverage stated: ‘adding impetus to do more research is a growing body of evidence on the returns on investments … there is mounting quantitative proof of the benefits of research to health, society and the economy.’ [24]. In support of this statement, they cited the UK study, Medical Research: What’s it Worth[25]. Further analysis also drew on studies from the US and Australia [2628]. However, the World Health Report also claimed that ‘Not all the benefits of research can be, or should be, measured in monetary terms. To capture the diversity of benefits from research, the Payback Framework evaluates outcomes under five headings: knowledge, benefits to future research and research use, benefits from informing policy and product development, health and health sector benefits, and economic benefits.’ [24].

While the first account of the Payback Framework was in 1996 [29], a major update was described in HARPS in 2004 [30]. Two reviews of studies assessing the wider impacts from health research have been published in HARPS, one from the Italian Cochrane Centre [31] and another from an Iranian team [32], both identifying the Payback Framework as the most widely used approach, including its influence on other frameworks. Papers published in HARPS that describe applications and developments of the Payback Framework include two on primary care research in Australia [33, 34].

There have also been important developments in the assessment of the impact of health research in North America. A panel of the Canadian Academy of Health Sciences (CAHS) built on the Payback Framework to recommend ‘A preferred framework and indicators to measure returns on investment in health research’ [35]. This was viewed as a ‘much awaited development in the Canadian health research community’ [36] and is being developed and applied by various research funders in Canada, for example by Alberta Innovates – Health Solutions[36]. The CAHS development of the Payback Framework has also been applied in Spain. An article in HARPS describing this application drew on interviews that demonstrated how projects on respiratory diseases funded by the Agency for Health Quality and Assessment of Catalonia between 1996 and 2004, ‘indicated changes in health services or clinical practice had resulted from research.’ [37]. There has, of course, been interest in these issues for many years, as traced by a paper considering some of the history of medical research in the USA [38]. In keeping with the growing interest, the Scientific Management Board of the National Institutes of Health (NIH) produced a report in 2014 exploring the current approaches to assessing the value of biomedical research, and recommended the establishment of a trans-NIH committee to develop a strategy and take forward work in this field [39].

Similarly, there are many developments in Europe, including in the Netherlands where an example from Leiden University Medical Center was reported in a paper in HARPS[40]. As noted above, the UK has probably seen most activity, and in addition to the approaches previously described, there have also been attempts to develop a performance monitoring framework for the English National Institute for Health Research [41].

Perhaps crucially in terms of taking the field forward, 2014 looks as though it could be an important year in demonstrating the impact of health research; this will be in several ways. First, there have recently been some key developments in the stream of work assessing the value of medical research. The original 2008 study in the UK, Medical Research: What’s it Worth[25], showed that on average every pound of public and charitable funding spent on cardiovascular and mental health research generated a very much higher level of benefits than the standard minimum required by the UK Treasury for the investment of public money. In June 2014, the same team published a follow-up study examining the value of UK cancer research and found that the best estimate for the rate of returns is even higher than in the previous study [42]. The particular significance of this 2014 study is that, given that cancer, cardiovascular disease, and mental health disorders account for approximately 45% of the current burden of disease in the UK, it can be concluded that investments in medical research produce a sizeable return in areas where there is a high morbidity [42].

A second factor making 2014 a pivotal year in this field is that December will see the results of the assessment of the 7,000 impact case studies submitted to the UK exercise to evaluate the quality and impact of university research. While this exercise covers all university researchers, well over 1,500 of the case studies will demonstrate the wider impacts of health and biomedical research from UK universities, including medical schools. The wider impacts eligible for consideration in the evaluation include impacts in the following categories: informing health policies and product development; the behaviour of practitioners and members of the public; improved health and health systems; and economic benefits. The database of case studies of the impact of health research will facilitate analysis of features of the full health research system in the UK that might have contributed to the impact. This, too, should help to demonstrate the value of the impacts made by funding health research, an increasingly important issue in many countries.

Building health research systems to meet the needs of health systems

The editors of HARPS believe it is useful to adopt a systems approach when considering how best to develop health research in all countries [43], and are pleased that such an approach was promoted in the 2013 World Health Report [24]. HARPS has published the findings from multi-country surveys of progress in developing health research systems, including in Latin American and Caribbean countries [44], the Economic Community of the West African States [45], and the 27 countries of the European Union [46]. Country-specific papers include ones on Panama [47], Guinea Bissau [48], Zambia [49], Solomon Islands [50], and England [51]. Contrasting systems at different stages of development have now been analysed and highlight a number of common key issues that are sometimes already being addressed, and often still need much more action. Such issues include research agenda setting, capacity building, and enhancing the use of research findings.

The importance of action at the systems level is increasingly becoming clear. In relation to agenda-setting, the involvement of a range of stakeholders is seen as important. HARPS recently published a paper that followed on from the overall analysis of the Panamanian health research system described above [52]; the follow-on paper examined previous attempts to set priorities for health research in Panama. To inform the approach taken, the analysis partly drew on an earlier paper in HARPS by Viergever et al. [53]. We also published a paper in 2014 in which Kothari et al. described the success of a process of consensus building between diverse stakeholders to build a public health systems research agenda in Ontario, Canada, using an approach developed in the USA [54]. HARPS is also interested in helping to promote research approaches that are relevant for addressing key issues. Hence, we have just published a major new article collection that was organised by Adam and colleagues at the Alliance for Health Policy and Systems Research: Advancing the Application of Systems Thinking in Health. It includes a commentary that sets out the benefits of using systems thinking in health [55], and an editorial outlining the full range of articles in the collection [56].

We have seen that HARPS is increasingly becoming a clearinghouse of challenges and solutions to strengthen health research systems. We intend that HARPS will continue to demonstrate how research systems can contribute to addressing key issues facing health systems. For example, the contribution of research systems to addressing existing Millennium Development Goals, and future ones, could become increasingly important as analysis of progress in achieving existing ones intensifies [57]. We hope to feature a future series of papers on a research programme related to this theme, in addition to papers on issues facing the health research systems such as how to analyse and reduce the time taken between the conduct of research and its impact on improving healthcare.

Of course, the system Bacon set out in the utopian vision of New Atlantis had many differences from modern systems, but some of the aspirations he set out provide a useful context to examine the progress being made. Health research systems are now being developed in many countries, and the impacts from health research, including on health systems, are now being more systematically assessed. At HARPS we shall aim to continue providing a platform for a wide range of contributions on these topics. In the words of the World Health Report, 2013: ‘All nations will benefit from taking a systematic approach to the monitoring and evaluation of research investments, practices, outputs and applications.’ [24].

Abbreviations

CAHS: 

Canadian Academy of Health Sciences

ESSENCE on Health Research: 

Enhancing Support for Strengthening the Effectiveness of National Capacity Efforts

HARPS: 

Health Research Policy and Systems

HEFCE: 

Higher Education Funding Council for England

LMICs: 

Low- and middle-income countries

NIH: 

National Institutes of Health

WHO: 

World Health Organisation.

Declarations

Acknowledgements

The authors thank Bryony Soper for most helpful comments on an earlier draft.

Authors’ Affiliations

(1)
Health Economics Research Group, Brunel University London
(2)
Universidad Anáhuac

References

  1. Bacon F: New Atlantis. 1627, [http://www.gutenberg.org/files/2434/2434-h/2434-h.htm]Google Scholar
  2. Ayah R, Jessani N, Mafuta E: Institutional capacity for health systems research in East and Central African schools of public health: knowledge translation and effective communication. Health Res Policy Syst. 2014, 12: 20-View ArticlePubMedPubMed CentralGoogle Scholar
  3. Jessani N, Lewy D, Ekirapa-Kiracho E, Bennett S: Institutional capacity for health systems research in East and Central African schools of public health: experiences with a capacity assessment tool. Health Res Policy Syst. 2014, 12: 21-View ArticlePubMedPubMed CentralGoogle Scholar
  4. Nangami M, Rugema L, Tebeje B, Mukose A: Institutional capacity for health systems research in East and Central African schools of public health: enhancing capacity to design and implement teaching programs. Health Res Policy Syst. 2014, 12: 22-View ArticlePubMedPubMed CentralGoogle Scholar
  5. Simba D, Mukose A, Bazeyo W: Institutional capacity for health systems research in East and Central African Schools of Public Health: strengthening human and financial resources. Health Res Policy Syst. 2014, 12: 23-View ArticlePubMedPubMed CentralGoogle Scholar
  6. Cole DC, Boyd A, Aslanyan G, Bates I: Indicators for tracking programmes to strengthen health research capacity in lower- and middle-income countries: a qualitative synthesis. Health Res Policy Syst. 2014, 12: 17-View ArticlePubMedPubMed CentralGoogle Scholar
  7. Boyd A, Cole DC, Cho D-B, Aslanyan G, Bates I: Frameworks for evaluating health research capacity strengthening: a qualitative study. Health Res Policy Syst. 2013, 11: 46-View ArticlePubMedPubMed CentralGoogle Scholar
  8. Bates I, Boyd A, Smith H, Cole DC: A practical and systematic approach to organisational capacity strengthening in the health sector in Africa. Health Res Policy Syst. 2014, 12: 11-View ArticlePubMedPubMed CentralGoogle Scholar
  9. Grudniewicz A, Hedden L, Kromm S, Lavergne R, Menear M, Sivananthan S, for The Canadian Association for Health Services and Policy Research (CAHSPR) Student Working Group: Capacity development in health systems and policy research: a survey of the Canadian context. Health Res Policy Syst. 2014, 12: 9-View ArticlePubMedPubMed CentralGoogle Scholar
  10. Lenters LM, Cole DC, Godoy-Ruiz P: Networking among young global health researchers through an intensive training approach: a mixed methods exploratory study. Health Res Policy Syst. 2014, 12: 5-View ArticlePubMedPubMed CentralGoogle Scholar
  11. Xue J, Murthy B, Tran NT, Ghaffar A: Goal setting and knowledge generation through health policy and systems research in low-and middle-income countries. Health Res Policy Syst. 2014, 12: 39-View ArticlePubMedPubMed CentralGoogle Scholar
  12. Rao KD, Arora R, Ghaffar A: Health systems research in the time of health system reform in India: a review. Health Res Policy Syst. 2014, 12: 37-View ArticlePubMedPubMed CentralGoogle Scholar
  13. Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M: The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Policy Syst. 2003, 1: 2-View ArticlePubMedPubMed CentralGoogle Scholar
  14. Kogan M, Henkel M: Government and Research. The Rothschild Experiment in a Government Department. 1983, London: HeinemannGoogle Scholar
  15. Lavis JN, Oxman AD, Lewin S, Fretheim A: SUPPORT Tools for evidence informed health Policymaking (STP). Health Res Policy Syst. 2009, 7 (Suppl 1): I1-View ArticlePubMedPubMed CentralGoogle Scholar
  16. El-Jardali F, Lavis J, Moat K, Pantoja T, Ataya N: Capturing lessons learned from evidence-to-policy initiatives through structured reflection. Health Res Policy Syst. 2014, 12: 2-View ArticlePubMedPubMed CentralGoogle Scholar
  17. Milat AJ, King L, Newson R, Wolfenden L, Rissel C, Bauman A, Redman S: Increasing the scale and adoption of population health interventions: experiences and perspectives of policy makers, practitioners, and researchers. Health Res Policy Syst. 2014, 12: 18-View ArticlePubMedPubMed CentralGoogle Scholar
  18. Kok MO, Schuit AJ: Contribution mapping: a method for mapping the contribution of research to enhance its impact. Health Res Policy Syst. 2012, 10: 21-View ArticlePubMedPubMed CentralGoogle Scholar
  19. Hegger I, Janssen SWJ, Keijsers JFEM, Schuit AJ, van Oers HAM: Analyzing the contributions of a government-commissioned research project: a case study. Health Res Policy Syst. 2014, 12: 8-View ArticlePubMedPubMed CentralGoogle Scholar
  20. Oliver K, Lorenc T, Innvær S: New directions in evidence-based policy research: a critical analysis of the literature. Health Res Policy Syst. 2014, 12: 34-View ArticlePubMedPubMed CentralGoogle Scholar
  21. Oliver K, Innvær S, Lorenc T, Woodman J, Thomas J: A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014, 14: 2-View ArticlePubMedPubMed CentralGoogle Scholar
  22. Innvaer S, Vist G, Trommald M, Oxman A: Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002, 7: 239-244.View ArticlePubMedGoogle Scholar
  23. Higher Education Funding Council for England (HEFCE): Memorandumsubmitted by the Higher Education Funding Council for England. In TheImpact of Spending Cuts on Science and Scientific Research. Sixth Report ofSession 2009-10. Volume 11: Oral and Written Evidence, House of CommonsScience and Technology Committee. HC 335-11, Volume 11. London: TheStationery Office Limited; 2010.Google Scholar
  24. World Health Organization: The World Health Report 2013: Research for Universal Health Coverage. 2013, Geneva: WHOGoogle Scholar
  25. Health Economics Research Group: Medical Research: What’s It Worth. Estimating the Economic Value of Medical Research in the UK. 2008, London: UK Evaluation ForumGoogle Scholar
  26. Funding First: Exceptional Returns. The Economic Value of America’s Investment in Medical Research. 2000, New York: Albert & Mary Lasker FoundationGoogle Scholar
  27. Access Economics: Exceptional Returns: The Value of Investing in Health R&D in Australia II. 2008, Canberra: Access Economics, [http://www.asmr.org.au/ExceptII08.pdf]Google Scholar
  28. Johnston SC, Rootenberg JD, Katrak S, Smith WS, Elkins JS: Effect of a US National Institutes of Health programme of clinical trials on public health and costs. Lancet. 2006, 367: 1319-1327.View ArticlePubMedGoogle Scholar
  29. Buxton M, Hanney S: How can payback from health services research be assessed?. J Health Serv Res Policy. 1996, 1: 35-43.PubMedGoogle Scholar
  30. Hanney SR, Grant J, Wooding S, Buxton MJ: Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK’s Arthritis Research Campaign. Health Res Policy Syst. 2004, 2: 4-View ArticlePubMedPubMed CentralGoogle Scholar
  31. Banzi R, Moja L, Pistotti V, Facchini A, Liberati A: Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews. Health Res Policy Syst. 2011, 9: 26-View ArticlePubMedPubMed CentralGoogle Scholar
  32. Yazdizadeh B, Majdzadeh R, Salmasian H: Systematic review of methods for evaluating healthcare research economic impact. Health Res Policy Syst. 2010, 8: 6-View ArticlePubMedPubMed CentralGoogle Scholar
  33. Kalucy EC, Jackson-Bowers E, McIntyre E, Reed R: The feasibility of determining the impact of primary health care research projects using the Payback Framework. Health Res Policy Syst. 2009, 7: 11-View ArticlePubMedPubMed CentralGoogle Scholar
  34. Reed RL, Kalucy EC, Jackson-Bowers E, McIntyre E: What research impacts do Australian primary health care researchers expect and achieve?. Health Res Policy Syst. 2011, 9: 40-View ArticlePubMedPubMed CentralGoogle Scholar
  35. Frank C, Nason E: Health research: measuring the social, health and economic benefits. CMAJ. 2009, 180: 528-534.View ArticlePubMedPubMed CentralGoogle Scholar
  36. Graham KER, Chorzempa HL, Pamela A, Valentine PA, Magnan J: Evaluating health research impact: Development and implementation of the Alberta Innovates – Health Solutions impact framework. Res Eval. 2012, 21: 354-367.View ArticleGoogle Scholar
  37. Solans-Domènech M, Adam P, Guillamón I, Permanyer-Miralda G, Pons JMV, Escarrabill J: Impact of clinical and health services research projects on decision-making: a qualitative study. Health Res Policy Syst. 2013, 11: 15-View ArticlePubMedPubMed CentralGoogle Scholar
  38. Cozzens SE: Quality of life returns from basic research. Health Res Policy Syst. 2010, 8: 18-View ArticlePubMedPubMed CentralGoogle Scholar
  39. National Institutes of Health: Scientific Management Review Board Report on Approaches to Assess the Value of Biomedical Research Supported by NIH. 2014, [http://smrb.od.nih.gov/documents/reports/VOBR%20SMRB__Report_2014.pdf]Google Scholar
  40. Mostert SP, Ellenbroek SPH, Meijer I, van Ark G, Klasen EC: Societal output and use of research performed by health research groups. Health Res Policy Syst. 2010, 8: 30-View ArticlePubMedPubMed CentralGoogle Scholar
  41. El Turabi A, Hallsworth M, Ling T, Grant J: A novel performance monitoring framework for health research systems: experiences of the National Institute for Health Research in England. Health Res Policy Syst. 2011, 9: 13-View ArticlePubMedPubMed CentralGoogle Scholar
  42. Glover M, Buxton M, Guthrie S, Hanney S, Pollitt A, Grant J: Estimating the returns to UK publicly funded cancer-related research in terms of the net value of improved health outcomes. BMC Med. 2014, 12: 99-View ArticlePubMedPubMed CentralGoogle Scholar
  43. Hanney SR, González-Block MA: Organising health research systems as a key to improving health: the World Health Report 2013 and how to make further progress. Health Res Policy Syst. 2013, 11: 47-View ArticlePubMedPubMed CentralGoogle Scholar
  44. Becerra-Posada F, Minayo M, Quental C, de Haan S: National research for health systems in Latin America and the Caribbean: moving towards the right direction?. Health Res Policy Syst. 2014, 12: 13-View ArticlePubMedPubMed CentralGoogle Scholar
  45. Sombié I, Aidam J, Konaté B, Somé TD, Kambou S: The state of the research for health environment in the ministries of health of the Economic Community of the West African States (ECOWAS). Health Res Policy Syst. 2013, 11: 35-View ArticlePubMedPubMed CentralGoogle Scholar
  46. Conceição C, McCarthy M: Public health research systems in the European Union. Health Res Policy Syst. 2011, 9: 38-View ArticlePubMedPubMed CentralGoogle Scholar
  47. Romero LI, Quental C: The Panamanian health research system: a baseline analysis for the construction of a new phase. Health Res Policy Syst. 2013, 11: 33-View ArticlePubMedPubMed CentralGoogle Scholar
  48. Kok MO, Rodrigues A, Silva AP, de Haan S: The emergence and current performance of a health research system: lessons from Guinea Bissau. Health Res Policy Syst. 2012, 10: 5-View ArticlePubMedPubMed CentralGoogle Scholar
  49. Chanda-Kapata P, Campbell S, Zarowsky C: Developing a national health research system: participatory approaches to legislative, institutional and networking dimensions in Zambia. Health Res Policy Syst. 2012, 10: 17-View ArticlePubMedPubMed CentralGoogle Scholar
  50. Redman-MacLaren ML, MacLaren DJ, Solomon J, Muse A, Asugeni R, Harrington H, Kekuabata E, Speare R, Clough AR: Research workshop to research work: initial steps in establishing health research systems on Malaita, Solomon Islands. Health Res Policy Syst. 2010, 8: 33-View ArticlePubMedPubMed CentralGoogle Scholar
  51. Hanney S, Kuruvilla S, Soper B, Mays N: Who needs what from a national health research system: lessons from reforms to the English department of health’s R&D system. Health Res Policy Syst. 2010, 8: 11-View ArticlePubMedPubMed CentralGoogle Scholar
  52. Romero LI, Quental C: Research for better health: the Panamanian priority-setting experience and the need for a new process. Health Res Policy Syst. 2014, 12: 38-View ArticlePubMedPubMed CentralGoogle Scholar
  53. Viergever RF, Olifson S, Ghaffar A, Terry RF: A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst. 2010, 8: 36-PubMedPubMed CentralGoogle Scholar
  54. Kothari A, Regan S, Gore D, Valaitis R, Garcia J, Manson H, O’Mara L: Using an integrated knowledge translation approach to build a public health research agenda. Health Res Policy Syst. 2014, 12: 6-View ArticlePubMedPubMed CentralGoogle Scholar
  55. Peters DH: The application of systems thinking in health: why use systems thinking?. Health Res Policy Syst. 2014, 12: 51-View ArticlePubMedPubMed CentralGoogle Scholar
  56. Adam T: Advancing the application of systems thinking in health. Health Res Policy Syst. 2014, 12: 50-View ArticlePubMedPubMed CentralGoogle Scholar
  57. Kuruvilla S, Schweitzer J, Bishai D, Chowdhury S, Caramani D, Frost L, Cortez R, Daelmans B, de Francisco A, Adam T, Cohen R, Alfonso YN, Franz-Vasdeki J, Saadat S, Pratt BA, Eugster B, Bandali S, Venkatachalam P, Hinton R, Murray J, Arscott-Mills S, Axelson H, Maliqi B, Sarker I, Lakshminarayanan R, Jacobs T, Jacks S, Mason E, Ghaffar A, Mays N: Success factors for reducing maternal and child mortality. Bull World Health Organ. 2014, 92: 533-544.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© Hanney and González-Block; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Advertisement