TY - STD TI - European Parliament and the Council. Regulation (EU) No 1291/2013 of the European Parliament and of the Council of 11 December 2013 establishing Horizon 2020 – the Framework Programme for Research and Innovation (2014-2020) and repealing Decision No 1982/2006/EC Text with EEA relevance. http://data.europa.eu/eli/reg/2013/1291/oj. Accessed 8 Dec 2017. UR - http://data.europa.eu/eli/reg/2013/1291/oj ID - ref1 ER - TY - STD TI - 2014 Italian Presidency of the Council of the European Union. Rome Declaration on Responsible Research and Innovation in Europe. https://ec.europa.eu/research/swafs/pdf/rome_declaration_RRI_final_21_November.pdf. Accessed 8 Dec 2017. UR - https://ec.europa.eu/research/swafs/pdf/rome_declaration_RRI_final_21_November.pdf ID - ref2 ER - TY - STD TI - European Commission. Science With and For Society. https://ec.europa.eu/programmes/horizon2020/en/h2020-section/science-and-society. Accessed 8 Dec 2017. UR - https://ec.europa.eu/programmes/horizon2020/en/h2020-section/science-and-society ID - ref3 ER - TY - STD TI - Trudeau J. Minister of Innovation, Science and Economic Development Mandate Letter. http://pm.gc.ca/eng/minister-innovation-science-and-economic-development-mandate-letter. Accessed 8 Dec 2017. UR - http://pm.gc.ca/eng/minister-innovation-science-and-economic-development-mandate-letter ID - ref4 ER - TY - STD TI - Government of Canada. Policy on Results. https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=31300. Accessed 8 Dec 2017. UR - https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=31300 ID - ref5 ER - TY - STD TI - Australian Government. National Innovation and Science Agenda Report. https://www.innovation.gov.au/page/national-innovation-and-science-agenda-report. Accessed 16 Jul 2017. UR - https://www.innovation.gov.au/page/national-innovation-and-science-agenda-report ID - ref6 ER - TY - STD TI - Science of Science Policy. Advancing Science Policy through Science. https://archive.is/sVXly. Accessed 21 Jan 2018. UR - https://archive.is/sVXly ID - ref7 ER - TY - BOOK AU - Fealing, K. H. PY - 2011 DA - 2011// TI - The Science of Science Policy: A Handbook PB - Stanford Business Books CY - Stanford ID - Fealing2011 ER - TY - STD TI - Centre for Science and Policy. Research and Policy Engagement. http://www.csap.cam.ac.uk/. Accessed 22 Jan 2018. UR - http://www.csap.cam.ac.uk/ ID - ref9 ER - TY - JOUR AU - Lane, J. PY - 2010 DA - 2010// TI - Let's make science metrics more scientific JO - Nature VL - 464 UR - https://doi.org/10.1038/464488a DO - 10.1038/464488a ID - Lane2010 ER - TY - JOUR AU - Lane, J. AU - Bertuzzi, S. PY - 2011 DA - 2011// TI - Research funding. Measuring the results of science investments JO - Science VL - 331 UR - https://doi.org/10.1126/science.1201865 DO - 10.1126/science.1201865 ID - Lane2011 ER - TY - JOUR AU - Smith, R. PY - 2001 DA - 2001// TI - Measuring the social impact of research JO - Difficult but necessary VL - 323 ID - Smith2001 ER - TY - JOUR AU - Macleod, M. R. AU - Michie, S. AU - Roberts, I. AU - Dirnagl, U. AU - Chalmers, I. AU - Ioannidis, J. P. PY - 2014 DA - 2014// TI - Biomedical research: increasing value, reducing waste JO - Lancet VL - 383 UR - https://doi.org/10.1016/S0140-6736(13)62329-6 DO - 10.1016/S0140-6736(13)62329-6 ID - Macleod2014 ER - TY - BOOK PY - 2013 DA - 2013// TI - How Science Goes Wrong ID - ref14 ER - TY - JOUR AU - Munafò, M. R. AU - Nosek, B. A. AU - Bishop, D. V. M. AU - Button, K. S. AU - Chambers, C. D. AU - Percie du Sert, N. PY - 2017 DA - 2017// TI - A manifesto for reproducible science JO - Nature Human Behaviour VL - 1 UR - https://doi.org/10.1038/s41562-016-0021 DO - 10.1038/s41562-016-0021 ID - Munafò2017 ER - TY - JOUR AU - Macilwain, C. PY - 2010 DA - 2010// TI - Science economics: what science is really worth JO - Nature VL - 465 UR - https://doi.org/10.1038/465682a DO - 10.1038/465682a ID - Macilwain2010 ER - TY - STD TI - Unknown quantities. Nature. 2010;465(7299):665–6. ID - ref17 ER - TY - STD TI - Must try harder. Nature. 2012;483(7391):509. ID - ref18 ER - TY - STD TI - Science – The Endless Frontier. A Report to the President by Vannevar Bush, Director of the Office of Scientific Research and Development, July 1945. https://www.nsf.gov/od/lpa/nsf50/vbush1945.htm. Accessed 8 Dec 2017. UR - https://www.nsf.gov/od/lpa/nsf50/vbush1945.htm ID - ref19 ER - TY - JOUR AU - Sherwin, C. W. AU - Isenson, R. S. PY - 1967 DA - 1967// TI - Project hindsight. A Defense Department study of the utility of research JO - Science VL - 156 UR - https://doi.org/10.1126/science.156.3782.1571 DO - 10.1126/science.156.3782.1571 ID - Sherwin1967 ER - TY - JOUR AU - Comroe, J. H. AU - Dripps, R. D. PY - 1976 DA - 1976// TI - Scientific basis for the support of biomedical science JO - Science VL - 192 UR - https://doi.org/10.1126/science.769161 DO - 10.1126/science.769161 ID - Comroe1976 ER - TY - JOUR AU - Griliches, Z. PY - 1992 DA - 1992// TI - The Search for R&D Spillovers JO - Scand J Econ VL - 94 UR - https://doi.org/10.2307/3440244 DO - 10.2307/3440244 ID - Griliches1992 ER - TY - JOUR AU - Kostoff, R. N. PY - 1994 DA - 1994// TI - Research impact assessment. Principles and applications to proposed, ongoing, and completed projects JO - Invest Radiol VL - 29 UR - https://doi.org/10.1097/00004424-199409000-00013 DO - 10.1097/00004424-199409000-00013 ID - Kostoff1994 ER - TY - JOUR AU - Pielke, R. PY - 2010 DA - 2010// TI - In Retrospect: Science – The Endless Frontier JO - Nature VL - 466 UR - https://doi.org/10.1038/466922a DO - 10.1038/466922a ID - Pielke2010 ER - TY - STD TI - Marjanovic S, Hanney S, Wooding S. A Historical Reflection on Research Evaluation Studies, Their Recurrent Themes and Challenges. http://www.rand.org/pubs/technical_reports/TR789.html. Accessed 8 Dec 2017. UR - http://www.rand.org/pubs/technical_reports/TR789.html ID - ref25 ER - TY - STD TI - Higher Education Funding Council for England. Assessment Framework and Guidance on Submissions. http://www.ref.ac.uk/2014/pubs/2011-02/. Accessed 22 Jan 2018. UR - http://www.ref.ac.uk/2014/pubs/2011-02/ ID - ref26 ER - TY - STD TI - European Commission. Past Evaluations: Evaluation of FP7. https://ec.europa.eu/research/evaluations/index_en.cfm?pg=fp7. Accessed 8 Dec 2017. UR - https://ec.europa.eu/research/evaluations/index_en.cfm?pg=fp7 ID - ref27 ER - TY - STD TI - European Commission. Horizon 2020 Evaluations. https://ec.europa.eu/research/evaluations/index_en.cfm?pg=h2020evaluation. Accessed 8 Dec 2017. UR - https://ec.europa.eu/research/evaluations/index_en.cfm?pg=h2020evaluation ID - ref28 ER - TY - STD TI - European Commission. High Level Group on Maximising Impact of EU Research and Innovation Programmes. https://ec.europa.eu/research/evaluations/index_en.cfm?pg=hlg. Accessed 8 Dec 2017. UR - https://ec.europa.eu/research/evaluations/index_en.cfm?pg=hlg ID - ref29 ER - TY - STD TI - van den Akker W, Spaapen J. Productive Interactions: Societal Impact of Academic Research in the Knowledge Society. https://www.leru.org/publications/productive-interactions-societal-impact-of-academic-research-in-the-knowledge-society. Accessed 22 Jan 2018. UR - https://www.leru.org/publications/productive-interactions-societal-impact-of-academic-research-in-the-knowledge-society ID - ref30 ER - TY - JOUR AU - Buxton, M. AU - Hanney, S. PY - 1996 DA - 1996// TI - How can payback from health research be assessed? JO - J Health Serv Res Policy VL - 1 UR - https://doi.org/10.1177/135581969600100107 DO - 10.1177/135581969600100107 ID - Buxton1996 ER - TY - JOUR AU - Hanney, S. R. AU - Gonzalez-Block, M. A. AU - Buxton, M. J. AU - Kogan, M. PY - 2003 DA - 2003// TI - The utilisation of health research in policy-making: concepts, examples and methods of assessment JO - Health Res Policy Syst VL - 1 UR - https://doi.org/10.1186/1478-4505-1-2 DO - 10.1186/1478-4505-1-2 ID - Hanney2003 ER - TY - JOUR AU - Hanney, S. R. AU - Grant, J. AU - Wooding, S. AU - Buxton, M. J. PY - 2004 DA - 2004// TI - Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK's 'Arthritis Research Campaign' JO - Health Res Policy Syst VL - 2 UR - https://doi.org/10.1186/1478-4505-2-4 DO - 10.1186/1478-4505-2-4 ID - Hanney2004 ER - TY - JOUR AU - Donovan, C. AU - Hanney, S. PY - 2011 DA - 2011// TI - The ‘Payback Framework’ explained JO - Res Eval VL - 20 UR - https://doi.org/10.3152/095820211X13118583635756 DO - 10.3152/095820211X13118583635756 ID - Donovan2011 ER - TY - JOUR AU - Greenhalgh, T. AU - Fahy, N. PY - 2015 DA - 2015// TI - Research impact in the community-based health sciences: an analysis of 162 case studies from the 2014 UK Research Excellence Framework JO - BMC Med VL - 13 UR - https://doi.org/10.1186/s12916-015-0467-4 DO - 10.1186/s12916-015-0467-4 ID - Greenhalgh2015 ER - TY - STD TI - King’s College London, Digital Science. The Nature, Scale and Beneficiaries of Research Impact: An Initial Analysis of Research Excellence Framework (REF) 2014 Impact Case Studies. http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/Analysis,of,REF,impact/Analysis_of_REF_impact.pdf. Accessed 8 Dec 2017. UR - http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/Analysis,of,REF,impact/Analysis_of_REF_impact.pdf ID - ref36 ER - TY - JOUR AU - Glover, M. AU - Buxton, M. AU - Guthrie, S. AU - Hanney, S. AU - Pollitt, A. AU - Grant, J. PY - 2014 DA - 2014// TI - Estimating the returns to UK publicly funded cancer-related research in terms of the net value of improved health outcomes JO - BMC Med VL - 12 UR - https://doi.org/10.1186/1741-7015-12-99 DO - 10.1186/1741-7015-12-99 ID - Glover2014 ER - TY - JOUR AU - Sussex, J. AU - Feng, Y. AU - Mestre-Ferrandiz, J. AU - Pistollato, M. AU - Hafner, M. AU - Burridge, P. PY - 2016 DA - 2016// TI - Quantifying the economic impact of government and charity funding of medical research on private research and development funding in the United Kingdom JO - BMC Med VL - 14 UR - https://doi.org/10.1186/s12916-016-0564-z DO - 10.1186/s12916-016-0564-z ID - Sussex2016 ER - TY - STD TI - Health Economics Research Group, Office of Health Economics, RAND Europe. Medical Research: What's It Worth? https://www.ohe.org/publications/medical-research-whats-it-worth. Accessed 8 Dec 2017. UR - https://www.ohe.org/publications/medical-research-whats-it-worth ID - ref39 ER - TY - STD TI - Haskel J, Hughes A, Bascavusoglu-Moreau E. The Economic Significance of the UK Science Base: A Report for the Campaign for Science and Engineering. http://www.sciencecampaign.org.uk/resource/UKScienceBase.html. Accessed 8 Dec 2017. UR - http://www.sciencecampaign.org.uk/resource/UKScienceBase.html ID - ref40 ER - TY - STD TI - Medical Research Council. Outputs, Outcomes and Impact of MRC Research: 2014/15 Report. https://www.mrc.ac.uk/successes/outputs-report/. Accessed 8 Dec 2017. UR - https://www.mrc.ac.uk/successes/outputs-report/ ID - ref41 ER - TY - STD TI - Research Councils UK. Research Outcomes Overview. http://www.rcuk.ac.uk/research/researchoutcomes/. Accessed 8 Dec 2017. UR - http://www.rcuk.ac.uk/research/researchoutcomes/ ID - ref42 ER - TY - JOUR AU - Hinrichs, S. AU - Grant, J. PY - 2015 DA - 2015// TI - A new resource for identifying and assessing the impacts of research JO - BMC Med VL - 13 UR - https://doi.org/10.1186/s12916-015-0364-x DO - 10.1186/s12916-015-0364-x ID - Hinrichs2015 ER - TY - JOUR AU - Ovseiko, P. V. AU - Oancea, A. AU - Buchan, A. M. PY - 2012 DA - 2012// TI - Assessing research impact in academic clinical medicine: a study using Research Excellence Framework pilot impact indicators JO - BMC Health Serv Res VL - 12 UR - https://doi.org/10.1186/1472-6963-12-478 DO - 10.1186/1472-6963-12-478 ID - Ovseiko2012 ER - TY - JOUR AU - Hanney, S. R. AU - Castle-Clarke, S. AU - Grant, J. AU - Guthrie, S. AU - Henshall, C. AU - Mestre-Ferrandiz, J. PY - 2015 DA - 2015// TI - How long does biomedical research take? Studying the time taken between biomedical and health research and its translation into products, policy, and practice JO - Health Res Policy Syst VL - 13 UR - https://doi.org/10.1186/1478-4505-13-1 DO - 10.1186/1478-4505-13-1 ID - Hanney2015 ER - TY - JOUR AU - Morris, Z. S. AU - Wooding, S. AU - Grant, J. PY - 2011 DA - 2011// TI - The answer is 17 years, what is the question: understanding time lags in translational research JO - J R Soc Med VL - 104 UR - https://doi.org/10.1258/jrsm.2011.110180 DO - 10.1258/jrsm.2011.110180 ID - Morris2011 ER - TY - JOUR AU - Ovseiko, P. V. AU - Greenhalgh, T. AU - Adam, P. AU - Grant, J. AU - Hinrichs-Krapels, S. AU - Graham, K. E. PY - 2016 DA - 2016// TI - A global call for action to include gender in research impact assessment JO - Health Res Policy Syst VL - 14 UR - https://doi.org/10.1186/s12961-016-0126-z DO - 10.1186/s12961-016-0126-z ID - Ovseiko2016 ER - TY - JOUR AU - Ovseiko, P. V. AU - Edmunds, L. D. AU - Pololi, L. H. AU - Greenhalgh, T. AU - Kiparoglou, V. AU - Henderson, L. R. PY - 2016 DA - 2016// TI - Markers of achievement for assessing and monitoring gender equity in translational research organisations: a rationale and study protocol JO - BMJ Open VL - 6 UR - https://doi.org/10.1136/bmjopen-2015-009022 DO - 10.1136/bmjopen-2015-009022 ID - Ovseiko2016 ER - TY - JOUR AU - Kalpazidou Schmidt, E. AU - Cacace, M. PY - 2017 DA - 2017// TI - Addressing gender inequality in science: the multifaceted challenge of assessing impact JO - Res Eval VL - 26 UR - https://doi.org/10.1093/reseval/rvx003 DO - 10.1093/reseval/rvx003 ID - Kalpazidou Schmidt2017 ER - TY - JOUR AU - Pollitt, A. AU - Potoglou, D. AU - Patil, S. AU - Burge, P. AU - Guthrie, S. AU - King, S. PY - 2016 DA - 2016// TI - Understanding the relative valuation of research impact: a best–worst scaling experiment of the general public and biomedical and health researchers JO - BMJ Open VL - 6 UR - https://doi.org/10.1136/bmjopen-2015-010916 DO - 10.1136/bmjopen-2015-010916 ID - Pollitt2016 ER - TY - JOUR AU - Adam, P. AU - Solans-Domènech, M. AU - Pons, J. M. V. AU - Aymerich, M. AU - Berra, S. AU - Guillamon, I. PY - 2012 DA - 2012// TI - Assessment of the impact of a clinical and health services research call in Catalonia JO - Res Eval VL - 21 UR - https://doi.org/10.1093/reseval/rvs024 DO - 10.1093/reseval/rvs024 ID - Adam2012 ER - TY - JOUR AU - Solans-Domènech, M. AU - Adam, P. AU - Guillamón, I. AU - Permanyer-Miralda, G. AU - Pons, J. M. AU - Escarrabill, J. PY - 2013 DA - 2013// TI - Impact of clinical and health services research projects on decision-making: a qualitative study JO - Health Res Policy Syst VL - 11 UR - https://doi.org/10.1186/1478-4505-11-15 DO - 10.1186/1478-4505-11-15 ID - Solans-Domènech2013 ER - TY - JOUR AU - Aymerich, M. AU - Carrion, C. AU - Gallo, P. AU - Garcia, M. AU - López-Bermejo, A. AU - Quesada, M. PY - 2012 DA - 2012// TI - Measuring the payback of research activities: A feasible ex-post evaluation methodology in epidemiology and public health JO - Soc Sci Med VL - 75 UR - https://doi.org/10.1016/j.socscimed.2012.03.044 DO - 10.1016/j.socscimed.2012.03.044 ID - Aymerich2012 ER - TY - STD TI - Spaapen J, Dijstelbloem H, Wamelink F. Evaluating Research in Context: A Method for Comprehensive Assessment. Second edn. The Hague: Consultative Committee of Sector Councils for Research and Development (COS); 2007. ID - ref54 ER - TY - JOUR AU - Spaapen, J. AU - Drooge, L. PY - 2011 DA - 2011// TI - Introducing ‘productive interactions’ in social impact assessment JO - Res Eval VL - 20 UR - https://doi.org/10.3152/095820211X12941371876742 DO - 10.3152/095820211X12941371876742 ID - Spaapen2011 ER - TY - STD TI - Association of Universities in the Netherlands (VSNU), Netherlands Organisation for Scientific Research (NWO), Royal Netherlands Academy of Arts and Sciences (KNAW). Standard Evaluation Protocol 2015-2021: Protocol for Research Assessments in the Netherlands. https://www.knaw.nl/nl/actueel/publicaties/standard-evaluation-protocol-2015-2021. Accessed 8 Dec 2017. UR - https://www.knaw.nl/nl/actueel/publicaties/standard-evaluation-protocol-2015-2021 ID - ref56 ER - TY - STD TI - National Science Foundation. Broader Impacts Improving Society. https://www.nsf.gov/od/oia/special/broaderimpacts/. Accessed 8 Dec 2017. UR - https://www.nsf.gov/od/oia/special/broaderimpacts/ ID - ref57 ER - TY - STD TI - US Department of Health and Human Services. STAR METRICS® -- Science and Technology for America's Reinvestment Measuring the EffecTs of Research on Innovation, Competitiveness and Science. https://www.starmetrics.nih.gov/. Accessed 8 Dec 2017. UR - https://www.starmetrics.nih.gov/ ID - ref58 ER - TY - JOUR AU - Trochim, W. M. AU - Marcus, S. E. AU - Mâsse, L. C. AU - Moser, R. P. AU - Weld, P. C. PY - 2008 DA - 2008// TI - The evaluation of large research initiatives JO - Am J Eval VL - 29 UR - https://doi.org/10.1177/1098214007309280 DO - 10.1177/1098214007309280 ID - Trochim2008 ER - TY - JOUR AU - Sarli, C. C. AU - Dubinsky, E. K. AU - Holmes, K. L. PY - 2010 DA - 2010// TI - Beyond citation analysis: a model for assessment of research impact JO - J Med Libr Assoc VL - 98 UR - https://doi.org/10.3163/1536-5050.98.1.008 DO - 10.3163/1536-5050.98.1.008 ID - Sarli2010 ER - TY - BOOK AU - Buxton, M. J. AU - Schneider, W. L. PY - 1999 DA - 1999// TI - Assessing the Payback from AHFMR-funded Research PB - Alberta Heritage Foundation for Medical Research CY - Edmonton ID - Buxton1999 ER - TY - JOUR AU - Frank, C. AU - Nason, E. PY - 2009 DA - 2009// TI - Health research: measuring the social, health and economic benefits JO - CMAJ VL - 180 UR - https://doi.org/10.1503/cmaj.090016 DO - 10.1503/cmaj.090016 ID - Frank2009 ER - TY - BOOK PY - 2009 DA - 2009// TI - Making an Impact: A Preferred Framework and Indicators to Measure Returns on Investment in Health Research PB - Canadian Academy of Health Sciences CY - Ottawa ID - ref63 ER - TY - JOUR AU - Caddell, A. J. AU - Hatchette, J. E. AU - McGrath, P. J. PY - 2010 DA - 2010// TI - Examining the impact of health research facilitated by small peer-reviewed research operating grants in a women's and children's health centre JO - BMC Res Notes VL - 3 UR - https://doi.org/10.1186/1756-0500-3-107 DO - 10.1186/1756-0500-3-107 ID - Caddell2010 ER - TY - JOUR AU - Montague, S. AU - Valentim, R. PY - 2010 DA - 2010// TI - Evaluation of R&D: from ‘prescriptions for justifying’ to ‘user-oriented guidance for learning’ JO - Res Eval VL - 19 UR - https://doi.org/10.3152/095820210X12827366906562 DO - 10.3152/095820210X12827366906562 ID - Montague2010 ER - TY - JOUR AU - Graham, K. E. R. AU - Chorzempa, H. L. AU - Valentine, P. A. AU - Magnan, J. PY - 2012 DA - 2012// TI - Evaluating health research impact: Development and implementation of the Alberta Innovates – Health Solutions impact framework JO - Res Eval VL - 21 UR - https://doi.org/10.1093/reseval/rvs027 DO - 10.1093/reseval/rvs027 ID - Graham2012 ER - TY - JOUR AU - Dobrow, M. J. AU - Miller, F. A. AU - Frank, C. AU - Brown, A. D. PY - 2017 DA - 2017// TI - Understanding relevance of health research: considerations in the context of research impact assessment JO - Health Res Policy Syst VL - 15 UR - https://doi.org/10.1186/s12961-017-0188-6 DO - 10.1186/s12961-017-0188-6 ID - Dobrow2017 ER - TY - JOUR AU - Shah, S. AU - Ward, J. E. PY - 2001 DA - 2001// TI - Outcomes from NHMRC public health research project grants awarded in 1993 JO - Aust N Z J Public Health VL - 25 UR - https://doi.org/10.1111/j.1467-842X.2001.tb00325.x DO - 10.1111/j.1467-842X.2001.tb00325.x ID - Shah2001 ER - TY - JOUR AU - Clay, M. A. AU - Donovan, C. AU - Butler, L. AU - Oldenburg, B. F. PY - 2006 DA - 2006// TI - The returns from cardiovascular research: the impact of the National Heart Foundation of Australia's investment JO - Med J Aust VL - 185 ID - Clay2006 ER - TY - JOUR AU - Kingwell, B. A. AU - Anderson, G. P. AU - Duckett, S. J. AU - Hoole, E. A. AU - Jackson-Pulver, L. R. AU - Khachigian, L. M. PY - 2006 DA - 2006// TI - Evaluation of NHMRC funded research completed in 1992, 1997 and 2003: gains in knowledge, health and wealth JO - Med J Aust VL - 184 ID - Kingwell2006 ER - TY - JOUR AU - Kalucy, E. C. AU - Jackson-Bowers, E. AU - McIntyre, E. AU - Reed, R. PY - 2009 DA - 2009// TI - The feasibility of determining the impact of primary health care research projects using the Payback Framework JO - Health Res Policy Syst VL - 7 UR - https://doi.org/10.1186/1478-4505-7-11 DO - 10.1186/1478-4505-7-11 ID - Kalucy2009 ER - TY - JOUR AU - Reed, R. L. AU - Kalucy, E. C. AU - Jackson-Bowers, E. AU - McIntyre, E. PY - 2011 DA - 2011// TI - What research impacts do Australian primary health care researchers expect and achieve? JO - Health Res Policy Syst VL - 9 UR - https://doi.org/10.1186/1478-4505-9-40 DO - 10.1186/1478-4505-9-40 ID - Reed2011 ER - TY - JOUR AU - Schapper, C. C. AU - Dwyer, T. AU - Tregear, G. W. AU - Aitken, M. AU - Clay, M. A. PY - 2012 DA - 2012// TI - Research performance evaluation: the experience of an independent medical research institute JO - Aust Health Rev VL - 36 UR - https://doi.org/10.1071/AH11057 DO - 10.1071/AH11057 ID - Schapper2012 ER - TY - JOUR AU - Milat, A. J. AU - Laws, R. AU - King, L. AU - Newson, R. AU - Rychetnik, L. AU - Rissel, C. PY - 2013 DA - 2013// TI - Policy and practice impacts of applied research: a case study analysis of the New South Wales Health Promotion Demonstration Research Grants Scheme 2000–2006 JO - Health Res Policy Syst VL - 11 UR - https://doi.org/10.1186/1478-4505-11-5 DO - 10.1186/1478-4505-11-5 ID - Milat2013 ER - TY - JOUR AU - Donovan, C. AU - Butler, L. AU - Butt, A. J. AU - Jones, T. H. AU - Hanney, S. R. PY - 2014 DA - 2014// TI - Evaluation of the impact of National Breast Cancer Foundation-funded research JO - Med J Aust VL - 200 UR - https://doi.org/10.5694/mja13.10798 DO - 10.5694/mja13.10798 ID - Donovan2014 ER - TY - JOUR AU - Cohen, G. AU - Schroeder, J. AU - Newson, R. AU - King, L. AU - Rychetnik, L. AU - Milat, A. J. PY - 2015 DA - 2015// TI - Does health intervention research have real world policy and practice impacts: testing a new impact assessment tool JO - Health Res Policy Syst VL - 13 UR - https://doi.org/10.1186/1478-4505-13-3 DO - 10.1186/1478-4505-13-3 ID - Cohen2015 ER - TY - STD TI - Commonwealth Scientific and Industrial Research Organisation (CSIRO). Our Impact. https://www.csiro.au/en/About/Our-impact. Accessed 8 Dec 2017. UR - https://www.csiro.au/en/About/Our-impact ID - ref77 ER - TY - STD TI - Australian Research Council. Engagement and Impact Assessment. http://www.arc.gov.au/engagement-and-impact-assessment. Accessed 8 Dec 2017. UR - http://www.arc.gov.au/engagement-and-impact-assessment ID - ref78 ER - TY - STD TI - Angelelli P, Gordon A, Di Marzo E, Peirano F, Moldovan P, Codner D. Investigación científica e innovación tecnológica en Argentina: Impacto de los fondos de la Agencia Nacional de Promoción Científica y Tecnológica. https://publications.iadb.org/handle/11319/382?locale-attribute=pt#sthash.gLmho4ez.dpuf. Accessed 8 Dec 2017. UR - https://publications.iadb.org/handle/11319/382?locale-attribute=pt#sthash.gLmho4ez.dpuf ID - ref79 ER - TY - JOUR AU - Angulo-Tuesta, A. AU - Santos, L. M. P. PY - 2015 DA - 2015// TI - Evaluation of the impact of maternal and neonatal morbidity and mortality research funded by the Ministry of Health in Brazil JO - Res Eval VL - 24 UR - https://doi.org/10.1093/reseval/rvv022 DO - 10.1093/reseval/rvv022 ID - Angulo-Tuesta2015 ER - TY - JOUR AU - Brambila, C. AU - Ottolenghi, E. AU - Marin, C. AU - Bertrand, J. T. PY - 2007 DA - 2007// TI - Getting results used: evidence from reproductive health programmatic research in Guatemala JO - Health Policy Plann VL - 22 UR - https://doi.org/10.1093/heapol/czm013 DO - 10.1093/heapol/czm013 ID - Brambila2007 ER - TY - JOUR AU - Kwan, P. AU - Johnston, J. AU - Fung, A. Y. AU - Chong, D. S. AU - Collins, R. A. AU - Lo, S. V. PY - 2007 DA - 2007// TI - A systematic evaluation of payback of publicly funded health and health services research in Hong Kong JO - BMC Health Serv Res VL - 7 UR - https://doi.org/10.1186/1472-6963-7-121 DO - 10.1186/1472-6963-7-121 ID - Kwan2007 ER - TY - JOUR AU - Probandari, A. AU - Widjanarko, B. AU - Mahendradhata, Y. AU - Sanjoto, H. AU - Cerisha, A. AU - Nungky, S. PY - 2016 DA - 2016// TI - The path to impact of operational research on tuberculosis control policies and practices in Indonesia JO - Glob Health Action VL - 9 UR - https://doi.org/10.3402/gha.v9.29866 DO - 10.3402/gha.v9.29866 ID - Probandari2016 ER - TY - JOUR AU - Yazdizadeh, B. AU - Majdzadeh, R. AU - Janani, L. AU - Mohtasham, F. AU - Nikooee, S. AU - Mousavi, A. PY - 2016 DA - 2016// TI - An assessment of health research impact in Iran JO - Health Res Policy Syst VL - 14 UR - https://doi.org/10.1186/s12961-016-0129-9 DO - 10.1186/s12961-016-0129-9 ID - Yazdizadeh2016 ER - TY - BOOK AU - Grant, J. AU - Culbertson, S. AU - Al-Khater, L. AU - Al-Heidous, A. AU - Pollitt, A. AU - Castle-Clarke, S. PY - 2013 DA - 2013// TI - QNRF Impact Measurement Framework PB - QNRF CY - Doha ID - Grant2013 ER - TY - STD TI - Morgan Jones M, Grant J, et al. Making the grade: methodologies for assessing and evidencing research impact. In: Dean A, Wykes M, Stevens H, editors. Seven Essays on Impact. DESCRIBE project report for JISC. Exeter: University of Exeter; 2013. p. 25–43. ID - ref86 ER - TY - STD TI - Guthrie S, Wamae W, Diepeveen S, Wooding S, Grant J. Measuring Research: A Guide to Research Evaluation Frameworks and Tools. http://www.rand.org/pubs/monographs/MG1217.html. Accessed 8 Dec 2017. UR - http://www.rand.org/pubs/monographs/MG1217.html ID - ref87 ER - TY - JOUR AU - Chalmers, I. AU - Bracken, M. B. AU - Djulbegovic, B. AU - Garattini, S. AU - Grant, J. AU - Gulmezoglu, A. M. PY - 2014 DA - 2014// TI - How to increase value and reduce waste when research priorities are set JO - Lancet VL - 383 UR - https://doi.org/10.1016/S0140-6736(13)62229-1 DO - 10.1016/S0140-6736(13)62229-1 ID - Chalmers2014 ER - TY - JOUR AU - Ioannidis, J. P. AU - Greenland, S. AU - Hlatky, M. A. AU - Khoury, M. J. AU - Macleod, M. R. AU - Moher, D. PY - 2014 DA - 2014// TI - Increasing value and reducing waste in research design, conduct, and analysis JO - Lancet VL - 383 UR - https://doi.org/10.1016/S0140-6736(13)62227-8 DO - 10.1016/S0140-6736(13)62227-8 ID - Ioannidis2014 ER - TY - JOUR AU - Al-Shahi Salman, R. AU - Beller, E. AU - Kagan, J. AU - Hemminki, E. AU - Phillips, R. S. AU - Savulescu, J. PY - 2014 DA - 2014// TI - Increasing value and reducing waste in biomedical research regulation and management JO - Lancet VL - 383 UR - https://doi.org/10.1016/S0140-6736(13)62297-7 DO - 10.1016/S0140-6736(13)62297-7 ID - Al-Shahi Salman2014 ER - TY - JOUR AU - Chan, A. W. AU - Song, F. AU - Vickers, A. AU - Jefferson, T. AU - Dickersin, K. AU - Gotzsche, P. C. PY - 2014 DA - 2014// TI - Increasing value and reducing waste: addressing inaccessible research JO - Lancet VL - 383 UR - https://doi.org/10.1016/S0140-6736(13)62296-5 DO - 10.1016/S0140-6736(13)62296-5 ID - Chan2014 ER - TY - JOUR AU - Glasziou, P. AU - Altman, D. G. AU - Bossuyt, P. AU - Boutron, I. AU - Clarke, M. AU - Julious, S. PY - 2014 DA - 2014// TI - Reducing waste from incomplete or unusable reports of biomedical research JO - Lancet VL - 383 UR - https://doi.org/10.1016/S0140-6736(13)62228-X DO - 10.1016/S0140-6736(13)62228-X ID - Glasziou2014 ER - TY - JOUR AU - Hicks, D. AU - Wouters, P. AU - Waltman, L. AU - Rijcke, S. AU - Rafols, I. PY - 2015 DA - 2015// TI - Bibliometrics: The Leiden Manifesto for Research Metrics JO - Nature VL - 520 UR - https://doi.org/10.1038/520429a DO - 10.1038/520429a ID - Hicks2015 ER - TY - STD TI - Wilsdon J, Allen L, Belfiore E, Campbell P, Curry S, Hill S, et al. The metric tide: report of the independent review of the role of metrics in research assessment and management. http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/The,Metric,Tide/2015_metric_tide.pdf. Accessed 8 Dec 2017. UR - http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/The,Metric,Tide/2015_metric_tide.pdf ID - ref94 ER - TY - STD TI - The International School on Research Impact. What is Research Impact Assessment (RIA)? http://theinternationalschoolonria.com/whatisRIA.php. Accessed 8 Dec 2017. UR - http://theinternationalschoolonria.com/whatisRIA.php ID - ref95 ER - TY - STD TI - Times Higher Education. World University Rankings. https://www.timeshighereducation.com/world-university-rankings. Accessed 8 Dec 2017. UR - https://www.timeshighereducation.com/world-university-rankings ID - ref96 ER - TY - STD TI - ShanghaiRanking. Global Ranking of Academic Subjects. http://www.shanghairanking.com/. Accessed 8 Dec 2017. UR - http://www.shanghairanking.com/ ID - ref97 ER - TY - STD TI - Centre for Science and Technology Studies. The CWTS Leiden Ranking. http://www.leidenranking.com/. Accessed 8 Dec 2017. UR - http://www.leidenranking.com/ ID - ref98 ER - TY - JOUR AU - Saldaña, J. AU - Mallette, L. A. PY - 2017 DA - 2017// TI - Environmental coding: a new method using the SPELIT environmental analysis matrix JO - Qual Inq VL - 23 UR - https://doi.org/10.1177/1077800416679143 DO - 10.1177/1077800416679143 ID - Saldaña2017 ER - TY - STD TI - Wooding S, Hanney S, Buxton M, Grant J. The Returns from Arthritis Research. Volume 1: Approach, Analysis and Recommendations. www.rand.org/content/dam/rand/pubs/monographs/2004/RAND_MG251.pdf. Accessed 8 Dec 2017. UR - http://www.rand.org/content/dam/rand/pubs/monographs/2004/RAND_MG251.pdf ID - ref100 ER - TY - STD TI - Wooding S, Hanney S, Pollitt A, Buxton M, Grant J. Project Retrosight. Understanding the Returns from Cardiovascular and Stroke Research: The Policy Report. www.rand.org/content/dam/rand/pubs/monographs/2011/RAND_MG1079.pdf. Accessed 8 Dec 2017. UR - http://www.rand.org/content/dam/rand/pubs/monographs/2011/RAND_MG1079.pdf ID - ref101 ER - TY - STD TI - Wooding S, Pollitt A, Castle-Clarke S, Cochran G, Diepeveen S, Guthrie S, et al. Mental Health Retrosight: Understanding the Returns from Research (Lessons from Schizophrenia): Policy Report. https://www.rand.org/pubs/research_reports/RR325.html. Accessed 8 Dec 2017. UR - https://www.rand.org/pubs/research_reports/RR325.html ID - ref102 ER - TY - STD TI - Guthrie S, Kirtley A, Garrod B, Pollitt A, Grant J, Wooding S. A 'DECISIVE' Approach to Research Funding: Lessons from Three Retrosight Studies. https://www.rand.org/pubs/research_reports/RR1132.html. Accessed 8 Dec 2017. UR - https://www.rand.org/pubs/research_reports/RR1132.html ID - ref103 ER - TY - STD TI - Australian Research Council. Excellence in Research for Australia. http://www.arc.gov.au/excellence-research-australia. Accessed 8 Dec 2017. UR - http://www.arc.gov.au/excellence-research-australia ID - ref104 ER - TY - STD TI - ACIL Allen Consulting. Benefits Realisation Review of Excellence in Research for Australia: Final Report. http://www.arc.gov.au/sites/default/files/filedepot/Public/ERA/Benefits%20realisation%20review.pdf. Accessed 8 Dec 2017. UR - http://www.arc.gov.au/sites/default/files/filedepot/Public/ERA/Benefits%20realisation%20review.pdf ID - ref105 ER - TY - JOUR AU - Bryson, J. M. PY - 2004 DA - 2004// TI - What to do when stakeholders matter JO - Public Manage Rev VL - 6 UR - https://doi.org/10.1080/14719030410001675722 DO - 10.1080/14719030410001675722 ID - Bryson2004 ER - TY - JOUR AU - Mitchell, R. K. AU - Agle, B. R. AU - Wood, D. J. PY - 1997 DA - 1997// TI - Toward a theory of stakeholder identification and salience: defining the principle of who and what really counts JO - Acad Manage Rev VL - 22 ID - Mitchell1997 ER - TY - BOOK AU - Eden, C. AU - Ackermann, F. PY - 1998 DA - 1998// TI - Making Strategy: The Journey of Strategic Management PB - Sage CY - London ID - Eden1998 ER - TY - JOUR AU - Bryson, J. M. AU - Patton, M. Q. AU - Bowman, R. A. PY - 2011 DA - 2011// TI - Working with evaluation stakeholders: a rationale, step-wise approach and toolkit JO - Eval Program Plann VL - 34 UR - https://doi.org/10.1016/j.evalprogplan.2010.07.001 DO - 10.1016/j.evalprogplan.2010.07.001 ID - Bryson2011 ER - TY - BOOK PY - 2009 DA - 2009// TI - The Principles and Practice of Change PB - Palgrave Macmillan CY - Basingstoke ID - ref110 ER - TY - BOOK AU - Gibbons, M. AU - Limoges, C. AU - Nowotny, H. AU - Schwartzman, S. AU - Scott, P. AU - Trow, M. PY - 1994 DA - 1994// TI - The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies PB - Sage CY - London ID - Gibbons1994 ER - TY - JOUR AU - Gibbons, M. PY - 1999 DA - 1999// TI - Science's new social contract with society JO - Nature VL - 402 UR - https://doi.org/10.1038/35011576 DO - 10.1038/35011576 ID - Gibbons1999 ER - TY - BOOK AU - Nowotny, H. AU - Scott, P. AU - Gibbons, M. PY - 2001 DA - 2001// TI - Re-Thinking Science: Knowledge and the Public in an Age of Uncertainty PB - Polity Press CY - Cambridge ID - Nowotny2001 ER - TY - STD TI - National Institute for Health Research (NIHR). Patients and the Public. https://www.nihr.ac.uk/patients-and-public/. Accessed 8 Dec 2017. UR - https://www.nihr.ac.uk/patients-and-public/ ID - ref114 ER - TY - STD TI - Agency for Health Quality and Assessment of Catalonia (AQuAS). Convocatòries PERIS 2016-2020. http://aquas.gencat.cat/ca/projectes/mes_projectes/avaluacio_convocatories_recerca/PERIS/. Accessed 8 Dec 2017. UR - http://aquas.gencat.cat/ca/projectes/mes_projectes/avaluacio_convocatories_recerca/PERIS/ ID - ref115 ER - TY - STD TI - American Evaluation Association. American Evaluation Association Public Statement on Cultural Competence in Evaluation. http://www.eval.org/ccstatement. Accessed 8 Dec 2017. UR - http://www.eval.org/ccstatement ID - ref116 ER - TY - JOUR AU - Milat, A. J. AU - Bauman, A. E. AU - Redman, S. PY - 2015 DA - 2015// TI - A narrative review of research impact assessment models and methods JO - Health Res Policy Syst VL - 13 UR - https://doi.org/10.1186/s12961-015-0003-1 DO - 10.1186/s12961-015-0003-1 ID - Milat2015 ER - TY - JOUR AU - Greenhalgh, T. AU - Raftery, J. AU - Hanney, S. AU - Glover, M. PY - 2016 DA - 2016// TI - Research impact: a narrative review JO - BMC Med VL - 14 UR - https://doi.org/10.1186/s12916-016-0620-8 DO - 10.1186/s12916-016-0620-8 ID - Greenhalgh2016 ER - TY - JOUR AU - Banzi, R. AU - Moja, L. AU - Pistotti, V. AU - Facchini, A. AU - Liberati, A. PY - 2011 DA - 2011// TI - Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews JO - Health Res Policy Syst VL - 9 UR - https://doi.org/10.1186/1478-4505-9-26 DO - 10.1186/1478-4505-9-26 ID - Banzi2011 ER - TY - JOUR AU - Hanney, S. AU - Greenhalgh, T. AU - Blatch-Jones, A. AU - Glover, M. AU - Raftery, J. PY - 2017 DA - 2017// TI - The impact on healthcare, policy and practice from 36 multi-project research programmes: findings from two reviews JO - Health Res Policy Syst VL - 15 UR - https://doi.org/10.1186/s12961-017-0191-y DO - 10.1186/s12961-017-0191-y ID - Hanney2017 ER - TY - JOUR AU - Buxton, M. AU - Hanney, S. AU - Jones, T. PY - 2004 DA - 2004// TI - Estimating the economic value to societies of the impact of health research: a critical review JO - Bull World Health Organ VL - 82 ID - Buxton2004 ER - TY - STD TI - Davies R, Mayne J, Befani B, Forss K, Stame N, Stern E. DFID Working Paper 38. Broadening the Range of Designs and Methods for Impact Evaluations. https://www.gov.uk/dfid-research-outputs/dfid-working-paper-38-broadening-the-range-of-designs-and-methods-for-impact-evaluations. Accessed 8 Dec 2017. UR - https://www.gov.uk/dfid-research-outputs/dfid-working-paper-38-broadening-the-range-of-designs-and-methods-for-impact-evaluations ID - ref122 ER - TY - JOUR AU - Gagliardi, A. R. AU - Simunovic, M. AU - Langer, B. AU - Stern, H. AU - Brown, A. D. PY - 2005 DA - 2005// TI - Development of quality indicators for colorectal cancer surgery, using a 3-step modified Delphi approach JO - Can J Surg VL - 48 ID - Gagliardi2005 ER - TY - STD TI - Treasury HM, Cabinet Office, National Audit Office, Audit Commission, Office for National Statistics. Choosing the Right FABRIC: A Framework for Performance Information. London: TSO; 2001. ID - ref124 ER - TY - BOOK AU - Wooding, S. AU - Nason, E. AU - Starkey, T. AU - Hanney, S. AU - Grant, J. PY - 2009 DA - 2009// TI - Mapping the Impact: Exploring the Payback of Arthritis Research PB - RAND Europe CY - Cambridge ID - Wooding2009 ER - TY - STD TI - American Evaluation Association. American Evaluation Association Guiding Principles for Evaluators. http://www.eval.org/p/cm/ld/fid=51. Accessed 8 Dec 2017. UR - http://www.eval.org/p/cm/ld/fid=51 ID - ref126 ER - TY - STD TI - INVOLVE. Plain English Summaries. http://www.invo.org.uk/resource-centre/plain-english-summaries/. Accessed 8 Dec 2017. UR - http://www.invo.org.uk/resource-centre/plain-english-summaries/ ID - ref127 ER - TY - BOOK AU - Wenger, E. AU - McDermott, R. A. AU - Snyder, W. PY - 2002 DA - 2002// TI - Cultivating Communities of Practice: A Guide to Managing Knowledge PB - Harvard Business School Press CY - Boston ID - Wenger2002 ER - TY - JOUR AU - Wang, X. AU - Liu, C. AU - Mao, W. AU - Fang, Z. PY - 2015 DA - 2015// TI - The open access advantage considering citation, article usage and social media attention JO - Scientometrics VL - 103 UR - https://doi.org/10.1007/s11192-015-1547-0 DO - 10.1007/s11192-015-1547-0 ID - Wang2015 ER -