Measuring research impact in Australia’s medical research institutes: a scoping literature review of the objectives for and an assessment of the capabilities of research impact assessment frameworks

Deeming, Simon; Searles, Andrew; Reeves, Penny; Nilsson, Michael

doi:10.1186/s12961-017-0180-1

Table 2 Capacity for existing research impact assessment frameworks to address specific objectives

From: Measuring research impact in Australia’s medical research institutes: a scoping literature review of the objectives for and an assessment of the capabilities of research impact assessment frameworks

Conceptual framework	Exemplar/s	Relevant research spectrum^a	Primary methods	Accountability (top-down)	Account. (bottom-up)/transparency	Advocacy	Steering	Value for money	Management/learning & feedback	Speed of translation	Prospective orientation of research
Balanced Scorecard	Young and Lindquist [42]	T1–T4	Quant. indicators	Partial	Yes	Partial	Yes	Possible	Yes	Possible	Possible
Canadian Academy of Health Sciences (CAHS) Impact Framework	CAHS [5]	T1–T4	Mixed Methods	Yes	Yes	Yes	No	Yes	Yes	No	Possible
Canadian Institutes of Health Research (CIHR) Impact Framework	Bernstein, Hicks [29]	T1–T4	Quant. Indicators, case studies	Yes	Yes	Yes	Possible	Yes	Yes/possible	No	Possible
Comprehensive Research Metrics Logic Model	Engel-Cox, Van Houten [55]	T3, T4	Quant. Indicators	Yes/possible	Yes	Yes	Possible	Yes	Yes	No	Possible
Decision Making Impact Model	Lavis, Ross [27]	T3, T4	Mixed methods	Yes	Yes	Yes	Possible	No/partial	Yes	Partial/possible	Yes
Economic Impact Assessment	Deloitte [41], Glover, Buxton [34]	T1–T4	Economic assessment	Yes	No	Yes	No	Yes	Partial	No	No/possible
Excellence in Research for Australia	ARC [50]	T1–T4	Peer-review, Quant. Indicators	Partial	No	Partial	No	No	Partial	No	No
Health Services Research Impact Framework	Buykx, Humphreys [51]	T1–T4	Quant. Indicators, survey	Yes	Yes	Yes	No/possible	Partial	Yes/partial	No	Possible
Hunter Medical Research Institute Framework to Assess the Impact from Translational Health Research	Searles [21]	T1–T4	Mixed Methods	Yes	Yes	Yes	Possible	Yes	Yes	Yes	Yes
Institute for Translational Health Sciences Kellogg Logic Model – World Health Organization Health Services Assessment Model	Scott, Nagasawa [47]	T1–T4	Mixed Methods	Yes	Yes	Yes	Yes/possible	Partial	Yes	No	Possible
Lean/Six-sigma Models	Schweikhart and Dembe [43]	T1–T4	Quantitative Indicators	Partial	Yes	No	Possible/no	No	Partial	Yes/partial	Yes
Matrix Scoring System	Wiegers, Houser [53]	T1–T4	Quant. Indicators	Yes	Yes/partial	Yes	No	No	Yes	No	No
Measurement of Research Impact and Achievement	National Health & Medical Research Council [48] (Not implemented)	T1–T4	Quant. Indicators, Self-report (externally verifiable)	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes/possible
Payback Model of Health Research Benefits	Buxton and Hanney [45]	T1–T4	Mixed methods	Yes	Yes	Yes	No	Possible	Yes	No	Possible
Process Marker Model	Trochim, Kane [26]	T1–T4	Quant. Indicators	No	Yes	Possible	No	Possible	Yes	Yes	Possible
RE-AIM Model	Glasgow, Vogt [54]	T4	Quant. Indicators	Partial/yes	Yes	Yes	No	Partial	Yes	Partial	Possible
Research Engagement for Australia	Australian Academy of Technology and Engineering [33]	T2, T3, T4	Quant. Indicators	Partial	No	Partial	No	Partial	Partial	No	No
Research Excellence Framework	Ovseiko, Oancea [28], Khazragui and Hudson [35]	T1–T4	Quant. Indicators, survey	Yes	No	Yes	No	Possible	Yes/possible	No	Possible
Research Impact Framework	Kuruvilla [40]	T2, T3, T4	Self-assessed survey	No	Yes	Yes	No	No/partial	Yes	No	No
Research Performance Evaluation Framework	Schapper, Dwyer [32]	T1–T4	Quant. Indicators, self-assessed survey	Yes	Yes	Yes	Yes	Partial	Yes	Possible	Possible
Research Utilization Ladder	Landry, Amara [36]	T3, T4	Self-assessed survey	No	Partial	No	Possible	No	Partial/no	Partial/no	Possible
Societal Impact Framework	Mostert, Ellenbroek [52], Van Ark G [58]	T3, T4	Quant. Indicators	Partial	Yes	Yes	Possible	No	Yes	No	Possible
Telethon Kids Institute Research Impact Framework	Telethon Kids Institute [46]	T1–T4	Mixed methods	Yes	Yes	Yes	Possible	Yes	Yes	Possible	Possible
Translational Research Organizations Performance Model	Pozen and Kline [44]	T1–T4	Quant. Indicators	Yes	Yes	Yes	Possible	Partial	Yes	Yes	Yes
Weiss Logic Model	Weiss [49]	T1–T4	Mixed methods	Yes	Yes	Yes	Possible	Yes	Yes	Possible	Possible

^aUtilises the Khoury, Gwinn [37] definition of T1–T4
Yes/No – Met/Did not meet specified criteria; Partial – Meets criteria, but incomplete representation; Possible – With adaptions, capable of meeting criteria

Back to article page

ISSN: 1478-4505

Contact us

Submission enquiries: Access here and click Contact Us
General enquiries: info@biomedcentral.com

Health Research Policy and Systems

Contact us