Skip to main content

Table 1 Thirty-six impact assessment studies: methods, frameworks, findings, factors linked to impact achieved

From: The impact on healthcare, policy and practice from 36 multi-project research programmes: findings from two reviews

Author, date, location Programme/speciality Methods for assessing health research impact/concepts and techniques Impacts found Factors associated with level of impact; comments on methods and use of the findings
Adam et al., 2012 [32]; Catalonia, Spain Catalan Agency for Health Information, Assessment and Quality – Clinical and health services research Bibliometric analysis; surveys to researchers
(99, 70 responded, 71%); interviews – researchers (15), decision-makers (8); in-depth case study of translation pathways 
Canadian Academy of Health Sciences framework
Overall, 40 principal investigators (PIs) (of the 70) gave 50 examples of changes; examples included 12 organisational changes of the centre/institution; two public health management; two legal/regulatory (some PIs might have given more than one of these: therefore, total for organisational/management/policy changes: possibly 17–23%, and 20% figure used in this analysis); 29 of the 70 (41%): changed clinical practice Interactions and participation of healthcare and policy decision-makers in the projects were crucial to achieving impact; the study showed that the Agency achieved the aim of filling a gap in local knowledge needs; study provided useful lessons for informing the funding agency’s subsequent action; the studies “provide reasons to advocate for oriented research to fill specific knowledge gaps” ([32], p. 327)
Alberta Heritage Fund for Medical Research, 2003 [33]; Alberta, Canada Alberta Heritage Fund for Medical Research – Health research Survey to PIs (100, 50 responded, 50%); interviews with decision makers and users
Version of Payback Framework
49% impact on policy; 39% changed behaviour; 40% health sector benefits Research teams with decision-makers or users more successful than those without
Bodeau-Livinec et al., 2006 [34]; France French Committee for the Assessment and Dissemination of Technological Innovations (CEDIT) – Health technology assessment (HTA) Semi-directive interviews with stakeholders affected by the recommendations (14); case studies used surveys in hospitals to examine impact of the recommendations (13)
No framework stated, but approach to scoring impact followed earlier studies of the CETS in Quebec reported by Jacob et al. [47, 48]
Widespread interest, “used as decision-making tools by administrative staff and as a negotiating instrument by doctors in their dealings with management....ten of thirteen recommendations had an impact on the introduction of technology in health establishments” ([34], p. 161); 7 considerable,
3 moderate: total 77%
Main factor fostering compliance with recommendations “appears to be a system of regulation” ([34], p.166) Reviewed other studies: “All these experiences together with our own work suggest that the impact of HTA on practices and… introduction of new technologies is higher the more circumscribed is the target of the recommendation” ([34], p. 167)
Brambila et al., 2007 [35]; Guatemala Population Council – Programme of Operation Research projects in reproductive health in Guatemala Key informant (KI) interviews; document review; site visits to health centres and non-governmental organisations implementing operational research interventions; scored 22 projects (out of 44 conducted between 1988 and 2001) on indicators: 14 process; 11 impact; 6 context
Developed an approach involving process, impact and contextual factors; drew on literature such as Weiss [18] and interactive approaches
Of the 22, 13 projects intervention effective in improving results, three interventions not effective; in 14 studies implementing agency acted on results; nine interventions scaled up in same organisation; five adopted by another organisation in Guatemala; some studies led to policy changes, mainly at the programme level (total 64% impact in combined policy and practice category) Highlighted how impact can arise from a long-term approach and the several 5-year cycles of funding “allowed for the accumulation of evidence in addition to the development of collaborative ties between researchers and practitioners, which ultimately resulted in changes to the service delivery environments” ([35], p. 242)
Buxton et al., 1999 [36]; United Kingdom NHS North Thames Region – Wide-ranging responsive mode R&D programme Questionnaires to PIs (164, 115 responded, 70%) and some bibliometric analysis for all projects and case studies (19); case studies included interviews with researchers and some users
Benefit scoring system based on two criteria (importance of the research to the changes, and level at which the change was made) was used to score questionnaire responses about the impacts and re-score the impact from each study on which a case study conducted
Payback Framework
41% impact on policy; 43% change in practitioner/manager behaviour; 37% led to benefits to health and health service The survey/case study comparison suggests “greater detail and depth of the case studies often leads to a somewhat different judgement of payback, but there is no evidence of a systematic under-assessment of payback from the questionnaire approach, nor, generally, of greatly exaggerated claims being made by researchers in the self-completed questionnaires” ([36], p. 196)
Caddell et al., 2010 [37]; Canada IWK Health Centre, Halifax, Canada, Research Operating Grants (small grants) – Women and children’s health Online questionnaire to PIs and co-investigators (Co-Is) (64, 39 responded, 61%)
Research Impact Framework: adapted
16% policy impact: 8% in health centre, 8% beyond; 32% said resulted in a change in clinical practice; 55% informed clinical practice by providing broader clinical understanding and increased awareness (average of 43% for practice impact); 46% improved quality of care An association between presenting at conferences and practice impacts; authors stress link between research and excellence in healthcare: “It is essential that academic health centres engage actively in ensuring that a culture of research inquiry is maintained” ([37], p. 4)
Donovan et al., 2014 [38];Australia National Breast Cancer Foundation – Wide range of programmes Documentary analysis, bibliometrics, survey of PIs (242, 153 responded, 63%), 16 case studies, cross-case analysis
Payback Framework
10% impact on policy – 29% expected to do so; 11% contributed to product development; 14% impact on practice/behaviour – 39% expected to do so Basic research – more impact on knowledge and drug development; applied research – greater impact in other payback categories; many projects had only recently been completed – more impact expected; in launching the report the charity highlighted how it was informing their funding strategy [92]
Expert Panel for Health Directorate of the European Commission’s Research Innovation Directorate General, 2013 [39]; European Union European Union Framework Programmes 5, 6, and 7 – Public health projects Documentary review: all 70 completed projects; 120 ongoing; KI interviews with particularly successful and underperforming projects (16); data extraction form constructed based on the categories from the Payback Framework, with each of the main categories broken down into a series of specific questions
Payback Framework
Appendix 1: only 6 out of the 70 completed projects did not achieve the primary intended output; 42% took actions to engage or inform policymakers; 4 (6%) projects change of policy, 22% expected to do so; 7 (10%) impact on health practitioners; 6 (9%) impact on health service delivery and 6 (9%) impact on health; 1 beneficial impact on small/medium-sized enterprise Used documentary review, therefore for completed projects had data about whole set; however, “Extensive follow-up of the post-project impact of completed projects was not possible” ([39] p. 9)
Comprehensive coverage of a programme without requiring additional data from the researchers; however, also shows the limitations of such an approach in capturing later impacts
Ferguson et al., 1998 [40]; United Kingdom NHS Northern and Yorkshire Region – Health Services Research (HSR) (two other programmes not included here) Desk analysis (bibliometrics), surveys to gather quantitative and qualitative data sent to all PIs and Co-Is in all three programmes: but only HSR projects asked about policy, so just the 32 HSR responses analysed here
Refer to Payback Framework; no attempt to develop own
Five HSR projects (16%) had a policy impact, i.e. “Better informed commissioning and contracting” ([40] p. 17); 5 (16%) led to a change in NHS practice, i.e. “More effective treatment, screening or management for patients” ([40], p. 16) This was part of a wider analysis, but in all three areas the projects were reactive; particularly difficult to make an impact with Primary and Community Care research
Gold & Taylor, 2007 [41]; United States of America Agency for Healthcare Research and Quality – Integrated delivery systems research network Documentary review of programme as a whole and individual projects (50); descriptive interviews (85); four case studies, additional interviews
No explicit framework described
Changes in operations; “Of the 50 completed projects studied, 30 had an operational effect or use” [41] (Operational effect or use is a broad term: so the 60% put into our combined impact category) Success factors: responsiveness of project work to delivery system needs, ongoing funding, development of tools that helped users see their operational relevance
Gutman et al., 2009 [42]; United States of America Robert Wood Johnson Foundation – Active living research A retrospective, in-depth, descriptive study utilising multiple methods; quantitative data derived primarily from a web-based survey of grantee investigators (PIs, Co-PIs), of the 74 projects: 68 responses analysed; qualitative data from 88 interviews with KIs
The conceptual model used in the programme “was used to guide the evaluation” ([42], p. S23).
Aspects of Weiss's model used for analysing policy contributions
Generally thought to be too early for much policy impact, but 25% of survey, 43% of interviewees reported a policy impact; however, policy impact in survey could be from active living research in general, not just the specific programme, and could include: “a specific interaction with policymakers (e.g. testifying, meeting with policymakers, policymaker briefings, etc.) or direct evidence of the research findings in a written policy” ([42], p. S33) Only 16% of grants had been completed prior to the year of the evaluation; some approaches “worked well, including developing a multifaceted, ongoing, interactive relationship with advocacy and policymaker organizations” ([42], p. S32); grantees who completed both interviews and surveys generally gave similar responses, but researchers included in the random sample of interviewees gave higher percentage of policy impact than researchers surveyed; questions slightly different in the interviews than in the surveys
Hailey et al., 1990 [43]; Australia National Health Technology Advisory Panel – HTA reports Looked at technologies (20) covered by HTA reports from the panel up to end of 1988. Little provided on methods – presumably desk analysis, just states comparing recommendations, assessments and policy activities
No framework described
Out of the first 20 technologies covered by HTA reports there had been significant impact in 11 and probable influence in three: 70% in total Timing crucial factor for impact; at the margin of our inclusion criteria since it is based more on panel recommendations than a programme of research, but took first 20, not a selection
Hailey et al., 2000 [44]; Canada Canadian province (not stated) – HTA brief tech notes Interviews with those requesting the 20 brief HTA notes (i.e. reviews); checks on quality of the reports made using desk analysis and comments from experts
No framework described
14 (70%) had influence on policy and other decisions These HTA brief reviews were directly and urgently requested by users; at the margin of our inclusion criteria since it is not clear to what extent it was a research programme
Hanney et al., 2007 [6]; United Kingdom National Health Service (NHS) – HTA programme Multiple methods: literature review, funder documents, survey all PIs of projects between 1993 and 2003 (204, 133 responses, 65%), case studies with interviews (16)
Payback Framework
Technology Assessment Reports (TARs) produced for the National Institute for Health and Clinical Excellence (NICE): 96% impact on policy, 60% on clinician behaviour; primary and secondary HTA research: 60% impact on policy, 31% on behaviour
Average for programme: 73% impact on policy, 42% on behaviour; case studies showed large diversity in levels and forms of impacts and the way in which they arise
Different parts of the programme had different impact levels; key factors in achieving impact – agenda setting to meet needs of healthcare system, generally high scientific quality of research, existence of a range of ‘receptor bodies’ to receive and use findings, especially demonstrated for the NICE TARs; pre- and post-interview scoring showed reasonable correlations: suggests most survey responses not making exaggerated impact claims
Hanney et al., 2013 [45]; United Kingdom Asthma UK – All programmes of Asthma research Survey of all PIs (153, 96 responses, 59%), documents, case studies (14) involving interviews and some expanding the approach to cover role of chairs and centre
Payback Framework
13% impact on policy; 17% product development; 6% health gain; but case studies reveal some important examples of influence on guidelines, some potentially major breakthroughs in asthma therapies, establishment of pioneering collaborative research centre Many types of research and modes of funding – long-term funding of chairs led to important impacts; comparison of evidence from surveys and case studies on same projects showed generally exaggerated claims not made; study informed strategy of the medical research charity
Hera, 2014 [46]; Africa Africa Health Systems Initiative Support to African Research Partnerships Documentary review; interviews at programme level; project level information – for six projects, workshops, for the remaining four a total of 12 interviews; participant observation of end-of-programme workshop and presented some preliminary findings
Key element of the design – adoption of an interactive model of knowledge translation
Policy impact was created during the research process: 7 out of 10 projects reported policy impact already, “The policy dialogue is not yet complete and further uptake can be anticipated” ([46], p. 3) Research teams who started the policy dialogue early and maintained it throughout the study, and teams that engaged with decision-makers at local level, district and national levels simultaneously were more successful in translating research results into policy action” ([46], p. 1); timing of evaluation – too early for some impact, but programme’s interactive approach led to some policy impact during project
Jacob & Battista, 1993 [47]; Quebec, Canada Quebec Council on Health Care Technology Assessments (CETS) – HTA Case study analyses of impact on decision-making and cost savings of reports in first 4 years (10) Scored for policy influence – critical incidents used Interviews (45) with scientific and political partners, and staff at CETS; documentary analysis also used Desk analysis of cost savings
Developed own CETS approach
Examined impact on decision-making and cost savings; 8 of 10 reports influenced decisions Identified a series of key features of the Quebec system that were favourable to HTAs making an impact; these include “A general receptivity to rationality in decision making…the health care system in Quebec is organized in such a way that information produced by the council can filter easily into the decision-making process” ([47], p. 571); this is an example of the receptor body playing an important role
Jacob & McGregor, 1997 [48]; Quebec, Canada Quebec Council on Health Care Technology Assessments (CETS) – HTA Comprehensive case study approach; similar to above on 21 reports in circulation sufficiently long for at least some impact to be estimated
Used own CETS approach
18 of 21 reports influenced policy (86%); 8 at the highest level Context was same as above; “The best insurance for impact is a request by a decider that an evaluation be made” ([48], p. 78) (not entirely clear if these 21 reports included 10 reports above)
Johnston et al., 2006 [49]; United States of America National Institute of Neurological Disorders and Stroke – All pre 2000 phase III clinical trials in this field Data on the effects of all 28 trials from desk analysis involving reviews, contact with PIs and others, and opinions of experts (4)
Health economic modelling used to estimate return on investment (ROI)
ROI analysis – a key example of a monetisation study
Six trials (21%) led to improvements in health: 470,000 quality-adjusted life years in 10 years since funding of 28 trials at cost of $3.6bn; the projected net benefit was $15.2bn; yearly ROI 46% (in total 8 studies, 29%, were identified as providing impact used in the analysis: two were cost savings only) The main purpose of this study was to assess the public ROI; however, it seems to be the only such study that attempted to identify whether any health improvements had resulted from each individual project in a programme (and it thus met our inclusion criteria)
Kingwell et al., 2006 [50]; Australia National Health and Medical Research Council (NHMRC) – Wide range of fields Survey of all contactable PIs completing in 1997 using a simplified version of NHMRC end-of grant report as the survey instrument (259, 131 responses, 51%)
No explicit framework
9% affect health policy; 24% affected clinical practice, 14% public health practice (in our analysis used the 24% as not clear how many might be duplicates); commercial potential: 41% Highlighted some projects with clinically relevant outcomes for showcasing to the community
Kwan et al., 2007 [51]; Hong Kong Health and Health Services Research Fund – Range of fields Adapted Payback survey sent to PIs of completed projects (205, 178 responses, 87%); statistical analysis including multivariate analysis
Payback Framework
Use in policymaking, 35%; changed behaviour, 49%; health service benefit, 42% Multivariate analysis found that investigator participation in policy committees as a result of the research and liaison with potential users were significantly associated with health service benefit, policy and decision-making, and change in behaviour; however, set out various limitations in the methods used
McGregor et al., 2005 [52]; Canada HTA Unit of McGill University Health Centre, Quebec – HTA 16 HTA reports examined; no account of concepts or techniques; presumably desk analysis of documents, etc. and insider account and informed by previous studies in Quebec (see Jacob [47] and [48] above) All 16 (100%) HTA reports incorporated into hospital policy and some cost savings Hospital’s HTA Unit combined researchers to synthesise evidence and a policy committee to make recommendations; success because “(i) relevance (selection of topics by administration with on-site production of HTAs allowing them to incorporate local data and reflect local needs), (ii) timeliness, and (iii) formulation of policy reflecting community values by a local representative committee” ([52], p. 263)
Milat et al., 2013 [53]; Australia New South Wales Health Promotion Demonstration Research Grants Scheme Semi-structured interviews with Chief Investigators (CI) (17) and end-users (29) of the 15 projects; thematic coding of interview data and triangulation with other data sources to produce case studies for each project
Case studies individually assessed against four impact criteria and discussed to reach group assessment consensus
Banzi Research Impact Model
10 out of 15 (67%) were in the moderate or high categories for impact on policy and practice combined (we did not have an economic category in our analysis and therefore decided not to include the combined health, social and economic category where 33% of the projects were rated as resulting in moderate or high impact) High impact projects' success: “due to the nature and quality of the intervention itself…, high quality research, champions who advocated for adoption, and active dissemination strategies. Our findings also highlight the need for strong partnerships between researchers and policy makers/practitioners to increase ownership over the findings and commitment to action” ([53], p. 14)
Molas-Gallart et al., 2000 [54]; United Kingdom Economic and Social Research Council AIDS Programme – Social aspects of AIDS 43 interviews with researchers of all 14 completed projects, then snowball approach for users: mapped network of researchers and users and post-research activity
Framework based on the interconnection of three major elements: the type of output, the diffusion channels and the forms of impact – later contributed to development of Social Impact Assessment Methods through the study of Productive Interactions [23]
50% of researchers claimed programme provided non-academics with tools to solve problems and been used to develop policies Concludes a two to three stage process required to assess impact (interview researchers first, then users); normal sampling techniques inadequate because impact not distributed along a normal distribution curve; detailed project-by-project qualitative analysis important
Oortwijn et al., 2008 [55]; The Netherlands ZonMw Health Care Efficiency Research Programme – HTA Survey data collected from PIs (43, 34 responses, 79%); case study analysis (including 14 interviews) of five HTA projects; developed and applied a 2-round scoring system
Payback Framework
10 projects (29%) had a policy impact, including 6 being cited in guidelines; 11 projects (32%) reported implementation of new treatment strategies: counted as informed practice The assessment was perhaps too soon after completion of the projects to witness benefits for many of projects; unlike most HTA programmes this had a large responsive mode element and most studies were prospective clinical trials
Poortvliet et al., 2010 [56]; Belgium The Belgium Health Care Knowledge Centre (KCE) – HTA, HSR and good clinical practice Documentary review; two group discussions: with 11 KCE experts, with 2 KCE mangers; interviews with stakeholders (20); web-based survey – total of 88 managers reported on 126 projects; nine detailed case studies; international comparisons with three agencies using documentary/literature review and interviews (3)
Developed own framework
58% of project coordinators thought the project contributed to policy development: more for HTA than good clinical practice or HSR; 16 of the 20 stakeholders said findings influenced decision making, four said not in their organisation; 30% coordinators thought the project contributed to changes in healthcare practice Factors linked to impact include involvement from “stakeholders in agenda and priority setting. The quality of KCE research itself is high and in general beyond discussion. The relevance of KCE research findings is generally judged as high” ([56], p. 111–2); some similarities with other/earlier findings about HTA being more likely to make impact
Reed et al., 2011 [57]; Australia Primary care research Online survey to 41 contactable CIs (out of 59 projects); asked impacts expected, how many achieved; some projects excluded as still underway, other refused; 17 completed out of 27 eligible
Payback Framework
Four projects (24%) influenced national/state policymaking, but 8 (47%) influenced decision making at organisational, local or regional level (combined nine separate projects (53%) had policy/organisational decision impact); despite further examples of quite high levels of impact, surveys showed “these perceived impacts affected the health service organizations, clinicians and patients who took part in the research projects” ([57], p. 4) (we included the lowest of the three figures given for this, 29%) The high level of use for policy and organisational decision making “reflects a high level of engagement of the researchers with potential users of their research findings” ([57], p. 5)
RSM McClure Watters et al., 2012 [58]; Northern Ireland, United Kingdom Northern Ireland Executive: Health and Social Care Research – All fields Desk analysis of documents and literature, consultations with stakeholders, survey informed by Payback Framework, three case studies, benchmarking. Surveys to all PIs for projects funded between 1998 and 2011 who could be contacted (169; 84 responses, 50%)
Payback Framework
19% impact on policy development; for impact on health and the healthcare system: 20% health gain; 14% improvements in service delivery; 17% increased equity (the 20% figure used in our analysis represents the most conservative overall figure); substantial leveraged funds for follow-on projects came from outside Northern Ireland Because Northern Ireland’s government did not contribute to the United Kingdom’s NIHR, researchers were not able to apply to the NIHR programmes. This “was seen by respondents as a major constraint to research activity… research was not seen as a priority within many organisations and that many key stakeholders in the health sector did not fully engage with research or see its benefits” ([58], p. 49); as a result of the assessment, Northern Ireland decided to subscribe to the NIHR
Sainty, 2013 [59]; United Kingdom UK Occupational Therapy Research Foundation – Occupational therapy PIs of completed project invited to complete a ‘personalized impact assessment form’ (equivalent to a survey) (11, 8 responded, 73%)
Two researchers provided an independent review of the collated findings
Becker Medical Library Model
Three projects (37.5%) reported local clinical application: “particular tools, clinical advice, or models that were the subject of research having been used in practice” ([59], p. 534) In relation to the clinically related activities of three projects: “Important to note, was the extent to which respondents highlighted this as being in the context of the participating services or host organizations” ([59], p. 534)
Shah & Ward, 2001 [60]; Australia NHMRC – Public health R&D committee Self-complete questionnaires to CIs funded in 1993 (55, 38 responses, 69%); combined with desk analysis – attempted some correlations between publications and impact
No framework stated
58% claimed research influenced policy; 69% influence on practice; 53% stated both Influence on policy, practice or both was not associated with peer-reviewed publication in an Australian journal” ([60], p. 558)
Soper & Hanney, 2007 [61]; United Kingdom NHS Implementation Methods Programme – Implementation research Postal survey of PIs (36, 30 responses, 83%) and potential users of the three projects in maternity care (227, 100 responses, 44%); poor response from other users to electronic survey; some desk analysis; interviews with key figures
Payback Framework
30% claimed impact on policy; 27% on practice; 54% of the midwives and perinatal care researchers surveyed said the findings from at least one of the three maternity care projects had influenced their clinical practice In this new field, the programme generated considerable enthusiasm among members of advisory and commissioning groups, and increased understanding and interest in the field; some projects made considerable impact, but IMP did not have a communications strategy and as a programme it highlighted some of complexities facing implementation.
The Madrillon Group, 2011 [62]; United States of America NIH – Mind body interactions and health program Mixed methods cross-sectional evaluation design; semi-structured interviews with 100% response rate – PIs of all 44 investigator-initiated projects and all 15 centres; impacts of centres scored by adapting the scales used previous in payback studies
Adapted version of Payback Framework
Projects: 34% influenced policies; 48% led to improved health outcomes; the centres and projects, “produced clear and positive effects across all five of the Payback Framework research benefits categories” ([62], p. xiii) Some projects were still in progress and it was too early to capture all the ‘latent’ impacts; conducted innovative analysis through examining three overlapping levels (programme, centre and projects); for assessing all projects used semi-structured interviews rather than surveys
Wisely, 2001 [63]; United Kingdom NHS – National R&D programme on primary/secondary care interface Survey of PIs of projects completed by April 2001 (63, 40 responded, 63%); desk analysis comparing grades for applications and quality of project
Payback Framework
35% used in policy/decision making; 27% led to changes in practice; 25% health service benefits arisen Some indication from limited data that applications graded as excellent more likely to lead to high quality projects with impact
Wisely, 2001 [64]; United Kingdom NHS – National R&D programme, mother & child care Survey of PIs of projects completed by April 2001 (39, 26 responded, 67%)
Payback Framework
27% used in policy/decision making; 31% led to changes in practice; 23% health service benefits arisen Some PIs thought that being part of a national R&D programme helped give their project greater credibility in the eyes of potential users
Wooding et al., 2009 [65]; United Kingdom Arthritis Research Campaign – Wide range of arthritis research Web-based tick list survey of PIs in 2007 of grants ending in 2002 and 2006 (136, 118 responses, 87%)
Developed from the Payback Framework was subsequently named the RAND/ARC Impact Scoring System
6 projects (5%) policy impact; 8% “generated intellectual property that has been protected or is in the process of being so” ([65], p. 37) (over 80% of grants generated new research tools) Much of the research funded was more basic and likely to inform further research rather than directly lead to impacts; also, it was probably too soon after the end of the projects to capture all the impact that might arise
Zechmeister & Schumacher, 2012 [66]; Austria Institute for Technology Assessment and Ludwig Boltzmann Institute for HTA – HTA Desk analysis identified all HTA reports aimed at use in re-imbursement or for disinvestment – 11 full HTA reports, 58 rapid assessments Descriptive quantitative analysis of administrative data and 15 interviews with administrators and payers
Analysis informed by Quebec studies – see above, Jacob [47] and [48]
Five full HTA reports and 56 rapid assessments “were used for reimbursement decisions”, four full HTAs and two rapid assessments “used for disinvestment decisions and resulted in reduced volumes and expenditure” ([66], p. 77) Total of 67 out of 69 used (97%); two full HTAs no impact; other factors also played a role: in only 45% of reports “the recommendation and decisions totally consistent” ([66], p. 81) In Austria, policymaking structures facilitate the use of HTA reports, but no mandatory requirement to do so; it is possible the decisions could have been made based on international HTA institutions, but unlikely because, to be used, HTA reports “need primarily to be in German language and they have to be produced within a time period that is strongly linked to the decision-making process” ([66], p. 77)