Table 1 Characteristics of included tools

Atkins et al. (2011) [34]

Factors organised by the PICOS framework, which may limit the applicability of individual studies

1. Population

1. Eligibility criteria; differences in demographics, representativeness, exclusion rate of participants; unrepresentative event rates

“Large differences between demographics of study population and community patients”, p. 1201

2. Intervention

2. Unrepresentative of current practice; not feasible; comparison with current standard; co-interventions; unrepresentative providers or provider training

3. Comparator

3. Inadequate dose; substandard alternative treatment

4. Outcomes

4. Composite outcomes; short-term/surrogate outcomes

5. Setting

5. Different standards of care; unrepresentative population or level of care

Bonell et al. (2006) [43]

Framework for empirically assessing and reporting generalisability of randomised trials

1. Can the intervention be delivered elsewhere?

1. Feasibility; coverage; acceptability

“Secondly, an intervention must achieve adequate coverage. This may depend on the overall comprehensiveness of health systems or on whether providers can reach people in other ways—for example, through outreach. Adequate coverage may be more difficult in some sites or sub-populations”, p. 346

2. Does the intervention meet recipients’ needs?

2. Similar needs

Burford et al. (2013) [27]

Questions to guide the assessment of the applicability of the findings of a systematic review to a specific setting

1. Studies conducted in same setting/findings consistent across settings/times?

“Are there any political, social, or cultural factors that may affect the implementation of this intervention?”, p. 1259

2. Are there important differences in on-the-ground realities and constraints that might substantially alter the feasibility and acceptability of an option?

2. Political, social or cultural factors that may affect implementation; acceptability to general public; ethically acceptable; does target population have sufficient means to receive/implement intervention; can intervention be tailored?

3. Are there important differences in health system arrangements that may mean an option could not work in the same way?

3. Organisation responsible for intervention and organisational structure as a barrier to implementation; capacity to implement

4. Are there important differences in the baseline conditions that might yield different absolute effects even if the relative effectiveness was the same?

4. Baseline prevalence; population characteristics

5. What insights can be drawn about options, implementation, and monitoring and evaluations?

5. Sufficient resources; skills/training for providers

Cambon et al. (2013) [28]

Tool to analyse transferability and to support the development and adaptation of health promotion interventions (primary research) to new settings

1. Population

1. Epidemiologic and sociodemographic characteristics; cognitive, cultural, social and educational characteristics; motivation; accessibility; climate of trust between providers and recipients; recipient population’s belief in the intervention’s utility; recipient population’s demand for intervention; recipient population’s perceptions of their health needs; acceptability to recipient population; participation levels; degree of involvement

“The epidemiologic and sociodemographic characteristics of the recipient population are similar in the primary and replica interventions Subcriteria: Socioeconomic characteristics (rate of unemployment); demographic characteristics (age, sex); health status”, p. 12, Supplementary file 2

2. Environment

2. Supportive institutional environment; supportive other elements of context; partnerships

3. Implementation

3. Intervention methods; resources available; skills of providers and project leader; providers’ belief in intervention’s utility; acceptability to providers; mobilisation of providers

4. Support for transfer

4. Adaptations; transfer elements prepared and provided; knowledge transfer process

Green & Glasgow (2006) [44]

Evaluation questions related to the RE-AIM dimensions to aid the planning, conduct, evaluation and reporting of studies having the goal of translating research into practice

1. Reach (individual level)

1. Participation rate and representativeness

“Participation rate among intended audience and representativeness of these participants. Questions to ask: What percentage of the target population came into contact with or began program? Did program reach those most in need? Were participants representative of your practice setting?”, p. 133

2. Effectiveness (individual level)

2. Impact on key outcomes and quality of life; consistency of effects across subgroups

3. Adoption (setting and/or organisational level)

3. Participation rate and representativeness of settings

4. Implementation (setting and/or organisational level)

4. Level and consistency of delivery

5. Maintenance (individual and setting levels)

5. Long-term effectiveness (individual level); sustainability and adaptation (setting level)

Gruen et al. (2005) [35]

Questions to assess the generalisability of findings of systematic reviews

1. Relative importance of the health problem

1. Occurrence and severity of health problem

“Relative importance of the health problem: do the occurrence and severity of the health problem vary significantly between settings and how might this affect the intervention’s potential benefit to the population?”, p. 480

2. Relevance of outcome measures

2. Meaningfulness of outcome measures

3. Practicality of the intervention

3. Factors that may affect feasibility

4. Appropriateness of the intervention

4. Are other interventions more appropriate

5. Cost-effectiveness of the intervention

5. Costs and benefits

Khorsan & Crawford (2014) [29]

Tool to assess the external validity of randomised controlled trials and non-randomized studies in healthcare interventions

1. Recruitment

1. Identification and recruitment

“Recruitment: Did the study identify the source population for participants and describe how the participants were recruited from that source population?”, p. 8

2. Participation

2. Representativeness

3. Model validity

3. Representativeness of staff, places and facilities

Lavis et al. (2004) [45]

Approach to assess the local applicability of systematic reviews of health systems research

1. Could it work?

1. Structural elements of the health system

“Could it work? Are there important differences in the structural elements of health systems (or health system subsectors such as pharmaceuticals or home care) that mean an intervention could not work in the same way as in the countries where the research was done – e.g., institutional attributes such as the degree of integration in service delivery”, p. 1618

2. Will it work? (or what would it take to make it work?)

2. Perspectives and influence of health system stakeholders; other health system challenges; power dynamics and on-the-ground realities and constraints (and ability to change these)

3. Is it worth it?

3. Balance of benefits and harms

Lavis et al. (2009) [37]

Questions to guide the assessment of the applicability of a systematic review’s findings to a specific setting

1. Were the studies included in a systematic review conducted in the same setting or were the findings consistent across settings or time periods?

2. Are there important differences in on-the-ground realities and constraints that might substantially alter the feasibility and acceptability of an option?

3. Are there important differences in health system arrangements that may mean an option could not work in the same way?

4. Are there important differences in the baseline conditions that might yield different absolute effects even if the relative effectiveness was the same?

5. What insights can be drawn about options, implementation, and monitoring and evaluation?

“Are there important differences in on-the-ground realities and constraints that might substantially alter the feasibility and acceptability of an option?”, p. 4

Schoenwald & Hoagwood (2001) [46]

Dimensions and variables that can be used to compare conditions in research settings and practice settings

1. Intervention characteristics

1. Nature of treatment theory; focus of treatment; specification of treatment; similarity of new and prevailing treatment; complexity and clarity of intervention model

“Intervention characteristics: Nature of the treatment theory, including the relative weight of the theoretical, empirical, and clinical base; focus of the treatment: specific vs diffuse; specification of the treatment, including whether manuals are used and how comprehensive and prescriptive they are; similarity of the treatment to the prevailing practice for treating the identified problem or problems; complexity of the intervention model; clarity of the intervention model” p. 1194

2. Practitioner characteristics

2. Specialised training; adherence monitoring; clinical supervision, supervisor; type of practitioner; endorsement of intervention model; salary; anticipated job longevity

3. Client characteristics

3. Referral problem; family context; referral source; age, gender, ethnicity

4. Service delivery characteristics

4. Frequency, length and location of sessions; source of payment for service

5. Organisational characteristics

5. Organisational structure including hierarchy; personnel policies; organisational culture and climate; size; mission; mandate

6. Service system characteristics

6. Policies and practices of referral/payers; financing; legal mandates; interagency working relationships

Young & Borland (2011) [30]

Five dimensions to consider in order to generalise knowledge to practice from any given corpus of research

1. The nature of the problem or issue being intervened in

2. The characteristics of the population that is the target of the intervention

3. The context of the intervention

4. The nature of the intervention mechanism itself

5. Framework (the formal or informal set of beliefs that frame the intervention)

“The nature of the problem or issue being intervened in: The task here is to map those characteristics of the problem that are relevant to the choice of intervention. There are two dimensions to this; variation of the focal behaviour around whatever is normative and the focal behaviour’s relationship with conceptually related behaviours”, p. 263