Table 3 Included studies on needs assessment, monitoring and evaluation (NaME) of health research capacity development (HRCD) at the individual and organizational level

1

Ajuwon [34]

Nigeria (LMIC)

Physicians, dentists, nurses, laboratory scientists, and public health professionals of 29 governmental and two non-governmental organizations^d

To evaluate training on research ethics

Workshop

2. Multi-study approach: expert study AND Intervention study in pre-post-test design

Individual

Definition of needs: quality of ethics review, good ethical consideration, planning and implementation of ethics training

Outcome evaluation: knowledge and ethical reasoning

Focus group discussions and in-depth interviews for needs assessment (qualitative); 23-item-questionnaire for pre- and post-course evaluation (quantitative)

2

Ali [43]

13 African countries^e

Health professionals, ethics committee members, scholars, journalists and scientists (n = 28)

To evaluate the Johns Hopkins-Fogarty African Bioethics Training Programme (FABTP)

One-year non-degree training

1.2.1 Cross-sectional study

Individual

Outcome evaluation: grants, publications, participants’ teaching activities

FABTP evaluation framework: Individual development (qualitative); Programme evaluation (quantitative)

3

Barchi [44]

Botswana (UMIC)

University faculty members^e, community and governmental staff, research staff from non-governmental organisations, students (n = 71)

To evaluate training on research ethics

One-semester training programme

1.1.1.1 Intervention study in randomized controlled design

Individual

Outcome evaluation: knowledge and critical reasoning

Pre- and post-training delivery of Family Health International 40-item-test (quantitative); Self-constructed post-training case work with ethical challenges (quantitative)

4

Bates [21]

Ghana (LMIC)

Clinicians, physiotherapists and hospital managers^d

To develop an evidence-based tool to guide the design, implementation, and evaluation of health research capacity development programmes

Not described further

1.2.2 Theoretical study

Individual and organizational

Mapping of the developed evaluation tool to identify needs and gaps: role of partners, institutional research support services, diplomas, research scope, educational quality assurance, publications, grants, use and dissemination of research within and outside of the organization

Validation of proposed framework by mapping it with participants’ and institution’s experiences to derive needs (qualitative)

5

Bates [45]

Ghana (LMIC)

Health professionals: medicine, physiotherapy, pharmacy and health management (n = 15)

To evaluate the effectiveness of a 1-year part-time course in research skills

One-year part-time course

2. Multi-study approach: Intervention study in pre-post-test design AND Cross-sectional study

Individual

Outcome evaluation: process and content of course delivery, competencies and confidence

Analysis of students’ research proposals and projects (quantitative); Research Self-Efficacy Scale (quantitative); Analysis of learners’ reflective commentaries (grounded theory approach) (qualitative); Course evaluation (nominal group technique) (qualitative); Pre- and post-test delivery of “Stages Of Change” tool (quantitative);

6

Bates [29]

Ghana (LMIC), Kenya (LMIC), Malawi (LIC) and Democratic Republic of Congo (LIC)

Four case studies with health-related research projects from four different African countries

To develop indicators to monitor the building of sustainable health research capacities

Not described further

1.2.2 Theoretical study

Individual and organizational

Definition of needs: list of capacity gaps, list of critical and supporting stakeholders Outcome evaluation: publications and/or presentations at national/international meetings, expanded skills and workforce, reduction of input of northern partners, long-term funding

Researchers mapped their framework (Bates et al. [21]) with four case studies to derive generalizable indicators (qualitative)

7

Bullock [46]

United Kingdom (HIC)

Healthcare managers from 10 sites within the National Health Service (NHS)^e

To improve quality of health research by involving healthcare managers in research projects

12-months fulltime programme

1.2.3 Expert study

Individual

Outcome evaluation: motivation, arrangements, experiences, lessons learned and quality improvements of the research and programme

Adapted version of Kirkpatrick’s framework [47, 48] for guiding and coding of semi-structured face-to-face interviews (qualitative);

8

Cooke [49]

United Kingdom (HIC)

General practitioners, nurses, social workers, pharmacists^d

To find indicators to evaluate the “Designated Research Team” (DRT) approach to build health research capacity in primary and community care settings

Training, mentorship, supervision, partnership development, protected time for research

1.2.2 Theoretical study

Individual/team

Outcome evaluation: constructing and applying indicators

Mapping of Cooke’s framework (Cooke [8]) with a case to derive literature-based and expert-based indicators for evaluating the DRT (qualitative)

9

Corchon [50]

Spain (HIC)

Clinical nurses (n = 170)

To develop nursing research capacity in clinical settings

Mentoring, research courses and journal clubs

1.1.1.2 Intervention study in non-randomized controlled design

Individual

Outcome evaluation: research knowledge, skills, competencies, attitudes, facilitating factors and barriers

Pre- and post-training delivery of Nursing-research-questionnaire (control) (quantitative); Research-knowledge-objective-test (intervention) (quantitative); Facilitators and barriers scale (intervention) (quantitative)

10

Dodani [51]

Pakistan (LMIC)

Health professionals^e (n = 56)

To strengthen research capacities through a research skills training workshop in collaboration with the University of Pittsburgh

9-day research training workshop

1.1.2.1 Intervention study in pre-post-test design with 1 year follow-up

Individual

Outcome evaluation: knowledge

Self-constructed 20-item multiple choice questionnaire (quantitative)

11

Du Plessis [52]

Republic of South Africa (UMIC)

Nurses, other health-related researchers, and national and nternational stakeholders^d,e

To understand the stakeholders’ and nurses’ opinion of meaningful research

Study to prepare any HRCD activity

1.2.3 Expert study

Individual and organizational

Definition of requirements: description of meaningful research

Qualitative secondary analysis with re-exploration of existing data from a Delphi study and focus group discussions

12

Finch [53]

Australia (HIC)

Speech language pathologists (SLP) (n = 158)

To investigate the current research interest, confidence, and experience in the SLP healthcare workforce, and factors that predict research engagement

Study to prepare any HRCD activity

1.2.1 Cross-sectional study

Individual

Analysis of current state: research skills, research participation

Research spider tool and additional questions on research participation (quantitative)

13

Golenko [22]

Australia (HIC)

Allied health senior managers (n = 9)

To describe and analyse allied health senior managers’ perspectives of how organizational factors impact research capacity development

Study to prepare any HRCD activity

1.2.3 Expert study, part of Holden et al. [54]

Organizational

Definition of requirements: organizational factors and support for research-capacity building (RCB), barriers and motivators, research culture

Qualitative study with semi-structured interviews

14

Green [35]

United Kingdom (HIC)

Senior staff with teaching role (nurses and midwifes) (n = 34)

To examine the development of nursing and midwifery research capacity from the faculty perspective

Analysis of institutionalized CD activities

2. Multi-study approach: two expert studies AND Theoretical study

Individual and organizational

Outcome evaluation: research culture, management and organization, problems and challenges, wider context

A case study approach using three types of qualitative methods: Interview; Focus group discussions; Document analysis

15

Henderson-Smart [55]

Australia (HIC), Malaysia (UMIC), Philippines (LMIC), Thailand (UMIC)

Local researchers of four sites from South East Asia^d,e

To improve the health of mothers and babies in South East Asia by using and generating relevant evidence

Training and support for generating, using and dissemination of evidence

1.1.2.1 Intervention study in pre-post-test design

Individual and organizational

Outcome evaluation: adherence to recommended clinical practices and health outcomes, involvement in evidence-based practice, local barriers

Patient chart analysis if best evidence practice had been followed (qualitative); Survey and document analysis: Involvement in evidence based practice; research activities (mixed); Surveys and interviews: Local barriers to practice change (mixed)

16

Holden [56]

Australia (HIC)

Allied health professionals ^e (n = 134)

To develop and validate a questionnaire to evaluate the effectiveness of research culture building activities on individual, team and organizational level

Not described further

1.2.4 Validation study

Individual/team and organizational

Needs and outcome evaluation

The research capacity and culture tool (RCC) (quantitative)

17

Holden [54]

Australia (HIC)

Multidisciplinary primary healthcare teams^d,e (8 teams)

To evaluate the effectiveness of a DRT approach to build research capacities using RCC

Supporting teams to conduct small research projects with a multi-strategic approach

1.1.1.2 Intervention study in non-randomized matched-pairs design

Individual/team, and organizational

Outcome evaluation: individual, team and organizational domain

RCC (intervention and control) (quantitative); Qualitative data on contextual information (intervention and control); Qualitative data on team related aspects (intervention)

18

Hyder [32]

Pakistan (LMIC)

Local researchers^e (n = 54)

To evaluate the current state and impact of human resource development for health research at doctoral level

Training on health research skills

1.2.1 Cross-sectional study

Individual

Outcome evaluation: training programme characteristics, contributions through research, publications

Impact evaluation: teaching activities after returning to Pakistan

Self-constructed questionnaire (quantitative)

19

Hyder [57]

Sub-Saharan Africa

Selected trainees from Sub-Saharan Africa^e (n = 12)

To assess given outputs of “The Johns Hopkins-Fogarty African Bioethics Training Programme” (FABTP)

Courses on bioethics, research ethics and research methodology

1.2.1 Cross-sectional study

Individual

Outcome evaluation: enhanced knowledge, new skills, publications, research grants, number of students taught

FABTP evaluation framework: Informal progress notes and evaluation forms (mixed); Transcripts from trainees’ coursework (qualitative); Resumes (qualitative); Formal progress notes (qualitative)

20

Jamerson [30]

United States of America (HIC)

Undergraduate, masters and doctoral nursing students (n = 30)

To describe a training on nursing research capacities

Collaboration between nursing students and clinician researchers

Not mentioned

Individual

Outcome evaluation is unclear

Evaluation design, methods and tools are not described

21

Janssen [36]

New Zealand (HIC)

Physical therapists and clinical managers (n = 25)

To explore the experiences of physical therapists and clinical managers conducting research facilitated by Participatory-Action-Research (PAR) approach

Supporting physical therapists and clinical managers in initiating and conducting research by PAR approach

Multi-study approach: Intervention study in pre-post-test design and 1 year follow-up AND Theoretical study 1.2.3 Expert study

Individual and organizational

Outcome evaluation: experiences related to the initiated research process, motivation, research confidence and orientation

Semi-structured interviews at the end of the intervention and 1 year later (qualitative); Field notes (qualitative); Reflections of PAR groups (qualitative); Three questionnaires in pre-post-test design with 1 year follow-up (quantitative): Edmonton Research Orientation Survey, two visual analogue scales

22

Jones [58]

Australia (HIC)

General practitioners (n = 11)

To determine research training needs and barriers

Study to prepare any HRCD activity

1.2.3 Expert study

Individual and organizational

Analysis of current state: experiences with research, level of research skills, perceived barriers

Grounded theory approach: Semi-structured face-to-face or telephone interviews (qualitative)

23

Kwon [59]

United States of America (HIC)

Community-based organizations (CBO) and partners (n = 27)

To assess the resources and needs for research capacities of CBOs

Study to prepare any HRCD activity

1.2.1 Cross-sectional study

Organizational

Definition of needs: organizational characteristics, involvement in research, research related training, infrastructure

Face-to-face group discussions (qualitative); Online questionnaires (quantitative)

24

Lazzarini [60]

Australia (HIC)

Podiatrists (n = 70)

To report the research capacity of podiatrists

Study to prepare any HRCD activity

1.2.1 Cross-sectional study (part of a longitudinal observational study)

Individual/team and organizational

Analysis of current state: individual research skills, team and organizational aspects of research

Electronic survey (quantitative); RCC tool (quantitative)

25

Levine [24]

United States of America (HIC)

Principal investigators of two research programmes (n = 15)

To evaluate two healthcare research capacity development programmes and their sustainability

Two capacity development programmes on health research infrastructure

1.1.2.1 Intervention study in pre-post-test design with 6 years follow-up

Organizational

Analysis of current state: level of research activities

Outcome evaluation: research infrastructure strategies, project barriers and facilitators, process variables, success variables

Mixed-method approach guided by a self-constructed framework: Interviews (qualitative); Secondary sources like annual reports or grant applications, etc. (quantitative); Surveys (quantitative)

26

Mahamood [25]

Bangladesh (LMIC)

Managers, key researchers and external partners^d

To assess structural and organizational aspects of research capacity development activities

On-going research activities and capacity development strategies

1.2.1 Cross-sectional study

Organizational

Outcome evaluation: perceived problems and issues, structural and organizational performance indicators, financial indicators

Mixed-method approach to re-assess defined issues (guided by a self-constructed framework): Interviews (qualitative); Questionnaires (quantitative); Financial analysis (quantitative); Structural analysis of investigated institution (qualitative)

27

Mayhew [28]

Republic of South Africa (UMIC) and Thailand (UMIC)

Programme staff (n = 25) from two partners in South Africa and one in Thailand^e

To strengthen health economics-related research capacity through partnerships

North-southern partnerships in research, teaching and communication of new knowledge

Multi-study approach: Theoretical study AND Expert study

Individual/team, organizational and partnerships

Outcome evaluation: characteristics of participants, publications, projects initiated, effects from partnerships

Mixed-method approach guided by evaluation framework: In-depth interviews (qualitative); Document analysis (qualitative); Annual reports and other programme reports (quantitative)

28

McIntyre [61]

Australia (HIC)

Different health practitioners^e (n = 105)

To build research capacity and to increase the number of health practitioners with knowledge and skills in health research

Researcher development programme

1.2.1 Cross-sectional study

Individual

Outcome evaluation: knowledge, attitudes and practice in relation to research

Measuring the impact of the training by applying an online-questionnaire (quantitative)

29

Minja [62]

Various developing countries^e

Participants (n = 128) and institutions (n = 20) of three different capacity development grants^e

To identify factors that positively influenced and improved the research capacity and career development of grant recipients

30 years training in tropical disease

1.1.2.1 Intervention study: Pre-post-test design study

Individual and organizational

Outcome evaluation: indicators on individual career development, research skills and productivity, indicators on institutional infrastructure and development

Mixed-method approach: three standardized questionnaires for individuals (quantitative); In-depth interviews (qualitative); Questionnaires for institutions (quantitative)

30

Moore [63]

United Kingdom (HIC)

Nurses, midwives, and managing staff within NHS foundation trust (n = 16)

To develop infrastructure for research capacity development

Study to prepare any HRCD activity

1.2.3 Expert study

Organizational

Analysis of current state: barriers and facilitators of the research process

Observing researchers in their natural field by applying the “Action research strategy”: Semi-structured individual interviews (qualitative)

31

Njie-Carr [27]

Uganda (LIC)

Clinicians, community health workers, and administrative staff (n = 43)

To evaluate a research capacity development programme (preparing for the implementation and evaluation of a mobile phone based healthcare training on HIV/AIDS)

Training to conduct and evaluate a mobile-phone-based healthcare programme

1.1.2.1 Intervention study in pre-post-test design

Individual/team and organizational

Definition of needs: pre-training assessment

Outcome evaluation: structural and organizational aspects of trainings, research knowledge, skills and confidence

Cooke’s evaluation framework (Cooke [8]): three questionnaires were constructed and delivered at three time points (quantitative): Situational analysis: Pre-training assessment; Interim evaluation of RCB activities; Final or post-training evaluation of RCB activities

32

Otiniano [64]

United States of America (HIC)

Community health workers in Latino communities (n = 8)

To present case studies of eight health promoters who participated in a health policy research programme

3-days course on research terminology and methods and a workshop conducted by the course participants to train their colleagues

1.2.1 Intervention study in pre-post-test design

Individual

Analysis of current state: experiences with data and milestone tracking

Outcome evaluation: extent to which new skills were developed

Pre-training assessment: analysis of an application survey (quantitative); Milestone tracking for peer teaching workshops in health research (quantitative); Post-training assessment: qualitative phone interviews guided by the “Grounded Theory” method (qualitative)

33

Pager [65]

Australia (HIC)

Allied health professionals^e (n = 84)

To gain a better understanding of how motivators, enablers, and barriers impact research activities within allied health professions

Study to prepare any HRCD activity

1.2.1 Cross-sectional study

Individual/team, and organizational

Analysis of current state: research motivators, enablers and barriers

Written version of research capacity and culture (RCC) tool (quantitative); Tool is broadened to questions on motivators, enablers and barriers on individual and team level (quantitative)

34

Perry [66]

United Kingdom (HIC)

Participants (nurses, midwives, and allied health professionals) and managers (n = 98)

To evaluate the extent to which a research facilitator can provide and improve research skills

Programme on research development, knowledge and implementation

Multi-study approach: Intervention study in pre-post-test design AND Expert study

Individual

Outcome evaluation: processes and activities (participants) and impact of the training (managers)

Mixed-method approach guided by a self-constructed framework: Questionnaire on opinions und perceptions of participants: comparison with previously defined objectives (quantitative); Semi-structured interviews with managers (qualitative)

35

Priest [67]

United Kingdom (HIC)

Nurses, social scientists^d

To evaluate nursing lecturers’ research capacity by involving them as co-researchers in a research project (for details of this project cf. Green et al. [35, 68] and Segrott et al., [69])

Programme to integrate neophyte researchers in a research project with experienced researchers

1.2.1 Cross-sectional study

Individual

Outcome evaluation: reasons for becoming a member of the study, experiences in terms of benefits and problems

Questionnaire with open-ended questions (mixed); Comparison of these findings with the findings of the main study (Green et al. [35, 68], Segrott et al. [69]) (quantitative)

36

Redman-Maclaren [70]

Australia (HIC) and Solomon Islands (LMIC)

Solomon Islander and Australian researchers^e (n = 10)

To explore the benefits of a collaborative research capacity development strategy for both Australian and Solomon Islander researchers

Two-week workshop on research design, data collection and reporting with teaching strategies

1.2.3 Expert study

Individual and organizational

Outcome evaluation: benefits, barriers, experiences, future development

Grounded theory method was applied: four open ended questions either in a face-to-face interview or in written form (qualitative)

37

Ried [71]

Australia (HIC)

Primary healthcare professionals^e (n = 89)

To develop and assess research and evaluation skills among primary healthcare professionals

Study to prepare any HRCD activity

1.2.1 Cross-sectional study

Individual

Analysis of current state and definition of needs: current level of participation in research, level of experience in 10 specific research skills, publication and funding record, interest in training, etc.

Questionnaire with five topics; Visual research spider tool (part of the questionnaire) (quantitative)

38

Salway [72]

United Kingdom (HIC)

Public health staff (n = 10)

To evaluate and identify elements of learning of participants within a certain research capacity development programme

5-month research capacity development programme

1.2.1 Cross-sectional study

Individual

Outcome evaluation: participants perception of learning, experiences, programme content and programme structure

Post workshop evaluation forms (quantitative); Final evaluation with structured and open ended questions (mixed); Follow-up evaluation 12 months later with three open ended questions (mixed)

39

Suter [31]

Canada (HIC)

13 case reports

To describe the process used by the Community of Practice to initiate research capacity development

Study to prepare any HRCD activity

1.2.2 Theoretical study

Individual and organizational

Definition of requirements: research and evaluation skills, support of research and evaluation, building linkages, ensuring dissemination, building sustainability, creating appropriate infrastructure

Mapping recommendations of 13 case reports against Cooke’s framework (Cooke [8]) (qualitative)

40

Webster [73]

Australia (HIC)

Health professionals^e, managers and mentors (n = 25)

To gain better understanding of the impacts of research programme from the participants’, managers’, and mentors’ perspectives

2-years health research capacity development programme

1.2.3 Expert study

Organizational

Outcome evaluation: effectiveness of the partnership, leadership, workforce development, resource allocation and organizational change strategies

Semi-structured interviews (qualitative)

41

Wilson [74]

Sites outside the United States of America^e

Clinical research managers^e (n = 166)

To describe the development, implementation, and evaluation of a distance-based continuing education programme for study coordinators outside of the United States of America

2-years online programme on clinical research

1.1.2.1 Intervention study in pre-post-test design

Individual

Outcome evaluation: participants perceptions on the course and teaching strategies, level of knowledge, logs on participants capacity development activities

Modified standard course, teaching and overall programme evaluation forms from the University of Alabama (quantitative); 21-item investigator-developed online survey to assess students’ level of knowledge at pre and post course time 10-item survey for withdrawals were constructed (quantitative)

42

Wootton [75]

Two countries^e

Researchers^e (n = 82)

To generate a useful “research output score” out of three indicators to measure individual research output

Not described further

1.2.4 Validation study

Individual

Outcome evaluation: development and testing of the “research output score”

Definition of three indicators, which build the “research output score”: grant income, publication and number of PhD students supervised; Application of indicators/research output score in different research departments/countries (quantitative)