Table 1 Characteristics of international funders’ evaluations of LMIC health research capacity strengthening (RCS)
Evaluations by international funder | Health RCS characteristics | Relation of evaluator to funder | Evaluation characteristics | ||
|---|---|---|---|---|---|
| Â | Project, programme(s), organisation | Period (duration) covered by the evaluation | Â | Timing | Main approaches/methods |
RCS programme – with streams, health sector | 1960+ (48 years) | Funder staff | Periodic review | Analysis of existing award data, alumni evaluation survey, 15 case studies, and 5 telephone interviews of selected scholarship recipients; impact assessment. | |
Carnegie[25] | RCS initiative with networks | 2008–2010 (2 years) | Contract evaluation organisation | Mid-term | Desk review or initiative and network documents, interviews and focus groups with stakeholders (key staff and students within each network). |
Danida[26] | Health research programmes of which health RCS is a part | 1997–2006 (10 years) | Contracted evaluation organisation | Periodic review | Components were: a) country reports with visits; b) desk study review of projects; c) institutional questionnaires for Danish research groups; d) ‘internal’ [Danish organisations] individual staff questionnaires; e) ‘external’ [non-Danish other HIC funder] questionnaires and interviews; f) literature review of publications supported; g) evaluation document analysis; and h) health-related project database analysis. |
DfID[27] | Project Health research council | 2008–2010 (2 years) | External programme evaluation team | Mid-term review | Desk review of organisational, programme, and project documentation; site visit with interviews of stakeholders, beneficiaries, non‒beneficiaries, funders, and secretariat; in‒depth case studies of selected grantees and their institutions; and evaluation of the grants selection process. |
EDCTP[28] | Health research partnership | 2007–2009 (2 years) | Independent external panel | Periodic review | Documentation analysis, meetings/discussions and interviews with organisational representatives, questionnaire survey of researchers, site visit, conference attendance and country case study. |
Health research programme with projects | Roughly 2001–2008 (7 years) | Contracted evaluation team | Special review | Conducted a gender audit at three levels – institutional, programmatic, and project (review of 15 projects) – through documentation review; search of guidelines and strategies of other organisations working on policy, health and gender issues; review of a previous internal gender survey; gender questionnaire to assess capacity development needs; and individual interviews with funder staff. | |
NIH-FIC (1) [33] | Health RCS programme | 1992–2003 (11 years) | Contract evaluators | Periodic review | Outcome evaluation using NIH-FIC evaluation framework and FIRCA logic model. Administrative data collection and review, interviews with programme stakeholders, census surveys of the US principal investigators and international research collaborators, bibliometric analysis of publications, and site visits. |
NIH-FIC (2) [34] | Health RCS programme | 2002–2008 (6 years) | Contract evaluation team | Mid-term review | Programme implementation and preliminary outcomes. Data collection methods included two online surveys (GRIP awardees, unsuccessful applicants with scored applications). Supplementary data from administrative sources and databases, MEDLINE, and from interviews with US-based mentors, FIC staff members, and programme partners. |
Health RCS & health research programmes | 2005–2008 (4 years) | (2008) Committee of three experts & two secretariat members (2009) Contract evaluators | Mid-term review | (2008) Background document review, discussions with programme coordinators, site visits with interviews, formulate recommendations, and discuss with Programme Committee. (2009) Not specified but included: programme document review, programme logic construction, projects’ progress reports analysis, and stakeholder interviews. | |
Sida[37] | Linked health RCS project funding (three routes) | 1999–2005 (6 years) | Contract evaluators | Mid-term for re-formulation | Emailed questionnaires to institutions, individuals, and graduates. Interviews during site visits and evaluation seminar at main site. |
Organisation’s entire set of health RCS programmes | 2000–2008 (9 years) | Contracted institute evaluation team | Periodic review | Questionnaires (individuals, research groups, and institutions), selected in-depth interviews, institutional site visits with stakeholder semi-structured interviews. | |
Wellcome trust[40] | Health RCS project – Consortium | 2009–2011 (2 years) | Contract evaluation organisation | Mid-term (Second annual) | Real-time, monitoring and evaluation with mutually agreed framework of qualitative and quantitative indicators. Analysis in the light of all consortia within the programme of which this project is a part. |