Published evidence indicates that since a key aim of any capacity strengthening programme is to promote change, such programmes should be based on an explicit theory of change [12]. For programmes concerned with strengthening research capacity, the first stage in the change pathway involves ensuring there is demand for the programme and identifying the potential stakeholders. This is followed by defining the programme’s purpose and making explicit the links between activities, outputs, and outcomes [23, 24]. The theory of change also requires clarity about how the context and any underlying assumptions may influence whether the goal is achieved. Using the principles of the theory of change combined with published evidence about successful strategies for capacity strengthening [25–27], we developed a five-step approach for designing and conducting research capacity strengthening programmes. We tested our five-step approach in capacity strengthening programmes implemented across multiple countries in sub-Saharan Africa. In this paper, we use three of these programmes as case studies to illustrate how our approach worked in practice.
Define the goal of the capacity strengthening programme
The first stage in our research capacity strengthening process was to ensure that the intervention addressed local priorities and had the potential to be viable, affordable, and sustainable. Once the intervention was broadly agreed upon, we defined the goal of the capacity strengthening intervention, as well as an overarching framework that linked the goal to activities, outputs, and outcomes. This was done in consultation with the direct beneficiaries since these were usually the major drivers of the capacity strengthening programme. These beneficiaries articulated and agreed upon a clear goal for the programme [26] and determined how much the operational focus for capacity strengthening should be extended beyond the institutions, to incorporate individuals, and the broader health system. Some programmes, operated at more than one level. During this process we encouraged beneficiaries to carefully consider the social, political, and economic context in which they worked, and other assumptions that might influence the success of their capacity strengthening plans. A wider group of stakeholders – people who had an interest in the programme but were not directly involved – were then engaged to help refine the goal and to frame more specific objectives [28]. At this point, we verified with the programme funders that the refined goal also met their requirements [27]. This was important as a lack of agreement about the goal will make it harder for capacity strengthening to be achieved, as each cadre of stakeholders might retain different expectations, thereby potentially creating an ineffective programme and acrimonious relationships.
Case study 1 provides an example of how this step worked in practice. The primary beneficiaries were laboratory managers and heads of departments and they proposed that their laboratory should become a regional centre of excellence for lymphatic filariasis. A key driver for this goal was the World Health Organisation targets for global control of neglected tropical diseases. The operational focus for capacity strengthening was the laboratory and its host research institution. Discussions with a wider group of stakeholders such as Ministry of Health programme managers and laboratory quality assessors, facilitated agreement concerning the goal, and the objectives and activities needed to achieve the goal. These included activities to enhance individuals’ skills (e.g., a course for trainers of molecular techniques) as well as processes at the organisational level (e.g., obtaining international accreditation).
Describe the required capacity needed to achieve the goal
Our second step involved collating evidence about the optimal capacity needed to achieve the goal, such as published peer-reviewed research and grey literature including reports, guidelines, and recommendations. This detailed collation and synthesis of relevant evidence to underpin and provide rigour to the subsequent ‘needs assessment’ step is the most innovative part of our process. The evidence we sought was specific to the particular programme and organisational context. Although organisations were generally the focus of our capacity strengthening activities, we included evidence (published and grey literature, and expert opinion) that encompassed individuals, organisations, and systems because of the critical interdependencies across the three levels. For each programme, collation and synthesis of the evidence produced a long and cumbersome list which needed refinement by grouping related capacities to create a smaller set of optimal capacities. It was important not to discard any of the capacities on the list during this refinement process because this ‘optimal’ list constituted a holistic set of capacities which would act as a benchmark against which to compare existing capacity and to identify any gaps (i.e., it was the compass for the needs assessment). Capacity strengthening needs assessments that are not based on an evidence-informed and holistic benchmark may overlook important capacity gaps. Since many components of a capacity strengthening plan are interdependent, failing to address a critical gap may result in non-achievement and non-sustainability of the whole programme.
Experience has taught us that a broad and systematic trawl for relevant evidence and current good practices is essential for developing an optimal set of capacities tailored to each capacity strengthening programme. For example, in case study 2, an extensive literature search failed to identify any single instrument that could be used to evaluate all the policies and processes required by universities to run successful doctoral programmes. Therefore, we pooled relevant information from sources such as the Code of Practice from the UK Quality Assurance Agency [29], institutional standards for the contents of doctoral programmes, quality assurance guidelines for educational courses, recommendations concerning research skills to be acquired by doctoral students [14, 30–32], a framework for managing institutional quality systems [33], and Personal Development Planning approaches for African doctoral students [unpublished data]. In our case study 2, checking the final, refined list of required capacities against the original list extracted from multiple sources ensured that we had not discarded any important ones during the process.
Determine the existing capacity and identify any gaps compared to the required capacity
In the third step, we used the list of optimal capacities to guide data collection on capacity gaps and needs. Published evidence indicates that, because of the diversity, uniqueness, and complexity of each setting, a mix of different tools should be used when conducting needs assessments for capacity strengthening programmes [26]. We therefore developed specific data collection tools tailored to the context and project goal. As in case study 2, these tools often included an interview guide to engage stakeholders in discussion about existing capacity, capacity gaps, and challenges to strengthening capacity; a checklist based on optimal capacity, for use with programme beneficiaries to assess existing capacity against specific criteria; a list of documents to be reviewed; and an observation guide for visits to facilities. To help us analyse the data collected using these different tools, we developed a matrix to collate all the data into one table. Together, the tools were used to document existing capacity and to highlight any gaps in capacity. As no single individual, method, or document was likely to be able to provide complete and accurate information about capacity gaps, we triangulated the data collected across at least two sources and resolved any discrepancies in consultation with stakeholders. The latter process engaged a wide range of stakeholders [34, 35] and was invaluable for understanding some of the reasons behind capacity gaps and for devising recommendations that we could act upon prospectively. This prospective approach contrasts with the retrospective way that resource, governance, and management gaps are usually identified [4]. Joint problem-solving with stakeholders was also important for identifying strengths, prioritizing critical capacity gaps, and transforming what evidence indicated was optimal into what was feasible and practical in the context of each programme. The gaps, reasons for the gaps, discrepancies, and resolutions, and potentially sustainable solutions for filling capacity gaps were all mapped onto a matrix and used to underpin an action plan. Examples of critical priorities, that we identified through this process in the case studies, were the appointment of a quality systems manager in case study 1 and training and mentoring for PhD supervisors in case study 2.
Devise and implement an action plan to fill capacity gaps
The information gathered through a needs assessment has to be transformed into knowledge which is useful for decision making through reflection and sharing with others [26]. Therefore, in the fourth step, we worked with the beneficiaries of each programme to turn the list of priority capacity gaps into an action plan. The action plan was unique to each programme and had a goal, objectives, activities, and qualitative and quantitative indicators of progress. The plan named individuals responsible for actions. Based on the programme-specific theory of change, we were able to anticipate what indicators might be suitable for monitoring progress. However, it was important that each plan was regarded as flexible and able to be revised, as necessary, because we could not be sure at the outset which activities of the action plan would meet the objectives.
In many cases, the action plans contained activities with no, or minimal, cost implications, such as setting up new committees or re-allocating tasks among existing staff. Inevitably, some activities to strengthen research capacity required resources to cover, for example, the cost of training, or specific equipment. In these cases, the beneficiaries drafted budgets to negotiate with their funders and institutions to cover these costs. Due to the explicit buy-in of stakeholders and the evidence-informed approach, the plans and budgets had sufficient legitimacy to be used as advocacy tools for mobilising support. For example, the systematic identification of poor internet access as a priority capacity gap for African universities (case study 2) enabled university authorities to persuade a group of international funders to jointly contribute to upgrading internet access for postgraduate students.
Learn through doing; adapt the plan and indicators regularly
The process of strengthening capacity must be iterative and flexible [12]. This allows the plan, activities, and indicators to be revised as processes, relationships, structures, and agendas change. In capacity strengthening programmes, organisations typically move along a change pathway through learning cycles of action, reflecting on the changes, as they happen, and the indicators documenting them, and identifying new priorities. This process ensures that progress can be monitored, that decision-making is auditable, and that capacity is strengthened (See the five-step pathway below). We used learning cycles of action and reflection to inform revisions to the plans and indicators (Figure 1). As our capacity strengthening programmes developed, they passed through four overlapping phases – awareness, learning-by-doing, expansion, and consolidation [15]. With each phase, the programmes became more mature, some capacity gaps were filled, and new priorities emerged. We regularly re-visited the monitoring indicators because, although helpful up to that phase, indicators had to become more sophisticated as the programme matured, reflecting the increasing complexity of the programme [15]. For example, in case study 3, early indicators included course attrition rates and examination marks, whereas later indicators included evidence that research had influenced clinical practice and institutional systems [16, 17].
Five-step pathway for designing health research capacity strengthening programmes
-
1.
Define the goal of the capacity strengthening project
This necessitates harmonising the expectations and objectives of the most critical stakeholders including developing country partners, people involved in ensuring sustainability of the activities in the long term and the funding organisation.
-
2.
Describe the required capacity needed to achieve the goal. This will require a search for the best evidence to describe the ‘optimal’ capacity collated from, for example, peer-reviewed published papers or expert groups, including evidence from outside the health sector.
-
3.
Determine the existing capacity and identify any gaps compared to the required capacity. The evidence from step two is formatted into a set of qualitative and quantitative data collection tools to identify existing capacity and capacity gaps. Data is collected from stakeholders with different perspectives; discrepancies are highlighted and resolved through further discussion.
-
4.
Devise and implement an action plan to fill the gaps
The prioritised list of capacity gaps is transformed into an action plan which includes objectives, activities, deliverables and monitoring indicators, and measures to facilitate sustainability.
-
5.
Learn through doing; adapt the plan and indicators regularly. Results from experimentation and learning, and regular discussions with those responsible for monitoring progress are used to refine the plan. Progress indicators become more sophisticated as the programme matures and capacity is strengthened.
Planning for sustainability
The definition of sustainable capacity varies from programme to programme and needs to be agreed upon by the primary beneficiaries at the outset. Sustainability may mean, for example, that the programme activities are incorporated into the structures of the original organization, that they are integrated into another institution, or that the programme itself becomes an autonomous agency such as an independent charitable organisation. In a recent analysis of sustainable programmes in Africa, we demonstrated that, typically, sustainability meant that a programme had achieved financial independence and local autonomy in decision-making (see Enablers and challenges for sustainably strengthening health research capacity below). Therefore, in our capacity strengthening programmes, strategies for promoting the sustainability of ongoing activities, and their accompanying indicators, were woven into the objectives, action plan, and monitoring approach from the outset. For example, in case study 1, external accreditation of the laboratory was deemed essential for it to become a regional reference centre underpinned by a viable business strategy, so all the activities needed to achieve accreditation were reflected in the laboratory’s capacity strengthening plan from the outset. Incentives to retain staff, through further education or salary adjustments has been another key strategy which health research organizations have employed to promote sustainability [36]. In case study 3, the contentious decision to charge course fees, to students pursuing a professional course in research skills, eventually contributed to the financial security of the course and concomitantly motivated students to complete the course.
Enablers and challenges for sustainably strengthening health research capacity [15]
Enablers associated with sustainability
● Early engagement of stakeholders and explicit plans for sustaining efforts
● Ongoing learning and quality improvement cycles
● Investment in core resources (people, funds, committees, systems)
● Institutionalisation of new capacity
● Evidence of problem solving, decision-making and innovation
Challenges to achieving sustainability
● Turnover of staff and stakeholders
● Embedding new activities into existing systems
● Securing funding
● Influencing policy and programs