The potential of an online educational platform to contribute to achieving sustainable development goals: a mixed-methods evaluation of the Peoples-uni online platform

Peoples-uni: what it hopes to achieve

Peoples-uni was conceived as a response to the massive health problems facing developing country populations, the need to build capacity to study the causes of and solutions for these problems, and a lack of affordable and accessible educational opportunities that would help build this capacity. The mission of Peoples-uni is “[t]o contribute to improvements in the health of populations in low- to middle-income countries by building Public Health capacity via e-learning at very low cost” (http://www.peoples-uni.org/content/overall-objectives). Box 1 describes the objectives of Peoples-uni.

Brief history

Started in 2007 with registration as a charity in the United Kingdom, the first module was piloted in 2008 and followed by the offering of a few modules later that year. Modules were developed, and then delivered, by teams of volunteers with academic or service experience in public health, using a common structure and populated by Open Educational Resources. The number of modules grew over time, partly by serendipity reflecting the interests of volunteers, and partly by identifying the need to build capacity in the foundation sciences of public health and the problems facing developing country populations. The IT and educational administrative infrastructure also grew over time to meet all the requirements to enrol and support students and volunteer tutors, run modules, assess competence and report results as in a high-quality educational organisation. Students came mainly from Africa, but also from India and other parts of the developing world. Unlike some other capacity-building programmes, for example, the International Clinical Epidemiology Network, where capacity was built within institutions, students enrolled as individuals (although some organisations sponsored some of their staff).

To reflect Objective 3 of the requirement for the education to be at the master’s level, while being classed as a charity rather than a registered provider of higher education, Peoples-uni aimed to enter into partnerships with those who could provide a credible academic award to the students. Manchester Metropolitan University (MMU) in the United Kingdom performed this role for 128 students who enrolled over a 2-year period in a Master of Public Health degree, but following the termination of this partnership, it has proved difficult to identify another academic partner (note: during the time that MMU were validating the programme, some students graduated with an MPH from them, while others graduated from Peoples-uni having taken the same course with the same input and assessment processes. The evaluation we report here includes both groups of graduates).

The use of volunteers, Open Educational Resources and a fully online educational infrastructure allowed courses to be provided at low cost, affordable to most students, yet with a bursary scheme for those unable to pay fees.

How this differs from Massive Open Online Courses (MOOCs)

The rise of MOOCs occurred at roughly the same time as Peoples-uni was evolving. However, there are a number of differences in the educational approach, including the MOOCs’ heavy reliance on specially created content – mainly videos – rather than the Open Educational Resources used extensively by Peoples-uni. MOOCs do not usually make their resources available for later use by students, while Peoples-uni modules are published under a Creative Commons licence. MOOCs have much less insistence on contributions to discussion forums or tutor facilitation than Peoples-uni, and there are no assignments other than quizzes. Finally, although mini-credits are now being offered by some MOOCs, academic credit is not the general aim.

Capacity-building for public health: what Peoples-uni hopes to achieve in terms of building capacities

As expressed in Objective 6, Peoples-uni from the start wanted to support its graduates in the role for which they have been trained in the programme, that is, to teach, perform research, and implement evidence-based health policy and advocacy to improve the health of their populations. While the competences gained during the educational programme, and the academic award itself, are important outcomes, the final endpoints of the programme would be the way in which graduates can use their new skills to impact on population health. An active Alumni group has gone some way towards this [5], but Peoples-uni continues to seek mechanisms to fully realise the potential of the programme.

Brief literature review

In 2007, the date that Peoples-uni was established [3, 4], a paper outlined the extent of the need for education in public health in Africa, stating that “[o]ver half (55%) of countries do not have any postgraduate public health programme” [6]. There have been some evaluations of postgraduate public health programmes, including those aimed at low- and middle-income country (LMIC) populations, most of which have reported positive outcomes in terms of skills gained [7–11]. We are not aware of evaluations of online master’s level courses in public health aimed at an LMIC audience, although distance learning programmes in Africa have been described [12, 13]. Of relevance to the use of Open Educational Resources (OER), Angell et al. [14] reported on the limited literature on the availability of OER in public health, and subsequently developed a model for the release of OER in public health, which they hoped would “provide practical assistance and encouragement for the academic public health community to create and share OER” [15]. A thought-provoking paper [16] identifies OER and online learning as “One of the key concepts in the right to education is access: access to the means to fully develop as human beings as well as access to the means to gain skills, knowledge and credentials….to address the global education gap”. Addressing the public health global education gap nicely summarises Objective 1 of the Peoples-uni. A fundamental question that this evaluation explores is whether Peoples-uni can help address this gap.

Evaluation of Peoples-uni

The original theory of change for Peoples-uni is described in Fig. 1. Its aspirations of key outputs included collaborative, multi-country research, education using online courses and courses from partners, advocacy efforts, and impacts on evidence-based health policy. Key inputs included development of courses, international volunteer tutors, IT infrastructure, a growing base of alumni, and partnering with universities and other organisations. The tutors would not only teach but would also offer guidance for both research projects and dissertations. Peoples-uni was supported by a structure that included IT infrastructure, governance and a process of ongoing evaluation. The activities were mostly online to enhance collaboration across the course participants.

This theory of change has stood Peoples-uni well over the past decade, but much of the ideas that were generated above were by the planners, who were based in the United Kingdom, Australia and other Western settings. Therefore, there is a need to learn from the students who had successfully graduated from Peoples-uni. It is possible that the impact pathways by which Peoples-uni actually impacted individuals is different from what was originally envisaged. Further, the student participants in the evaluation provided suggestions on how Peoples-uni could be developed in the near future.

Data collection

The evaluation was conducted in two parts, an online survey and a telephone interview. In the first part, the students who had graduated with a master’s level award during the time that the course was accredited by MMU were sent an invitation to participate in an anonymous online survey (SurveyMonkey). The invitation was sent by the lead evaluator, deliberately external to the usual online communication system run by Peoples-uni. Questions were based on the objectives of the study, and reflected other evaluations in the literature. As part of the survey, respondents were asked to give their email address if they were prepared to take part in a subsequent interview survey. One reminder was sent to the group after 4 weeks.

From amongst the respondents agreeing to a possible interview, 18 were invited to participate in a semi-structured interview through phone or Skype. Selection was based on the desire to include a spread of participants on the basis of their graduation status (MMU or Peoples-uni) and country. The semi-structured format enabled the evaluation team to set the agenda with respect to the topics covered, and allowed interview responses to determine the kinds of information produced about those topics.

The interview guide was designed by the evaluation team, but reflected questions previously used in the literature [17]. The aim was to capture a rich understanding of the graduates’ interpretations of the Peoples-uni programme. The evaluation team reviewed rapport-building practices to ensure that the interviewer fostered a sense of safety and comfort for the interviewee. All of the interviews were completed over Skype or over the phone.

Each interview was audio-recorded and transcribed in Microsoft Word. Additionally, to ensure confidentiality for the respondents, any identifiers such as names or specific locations were removed from these transcripts before including them in this report and a participant number was assigned to each participant.

Analytical focus

The three broad evaluation questions described earlier guided the analysis. The summative focus of the analysis assessed progress through analysis of the structured survey against the goals of Peoples-uni. The qualitative interviews provided the data for understanding the impact pathways and formative feedback. As part of the qualitative analysis, two evaluators sought to identify key categories of impacts and feedback from the respondents. The analysis was informed by an understanding of the existing theory of change as well as recognition that the feedback helped to inform a refined theory of change.

Data were analysed using thematic content analysis, the most common method of qualitative analysis, which is both grounded in the literature and consistent with an interpretive theoretical paradigm. This study applied an inductive style of analysis to derive themes and explanations from a close reading of the data, rather than through an attempt to categorise the data into pre-existing concepts or ideas from theory. To explore graduates of the Peoples-uni programme’s interpretations and familiarise themselves with the data, two research team members re-read each transcript and its corresponding memo.

Ethical issues

As part of the application process for entry to Peoples-uni courses, students are informed that their anonymised information might be used for research into the outcomes of the education programme and are provided the opportunity to reject this. The invitation letter to the online survey made it clear that this was carried out by an independent organisation, and that participation was voluntary and anonymous. Those agreeing to be interviewed had to provide their names and email addresses.

For the interview phase, each participant was informed of the interview subject matter and was debriefed about the evaluation purposes afterwards. Prior to commencing their interview, each interviewer emphasised voluntary participation, and the right to confidentiality. Special care was taken to maintain respect and confidentiality of the participant and their information. Given these precautions, there were no anticipated harms from this evaluation.

Online survey

Interview survey

Were the objectives of Peoples-uni met?

It is too early to say if the first part of the mission of Peoples-uni, namely “[t]o contribute to improvements in the health of populations in low- to middle-income countries”, has been met, but the programme has clearly met the remainder “by building public health capacity via e-learning at very low cost” (http://www.peoples-uni.org/content/overall-objectives).

The programme has provided public health education, with the majority of graduates coming from Africa or the Indian sub-continent, and including a wide range of types of health professionals. Many said that they would not have been able to get an MPH without Peoples-uni. Further, it was encouraging to see that such a high proportion of the graduates were already passing their training on to others, and the ‘train the trainers’ objective was being met.

Although career development of the students was not specified as an objective of the programme, many of the questions in both parts of the evaluation cover this aspect. Career advancement, such as promotion, new positions or responsibilities, appreciation by the employer, as well as accessing the next educational step of a PhD, appear to have been welcome and frequent outcomes of the programme. Critical thinking was one of the interview themes that emerged, and is found as an underlying theme of much of the educational programme. It was also encouraging that such a high proportion of the respondents felt that the programme fits them to deal with local public health problems. The high proportion of respondents who had already published papers or started research is also a positive outcome of relevance to improving the health of their populations.

The existence of a growing cohort of well-trained public health professionals, who are already maintaining collaboration in education and research [5], is a resource to be nurtured and expanded. It is to be hoped that, in a next phase of the activities of Peoples-uni, a way to utilise this resource to further meet the objective of improving the health of populations will be found.

Building capacities for global health action: the salience of Peoples-uni at a time of SDGs

The evaluation also provided former students the opportunity to offer input to enhance future versions of Peoples-uni. Clearly, the leading challenge raised by most students was the need for accreditation. Getting accreditation from universities that are based in countries like the United Kingdom, Australia or New Zealand has proved to be a serious challenge. Consistent feedback from a number of students was to seek out accreditation from local universities. While we think that this is an attractive way forward, it also raises a number of challenging questions, such as (1) What are the incentives for local universities to collaborate with an online platform like Peoples-uni? (2) What are the mechanisms by which such partnerships can both help the quality of public health training in local universities without having negative economic impacts for the university? (3) What role should international organisations and funders like the UN or WHO or global foundations play in promoting such partnerships between an online platform and local universities? (4) Can action-focused partnerships be created between international organisations, online platforms and local Universities not only for accreditation but also to build capacities on the ground for action?

The tropEd Network for International Health in Higher Education is a good example of networked partnerships between universities, and has emphasised the importance of quality assurance in transnational higher education [20]. Other than partnering to provide an educational programme, forms of partnership to capitalise on the ‘products’ of educational programmes in collaborative research, evaluation and policy will require a different set of partners and models. Local, national and international health service providers now become possible partners. As described below, these will have the potential to help meet the SDG targets. It is important to remember that benefits for each member of a partnership is an essential component for success and sustainability (of note, since the paper was submitted for publication, Peoples-uni has established a partnership with Euclid University (www.euclid.int), which will offer an MPH to Peoples-uni graduates).

We believe that answering the above questions will be critical, not just for Peoples-uni but also to move towards more thoughtful models of capacity-building that pay attention to the inequities in access to education that exist between different countries. As we work towards achieving the SDGs, what role should accessible education through a platform such as Peoples-uni play in building capacities in settings where existing training in public health is weak? Could an online platform like Peoples-uni have a role to play in disrupting existing inequities in educational systems?

The feedback from multiple students clearly generated data that provided evidence that an online platform can help enhance individual-level access. Yet, Peoples-uni or online platforms are unlikely to be disruptors of existing orders without active partnership either at the local or national levels or partnerships with organisations like the UN, WHO or international foundations. One of the great strengths of Peoples-uni is that established networks of alumni are already working in settings that may have an important role in reducing inequalities in health and education.

Towards global health action

Study limitations

Although the evaluation was conducted externally in the hope that this would reduce any tendency to want to post only positive responses, we are not able to discount the possibility of response bias. Non-respondents might well have been less positive about the stated value of the programme. The response to the questionnaire part of the study was greater among those who said had gained their award from Peoples-uni than those whose award came from MMU, and this may have been because they were keen to reflect their views of the importance of the need for future university partnerships to improve the credibility of the award – a key and consistent finding of the study. In addition, we did not validate or triangulate the answers from other sources, such as by interviewing employers. Further, those interviewed were not representative of all graduates in terms of gender due to the selection criteria being restricted to geography and the body giving the master’s award. This was not an intended consequence, and may have biased the interview responses in an unknown direction.

The first of our study questions of whether Peoples-uni met its objectives, which include improvements to the health of populations, can only partly be answered by a survey of graduate experience and opinions. It is possible that the positive career developments described by the graduates might have occurred without their enrolment in Peoples-uni. It was also probably too early in the life history of the graduates to explore the wider impacts on society or the workplace. As part of the dissertation assessment, students are asked to reflect on the impact that the course has had on their work, and there are many anecdotal reports of such impact.

Finally, by focusing on the graduates of the programme, we appreciate that these represent the success stories. We are not able to capture the reasons why these people actually managed to enrol and then succeed while others who might have benefited missed out. A future evaluation might tackle this issue and help identify ways to open up education further via the Peoples-uni approach, especially for women.