- Open Access
A systematic review on academic research productivity of postgraduate students in low- and middle-income countries
Health Research Policy and Systems volume 16, Article number: 86 (2018)
While several individual studies addressing research productivity of post-graduate students are available, a synthesis of effective strategies to increase productivity and the determinants of productivity in low-income countries has not been undertaken. Further, whether or not this research from post-graduate students’ projects was applied in evidence-informed decision-making was unknown. Therefore, we conducted a systematic review of literature to identify and assess the effectiveness of approaches that increase productivity (proportion published) or the application (proportion cited) of post-graduate students’ research, as well as to assess the determinants of post-graduate students’ research productivity and use.
We conducted a systematic review as per our a priori published protocol, also registered in PROSPERO (CRD42016042819). We searched for published articles in PubMed/MEDLINE and the ERIC databases through to July 2017. We performed duplicate assessments for included primary studies and resolved discrepancies by consensus. Thereafter, we completed a structured narrative synthesis and, for a subset of studies, we performed a meta-analysis of the findings using both fixed and random effects approaches. We aligned our results to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.
We found 5080 articles in the PubMed (n = 3848) and ERIC (n = 1232) databases. After excluding duplicates (n = 33), we screened 5047 articles, of which 5012 were excluded. We then retrieved 44 full texts and synthesised 14, of which 4 had a high risk of bias. We did not find any studies assessing effectiveness of strategies for increasing publication nor citations of post-graduate research projects. We found an average publication proportion of 7% (95% CI 7–8%, Higgins I-squared 0.0% and Cochran’s Q p < 0.01) and 23% (95% CI 17–29%, Higgins I-squared of 98.4% and Cochran’s Q, p < 0.01) using fixed effects and random effects models, respectively. Two studies reported on the citation of post-graduate students’ studies, at 17% (95% CI 15–19%) in Uganda and a median citation of 1 study in Turkey (IQR 0.6–2.3). Only one included study reported on the determinants of productivity or use of post-graduate students’ research, suggesting that younger students were more likely to publish and cohort studies were more likely to be published.
We report on the low productivity of post-graduate students’ research in low- and middle-income countries, including the citation of post-graduate students’ research in evidence-informed health policy in low- and middle-income countries. Secondly, we did not find a single study that assessed strategies to increase productivity and use of post-graduate students’ research in evidence-informed health policy, a subject for future research.
Writing a thesis is a fundamental step for post-graduate studies globally . This process inculcates knowledge and skills for scientific enquiry, critical thinking, systematic problem-solving, and appraisal of scientific and lay claims. This if often followed by dissemination of the thesis results to the scientific community, of which publication in a peer-reviewed journal is the highest and most respected form . However, how much of this thesis work appears only in theses, and its citation particularly in public policy-related work, in low- and middle-income countries is the subject of our systematic review.
Our first aim was to conduct a systematic review of literature that identifies and assesses the effectiveness of approaches that increase productivity (proportion published) or the application (proportion cited) of post-graduate students’ research. Our second aim was to assess the determinants of post-graduate students’ research productivity and use.
We registered our protocol a priori in PROSPERO (CRD42016042819) and thereafter published it in a peer-reviewed journal . We thus present an overview of the methodological approach and highlight differences between the protocol and the actual conduct of this systematic review.
We report the electronic search for the PubMed/Medline database only (https://www.ncbi.nlm.nih.gov/pubmed/). We combined terms using Boolean logic ‘OR’ for synonyms and ‘AND’ across elements of PICOS (Population Intervention Comparison Outcome and Study design), as follows: ‘medicine’, ‘nursing’, ‘dentistry’, ‘pharmacy’ and ‘public health’ described the professional fields of interest; while ‘degree’, ‘doctor’, ‘post-doctor’, ‘master’, ‘fellow’, ‘resident’, ‘student’, ‘trainee’, ‘graduate’ and ‘post-graduate’ were intermediate transitional terms for the population of interest, setting or interventions. The terms describing the interventions of interest included were ‘mentor’, ‘grant’, ‘fund’, ‘supervise’, ‘workshop’, ‘seminar’, ‘conference’, ‘manuscript writing’, ‘scientific writing’, ‘academic writing’, ‘scholarly writing’, ‘grants writing’, ‘capacity-building’ and ‘research’.
We defined ‘productivity’ as the proportion of dissertations from which at least one manuscript was published and ‘use’ as the proportion of dissertations cited in peer-reviewed articles, technical reports or policy-related documents. Thus, the search terms for the outcome of interest were ‘abstract’, ‘thesis’, ‘dissertation’, ‘publication’, ‘poster session’, ‘poster presentation’, ‘book chapter’, ‘technical report’, ‘policy brief’, ‘policy dialog’, ‘evidence-informed policy’, ‘evidence-based policy’, ‘evidence-informed health policy’, ‘evidence-based health policy’, ‘decision-making’, ‘policy-making’ and ‘dissemination’.
We restricted these search terms to the title or abstract, and included terms for the outcomes to maximise relevance and efficiency. Further, in order to minimise the risk of an empty review, we did not enter specific terms for the study design as we intended to use all evidence types to describe the available range of interventions. See the full search string in Additional file 1. We found articles in French, Persian and Spanish, and used Google translator (https://translate.google.com/) for English translations during screening and full-text review.
In our targeted search, we screened the reference lists of included publications and retrieved full texts of articles likely to be eligible for inclusion. We contacted authors of included articles for any literature that they may be aware of.
Selection of studies
Using EndNote software version X7 (Thomson Reuters, 2015) we imported all identified titles, excluded duplicates, and screened and grouped these into relevant eligibility categories as described in our Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow chart .
Minimising bias in study identification and selection
A second reviewer (RS), an Information Scientist, validated the electronic search in PubMed by performing an independent and duplicate search. Similarly, a third reviewer (MO), screened all full texts excluded by the first reviewer (EAO). We resolved differences by discussion and consensus.
Criteria for considering inclusion of studies
We included published studies reporting at least one outcome of interest, and reporting on post-graduate research conducted in a low- and middle-income country.
Exclusion criteria for ineligible studies
Our exclusion criteria were studies about research conducted by Bachelor’s degree or undergraduate students’ or established university faculty not identified as post-graduate students, studies conducted in high-income settings or in low-income settings but by students from high-income settings, qualitative designs, non-empirical studies, syntheses, editorials or perspectives, studies published earlier than 1990, or those that were totally irrelevant.
We adapted a data extraction tool for observational studies we had developed for a previous systematic review . We then abstracted administrative, study design and primary outcome data on productivity measured as proportion of publications. We further abstracted our secondary outcomes of use of the research as measured by citations, time to publication, conference abstracts and additional outcomes describing the nature of the post-graduate students’ research (predominant types of research, first authorship status of student).
Handling of missing data
We denoted variables that were missing as not reported. We did not employ any statistical methods for handling missing data, neither did we contact authors for additional information.
Risk of bias of assessment of included studies
We adapted a tool we used in a published systematic review to assess for the risk of bias in the included studies . We considered the following seven aspects of bias [6, 7]: selection bias due to sampling or proportion of responders or baseline characteristics (and confounding), detection bias due to reliability of measurements used, and bias due to method of data analysis used for overall outcome and reporting biases. We categorised risk of bias as high, moderate or low, guided by the descriptive assessment questions in our tool (Additional file 2).
Synthesis of included studies
We employed a structured synthesis in which the units of analysis were findings from a single primary study. First, we described the characteristics of the included primary studies. Using the command ‘metaprop’ in Stata version 14.1 (Stata, College Station, Texas, USA), we constructed forest plots for proportions of published post-graduate work for both fixed and random effects models. The kind of data we abstracted would not permit assessment of the measures of effect, comparing two groups. We thus evaluated single group prevalence of publications, conference abstracts and predominant types of studies of the post-graduate work. We visually explored heterogeneity by inspecting the forest plots and statistically quantifying this using the I-squared statistic and tested for significance using Cochran’s Q. As we found a high level of heterogeneity we conducted a meta-regression testing the variables of duration of study, period of study and geographical region, before conducting a sensitivity analysis by excluding the very large study contributing 93% of the overall combined information. Finally, we wrote a narrative synthesis for the results for which we were unable to perform quantitative meta-analysis.
Differences between the published protocol and the actual study
In conducting this review, we employed practical approaches to circumvent unanticipated methodological challenges. We did not contact heads of academic or research departments in target universities as key informants, nor did we search additional grey literature-specific databases due to resource and time constraints of completing this doctoral project. Secondly, we did not find a single article reporting on the effects of strategies to increase productivity or increase the use of post-graduate students research and therefore synthesis was based on other relevant outcomes to map the field for future studies. Third, we did not assess the overall quality of evidence, as we did not find effectiveness studies or studies with comparison groups testing interventions for increasing productivity or use of post-graduate students research .
Systematic review flow, screening and inclusion
Our results are illustrated in the PRISMA flow chart (Fig. 1) and in Table 1, while Table 2 shows a summary of the risk of bias assessments. We retrieved 5080 titles and abstracts from two databases, PubMed (n = 3848) and ERIC (n = 1232). We updated the search in PubMed only and findings are as recent as July 17, 2017, as it provides the bulk of health-related literature. After excluding 33 duplicates, we screened all 5047 titles or abstracts, excluding 5012 mainly due to irrelevance (n = 4659) or various reasons not meeting eligibility criteria (n = 353). We retrieved a total of 44 full text articles, of which 9 were from additional targeted searching; we finally reviewed 14 of them, with only 12 in the statistical meta-analysis. The main reason for excluding the full texts was not containing at least one outcome of interest and the two studies were dropped in the meta-analysis either because they lacked the primary outcome  or the primary outcome was not reported in a way to permit synthesis .
Description of included studies
The studies we included were from Asia (n = 7), Africa (n = 5) and South America (n = 2), published between 2007 and 2014, including post-graduate students over a 30-year span (1974 to 2014). The smallest study had a sample size of 90, while the largest had 22,625. All studies were about post-graduate students pursuing Masters’ degrees, of which 4 included Doctoral students as well. Importantly, although many studies included cohorts of post-graduate students, all used a cross-sectional analysis. We included 1 study that did not report the primary outcome but described other secondary outcomes, and 7 studies that reported at least an additional outcome of interest.
Findings on the outcomes of interest
Publication proportion of post-graduate students’ research
We included 12 of the 14 review studies in the meta-analysis for the primary outcome, all together contributing work of 27,477 post-graduate students for the meta-analysis (Fig. 2). The proportion of post-graduate students research published ranged from 6% in Turkey  to 41% in Togo . We found an average publication proportion of 7% (95% CI 7–8%; Higgins I-squared 0.0% and Cochran’s Q, p < 0.01) and 23% (95% CI 17–29%; Higgins I-squared of 98% and Cochran’s Q, p < 0.01) using fixed and random effects models, respectively.
Meta-regression and sensitivity analyses
After excluding the study by Orzgen et al.  as it accounted for 92% of the review sample size, the results were 20% (95% CI 19–21%; Higgins I-squared and Cochran’s Q, p < 0.01) by fixed effects and 24% (95% CI 20–29%; Higgins I-squared of 92% and Cochran’s Q, p < 0.01) by random effects.
Our meta-regression results were not significant for duration of study span, period of the research or geographical region (data not shown).
Citation of post-graduate students’ research
Two studies reported on the citation of studies, which was our surrogate for use of post-graduate students’ research. In Uganda , this was 17% overall (95% CI 15–19%) and 4% specific for policy-related documents, while in Turkey , the median citation was 1 study (IQR 0.6–2.3).
Determinants of post-graduate students’ research
Only one included study reported on the determinants of productivity or use of post-graduate students’ research, and it suggested that younger students were more likely to publish and that cohort studies were more likely to be published .
Six studies reported the time to publication from completion of theses by post-graduate students. The earliest average time to publication was 2 years (Cameroon, Egypt) [14, 15], while the rest were 2.3, 2.8 and 3 years in Uganda , India  and Iran , respectively. Only three studies reported the proportion of abstracts presented in conferences as 54% in Togo, 11% in Cameroon and 2% in Uganda [12,13,14]. Post-graduate students were first authors of their work in 23%, 54%, 62% and 80% in Cameroon, India, Egypt and Turkey, respectively, and in nearly all their papers in Uganda [2, 12,13,14, 17]. Cross-sectional studies were the predominant designs of post-graduate students’ research projects (44–80%), while randomised trials were few (3–23%) [2, 9, 13].
We report that the majority of post-graduate students in low-income countries infrequently publish their research theses. Secondly, the most published studies are cross-sectional in design with hardly any clinical trials, likely because of feasibility considerations with higher logistic demands, particularly for students. Third, it is apparent that post-graduate students are not the first authors in a significant proportion of their published work. Nevertheless, the evidence of citations suggests that post-graduate students’ research work is used in some form.
Findings in relation to other systematic reviews
Although we did not identify data to support our primary objective, we found a systematic review about interventions for increasing scholarly productivity among residents in the United States and Canada, which are high-income settings . This review mapped the following approaches that were associated with increased scholarly productivity: protected research time, research curricula, research directors, dedicated research days, and research tracks, but with mixed effects on resident presentations or publications. It would be important to extend these single studies by explicitly testing the approaches found to be associated with productivity in low-income countries such as Uganda.
The proportion of publications in our review was relatively low. In a seminal paper by Dickersin et al. , students in John Hopkins University in the 1980s were found to publish less than the faculty. Although it is not clear why students published less, we found that students were not necessarily first authors of their work even when they published. Indeed, authorship would be an incentive for career advancement for which students and faculty would benefit. It is possible that power imbalances may explain this finding and, in some instances, discourage would be student authors from publishing.
With only a single study appropriately documenting the determinants of productivity and citation, we could hardly draw firm conclusions. The results herein suggested that younger students were more likely to publish and that cohort studies were more likely to be published. Bullen and Reeve  documented that Master of Public Health students in New Zealand mentioned that barriers to publication included a protracted publication process and a negative perception of the importance of the results. Future qualitative investigations in this setting would be informative.
Systematic review strengths and limitations
This is the first systematic review documenting the academic research productivity of postgraduate students in low- and middle-income countries. We have employed a robust and internationally agreed methodology  to conduct this systematic review with a sizeable sample of included studies and students.
Nonetheless, we report two main methodological limitations in our review. First, we did not find a single study assessing the effects of interventions to increase academic research productivity of post-graduate students. This could be explained by the identification of all the relevant studies or university reports, limited by grey literature beyond the reach of our review team. Nonetheless, we employed a robust and comprehensive search strategy beyond the electronic search, which enabled us access to articles for over a 40-year span and in languages other than English. Secondly, identifying the outcome of ‘use’ of students’ research in the policy process or decision-making in health remains a challenge, with citation being an imperfect proxy. An additional limitation was the fact that we did not explore grey literature more effectively, largely due to resources constraints.
Implications for future research and policy
It is clear that there is a sheer lack of evidence to effectively assess interventions to increase productivity and use of research conducted by post-graduate students in low- and middle-income countries. The cohort studies that exist did not analyse the data in a manner that permits comparisons of groups exposed to specific approaches, an area that can be strengthened. Future studies should be prospective, employing mixed methods to investigate interventions to increase productivity and citation of post-graduate students’ research as well as to identify unique aspects such as publishing in predatory journals. Additionally, it is becoming more common for early career researchers globally to publish during their candidature (PhD by publication), an area that can be examined further. Policy-makers and actors should invest in supporting research in this area.
We report a low productivity of post-graduate students’ research in low- and middle-income countries, including the use of post-graduate students’ research in evidence-informed health policy. Secondly, we did not find a single study that assessed strategies to increase productivity and use of post-graduate students’ research in evidence-informed health policy, a subject for future research.
Ogunyemi D, Bazargan M, Norris K, Jones-Quaidoo S, Wolf K, Edelstein R, et al. The development of a mandatory medical thesis in an urban medical school. Teach Learn Med. 2005;17(4):363–9. https://doi.org/10.1207/s15328015tlm1704_9.
Dhaliwal U, Singh N, Bhatia A. Masters theses from a university medical college: publication in indexed scientific journals. Indian J Ophthalmol. 2010;58(2):101–4. https://doi.org/10.4103/0301-4738.60070.
Obuku EA, Lavis JN, Kinengyere A, Mafigiri DK, Sengooba F, Karamagi C, et al. Where is students’ research in evidence-informed decision-making in health? Assessing productivity and use of postgraduate students’ research in low- and middle-income countries: a systematic review. Health Res Policy Syst. 2017;15(1):18. https://doi.org/10.1186/s12961-017-0169-9.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.
Ocan M, Obuku EA, Bwanga F, Akena D, Richard S, Ogwal-Okeng J, et al. Household antimicrobial self-medication: a systematic review and meta-analysis of the burden, risk factors and outcomes in developing countries. BMC Public Health. 2015;15:742. https://doi.org/10.1186/s12889-015-2109-3.
Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias). J Clin Epidemiol. 2011;64(4):407–15. https://doi.org/10.1016/j.jclinepi.2010.07.017.
Sterne JA, Hernan MA, Reeves BC, Savovic J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. https://doi.org/10.1136/bmj.i4919.
Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94. https://doi.org/10.1016/j.jclinepi.2010.04.026.
Ahmed Y, Kanyengo C, Akakandelwa A. Mapping postgraduate research at the University of Zambia: a review of dissertations for the Master of Medicine Programme. Med J Zambia. 2010;37(2):52–7.
Linch GF, Ribeiro AC, Guido LA. Graduate program in nursing at the Federal University of Santa Maria: trajectory and results. Rev Gaucha Enferm. 2013;34(1):147–54.
Özgen Ü, Egri M, Aktas M, Sandikkaya A, Özturk ÖF, Can S, Ozcan C. Publication pattern of Turkish medical theses: analysis of 22,625 medical theses completed in years 1980-2005. Turk Klin J Med Sci. 2011;31(5):1122-31.
Pitche PT, Onipoh DK, Tchangai-Walla KL. Scientific dissemination of medical dissertations at the University of Lome (Togo). Sante. 2007;17(2):117–20.
Obuku EA, Lavis JN, Kinengyere A, Mafigiri DK, Sengooba F, Karamagi C, et al. Academic research productivity of post-graduate students at Makerere University College of Health Sciences, Uganda, from 1996 to 2010: a retrospective review. Health Res Policy Syst. 2017;15:30. https://doi.org/10.1186/s12961-017-0194-8.
Munung NS, Vidal L, Ouwe-Missi-Oukem-Boyer O. Do students eventually get to publish their research findings? The case of human immunodeficiency virus/acquired immunodeficiency syndrome research in Cameroon. Ann Med Health Sci Res. 2014;4(3):436–41. https://doi.org/10.4103/2141-9248.133474.
Nour-Eldein H, Mansour NM, Abdulmajeed AA. Master’s and doctoral theses in family medicine and their publication output, Suez Canal University, Egypt. J Fam Med Prim Care. 2015;4(2):162–7. https://doi.org/10.4103/2249-4863.154622.
Motamed-Jahromi M, Leila Dehghani S. Nursing MSc theses: a study of an Iranian College of Nursing and Midwifery in two decades (1990–2010). Glob J Health Sci. 2014;6(5):118–24. https://doi.org/10.5539/gjhs.v6n5p118.
Sipahi H, Durusoy R, Ergin I, Hassoy H, Davas A, Karababa A. Publication rates of public health theses in international and national peer-review journals in Turkey. Iran J Public Health. 2012;41(9):31–5.
Stevenson MD, Smigielski EM, Naifeh MM, Abramson EL, Todd C, Li ST. Increasing scholarly activity productivity during residency: a systematic review. Acad Med. 2017;92(2):250–66. https://doi.org/10.1097/acm.0000000000001169.
Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA. 1992;267(3):374–8.
Bullen CR, Reeve J. Turning postgraduate students’ research into publications: a survey of New Zealand masters in public health students. Asia Pac J Public Health. 2011;23(5):801–9. https://doi.org/10.1177/1010539511417998.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. https://doi.org/10.1186/2046-4053-4-1.
Eftekhari Y, Rezaeian M, Zare Bidaki M, Arabshahi A. A survey on the status of article publication from defended medical theses in Rafsanjan University of Medical Sciences, School of Medicine during 1993–2007. J Rafsanjan Univ Med Sci. 2013;12(7):531–44.
Ogrenci A, Eksi MS, Ozcan-Eksi EE, Koban O. From idea to publication: publication rates of theses in neurosurgery from Turkey. Neurol Neurochir Pol. 2016;50(1):45–7. https://doi.org/10.1016/j.pjnns.2015.11.007.
Koca K, Ekinci S, Akpancar S, Gemci MH, Ersen O, Akyildiz F. An analysis of orthopaedic theses in Turkey: Evidence levels and publication rates. Acta Orthop Traumatol Turc. 2016;50(5):562–66. https://doi.org/10.1016/j.aott.2016.03.001.
Ticse R, Ygreda P, Samalvides F. [Publication of research projects for certification as medical specialists at a Peruvian university, 2007-2010]. Rev Peru Med Exp Salud Publica. 2014;31(2):292–6.
The authors acknowledge Benjamin Temper and Joanita Nangendo for initial review of the search strategy of the protocol.
The writing of this manuscript has been supported by the IDRC International Research Chair in Evidence-Informed Health Systems and Policies, jointly held by Nelson Sewankambo and John Lavis. Dr Ekwaro Obuku is a doctorate student at Makerere University College of Health Sciences, Kampala, Uganda. He is a beneficiary of this training grant number 104519–008, held by Prof Nelson K Sewankambo of Makerere University, College of Health Sciences, Kampala, Uganda, and Prof John N Lavis of McMaster University, Hamilton, Ontario, Canada.
Ethics approval and consent to participate
As this is a systematic review of literature this section is not applicable. Nonetheless, the School of Medicine Research & Ethics Committee, the Uganda National Council for Science & Technology (HS 3268) and the Office of the President of Uganda (ADM/154/212/01) approved this study.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplement 1. Feasibility of yield of literature of pilot electronic search strategy for post-graduate students’ research. Supplement 2. Updated search strategy as at 17th July 2017 in PubMed (https://www.ncbi.nlm.nih.gov/pubmed/). (DOCX 21 kb)
Risk of bias assessment tool: productivity of post-graduate students’ research in low- and middle-income countries. (DOC 59 kb)
About this article
Cite this article
Obuku, E.A., Lavis, J.N., Kinengyere, A. et al. A systematic review on academic research productivity of postgraduate students in low- and middle-income countries. Health Res Policy Sys 16, 86 (2018). https://doi.org/10.1186/s12961-018-0360-7
- Evidence-informed health policy
- Knowledge translation