The principal findings
We report low publication productivity among post-graduate students at Makerere University. In addition, older students published less and cohort studies were more likely to be published. In terms of first citations, one in five dissertations were cited mostly in peer-reviewed journal articles. At least one in 20 dissertations were cited in policy-related documents, signifying use in the policy process mostly by international actors.
Findings in relation to other studies
Most of the research work by post-graduate students at Makerere University remains unpublished and therefore less accessible to end-users including decision makers. Our finding of a low publication proportion (18%) is corroborated by studies in Cameroon [
12], Egypt [
9] and India [
8] at 14%, 20% and 30%, respectively. Similar studies in higher income countries reported higher but still sub-optimal publication at 40%, 45%, 45%, 65% and 66% in Iran [
27], New Zealand [
5], Canada [
28], Spain [
29] and the United States [
30], respectively.
Publication of post-graduate research findings is necessary, but not sufficient for its utility. Our paradoxical finding in publication versus citation of post-graduate research is substantiated in recent paper, that about 31% of published World Bank reports were never read (not downloaded at all) [
31]. Therefore, publication is not a guarantee that these results will be immediately useful. Nonetheless, publishing, particularly in open access journals, will, at the very least, nourish the body of empirical evidence readily available for use.
Post-graduate students’ dissertations were barely used to inform policy or practice guidelines. That most citations were by peer-reviewed articles suggests a predominant use in informing further research, with far less application in policy-related processes. What, then, could explain this very low utility? In two systematic reviews, decision-makers perceived publications aimed at a scholarly audience and irrelevance of publications as two impediments to the uptake of research for decision-making [
32,
33]. Clearly, the perennial health system challenges in low-income countries (health financing, governance and human resources) were under-researched, lending currency to the argument of irrelevance of the post-graduate students research. However, our results show alignment with health-related MDG priorities and Uganda’s disease burden, as further corroborated by a study of the University of Zambia [
13]. Perhaps this is an artefact of what is emphasised by academia (peer-reviewed publications versus evidence briefs for policy) or a documentation deficiency (including poor indexing of policy-related documents) or a selection bias, since most of these policy-related documents were of international agencies that we retrieved online. Until recently, Ugandan and Cameroonian policy documents rarely included references to research, reflecting the same trend in low-income settings [
34]. Assessing how research is used remains a subjective process without documented hard evidence, a subject for further inquiry.
Younger age at enrolment into the post-graduate courses independently predicted publication in peer-reviewed journals; the fact that publication output reduced among older post-graduate students may reflect shifting priorities or social responsibilities and resultant opportunity costs for the limited time [
5]. Noteworthy, marital status, which would imply less time available for research, did not impact on productivity of publications in the multivariable model in our study. Regardless of this finding, newly qualified post-graduate students are commonly caught up finding a job or career openings without time to think of immediate publication. This is a plausible explanation given that it took at least 2 years after dissertation completion to publish approximately 1 in 40 manuscripts from these dissertations. Our findings are augmented by studies from similar settings. Among the Doctoral and Masters dissertations in Egypt that were published, 63% were published after the first year [
9]. In an Indian study, only 8% of the dissertations were published in the first year after completion of the Masters degree [
8].
Since the type of research design used was a determinant of publication irrespective of scholarship funding, it is tempting to suggest that the quality of evidence from the research influenced publication. However, this contradicts a recent study where cross-sectional designs were significantly published more by pharmacy residents in the United States [
35]. Dhaliwal et al. [
8] showed that study design did not influence publication in India. Probably, peer-reviewer bias could have had an important bearing on the types of studies by post-graduates at Makerere University that eventually got published. It is plausible that such post-graduate students underwent their dissertation work using already established cohorts of patients that were managed by large collaborative programs. One could argue that they were more likely to be published because of the push from the partners managing the projects that supported the research.
The finding that only 2% of abstracts were presented in conferences may be misconstrued as low-level dissemination efforts. However, international conferences abroad are likely to have put their conference abstracts or proceedings online, whilst those organised in Uganda or within the African region preferred hard copy abstracts and reports without online archives.
Strengths and limitations
We highlight several strengths and a few limitations in our review. The strengths lie in the new information on previously un-researched outcomes (citation of research) and time-to-event indicators, among post-graduate students. The breadth of health disciplines covered (biomedical, clinical and public health), a large sample size and a long study period make our findings even more robust. Nonetheless, we encountered some shortcomings, particularly with regards to missing data or inaccuracies due to limited availability of records dating 20 years back. Although we used multiple corroborating sources to fill in information gaps, we did not perform multiple imputations of the missing data in the regression models. We acknowledge that citation of research is one of the many ways to identify research use. Still, it was challenging to document the outcome of use in policy-related documents, which we likely underestimated. Despite our anticipation of fewer outcomes in the third period towards 2010 because of a shorter study time period, this was not the case. Interestingly, what we observed was a general reduction in the time to outcomes (publication, citation and conference presentations), which could be explained by bias in the longer continuation of the first two periods or increased advocacy, supervision and funding support to publish post-graduate students research. We did not assess local use of the results of post-graduate research dissertations, say in the field at the level of district, by district health teams (as would be the case for Masters of Public Health students) or management of patients (as would be the case for clinical disciplines). Similarly, we may have missed abstract presentations in local conferences that do not have electronic records posted online. It is possible that publication bias, which is the tendency for investigators to submit manuscripts and of editors and reviewers to accept them, based on the strength and direction of the research findings, may account for the low publication proportion [
36]. We thus urge readers to interpret our results within this context.
Implications for policy on post-graduate research
Our results have implications for stakeholders in the post-graduate training enterprise at national and global level. First, we have documented the deficiency in publishing post-graduate research dissertations in a leading education and research training institution in Africa. Secondly, we have provided empirical evidence that post-graduate research is cited in policy-related documents, however minimal. Considered together, this calls for investment in promoting effective interventions that increase publication [
37] and active knowledge translation approaches that will link this research to policy [
32,
33]. A low-cost intervention such as obligatory publication of Masters dissertations may positively impact on productivity, particularly by scholarship funders, as is the case for doctoral students at Makerere University. Whichever intervention is chosen should consider the unique circumstances of older post-graduate students. Last but not least, education research funders should prioritise impact evaluation studies on post-graduate research dissertations.
Implications for future research
We identified opportunities for further inquiry. Most studies have emphasised productivity and citations, yet the ultimate goal is for this research to inform decision-making or policy formulation and hopefully improve population health. Research may be used conceptually (to enlighten the decision maker), instrumentally (to solve a problem at hand), or symbolically (political or tactical use to justify action or inaction) [
38,
39]. Although it is likely that post-graduate research was used either conceptually or instrumentally we did not distinguish the two. An in-depth qualitative investigation would inform how post-graduate research is used as well as identify potential interventions for knowledge translation in the local Ugandan policy environment.
Indeed, the ground is fertile for quasi-experimental designs testing interventions that have shown some promise in raising productivity elsewhere or prospective cohort studies assessing broader outcomes such as number of dissertations submitted for publication and those rejected, or comparisons with doctoral students or established faculty. Finally, mixed methods analyses about where and why post-graduate students publish (or not), including citometrics (impact factor), which were beyond the scope of our study, would improve understanding about the context-specific interventions to increase research productivity.