Scientific activity by medical students: the relationship between academic publishing during medical school and publication careers after graduation

What’s learnt in the cradle lasts to the tomb: a saying that applies to activities like riding a bicycle. But does it also apply to the involvement of clinicians in science? All clinicians should at least be able to use research in their clinical practice, a competency required by the Netherlands Federation of University Medical Centres, the U.S. Accreditation Council for Graduate Medical Education (ACGME) and the Canadian Medical Education Direction for Specialists (CanMEDS), among others [1‐3]. In addition, we need clinicians who conduct research themselves: clinician-scientists. However, there is a shortage of clinician-scientists, which is visible in multiple places in the world, for example in the United States, Canada, and Europe [4‐8].

This shortage is thought to lead to undesirable effects. For example, it has been argued that clinical practice and science have become too disengaged—into patient care on the one hand and basic research on the other [9]. As a result, medical research might lose clinical relevance, while clinical problems might remain unanswered. The question is how to stimulate clinicians to become and stay involved in research. An answer may lie in scientific education during medical training [10].

Formal scientific education can take various shapes and forms [11]. These may be categorized according to student involvement: students as audience or participants. In the forms where students are the audience, learning is quite passive. However, in the forms where students are participants, students learn actively about research, which has been asserted to be a much more effective form of scientific education [12].

The ultimate form of active learning in scientific education, it can be argued, is participation of students in the scientific process. Often, this takes the form of research projects, which usually take place in the graduate phase, but may also take place in the undergraduate phase [13‐15]. Undergraduate students are motivated to do research already early in their studies. This provides an opportunity to engage them in research early on in medical training [10]. The question is what the long-term outcomes are of such early engagement in research [4].

Here, we study whether publication during medical training, capped by authorship of one or more scientific publications, is associated with the post-graduation scientific activity of medical graduates. Are medical students who experience success in the sense that they successfully go through both the research process and the scientific publication process more likely to stay involved in research and keep publishing after graduation? There have been other studies that predict research engagement after medical training, but these often focus on either scholarly concentration or MD/PhD programs, not on the larger group of MD graduates, e.g., [16‐18]. In addition, many of these do not directly evaluate scientific publication as an outcome variable but rather the intention to be involved in research [18‐20].

We use bibliometric methods to study the relationship between pre-graduation and post-graduation publication. Bibliometric methods are especially suitable to study this relationship, as they can be used to track the scientific performance of individuals, reinforcing its strength by grouping the scores of individuals to larger sets of publications, with more robust bibliometric scores of citation impact as a result.

Specifically, we aim to study the following questions: are medical students who publish before graduation: 1) more likely to publish after graduation, 2) do they publish a greater number of papers after graduation, and 3) do they publish papers with a higher citation impact after graduation? If the answers to these questions are positive, authentic research learning opportunities during medical training and the opportunity to publish scientific work could impact students’ interest in a research career.

All 2005–2008 MD graduates from all eight Dutch university medical centres were included in the study. All eight agreed to participate and provided the names of their graduates. With 1658 graduates in 2005, 1832 graduates in 2006, 1990 graduates in 2007, and 2064 graduates in 2008 this study includes 7544 medical graduates. The study was approved by the Educational Institutional Review Board of Leiden University Medical Center (reference number OEC/ER7RC/20171212/1) on 12 December 2017.

In the Netherlands, in 2005–2008, medical school comprised 6 years of study, of which 4 years were pre-clinical and 2 years were clinical training. Students typically start medical school directly after finishing secondary school, which means that the majority of students are approximately 18–19 years old when starting medical school and they have not previously obtained an undergraduate degree [21]. Partly because of the nature of the medical school system, MD/PhD programs in the Anglo-Saxon tradition are virtually absent. Such programs do exist but typically only draw less than twenty medical students. When medical students pursue a PhD degree, they usually do so after MD graduation (either full-time or in combination with postgraduate medical specialty training). All eight medical schools provide scientific training in line with the national Blueprint for Medical Education [3], including a compulsory full-time individual research project of at least 14 weeks in pre-clinical training.

Therefore, we employed two complementary strategies. We automatically assigned publications to a subset of all graduates with relatively rare names, a strategy also employed in other studies [22, 23]. Additionally, we manually assigned publications to a 10% random sample from this group. We selected rare names based on the number of initials and the prevalence of the last name in the Web of Science (the number of unique combinations of last name and initials). We selected all graduates with three or more initials and a last name occurring in less than 1000 unique combinations of last names and initials, and with two initials and a last name occurring in fewer than 50 unique combinations. This resulted in a set of 4145 (out of 7544) MD graduates. In addition, we used an author clustering algorithm developed by the Centre for Science and Technologies Studies [24]. The algorithm sorts all publications in the Web of Science into clusters of publications presumed to be authored by the same person. We matched the graduates’ full names (last name plus all initials) to the most common full name in an author cluster. This decreases the chance of false-positive assignment, as all initials have to match. To further decrease this chance, the first publication in the cluster also had to be published between 6 years before (as it is quite unlikely that a medical student would publish before starting their studies) and 6 years after graduation. From the clusters we collected all articles, reviews, and letters published between 6 years pre-graduation and 6 years post-graduation. This has the added benefit that also papers on which students did not use all their initials are collected, which decreases the chance of false-negative assignment (of course as long as they have other publications with all initials listed).

As a measure of citation impact after graduation, we used the mean normalized citation score of the papers published after graduation [25]. We counted the number of citations to each paper between the year of publication and two years afterwards. Papers were counted fully, i.e., each paper counts equally, regardless of whether it was authored by one or multiple authors. The citation score was then normalized by scientific field, as the number of citations that publications receive is greater in some fields than in others [25]. By definition, the normalized citation score of a field is 1; a score higher than 1.2 is considered to be above field average, a score below 0.8 lower than field average.

For statistical analyses we used SPSS Statistics version 23.0.0 (IBM). To test whether group differences were statistically significant, we used 1) the chi-square test for the likelihood to publish after graduation, 2) the Mann-Whitney U test for the number of papers published after graduation (as data were not normally distributed nor could be transformed to become normally distributed), and 3) an independent samples t‑test for the mean normalized citation impact (MNCS; Box-Cox transformed with λ = 0.75 to follow normal distribution).

In this study, we found that medical students who published during their studies were almost twice as likely to publish after graduation, and published more papers after graduation. We also found these medical students had a slightly higher citation impact, albeit this was not statistically significant in the smaller group of manual publication assignment. This means that the early engagement of medical students in research leading to scientific publication is positively associated with sustained publication after MD graduation. Whereas this relationship may seem straightforward, no study has looked at the strength of this association before by using bibliometric methods. In addition, many studies on this topic have intended research involvement or interest in a research career as dependent variable rather than measures of actual research involvement [18‐20]. It is important to note that within the studied group of medical students, all students had been required to undertake a full-time individual research project of at least 14 weeks in pre-clinical training [3]. This means that rather than looking at the effect of undertaking a research project versus not undertaking such a project, we compared students who had published before graduation, which reflects an experience of success, to those who had not. In the comparison between these groups, we found that pre-graduation publication was associated with sustained publication, a higher number of publications and higher citation impact after graduation.

Social Cognitive Career Theory, and especially its key concept of self-efficacy, could explain why such a positive association exists [26]. Mastery of an activity leads to higher self-efficacy [27]. Early involvement in research leading to the publication of a student’s scientific work could increase research self-efficacy [20, 28], which could be an explanation of our results. The effect of a success experience during medical school is not the only possible explanation of the association we found, though, as the effect of self-efficacy is not limited to the period of medical training. Career interests already develop during childhood and adolescence [26]. Certain medical students could thus have developed a greater interest in research than others already before starting medical training [20]. If these students publish more often before and after graduation, it is a confounder of the association we found between pre-graduation and post-graduation publication.

Other explanations of the association we found are the extrinsic motivation to conduct research and selection effects. A previous study by our group showed that medical students have a high extrinsic motivation to do research, already in their first year. They expect it to improve their chances for their preferred residency spot [10]. A selection effect is at play if PhD advisors prefer to hire the recent MD graduates who have published during their studies as PhD candidates; this could also contribute to the association we discovered.

On a more general level, our results show that quite a number of medical students in the Netherlands published one or more papers in the 6 years after graduation: 1591 out of 4145, which is 38%. This finding seems to disprove the clinician-scientist shortage often reported upon [4‐8], and which we mentioned in the introduction. At the same time, we also noted in our Results section that the distribution of the number of publications is heavily skewed. Of the 38% who published after publication, almost one-third (31%) published only one paper. These graduates do not appear to have remained active clinician-scientists after graduation. In addition, the selection system for medical specialty residencies may have increased the number of graduates with post-graduation publications. As mentioned above, medical students are quite extrinsically motivated to pursue a PhD degree because it will increase their chances of a residency spot. It will therefore be interesting to repeat our study in a few years’ time to see how many medical graduates remain scientifically active after the period of residency spot competition has ended. Then, this basis for extrinsic motivation will have disappeared while other barriers to academic career involvement are still present, such as difficulties combining research, clinical care, and family and personal life [29, 30].

As mentioned in our introduction, there is currently a shortage of clinician-scientists [4‐8]. Medical students who publish during their studies are more likely to keep publishing after graduation, are more productive, and have a higher citation impact. Although this association could be also caused by other factors, there is good reason to assume that the association is at least partly caused by the success experience that publication during medical school gives students [19, 20]. Medical schools could alleviate the clinician-scientist shortage by providing students with more opportunities for authentic research experiences during medical training, including the opportunity to gain experience in the scientific publication process.

In conclusion, when it comes to early scientific publication by medical students, what is learnt in the ‘cradle’ indeed lasts. Although we cannot infer from our results whether it lasts until the tomb, we do know it lasts at least during the 6‑year period after graduation.