A scoping review of competencies for scientific editors of biomedical journals

“…journals, some of which have reported research for many decades, are still not producing articles that are clear enough to really judge a study’s conduct, quality, and importance—let alone to allow other researchers to reproduce it or build on it” [1].

Biomedical journals are the main route for disseminating the results of health-related research [2]. However, when examined more closely, the articles that journals publish are problematic; critical details are often missing or poorly reported, consequently reducing their quality, transparency, reproducibility, and usefulness for decision makers [3] – this is wasteful, diminishes scientific and fiscal value, and is unethical [4]. Authors and scientific journals share the majority of the responsibility for these shortcomings, as the former are accountable for the integrity of a study’s conduct and the accuracy of reporting of the content within the manuscript, while the latter are accountable for decisions regarding its publication. On the side of journals, it is scientific editors (by which we mean editors, and ultimately the Editor-in-Chief, who are tasked with making decisions about the content and policies of journals) who are accountable for all material published in their journals. Readers have a right to expect these editors to implement all reasonable actions that could lead to best practices within their journals, as well as journals having processes in place to ensure the quality of the papers they publish.

Unlike many other professional groups, such as clinicians and healthcare professionals, many scientific editors of biomedical journals operate largely without formal training and universal certification is not yet a high priority [5]. Instead, editors generally are invited to serve in their role by publishers, based on their expertise and stature in the field, since such expertise is essential for evaluating research and stature is important for establishing the reputation of the journal and attracting submissions. However, such expertise does not guarantee that editors have the background or training necessary to carry out their roles and responsibilities. Editors may or may not be paid for their role and financial support for the editorial role often does not include travel or training funds. Most editors work part-time as they continue their academic responsibilities in research and/or clinical work, with time for completing editorial responsibilities – much less training – being at a premium. Researchers and peer reviewers similarly have no international standardized formal training or certification as to research conduct, reporting, and evaluation, making the editor’s job even more demanding. This situation is highly problematic given that the consequences of deciding what gets published and the degree of quality that is acceptable impacts future research, decisions, and healthcare directly. Our view is that the lack of consistent training of editors reduces the value of the published literature, including its quality, transparency, and reproducibility, thereby reducing value for money to funders and the usability of research findings, ultimately degrading public trust in the research record [3]. However, we are unaware of any research that directly addresses this topic. Additionally, while the training of biomedical editors is an important mechanism to ensure the quality of the published literature, other important changes in tandem with this, including re-examining the training offered to peer-reviewers and training graduate students in study conduct, analysis, interpretation, and reporting, could also have a beneficial effect.

Some organizations, for example, the World Association of Medical Editors (WAME) [6] and the Committee on Publication Ethics (COPE) [7], provide rigorously-developed resources for biomedical journal editors free of charge, including guidance on the role of the medical editor, editorial policies, and listservs on which editors’ questions and issues are discussed. There are also individual websites and blogs, such as “Journalology” [8], that provide thoughtful commentary on current issues related to publication science. Several commercial groups offer short training courses for editors [9, 10]. However, for any comprehensive editor training program to work effectively and be assimilated into practice, it must be based on what the broader biomedical editor community considers to be core competencies.

We are unaware of any body of literature systematically identifying core competencies for biomedical editors, nor any agreement on or attempt at a consensus process to determine what they should be. For the purposes of this research, we borrow from the literature on competency-based continuing professional development to define competence as “the array of abilities across multiple domains or aspects of [practitioner] performance in a certain context” [11]. We thus define core competencies as the essential knowledge, skills, and behaviors necessary for the practice of scientific editing of biomedical journals. We believe it is important to develop a set of core competencies so that training programs can then be developed and tailored with the intent that all editors meet some basic globally agreed-upon standards. Other stakeholders, such as publishers (including medical associations who have their own journals), peer reviewers, and authors (researchers), also need to contribute to this effort. Herein, as a starting point, we report a scoping review of possible core competencies of scientific editors of biomedical journals.

The protocol for this project has previously been deposited in the University of Ottawa’s Digital Repository (uOttawa Research) prior to beginning the screening phase [12]. Our methodological approach was guided by the Arksey and O’Malley Framework [13], as well as the additional suggestions to this framework made by Levac [14]. Specifically, we undertook the six-step process of: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting results; and (6) consultation.

This scoping review identified 225 relevant publications, spanning more than 60 years, and involving authors from more than 20 countries. It produced a comprehensive list of possible competency-related statements for the scientific editor position within biomedical journals. This categorized list of statements will be used in a subsequent Delphi exercise aiming to ask a broad spectrum of scientific editors of biomedical journals to rate the importance of each statement in relation to performing their duties as a scientific editor. This data will then help inform a consensus meeting in which a select group of editors will collaborate to outline a set of core competencies.

Despite our finding that the competencies of editors have been discussed in the published literature since the mid-1950s, a recent systematic review [17] found no comparative studies of the effectiveness of training for editors. This is concerning, given the gate-keeping role that scientific editors play as guardians of the scientific record [23]. Trends in the number of publications annually would seem to indicate that, while the overall number of publications in this area has grown since the topic first emerged in the literature, interest peaked around 2011 and is now waning. The trend, however, is more reflective of editorial-type articles in journals, which have declined since 2011, while the number of research-based publications has remained relatively stable (yet sparse) during the same timeframe.

One possibility for the decline in total publications is that organizations may believe these issues have already been identified and they are working on training materials for them rather than publishing more research or commentaries about them in medical journals. A possibility for the continued lack of research in this area may be that the focus of many major funding agencies is to fund by disease (e.g. heart, kidney, cancer, diabetes). As journal editing, and the field of journalology more broadly, is a domain that spans across the spectrum of research on diseases, it is often difficult to find appropriate funding opportunities and even more difficult to convince specific disease-based funding agencies that the research merits their funding investment.

A major challenge that we anticipated as part of this scoping review was that a large proportion of the evidence may not be in the traditional research-based literature. For this reason, we placed a heavy emphasis on extensively searching the grey and non-research-based literature by including two related environmental scans as part of the larger program on core competencies for scientific editors of biomedical journals. However, it is still possible that we may not have captured every aspect of a biomedical editor’s work in our searches. In particular, some of the more tacit (e.g. difficult to describe) aspects of the work of editors may simply not be documented, or may reside in documents that may not necessarily be found in a database or Google search (e.g. resources residing behind membership or password-protected webpages, paywalls). We expect that any critical missing items will be brought forth in the training needs assessment, the Delphi, or the consensus meeting. Additionally, because of the subjective nature of this type of data extraction, it is possible that some competency-related statements were missed within the included publications. However, we feel that the likelihood of this was reduced as several members of the research team are editors. Finally, due to the broad inclusion criteria and the decision to preserve the wording used by authors to describe potential competencies as much as possible, it is likely that some items may not actually be competencies per se, but may instead describe tasks, behaviors, and knowledge related to competencies. However, these items are still useful in describing important aspects of editors’ work and will therefore contribute valuable information for the development of core competencies.

With the large number of competency statements and our desire to create a manageable list for use downstream in our program of research, our efforts to remove redundant and overlapping items in order to streamline the list may also have led to the elimination of some nuances between items that were subtly different from one another. While we implemented measures to ensure consistency in our methods (i.e. piloted forms, duplication of the classification exercise), ultimately there is a degree of interpretation and selectivity embedded in this process. Thus, our list of possible competencies may not include all of the competencies of biomedical editors. As noted above, the next phases of this project are designed to elicit any missing items.

Another limitation is that for feasibility purposes we only considered English and French articles, which raises the possibility that relevant information published in another language was missed. Similarly, the databases searched may not have included some journals from outside fiscally resourced countries.

To our knowledge, this scoping review is the first attempt to systematically identify possible competencies of editors. On its own, the review will serve to inform readers on the extent and nature of existing literature in this area, as well as the breadth of skills, abilities, tasks, knowledge, and training that may be necessary to fulfill the position of scientific editor at a biomedical journal. More importantly, the review is part of a larger program to develop a minimum set of core competencies for scientific editors of biomedical journals.

The purpose of the 203 competency-related statements generated here is to be the central tool used for a Delphi exercise involving scientific editors of biomedical journals from around the world. Subsequently, these statements will serve to stimulate discussion at a consensus meeting in which the goal will be for relevant stakeholders to agree upon a minimum set of core competencies for scientific editors of biomedical journals. This evidence-based approach will ultimately lay the groundwork for the development of specific competency-based training and certification for scientific editors of biomedical journals [5]. The development of core competencies and subsequent training represent critical steps toward ensuring that the publication of biomedical research truly represents a hallmark of quality and trustworthiness, both within and beyond the research community.