Background
A systematic review (SR) refers to secondary research that not only systematically synthesizes all available research evidence relevant to a particular topic, but also interprets, evaluates and appraises the quality of such results [
1]. With up-to-date results from SRs, relevant evidence becomes available for researchers to decide not only whether to embark on new research but also whether to continue supporting ongoing studies [
2,
3]. Many policymakers and stakeholders seek to use research evidence to influence policymaking, for whom systematic reviews have become an indispensable resources. [
4] With the dynamic nature of gathering and presenting research evidence, SRs must be up-to-date to enable different stakeholders such as researchers, funding bodies, and data-monitoring committees, to use reliable evidence to inform decision-making [
2].
Against this background, it is therefore important to ascertain whether the information published in the SRs are up-to-date. In their evaluation of 300 SRs published in Core Clinical Journals, Beller et al. [
5] found that the median time from the last search date to first publication was 8 months (~240 days). Another study reported that the median time taken for a research article published in top nursing journals from the end of data collection to first publication was 855 days [
6]. However, an interesting gap to explore has remained in the literature: no studies have been conducted to specifically examine the time lag for SRs published in nursing journals.
SRs with longer time lags of publication may appear to be less convincing for the stakeholders due to the concern with less updated evidence. However, such impact on decision-making also depends on the frequency of publication of primary studies in the topic. For example, if multiple new studies are published every year, one or 2 years’ delay in publishing the SR will imply the untimely exclusion of such newly published studies; otherwise, there is less impact of delayed publication on the timeliness of findings. A recently published meta-epidemiological study [
7] reported that 73% of the SRs published in top medical journals included 10 or more studies while Pollkki et al. [
8] reported a median of 15 studies were included among 39 SRs in high-impact nursing journals. Nevertheless, they did not examine the average number of those studies by the years of the SRs.
The time taken for completing the SRs also affects the time required for publication. Problem identification, search-strategy development, database search, study selection, data extraction, data analysis and synthesis are the basic steps in conducting systematic reviews. Intuitively, if copious records are retrieved from database, more time is expected for study selection and data extraction; therefore, the number of records retrieved may relate to the time taken for publication. Furthermore, performing meta-analysis requires specific training but it has been reported that many nursing researchers and students did not fully understand the statistical techniques of meta-analysis or the various important concepts that underpin it [
9]. Thus, the inclusion of meta-analyses in the SR may take more time to complete the review.
Another aspect to influence the speed of publication is the journal review process. Publishing a SR in a peer-reviewed journal usually involves several phases including manuscript submission, first decisions from editors, peer-reviews, follow-up decisions from the editors, and revision and re-submission. The whole process can be repeated numerous times if the SR is rejected at the editor’s first decision, thereby prolonging the publication process: it has been suggested that editors of nursing journals would take at least 8 weeks to first decide whether to accept a manuscript, or to request revisions [
10]. In addition, the timing of submission has been suggested to affect the time taken for the peer-reviews as it would be difficult for editors to recruit peer-reviewers during holiday periods [
11]. No studies have hitherto examined these effects in the publication process of SRs but they are clearly critical.
On the other hand, the methodological quality of SRs plays an important role in drawing conclusions on intervention effectiveness and carries implications for clinical decision-making. [
12] Although studies have examined the methodological quality of SRs [
8,
13‐
15], none of them have examined the association between the quality and the time taken for publication. Besides, Gomez-Garcia et al. [
16] reported that funding from academic institutions was associated with a higher quality of SRs and hence the funding sources may also affect the time lag.
Given the aforementioned uncertainties, the primary aim of this study is to evaluate the timeliness of SRs published in nursing journals upon first publication, as measured by the time taken from the last search date to actual publication. The total time taken was divided into stages, including the time from the last search dates to the acceptance, online publication, or full publication. The secondary aim is to explore the factors potentially associated with the time taken, such as study quality, funding source, number of submission, et cetera.
Discussion
The identification of comprehensive and up-to-date SRs is important for researchers when evaluating evidence relevant to their topic of interest [
25]. Our results revealed that the median time taken from the last search date to the online and full publication for the SRs in nursing journals were 455 and 669 days. These duration are markedly longer than those in medical journals (i.e., an average of 5.1 months from last search to first publication) [
5]. The lack of timeliness in this context may jeopardize end-users such as guideline producers or policy makers in developing guidelines or revising policies as up-to-dated information may be missed.
In our study, the median time from the last search date to publication was 669 days (i.e., approximately 22.3 months). It has been advocated that systematic reviewers should update their literature search biennially to determine new studies for inclusion into a previously completed SR for updating of evidence [
26]. Therefore, the time taken for publishing SRs in nursing journals is considered excessively long. Such time lags can actually be classified into three categories: from the last search date to submission, from submission to acceptance, and from acceptance to online publication.
The median time from the last search date to acceptance in nursing journals was 393 days, approximately 2.5-fold longer than those in Core Clinical Journals (5.1 months or approximately 153 days) [
5]. The time taken can be classified into two periods: from the last search date to submission and from submission to acceptance. From our results, the median time from the last search date to submission was 167.5 days. The time taken was expended on the authors’ preparation of the manuscript and attempts of submission to journals. According to Polit and Beck [
27], in conducting a SR, five steps are warranted after the literature search, i.e. selecting the studies, evaluating study quality, extracting and encoding data, analyzing and interpreting data (i.e., meta-analysis or synthesis), and writing the review. The time taken to complete these steps varies as this may depend on the researchers’ experiences. Therefore, having an experienced systematic reviewer within the authoring team may expedite these steps. For example, to register a new review with Cochrane Collaboration, a requirement is that at least one author must possess the experience in completing at least one Cochrane Review [
28]. This requirement ensures the timeliness and quality of the review process.
In addition, authors are suggested to conduct preliminary selection of appropriate journals, from which the resultant review will more likely be accepted. This will avoid delays caused by multiple attempts when submitting to different journals [
29,
30]. We contacted the corresponding authors of the 202 reviews and received replies from 62 authors, of whom 47 (75.8%) clarified that their papers were published in the journal in their first attempt. Therefore, such an effect should not be a critical factor among the SRs. However, as the SRs should be submitted before 2014, recall bias from the authors could not be ruled out.
The median time taken for SRs from submission to acceptance was found to be 153 days in this study. It includes the time for review and revision of the submitted manuscript. In comparison, the average time from submission to acceptance for journals from different disciplines was around 6.41 months (~192 days), which is longer than our result [
31]. Some common problems identified for the delays in the review process include discrepancies with journal guidelines, submission during peak periods, reviewers’ delays, and failure to address reviewers’ comments or not submitting revisions on time [
11]. For each of these problems, solutions are suggested to avoid unnecessary delays. As regards discrepancies with journal guidelines, although most authors adhere to the guidelines of the journals, when their manuscript is rejected by one journal, they may directly send the rejected manuscript to another without making any necessary changes. Further delays may follow when the editorial office of the second journal realizes stylistic and guideline breaches of the unedited manuscript and, in rejection, return it to the authors [
32].
Some suggested that manuscript submission during holiday periods would probably lengthen the review process. There appears to be a propensity among some researchers to make use of the holiday to finalize and submit their manuscripts. Nevertheless, during this period, journal editors will find it difficult in recruiting potential reviewers, as some may be on vacation. [
11]. However, no such observation can be concluded from our findings.
Reviewers’ delays may be resolved by shortening the timeframe for review. According to a recent study, some journals do not provide any deadline for their reviewers, whereas others allow up to 1 year [
33]. In the same study, an experiment was conducted where all the participating journals were requested to shorten their current imposed review period by a minimum of 1 week. The results revealed a reduction in the editing time by at least 1 week on average, with several participating journals improving by more than 2 months [
33]. In addition, the study revealed that reviewers were more likely to accept invitations with a shorter response timeframe [
33].
The last problem concerns authors’ failures to address reviewers’ comments or not submitting revisions on time. The time estimated for nursing journals to complete the review is at least 8 weeks, when the editor will then make the first decision to accept or reject the manuscript, or to request revisions [
10]. After receiving comments from the editor and reviewers, the authors should adequately address their comments by highlighting any changes in the manuscript and drafting a point-to-point response letter to expedite decision [
10]. In addition, the authors should adhere to the deadline set by the journals to avoid their manuscripts being treated as new submissions or facing new reviewers [
11]. The aforementioned strategies may collectively mitigate the four potential problems underlying excessive delays and thereby shorten the time taken from the last search date to the acceptance date. From the perspective of the journal editors, they may consider emulating the practice of Cochrane Database of Systematic Reviews where an updated literature search is required if the initial search was conducted more than 6 months prior to the review of the manuscript, as has been adopted by some General Medical Journals such as
Annals of Internal Medicine. This will lessen the time lag between the search and the SR publication. Unless such an updated search before acceptance is made compulsory by journals, it is probable that some authors will not prioritize it since their results and, by extension, the discussion and conclusion may change.
The median time taken from acceptance to online publication of SRs published in nursing journals was 59 days. Such a duration may admittedly not be considered long but this is in fact twice the median time taken compared with papers found on PUBMED 2014 (approximately 25 days) [
34]. This timing varies among different publishers and with current technology the situation should improve. The median time taken from online publication to full publication was 212 days (approximately 7 months). One possible reason underlying the time lags is the overwhelming number of accepted articles that is not commensurate with the publishing space in the journals. Nevertheless, this issue may become less critical with the increasing availability of the online early views in many journals for readers. Our results show that 149 (73.8%) SRs indicated their online publication dates; hence, it may be concluded that at least 73.8% of the SRs were made available earlier than the full publication date. Our results also suggest that the average time taken from acceptance to publication is 242 days. This is considered an improvement compared with the circumstance 10 years ago when the publication of SRs would usually take more than a year in most nursing journals [
35].
According to Item 7 in the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines, authors of SRs should include all information sources (e.g., databases with dates of coverage, and the search and last search dates) [
36]. Among the 202 SRs identified, 172 (85.1%) provided the last search date, which is slightly lower than Core Clinical Journals (90.3%) [
5]. An overwhelming majority numbering 198 (98.0%) of the identified SRs stated the databases used for their search. This is comparable with the figure previously reported (98.3%) by Beller et al. [
5]. Against this background, Liberati et al. [
37] further suggested that this information would allow researchers to assess the scientific value of the review as the publication time lags may necessitate updates of SRs by the respective authors.
The quality of our included SRs was in general low with a median score of 3 out of 11. It is lower than the journals in Orthodontic (with mean scores 3.5 to 5.66) [
23] or Pain related SRs (median: 6; IQR: 4–7) [
13]. Two potential reasons underlie this low quality, the first of which concerns the lack of adherence among the authors to the PRISMA guideline in their reviews. Only 30 out of the 107 journals mandated the use of the PRISMA guideline [
20]; accordingly, there might have been inadvertent omission of information in reporting their results. For example, the independent selection of papers might have been performed but this might not be reflected in the published review. The other potential reason concerns our stringent conformance to the AMSTAR criteria in examining the methodological quality. The generally low quality of our included SRs might be attributed to some AMSTAR criteria that were unfeasibly difficult to fulfill: for example, the last item on the AMSTAR checklist – “
Was the conflict of interest included?” – requests the authors to indicate all sources of funding or support for the SRs and each of the studies included therein. The compliance to this item was often low and has indeed been reported to be approximately only 10% in the field of pain research [
13]. Some studies would remove this item in evaluating the quality of SRs [
38] whereas others would (more leniently) consider the disclosure of funding only for the authors but not for the individual studies included in the SRs [
39]. Such refinements of definitions of the AMSTAR criteria may lead to a higher quality for the included studies.
Strength and limitations
To the best of our knowledge, this is the first study focusing on the publication time lags of SRs published in nursing journals. The study sample is reasonably large with 202 SRs. Robust procedures have been adopted such as independent data extraction and evaluation, to ensure the accuracy of the results. Nevertheless, certain limitations of this study deserve attention. Firstly, only SRs published in 2014 were included as this study was conducted primarily in 2015 when 2014 was then the latest available full year. We did not include any preceding years because of limited resources but we believe that the observed pattern would be similar for 2015 or 2016. Secondly, the methodological quality of the SRs was generally low. Thirdly, non-English SRs were excluded due to language barriers of the authors; however, the time taken is expected to be similar or even longer. Fourthly, as a database search was used to identify the SRs, the possibility cannot be discounted that some articles might be missed if they had not included any words related to SR or mete-analysis. However, this method is considered acceptable. Fifthly, although the authors were contacted for additional information, only 2 questions were posted by e-mail to ensure an acceptable response rate. Further studies are recommended to explore the topic from both the journal editors’ and authors’ perspectives. Finally, given the absence of submission dates of most SRs, further analysis could not be conducted for most of them.