It has been estimated that about 45% of abstracts that are accepted for presentation at biomedical meetings will subsequently be published in full. The acceptance of abstracts at meetings and their fate after initial rejection are less well understood. We set out to estimate the proportion of abstracts submitted to meetings that are eventually published as full reports, and to explore factors that are associated with meeting acceptance and successful publication.
Methods
Studies analysing acceptance of abstracts at biomedical meetings or their subsequent full publication were searched in MEDLINE, OLDMEDLINE, EMBASE, Cochrane Library, CINAHL, BIOSIS, Science Citation Index Expanded, and by hand searching of bibliographies and proceedings. We estimated rates of abstract acceptance and of subsequent full publication, and identified abstract and meeting characteristics associated with acceptance and publication, using logistic regression analysis, survival-type analysis, and meta-analysis.
Results
Analysed meetings were held between 1957 and 1999. Of 14945 abstracts that were submitted to 43 meetings, 46% were accepted. The rate of full publication was studied with 19123 abstracts that were presented at 234 meetings. Using survival-type analysis, we estimated that 27% were published after two, 41% after four, and 44% after six years. Of 2412 abstracts that were rejected at 24 meetings, 27% were published despite rejection. Factors associated with both abstract acceptance and subsequent publication were basic science and positive study outcome. Large meetings and those held outside the US were more likely to accept abstracts. Abstracts were more likely to be published subsequently if presented either orally, at small meetings, or at a US meeting. Abstract acceptance itself was strongly associated with full publication.
Conclusions
About one third of abstracts submitted to biomedical meetings were published as full reports. Acceptance at meetings and publication were associated with specific characteristics of abstracts and meetings.
The online version of this article (doi:10.1186/1471-2288-3-12) contains supplementary material, which is available to authorized users.
Competing interests
None declared.
Authors' Contributions
Erik von Elm initiated and designed the project, searched, extracted, and analysed the data, and is study guarantor. Michael C Costanza provided guidance on the statistical analyses and their interpretation. Bernhard Walder cross-checked data. Martin R Tramèr initiated and designed the project, cross-checked data, and is study guarantor. All authors participated in discussing the results and in writing the paper.
Background
Dissemination of results from research is a key feature of biomedical science. Over 40 years ago, the filtering process from the stage of designing a study until publication of the results was likened to a biological metabolism [1]. Abstracts that are presented at biomedical meetings often provide the first evidence of a research activity. Since abstracts are usually published in proceeding volumes or in journal supplements, they can be followed up until successful publication as full reports. Thus, through follow-up of abstracts from meeting submission to subsequent full publication, some aspects of this "metabolism" [1] can be studied. Systematic review of reports that focused on the follow-up of meeting abstracts identified some specific characteristics of the abstracts that were associated with successful publication [2, 3]. The average publication rate of abstracts was shown to be about 45% only, suggesting that more than half were not fully published after presentation at the meetings [2, 3]. Despite this important finding, some elements of the selection process of scientific data remained unclear. For instance, it is not well understood what happens to abstracts submitted for presentation at meetings. The rate of meeting acceptance has not been subject to systematic review, and factors that determine acceptance of abstracts at meetings have not been systematically investigated. Finally, the rate of abstracts that are rejected at meetings but still published is unclear. This additional knowledge would improve our understanding on the "metabolism" of scientific data. Our study focused on five objectives. First, to estimate, based on published data, the acceptance rate of abstracts submitted to biomedical meetings. Second, to better describe the time-course of subsequent full publication of abstracts. Third, to identify abstract and meeting characteristics that are associated with abstract acceptance at meetings and their subsequent publication. Fourth, to investigate the fate of rejected abstracts. And fifth, to estimate the rate of abstracts that are submitted to biomedical meetings and, whether accepted at the meetings or not, subsequently published in full.
Methods
Dissemination of biomedical information
We defined three flows of abstracts (Figure 1). Flow 1 described abstracts that were submitted to a meeting and accepted for presentation. Flow 2 described accepted abstracts that were published subsequently as full reports. Flow 3 described abstracts rejected by a meeting that were published subsequently despite rejection. Abstracts were considered as unpublished if they could not be retrieved as full reports, including papers that were still in the publishing process (in press), not easily accessible (grey literature), or not published at all (file drawer).
Figure 1
Flows of abstracts from submission to a meeting until full publication.
×
Anzeige
Systematic search
We searched in MEDLINE, OLDMEDLINE, EMBASE, Cochrane Library, CINAHL, and BIOSIS for studies that provided data on Flows 1 to 3. Searches were conducted until March 31, 2001 without restriction for language or publication format. The search strategy included multiple free text keywords and combinations thereof: abstract, summary report, poster, poster presentation, oral presentation, podium presentation, presented, meeting, congress, conference, convention, submission, submitted, acceptance, accepted, acceptance rate, rejection, rejected, publication, published, publication rate, and publication pattern. Fate was searched as a title word. Citations of previously published similar analysis [3] were searched using Science Citation Index Expanded [4]. Bibliographies of retrieved articles, and the proceedings of the first three International Congresses on Peer Review in Biomedical Publication were checked. Authors were contacted to clarify ambiguity in the reports and to provide supplementary data if necessary. Internet searches were conducted for additional information on the meetings.
Inclusion / exclusion criteria
We included reports on the acceptance of abstracts submitted to biomedical meetings and follow-up studies on the full publication of abstracts submitted to or accepted at biomedical meetings. Only studies on individual and identifiable biomedical meetings were included; studies on large thematic collections of abstracts from a variety of meetings were excluded. We checked whether there was overlap in the included meetings. Reports on meetings in social science including psychology were excluded.
Reported details on follow-up period, the searched databases and other sources, and the procedure used to match an abstract with its corresponding full publication were identified as indicators of methodological quality of follow-up studies. For inclusion, these key components had to be described adequately.
Data extraction
For each identifiable meeting we extracted information on the most frequently reported meeting characteristics; these were year, country, size, and speciality of the meeting. We hypothesised that they may be associated with meeting acceptance and full publication of abstracts. Size was defined as the number of abstracts presented at the meeting. Data from related specialities (for instance, anaesthesia and emergency medicine) were combined. When a report provided information on more than one meeting, data were extracted separately for each meeting.
Anzeige
We also extracted dichotomous data on abstract characteristics that were potentially associated with acceptance or subsequent publication, for instance, study outcome (positive / negative), topic of research (basic / clinical), and type of presentation (oral / poster). Definitions of these characteristics were taken as provided in the original reports.
Two investigators extracted all relevant data independently; discrepancies were resolved by discussion with the other investigators.
Statistical analyses
Acceptance rate was defined as ratio of number of accepted abstracts to number of submitted abstracts. Multiple logistic regression was used to investigate associations between abstract acceptance and meeting characteristics. Results were expressed as adjusted odds ratios (OR) with 95% confidence intervals (95%CI).
Publication rate was defined as ratio of number of subsequently published abstracts to total number of accepted abstracts. The delays until full publication were used as time-to-event data for survival-type analyses. Full publication was defined as the event; abstracts that remained unpublished were censored at time of last follow-up. Results were plotted as Kaplan-Meier (product limit) curves. A follow-up interval was defined as the delay between time points at which publication rates were reported during the follow-up period. For instance, if publication rates were reported for each year elapsed since the date of the meeting, the follow-up interval was one year. The average follow-up interval was calculated for each included meeting. We assumed that publication occurred always at the end of the follow-up intervals. Since average follow-up intervals of meetings varied, we performed sensitivity analyses using different maximum lengths of average follow-up intervals. Differences in subsequent full publication according to meeting characteristics were investigated using logrank tests.
Data on abstract characteristics were combined using the Mantel-Haenszel method and a fixed-effect model. Results were expressed as odds ratios (OR) with 95% confidence intervals (95%CI).
Results
Retrieved and analysed studies
We retrieved 84 potentially relevant reports. We excluded 14: three were not about biomedical meetings [5‐7], six were on collections of abstracts across meetings that were not identifiable individually [8‐13]; and in five, relevant data could not be extracted [14‐18]. We analysed data from 64 studies published in 70 reports [1, 3, 19‐86] (see Additional file 1). Four of those studies were published in two full reports each and with partial overlap of the data [1, 45, 46, 50, 51, 62, 74, 84]. Another study was published in three abstracts with substantial redundancy of the reported data [79‐81]. All redundant data were disregarded.
Flow 1 – From submission to acceptance
Twenty-one studies provided information on Flow 1 (Additional file 1, Figure 1). Of two studies reporting on the same meeting [21, 79], only one was included [21]. The 20 remaining studies reported on 43 biomedical meetings held between 1976 and 1999 (median, 1990). Most meetings (79%) were in paediatrics (37%), internal medicine (28%), or surgery (14%). Of 14945 submitted abstracts, 6815 (46%) were accepted. The lowest acceptance rate was 25% [78], and the highest was 86% [23]. Abstracts were more likely to be accepted when the meeting was held after 1990, when it took place outside the US, or when it was larger than the median size (89 abstracts) of the meetings (Table 1). Abstracts submitted to an internal medicine or paediatric meeting were accepted more often than those submitted to a surgical meeting. Abstract characteristics that were associated with acceptance at meetings were basic science and positive outcome (Table 2).
Table 1
Flow 1, from submission to acceptance. Meeting characteristics associated with abstract acceptance
Meeting characteristic
Number of abstracts
Odds ratio*
95% conf. interval
Year (median: 1990)
1990 or before
8828
1.00
After 1990
6117
1.55
1.43–1.68
Country
US
12540
1.00
Other§
2405
1.50
1.33–1.69
Size (median: 89 abstracts)
Smaller than median
2764
1.00
Larger than median
12181
1.35
1.22–1.49
Speciality†
Surgery
1594
1.00
Paediatrics
4634
1.19
1.02–1.39
Internal medicine
4182
1.50
1.30–1.72
*Odds ratio > 1 indicates a higher likelihood of acceptance compared with the reference group. §Canada, Germany, UK †Only specialities with 5 or more meetings are included
Table 2
Flow 1, from submission to acceptance. Meta-analyses of abstract characteristics associated with abstract acceptance.
Flow 2 – From acceptance to subsequent publication
Fifty studies provided information on Flow 2 (Additional file 1, Figure 1). They reported on 19123 abstracts presented at 234 biomedical meetings held between 1957 and 1998 (median, 1992). The majority (53%) of meeting specialities were surgery (25%), internal medicine (13%), anaesthesia/emergency medicine (9%), and paediatrics (6%); the remaining 47% comprised a broad range of 27 different specialities.
Anzeige
There were large variations in both follow-up periods (range, 8 months [73] to 300 months [39]) and publication rates (range, 0%[53] to 82% [53]). Numerous studies reported on only one publication rate resulting in one long follow-up interval. In sensitivity analyses, the shape of the Kaplan-Meier curve differed depending on the maximum length of the average follow-up interval (Figure 2). Average follow-up intervals ≤ 1 year were provided for a subgroup of 6383 abstracts or 33% of all abstracts. For those, the publication rate after one year was 12% (95%CI 11–13%), after two years was 27% (26–29%), after three years was 37% (36–38%), after four years was 41% (40–43%), and after six years was 44% (43–46%). The shape of the Kaplan-Meier curve suggested a plateau at about 45%; however, there was little data for follow-up periods longer than six years. Including data from studies with an average follow-up interval of up to two years did not change the shape of the curve. When studies with an average follow-up interval of up to four years or all studies were included, the analyses tended to yield lower publication rates until the fourth to sixth year, and higher publication rates thereafter (Figure 2).
Figure 2
Flow 2, from abstract acceptance to subsequent full publication. Combined (Kaplan-Meier) publication rates with sensitivity analyses.
×
To investigate meeting characteristics, only the subgroup of studies with the most detailed follow-up intervals (average ≤ 1 year) was analysed. Meetings smaller than the median size (109 abstracts), and US meetings (as compared to all other countries) were associated with an increased likelihood of subsequent publication (Figure 3). Abstracts from paediatric meetings tended to have a higher publication rate than those from surgery meetings which, in turn, were published more often than those from anaesthesia/emergency medicine meetings. For meetings held after 1992 (median) versus before there was no clear difference between Kaplan-Meier curves. Abstract characteristics that were associated with subsequent publication were basic science, positive outcome, and oral presentation (Table 3). Study sample size, investigated in two reports, had no impact on rates of subsequent publication [3, 43].
Figure 3
Flow 2, from abstract acceptance to subsequent publication. Meeting characteristics associated with full publication.
×
Table 3
Flow 2, from abstract acceptance to subsequent publication. Meta-analyses of meeting characteristics associated with full publication
Abstract characteristic
Fully published / accepted abstracts
Fully published / accepted abstracts
Odds ratio (95% conf. interval)
Topic of research [44, 48, 51, 52, 54, 60] Basic vs clinical
239/401 (60%)
1218/2991 (41%)
2.29 (1.75–2.98)
Outcome of studies [3, 43, 52, 54, 63, 75] Positive vs negative
Flow 3 – Full publication despite meeting rejection
Nine studies provided information on Flow 3, with eight of those reporting on analysable data on the fate of both accepted and rejected abstracts (Additional file 1, Figure 1). These eight studies reported on 4414 abstracts submitted to 24 biomedical meetings held between 1976 and 1995. Most meetings were in paediatrics (50%), internal medicine (25%), or surgery (17%). Follow-up periods ranged from 36 months [3] to 88 months [77]. Full publications were identified for 1042 (52%) of 2002 accepted abstracts, and for 641 (27%) of 2412 rejected abstracts, respectively. Abstract acceptance was strongly associated with full publication (OR 2.95, 95%CI 2.59–3.36).
In four studies, authors were queried on the reasons why their manuscript had not been published [3, 75, 82, 84]. Of 549 valid responses, lack of time was the most cited reason (31%), followed by low priority (21%), prior rejection (10%), problems with co-authors (9%), anticipated rejection (8%), and negative study results (3%).
Anzeige
How many abstracts get published?
The number of submitted abstracts (accepted or not) that were eventually published in full could be estimated in two different ways. First, in 11 studies (39 meetings), abstracts were followed up from submission to full publication [73‐79, 82, 83, 85, 86]. Follow-up times ranged from 8 to 127 months. Of 7672 abstracts that were submitted to biomedical meetings, 2415 (31%) were eventually published in full.
The second estimation yielded a very similar number. Of all submitted abstracts, about 46% were accepted for presentation, while 54% were rejected (Flow 1). Of the accepted abstracts, 44% (20% of all submissions) were subsequently published as full reports within six years (Flow 2). Of the rejected abstracts, 27% (15% of all submissions) were published as full reports despite rejection (Flow 3). Taken together, of all submitted abstracts, 35% were eventually published.
Discussion
About one third of abstracts submitted to biomedical meetings is eventually published as full reports. This estimation was derived from two different methodological approaches and both gave very similar results. It is based on the assumption that the analysed abstracts are representative samples of all abstracts. Also, for rejected abstracts, the time-course until publication remained obscure. However, the estimate implies that about 2/3 of all submitted biomedical research abstracts will not get published. Selective underreporting of research may be seen as a form of scientific misconduct [87], and it may result in publication bias. It is important, therefore, to understand why some abstracts are accepted at meetings and others are not, and why some of the accepted abstracts are published subsequently and others are not.
Ideally, all published full reports are based on data from sound scientific studies. Accordingly, peer review of meetings and journals may be regarded as a putatively laudable mechanism to filter out invalid studies. Abstract acceptance and oral presentation were associated with subsequent full publication. One might hypothesise that only the most valid studies would be selected for presentation at the meetings, and that among those only the best would be chosen for oral presentation and for subsequent full publication. Unfortunately, there is no reliable evidence to support this view. One report found prospective study design, affiliation with a university, and randomisation to be associated with abstract acceptance [76]. Another report showed an association between the originality of a study and abstract acceptance [74]. However, in the same report, a quality score was not significantly associated with abstract acceptance [74].
Anzeige
We identified five factors that possibly play a role when abstracts are filtered on their way from meeting submission to subsequent publication. First, abstracts that reported on a positive study outcome were more likely to be published subsequently, as already shown in a previous analysis [2, 3]. Positive outcome was defined differently in the included studies; we decided to combine these data despite diverging definitions. When authors were asked why their study had remained unpublished, less than 5% indicated that a negative study result was the reason [82, 84]. Low priority, anticipated rejection, or lack of time were more often cited, but these reasons may be related to negative study outcome [82, 84]. When investigators were surveyed for reasons of non-completion or non-submission of work initially presented at a meeting, similar answers were found [88]. Our analysis provides evidence that positive outcome bias is likely to occur already at an earlier stage, i.e. at abstract submission. Peers who select abstracts submitted to biomedical meetings might favour studies that report on positive results. The authors themselves may be not only less enthusiastic about publication of negative study results [89], but also about their prior submission to scientific meetings.
Second, abstracts that reported on basic (as opposed to clinical) research were more likely to be accepted at biomedical meetings and to be published subsequently. Differences in the quality of conducting and reporting of basic and clinical research may explain this finding. However, in both domains, studies that investigated the relationship between the quality of abstracts and their likelihood of publication came to contradictory conclusions [22, 90]. Whether the abstracts included in the analysed follow-up studies were representative for basic or clinical research could not be assessed. It is conceivable that, in basic research, only the most important findings were submitted for presentation and later published. When chairpersons and senior research advisors were asked to rate robustness and quality of research projects performed in their departments, they consistently rated basic research highest [91]. Assuming a significant involvement of these research leaders in the reviewing of meeting abstracts and journal manuscripts, this implies the existence of a bias in favour of basic research.
Third, meetings comprising a larger number of abstracts had higher acceptance rates. But abstracts presented at smaller meetings were more likely to be published subsequently. Since meeting organisers often wish to attract the maximum number of attendees, a less rigorous selection of abstracts may result for larger meetings. At smaller meetings the peer review process may be more stringent, leading to selection of higher-quality papers that, in turn, would be more likely to be published eventually. However, numerous factors determine the size of a meeting and make the relationship with the acceptance of abstracts difficult to interpret.
Fourth, abstracts submitted to US meetings were accepted less often, but when accepted, were more likely to be published subsequently. This suggests that filtering mechanisms at US meetings are different from those at non-US meetings, but once again other underlying factors may play a role.
Fifth, the most frequently investigated specialities had different rates of acceptance and publication. Internal medicine and paediatrics had higher abstract acceptance rates than surgery. Paediatrics and surgery meetings had higher full publication rates than anaesthesia/emergency medicine meetings.
The analysed reports did not allow to explain these disparities.
Choosing an appropriate statistical approach for combining the time-to-event data presented a methodological challenge since publication rates were reported in widely varying average follow-up intervals. In previous analyses a single publication rate after the median follow-up time was calculated [2, 3]. We estimated the time-course of publication rates using the techniques of survival analysis. Then we performed sensitivity analyses by excluding data from reports with less detailed follow-up intervals. Based on reports with the most detailed follow-up (intervals ≤ 1 year), the publication rates increased to about 44% at six years after meeting presentation. When data from reports with longer follow-up intervals were included in the analyses, estimates differed. Generally, publication rates were lower for observation periods up to six years, and were higher for those longer than six years. Thus, for this type of combined time-to-event analyses, there may be an argument to only include data from studies with short follow-up intervals, i.e. more detailed follow-up.
Often, acceptance rates were reported as a secondary outcome of studies that investigated the functioning of scientific meeting committees. Therefore, we may have missed other relevant studies, and the retrieved studies may not be representative. Also, the methodology of included studies varied with regard to the length of follow-up after a meeting, follow-up intervals, type and number of data sources, and criteria of matching between abstracts and subsequent full reports. In an attempt to take this heterogeneity into account, we conducted sensitivity analyses with subgroups of studies with different maximum follow-up intervals. This resulted in different Kaplan-Meier curves.
We decided pre hoc to exclude studies on collections of abstracts that originated from a variety of meetings, since they did not provide information on the individual meetings. Two studies with good methodological quality were among those [8, 9]. The publication rate of abstracts from the Oxford Database of Perinatal Trials was reported to be 39% after at least four years of follow-up [8]. For abstracts collected by the Cochrane Cystic Fibrosis Group, the publication rate after five years was 40% [9].
There are still unresolved questions, and these could be addressed in further studies. For instance, we did not study what happens to scientific data before abstract submission, nor did we look at alternative pathways of dissemination. An unknown number of manuscripts are submitted to journals directly and arrive at full publication without former presentation at meetings. Also, criteria applied by peers to select abstracts for meeting presentation are not well understood. Finally, we could not draw any conclusions on the relationship between study quality and the likelihood of abstract acceptance and subsequent full publication.
Conclusions
Of abstracts that are submitted to biomedical meetings, only about one third is subsequently published in full. Both meeting acceptance and full publication are associated with specific characteristics of the abstracts themselves and of the meetings. Our study confirms knowledge from previous similar analyses that concentrated on the flow of scientific data from presentation at a meeting to subsequent full publication [2, 3]. It integrates this previous knowledge with additional relevant information, i.e. the rate of abstract acceptance at meetings, and the fate of rejected abstracts. Thus, our analysis adds to an improved understanding of how scientific data are disseminated.
Acknowledgements
Dr Tramèr is the recipient of a PROSPER grant (N° 3233-051939.97/2) from the Swiss National Science Foundation. Dr von Elm's salary was provided by the Swiss National Science Foundation (N° 3200-052199.97/1 and 3200-064800.01/1). We thank the numerous authors who responded to our enquiry. We are grateful to Daniel Haake from the Library of the Geneva Medical School for his help in searching electronic databases, and to Greta Poglia who assisted in extracting and cross-checking data. Preliminary results of this study were presented at the 4th International Congress on Peer Review in Biomedical Publication, Barcelona/Spain, September 14–16, 2001.
Competing interests
None declared.
Authors' Contributions
Erik von Elm initiated and designed the project, searched, extracted, and analysed the data, and is study guarantor. Michael C Costanza provided guidance on the statistical analyses and their interpretation. Bernhard Walder cross-checked data. Martin R Tramèr initiated and designed the project, cross-checked data, and is study guarantor. All authors participated in discussing the results and in writing the paper.
