Background
Methods
Search strategy and study selection
Outcome measures
Data collection
Quality assessment of included systematic reviews
Evidence synthesis
Results
First author, publication year | Study design | Field of study | Data sources for abstracts | Data sources for full reports | Study search frame | Numbers of included abstracts/full reports | Study country of primary studies | Study sample size in primary studies |
---|---|---|---|---|---|---|---|---|
Abstracts followed by subsequent texts within the same publications (n = 6)
| ||||||||
Boutron 2010 [4] | Survey | Composite fields | Medline via PubMed | Same as abstracts | Full reports: 2006 December | 72/72 | NR | For full reports: Median 84 (range: 4 to 6848) |
Harris 2002 [18] | Survey | Psychology | Eight American Psychological Association journalsa | Same as abstracts | From years 1997 to 1998 | 400/400 | NR | NR |
Lehmen 2014b [12] | Systematic review | Spinal studies | Three spinal journalsc | Same as abstracts | From years 2001 to 2010 | 40/40 | NR | NR |
Ochodo 2013d [3] | Systematic review | Diagnostic accuracy studies | Journals with an impact factor of 4 or higher | Same as abstracts | Between January and June 2010 | 126/126 | NR | For full reports: Median 151 (range: 12 to 20,765) |
Pitkin 1999 [10] | Survey | General medicine | Five major general medical journalse and a consecutive sample of articles published in the CMAJ | Same as abstracts | For the five journals: Jul 1, 1996-Jun 30, 1997; For CMAJ: Jul 1 1996-Aug 15, 1997 | 264/264 | NR | NR |
Ward 2004 [11] | Survey | Pharmacological studies | Six pharmacy-specific journalsf | Same as abstracts | From June 2001 to May 2002 | 243/243 | NR | NR |
Abstracts in conferences and meetings (n = 11)
| ||||||||
Bhandari 2002 [2] | Survey | Orthopedics | The 1996 scientific program of the sixty-third Annual Meeting of the American Academy of Orthopaedic Surgeons | Medline and PubMed | Abstracts: 1996; Full reports: 1996-2001 | 465/159 | For full reports: Most in North America (93.1%) | For full reports: Median 49 (range: 2 to 8141) |
Davies 2002 [22] | Survey | Perinatology | The first annual Perinatal Society of Australia and New Zealand Congress in 1997 | Medline | Abstracts: 1997; Full reports: up to Oct 2000 | 172/83 | NR | NR |
Dyson 2006 [27] | Survey | Veterinary anesthesiology | Conference abstracts from 1990 to 1999 annual meetings of the American College of Veterinary Anesthesiologists | Entrez PubMed and CAB Direct | Abstracts: 1990 - 1999; Full reports: 1990 - 2006 | 283/201 | NR | NR |
Hopewell 2006 [7] | Survey | Oncology | American Society of Clinical Oncology annual conference in 1992 | The Cochrane Central Register of Controlled Trials and PubMed | Abstracts: 1992; Full reports: up to 2002 | 209/37 | NR | For full reports: Median 120 (range: 8 to 612) |
Klassen 2002 [23] | Survey | Pediatrics | The proceedings from the Society for Pediatric Research | PubMed, EMBASE, Cochrane Library, CINAHL, Web of Science, Current Contents, and HEALTHSTAR | Abstracts: 1992 - 1995; Full reports: up to July 2000 | 447/264 | NR | For full reports: Median 45 (interquartile range: 20 to 116) |
Kottachchi 2010g [19] | Systematic review | Inflammatory bowel diseases | All abstracts of Phase III randomized controlled trials in inflammatory bowel disease accepted at Digestive Disease Week | MedLine, PubMed, EMBASE, and Google Scholar | Abstracts: 1998 -2003; Full reports: 1997-2009 | 82/64 | For full reports: Europe (53%, 34/64); North America (28%, 18/64); and others | For full reports: Mean 94 (SD: 96) |
Preston 2006 [24] | Survey | Orthopedics | The annual meeting of the Orthopaedic Trauma Association | PubMed | Abstracts: 1994 - 1997; Full reports: up to 2005 | 254/137 | For full reports: Most in North America (93%) | For full reports: Mean 121 (standard deviation: 179) |
Rosmarakis 2005 [20] | Survey | Infectious diseases and microbiology | From the first session of 7 of 15 major research categories presented in the 1999 and 2000 Interscience Conference on Antimicrobial Agents and Chemotherapy | Index Medicus | Abstracts: 1999 - 2000; Full reports: from 1999 to 2004. March | 190/51 | NR | NR |
Snedeker 2010 [21] | Survey | Veterinary pre-harvest or abattoir-level interventions against foodborne pathogens | Ten conferences/meetingsh which involved presentations on pre-harvest or abattoir-level food safety | Four databases: Agricola, CAB Abstracts, Web of Knowledge, and Scholar’s Portal | From years 1995 to 2004 | 59/59 | NR | For full reports: median 5 (range 1 to 35) |
Toma 2006 [25] | Survey | Cardiology | Proceedings booklets and related Web sites for the American College of Cardiology scientific meetings (1999-2002). | PubMed, MEDLINE, EMBASE, and the Cochrane Cochrane Central Register of Controlled Trials databases | Abstracts: 1999 - 2002; Full reports: up to 2005 | 148/148 | NR | For full reports: Median 452 (interquartile range: 173 to 1715) |
Turpen 2010 [26] | Survey | Urology | Annual meetings of the American Urological Association | PubMed | Abstracts: 1999 - 2002; Full reports: up to 2007 | 126/79 | NR | NR |
First author, publication year | Definition of inconsistency between abstracts and full reports | Main findings of inconsistent reporting | Authors’ conclusions |
---|---|---|---|
Bhandari 2002 [2] | Inconsistencies including minor differences (in study title, number of authors, presentation of all outcomes, and authors’ data interpretations) and major differences (in study objective and/or hypothesis, study design, primary or any secondary outcome measure, sample size, analyses, results, and precision measures. | Major inconsistency found in designating a primary outcome measure (14%), and results for primary outcome measure (19%) | The overall abstract-reporting quality was inadequate. The use of abstracts as a routine guide to orthopedic practice requires to be reconsidered. |
Boutron 2010 [4] | In trials with primary outcome showing statistically non-significant results, the spin of reporting were (1) with a focus on statistically significant result; (2) with interpretation of non-significant results as showing equivalent or comparable treatment effectiveness; and (3) with an emphasis or claim of beneficial effect. | Spin identified in the abstracts of Results (38%) and Conclusions (58%) sections. Among the Conclusions section of abstracts, 24% focusing only on treatment effectiveness | Result reporting and interpretation in abstracts was frequently inconsistent with full reports in RCTs with non-significant findings. |
Davies 2002 [22] | Abstracts were considered discordant with full reports if abstracts reported different sample size, or different primary aims and/or conclusions. | Discordance found in primary aims (25%), conclusions (35%) and sample sizes (39%). | Considerable differences found between abstracts and full reports in perinatology |
Dyson 2006 [27] | Major differences defined as inconsistency on major results and conclusions; minor differences defined as inconsistency that would not change the overall clinical approach | Major differences existed in 7%, among which half of these inconsistencies could affect clinical action by changing the emphases of the conclusions. | Caution must be exercised in using information from conference abstracts in veterinary science |
Harris 2002 [18] | Abstract rated as deficient: 1) if it contained information or a claim that was inconsistent with the body of the article (labeled discrepancy), 2) or if information or claim was reported in the abstract but not in the article (labeled omission) | Proportion of deficient abstracts ranged from 8% to 18% across journals, with an average of 13% over the entire sample | Readers should be aware that abstract-full-report inconsistencies are not uncommon in psychology. |
Hopewell 2006 [7] | Inconsistencies defined as any differences in objectives, study designs, study quality, participants, interventions, outcomes, results, and conclusions. | 16% of abstracts differed in primary outcomes, 54% in number of participants randomized and 78% in number of participants analyzed. | Information given in oncology conference abstracts is unstable and needed to be improved. |
Klassen 2002 [23] | Differences in abstract-full-report pairs included conclusions, outcomes, effect sizes, and sample sizes. | 5% of abstracts changed the conclusions regarding treatment efficacy, 13% had different effect sizes for outcomes, 59% had different sample sizes. | Significant differences between conference abstracts and subsequent full reports were found in pediatrics research. |
Kottachchi 2010 [19] | Inconsistencies including minor (changes in number of authors and study title) and major (changes in study hypothesis/design, measurements in primary/ secondary outcomes, changes in sample size, statistical analysis, or different study results or measures of precision) inconsistency. | Minor change in number of authors (55%) and study title (70%). Major change in study design (5%), sample size (37%), primary outcome (28%), secondary outcome (31%), and conclusion (6%). | A substantial inconsistency was found when comparing abstracts with full reports in digestive diseases. |
Lehmen 2014 [12] | Abstracts considered to have a deficiency if they had data that were inconsistent with or not found in full reports, or if they did not report pertinent negative results | 75% of the abstracts had at least one 1 deficiency | A surprisingly high percentage of inconsistency between abstracts and full reports was reported in spinal RCTs. |
Ochodo 2013 [3] | Abstracts defined as overly optimistic if they chose to report the best results only, or if they reported stronger recommendations or conclusions than in the full reports | 23% of the abstracts were overly optimistic | Abstracts were frequently found to be misreported and overly optimistic in diagnostic accuracy studies. |
Pitkin 1999 [10] | Abstracts considered deficient if they reported different data from full reports, or they provided data that could not be found in full reports | Deficient abstracts varied from 18% to 68% | Even in large-circulation general medical journals, data in abstracts were commonly inconsistency with full reports. |
Preston 2006 [24] | Inconsistencies were discrepancies in study objective and/or hypothesis, study design, primary outcome measure, sample size, statistical analysis, results of primary/secondary outcomes, and conclusions. | 29% abstract-full-report pairs had at least one inconsistency. Inconsistencies found in conclusions (7%), primary outcome measures (4%), sample size (18%), results for primary (8%) and secondary (29%) outcomes. | Inconsistencies were frequently observed. Most conclusions remained unchanged. |
Rosmarakis 2005 [20] | Difference between abstracts and full reports categorized into minor or major; difference in any number by 10%, or statistically non-significant results changed to be significant (or vice versa) was considered major | Difference found in 59% pairs of abstracts and full reports, among which 77% was major difference | Significant inconsistencies were found between abstracts and full reports in infectious diseases and microbiology. |
Snedeker 2010 [21] | Difference in abstract-paper(s) match including number of authors, study objectives, pathogen(s), intervention(s), species, sample size, housing, number of bacterial outcome measures, intervention effect, and overall conclusion | One-third (32%) of matches had different results; 14% differed in the direction of intervention effect; 26% significantly differed in outcome results; 11% differed in overall conclusion on efficacy of the intervention | Abstracts may not always accurately report the same information as in full reports in the field of pre-harvest and harvest-level food safety. |
Toma 2006 [25] | Inconsistencies included differences in the study designs, purpose of trials, sources of funding, allocation concealment, sample size, results, and conclusions. | 24% of abstracts had different sample size, 41% had different treatment effect estimates. | Inconsistencies between meeting abstracts and subsequent full reports were not uncommon in cardiology. |
Turpen 2010 [26] | Inconsistencies included any differences in study design and results | 29% abstract-full-report pairs had different numbers of participants randomized, 70% had unidentifiable primary outcome. | Abstract provided inconsistent results that could not allow urologists to critically appraise study validity. |
Ward 2004 [11] | Abstracts considered deficient if they had data but not in full reports (omission), inaccurate factual information that differed from full reports, inconsistency in following the “Instructions for Authors” for respective journals, or difference in the information placement between abstracts and full reports | 61% of the abstracts had at least one deficiency. 25% had an omission; 19% had qualitative inaccuracies; 25% had quantitative inaccuracies; 5% were inconsistent with the “Instructions for Authors”; 14% had information placement difference | Improvement is needed to rectify the inconsistency of abstract reporting in pharmacy-specific journals. |
Study-validity-related factor | Number of included studies (reference numbers) | Number of abstract-full-report pairs | Main findings of inconsistent reporting |
---|---|---|---|
Research question or objective | 274 | Two studies reported high level (98% - 99%) of consistency for study objectives; One study found 10% difference in both study objectives and conclusions | |
Population or sample size | 1121 | Sample sizes in abstracts were found to be smaller (9%), be different from full reports (17% - 78%), or have insufficient information on numbers of enrolled and analyzed participants/subjects (44% - 59%). | |
Intervention or exposure | 1 ([21]) | 59 | Full reports provided different/additional pathogens and/or interventions in two abstract-full-report pairs (3%). |
Comparator | 0 | 0 | – |
Outcome measure | 647 | It was found that inconsistency existed in designating a different primary outcome (4% - 28%), outcome measures were different (59%) between abstracts and full reports, or primary outcome was not stated in abstract (70% - 77%). | |
Study duration | 1 ([20]) | 51 | Sixteen abstracts (31%) reported different study period and/or population from full reports. |
Study design | 223 | High level of consistency was found for study design (95% - 99%). | |
Statistical analysis | 1 ([2]) | 159 | Few abstracts (8%) reported the same statistical methods as in the full reports. |
Result presentation | 1131 | Results in abstract were different from full reports (13% - 41%), with a statistically significant change leading to a change of study conclusion (6% - 32%), not reporting pertinent negative (40%) and pertinent positive (90%) findings, or selectively reporting favorable results (6%). | |
Result interpretation | 456 | Result interpretation in abstracts was found to be inconsistent (4% - 15%), or overly optimistic (23%). | |
Conclusion or recommendation | 896 | Conclusions in abstracts were reported to be inconsistent (15% - 35%), or with stronger statements than in full reports (17%). |
First author, publication year | Study design | Field of study | Numbers of abstract-full-report pairs included for analyses | Factors related with inconsistent reporting | Association between factors and inconsistency |
---|---|---|---|---|---|
Bhandari 2002 [2] | Survey | Orthopedics | 159 | Time from abstract presentation to the publication of the full report | Longer time to publication of full reports significantly increased the likelihood of an inconsistency (odds ratio = 1.5 for per-month increase, p < 0.01) |
Rosmarakis 2005 [20] | Survey | Infectious diseases and microbiology | 51 | Time from abstract presentation to publication of full reports | A trend found between longer time to publication of full reports and increased inconsistency (odds ratio = 1.76 for per year of delay, p = 0.07) |
Snedeker 2010 [21] | Survey | Veterinary pre-harvest or abattoir-level interventions against foodborne pathogens | 59 | Time from abstract presentation to publication of full reports | Longer time to publication related with fewer outcome measures in full reports (than in abstracts) (p = 0.03) |