Definition of 'unpublished'
We classified trials as unpublished if detailed methods and results were not reported in either a peer-reviewed journal article [
8] or book chapter, i.e. not published in full. We considered forms of written information including abstracts, drug company reports, letters, literature reviews, and online trials registers to be less than full publications, which we term 'sources.' If it was unclear whether a trial had or had not been published from the titles of references, the trial source(s) were retrieved. We considered a study in the trials register to be unpublished if we could not match the details of study methods and results to any subsequent full publication.
Classification of trial completion status
We assigned a completion status of complete, ongoing, temporarily suspended, planned, never started or unclear to unpublished trials. We classified trials as complete if the most recent source reported results and gave no indication that these results were either interim or preliminary, or that patient recruitment or data analysis would continue. If the trial could not be classified as complete after reviewing the most recent source, we searched for the trial using the acronym or intervention on three online databases (Clinical trials.gov
http://www.clinicaltrials.gov, The Internet Stroke Center
http://www.strokecenter.org and Current Controlled Trials
http://www.controlled-trials.com. Trials were classified as complete or ongoing depending on the status described by these databases. If completion status remained unknown, we attempted to contact investigators at least twice. If investigators did not reply by 31
st December 2008, the trial completion status was classified as unclear.
One author (LG) extracted data on trial designs, methods, and results. We recorded sample size, number of centres, type of intervention, comparison groups, method of randomisation, blinding procedures, and allocation concealment for each trial. If details were unclear, trial sources were referred to a second author (PS and/or MB). We classified trials as single or multi-centre according to descriptions provided by investigators. If such descriptions were not available, we noted investigators' institutional affiliations and arbitrarily classified trials with one affiliated hospital or clinic as single centre, and trials with more than one affiliated hospital or clinic as multi-centre, accepting that in rare instances, investigators from multiple hospitals or clinics would participate but only one centre would be responsible for recruiting patients.
We extracted all available data on treatment effects on clinical outcomes (e.g. death, death or dependency and adverse effects) and on non-clinical outcomes (e.g. physiological variables or laboratory measures of uncertain clinical significance). We classified results as qualitative or quantitative and noted whether statistical significance was reported. We extracted data on the most important clinical outcome reported, such as death or functional scales, from the most recent source. In the absence of data on death or dependency, we extracted data on the studies' pre-defined primary outcome. If no primary outcome was defined, we extracted data on the first outcome described in the methods.
We classified the reported effects of interventions as beneficial, harmful, or neutral based on statistical significance, or percentages of participants having a particular outcome if statistical data were not available. We defined trials as beneficial if the intervention was reported to have had a favourable effect compared with the control treatment; as harmful if the treatment effect was adverse; and neutral if no differences were reported between intervention and control groups. Where only quantitative results were available but statistical data were not provided, we classified trials as neutral only if the results were exactly the same for each intervention group. Where results were available for different time points, we recorded only the latest time point. We classified results as 'unable to analyse' if results were not reported separately for each intervention group.
We also extracted data on the number and type of sources reporting a trial, the investigators, country of origin, funding, and date of the latest source. If an investigator appeared to have been involved in multiple completed but unpublished trials, we cross-referenced their institutional affiliation in the sources for the different trials. We contacted investigators if it was still unclear whether they had been involved in multiple trials. We used the institutional affiliation of the first investigator of the latest source to determine the country of origin of each trial. We classified trials as receiving funding from pharmaceutical companies when this was explicitly stated in a source or if any of the investigators were affiliated to a pharmaceutical company.
If any of these data could not be extracted from the most recent source, we checked earlier sources. If data were still not available, they were classified as not reported.
Calculating the prevalence of unpublished studies
We did not have the resources to fully check the eligibility of each of the published studies. We therefore estimated the proportion that would not meet all our eligibility criteria by assessing in detail a random 20% sample of the published studies. We listed published studies by date of most recent publication, and used random.org [
14] to select 20% of studies in each of the following date ranges: 1980-1984, 1985-1989, 1990-1994, 1995-1999, 2000-2004, 2005-2009, pre-1980 and unknown date. The percentage of published studies which otherwise met the inclusion criteria (which we term 'eligible published studies') was calculated and applied to the entire list of published studies. To calculate the point prevalence of unpublished studies, we divided the number of unpublished studies by the sum of the number of unpublished studies and the number of eligible published studies. We used the normal approximation method to calculate 95% confidence intervals (CI) [
15].