Randomised trials relevant to mental health conducted in low and middle-income countries: a survey

Detailed methods for this study have been published previously in this journal [6]. In brief, an earlier project searched the Cochrane Library's Central Register of Controlled Trials (CENTRAL) plus 25 biomedical bibliographic databases to identify randomised or possibly randomised trials from low and middle-income countries published in 1991, 1995 or 2000. Countries were classified using World Bank definitions, categorizing by Gross National Income per capita and by geographical region [7]. For this paper, the electronic records of these initial searches were searched again to identify studies relevant to mental health. These records included titles, keywords, and, where available, abstracts. Word roots based on Mesh headings within "Mental Health" were utilised with extra terms in Spanish, Portuguese and French. In addition, for 10% of the records not identified as being relevant to mental health by the electronic search, full text copies were checked to assess whether this was truly the case. For each study identified as potentially relevant to mental health, the author names were checked in PubMed for additional citations from the three sample years (1991, 1995 and 2000). Full text copies for all trials potentially relevant to mental health were sought.

The criteria used to assess eligibility were:

(i) Randomised or possibly randomised trials – this included randomised trials, quasi randomised studies, and controlled trials which could possibly be randomised.

(ii) Published in 1991, 1995 or 2000.

(iii) At least one participant from a low or middle-income country as stated in the text, or if not stated this was assumed if reports were from databases local to low and middle-income countries and reported in local languages.

(iv) Participants were either people who had a mental health problem, or they had been identified as being at increased risk of developing a mental health problem.

Two authors (RJSS and MJ) independently assessed full text copies of each record identified by the search. Discrepancies were resolved by discussion.

Data on study quality and content were extracted onto a specifically designed data extraction form. Each trial was extracted onto a separate data extraction sheet. Hence, if two trials were reported in the same paper, these were extracted separately. For English language reports, data were extracted by two people working independently. For studies in other languages data were extracted twice for a random sample of 10% of papers. Extraction was by someone fluent in the relevant language. In the few cases where full text was not available (n = 12), data were extracted from the title plus abstract (n = 10), or title alone if that was all that was available (n = 2).

Health conditions of participants in the trials were categorised using the broad categories of the International Classification of Diseases tenth edition (ICD-10). The interventions being evaluated were classified as drug, physical non-drug, such as Electro-Convulsive Therapy (ECT) or psychosocial. Drugs were further classified as to whether they are listed on the WHO List of Essential Medicines [8]. Outcomes were classified using a system developed for a survey of trials of schizophrenia [9].

Data were entered into Meerkat software [10] and compared. For each variable, 80% agreement was considered acceptable. If the agreement fell below this, possible reasons were investigated and data re-extracted. To assess accessibility, each citation was searched for on PubMed in August 2006 using author AND year or failing this title.

Characteristics of the trials in terms of where they were conducted, the language of report, health problems being addressed, interventions and outcomes were described. Frequencies were compared using χ² for linear trend. Topics covered by the trials were compared, by region, with the WHO revised 2002 burden of disease estimates [11].

Global burden of disease estimates for mortality and YLD (years lost to disability) are available for both economic groups (high, upper middle, lower middle and low) and broad geographic regions [12]. These estimates are further categorised by cause e.g. unipolar depression. We calculated an estimate of how trial-based research activity reflects burden of disease as a 'research: need' ratio, by dividing the number of people randomised with each category of mental health problem by the WHO's Years Lost due to Disability (YLD) for the same category. The findings were compared with each other in order to investigate which disorders and populations are the focus of a relatively high or low research activity.

Adequate allocation concealment was described for 12% of studies (21/177), and method of sequence generation was stated for 17% (30/177). For 62% of studies (109/177) there was information about blinding of the intervention; 77/109 were reported to have been double blind. One in five reports mentioned an ethics committee (34/177), and a third mentioned consent (61/177). A source of funding was mentioned for 25% of studies (45/177).

People aged 18–60 years were included in 79% of the studies (135/177). Of these, 39 also included participants who were over 60 years, and 12 also included those who were younger than 18 years. Seven studies included only people aged more than 60 years, and 22 included only people younger than 18 years. Age of the participants was not clear in 13 trials (7%). People of both sexes were recruited to three quarters of the trials (135/177). Participants were male only for 19 studies and female only for 7. Gender of participants was not clear for 25 reports (15%).

The most common health problems for participants in the studies were unipolar depression, (36/177) and schizophrenia (36/177) (Table 1). Three trials (1%) addressed prevention rather than treatment.

Most studies (80%) evaluated pharmacological interventions (Table 2); a third of these drugs (47/140) were listed by WHO as essential drugs.

Mental state was the most common outcome, reported by three quarters of studies (129/177) (Table 3). Less than half the studies reported adverse effects (78/177). For 83% of studies (147/177) at least one scale was used to rate outcome (median = 1, range = 1–6). The most frequently reported scales were Hamilton rating scale for depression (HAM-D) (24/177), and the Hamilton Anxiety Scale (HAM-A) (18/177). Nine reports did not include data on outcome.

Overall, 41% of the reports could be identified on PubMed when searched in August 2006 (73/177). The proportion identified decreased from 68% in 1991 to 34% in 2000 (χ² for linear trend 10.92, 1 df, p = 0.001). Of the 80 reports in English, 54 (68%) were identified in PubMed, compared with nine (13%) of the 71 reports in Chinese. The proportion of Chinese reports identified on PubMed decreased from (6/7) 86% in 1991 to (2/53) 3% in 2000. In total 10 (40%) of the 25 reports in other languages were identified in MEDLINE. 66% (117/177) of citations were identified on Cochrane's CENTRAL during the original search, again the proportion located dropped from 80% in 1991 to 57% in 2000.

Unipolar depressive disorders and schizophrenia have the highest burden of disease as assessed by years lost due to disability (Table 4). However, more people with drug use disorders were randomised per year lost due to disability than any other type of mental health disorder.

Most trials were small and few involved more than one country. The value of international trials has been demonstrated in cancer, heart disease and perinatal care; such studies enable large numbers to be recruited [14], ensure wide generalisability of results [15] and facilitate research capacity building [16]. Conducting large international trials in mental health would be likely to have similar advantages.

Participants in the trials reported here had a wide range of mental health problems, but the most common were depression and schizophrenia. Overall, 80% of the studies evaluated the effects of one or more drugs. This is the same level as for trials of schizophrenia overall [9], and hence may reflect a greater acceptance of quantitative evaluation for drugs as compared to other forms of therapy within mental health. The low level of community based and prevention studies highlights a major gap for future research. Such trials should be a priority.

Most studies reported using scales to assess outcomes. Nevertheless, interpretation of scales can be difficult and they are rarely used in clinical care. It would therefore be more relevant for trials to report outcome measures that have more intuitive meaning to patients, their families, and clinicians. It would seem a shame for those rare trialists in low and middle-income countries to dissipate their energies recording outcomes that are difficult to use in everyday practice.

The low level of reporting on how the study was funded is not uncommon. For example, funding source was reported for 39% to 51% of surgical trials [17, 18]. Similarly, many studies describing the quality of randomised trials do not report on whether ethics approval and consent are explicitly mentioned.

The CONSORT (Consolidated Standards of Reporting Trials) statement of 1996 [19], updated in 2001 [20] gives recommendations for how to report randomised trials using a 22 item checklist. Some journals have adopted CONSORT; although this endorsement is associated with improved reporting of randomised trials, reporting is often still not adequate [13, 21]. Considerable scope remains for improving the reporting of randomised trials. In 2003, only 22% of high impact general and internal medicine journals mentioned CONSORT in their instructions for authors, and many used ambiguous language regarding what was expected from authors [22]. Nevertheless, as funding source, ethics approval and consent are not included in the CONSORT checklist, the quality of reporting for these items may not improve unless researchers are made more aware of their relevance.

The proportion of trials from different geographic regions varied greatly; for example only three trials were identified from the Middle East or North Africa region (comprised of Algeria, Djibouti, Egypt, Iran, Iraq, Jordan, Lebanon, Libyan Arab Jamahiriya, Malta, Morocco, Occupied Palestinian Territory, Oman, Saudi Arabia, Syrian Arab Republic, Tunisia and Yemen). In this largely Arabic speaking part of the world, with a population of 300 million people, mental health literature may be not be easily accessible by searching in English, and many journals are not indexed on any database [23]. Nevertheless, despite searching African, Arab, Iranian and several Egyptian and Eastern Mediterranean biomedical databases, only three studies were identified. The religious and cultural individuality of this region, along with the presence of conflict, could make evaluative mental health research enormously difficult.

In contrast, China produced almost half (46%) of all trials in mental health, and accounted for most of the increase in annual output across the decade 1991 to 2000. Although the number of mental health trials does seem to be linked to national wealth [24], the increased trial activity in China is unlikely to be driven by income alone. With rapidly increasing numbers of trials there has been concern about the quality of randomised trial conducted in China [25]. Evidence of similar trends [26] are beginning to happen in randomised trials of traditional Chinese medicine which have increased in both quality and quality since 2000 [27].

Since 2001 it appears that China continues to be dominant in its production of mental health trials as evidenced in a recent review of mental health trials from low and middle-income countries where two thirds of trials conducted in low and middle-income countries were from China [28]. This review differs from our survey in that it covers a different time period and has a narrower range of interest concentrating on interventions for schizophrenia, depression, developmental disability and alcohol dependence.

Forty five percent of these reports were in English. 'Positive' trial reports are more likely to be published in English than in other languages [29]. Less than half of these studies from low and middle-income countries were cited on PubMed; even for reports in English one third were not available in PubMed. Research from low and middle-income countries is therefore underrepresented [30, 31]. Proportions of these trials accessible through Cochrane's CENTRAL register are somewhat better. Although CENTRAL is the most comprehensive database of randomised trials, one third of these mental health studies were not available on it.

Although the number of trials conducted in each year is increasing across time, accessibility of their citations on bibliographic databases, such as PubMed, seems to be decreasing. The time difference from publication to our assessment of accessibility was greatest for studies from 1991, nevertheless this seems unlikely to explain the decline in accessibility within PubMed over time, as by 2006 studies published in 2000 should have been indexed and available. The lack of availability on MEDLINE of trials reported in Chinese along with the large increase in trials originating in China seems an important factor. Indexing has been found to be poor with only 95 of the total 4 959 journals which are listed in MEDLINE being published in China [32]. Reports of randomised trials are not always easy to identify, and efforts by institutions such as the Cochrane Collaboration and WHO to make them more accessible should continue.

Using the crude estimates within our sample, more people with 'drug use disorders' (other than alcohol misuse) have been randomised per year-lost-though-disability in low and middle-income countries than people in the other categories of mental health problems. This may reflect the greater availability of funding for research relevant to hepatitis or HIV/AIDS, compared with other mental health problems such as schizophrenia, dementia, affective or anxiety-linked problems, alcohol related difficulties and psychological trauma.

Accessibility to trials conducted in low and middle-income countries should be improved by prospective registration of all future trials, as endorsed by the International Committee of Medical Journal Editors (ICMJE) and the World Health Organization's International Clinical Trials Registry Platform (WHO-ICTRP) [33, 34]. China, India and Sri Lanka now have national trial registries that form part of the WHO-ICTRP network of Primary Registers [34]. Registers in Latin America and Africa are also preparing to meet international requirements for trial registration [35, 36]. Prospective registration as a pre-requisite to publication is also beginning to receive local support from medical journal editors [37].

In addition to improving the accessibility of trial reports prospective registration of trials could also improve the internal validity of trials. By encouraging reporting of elements identified in the CONSORT statement in trial protocols (method of randomization, allocation concealment, blinding and attrition) trial design will be optimised during the planning of trials [38]. Trial registration is also being used in some countries to improve the planning and reporting of ethical considerations such as informed consent, funding sources and ethics committee approval [39].

We would also suggest that coordinated action is required from medical journal editors in low and middle-income countries to adhere to the ICMJE uniform requirements for submitting final reports for trials [40]. This would require details of ethics committees, informed consent and funding sources in addition to the use of CONSORT checklists and flow diagrams. Provision of clear instructions to authors regarding CONSORT and ICMJE requirements and appropriate checklists, education of peer reviewers, and journal policies that consistently preclude publication of trials that do not meet these requirements are also suggested [41].

Initiatives to provide training for ethics committees, researchers, investigators, regulators and students have commenced in some parts of the developing world as a crucial first step in building capacity for ethically sound research [42, 43]. However, local and regional regulatory frameworks and legislation are also needed to interpret international guidelines on the standards of care in the context of local constraints.

Strengths of our study are that we utilised comprehensive and extensive electronic searches of not only the Cochrane Library's CENTRAL, but also of biomedical bibliographic databases from low and middle-income countries. Full text copies were available for a high proportion of citations. Previous surveys of mental health research from low and middle-income countries have either concentrated on particular regions or have investigated only selected journals or individual databases [30, 31].

Despite the extensive searches, it is likely that we failed to identify some relevant studies. Due to time constraints, we were not able to search all the bibliographic databases which include journals from low and middle-income countries, and new databases have become available since our searches were completed. Other factors are that poor indexing within some databases may have impaired retrieval, and some journals may not be indexed or only partially indexed.