Reporting quality of randomized controlled trials in patients with HIV on antiretroviral therapy: a systematic review

To allow for correct evaluation of clinical trial results, readers require comprehensive, clear, and highly transparent information on the methodology used and the results obtained. The Consolidated Standards of Reporting Trials (CONSORT) statement [1‐3] aims to improve the scientific quality of reports of randomized controlled trials (RCT) and was developed in collaboration with clinical trial methodologists, guideline developers, knowledge translation specialists, and journal editors. This statement evaluates the quality of the reporting of clinical trial results, but not of the clinical trial itself. The first version of the CONSORT statement was developed in 1996 and was thereafter revised twice, in 2001 and 2010 [1‐3]. The current CONSORT 2010 statement is composed of a checklist of 25 topics and includes 37 items and a flow diagram. It has been recommended that the statement be used in conjunction with the CONSORT 2010 Explanation and Elaboration [4], which was developed simultaneously with the statement [4].

Systematic reviews evaluating the reporting quality of clinical trials in the field of HIV/AIDS have included various articles. Some reports assessed the quality of reporting of adverse events using the CONSORT 2001 extension for adverse events reporting [5, 6]. Another report targeted studies in which HIV interventions were performed by a pharmacist and their validity was assessed using the CONSORT 2010 statement and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement criteria [7]. A further report targeted non-inferiority RCTs [8] and yet another report evaluated the performance and reporting quality of RCTs aimed at prevention and treatment of HIV/AIDS in Africa [9]. However, to our knowledge, no study has comprehensively reviewed and evaluated the quality of reports on RCTs of antiretroviral therapy (ART), the standard therapy for HIV/AIDS using the CONSORT 2010 statement.

The CONSORT statement does not include a detailed protocol deviation item checklist. On the other hand, the report by Sweetman and Doig [10] classified protocol deviations into five items, i.e. enrollment, randomization, study intervention, patient compliance, and data collection. These items were defined as indices for the evaluation of clinical trial quality, with the purpose of examining relationships between protocol deviation reporting and the characteristics of clinical trials. Based on the results of their study, they proposed addition of the five protocol deviation items to the CONSORT statement in order to clarify the requirements for protocol deviation reporting. In the field of HIV/AIDS, compliance with medication is an important index, because it is considered important for sustained HIV suppression and reduction of the risk of drug resistance [11]. Several compliance-related studies have been reported, including a study using a device that informs HIV patients with impaired memory when to take their medication [12] and another study on compliance rate and outcomes [13‐17].

According to the 2016 fact sheet of the Joint United Nations Programme on HIV/AIDS (UNAIDS), there were 36.7 million HIV-infected individuals alive as of 2015 and 70% of those people resided in sub-Saharan Africa (19 million people in Eastern and Southern Africa and 6.5 million people in Western and Central Africa) [18]. The majority (33 of 48 countries) of sub-Saharan Africa countries are poorly developed countries [19]; this area includes many regions in which the literacy rate is low [20]. The United Nations defines “least developed country” based on the following three criteria: per capita income; human assets; and economic vulnerability [21]. It has been shown that informed consent processes require more effort in developing countries than in advanced countries [22] and it has been recommended that a local translator write the information pamphlet as well as the consent form and that informed consent should be acquired in the local language [23]. However, there has been no detailed review of the reporting on obtaining informed consent in RCTs in the field of HIV/AIDS.

Thus, this study aimed to evaluate the reporting quality of articles on RCTs of ART developed in the field of HIV/AIDS, by adding evaluation items related to protocol deviations and ethical aspects to the CONSORT 2010 checklist items.

In this study, the quality of reports on RCTs of anti-HIV therapy was evaluated comprehensively and systematically, based on the CONSORT 2010 checklist, protocol deviations, and ethical considerations, which has not been previously reported.

The quality of reported articles evaluated using the CONSORT 2010 checklist improved over time (65% in Period 1, 67% in Period 2, 79% in Period 3; p < 0.0001), although seven items had low reporting rates of < 50%, regardless of the study period (“Methods”: 6/17 items; “Other information”: 1/3 items). The six items in the “Methods” section were randomization or blinding (#9, #10, #11b), explanation of any interim analyses and stopping guidelines (#7b), and criteria changes after trial commencement (#3b, #6b). The one item in the “Other information” section was where the full trial protocol can be accessed (#24). Some journals require public posting of protocols as a condition of acceptance for publication [25]. Therefore, it is possible that articles published in the future may improve the reporting rate of item #24.

“Registration” (#23) was not reported at all in Period 1, but the reporting rates in Periods 2 and 3 were moderate (52%) and high (95%), respectively. The International Committee of Medical Journal Editors (ICMJE) requires the registration of clinical trials in a public trials registry. This abrupt increase after the second revision may involve the requirement for registration in Clinicaltrials.gov, etc., since 2000. “Registration” (#23) was added in the CONSORT 2010 revision.

Several studies have reported on compliance and outcomes of studies in the field of HIV/AIDS [13‐17] and have demonstrated the close relationship between compliance rates and outcomes. In particular, in this study, the rate after 2010 was very high, at approximately 90%. This reflects the importance of compliance in this disease field. Although both the study by Sweetman and Doig and this study showed low reporting rates (approximately 30%) in overall deviations, the rate can likely be improved by means of a data manager, as management of deviation reporting is within the scope of data management.

In terms of ethical aspects, the rates of reporting on obtaining informed consent and approval by an ethics committee or IRB were high throughout the study period, as previously reported [26]. This is likely because the included trials complied with the Declaration of Helsinki [27], adopted in 1964, and the ICH-GCP guidelines [28], completed in 1996. The rates for reporting on COIs were 40%, 52%, and 85% in Periods 1, 2, and 3, respectively, and thus increased over time. Reporting on COIs was mentioned in the uniform requirements for manuscripts submitted to biomedical journals, published in 1988 [29, 30] by the ICMJE. However, COIs were reported in > 50% of the articles in Period 2, after 2001, indicating that the revision of the COI section in 2001 [31] and the influence of 21 CFR Part 54, described in 1999 [32], were likely to have facilitated reporting of COIs.

This study has several limitations. Although PubMed is one of the largest medical publication databases, we did not evaluate reports that were not indexed in PubMed. Also, we searched for keywords in the Title/Abstracts only and did not search the literature by using relevant subject headings. The second limitation regards scoring. It was simple to score 0 or 1 for about half of the items (17/37 items), but some items could not be determined dichotomously (0 or 1), so we chose intermediate score levels (0.25, 0.5, and 0.75) for some items. This step was subjective in nature. Therefore, we provided a comprehensive description of item scoring in the “Methods” section. We defined the method to evaluate items such as “important changes to methods after trial commencement (such as eligibility criteria), with reasons (#3b)” and “any changes to trial outcomes after the trial commenced, with reasons (#6b)”. But these items are difficult to assess unless the clinical protocol was reviewed. The third limitation is that there is no consensus if it is important to report ethical items for HIV/AIDS RCT articles. We found that earlier reports stating that informed consent required additional effort, due to low literacy rates and differences in language in developing countries [22, 23]. However, we did not find trials involving participants exclusively from sub-Saharan Africa, so we could not assess the difference of reporting rates of ethical items between studies in developing and developed countries. Articles on trials targeting multiple regions accounted for 85% of the individuals included in this study. Although it is likely that, in such cases, informed consent forms were prepared in the language of the respective region, we could not confirm whether there were any problems related to language, as the articles did not report on language issues. The final limitation is that we were unable to find out when the author guidelines of each journal requested CONSORT compliance, so we evaluated all articles consistently under the CONSORT 2010 guideline. There was a possibility of including the journals that did not require authors to follow the CONSORT guidelines.

In conclusion, the representative RCT articles in the field of HIV/AIDS examined in this study, had reporting rates for items defined by CONSORT of approximately 70%; these rates additionally improved over time. However, the reporting rate was still low for some items related to randomization or blinding and for the description of changes after the trial commenced. Additionally, protocol deviations were not comprehensively reported.