Systematic review and meta-analysis of ivermectin for treatment of COVID-19: evidence beyond the hype

Despite the efforts and the relative success of vaccination against the coronavirus disease 2019 (COVID-19) worldwide, it is possible that the pandemic will persist for a long period, due to the ascension of outcoming variants and anti-vaccine movements around the world [1‐3]. In this context, several drugs, alone or in combination with other drugs, vitamins and minerals were studied to verify the possibility of mitigating the effects of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) or the symptoms of covid-19 [4]. However, until the present date, few pharmacological therapies have been shown to be effective in reducing the number of hospitalizations, mechanical ventilation and death [5, 6].

Using existing medicines that are widely available and at low cost has appeal [7]. The anti-parasite drug ivermectin has grabbed a lot of attention as a potential drug to treat COVID-19, and has become widely used off label, especially in Latin America, to control COVID-19 [8, 9]. The drug has been used for decades to treat parasitic infections, and it is on the list of essential drugs from the World Health Organization (WHO), being considered safe and effective once it is administered in appropriate doses. Several in vitro studies have shown an antiviral effect of ivermectin for flaviviruses, such as Dengue, yellow fever, Zika virus and SARS-CoV-2 as well [10‐12]. In cells infected by SARS-CoV-2, ivermectin has been shown to inhibit the attachment of the virus's spike protein to the human cell membrane, which was able to reduce the viral RNA concentration by almost 5000-fold, what increased hopes for clinical benefit in prevention and treatment of COVID-19 [10, 13, 14]. Nonetheless, for the antiviral effect observed in cell culture to be effective in humans, the dose of ivermectin required would be 17 times higher than the maximum safe dosage allowed per day for patients [10].

The enthusiasm brought by the mechanistic effect has also been questioned in a recent network meta-analysis on the effects of ivermectin on viral clearance. When authors analyzed the data using a fixed effects model approach, there was a significant effect of ivermectin on reducing viral clearance (OR 2.32, 95% equal-tailed credible intervals [CrIs] 1.38–3.94); this effect was nonsignificant when they used a random effects model approach (OR 2.70, 95% CrI [1.24, 6.12]), [15] demonstrating how choosing or not the proper methods for analysis can influence the results.

Multiple clinical trials were carried out to assess clinical outcomes [16‐19], with conflicting evidence, and some of these studies were withdrawn given concerns about serious data inconsistencies and research fraud [20‐24] Yet, the drug has been widely used, and has even been called a “COVID-19 miracle drug”, supported by vaccine opponents, or even by health authorities in some countries [25]. However, there have been reports from different countries of people being hospitalized after self-medicating and developing serious adverse effects due to ivermectin [26]. The need for evidence synthesis is imperative for medical doctors and the community. Systematic reviews have been published on the topic, but in the majority of them a retracted trial represented more than 10% of the overall effect, [20, 21, 27, 28] which overestimated the benefits. One of those was recently reassessed after the exclusion of this trial, which changed the results from reducing mortality to lack of benefit in this outcome [17]. In another systematic review restricted to studies comparing the drug to placebo or standard care, the authors concluded they were uncertain about the efficacy and safety of ivermectin [29].

As the current evidence on the benefits of ivermectin to treat people with COVID-19 is still debatable and there is a risk of serious adverse events, the WHO living guideline recommends that the drug should only be used within clinical trials, and the IDSA's guideline suggests not using it [4, 30]. Therefore, there is still a gap of reliable and updated evidence synthesis of the effect of ivermectin in COVID-19 patients. This study aims to summarize and critically appraise the evidence of randomized controlled trials (RCTs) of ivermectin, assessing clinical outcomes in inpatients or outpatients with COVID-19.

This systematic review and meta-analysis was based on recommendations from the Cochrane Guidelines for Systematic Reviews of Interventions and was written according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [31, 32]. The review protocol was registered at the PROSPERO (CRD42021257471).

We conducted a comprehensive search on the impact of ivermectin for the management of patients with COVID-19 and observed that ivermectin does not have an effect in reducing mortality or mechanical ventilation in patients with COVID-19. Despite the low quality of evidence, this effect was consistent when comparing ivermectin vs. placebo, and ivermectin associated with SOC vs. SOC, as well as in sensitivity analysis. Additionally, there was very low quality of evidence of no increase in risk of adverse effects.

Despite not being recommended in current COVID-19 guidelines by WHO and IDSA [4, 30] the prevalence of self-medication during COVID-19 course was high, and ivermectin was one of the medications commonly used, as shown in a recent systematic review [66]. This may be related to self-medication, misinformation in the media, science denialism and low access to health services combined with the low cost of ivermectin, and the belief that it has a safe adverse effect profile [66, 67]. In Brazil, for example, the Ministry of Health included the medication in its COVID-19 guidelines. Up to August 2021, estimates from the Brazilian Parliamentary Commission of Inquiry showed that only one pharmaceutical company sold more than 83 million US dollars in ivermectin [68, 69].

Living systematic reviews may have changed this scenario, as they are supposed to incorporate all new relevant evidence as they become available [70]. Nonetheless, with the overwhelming number of studies published in COVID-19 pandemic area, keeping the living reviews updated is a challenge difficult to overcome. This challenge is even more complex when the living reviews propose to assess different comparisons with multiple drugs and evolving SOC. Ivermectin has been the subject of two systematic reviews. The British Medical Journal's living review was last updated not so recently, in April 2021, and suggested a possible reduction in mortality in patients who used ivermectin, when compared to standard of care (RR 0.31 95% CI 0.14–0.072). The authors highlighted the fact that data was limited by extremely few events, leading to very serious imprecision, and serious risk of bias [28]. The other living review, by the Pan American Organization, has been recently updated in December 2021, and analyzed the evidence from 14 studies. The authors reported that pooled estimates suggested mortality reduction with ivermectin (RR 0.50 95% IC 0.29–0.87), an effect that was no longer apparent when a subgroup analysis of the three studies classified as low risk of bias was performed (RR 0.96 95%CI 0.58–1.59) [71]. These two living reviews were not updated after Elgazzar et al., Samaha et al. and Pott-Junior et al. studies were retracted. We reckon that another update must also take into account the fact that studies that compared ivermectin with hydroxychloroquine may now be clinically inappropriate, and therefore should not be kept in the pooled analysis studies, as evidence of harm with the use of hydroxychloroquine is now robust [38].

Differently from our results, a recent review has found substantial differences in the results of studies with or without important methodological limitations, highlighting that important benefits associated with ivermectin were based on potentially biased results [72]. Because this review was mainly interested in investigating “bias as a source of inconsistency”, as stated in the review title, the authors included in the pooled analysis Elgazzar et al. retracted trial (and studies which were retracted after the publication of the systematic review), as well as studies in which ivermectin was compared to other drugs, such as hydroxychloroquine [64, 73]. We strongly believe the best body of evidence now available should not include such studies. Additionally, several studies were published after the publication of this review [7, 45, 52, 53].

In the present study, the certainty of the evidence on mortality and need for mechanical ventilation was ranked as low (GRADE) due serious concerns about risk of bias and imprecision. Methodological limitations were mainly due to lack of adequate blinding of patients and outcome assessors and high number of losses after randomization. Additional concerns included the fact that some studies were not pre-registered prior to enrolling patients, others had the protocol modified, and the majority did not report the funding source, although three of them were sponsored by pharmaceutical companies. However, as mortality and mechanical ventilation are hard endpoints, and our findings were negative, those sources of bias might not have had a great impact on these outcomes. Instead, it could have been influential for assessing adverse effects. In fact, studies usually assess overall adverse effects, without separating them according to severity, and this is a limitation addressing this outcome. For example, in Okumus et al., while patients who took ivermectin had serious neurological adverse effects which require drug discontinuation, the control group had only nausea, vomiting or two-fold increase in alanine transaminase. None of these side effects were severe enough to require termination of treatment in the control group [58].

Another limitation is the low event rate. Among the 22 studies that could be assessed for mortality, 11 did not have any events. COVID-19 severity varied among the different studies, but the majority of them included patients with mild to moderate disease. Therefore, mortality is expected to be very low in this context. The same applies for mechanical ventilation requirement.

One strength of the present review is to have applied strict methodological criteria, to have performed a broad search in several databases, and to be comprehensive, analyzing not only studies comparing the drug to placebo, but also those in which ivermectin associated with SOC was compared to SOC, in a stratified analysis. This is different from a recent Cochrane review, with the last search performed in May 2021, which selected only placebo-controlled studies. Only two studies were included in the pooled analysis to assess mortality and mechanical ventilation requirement [29]. As aforementioned, a recent network meta-analysis has shown how different statistical approaches (random vs. fixed) lead to different results on the effects of ivermectin on viral clearance [15]. This shows how misleading results may be when inappropriate methods are used. Random effects model is more appropriate in this context, as studies included patients with heterogeneous disease severity and the management was different among the different studies. This could be observed by the several definitions of standard of care. As we should not assume a common effect size to all studies included, and the goal of the analysis is to generalize to a range of scenarios, a random effect approach is recommended [74].

The urgent demand for treatment options for COVID-19 has created the need for randomized clinical trials. Scientists tested several approved drugs against the disease, “throwing every already-approved drug” [9], and the rush to conduct those trials led to conduct and publication of studies with varied quality and important methodological limitations. This “provided fertile ground for even poorly evidenced claims of efficacy to be amplified, both in the scientific literature and on social media” [20]. This is very problematic, as results from studies with high risk of bias were quickly widespread in clinical practice and public policy and SOC were also adopted in a rush in different countries. Even worse, different governments were reluctant to change their protocols after the evidence had shown that some drugs should not be used. In Brazil, for example, a huge polarization and politicization disseminated the SOC supported by the Brazilian President, known as “kit COVID-19” which included hydroxychloroquine, ivermectin and azithromycin [8]. Consequently, studies have included varied and clinically inappropriate options as they defined their comparators as SOC.

For example, months after evidence that hydroxychloroquine may increase the risk of death was available [75] a research protocol of Beltran-Gonzalez et al. study was registered, in Brazil [50]. With regard to antibiotics as SOC in patients with COVID without evidence of bacterial pneumonia (for example doxycycline or azithromycin), the World Health Organization advised against the practice in May 2020; still, two studies which started recruitment at that month kept antibiotics in their definition of SOC [56‐60].

There are ethical considerations in this regard, exposing patients to harm and, in the case of antibiotics, contributing to the emergence of antibiotic resistance [70], which is a major issue worldwide. In a letter to the editor about Podder et al., Meneses pointed out important ethical issues. He observed that the authors mentioned approval of their study by the director of the health center, but apparently there was methodological and ethical evaluation by an institutional board [76].

Niaee et al., which included hydroxychloroquine in the comparison arms and was the main study which has shown benefits of ivermectin in clinical outcomes after the aforementioned retractions [62], has been recently questioned. An editorial note was published in October 2021, reporting concerns about various aspects of the study, including possible problems in the randomization of participants [20]. This raises concerns that flawed evidence in studies with ivermectin or other drugs may impact in systematic reviews.

Our meta-analysis was innovative for using the Living Overview of Evidence database (L.OVE, issued by Epistemonikus) for a comprehensive search, in addition to the traditional search. L.OVE is a digital tool that compiles articles from several databases, including preprint databases, kept up to date through computational algorithms [77]. A previous analysis has shown that it may be more efficient than the traditional search [78]. Consequently, in terms of databases, our search was broader than other systematic reviews on the topic, and the tool made it easier to update the search regularly. Furthermore, we tried to minimize potential biases in the review process by following the methods recommended by the Cochrane Collaboration [31] and set out in our published PROSPERO protocol [32], which provides transparency in the review process. Additionally, we presented a summary of findings table with GRADE results and assessment, in accordance with the new standards required by Cochrane.

In a recent publication, the authors reflect that besides the retracted studies, several other studies which claim a benefit for ivermectin may be similarly fraught. They highlighted unexplainable mismatches between trial registry updates and published patient demographics and timelines that are not consistent with the veracity of the data collection [20]. Therefore, it is of utmost importance for authors of systematic reviews, before following strict methodological criteria, to keep updated with possible new study retractions. Our sensitivity analysis did not show any difference in the point estimates when individual studies were removed, so we do not expect large changes in point estimates.

Thousands of supporters, many of them anti-vaccine activists, have continued to vigorously campaign for ivermectin use, claiming that the real evidence is being ignored. In the context of misinformation infodemics, some sites have published systematic reviews with meta-analysis on the effectiveness of the use of ivermectin in COVID-19 outcomes (https://​ivmmeta.​com and Home—FLCCC|Front Line COVID-19 Critical Care Alliance (covid19criticalcare.com) Most reviews have not undergone peer review, do not show the criteria used in the selection of RCT's, do not present records and statistical criteria for evaluating the effect and heterogeneity between studies. According to Roman et al., these sites contribute to misinformation of patients, their families, the general population and health professionals who cannot critically analyze scientific studies.

We believe this transparent and thorough summary may contribute to disseminate truthful evidence. Despite the limitation in the analysis of adverse effects, previous studies list some serious adverse effects, such as toxidermias, encephalopathies, confusional disorders [79]. Associated with the lack of clinical benefit, this should be considered when managing patients with COVID-19.

The evidence suggests that ivermectin does not reduce mortality risk and the risk of mechanical ventilation requirement. Although we did not observe an increase in the risk of adverse effects, the evidence is very uncertain regarding this endpoint.