Evaluation of confounding in epidemiologic studies assessing alcohol consumption on the risk of ischemic heart disease

Over the past few years, there have been a growing number of studies outlining both harmful and potentially protective effects of alcohol consumption on the risk of various health-related outcomes, including ischemic heart disease [1‐7]. Many of these studies, which have gained widespread attention in the media [8, 9], have the potential to influence consumer behavior and can create uncertainty and false notions regarding healthy or unhealthy practices [10, 11]. The burden of disease from alcohol is undoubtedly high [8], but there is less clarity about estimates of risk (or even protection) with low levels of consumption [6‐8, 12, 13]. Typically, the reported associations between alcohol and health-related outcomes come from observational studies, which have inherent methodological limitations that generate bias and confounding [14, 15].

Confounding is the bias resulting from the presence of common causes of exposures and outcomes [14, 16, 17]; thus confounders can distort observed exposure-outcome associations [14, 18, 19]. Although there are numerous techniques that can be used to account for confounding in observational research, it is very challenging to completely exclude the impact of unmeasured residual confounding [17]. Furthermore, many potential confounders may be unknown to researchers and can be difficult to identify or measure. Among different investigators studying the same exposures and outcomes, there may be a lack of consensus about potential confounding variables that should be considered as matching (i.e. the selection of comparators or comparison groups with respect to one or more potential confounders), adjustment (i.e. the inclusion of potential confounders in multivariate analyses), or stratification variables (i.e. the fixing of levels of confounders by producing groups (strata) within which confounders do not vary and evaluating associations within stratum of the confounder(s)). As a result, individual studies may evaluate different confounders and/or report on certain subsets of a larger pool of potential variables in published articles.

Recently, the Global Burden or Disease (GBD) 2016 Alcohol Collaborators published a meticulous systematic analysis of alcohol burden across the world, which included separate meta-analyses for 23 health outcomes [8]. Ischemic heart disease was the only outcome with significant evidence for a J-shaped curve [8], supporting previous claims that lower-volume alcohol intake may be associated with no harm or even protective effects [9, 20, 21]. However, all of these results come from observational studies, where confounders may contribute to both favorable or unfavorable associations [22]. For instance, it has been proposed that some or all of the U-or-J-shaped dose-response trends may be attributable to unmeasurable characteristics that are associated with alcohol consumption and cardiovascular outcomes [13, 22]. While some meta-analyses have suggested that adjusting for common confounders, such as smoking, age, and sex, does not alter the observed effect estimates [9], others claim that individual studies with adjusted effect estimates have lower (attenuated) protective effects [20].

Concerns have been raised that confounding factors may affect the results obtained for the alcohol-ischemic heart disease relationship, as well as their consistency and reproducibility across different studies [21]. Therefore, we systematically assessed whether individual observational studies evaluating the impact of alcohol on ischemic heart disease considered the same, similar, or different confounders and how much heterogeneity existed on how these confounders were defined and operationalized in matching, stratifying, or adjusting for them in the analyses. Additionally, we examined how authors of the individual studies considered confounding bias when interpreting their findings.

Our analysis suggests that there is substantial variation in how adjustment, stratification, and matching confounders are defined, operationalized, and discussed across observational studies evaluating the impact of alcohol consumption on the risk of ischemic heart disease. While the majority of articles accounted for smoking, age, and BMI, these variables were rarely measured the exactly same way, and no two models considered the same higher-level confounder domains. Two-thirds of the articles specifically mentioned confounding bias in their Abstract and/or Discussion sections, but less than 2% claimed that their main findings were likely to be affected by residual confounding. Very few articles called for cautious interpretation due to confounding. Given the lack of standardized approaches for selecting and adjusting for confounders, and the inadequate discussions regarding the importance of confounding, individual findings from observational studies assessing the impact of alcohol consumption on ischemic heart disease may need to be interpreted with caution.

Most of the largest multivariate models in observational studies evaluating the impact of alcohol on the risk of ischemic heart disease accounted for smoking, age, BMI, and physical activity. Age and smoking are two of the most important risk factors for ischemic heart disease, and studies have regularly found that these variables confound the alcohol-ischemic heart disease relationship [21]. Although evidence on the influence of physical activity and BMI are sparse [21], a previous evaluation suggested that low BMI and leisure-time physical activity are more common among never-drinkers than among light drinkers [28]. However, the authors noted that the differences were unlikely to be large enough to explain the lower risk observed among light drinkers compared to abstainers [28]. Numerous studies have also indicated that drinking pattern and type of beverage are important confounders [29]. However, we only identified three studies evaluating drinking history in their largest multivariate models. Furthermore, other proposed potential confounders, including cognitive function, dietary habits, and socioeconomic status [12, 13, 30], were rarely evaluated. Overall, it is unclear whether the lack of consistency across articles reflects the fact that there is little consensus about which variables are potential confounders or the beliefs that the protective effect of alcohol consumption on the risk for cardiovascular disease is independent of how well studies control for confounding [13, 29, 31, 32].

We also found that articles rarely measured similar adjustment, matching, or stratification variables the same way. For instance, although standard categories for BMI have been proposed in the literature, only one-third of the categorical BMI variables in our sample had the same cut-off levels as at least one other study. These findings build upon previous concerns that it is often difficult to determine how categorical or continuous adjustment variables are treated in analyses [33, 34]. Moreover, different treatment of variables, including incorrect adjustment for continuous confounders, can have an impact on the observed estimates or result in residual confounding [35].

There are a number of reasons that could explain the variability in the adjustment, stratification, and matching variables. Although it is possible that authors may not be able to measure all potential confounders, and therefore are prevented from considering them in multivariate models, it is more likely that there is a lack of consensus about what should be considered as matching, adjustment, or stratification variables. Furthermore, researchers may not be reviewing previously published models to determine potentially important confounders. Different studies may have different rigor in measuring some variables, and this can affect whether investigators want to use these variables in their analyses. It is also possible that certain models are preferentially reported or excluded due to biases and potential conflicts of interest, especially if the unreported multivariate models resulted in less desirable results [27]. However, the optimal choice of covariates may be difficult to identify and consensus may be elusive even with the best intentions. While adjusting for a large number of potential confounders is often appropriate and necessary, it can be particularly challenging to differentiate between potential confounders and variables that may be in the path that explains the effect of a risk factor, which should not be adjusted for. Field-wide systematic exposure assessments may help standardize variable adjustments and identify the full range of potential effect estimates due to different modeling considerations (i.e., vibration of effects) [36]. Small changes in modelling choices, including exposure and outcome definitions, covariates considered, and statistical methods, can have a major impact on effect estimates observed in observational studies, and can even flip the direction of effects [36]. Furthermore, greater transparency when it comes to the choice, measurement, and impact of potential confounding variables is necessary. Without these efforts, the associations reported in observational studies of alcohol consumption on ischemic heart disease may need to be interpreted with great caution.

Our findings, which suggest that meta-analyses of observational studies evaluating the impact of alcohol consumption on the risk of ischemic heart disease are unlikely to identify effect estimates that have been adjusted for the same variables, are generalizable to other fields. When it comes to performing meta-analyses of observational studies, there are no clear rules regarding the prioritization of adjusted or unadjusted effect estimates. According to the Cochrane Handbook for Systematic Reviews of Interventions, review authors should record both adjusted and unadjusted effect estimates, but “no general recommendation can be made for the selection of which adjustment estimate is preferable” [37]. Instead, review authors are advised to consider the estimates from the models adjusted for the maximum number of covariates, the estimates from the primary models, or the estimates from the models with the largest number of confounders that are identified as important. Previous evaluations suggests that meta-analyses of observational studies evaluating the relationships between type 2 diabetes and cancer and environmental risk factors and dementia often only identify effect estimates from models that consider age and sex, despite a large number of other measurable confounders [38, 39]. Other assessments indicate a lack of consistency among the adjustment variables considered across individuals studies included in meta-analyses [40‐42]. To facilitate the inclusion of effect estimates in meta-analyses, the raw data of individual studies should be made available to review authors in order to generate effect estimates across studies using the same or similar confounders. However, raw data are currently rarely available for observational studies [23, 25]. Even if they were available, it is likely that different datasets may vary substantially in what variables they have recorded.

We found that most authors mentioned the concept of confounding, which is consistent with prior evaluations of considerations of confounding in epidemiological studies [26]. However, we found that authors rarely explicitly state that their main findings should be interpreted with caution due to confounding. This is significantly lower than what has been previously observed among surveyed samples of high-impact observational studies and research focusing on medical interventions [34, 43]. Moving forward, more transparent reporting and discussions regarding the selection of confounders, including the potential impact of residual confounding, are necessary to ensure that observational associations on the relationship between alcohol consumption and ischemic heart disease can be properly interpreted.

This study has a number of potential limitations. First, our sample included 87 observational studies identified by a previous meta-analysis. Therefore, some articles may have been missed and the results may not be generalizable to all observational studies focusing on alcohol-health related outcomes. Although there may have been multiple reports of the same study, different authors evaluating the same data sources could have considered the same or different variables. Second, our study includes articles published between 1981 and 2015, and reporting practices may or may not have changed over time [43]. Our post-hoc analysis suggests that articles published after 2010 were more likely to mention confounding and biases. While these findings may suggest improvements over time, our sample was not designed to assess trends. Third, for each article, we focused only on the covariates included in the largest model. However, it is possible that authors may have considered additional potential confounders that were not eventually included in the largest model. Moreover, some variables included in the largest models may have been considered as predictors worth capturing, instead of potential confounders. However, it is difficult to assess which variables were explicitly deemed to be potential confounders. Fourth, given the different patient populations, it also may not have made sense to adjust for the same characteristics in all studies (e.g., adjusting for sex in a study of only women). This is why we reported separate results for studies that included both genders, only men, and only women. It is possible that some additional confounders beyond gender were dealt with by restriction, i.e. by using eligibility criteria that restricted upfront the study population to have the same value for a potential confounder.

Our evaluation shows that although most authors mention confounding bias when interpreting their study findings, they rarely call for results to be interpreted with caution. However, the high variation in how confounders were defined and handled suggests that the results and their interpretation in these studies may have been affected by definition and handling choices.