Reporting of health equity considerations in cluster and individually randomized trials

What is already known on the topic

What this study adds

Policy-makers in most countries have committed to reducing inequalities in the Sustainable Development Goals [1]. There is a great need for more empirical evidence on efforts to redress health inequities. Health inequity is defined as “differences in health that are unnecessary, avoidable, unfair, and unjust” [2]. Fairness is a normative concept, and there are differing perspectives on what is considered unfair and these may differ across settings and time [3]. Health inequalities are considered unfair when they can be avoided, prevented, or mitigated [4]. Not all health differences are health inequities; for example, age differences in Alzheimer’s disease prevalence would not be considered an age-related inequity because the risk increases with increasing age [5]. Also, judgments about fairness may need to take into account opportunity costs in sectors outside of health when redressing health inequalities.

Throughout this paper, we use the term “socially disadvantaged” to denote that people are disadvantaged by differences in distribution of power and resources which structure their living and working conditions and affect their opportunities for health [4]. We recognize that this terminology may be seen as labeling or stigmatizing, which is not our intent. Some people and populations may prefer other terms to fit their context such as marginalized, living in a vulnerable situation or under-served [6].

Different frameworks are available to describe factors associated with health inequity. We use the mnemonic PROGRESS-Plus which stands for place of residence, race/ethnicity/culture/language, Occupation, gender/sex, religion, education, socioeconomic status, and social capital [7, 8]. We also recognize additional characteristics as the “Plus” characteristics, such as: individual characteristics (e.g., age, disability); features of relationships; and time-dependent transitions (i.e., when a person is temporarily at a health disadvantage) [8‐10]. As stated above, differences in health outcomes across these characteristics are not always inequitable. We only consider them to be inequities when differences in health across these characteristics are produced or exacerbated by social disadvantage. For example, the difference in average life expectancy across neighborhoods associated with neighborhood-level socioeconomic deprivation in the UK and education levels in Norway is considered inequitable by many [11].

Randomized controlled trials (RCTs) can provide evidence about the effectiveness of an intervention and of the effects on health equity. However, many trials do not include people who may experience social disadvantage, and even if they do, results are rarely disaggregated for specific populations [3]. Under-representation of many populations that commonly experience social disadvantage has been well-documented; people living on low income, ethnic minorities, women, and older adults are often under-enrolled or excluded [12‐14] despite ethical guidelines typically stating that under-representation should be avoided [15]. When trials do include adequate numbers of participants representing these groups, the authors often fail to report basic sociodemographic details, such as the socioeconomic status of participants [16], and rarely consider subgroup hypotheses across PROGRESS-Plus characteristics [17, 18]. Carefully planned subgroup analyses can play a vital role in equity-relevant trials. For example, they can be used to investigate whether an observed total treatment effect is consistent across subgroups in the population [19]. They can also be used to identify subgroups with better or worse outcomes, or those experiencing potential harms. Pre-planned subgroup analyses can be used to test intervention effects in groups specifically targeted by an intervention, or believed to be resistant to treatment. However, subgroup analyses are subject to multiple methodological challenges which can complicate their interpretation, including increased risks of type I error due to multiple testing and low statistical power [20]. Some reviews have found that subgroup analyses are often poorly justified, infrequently pre-specified, and inadequately reported [21]. While such challenges may seem to discourage their use, lack of disaggregation or descriptive detail means that judgments about the policy relevance of the evidence to people who live with social disadvantage may be difficult for decision-makers who are using the evidence to inform and implement policies, programs, or individual patient care. Additionally, this lack of disaggregation limits the information available for systematic review authors and makes it impossible to explore differential effectiveness of the intervention. While underpowered subgroup analyses in trials should be interpreted with caution, they can provide useful information about the nature and direction of any potential effect and, when combined with data from other studies, can make more informative meta-analyses [22]. This inadequate consideration of potentially disadvantaged populations is often described as a limitation by those who rely on research to make decisions concerning socially disadvantaged populations [23, 24].

To address these concerns and the calls made in international public health policy documents as well as commitments from research funders and journal editors [25‐28], we have developed a reporting guideline for health-equity-relevant RCTs (CONSORT-Equity), which aims to increase the transparency and completeness of reporting of equity analyses to improve their usefulness for health-equity-relevant decision making [29].

Not all published trials are relevant to decisions concerning equity [3]. We define health-equity-relevant trials using the following criteria:

Cluster randomized trials (CRTs), that is, trials in which the units of randomization are coherent groups, communities, schools, or medical practices, rather than separate individuals, have unique methodological requirements, and also, unique equity considerations [30]. They pose additional challenges for assessing effects on health equity compared to individually randomized trials because clusters may include individuals experiencing different levels of social disadvantage and/or clusters may have different experience of social disadvantage. In addition, populations that are socially disadvantaged across one or more PROGRESS-Plus characteristics may be hidden within the clusters.

This study aimed to provide a descriptive assessment of the reporting of health-equity considerations in a random sample of 200 health-equity-relevant trials (100 individually randomized trials, RCTs and 100 CRTs). The preliminary results of this study were used to inform the development of the CONSORT-Equity 2017 Reporting Guideline [29]. We also used this as an opportunity to conduct a “baseline” assessment of the reporting of health-equity considerations in these trials; that is, before the publication of the CONSORT-Equity 2017 guidelines, by tabulating the proportion of trials reporting each CONSORT-Equity 2017 item in these equity-relevant trials.

This study was conducted to inform the development of a reporting guideline for health-equity-relevant trials (CONSORT-Equity) [29] and to serve as a baseline assessment of quality of reporting before publication of the CONSORT-Equity extension. The CONSORT Statement and its many extensions provide guidance for reporting trials completely and transparently and the health-equity extension focuses on items that improve the reporting of equity-relevant trials.

In our sample of health-equity-relevant trials, 25% of CRTs and 49% of RCTs reported a subgroup analysis by a PROGRESS characteristic. Of these, the most commonly reported subgroup analysis was across sex and it was only reported in 19% of studies (13% CRT, 24% RCT). Other PROGRESS characteristics were assessed using subgroup analyses in less than 10% of the studies included in our sample. There were almost twice as many studies which adjusted for PROGRESS characteristics as studies which performed subgroup analyses. While the approach to subgroup analysis differs across disciplines, these studies may represent a missed opportunity for understanding effects in socially disadvantaged populations. About two thirds of these subgroup analyses met one or more of the Yusuf criteria for quality of subgroup analyses. Only one out of 200 trials reported formally planning for a subgroup analysis during the design of the trial, by considering the sample size and power available for analysis. Because sample-size calculations are usually based on the primary comparison of interest; that is, between treatment arms, rather than on differential effects within subgroups, it is likely that many studies may have been underpowered for subgroup analyses. We believe that, even with insufficient power, if data were presented as disaggregated by important PROGRESS characteristics in all randomized trials, then it could be used for hypothesis-generation for future studies as well as in subsequent meta-analyses or other studies where greater power could be achieved. However, the results should be interpreted with caution and authors should state that type-II errors are possible.

CRTs had greater prevalence of reporting baseline assessment of place of residence and socioeconomic status than RCTs, possibly because balance across these characteristics is often checked to assess adequacy of cluster randomization. However, CRTs were about half as likely to report subgroup analyses across one or more PROGRESS characteristics than RCTs. Possible reasons are that CRTs have larger sample-size requirements than RCTs and there may have been inadequate numbers of clusters and/or participants to allow subgroup analyses to be conducted, and because statistical analyses are more complicated in CRTs. However, it may also be that we found that CRTs are more likely to be focused on an equity issue (64% compared to 55% of RCTs) and more likely to be conducted in a resource-poor setting (40% compared to 22%) and, therefore, conducting subgroup analyses may have been less relevant.

Our study has some limitations. First, we selected studies which met our criteria for being health-equity-relevant; therefore, the study was not designed to consider how well CONSORT-Equity items are reported in the clinical trials’ literature in general. We did not extract descriptive information about each included study, such as the type of intervention being assessed, so cannot assess whether reporting is different depending on the field of study. We have also only provided a description of what has been reported in these trials and we have not made judgments about how the studies were conducted or what should have been reported. We excluded studies at the abstract stage if they did not meet our criteria for health-equity-relevance which may have excluded some studies that did report on equity analyses in the full text but not the abstract during our screening process. This may have resulted in missing some studies which were health-equity-relevant. It is not possible to determine whether these would have different design or reporting characteristics from those included in our sample. Our search was restricted to the most recent years (2013–2015) as we were interested in describing the most recent reporting practices to inform the development of the CONSORT-Equity reporting guideline. It is important to bear in mind that the reporting of clinical trial results is usually dictated by their study protocols, which typically are 4 to 6 years older than the actual publication. Therefore, we may not have captured the very latest improvements in methods to report health-equity-relevant information from randomized trials. Finally, we did not assess all the CONSORT-Equity items because the final version of the equity extension was developed in 2017 after we had completed data extraction. The items which were not collected were added at the CONSORT-Equity meeting or during the development and included items related to eligibility, context, comparator, stratified randomisation, ethical clearance, implementation, intervention-generated inequities and limitations (e.g., power) to assess effects on health equity.

Our results provide a baseline estimate of the completeness of reporting of the recommended CONSORT-Equity items. These results show that even though these were all health-equity-relevant trials, only 73% described a rationale for their focus on equity socially disadvantaged population, only 25% reported engagement with communities or individuals who are socially disadvantaged, and only 20% mentioned importance of outcomes for socially disadvantaged populations. Because PROGRESS-Plus characteristics may be sensitive, these findings may be limited by the information that participants were willing to disclose [38].

In this sample of health-equity-relevant trials, we found few subgroup analyses that could inform equity decisions. However, many more trials recorded equity-relevant baseline descriptors, which suggests that there was an opportunity to consider how the analyses might inform equity considerations, including a-priori specification of hypothesis tests about treatment effects across relevant subgroups. Our findings concur with other studies which have found little detail about characteristics, such as gender and race/ethnicity/socioeconomic status, in studies to inform decisions about mitigating or redressing health inequities [16, 39‐41]. The added value of this paper is that we have identified a dearth of studies reporting adequate consideration of relevant subgroup hypotheses and pre-planned subgroup analyses across several PROGRESS characteristics. This limits their usefulness for informing decisions where health equity is an important consideration. In addition, we have identified deficiencies in reporting details such as the rationale for subgroup analyses and power for these analyses as well as details about the process of recruitment, engagement with people with lived experience and reporting of disaggregated data. This information can be used to guide future research and research reporting.

When health-equity considerations are poorly reported in trials, the applicability of the evidence cannot be fully assessed, and the research will be of less value for reaching important health-policy goals, including Sustainable Development Goals SDG5 (gender equality) and SDG10 (inequality within and between countries) and the calls to provide universal health coverage [42‐46]. When these population characteristics are adjusted for but not compared, we are unable to make any conclusions about their potential impact on differential effectiveness of the intervention. This can result in a loss of important policy-relevant or equity-relevant information. In addition, we cannot determine whether populations are included but hidden within trials or have purposefully been excluded or under-represented, in which case the validity of the results to determine prevention and care for the under-represented groups is compromised.

Presentation of disaggregated data, accompanied by formal subgroup analyses where appropriate, would allow us to better understand the equity implications of interventions. It is important to note that subgroup analyses should be carefully planned and informed by well-formulated hypotheses: the higher the number of subgroup analyses, the higher the risk of spurious findings due to multiple statistical tests [47]. When planning a trial, investigators need to consider potential population differences in baseline risk of the condition or problem being studied and the possibility of differential effectiveness of the intervention and decide whether subgroup analyses are appropriate and can be accommodated in the sample-size calculation [48‐50]. If additional subgroup hypotheses of interest are identified during the analyses, they should be clearly presented as exploratory and interpreted with caution [51‐53]. Presentation of disaggregated results across relevant subgroups is important so that the data can be used in meta-synthesis. Reporting needs to improve to explicitly state and justify any pre-planned hypotheses and present results for all such analyses, describe relevant sample-size considerations, clearly identify exploratory analyses and their hypotheses, and report disaggregated data for relevant subgroups.

One mechanism to facilitate the availability of disaggregated data for meta-synthesis might be to share data in online repositories, which would allow meta-studies to be conducted which may have greater power to detect subgroup differences. In addition, trial registries could request details about planned subgroup analyses and the availability of equity variables. The CONSORT-Equity reporting guideline aims to improve the transparency and completeness of equity considerations in RCTs and CRTs by providing authors with a checklist for reporting. However, this reporting guideline is only one step towards more equity-relevant research which is needed to ensure that inequities are not perpetuated or worsened by programs and policies.