Intervention effect estimates in randomised controlled trials conducted in primary care versus secondary or tertiary care settings: a meta-epidemiological study

Many studies have highlighted the lack of adherence to clinical practice guidelines by general practitioners (GPs) [1, 2]. GPs generally argue a lack of relevance or difficulties in applying guidelines [3, 4], and they particularly express concerns about the applicability of clinical trial results used to establish guidelines to their own patients [5, 6]. The first reason is that many randomised controlled trials (RCTs) include highly selected patients, which restricts the generalizability of their results [7]. As an example, a study found that less than half of primary-care patients would have been included in hypertension trials [8]. The second, highly related to the first one reason is that in most RCTs, patients are recruited in secondary- or tertiary-care settings [9] rather than in primary-care. However, as compared with secondary- or tertiary-care setting patients, primary-care patients have less severe disease and more undifferentiated symptoms [10] but more multi-morbidity [11]. Such differences may lead to differences in benefits and harms of the intervention. Our hypothesis is that intervention effect estimates differ between primary care setting randomised controlled trials (PC-RCTs) and secondary or tertiary care setting RCTs (ST-RCTs).

We performed a meta-epidemiological study to assess whether intervention effect estimates differ between PC-RCTs and ST-RCTs.

This was a meta-epidemiological study, a study design used to compare intervention effect estimates between trials with and without a characteristic of interest [12]. In this study, we focused on the setting, with the aim of comparing PC-RCTs to ST-RCTs. A meta-epidemiological study is performed by generally using a two-step approach and, contrary to traditional epidemiological studies, units of analysis are studies rather than patients [13]. First, for each selected MA, we assessed the difference in intervention effects between studies with the characteristics of interest (i.e., primary care setting) and without the characteristics of interest. This step involved using a meta-regression for each selected MA (with the intervention effect considered the outcome and the characteristic of interest the independent variable). Second, results from the meta-regression were meta-analysed over the different MAs.

In this meta-epidemiological study of 76 MAs, we did not observe a significant difference in intervention effect estimates between randomised controlled trials conducted in primary care versus secondary or tertiary care settings. Consistent results were found in subgroup and sensitivity analyses. Our result and the way we worked to obtain it satisfy the 10 criteria as treated by Moustgaard et al. [ 17].

To our knowledge, our meta-epidemiological study is the first to compare intervention effect estimates between PC-RCTs and ST-RCTs. The number of MAs analysed is the major strength of this study. In 2016, Dechartres et al. found that the median number of MAs per meta-epidemiological study was 29 versus 76 MAs in our study [18]. Nevertheless, we restricted our study to Cochrane reviews, which could limit the generalizability of our results. Cochrane reviews were shown to have better methodological quality [19] and to be better reported [20] than non-Cochrane reviews. We performed an analysis of MAs of binary outcomes, so our results cannot be extrapolated to MAs with continuous outcomes. MAs with continuous outcomes have more subjective outcomes, are less frequently blinded [21]and have higher heterogeneity [22] than those with binary outcomes. Finally, another limitation may be the definition of primary care settings. Primary care is often difficult and poorly defined in the literature and can include a large range of professions and professionals [14]. In Cochrane reviews, the description of the type of healthcare professionals in charge of including, following up and assessing patients was seldom reported. This omission could have led us to misclassify some trials or also to exclude some trials because of uncertainty. Nevertheless, the selection and the extraction process were performed in duplicate by health professionals (i.e., a pharmacy resident, a general medicine resident and a professor of general medicine in case of discrepancy), which helped limit the uncertainty of some terms and reduced the risk of errors.

Our result does not support GPs’ general feeling that results from PC-RCTs differ from those derived from ST-RCTs [3, 5]. Our study is sufficiently powered, and the ROR point estimate was very close to 1, which indeed supports the conclusion of an absence of differences in intervention effect estimates between PC-RCTs and ST-RCTs. However, among the 76 MAs, 29 (38.2%) were related to psychiatry (mainly depression) or addictology (mainly tobacco use) and 10 (13.2%) to pneumology. Very few MAs were dedicated to cardiovascular conditions (n = 4, 5.3%) or diabetes (n = 1, 1.3%). So the medical fields represented in our study do not well match those mostly frequently encountered in primary care, typically cardiovascular diseases (13.6%), muskuloskeletal troubles (12.6%) and pneumology (12.5%), with psychatric diseases representing only 8.5% of the consultations [23]. Therefore, how we selected the MAs for our study led to a sample of diseases that does not perfectly reflect the diseases mainly managed in primary care. Other studies should probably focus on specific medical fields or diseases rather than considering all MAs, whatever the medical field.

Another point is that we focused on binary outcomes, but in cardiovascular diseases or diabetes, most trials use surrogate or intermediate endpoints, which are mainly continuous ones, such as cholesterol level, blood pressure or HbA1c level [24, 25]. This situation may also have led to discarding some MAs focused on diseases that are highly prevalent in primary care.

Finally, although the present study focused on the settings, intervention effect estimates also depend on the target population, defined by selection criteria and/or the type/intensity of the intervention assessed. Due to the limited size of our study, no adjustment on these characteristics could be done while analyzing data. Therefore, one of the remaining questions, which is beyond the scope of the present work, is whether patients included in PC-RCTs are similar to patients included in ST-RCTs.

We found no significant difference in intervention effect estimates between PC-RCTs and ST-RCTs. Nevertheless, the main medical fields selected are not fully representative of primary care consultations. Further studies focusing on medical fields highly prevalent in primary care are needed.