Background
Cluster randomized trials (CRTs) are increasingly used to evaluate the effectiveness of health-care interventions [
1]. In CRTs, patients are nested within clusters such as hospitals, communities or practices, and interventions are applied at cluster levels but outcomes are measured at the individual level. It is expected that individuals in the same cluster, e.g. geographical area, hospital, would have more similarities compared to individuals in different clusters [
2]. This similarity may occur because patients in the same clusters interact with each other and receive care from the same practitioners. Intraclass correlation coefficient (ICC) is used to determine the degree of within-cluster dependence and it plays an important role in estimating sample size for cluster randomized trials [
3].
In cluster randomised trials, traditional sample size estimation techniques for RCTs lead to underestimation of the required sample size [
3,
4] and because individuals in the same organization or clusters are not independent, the sample size must be inflated [
4]. The degree of the increase in sample size is a function of both ICC and cluster sizes where generally a greater ICC requires enrollment of a greater number of patients in the trial [
3].
Chronic heart failure (CHF) is a major health problem worldwide associated with a high prevalence, mortality rate and hospital costs. It is estimated that about 5.7 million people are afflicted by CHF in the United States [
5] and in Europe approximately 5% of all acute medical admissions are HF-related [
6]. The estimated direct and indirect costs of HF management in the United States totaled $39.2 billion in 2010 [
7]. It was reported that the cost of HF care is two times higher than that of breast cancer, and three times higher compared to costs of colorectal and lymphoma cancer care in the USA [
8].
Care Pathway (CP) is a complex intervention for the mutual decision making and organisation of care processes for a well-defined group of patients during a well-defined period [
9]. CPs are also known as “integrated care pathways”, “critical pathways”, “care plans”, “clinical pathways”, “care maps” and “care protocols” [
10]. CPs have become a popular tool to improve quality of health-care services provided to CHF patients by reducing the risks of mortality [
11‐
13] and readmission [
14,
15] leading to shorter length of stay [
13,
14] and lower costs [
13,
14]. CPs are utilized to evaluate one specific care plan. Each hospital included in CPs uses the same protocol in practice. Many articles are available in Medline and medical databases describing the concept and success of CPs [
9‐
11]. According to the European Care Pathway Association (an international non-profit association), the defining characteristics of CPs include: (i) an explicit statement of the goals and key elements of care, based on evidence, best practice, and patients’ expectations and their characteristics; (ii) the facilitation of the communication among the team members and with patients and families; (iii) the coordination of the care process by coordinating the roles and sequencing the activities of the multidisciplinary care team, patients and their relatives; (iv) the documentation, monitoring, and evaluation of variances and outcomes; and (v) the identification of the appropriate resources. The aim of a care pathway is to enhance the quality of care, across the continuum, by improving risk-adjusted patient outcomes, promoting patient safety, increasing patient satisfaction, and optimizing the use of resources [
16].
In order to conduct a well-designed CRT with the aim to show the effectiveness of CPs for a CHF treatment, ideally, the ICC should be known beforehand to estimate required sample size and statistical power to decrease the chances of type II error [
17]. A range of ICCs has been reported previously [
17‐
22]; however, it is often difficult to obtain an appropriate ICC value for a specific study from the published ICC estimates. The present study reports ICCs for both demographical and outcome variables from cluster randomized trials of heart failure patients in the setting of usual care (UC) and care pathways (CP). Reported values of ICCs obtained from the UC and care pathway trials for some common outcomes would be useful to estimate sample size in future CP trials.
Results
ICCs and 95% confidence intervals (CIs) were calculated for 429 heart failure patients (CP, n = 214 and UC, n = 215). Data were collected from 14 hospitals. Cluster size ranged between 30 to 32 and average cluster size was 30.64. For ICC estimates of baseline characteristics, all centers was used because the intervention has not yet occurred. Baseline characteristics and outcome variables of the study are presented in Table
1 and Table
2 respectively. Mean age of the study participants was 81.66 ±8.41 years (range: 50–99) and 49.4% of them were males. The most common health problems among the study patients were hypertension (73.7%] and co-morbidities such as COPD, diabetes and smoking (32.9%]. 48.5% of the patients admitted following referral by a general practitioner.
Table 1
Baseline characteristics of the sample in clusters
| | | | 6.7 | 40.0 | 53.3 | | |
CP 2 | 30 | 80.50 (8.32) | 56.7 | 3.3 | 56.7 | 40.0 | 66.7 | 20.0 |
CP 3 | 31 | 79.39 (9.32) | 48.4 | 12.9 | 58.1 | 29.0 | 71.0 | 29.0 |
CP 4 | 30 | 82.00 (7.49) | 46.7 | 6.7 | 63.3 | 30.0 | 73.3 | 30.0 |
CP 5 | 31 | 82.13 (8.54) | 41.9 | 3.2 | 71.0 | 25.8 | 64.5 | 29.0 |
CP 6 | 31 | 83.39 (7.32) | 48.4 | 16.1 | 54.8 | 29.0 | 71.0 | 19.4 |
CP 7 | 31 | 82.29 (7.00) | 51.6 | 3.2 | 41.9 | 54.8 | 80.6 | 29.0 |
UC 8 | 30 | 81.43 (8.61) | 50.0 | 6.7 | 46.7 | 46.7 | 83.3 | 50.0 |
UC 9 | 30 | 78.77 (10.08) | 53.3 | 3.3 | 50.0 | 46.7 | 83.3 | 43.3 |
UC10 | 32 | 78.94 (9.610) | 46.9 | 0.0 | 56.3 | 43.8 | 71.9 | 43.8 |
UC11 | 31 | 76.23 (8.09) | 51.6 | 6.5 | 51.6 | 41.9 | 74.2 | 38.7 |
UC12 | 30 | 80.53 (7.32) | 53.3 | 6.7 | 40.0 | 53.3 | 86.7 | 43.3 |
UC13 | 30 | 82.60 (7.74) | 40.0 | 10.0 | 50.0 | 40.0 | 86.7 | 40.0 |
UC 14 | 32 | 79.22 (6.48) | 62.5 | 15.6 | 75.0 | 9.4 | 40.6 | 9.4 |
Total | 429 | 80.66±8.41 | 49.4 | 7.2 | 54.1 | 38.7 | 73.7 | 32.9 |
Table 2
Outcome variables of the present study in clusters
CP 1 | 30 | 10.97 (5.44) | 2227.30 (591.83) | 42.9 | 53.6 | 3.6 | 6.7 | 96.7 | 6.7 |
CP 2 | 30 | 13.13 (5.94) | 2171.77 (470.22) | 39.3 | 57.1 | 3.6 | 3.3 | 73.3 | 0.0 |
CP 3 | 31 | 9.13 (4.33) | 2077.87 (550.71) | 48.3 | 51.7 | 0.0 | 6.5 | 77.4 | 9.7 |
CP 4 | 30 | 8.27 (4.82) | 2113.47 (550.89) | 46.4 | 50.0 | 3.6 | 6.7 | 73.3 | 13.3 |
CP 5 | 31 | 10.35 (4.72) | 2089.52 (482.24) | 31.0 | 65.5 | 3.4 | 6.5 | 58.1 | 6.5 |
CP 6 | 31 | 9.48 (4.86) | 1969.10 (571.62) | 50.0 | 46.7 | 3.3 | 3.2 | 64.5 | 16.1 |
CP 7 | 31 | 11.13 (4.98) | 2234.29 (489.18) | 31.0 | 65.5 | 3.4 | 6.5 | 90.3 | 3.2 |
Total CP | 214 | 10.35 (5.17) | 2125.56 (530.93) | 41.3 | 55.7 | 3.0 | 5.6 | 76.2 | 7.9 |
UC 8 | 30 | 12.33 (7.63) | 2207.53 (591.92) | 24.0 | 64.0 | 12.0 | 16.7 | 83.3 | 13.3 |
UC 9 | 30 | 10.93 (6.11) | 2322.50 (488.25) | 45.5 | 40.9 | 13.6 | 20.0 | 76.7 | 10.0 |
UC10 | 32 | 10.13 (5.83) | 2290.13 (525.57) | 23.1 | 61.5 | 15.4 | 15.6 | 78.1 | 12.5 |
UC11 | 31 | 11.26 (6.00) | 2243.74 (480.45) | 11.1 | 74.1 | 14.8 | 12.9 | 77.4 | 16.1 |
UC12 | 30 | 9.43 (6.13) | 2380.37 (719.91) | 8.0 | 84.0 | 8.0 | 16.7 | 90.0 | 20.0 |
UC13 | 30 | 11.70 (6.82) | 2150.43 (613.75) | 25.9 | 66.7 | 7.4 | 10.0 | 76.7 | 23.3 |
UC 14 | 32 | 14.06 (7.65) | 1898.31 (490.99) | 77.8 | 22.2 | 0.0 | 15.6 | 25.0 | 3.1 |
Total UC | 215 | 11.42 (6.69) | 2211.21 (574.76) | 30.7 | 59.2 | 10.1 | 15.3 | 72.1 | 14.0 |
The mean length of hospital stay (LOS) for UC and CP patients was 11.42 ± 6.69 and 10.35 ± 5.17 days, with a mean cost of hospitalization of €2211.66 ± €574.76 and €2125,66 ± €530.93, respectively. In-hospital mortality rates were 15.3% in UC and 5.6% in the CP groups and the mean rates of unscheduled readmission for UC and CP were 14.0% and 7.9% respectively.
Tables
3 and
4 provide estimates of intraclass correlation coefficients, 95% confidence intervals and design effects for baseline characteristics for each cluster and outcomes of the clustered sample selected from 14 hospitals (7 CP, 7 UC).
Table 3
Intraclass correlation coefficients for baseline characteristics of the sample selected from 14 hospitals
Age* | 0.025 | 0.000-0.102 |
Disease Severity at Admission (NHYA)†
| 0.046 | 0.007-0.145 |
Admitted from Home or Referred by GP†
| 0.058 | 0.000-0.128 |
Co-morbidities†
| 0.027 | 0.000-0.066 |
Hypertension†
| 0.043 | 0.000-0.101 |
Table 4
Intraclass correlation coefficients for the outcomes variables of study
LOS (days)* | 0.020 | 0.000-0.184 | 0.063 | 0.007-0.311 |
Cost (€)* | 0.046 | 0.001-0.265 | 0.001 | 0.000-0.107 |
In-hospital mortality†
| 0.001 | 0.000-0.003 | 0.001 | 0.000-0.003 |
Disease Severity at Discharge (NYHA)†
| 0.182 | 0.062-0.554 | 0.000 | 0.000-0.076 |
AOS†
| 0.203 | 0.059-0.436 | 0.069 | 0.003-0.155 |
Unscheduled readmission†
| 0.004 | 0.000-0.036 | 0.010 | 0.000-0.046 |
ICCs for baseline characteristics ranged from 0.025 to 0.058. The median value and interquartile range for ICCs of baseline characteristics was 0.043 [0.026-0.052]. Among baseline characteristics, the highest ICCs were found for referrals by a general practitioner or admission from home (ICC = 0.058) and hypertension (ICC = 0.043) (Table
3). Therefore, to achieve the same statistical power for an individual randomized trial as would be obtained by a CRT, the number of subjects enrolled in the study should be multiplied by 2.68 and 2.25, respectively for a mean cluster size of 30. As shown in Table
4, ICC estimates of outcome variables in the CP group ranged from 0.001 for in-hospital mortality and disease severity at discharge to 0.069 for appropriateness of the stay (AOS) and median value and interquartile range for outcome measures ICCs was 0.006 [0.001-0.065]. The estimates for appropriateness of the stay, length of stay were 0.069 [0.003-0.155] and, 0.063 [0.007-0.311], respectively.
ICC estimates of outcome variables in the UC group ranged from 0.001 for in-hospital mortality to 0.203 for appropriateness of the stay (AOS) and median value and interquartile range for ICCs of outcome variables was 0.033 [0.003-0.187]. The estimates for appropriateness of the stay and disease severity at discharge were 0.203 [0.059-0.436], and 0.182 [0.062-0.554], respectively.
Discussion
In the present study, ICCs and their associated 95% confidence intervals were calculated for clinical and patient-related outcome variables based on the results of an Italian multi-center cluster randomized trial of heart failure.
In recent years, the need to have published ICCs from different CRTs was put forward to help planning future studies [
29‐
31]. Also several studies reported estimates of ICCs for various outcomes and for different treatment modalities [
18,
29,
32]. However, this is the first study to present ICCs for a cluster randomized trial of care pathway. In addition, most of the studies have reported ICCs obtained in the setting of primary or residential care [
19,
22,
29,
33,
34]. This study provides ICCs for a specific in-hospital treatment. Moreover, while some studies reported ICCs for many cardiovascular interventions [
18,
22,
25], none of them reported ICCs for outcomes such as in-hospital mortality or length of hospital stay to determine effectiveness and efficiency of heart failure treatment. Although a well-designed CRT was conducted with the aim to show the effect of pharmacological treatment on heart failure [
35,
36], ICC estimates are still lacking. To our best knowledge, this is also the first study to present ICCs for multiple outcome variables in an effort to evaluate effectiveness and efficiency of in-hospital treatment of heart failure.
We hope that our estimates will be helpful in designing a clinical experiment not only for CPs but also for any clustered RT in heart failure patients. However, we believe that there is one critical point that should be considered by any researcher who would intend to use these estimates. We often observed less variation in different clusters in the CPs compared to the UC in many cases because all of the hospitals in the care pathway group used the same protocol and they were informed and trained in the same way; thus, ICC estimates tended to be lower, which implies the presence of larger within- cluster variance. The variance within clusters may be reduced by adjusting the subject-specific covariates in order to improve the accuracy of the ICC estimation.
It has been reported that ICCs were usually between 0.01 and 0.02 in human studies [
34] and the Minnesota Heart Health Program Trial, the largest community trial for prevention of coronary heart disease to date, found ICCs which generally ranged between 0.002-0.012. In our study, the range of ICCs was wider (Tables
3 and
4).
We conducted a literature search to identify ICC estimates of variables similar to those in our study to compare our results. Previously published ICC estimates were available only for hypertension. We observed a moderate dependency for hypertension and previous estimates were also rather low for hypertension.
In primary care, physician practices would be expected to be more independent and previous studies showed that the ICC estimates derived from secondary care were greater than those from primary care [
37]. In contrast to other studies, we found greater ICCs for outcome variables such as LOS, AOS and cost, whereas the ICCs for baseline characteristics tended to be lower. High ICCs estimated for LOS, AOS and cost indicated that patients staying in the same hospital shared many common characteristics other than patients in other hospitals. In other words, for these outcomes, HF management was more likely to be influenced by the practice itself or to be related with physician’s practice style. A care pathway is performed to encourage physicians practice more consistently. If the intervention is successful, it is expected that ICC based on post-intervention data would be smaller than an ICC based on pre-intervention data [
38]. Based on our data, ICC value for the disease severity at admission [ICC = 0.046] was higher than that of disease severity at discharge in the care pathway group [ICC = 0.000]. This result confirms the success of the intervention and also consistency of the ICC estimates.
Estimation of the effect size or minimally important effect of intervention is another important consideration for sample size calculation. It is usually obtained from the published literature. Since a CRT design was used in the current study, our results may be helpful for estimating the effect size in future randomized trials of in-hospital treatment of heart failure.
Adjustment for the data variations in cluster was advised in many publications with regard to baseline covariates and factors such as age and gender based on the idea that different distributions across clusters could have an effect on ICC estimations. Several methods for adjustment have been discussed [
39,
40]. However, we did not make any adjustment because our clusters were very similar to each other with respect to baseline characteristics (Table
1). The results of this study were derived from patients between 50 and 99 years of age and may be extrapolated to other patients older than 50 years of age.
Many estimators have been proposed for binary outcomes [
41] and point estimation by ANOVA is most commonly used for calculating ICC for continuous data [
42]. Also there are several published studies which reported ICCs and each of them reported their ICC values in their own way. To ensure that all of the necessary information was presented in our report, we provided compressive information for reporting ICCs by using a framework reported by Campbell et al. in 2004 [
37].
Since sample size estimation is a key element of a clinical trial, many new techniques have been introduced for cluster randomized clinical trials in the recent publications. Campbell et al. was performed a study to generate empirical estimates of ICCs and to explore factors which may affect their magnitude [
38]. Furthermore, Turner et al. provided Bayesian methods of analysis for cluster randomized trials with binary outcome data [
43]. Campbell et al. presented the development of a sample size calculator for cluster randomized trials and described strategies for increasing the sample size [
44]. This calculator allows the investigator to trade-off the options for achieving the appropriate sample size. Most cluster randomized trials have unequal cluster sizes. Moreover, Eldridge et al. (2006) [
45] showed how coefficient of variation of a cluster size can be used to deduce the possible effect of unequal cluster sizes for various types of analyses and both continuous and binary outcomes. Their simple formula provides a good estimate of sample size requirements for trials analysed using cluster-level analyses weighted by cluster size and a conservative estimate for other types of analyses. Furthermore, Sample size formulae for CRCTs with a fixed number of clusters for both continues and binary outcomes were systematically outlined by Hemming et al. [
46]. In addition that Rotondi and Donner (2012) provided an evidence-based approach [
47] and authored an R package to facilitate sample size estimation in this design (CRT Size). In this approach, sample size for clustered randomized trials was estimated taking into account the role of the planned trial on a future meta-analysis [
48].
In the cluster randomized trials, the design effects are calculated using the cluster sizes from the existing data set and anyone planning a trial will need to calculate their sample size on their own. Thus, we did not report the design effects because it would not be useful for researchers planning their own trials.
There are some potential limitations of this study. The first limitation is the absence of any published ICC values for any care pathway or in-hospital treatment of heart failure. We have only been able to compare our results with ICCs estimated for different kinds of treatments. Second limitation of this study was that the data were collected between 2003 and 2004, to the best of our knowledge; this is the latest multicenter cluster randomized trial to show the effectiveness of CPs on the heart failure treatment. Also, no ICC values have been reported in literature for hospital treatment of heart failure up to date. But there is no evidence about the timeless of the material. So the readers should be aware of that changes in the treatment methodology can affect the ICC estimations and one should consider limitations of this study while using the ICC estimation of the present study.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
SK, KV and MP participated in designing the study and MP provided the data from a previously conducted CP trial. SK conceived of the study and wrote the first draft of the paper. MP and KV helped writing the final manuscript and contributed to discussion of the results. All authors read and approved the final manuscript.