Sample size considerations for trials using cerebral white matter hyperintensity progression as an intermediate outcome at 1 year after mild stroke: results of a prospective cohort study

White matter hyperintensities (WMHs) are lesions in the white matter of the brain visible on magnetic resonance imaging (MRI). They are common and associated with stroke [1, 2], dementia [3, 4] and physical disability [5]. There is increasing interest in using WMH as an outcome in trials investigating cerebral small vessel disease and associated diseases such as stroke and dementia [4]. However, the mechanism for the development of WMH is not fully understood, and research, including randomised controlled trials (RCTs), into the role of modifiable risk factors such as hypertension, diabetes, smoking and hyperlipidaemia is ongoing [6].

Measurement of WMH volumes is not trivial, and there is not yet a consensus on how it should be done [7]. It is labour- and resource-intensive, requiring much time from staff with specialist image analysis skills. Fazekas scores are much easier to obtain as they may be read from brain scans within a few minutes by individuals with appropriate training. The Fazekas scale gives scores ranging from 0 to 3 each for periventricular white matter and deep white matter, which can be summed to give a total score from 0 to 6, with lower scores indicating less WMH burden.

Any trial using WMH as an outcome should adjust for important predictor variables such as baseline WMH and age. Also, prediction of WMH for an individual patient is difficult because there is much variability in WMH progression between patients [8]. However, previously published WMH sample size calculations have used unadjusted estimates [9, 10]. Therefore, we studied the predictors of WMH in a high-risk population, namely patients with mild stroke, in whom WMHs are an adverse prognostic marker [11, 12] and who would therefore benefit from interventions to reduce WMH progression. We examined ways, using study design, covariate adjustment and statistical analysis, to reduce sample size for clinical trials of WMH progression. We present our results on predictors of WMH and sample size calculations for hypothetical RCTs.

We recruited 264 patients from 10 May 2010 to 24 December 2012, and 190 had imaging data at 1 year. Of the 74 with no 1-year WMH volumes, 33 declined to participate, 24 were unwell, 5 had died, 6 had various other reasons for not participating and 6 had incomplete scans that did not allow calculation of volume data.

Patients who completed follow-up MRI were younger by 5.7 years than patients who did not have another scan. MAP, PP and diastolic and systolic blood pressures were not significantly different between those with and without follow-up MRI (all estimated differences <0.6 mmHg, p > 0.73 for all). There was also no significant difference for stroke subtype, hypertension, diabetes, smoking status or hyperlipidaemia (p > 0.16 for all). Baseline Fazekas scores tended to be higher in patients without follow-up (p = 0.049) (Table 1).

At follow-up, the mean (SD) WMH volume was 23.2 ± 23.3 ml. The mean difference between follow-up and baseline was an increase of 1.27 ml (95% CI 0.06–2.48, p = 0.040). A total of 124 patients had an increase (mean increase 5.5 ± 5.8 ml) in WMH volume, whereas 66 had a decrease (mean decrease 6.6 ± 6.9 ml) (Fig. 1). The median Fazekas score at follow-up was 3 (IQR 2–4), a non-significant increase of 0.5 from baseline (2.5; IQR 2–4). There was no statistically significant relationship between blood pressure and WMH change after adjustment (Table 2, Fig. 2).

We expected and checked for collinearity with variance inflation factors, because MAP, PP and diagnosis of hypertension are all ways of assessing blood pressure, and Fazekas score and baseline WMH volume are both ways of assessing WMH burden. The blood pressure variance inflation factors were 1.4, 1.6 and 1.2, respectively, for MAP, PP and diagnosis of hypertension, indicating that they are not a source of collinearity. However, Fazekas score and baseline WMH volume have variance inflation factors of 3.4 and 3.1, respectively. This may or may not indicate an acceptable degree of collinearity to researchers, so the results should be interpreted cautiously, though it should be noted that dropping Fazekas score from the model gives similar results.

The predictor that explained most of the variation in follow-up WMH volume was baseline WMH volume in both univariate and multivariable analyses (Table 2, Fig. 1a). Fazekas score was also related. Uncertainty in the univariate analyses (evidenced by wide CIs) and changes of directions for some predictors (e.g. diabetes) were observed. Individual changes in WMH volume were varied, and WMH decreased in some patients, stayed the same in some, and increased in others, reflected in the wide SD. MAP was negatively associated with WMH progression in adjusted analysis (Table 2), as suggested in Fig. 1, where there are more patients with higher starting blood pressure with decreasing WMH.

The sample size calculations for WMH volume were based on an RCT in which the treatment was hypothesised to stabilise WMH volume as at trial entry by preventing progression of WMH. In the control group, WMH would increase, as seen in our observational data at approximately by 1 ml per year, and the WMH volume SD at follow-up would be 23.3 ml. We then adjusted the sample size for inclusion of predictors in the analysis. The R ² values are taken from regression models with follow-up WMH volume as the outcome, and they are used to calculate the reduction in sample size that would happen by including adjustment for predictors by reducing the unexplained variability in outcome. We then recalculated the sample size for a 2-ml difference between active and control groups and for 2 years of follow-up, assuming the same per-year increase in WMH. The sample size calculations for Fazekas score are based on our baseline and follow-up data percentages of patients in each category.

If the RCT had allowed for 2 years of follow-up and with the same assumptions so that the control group WMH volume would increase by 2 ml, then the sample size per group would be 2128 (4256 in total). Adjustment for all predictors reduces this to 205 per group (410 in total) before allowing for dropouts (Table 3).

If Fazekas score were the outcome variable, then sample sizes would increase by a factor of 5.04 between not adjusting and adjusting for baseline Fazekas score, assuming a constant OR for treatment versus control in all baseline Fazekas categories. This assumption is not likely to be true and would need to be explored with a simulation study. Other predictors would inflate the sample size less, given the same assumption (Table 3). The large sample size for the comparison of medians is a result of most patients’ Fazekas scores not changing over the course of 1 year.

WMH volumes are linked to cognitive decline, stroke, dementia and death. Although their causes have not yet been fully ascertained, treatment of risk factors such as blood pressure may limit their progression. These interventions are being tested in ongoing trials. We studied a well-defined patient sample with mild non-disabling ischaemic stroke, a group likely to be of interest to researchers in stroke and dementia. The WMH volumes were measured carefully with a well-developed methodology. In our observational cohort of patients followed for 1 year, there was wide variation in WMH volume change from patient to patient, making prediction for individual patients difficult. Roughly one-third of our sample showed a decrease in WMH, and two-thirds showed an increase. The extent of change also varied widely, with a mean change of 1.27 ml but a range from −32.0 ml to +29.1 ml. The predictor that explained most of the variation in WMH progression was WMH volume or Fazekas score at presentation.

Previous estimations of average rates of progression of WMH were similar to ours at about 1 ml per year, but they did not seem to identify the relatively large proportion whose WMH decreased [8, 9]. This may reflect the fact that other populations, such as community-dwelling older subjects as in the Leukoaraiosis and Disability Study [10] or the Austrian Stroke Prevention Study [41], may be less variable than patients presenting with an acute disease such as stroke.

Researchers seeking to undertake RCTs of treatment to prevent WMH progression need to be aware that the variation in WMH volumes, at least in stroke patients, can lead to prohibitively large sample sizes unless an efficient study design is used. Fortunately, making RCTs more efficient need not be difficult. Analyses that use a baseline measurement as a predictor are known to be more efficient than studies comparing follow-up measurements only, or even those that use difference between baseline and follow-up [42‐44]. Our calculations for hypothetical studies show that it is possible to reduce the required sample size by more than 90% by including some routinely collected data and a baseline measurement. Increasing duration of follow-up to 2 years also helps reduce sample size but increases cost per patient substantially as well as the number of dropouts.

However, a trial of 1642 patients (followed over 1 year, 2054 allowing for dropouts) is a massive undertaking, especially if all must have baseline and follow-up MRI and all images must be analysed for WMH volume, which has to be done centrally by trained operators, including careful exclusion of the index and any old stroke lesions. Of note, 74 (28%) of 264 in our cohort did not have MRI at 1 year. Failure to have repeat MRI is not uncommon in observational studies and trials.

We did not allow for a range of dropout rates, and we present data for only a 20% per year dropout rate. This (and other rates) of course increases the required sample size. Here, just under one-third (28%) did not have 1-year MRI. Although this might have been different in an RCT (rather than an observational study), we have no evidence that patients are more likely to comply with imaging in trials. Moreover, 29 (39%) of 74 patients who dropped out did so for reasons of health, which is likely to be a source of bias; it seems reasonable to suppose that people with worse general health have less healthy white matter and hence more WMH progression. This bias would have a greater effect in studies with longer follow-up, because dropout is generally greater in long-term studies. The attractive sample size estimate for the 2-year study needs to be balanced against this bias. Statistical methods for dealing with missing data generally assume that the data are missing at random. Although the methodology of dealing with data that are not missing at random is developing, and although it is possible that data more readily obtainable at follow-up could be used in multiple imputation, any results would need to be interpreted very cautiously. In particular, the sensitivity of results to assumptions about the relationship of missingness and observed data would require careful exploration. Additionally, the longer the follow-up, the greater the risk of genuine changes in MRI scanner performance or in scanner replacement, which could inflate variance and further alter the detection of WMH. Finally, 6 (8%) of the 74 subjects without follow-up MRI data actually had the scan, but their data were unsuitable for WMH volume measurement because of patient movement within the scanner or completing only part of the scan.

Over 10% (33 of 264) of patients (45% of the 74 who did not have 1-year MRI) did not have follow-up WMH volume data because they found the first MRI scan unpleasant. Although careful screening of patients with claustrophobia and other contraindications is essential, it is not possible to predict which patients will decline follow-up because it involves MRI.

We know that the patients with follow-up were younger and had lower Fazekas scores than the entire baseline group. Although we adjusted for age, Fazekas score and other predictors, the baseline and follow-up groups may differ in other ways that we have not been able to account for.

We have not allowed for the effects of between-scanner variation on measurement of WMH volume, but large trials (even to get the minimum number of 642 patients calculated here) require multicentre participation, which inevitably requires use of different scanners and introduces between-scanner variation. Whether the Fazekas score (or other visual scores) are less susceptible to between-scanner variation is not known and would be an obvious target for testing to inform future trials in stroke and dementia.

Fazekas scores are more easily measured than WMH volumes. However, calculation of the sample size for scales such as the Fazekas scale is more complex and requires data that may not be readily available. The results shown in Table 3 suggest that the sample size is sensitive to small changes in the assumed OR—a small decrease in OR of just 0.04 caused the sample size to more than double, and a small increase in OR of 0.05 more than halved the number. Therefore, researchers should use the most precise estimates available to ensure than any ordinal logistic regression analysis is suitably powered. Also, sample size may counterintuitively increase when estimates are adjusted for baseline covariates [45]. Even so, it is recommended that researchers use adjustment in logistic regression both to balance the trial groups for important outcome predictors and to have estimates that are more readily applicable to individual patients [34]. Researchers need to undertake a simulation study to explore assumptions and the sensitivity of the estimated sample size to those assumptions. A compromise for the significant organisational effort, cost and ‘noise’ involved in assessing baseline WMH volume that captures the similar prognostic information might be to use the baseline Fazekas score and adjust for covariates and use WMH volumes as outcome. It is also important to tie WMH to a relevant clinical outcome such as cognition, recurrent stroke, other vascular events or death, and possibly a measure of mobility, to capture the physical effects of small vessel disease.

In our statistical model, we used raw WMH volumes and checked model performance by examining residual behaviour and linearity assumptions. Some aspects of model performance were improved by using the logarithm of the WMH volumes rather than the raw data. The greatest improvements occurred when we used logarithms for both baseline and follow-up data. In our opinion, the improvement in model performance with logged data was not great, so we chose to present results for the raw data for ease of interpretation. However, we recognize that this is a subjective choice and that others might choose differently. Unsurprisingly, the models with raw and logged data all revealed baseline WMH to be positively related to follow-up WMH, with high R ² values of 0.90 for raw baseline and follow-up and 0.87 for logged baseline and follow-up. Therefore, the reduction in sample size would be similar, regardless of whether logged or raw data were used.

Although our sample size for the comparison of medians gave an unachievable sample size (174,432 before allowing for dropouts), this does not mean that analysis with medians in other contexts is unfeasible. Researchers wishing to use medians could explore the non-parametric ANCOVA models with simulation studies, not only to estimate sample size but also to test the suitability of the ANCOVA models for the kind of data they are likely to obtain. This is beyond the scope of this paper, but it could be a subject for further research.

We show, in a fairly typical population of patients presenting to a regional hospital with non-disabling stroke and managed according to current UK guidelines with anti-hypertensives, statins, antiplatelet drugs and lifestyle advice, that WMH may decrease, stay the same and increase over the course of 1 year after stroke. Baseline MAP was inversely associated with WMH volume in multivariate analyses. This is counterintuitive, but it may have occurred if patients with higher initial MAP were diagnosed when presenting to the stroke clinic and were started on anti-hypertensive medication, or if they already had a diagnosis of hypertension but were not taking their prescribed anti-hypertensive medications and were motivated by their stroke to take their medications, leading to better blood pressure control, which may have contributed to reducing the development of WMH. In general, WMH are assumed only to increase in size, but the fact that they may decrease as shown here (in 35%) adds to the variation in follow-up volumes, thus inflating sample sizes, and indicates that they do not consist only of permanently damaged tissue. However, it would seem that WMHs are much more dynamic, at least in the year after mild stroke, than previously thought. Reasons for this dynamic WMH change, as well as the primary mechanisms underlying WMH, should be determined urgently.

Sample sizes for RCTs using WMH as an outcome can vary enormously according to measurement of WMH, adjustment for predictors, and method of analysis. Smaller sample sizes will be obtained in studies that use WMH volumes rather than Fazekas scores, but measuring volumes is resource-intensive. Dropout rates and scanner drift could impact results. Triallists undertaking studies using WMH as an outcome need to consider their study design and analysis plan carefully to maximise efficiency.