Examining variation across treatment clinics in cancer patients’ psychological outcomes: results of a cross sectional survey

Non-modifiable individual characteristics such as age and gender [2] have been associated with psychological outcomes. Potentially modifiable individual factors such as coping strategies [6] and social support [7] have also been linked to poorer psychosocial outcomes. The association between disease and treatment characteristics has also been examined. Compared to other people with cancer, those undergoing current treatment [8], and people diagnosed with poor prognosis cancers such as lung, or cancers with complex treatment regimens such as haematological cancers tend to have poorer psychological outcomes [2, 9, 10].

It is also possible that there are differences between clinics in patient psychosocial outcomes. Variation in patient outcomes has been observed across a number of other areas of health care. For example, the Dartmouth Atlas of Health Care has documented a fivefold regional variation in rate of leg amputations for patients with diabetes [11]. This may be due to regional variation in the management of diabetes across indicators such as blood glucose testing and control, blood cholesterol testing and control, smoking cessation and foot care to identify and treat ulcers [11]. Similarly, regional variation in unplanned readmissions following treatment for peripheral artery disease ranged from 1 in 3 to 1 in 10 [11]. After adjustment for casemix, analyses of national claims data in the US showed that variability in mortality following cancer resection was influenced by both hospital volume and surgeon volume [4].

Several other studies have provided evidence of variation in factors that potentially relate to psychological outcomes. Jacobsen and colleagues reported variation in indicators of psychological care across 11 medical outpatient clinics. The study relied on medical records data, so it was unclear to what extent the variation observed related to actual differences in care patterns or to variation in the quality of documentation. Variation in patients’ psychological outcomes was not examined. There is some evidence of variation in related outcomes such as patient experience and pain across cancer clinics. A Korean study involving 34 palliative care centres revealed significant variation in pain management outcomes between centres. Higher scores for human resource adequacy were associated with better pain outcomes suggesting that organisational factors played a role in the observed variation in pain outcomes across centres [12].

Despite evidence for between-clinic variations in care practices which may be expected to influence psychosocial outcomes, there has been limited exploration of whether psychosocial outcomes vary across clinics. Our previous study involving radiation oncology clinics indicated that rates of anxiety varied from 24 to 34%; while depression varied from 13 to 20% [13]. These differences were not statistically significant, and conclusions of the study were limited by the small sample of clinics. To our knowledge, no other studies have explored the question of whether psychological outcomes vary across cancer clinics.

Therefore, the aim of the present study was to examine in a large sample of medical and haematology clinics: (a) whether there is significant variation in patient anxiety and depression across clinics independent of sociodemographic, disease and treatment factors.

Between-clinic variation in anxiety or depression scores (as determined by the HADS) was initially examined through boxplots, and then formally assessed through random effect linear regression models with anxiety or depression score as the outcome and a random intercept for clinic effect. Model-based estimates of the site level means (i.e. partially pooled estimates) with 95% confidence intervals were displayed as caterpillar plots for comparison. The statistical significance of the clinic random effect variance was assessed by comparing the model with a random effect to the equivalent linear model without the random effect, the corresponding likelihood ratio test (LRT) is assumed to follow a 50:50 mixture of chi-square distributions. The intra-class correlation (ICC) was reported, and where clinic variation in anxiety or depression was found to be statistically significant (p < 0.05), we reported the corresponding 95% confidence interval for the ICC obtained from bootstrapping. We also determined if this variation could be explained through differing patient level characteristics by including the following patient demographic (age, sex, marital status, highest level of education, smoking status, concession card status, private health insurance status, Aboriginal and /or Torres Strait Islander status and country of birth) and disease and treatment characteristics (cancer type, time since diagnosis, stage at diagnosis and treatments received) as fixed effects in the mixed-effects regression model and assessed the significance of the between-clinic variation using the method described above. Classification of categorical variables was as shown in Table 1. We also present results for estimates of the ICC on the probability scale for depression and anxiety dichotomised at a cut point of 8 or more. Logistic regression within a generalised estimating equation framework assuming a compound symmetric residual correlation matrix was used for these analyses.

Mean depression scores ranged from 3.7 to 5.4, with an overall mean of 4.57 (SD = 3.78). No evidence was found for variation in HADS depression scores among clinics (random effect variance LRT p value = 0.1232), (see Fig. 1), therefore no further analysis for the depression outcome was undertaken. The overall proportion of those reporting elevated depression scores (≥ 8) was 21%, (n = 588) with rates varying from 12 to 32% across clinics and an intra-class correlation (ICC) of 0.0001.

The overall mean anxiety score was 5.24 (SD = 4.07) with mean scores ranging from 3.9 to 6.4. There was evidence for variation in HADS anxiety scores between clinics (random effect variance LRT p value = 0.0002), with the ICC estimated to be 0.011 (95% CI 0.005, 0.024) (see Fig. 2). The overall proportion of those reporting elevated anxiety scores (≥ 8) was 27%, with rates varying from 12 to 38% across clinics and an ICC of 0.0067.

To our knowledge, this is the first large scale study to examine whether treatment clinics potentially contribute to variation in psychological outcomes for cancer patients. Previous studies suggest that clinic characteristics are associated with unwarranted variation in important outcomes such as pain [12] and mortality rates following surgery [4]. Jacobsen’s study [16] found that indicators of psychosocial care varied across 11 medical oncology clinics; therefore, it is reasonable to expect variation in psychological outcomes may also occur. Therefore, our finding that, once patient factors were controlled for, there was no significant variation across clinics for depression or anxiety is surprising.

A number of hypotheses may explain why between-clinic variation in psychological outcomes was not identified in the present study.

In Australia, while clinical practice guidelines were developed for psychosocial care of adults with cancer in 2003 [17], these have subsequently been rescinded. There is no requirement for specific distress screening processes, or provision of specific interventions, in order to gain accreditation. In the absence of policy levers to ensure implementation of specific care processes, individual clinicians have a high level of discretion regarding the processes (or lack thereof) which they put in place to detect and treat anxiety, depression and related outcomes. Therefore, our results may reflect that within clinic variability in the process of care used by clinicians within a clinic was greater than between clinic variation in processes.

It is also possible that there is little within-clinic variation, and that clinics implement very similar care processes. This would also account for our finding of no variation in outcomes. While private hospitals represent approximately 47% of all hospitals in Australia [18], only one private hospital was included in our sample due to prohibitive costs associated with obtaining research ethics and governance approvals. A greater representation of private clinics may have introduced greater variability as it is the nature of private enterprise to strive to differentiate oneself from the crowd by applying innovations in products and processes.

While numerous studies have demonstrated the efficacy of specific treatments to improve the psychological well-being of cancer patients [19‐22], effectiveness studies are more limited. Efficacy studies are undertaken in optimal circumstances with a focus on internal validity [23]. Often, this is achieved by controlling for context in a way that does not reflect the real world. Patient eligibility criteria, for example, are carefully delineated (often excluding individuals with commonly occurring co-morbidities, culturally diverse backgrounds and limiting age range) [23]. Furthermore, clinicians involved in intervention trials may be biased toward those who are already committed to improving psychological care. Additionally, in well-controlled research, the fidelity of adherence to the treatment protocol is usually carefully monitored and maintained [23]. These experimental conditions may not be achieved in everyday clinical practice, diluting the potential effectiveness of an intervention and limiting the generalisability of results. Therefore, if efficacious interventions are implemented poorly, or are ineffective in real-world settings, then variability in care processes may not translate into variability in psychological outcomes of patients. There is a need for the field to undertake effectiveness studies (sometimes referred to as pragmatic trials) where the power of the intervention is examined in real-world circumstances. In such studies, the evaluation design is able to take proper account of context by allowing a more realistic eligibility criteria, ensuring a range of clinicians are involved, and allowing for the expected variation in treatment fidelity.

The Hospital Anxiety and Depression Scale is a commonly used measure of psychosocial well-being. It has acceptable psychometric properties, and scores are not confounded by the physical symptoms of cancer or its treatment [15]. However, it remains a screening tool, and unlike a structured clinical interview, is not diagnostic. Therefore, it is possible that measurement error may have impacted on the accuracy of the anxiety and depression scores. In an effort to overcome uncertainty about the optimal clinical cut-point for cancer populations [13], analysis for this paper was undertaken using the HADS score as a continuous variable. Examining the HADS score in this manner increases the statistical power of the analysis.

As discussed above, our results must be interpreted in light of several limitations which may have affected the results. These include potential measurement error, and a hospital sample which consisted almost entirely of large metropolitan public hospitals. Further, our study did not allow us to rule out possible explanations for the lack of variability observed such as there being a high degree of within clinic variability in the delivery of care.