Combining QOF data with the care bundle approach may provide a more meaningful measure of quality in general practice

The condition-specific care bundles and associated individual components were selected from the ‘Quality and outcomes framework guidance for GMS contract 2009/10’ by the authors [15]. Studying QOF data was a pragmatic choice, as it provided a measure of current evidence-based clinical practices. We agreed selection criteria to help identify potentially suitable clinical conditions and indicators according to previous published guidance [24] and local practical considerations. All QOF sections and indicators were initially considered. The selection criteria (Table 1) were applied in a step-wise manner as illustrated in Figure 1 to exclude unsuitable indicators and conditions.

Five QOF clinical conditions fulfilled the pre-specified inclusion criteria: Stroke/TIA, secondary prevention of coronary heart disease (CHD), diabetes mellitus (DM), chronic kidney disease (CKD) and chronic obstructive pulmonary disease (COPD). The 24 retained indicators are shown in Table 2.

A total of 12/14 practices agreed to participate. Nine practices were able to provide data in the format requested to allow meaningful analysis (64.3%). The combined patient population on the 31^st March 2010 was 56,948. The prevalence of the selected clinical conditions in all practices was: CKD 3.4% (range 2.5% to 4.9%), stroke/TIA 2.2% (range 1.7% to 2.7%), diabetes mellitus 4.2% (range 3.6% to 5.5%), CVD 4.6% (range 4.2% to 6.2%) and COPD 2.3% (range 0.8% to 3.8%). Further demographic and disease register details of participating practices are shown in Table 3.

The minimum and maximum payment thresholds for the selected indicators ranged from 40-50% and 60-90% respectively. All practices achieved the minimum threshold for all indicators and the vast majority also achieved the maximum threshold for most indicators.

The composite measures of the selected conditions (care bundles) varied for all practices and were as follows: CKD 69.0% (range 64.0-81.4%), stroke/TIA 74.1% (range 51.6% to 82.8%), Diabetes mellitus 58.4% (range 50.3% to 65.2%), CVD 64.0% (range 47.8% to 71.9%) and COPD 82.0 (range 47.9% to 95.8%). Care bundles with more components such as DM and CVD generally had lower compliance rates.

The diabetes mellitus care delivered by all practices as specified by individual indicators ranged from 82.9% (DM24) to 98.5% (DM15), yet overall care bundle compliance was substantially lower at 58.4%. In other words, 58.4% of all eligible patients received all the care specified by the bundle and 41.6% of patients did not receive one or more care components. For CVD care, individual indicator achievements ranged from 82.6% (CVD10) to 99.0% (CVD11) with 64.0% of patients receiving all specified care. The COPD care delivery was the most reliable, with 82.0% of patients receiving all components. However, overall the results suggest that the composite bundle approach highlights a greater level of variation within and between practices in evidence based care delivery and arguably therefore provides a more informed and discriminatory measure of care quality than the current, largely homogenous approach. The variation in the individual indicator and composite measures for each clinical condition and practice can be observed in Table 3.

This small study is the first known attempt to measure composite care delivery for specific clinical conditions in UK primary care using a care bundle approach. We identified five suitable clinical conditions for care bundles: COPD, CKD, DM, TIA/Stroke and CHD. Three to eight components were chosen and adapted from QOF indicators for each bundle. Compliance with individual bundle components was generally high, but substantially lower overall for the ‘all or nothing’ bundles. We would suggest that this ‘gap’ in performance may be a more valid reflection of the variation in expected care delivery at the patient level than current individual measures presently indicate.

There are at least five possible reasons for the observed ‘gap’ between individual component and overall bundle compliance. Firstly, the Framework financially incentivizes an individual indicator approach so this becomes the practice focus. Secondly, some components are easier to deliver than others. Thirdly, care bundle compliance (the composite measure) will always have a value equal to or lower than that of the indicator with the lowest performance - consequently there will always be a gap, although it is unknown whether or when the size of the ‘gap’ becomes of clinical significance. Fourthly, the ‘gap’ will usually increase exponentially with the rise in the number of bundle components. Finally, natural variation is inherent in all health care systems and will be affected by the effectiveness and efficiencies of local processes and team efforts.

Variation in care is introduced ‘naturally’ through different and often complex patient presentations and ‘artificially’ through individual clinician differences and priorities, extenuating circumstances and local systems. ‘Patient characteristics’ and ‘deprivation’ have previously been identified as the two factors which are strongly associated with variation (‘the gap’) in primary health care settings in Scotland [12], England [11, 13, 39], the Netherlands [40], Australia [41] and the USA [42].

The care ‘gap’ between components and care bundles in our study was observed for all clinical conditions and participating practices. This ‘gap’ (variation) may be at least partially accounted for by hidden patient and deprivation factors. The implication may be that the care bundle method may measure the gap but could not necessarily help to eliminate it.

In UK secondary care settings, reported compliance with a variety of clinical care bundles ranges from 19-52% [32, 33]. Low compliance rates have important patient safety implications, as a positive and significant association has been found between compliance rates and clinical outcomes such as mortality [27, 28, 32]. A similar association has not yet been shown for primary care. Although the participating practices’ compliance with all the selected bundles was ≥ 58% it is not known whether compliance can be compared meaningfully across settings and for different care bundles. It would also be desirable to determine to what degree different compliance levels may impact on clinical outcomes in primary care settings.

We adapted care bundles from the QOF, currently the most ambitious, comprehensive and largest P4P scheme and quality measure in international healthcare and a ‘natural experiment in progress’ [43]. Framework research and experience may therefore be applicable to our study findings. For example: the Framework can cause a ‘street lamp effect’, i.e. neglect of health care activity and clinical conditions which do not attract financial incentives; it can create a de-personalizing ‘box ticking culture’; it is vulnerable to data distortion and potential gaming; it is accelerating a transition to nurse-led primary care; there is tension between the different QOF roles as a quality improvement method, regulatory framework or remuneration mechanism; the Framework promotes simplicity over complexity and measurability over meaningfulness. These valid concerns could similarly be raised about the care bundle method, with no clear answers at present [19, 43‐46].

The impact of QOF on clinical outcomes continues to be debated. There is no evidence of any ‘discernable effect’ on hypertension-related outcomes [47], while new depression indicators failed to improve disease detection or treatment [46]. In addition, associations between QOF measures and all-cause mortality and emergency admissions have been found to be ‘small and inconsistent’ [39]. On the other hand, evidence of improved care quality for patients with specific chronic diseases and a ‘trend towards improvement’ in process and outcome at least partly attributable to QOF have recently been reported [44, 48‐50]. It is unclear what impact (if any) surrogate measures such as care bundles would have on clinical outcomes, how this could be feasibly measured and what period of time would be required.

‘Time-interval’ indicators have been proposed as an alternative and more reliable measure to ‘point-in-time’ indicators to assess care quality [51]. Performance is evaluated over a specified period rather than at a single point in time. Mabotuwana et al applied this approach to hypertension management and found it feasible but with lower measures for ‘interval indicators’ compared with point-in-time indicators. The differences, similarities and potential benefits of any one method, whether care bundles, time-interval or QQF indicators are unclear.

The care bundle method may be a useful new care quality measure and help to reduce healthcare variation. It may also help to better differentiate the care quality of practices, given that their current QOF scores are now broadly comparable. To implement the bundle a number of challenges would have to be overcome, for example: resistance from practices who are financially disadvantaged by ‘lower’ measures than the current thresholds; accounting for ‘natural’ variation so that practices are not unfairly penalized; existing information technology systems would have to be redesigned; a full evaluation to identify both impact and unintended consequences (e.g. a proportion of patients may end up receiving ‘no care’) would be necessary. If this new measure is considered desirable, it would have to be promoted and incentivized.

Reducing variation in quality can decrease costs if the care ‘gap’ is large, but costs increase as the gap narrows until there is a net expense [52]. Evaluation of care bundle implementation in some secondary care settings has found them to be cost-effective [53]. However, implementing interventions require initial financial and resource investment. McNeill et al found that only 20/265 (12%) of acute medicine units in the UK had the minimum facilities to comply with the ‘surviving sepsis care bundle’ [34]. However, the applicability of care bundles in acute settings cannot be assumed to translate to QOF-bundled components because care packages over 9-15 month periods.

The financial implications of quality-reporting programmes vary greatly. The QOF ‘probably’ represents value for money [54]. Adapting the Framework’s reporting system to allow composite quality measures would require substantial investment and feasibility and acceptability concerns would have to be addressed. Known incentives that facilitate increased practice participation include: financial payments, staff training and providing technical support [54, 55].

Implementing care bundles and increasing compliance with them relies not only on individual health care professionals but even more on the availability of adequate resources, support systems and leadership. Aligning care bundles with larger quality improvement initiatives and providing related training also appears to be an important factor influencing success and impact [28]. These issues need to be considered if the care bundle approach is to be successfully implemented in primary care settings.

The threshold ‘ceiling’ for maximum financial reward is set at ≤ 90% achievement. Practices could hypothetically ‘stop’ delivering care once they reach these thresholds without financial penalty. Our findings indicate that many indicators with relatively low achievement ceilings were still performed for the vast majority of patients which seems to suggest that thresholds are an unlikely reason for the observed ‘gap’.

Exception reporting has previously been found to be consistent across practices and regions and accounts for a tiny minority of cases [38]. Concerns that practices may be ‘gaming’ results through excessive or inappropriate exception reporting have not been substantiated. However, exception reporting combined with less than 100% maximum threshold targets introduce an incentive ceiling [18].

Our findings are based on a small, convenience sample of practices. We did not take into account patient list size, geographical distribution (degree of remoteness), socio-economic class and level of deprivation or prevalence of disease. Our findings may still have wider application, as different degrees of remoteness, access and practice size do not significantly contribute to variation in care quality as measured by QOF results [56, 57]. A recent review found that inequalities in chronic disease management have largely persisted after introduction of QOF (even though quality of care may have increased) [58]. While deprivation is a major contributing cause to overall care, it has not been shown to affect QOF scores. For example, Hamilton et al found that the impact of QOF on diabetes mellitus care was comparable in affluent and deprived areas [59].

We decided to code exception reported indicators as if the described care had been delivered. The consequence was a ‘smaller’ compliance ‘gap’, but one that may have more acceptability to frontline staff. Exclusion systems have an important role to help militate against the potential impact of socio-economic, patient and other factors out-with the practice team’s control. Simple care processes are mitigated more than complex ones and the additional work required in deprived areas is not always being rewarded [60]. We therefore purposefully selected ‘simple’ care bundle components in preference to ‘complex’ ones where possible. We accept that there are many other potential care bundle topics for general practice. However we chose a pragmatic approach to give an initial indication on the feasibility of the concept as applied to QOF data. Others may have chosen alternative approaches and clinical topics for bundles.