We conducted ethnographic visits to 17 of the 196 adult ICUs in England that participated in Matching Michigan. Most (96%) of ICUs in England took part in the program, but we were able to secure access to two non-participating units. Thus, 19 ICUs were included in the study in total. We conducted around 910 hours of observations and 98 semi-structured across the 19 ICUs. Our analysis is focused primarily, though not exclusively, on the 17 units that participated in the program. We carried out 29 telephone interviews with training event participants.
Design and execution of the program
Matching Michigan was not an exact replica of Keystone across a range of dimensions (Table
1).
Table 1
Selected differences and similarities between the Keystone project and
Matching Michigan
1. One cohort | 1. Four cohorts (97% of English ICUs), including one pilot |
2. Kicked off with 6 weeks of ‘immersion’ weekly teleconferences | 2. Kicked off with data collection training |
3/Whole-state workshops every six months—1.5 or 2 days (overnight), gradually becoming participant-led | 3. Each cohort attended two ‘training events’ (0.5 or 1 day)—data collection and intervention |
4. Continuous contact via teleconferences with 100~200 | 4. Teleconferences only at the beginning; discontinued after poor attendance. Webinars continued, but generally not well attended. |
5. 5/6 months getting started with data collection & implementing the comprehensive unit-based safety program and daily goals; then Ventilator Acquired Pneumonia (VAP) and CVC-BSI interventions. | 5. Initial period (3-6months according to cohort) of data collection only, then all interventions in any order. No VAP intervention. |
6. Interactive web-based data entry tool allowing comparison with others | 6. Interactive web-based data entry tool allowing comparison with others |
7. Program team asked for infection rates to be reported by infection control practitioners independent of the ICUs. | 7. ICUs allowed to determine method of data collection and reporting for themselves. Detailed definitions and guidance provided. |
8. Targeted adult ICUs primarily | 8. Targeted both adult and paediatric ICUs |
9. Led by collaboration between prestigious out of state university and the state hospital association | 9. Led by government agency |
Some differences between the two programs were superficially minor; others were more far-reaching. The original Keystone program sought to drive clinicians’ use of five evidence-based practices through a combination of technical interventions (e.g., a checklist summarizing the five practices and a dedicated line insertion trolley/cart) and ‘adaptive’ interventions (e.g., the Comprehensive Unit-based Safety Program, or CUSP, intended to help in altering culture and behavior). Matching Michigan, in an example of a minor difference, used the term ‘non-technical’ to describe the interventions known as ‘adaptive’ in Keystone. These non-technical interventions were not labelled as the CUSP (the term used in Keystone), but their content was largely the same as the original.
The content of the technical interventions used in Matching Michigan was largely similar to Keystone. What was much more different was the provenance and freshness of these technical interventions for the target audience. Where Keystone had introduced a set of five evidence-based procedures summarized into a checklist that was new to participating units, Matching Michigan’s procedures were based on two pre-existing Department of Health CVC ‘bundles’ of procedures known as ‘High Impact Interventions.’ The two bundles (one for insertion and one for ongoing care) had already been established policy since 2007, two years before the program launched. Thus, apart from an updating of the insertion bundle to recommend that hats and masks be worn during CVC insertion, the evidence-based procedures were not new in England at the launch of the program, and they were closely associated with established central government policy. English ICUs had already had significant exposure to the evidence on procedural good practice for insertion and care of catheters by the time the program launched; the bundles had been heavily promoted by the Department of Health and by other agencies over a significant period.
The organization of Keystone and Matching Michigan differed significantly, though in ways that might not be evident from a quick examination of the program components. Both programs held meetings for staff from participating ICUs. However, in contrast to Keystone’s model of initial immersion coaching and six-monthly residential workshops, with all participating units meeting at the same time, Matching Michigan offered two non-residential training events for each of the four clusters as they entered the program. The first training event for each cluster covered data definitions and data collection and the second the program’s interventions. These training events were described by participants as professionally organized and effective in stimulating interest. Participants saw the opportunity to hear from peers as especially useful both for leadership and for practical learning about how to improve practice:
‘I thought it was really well organized and quite inspirational really, you know, there’s a big culture thing to consider when you’re making quality improvements, and that is, if another consultant [attending physician] has done it, and is signed up to it, that has huge impact.’ (Senior nurse, participant 25)
The training events did, however, encounter some challenges. Some ICUs sent large, eager teams to the training events; others sent one or two individuals, who had sometimes been unwillingly volunteered. Participants understood that they were being asked to collect data and ensure the CVC care bundles were fully implemented, but some appeared to have difficulty in understanding what they were required to do for the non-technical elements of the program. The sessions were inclined to become bogged down in distracting criticisms of the definitions or anxieties about the effort required to collect the data:
‘[The first training session] featured a lot of sniping between consultant microbiologists about how the [program] actually defines associated and [related] CVC-BSIs.’ (Infection control nurse, participant 11)
Participants from different ICUs did not meet again after the training sessions, and efforts to engage them in teleconferences and webinars suffered from low participation rates. This meant that the ICUs involved in Matching Michigan did not have the experience of being part of a collaborative community working together towards shared goals theorized to have occurred in Keystone.
National context
At
Matching Michigan training events, lack of clarity about how well or how consistently the recommended practices were being implemented was presented as a rationale for the program. It was suggested to participants that the program would, through its interventions, promote adherence to the procedures specified in the Department of Health bundles and have a more general impact on patient safety.
Matching Michigan was promoted by the organizers as a clinically led, cooperative program, and participants were assured that their infection data would be used only for learning. However, the national context into which
Matching Michigan was introduced meant that the program faced some scepticism. In contrast to Keystone, a centrally-led emphasis on infection control had been a feature of the English NHS since the mid-2000s (Table
2).
Table 2
History of infection control efforts relevant to central venous catheters
2001 | Mandatory reporting to the Health Protection Agency (HPA) of MRSA bacteraemia. |
2003 | Report of the Chief Medical Officer: Winning ways: guidance to reduce healthcare associated infection in England. |
2004 | |
2005 | DoH Saving Lives program—NHS High Impact Interventions (NHS-HII), modelled on Institute for Healthcare Improvement bundles. |
2006 | Health Act 2006: Department of Health Code of Practice gives new powers of inspection to the Healthcare Commission. Superseded by the Health & Social Care Act 2008 |
2008 | 2008 Health and Social Care Act 2008: required registration with the Care Quality Commission: duty to protect patients against HCAIs. New code of practice. |
| |
2008 | |
2009 | Some NHS trusts participated in CQUIN (Commissioning for Quality and Innovation) schemes that made a percentage of their incomes dependent on demonstrating compliance |
2011 | Mandatory reporting of MRSA and Escherichia coli bacteraemia. |
These government-led efforts were widely seen by staff as harsh and coercive. Matching Michigan was perceived by some as just the latest in a long parade of similar top-down initiatives:
‘There’s a huge amount about [the infection control agenda] that’s dogmatic … with the consequences of not hitting the targets being so dire [and] unforgiving central policies … And I think a lot of it has been driven like that.’ (Consultant, participant 20)
Those who saw Matching Michigan simply as another externally mandated program had little inclination towards genuine engagement. A multi-disciplinary team was appointed to lead the program, including a senior ICU physician who had previously been president of the European Society of Intensive Care Medicine (JB, one of the authors of this article), and a senior clinical intensive care nurse who had been head of the UK Critical Care Nurses’ Association. Despite this, the location of the program in a government agency rather than a professional organization or research collaboration appeared to contribute to an alienating sense of ‘distance’ on the part of some front-line clinicians: contrary to what was theorized to have occurred in Keystone, Matching Michigan was often seen as imposed from outside and lacking in professional ownership. Some participants remained suspicious about the potential for the data to be used for performance management or public shaming purposes:
‘Because the NPSA is a Department of Health arm’s length body, there is certainly a view by a lot of clinical staff that oh, if it’s coming down from there then, you know what that is all about. So I think there is certainly a limitation attached to that.’ (Senior Nurse, participant 104)
‘[What was needed was] a consultation so that it feels like what we are implementing is coming from within, first of all…If we had taken what Matching Michigan had done, not called it Matching Michigan but taken the same things and then applied it as something that came through for example the Intensive Care Society or [another professional society]… it was dumped on top of us from above and we had no option in it.’ (Consultant, participant 3)
Variability in ICUs’ responses to the program
We found considerable variability across individual ICUs in their responses to
Matching Michigan’
s calls for data and implementation of the technical and non-technical interventions it recommended. Interviews and ethnographic observations suggested that local, unit-level responses to the program could be largely distinguished not by the degree of compliance with the program’s requirements, but by the extent to which staff in units attributed their behavior and practices to
Matching Michigan. We identified three characteristic responses:
1
Transformed (one unit) where the program was seen by staff as having produced radical improvement in care.
2
Boosted (five units), where the program was credited with having reinforced existing good practice or supported further improvements.
3
Low Impact (11 units), where staff attributed little of their behavior and practices to the program, instead seeing the influences on what they did as coming from elsewhere.
We did not find a relationship between which cluster ICUs joined and their response to the program, but our study design does not fully exclude this as a possibility. Despite the variability we observed, most staff from across the ICUs believed that both their practices and their responses to the program were ‘normal’ and typical of all. This is likely to have occurred because of limited contact between program participants.
Collecting data on infections
All 17 ICUs in the ethnographic study that were participating in Matching Michigan developed systems for collecting data on CVC bloodstream infection rates, and all but one reported their rates to the program’s central database for at least some months of the program. On 16 of the participating units, staff attributed the introduction of a data collection system to Matching Michigan; the exception was a unit that had begun collecting and feeding back data as part of their involvement in an earlier initiative. For the 11 Low Impact units, establishing the data collection system was the most prominent, or only, feature of their response to the program.
Technical interventions
Because our study was not an audit, and we did not use a structured tool to assess practices, we did not produce precise estimates of compliance with Matching Michigan’s technical standards. Nonetheless, our observations suggested that the evidence-based practices summarized in the CVC care bundles, and the technical interventions to support them, were being implemented across all of the units in our study. On all units where we conducted observations (including the two not participating in Matching Michigan), the care bundles were known to staff and were widely used. Hygiene practices during insertions were mainly very good: chlorhexidine was routinely used to prepare patients’ skin, and handwashing was consistently good. Most ICUs were using a dedicated central line cart and/or a pack; most, too, were using full barrier drapes. Most were aware of the need to monitor CVCs and to remove them as soon as possible, though there was evidence that removal practices varied by individual physician. However, use of a checklist of good practice was variable: only eight of the 17 participating ICUs recognized that a checklist was intended to be used concurrently with CVC insertion (rather than as post hoc audit tools), and only two were fully consistent in using a checklist in this way.
It was usually not possible to identify Matching Michigan as directly influencing technical practices and behaviors; indeed, on any given unit it was difficult to isolate the effects on practices of any single program, initiative or intervention. Instead, most participants gave accounts of an incremental history of improving technical practice. The main exception to this was the Transformed ICU. This unit demonstrated a high level of consistency in compliance with the technical interventions, and staff in this unit explicitly and confidently attributed recent improvements to their participation in Matching Michigan. The five Boosted ICUs also demonstrated high (though not always perfect) reliability in applying the technical interventions. They credited the program with helping them sustain or enhance improvements already made, but did not identify it as the only influence on their behavior in relation to CVC care. In the 11 Low-Impact ICUs and in the two non-participating units, staff did not identify the program as having any influence on their behaviors and practices. In some of these units, compliance was generally high, but was less consistently good on others, and in some units appeared to be strongly influenced by which senior staff were on duty:
[Consultant] said, ‘we just have to be vigilant about thinking about how long the lines have been in.… He is careful to check patients’ [central lines]. It wasn’t something I necessarily saw with the other consultants. I didn’t see that they were very explicitly vigilant about how long the central lines were in,’ (Fieldnotes)
Innovations such as bio-patches, specially coated catheters, and new techniques for aseptic practice were introduced by many units during the period of our observations. On the whole, staff did not attribute these new developments to Matching Michigan, although staff on some of the Boosted and Transformed units described using the program strategically to implement the changes. Telephone interviews suggested similar behaviors:
‘What I am doing is focusing on our own needs though the project. Matching Michigan was merely a hook because it enabled us to do all the things that we wanted to do.’ (Senior nurse, participant 119)
Non-technical interventions
Participants’ understanding of
Matching Michigan’s non-technical interventions (Figure
1), intended to change culture and behavior, was generally weak. ICUs were, as they had been in Keystone, asked to assemble a local
Matching Michigan safety team including nurses, doctors, and senior executives to provide leadership and coordination for the program. On the Transformed and Boosted ICUs, the safety team generally functioned well. On the remaining ICUs, it existed in name only or did not function optimally. In one ICU, it only ever met by email; in many others, its main function was the production of infection data. The safety teams varied in the commitment and enthusiasm they invested in the program. They were often unsure of what the non-technical interventions required them to do, were sceptical of the benefits of the interventions, or did not ensure that the interventions were implemented. Most (11) struggled to involve executives.
Safety Surveys—one of the non-technical interventions intended to provoke local discussion and reflection—were distributed in only three of the 17 ICUs. In very few ICUs was there evidence that ‘learning from one defect a month’ was introduced as a result of Matching Michigan, nor was formalised or standardised practice in relation to daily goal-setting newly implemented. Only three ICUs publicly displayed CVC-BSI data so that staff were aware of their own unit’s rate. Strongly hierarchical and sometimes negative safety cultures persisted on some ICUs:
‘Working with different consultants is that I would say probably without exception as registrars [residents] we do what the consultant tells us to do, certainly where they’ve got very strong personalities, so I’ve put central lines in with just a pair of gloves with the individual who does it like that …because that’s how he does it, he won't let you do any other way and as registrars you're under pressure to do things as the consultant wants you to do them …[or] you know makes the rest of your day miserable.’ (Junior doctor, participant 30)
Influences on variability in responses to the program
Our analysis suggested that much of the units’ variability in response to Matching Michigan could be explained by variability in infection rates and measurement, local histories, and local leadership.
Infection rates and measurement
Program organizers emphasized at training events that the impact of previous efforts to control CVC-BSIs was unknown, given the absence of a national infection data collection system. They also emphasized that unless ICUs knew their own rates they could not be sure that they were providing safe care. It was anticipated that discovery of high rates of infection would stimulate change:
‘What we did not have in this country was a measure of the outcome.... Key stakeholders in the country [were saying to us], ‘well we are fine in England, we are already matching Michigan. We have had the technical interventions. We have got it all sorted in England. So go away.’ And I actually did say to them, ‘Well, could you tell me what your rates of infection are?’ And they said ‘Oh I don’t know.” (Matching Michigan program team member)
The starting rate for the program turned out to be quite different from Keystone, where the initial mean rate was 7.7 BSIs per 1000 catheter days. The first of the four clusters of adult ICUs in Matching Michigan reported an initial mean rate of 3.7 BSI per 1000 catheter days. Though this was less than half the initial rate in Keystone, it did suggest room for improvement. However, each of the three successive clusters joined the program on the trend line, with an initial infection rate similar to the post-intervention level of the preceding cluster. This indicated that substantial improvement occurred outside the program throughout the period it was running. In any given month, almost two-thirds of units across the program were reporting no infections.
In the Transformed and the Boosted ICUs, collecting infection rate data had a generally positive impact on practice. These units established relatively robust data collection systems [
20] and the data were generally accepted by staff locally as credible. Discovery of a low infection rate did not necessarily undermine the program’s aims; low rates on these units were used to celebrate and reinforce good practice. Discovering previously unrecognized high rates of CVC-BSIs, on the other hand, did have the intended program effect of driving change on these units:
‘If I’m honest right before we started, we didn’t think we were that bad.… We thought, you know, [we] don’t really have a problem with central line infections. But I think what it was, nobody ever looked to see whether we were any good … and when we compared our infection rates, actually they were far worse than any of us ever realized.’ (Senior nurse, participant 43)
‘It was hard to sit and have the error of your ways pointed out when you actually already knew [how things ought to be done]. So, I think enough of us felt like that we put the wheels in motion really.’ (Consultant, participant 42)
However, Matching Michigan data on infection rates could have effects that were in the opposite direction from those intended. Low-impact units varied substantially in the extent to which they established robust local data collection systems and how far the data were regarded as credible by clinicians locally. On some units, evidence of high infection rates was dismissed as poor quality data, and change was thus stymied. On some others, low infection rates were taken as evidence that no change was needed—but the data were not always collected accurately enough to justify such a conclusion. This meant that sometimes very hierarchical cultures that were not fully supportive of patient safety were reinforced, and that opportunities for improvements in practice went unrecognized.
Local histories
Local histories of efforts to control CVC-BSIs were deeply implicated in the differing responses to the program. In the Transformed unit, previous attempts to improve practice had been largely ineffectual. The program was seen by staff locally has having provided the tools and techniques they needed to make change, where nothing had seemed to work before:
[Consultant says] ‘Matching Michigan has genuinely made a real difference here … We’ve seen some really quite big changes around here … you can see real actual evidence of improvement in patient outcomes.’ (Fieldnotes)
On the five Boosted units, the program was absorbed as part of a local narrative of cumulative improvement. Staff on these units described having already recognized, long before Matching Michigan, that central line infections were a problem that they needed to tackle. These units had made significant gains before Matching Michigan, but recognized that there was still some room for further improvement. On these units, local leaders were keen to drive improvement, but had sometimes experienced barriers or resistance to change. These individuals explained how the program helped them consolidate their gains or make new improvements. Having a national, centrally led program was helpful because it enabled them to ask for resources, demonstrate that proposed changes were evidence based, persuade reluctant colleagues that conformity with good practice was now compulsory, and learn how their infection rates compared with other units:
‘We already had things like a lines trolley in place and I think that worked reasonably well. Perhaps wasn’t always stocked as well as it should be.... Being involved in Matching Michigan project certainly sort of tightened up vigilance.… We certainly didn’t have an insertion checklist, and this certainly wasn’t the sort of the culture whereby everybody felt that line infections were pretty much preventable. [The change was] the continuous measurement system, … actually doing the checklist and publishing the results that we were getting back.’ (Consultant, participant 114)
In the Low-Impact units, a few staff were resentful or hostile to the program, but more commonly they expressed apathy, exasperation, and bewilderment. On these units, staff argued that they had already invested heavily in changing practices to reduce CVC-BSIs in response to previous policy initiatives, that the prevalence of CVC-BSIs had already greatly diminished, and that the program was a largely superfluous data collection exercise. ICU staff on the Low-Impact units were often unsure what was new or distinctive about Matching Michigan, given that the technical practices were the same as those that had already been adopted as policy. Many saw it as addressing a problem that they believed had already been solved:
‘Compared to what we were already doing it seemed to be that wearing a hat [was the only difference] basically. We were already going over to packs anyway.’ (Consultant, participant 110)
Staff reported that they could not understand why CVC-BSIs in ICUs were being targeted by a dedicated program, when what they saw as other, more pressing problems—such as ventilator-acquired pneumonia or CVCs in non-ICU settings—were being neglected. Some saw the program as a failure to respect what had already been achieved by ICUs. Even its title caused resentment: in one unit, a file about the program was defaced, with ‘Matching Michigan’ scribbled out and ‘Exceeding Michigan’ written in:
‘There is sort of an attitude that, actually, it’s come in a little too late.’ (Senior nurse, participant 88)
What was notable about these Low Impact units was that these views were held not only by the staff ‘on the ground,’ but often also by those on the safety team set up locally to implement the program. These local leads were not always fully convinced of the need for, or value of, Matching Michigan.
Leading the program into practice: the importance of creating local coalition and consensus
Perhaps the single most important influence on program response by individual units—either in promoting or resisting change—was the extent of consensus and coalition among the senior medical and nursing staff on individual ICUs. The commitment, characteristics, and skills of local leads were pivotal. Transforming or boosting of efforts was most likely to occur when those locally charged with implementation were sincere in their beliefs about the value of the program, were able to create transdisciplinary alliances, had local credibility among peers, were prepared to tolerate debate but exercise firmness, and used multiple tactics including role modelling, persuasion, sanctioning, reminders, and constant feedback:
‘Cultural change is the biggest threat, because all of a sudden it fundamentally means that what you’ve been doing so far is maybe wrong, or people don’t value what you’ve done for the last 15 years. People think ‘oh, there’s the smartarse telling us how we’re supposed to do things.’ So there were a lot of discussions and persuasion.’ (Consultant, participant 106)
[Consultant says] ‘I think it’s been successful because it’s a unifying program, it’s one of the few things that we’ve done that hasn’t been just a doctor thing, or just a nurse thing, it’s involved the doctors and the nurses together.’ (Fieldnotes)
Authoritative and unwavering support from senior consultants was especially important in enabling nurses to act as a disciplinary force for junior doctors, who performed most CVC insertions:
‘So the fact that the lead consultant was passionate about it helped us to bring about change. [We could] actually [say to the junior doctors,] ‘You will gown up! You will put a mask on!’ And that was coming from [the lead consultant] as much as it was from [the nurses].’ (ICU Outreach Nurse, participant 111)
In the transformed unit, collecting data for the program revealed previously unrecognized high rates of CVC-BSIs. This shocked unit staff into action; a local leadership team emerged who used the program’s tools and techniques to secure change, and collecting data over time confirmed that the interventions were effective in reducing rates to zero from an initial high rate. Matching Michigan, staff in this unit reported, was the first that came from and was owned by the ICU, rather than being imposed from the outside. Led by a young and determined entrepreneurial consultant, a team was developed that crossed intra-professional hierarchies and inter-professional work domains. This individual and his consultant colleagues went about rebuilding the unit culture according to the program’s goals. He described how he and his colleagues modelled Matching Michigan’s preferred practice for the insertion of CVCs and insisted on compliance from junior doctors. Both junior and senior medical trainees, as well as nurses were included in the data collection process. He used the introduction of the checklist and observation as ways of flattening unit hierarchies and empowering nurses and junior medics to act on any breaches of aseptic technique they observed:
In the very beginning we made sure that it was the charge nurse, or one of the sisters, who would take the role of the observer. So [we chose someone who would] feel more confident, and was actively encouraged to interrupt if there was something [wrong with our technique]…[We did this] so that the juniors could see. And if one of the consultants was putting the central line in we would make sure that [a nurse observed] us as well. To make sure that people see it applies for us exactly as for anybody else…it’s the same rules apply for everybody. If we put a line in or [one of the junior medics does], it doesn’t matter.’ (Consultant)
In the units where Matching Michigan had less impact, senior consultants were not persuaded that all elements of the program were grounded in high quality evidence, saw it as an illegitimate policy or bureaucratic intrusion into professional work, or deemed it irrelevant to their concerns and interests. Apparently minor issues—such as the recommendation to wear hats and masks during CVC insertion—caused irritation and resistance in some settings because they were perceived to lack sound evidence. If local program leads did not successfully build consensus among senior staff that central line infections were a problem, and that Matching Michigan was the right answer to the problem, the program was inclined to stall:
‘I recall from reading that there wasn’t anything that made me change my practice. I haven’t seen any people [using hats and masks] and I certainly wouldn’t you know ask my juniors to do that.’ (Consultant, participant 87)
‘It can be a struggle. It can even be difficult if you’ve got one [consultant] who’s a real advocate and saying, ‘Yeah this is the best thing since sliced bread!’ And then you’ve the six others who are like, ‘What a load of crap, we’re not using it.” (Nurse, participant 91)
Executive involvement that generated enthusiasm, conferred authority, and allocated resources was helpful to the implementation of the program in local settings. Where executive involvement was limited to exhorting the ICUs to produce ‘the numbers [of infections],’ it reinforced a view of the program as a mandated, performance management national audit, and risked undermining the program’s aims:
‘They were getting e-mails [from the hospital executive] saying, ‘Why haven’t you submitted your data? So you go back to just tick boxing.… There’s a complete lack of interest now in line insertion.’ (Senior nurse, participant 40)