Background
Methods
Sampling and recruitment
Data collection and management
Analysis approach
Results
MAIN FINDING 1: Implementation supports provided by the government or other organisations were identical for each QBP and did not address differences in hospitals’ highly variable capacity to manage change
“Some of the hospitals – where we know they’ve invested in developing their team, developing their capacities for quality improvement, their board is already focused on quality improvement, they take their quality improvement process seriously, the quality committee is engaged at the board level, leadership is engaged – in that kind of a culture, it will be easier for them to take on a new priority QBP implementation than in a hospital that hasn’t done that ground work within their organisation already.”Source: QBP Designer, Ministry of Health, Level 1 – 006
“The capacity to even think this through, have the data capacity, I think it’s highly variable. I think the big academic hospitals have been on it. They’ve got mature decision support systems, they’ve got mature case costing systems, they’ve got in-depth clinical leadership, and many of them were on the panels. I think that they are in a very different place than many of the other community hospitals that really have very, very thin decision support capability, and not necessarily a tonne of depth around quality and clinical quality.”Source: Senior Executive, QBP Adoption Supporter, Level 2 – 015
“P: ALC [Alternative Level of Care] numbers have gone up in the last couple of weeks so I'm challenging my teams to do something about that. R: That seems to be the thing that you're spending most of your attention on. It’s not QBPs so much… P: Definitely, no. R: Yeah, okay, I understand. P: If I can solve that…QBPs is the least of my worries at this time.”Source: Senior Executive, Hospital, Level 3 – 026
“… in terms of the systems of data analytics that are required to really do it well… we don’t have some of the key enablers that will be necessary to really get us to perfection or a lot closer to perfection than we are today.”Source: Senior Executive, Hospital, Level 3 – 024
Sub-finding 1a: Some QBPs were more difficult to implement than others, requiring unique adoption supports to support change management
“Then there are two other QBP things that we’re wrestling with. One is heart failure, and the other one is pneumonia. We’ve had a lot of trouble in this hospital, getting people to put patients into those QBPs. R: What do you mean by, put patients into those QBPs? P: Enrol them into the pathway. R: Okay, so follow the pathway? P: Yeah, because people come in, and they don’t say, I’ve got heart failure, they come in and say, I’ve got shortness of breath, so we need a shortness of breath pathway. Then blood work is done, an x-ray is done, we figure out, oh, it was actually an infarct that caused the shortness of breath, or it was pneumonia, or it was heart failure, or it was a chest infection that produced a bit of heart failure, so where do you put that patient? When you get away from a very well-described thing like renal failure, a hip replacement, we’ve had a lot of difficulty, and I know other places have had a lot of difficulty with these medical ones.”Source: Senior Executive, Hospital, Level 3 – 028
“When you have the nice, neatly defined episode of care, it’s pretty easy, like hip fractures, knees, and surgical things. They lend themselves very well to QBPs. It works well. The medical ones are more of a challenge, like CHF [congestive heart failure], COPD [chronic obstructive pulmonary disease]. The hospital is only a small component of that person’s healthcare journey… the way our system is designed and organised, we have precious little influence or control over probably 90% of their healthcare experience.”Source: Senior Executive, Hospital, Level 3 – 022
Sub-finding 1b: External QI adoption supports enabled QBP implementation
“I think a key enabler is having some sort of an agency or focus where you know that you’ve already got traction in the system. For example, for our cardiac and vascular QBPs, we already work with those end users and stakeholders. And so, when it came time for us to go out and do some dissemination and some road shows to each of the hospitals, we already had those connections.”Source: Senior Executive, QBP Adoption Supporter, Level 2 – 021
“We know that the first wave of QBPs that were created, which were, of course, closely linked to the original wait times activity, that those are in very advanced stages of adoption. But, subsequent QBPs, subsequent batteries of QBPs, at a minimum we know are implemented in a highly non-standardised manner across the province, and we know that in some parts of the province they’re not implemented at all.”Source: Senior Executive QBP Adoption Supporter, Level 2 – 020
“Having some of those provincial organisations out there assisting hospitals with less internal capacity is a good thing, and I think the OHA [Ontario Hospital Association] has also started to do some more analytics support for its organisations, especially small hospitals. So, I think all those are good strategies at the provincial level, to ensure people don’t get left behind.”Source: Senior Executive, QBP Adoption Supporter, Level 1 – 014
“All of the things that the Ontario Stroke Network has been able to do around the stroke QBP are just totally the sorts of things that have contributed to success in terms of adopting clinical best practices. One of the things to remember is that there’s nothing magical about the stroke handbook. Pretty much all the recommendations from that were taken from the existing Heart & Stroke guideline. But, I think the Stroke Network was very adept in terms of capitalising on the QBP introduction, to sort of make this case for clinical practice change, and so they did very well as a result whereas, if you look at COPD, there was much less of that going on. And, I think pneumonia was…a perfect example in that there’s no Ontario pneumonia network. There wasn’t much activity going on in that area, there’s no pneumonia scorecard, any of this stuff brought in feedback. And, so there’s actually probably been very little impact of that pneumonia QBP handbook in terms of clinical practice change.”Source: Source: Senior Executive, QBP Adoption Supporter, Level 1 – 014
Sub-finding 1c: Adoption supports were lacking to facilitate collaboration among physicians, and between physicians and the broader clinical care teams, in applying best practice episode of care pathways
“My guess is most clinicians have never read a QBP clinical handbook. So, they can’t make change if people don’t read them.”Source: Senior Executive, QBP Adoption Supporter, Level 2 – 018
“P: I’m guessing I wouldn’t have to walk very far down the hall before I could find a doctor who would be grumpy about this or that in terms of what is in there. But I don’t think that is the big problem. The big problem is, what mechanisms and what tools do we have in place to relentlessly drive towards compliance standardisation? And it’s very difficult because these guys, they don’t work for me. R: The physicians? P: Yes. R: (Laughs.) Right, I suppose. P: It’s the truth. They don’t. R: They are not employees, in that sense, you mean? P: Not only are they not employees, they are essentially tenured. So, not only is it very difficult to give them direction, it’s very difficult to do anything when they don’t comply with it.”Source: Senior Executive, Hospital, Level 3 – 024
“And so, that is our challenge, and we want to do better. We want to do well. We’re struggling. We keep going back and we meet with one group of docs, another group of docs, our Emergency docs. And to date, we have not found the magic to trigger it because it seems so complex, like I said, with other systems involved. So, we’re going to continue trying because we want to do well, but it is very difficult.”Source: Senior Executive, Hospital, Level 3 – 042
“I think what I originally was hoping for, looking at the QBPs, I think there was hope to have a little stronger clinical engagement and how do you engage clinicians better and use existing infrastructures in the province to help create communities of practice. I don’t think that really materialised. And I think that is what we’re seeing now. It’s great that you throw a best practice at me. It’s great that you show me the baseline results. It’s great that you pay me an average price. But I have no idea how to move this forward.”Source: QBP Designer, Ministry of Health, Level 1 – 005
MAIN FINDING 2: There was a mismatch between the provincial adoption supports developed in response to the policy (handbooks with evidence-based episode of care pathways) and those needed by different hospital leadership groups
“If you pull back the camera lens, health system funding reform, including quality-based procedures, was entirely designed and primarily implemented by the Ministry in a command-and-control function with virtually no genuine engagement and collaboration with the very sector that is supposed to implement the change. There was a very wide range of meetings and groups that came together under Ministry sponsorship from time to time, but they were inauthentic and simply masquerading as mechanisms of, again, consultation, collection of evidence and guidance, and refinement of tactic and strategy. It’s in the last year the Ministry has come to genuinely appreciate that the trajectory of HSFR [Health System Funding Reform] in general is no longer viable, and that a fundamental recalibration is needed. And, so they’ve abandoned the old committee structure, which was representational only, it had no other real authentic function, and we’ve reconvened a new governance collaborative.”Source: Senior Executive, QBP Adoption Supporter, Level 2 – 020
“You can’t just write a handbook, throw the brick over the fence and assume your job is done. And, there are pockets of change management support for different kinds of QBPs in the province. The OHA [Ontario Hospital Association], for instance, partnered with the OMA [Ontario Medical Association] and did regional programming for clinicians on one type of QBP last year. But, the scale of change management supports that are required on a sustained and system basis are enormous. The Ministry deserves full criticism of this. They paid no attention to the change management requirements of QBP implementation.”Source: Senior Executive, QBP Adoption Supporter, Level 2 – 020
“But to some degree there has not been change management dollars behind change. And fundamentally at the end of the day, I think most people understand that change actually costs to implement and then you get the benefits down the line.”Source: Senior Executive, Hospital, Level 3 – 023
Sub-finding 2a: Hospital finance administrators lacked adoption supports aimed at informing financial operations, which undermined their role in change management and impeded QBP implementation
“I think the hospitals are pushing back and saying, slow down, because this is tougher to manage than we thought and it’s got all kinds of complication in the implementation. And even if they are done very, very well, the capacity of hospitals to really get their arms around what they should be doing with them is a process. There is quite variable capacity in hospitals to deal with that, I think. So, I think hospitals are pushing back and saying, slow down. I think the execution needs to be improved for the whole QBP and health system funding reform process.”Source: Senior Executive, QBP Adoption Supporter, Level 2 – 013
“The pushback we’re getting is largely from administrators, right? This is too much, too soon, too fast. I think I’m hearing it largely from the administrative side of the house where they’re seeing the risks of not being able to predict the volumes, not being able to plan, having to change processes, all this kind of thing.”Source: QBP Designer, Ministry of Health, Level 1 – 006
“Although nobody would ever say it this way, I think what it really means is, if you want us to figure out how to use these new funding formulas to maximise our revenue, we need time. And they’re really complicated and these incentives sometimes act in confusing and conflicting ways, particularly because we didn’t shift to just the straight up rate by volume funding mechanism. If we had, it would have been very clear, you’ve got to identify your admissions, or your costs are greater than your revenues, and work to reduce costs on those, and maybe reduce volumes there.”Source: Senior Executive, Adoption Supporter, Level 2 – 018
“No one has shown me any information that would say, you know what, we’ve mapped out and costed the perfect care pathway for the CMG [case mix group], and when you do that, this is what you should have. You should have a length of stay at this. You should have nursing coverage at that. You should have this and that. In a perfect world, this would be a cost structure. It’s still based on average performance across the province, so it’s a bit of an anomaly to think that we’ve actually tried to provide all the appropriate operational guidance, clinical guidance, to hit the mark that was, I think, fully intended at the outset.”Source: Senior Executive, Hospital, Level 3 – 023
Sub-finding 2b: There was no process to mobilise and support physician leaders and promote them as champions and change agents who could enable system wide readiness for change and adoption of QBPs
“Yes, the handbooks are useful, but what you really need is having strong clinical champions, having a network, having timely data on quality performance that you can provide to hospitals.” Source: QBP Designer, Adoption Support Agency, Level 1 – 014
“I think they vary by QBP to the extent that there are different levels of clinical engagement and support in these different service areas in Ontario. And in some areas like cancer we have a really mature and robust clinical champion and a formalised network of clinical leads that are able to do knowledge transfer really effectively through Cancer Care Ontario. That’s an outlier. And then we have less formalised clinical leadership networks, not only paid physicians, clinical leads, and then in some areas we really don’t have any infrastructure like that at all in the province. To that extent it varies because of the way that we’ve supported clinicians in being champions and engaging with their peers in knowledge transfer and exchange. And that’s just really varied by clinical area in the province. I guess I would say again I don’t know that it’s about personalities as much as to what degree we’ve supported that clinical peer knowledge translation function in the province.”Source: QBP Designer, Ministry of Health, Level 1 – 006
“R: How long does it take, or did it take, to implement a new QBP from when it’s announced? P: Oh. Some of them were really... R: That was a big groan. P: Yeah, some of them were lickety-split easy and some of them hard, hard, hard, hard, hard. R: Can you differentiate between the easy and the hard and why some were easier and some were hard? P: Uh, yeah, yeah, yeah, I think it generally had to do with how much clinical engagement and clinical consensus you could generate and where you had the strongest physician champions. The physician champions were critical because… I mean, at the end of the day, we would have the support of the whole team, absolutely, but they don’t admit, discharge, or order.”Source: Senior Executive, Hospital, Level 3 – 043
Sub-finding 2c: Unique adoption supports intended for particular hospital leadership groups were implemented too late or not at all
“In terms of the systems of data analytics that are required to really do it well, and that hospitals and healthcare are lacking…we don’t have some of the key enablers that will be necessary to really get us to perfection or a lot closer to perfection than we are today.”Source: Senior Executive, Hospital, Level 3 – 024
“They would rather hand-write all the orders, and for the life of me, I can’t figure out why. They would rather scrawl and write exactly what they want, rather than go through this lovely pre-printed thing, check, check, yes, no, this is how it should be, and I don’t understand it. We have put them out over and over again. R: The pathway? P: We have launched... the pathways, yes R: The order sets? P: The order sets. We have launched them. We have re-launched them. We have gone around and we have talked to our internal medicine specialists, our hospitalists, our cardiologists and everybody and say, they are right here in Emerg. Oh, you know, when I go there and open the door, they are never there. And we go back to Emerg and we put them in red, and then we put up big signs, flashing light bulbs, like the blue light special. We do everything we can, and they won’t get used.”Source: Senior Executive, Hospital, Level 3 – 042
“You’re in the process of already creating standardised order sets for those. So, much of what drives the care that the patient gets is based on what the physician orders and so these order sets are the tool that really enhances the success of the QBP.”Source: Senior Executive, Hospital, Level 3 – 030
Sub-finding 2d: Monitoring and evaluation tools to assess the uptake of QBPs, and their impact, were not part of the universally available adoption supports
“There is virtually no provincial understanding about spread and scale of QBP adoption but for some heat maps that have been sponsored by Local Health Integration Networks. That’s the only mechanism that allows us to evaluate the scale of QBP adoption in the province.”Source: Senior Executive, Adoption Supporter, Level 2 – 020
“I don’t know the measures that are out there to even tell me that some of them have been implemented, other than a process measure that says someone wrote a handbook. Nobody is measuring the quality of delivery according to best practice, so how would we know?”Source: Senior Executive, Adoption Supporter, Level 2 – 021
“There are some indicators, yes, absolutely, but there are many indicators that don’t exist. And, then this gets to a more kind of systemic issue, which is there’s a sense that the sector is overwhelmed in that it’s asked to conduct so much performance measurement, and all of it is given equal importance, that the sector simply doesn’t know which priority the government feels is more significant over the other. Some indicators are publicly reported, others are for boards only. Others are for management teams only. But, it’s not just QBP indicators, but there are hundreds and hundreds of indicators for completely different quality improvement, quality assurance initiatives that compete with QBP, and so you have a confounding situation on your hands.” Source: Senior Executive, Adoption Supporter, Level 2 – 020
“I’m not sure that anybody really has told us what the desired effect is. I don’t mean to be flippant but I just think it’s assumed all these wonderful things will happen. I think it’s early days to tell, to tell you the truth. I’m not sure they’re going to be measured locally. I think it’s almost at the system level. I would say to you at the gross system level for a primary unilateral hip or unilateral knee, are your costs stable, has your length of stay come down, and what are your outcomes? R: Do you collect those data, the ones you just mentioned? P: There’s very little outcome data available to us for anything that we do with patients. That's one of the things I think the whole system grapples with. We have lots of process indicators. We don’t have a lot of outcome indicators.”Source: Senior Executive, Hospital, Level 3 – 022
MAIN FINDING 3: In response to QBPs, hospitals sometimes focused more on containing the costs of care than on improving adherence to each of the evidence-based clinical recommendations in the episode of care pathways, potentially undermining the policy intent
“People are saying oh, it’s just too complex, too complicated, I’ll leave it to the finance people to deal with. And, so, I think you kind of lose that emphasis on quality once you begin to disengage, I think, for people that are in that quality side of the house.”Senior Executive, Hospital, Level 1 – 008
“We use this network of physicians, and clinical leadership, and stuff like that, and we use them helping us guide the strategy of all this, but the ministry uses the CFOs [Chief Financial Officers] to help guide all the strategy. They’re more interested in, make it simple, make sure my hospital doesn’t get hosed somehow, and I get way more money than I used to have.”Source: Senior Executive, Adoption Supporter, Level 2 – 017
“… one of those key things this has done has been a catalyst for conversations between the clinical and administrative sides of the hospital. But having said that, they’re not necessarily seeing eye to-eye, right? (Laughter) So, I think it surfaces as some tensions as well.”Source: QBP Designer, Ministry of Health, Level 1 – 006
“So, we got a reduction in volume of cases, but the math would say that they were less complex cases so they should cost you less and we’re going to give you less funding. But the reality is a hip patient still went into a bed, there was still a nurse, there was still a meal delivered, there were still all these things. The length of stay didn’t drop the bottom out. So, it was a very mathematical thing to suggest that it cost us less to do the same number of hips the next year that we did last year, and yet funding went down, right?”Source: Senior Executive, Hospital, Level 3 – 023
“There’s lots of change, for sure, right? And, layoffs and closure of… winding down of, FTE [full-term employment] positions is absolutely one of them, and so is hiring new positions and new people. So, it’s never fair to think that… it might be a cut in and of itself in that moment in time, but at an organisational basis and on a regional basis and on a provincial basis, there’s still continued growth overall. So, there’s change happening, but to the employees who it affects, yeah, it’s pretty darned important.”Source: Senior Executive, Adoption Supporter, Level 2 – 020
“Typically what happens is there’s a QBP handbook comes out and the folks start to take a run at it, and they go, ‘our length of stay is a little high’ and things like that, and they do go to work on that and they bring it in line.”Source: Senior Executive, Hospital, Level 3 – 023
“I think there’s probably some indicators we’re getting better outcomes. My only point is, nobody has that conversation. When you’re actually a hospital [executive]… when we get our numbers, I don’t hear everybody rushing into my office saying, oh my God, we did so well on QBPs this year, we had such better outcomes of patient experience. I hear from my colleagues, oh, how did you do on QBPs, did you lose or did you gain money. Then, it’s not a quality indicator.”Source: Senior Executive, Hospital, Level 3 – 031
“If you don’t deliver the QBPs, you have to give the money back. If you deliver more than what you are funded for, you are not paid for it. So, for planned care, yes, we can cap at that. For unplanned care, we have no ability to cap. R: So, what do you do? Where do you get the money? P: Well, it has to come from the hospital. We have to balance the bottom line. R: So, you take it out of the globe? P: We take it out of the globe.”Source: Senior Executive, Hospital, Level 3 – 025
Discussion
Internal capacity for change | |||
---|---|---|---|
High | Low | ||
Complexity of change | High | Less external support | Most external support |
Low | Least external support | Least external support |