Background
-
What are the influences on implementation of guidelines?
-
How useful are guidelines perceived to be?
-
To what extent is the linear evidence-guidelines-practice model fit for purpose?
Methods
Participants and recruitment
-
A unitary authority in the north of England
-
An outer London borough
-
A two-tier authority with rural areas
-
An inner London borough
Role | Site 1 | Site 2 | Site 3 | Site 4 |
---|---|---|---|---|
Director of Public Health Public health team Clinical Commissioning Group Other LA departments District council Councillor | 1 2 1 4 0 2 | 1 2 1 2 0 1 | 1 2 2 0 2 2 | 1 3 0 0 0 1 |
Total | 10 | 7 | 9 | 5 |
Procedure
Interview schedule development
Data collection and transcription
Participant ID | Role | Pre-2013 role | Wave |
---|---|---|---|
1 Off 1 | Director of Public Health | Other health | 2 |
1 Off 2 1 Off 3 | Public health team Public health team | NHS NHS | 2 3 |
1 Off 4 | Clinical Commissioning Group | – | 3 |
1 Off 5 1 Off 6 1 Off 7 1 Off 8 | Other LA departments Other LA departments Other LA departments Other LA departments | Local government Local government Local government Local government | 2 3 3 3 |
1 Cllr 1 1 Cllr 2 | Councillor Councillor | – – | 2 2 |
2 Off 1 | Director of Public Health | NHS | 1 |
2 Off 3 2 Off 4 | Public Health team Public Health team | NHS NHS | 3 1 |
2 Off 5 | Clinical Commissioning Group | – | 2 |
2 Off 6 2 Off 7 | Other LA departments Other LA departments | Local government NHS | 2 3 |
2 Cllr 2 | Councillor | – | 1 |
3 Off 1 | Director of Public Health | Other | 1 |
3 Off 2 3 Off 3 | Public health team Public health team | NHS NHS | 1 1 |
3 Off 4 3 Off 5 | District council District council | Other Local government | 2 3 |
3 Off 6 3 Off 7 | Clinical Commissioning Group Clinical Commissioning Group | – – | 2 3 |
3 Cllr 1 3 Cllr 2 | Councillor Councillor | – – | 3 2 |
4 Off 1 | Director of Public Health | Other | 1 |
4 Off 2 4 Off 3 4 Off 4 | Public health team Public health team Public health team | Local government NHS Other health | 1 1 1 |
4 Cllr 1 | Councillor | – | 1 |
Data analysis
Results
Role of context in implementation
I don’t think it’s as straightforward as to say it was all implemented, and therefore it was all evidence-based. (Site 2, Off 3)
Context is important
And I know putting it on the web’s brilliant and I do think the pathways do help, but the only problem with the pathways is you see the recommendation of it out of context and its bias of recommendations (don’t serve who they are supposed to) (Site 4, Off 4)
We tend to talk about evidence-based decision making…Which is part of this delusory narrative… there are other things that have a place…A big bit of which, equally important, is public preference… And there are other things, in that model, such as, what resources you’ve got…Because there’s no point in the guidance being separated from capacity…So, I think, one of the things that would help in the future, is, for a bit more work to be done, on that conceptual model. (Site 1, Off 1)The challenge for NICE is that this is not an environment where you can be driven totally by evidence or by science (Site 2, Off 1)The world’s more complex, than the NICE evidence base, really, can do it. (Site 1, Off 1)
In relation to any of these major public health issues, it’s a range of different things and, kind of, no one knows the exact balance because that answer will be different with different people in different communities. (Site 4, Cllr 1)
What’s the difference between, this kind of platonic, perfect world, in which evidence-based decision making happens… And everything is rational. And, the real world, where, a whole lot of other factors, become part of the decision making process. And, what you’re trying to do is, balance, and make those as effective and rational as possible, within certain parameters…And that situational decision making, is something which we haven’t got a model for…The model is based, on here’s the evidence, everybody should do what the evidence says, because the evidence is right, and this is what it says. And, it’s kind of really, I just think, we should be a bit more sophisticated, in the traction model, or something. I can’t, it’s… I’m not quite describing it properly, but the way it gets taken up in the real world (Site 1, Off 1)
This is about the evidence we use, kind of, transferring with us into local authorities, where the evidence needs to speak the different language that we need to speak (Site 1, Off 2)
Limited resources
I met with the obesity team last week in the council and they said they have no time, they spend their whole time reviewing contracts and planning to get new contracts set up for 2016. So if I want to do anything in driving forward the obesity strategy, I must provide management resource from the CCG. (Site 1, Off 5)
For the last maybe 20, 30 years, funding has been quite stable, but then in the last five years, we’ve had to change rapidly and do things very differently, and sometimes, out of adversity, you actually come up with better solutions (Site 1, Off 5)
Political influences
I know councils don't like being told what to do. They don’t like dicta and whereas the NHS might be able to say through its hierarchy of command and control, NICE has said this is, you know, this has passed through the hurdles, it must be implemented within three months, they can do that. You can't tell a local authority to do that. They say, well, thank you for your opinion. We'll weigh it up carefully and we'll do what our voters tell us. Welcome to democracy. (Site 3, Off 1)1
The NHS has a very strong narrative, that it tells itself, that it’s evidence-based, and scientific, and rigorous, and everything like that. And that local government decides thing on whim, and political priorities. My experience of coming over, in to local government, was actually, that the truth of that is much more complex (Site 1, Off 1)
Well, as you know, every politician works on an anecdote. We have to use evidence either to support or refute the anecdote and sometimes you get overruled. If you manage to … ensure the evidence base is followed 75% to 85% of the time probably in this environment, we’re doing pretty well. (Site 4, Off 1)
How useful guidelines are perceived to be
Evidence misses complexity and scale
I would say, is that some people have a view, that NICE guidance is not realistic, to the local situation, sometimes. (Site 2, Off 4)
There was a really broad recommendation and it was about doing the system change stuff, but because the evidence was based on a big system change, we know that worked, but actually what you want to understand is smaller scale stuff. What difference does this particular intervention do? And it’s quite hard to do that if, you know, it’s been implemented as part of the big package of things, and also the context is just really important. (Site 4, Off 4)
National evidence is important in terms of big areas, volume areas. I think where it is less effective is being able to allow you to translate that to your local population, and, depending on your population mix, something which is a big issue nationally may not necessarily translate across to your local population. So, let’s take, for example, in xxxx, you know, around 26% ethnic minority communities; that isn’t the picture nationally. So, you see, the problem that you’ve got is to meet and understand and reflect your local needs. (Site 1, Off 5)
It will give you very sophisticated analysis of, how to increase x target group, to three times 30 per week. But, it misses some of the big, systemic questions. It’s always down to a small project level, intervention level. It doesn’t look at mainstream world, changing, the way it is. (Site 1, Off 1)
I try to get my Health Intelligence Team when they’re looking at it, to actually look at the research that was underpinning it as well, because I think sometimes if it’s based on these big studies and there are all these big, grand recommendations you think, actually that’s not always helpful locally when you’re doing some bits of it but you just want to know, will this extra thing work or not? (Site 4, Off 4)
Research evidence is out of date
The councillors knew exactly what was going on, how it would work, who, or how it was working, how it was evolving. It was in real time, happened in the last 12 months. The problem with NICE guidance is, it would never capture that sort of phenomenon. …NICE guidance, by definition, can only include evidence, that’s based on well-funded research, that appears in peer reviewed journals, that meets research threshold and criteria, that by definition, by the time it ends up in an effectiveness review, is three years old, at least. So, it’s always behind time, influenced by who’s funded the research, and the way they view and frame their world, and actually, their economic interest as well, in terms of their clinical guidance. So already, you’ve got a research time, and framing bias, in the NICE guidance. And, that’s very problematic, because, in, when you’re trying to solve complex problems, in real time, in a democratic setting, you need a different way of working, and a different, a different framing of solutions, and what evidence counts (Site 1, Off 1)
Specifying process not just outcome
I think sometimes NICE recommendations can seem a little bit vague at times, quite abstract (Site 2, Off 7)I think sometimes the recommendations aren’t specific enough. It’s really not clear. …some of the recommendations sometimes come across as an outcome that you want to achieve, whereas I think practically quite often you want recommendations about what activity would give you that outcome. (Site 4, Off 4)
I remember when I went reading guidance and thinking, god this is so turgid and just like thinking, I’m just not going to read it. Now I’ve got used to it, I don’t really have a problem … just think it feels a bit like an academic report. (Site 4, Off 4)
I think if you can ground them… more sort of practical examples, case studies, things like that, but again it’s always got to be concise (Site 2, Off 7)
Using local evidence
I don’t necessarily think [the local authority understanding of what evidence is] is always a different standards thing, it's just a different interpretation of how significant evidence should play in decision making. (Site 4, Off 3)People… will dismiss it, because they feel it was done under… research conditions, and doesn’t really apply to the real world. (Site 2, Off 4)
Local evidence provides context for national recommendations
We’re doing something similar looking at the obesity pathway, because we’re bringing in people from… because obviously this is a system wide thing that needs to happen, and we have used guidance to come up with recommendations, but we’ve invited them to feed in. So, you know, this is what the guidance says, what does the local evidence say? So it does help us to have conversations about things, I think, and they have different types of evidence sometimes as well. (Site 4, Off 4)We will go to what happens in [this part of] London, who else is doing something, or what happens in, I think it’s xxxx or xxxx, which borough is similar demographically to [this borough], and what’s happening in these areas. So we will either follow the national Guidance or we will look at other areas and see what best practice we can then apply to xxxx (Site 2, Off 6)We’ve got a very diverse borough. It’s quite different demographically to the rest of xxxx, the rest of the county. I think there’s something about taking the national evidence, taking the national guidance and then always asking the question what does this mean for us. Does this speak to our community, you know? Does this take into account ethnicity? Does this take into account our levels of deprivation, etc? Does this apply in this case? And often it does. Sometimes you might think, actually, there’s shifts we need to make. It’s about localising the national. I probably do start with the national. (Site 2, Off 2)
It’s how you transfer that into a local area, I think that’s the art of public health, I guess, because there are – does it – is it – is – can you apply it to the local area? And I think you would always use that as a framework to go and then audit, check what we’re doing locally, is it in line with this, that and the other? So you check all that out. And where there is a difference it’s understanding, why is there a difference? (Site 1, Off 3)
So do acknowledge that any local intelligence from research or any local engagement activities or surveys or fieldwork; it’s less strong because it’s not been through that rigour of process, but valid as a piece of… for what it is in terms of, this is public opinion, service user feedback, it’s a sense of opinion or feeling around this topic of what needs to be done and what the priority areas might be to be addressed. (Site 1, Off 3)
Local evidence can be more persuasive than national recommendations
I think, very practical evidence of, you know, showing me a, sort of, case study, if you like, which has been done on the scale somewhere else. And so they would be probably the most persuasive types of evidence, and I think the more it’s, sort of, national or the more it’s vague or it’s less specific, then the less persuasive that would be (Site 4, Cllr 1)If there’s a national guideline for something, we should aim to hit the target but make sure that it’s not to the detriment of what we’re providing for a person. Sometimes we miss targets because we are doing things differently. So targets are all well and good as long as they are effective. (Site 3, Cllr 2)It might be NICE guidance, but we might say, well that won’t work in xxxx, because we know of x, y, z issues. (Site 2, Off 6)But that sense of, again, NICE guidance, well this is what is says but it doesn't fit our local area, we didn't see any fit for us. Without necessarily looking at the detail of it. Just, it's coming from outside therefore we probably shouldn't have anything to do with it. (Site 4, Off 3).