Background
Methods
Setting: primary care networks
Research training
Documents/processes/systems |
---|
➢ Project background, including development of the interventions (HEALD and TeamCare) |
➢ Contact lists (research team, participating PCN staff across sites) |
➢ Protocol to achieve member physician support |
➢ On/Off study design timetable/schedule |
➢ Patient recruitment algorithm/study flow, including: ▪ Protocol for non-response to patient invitation letter |
➢ Patient recruitment and tracking system (Excel) |
➢ Short screening survey package, including: ▪ Sample patient invitation letter from PCN ▪ Information letter |
➢ Eligibility & incentives, including: ▪ Inclusion/Exclusion criteria ▪ Incentive |
➢ Telephone screening script and protocol |
➢ Frequently Asked Questions |
➢ Checklist for packages sent to participants (HEALD specific) |
➢ Data collection matrix |
➢ Clinical protocols/instructions for intervention and active control groups, including: ▪ Informed consent ▪ Data collection of clinical & physical measures ▪ Discharge or end of study |
➢ Information letters & consent forms |
➢ Data collection forms & surveys |
➢ Instructions for data entry & management |
Training activities & supports by research team | Members of the PCN team |
---|---|
▪ Individual detailing sessions on: ▪ Obtaining informed consent ▪ Data collection and documentation (e.g., assigning study IDs) ▪ Data management (e.g., proper secure storage of patient files) ▪ Patient recruitment algorithms/study flow, including mail out of screening survey packages and reminder postcards ▪ Eligibility criteria ▪ Telephone screening script & protocol ▪ Preforming clinical and physical measures | ▪ Exercise specialists and care managers ▪ Administrative personnel |
▪ On-site training on patient recruitment & tracking system (e.g., Access database) | ▪ Exercise specialists and care managers ▪ Administrative personnel |
▪ On-site Point of Care training | ▪ Exercise specialists and care managers |
▪ On-going support by research team via regular in-person PCN site visits, telephone, electronic mail, and quarterly bulletin | ▪ Exercise specialists and care managers ▪ Administrative personnel |
Study design
Data collection
Timeline | Key informant group(s)a | Selected interview questionsb |
---|---|---|
Baseline | ED & CDM | Why did your PCN decide to implement and deliver HEALD and TeamCare? In other words, why did you think the interventions were a good fit for your PCN? |
Describe why you provided a rating of Xc to describe the level of commitment of the PCN leadership (i.e., PCN Board and/or Management) to the HEALD and TeamCare interventions. | ||
At this time, do you believe that PCN staff is capable of implementing HEALD and TeamCare? Why? | ||
At this time, do you believe HEALD and TeamCare are appropriate for PCN staff, patients, and/or PCN member physicians? Why? | ||
What are the anticipated benefits of HEALD and TeamCare, if any, for staff? What are the anticipated disadvantages, if any, to staff? | ||
Do you have any other comments about HEALD or TeamCare? | ||
Midpoint | ED, CDM, CM, & ES | To date, what were the challenges or barriers in getting HEALD and TeamCare up and running? How were these challenges or barriers addressed? |
To date, what has worked well in getting HEALD and TeamCare up and running? | ||
Looking back, what would you have done differently or do you think should have been done differently. | ||
What advice or recommendations would you give to someone else in your position that is about to implement HEALD and TeamCare into their organization? | ||
What assumptions did you make about HEALD and TeamCare (e.g., your role, interventions, how it was going to work)? | ||
In your opinion, are HEALD and TeamCare sustainable? If yes, how do you see this being achieved? If no, why? | ||
Do you have any other comments about HEALD or TeamCare? | ||
Post-Intervention | ED, CDM, CM, & ES | During your midpoint interview, we focused mainly on implementation of TeamCare. Do you have anything else to add regarding what worked well, what didn’t work well, and suggestions for improvement? |
Thinking back to when we first asked you to be involved, what assumptions did you make about HEALD and TeamCare (e.g., your role, the intervention, how it was going to work)? | ||
In your opinion, should the PCN continue to use this model of care (i.e., HEALD and TeamCare)? Why or why not? | ||
ED & CDM | What components, if any, of HEALD and TeamCare will be sustained in your PCN? Describe how each component will be sustained. | |
What components, if any, of HEALD or TeamCare will not be sustained? Describe the barriers to sustaining these components. What would make it sustainable? | ||
CM & ES | Tell me about your (job) satisfaction in this role compared to your previous roles. | |
Based on your experience, would you consider doing this role again? Why or why not? | ||
What will you take, if anything, from your experience with HEALD and TeamCare into your future work/roles? | ||
ED, CDM, ES, & CM | Do you have any other comments about HEALD or TeamCare? |
Data analysis
Results
Key findings | Supporting quotation(s) |
---|---|
Theme 1: Eager to participate in primary care implementation research | |
▪ Supportive of implementation research in primary care setting to determine effectiveness of interventions and to use evidence to inform decision-making |
I mean it’s participating in research, definitely. Because we want to prove what we’re doing is working, right? Or else, why are we here? (PCN manager-HD/TC).
|
So there are millions of dollars being spent and very little actually researching whether it’s effective or if there are better ways to do it… (PCN manager-HD/TC).
| |
Looking at more informing our future business planning…We have two years to evaluate how effective this is and it’s a really easy sell for the Board after the fact to say, look, this is the effect of it (PCN manager-HD/TC).
| |
I think that the results are going to be important in terms of trying to actually put some dedicated funding into the continuation of it. Often with research, we’re there with the resources, but then once the research project finishes, the reserves are spent. So really making it a continued priority in the PCN, to continue putting resources towards these two projects…with the business planning process (PCN manager-HD/TC)
| |
▪ Implementation research perceived as external to PCN (i.e., university-based) |
Right now it’s a research study. But I think as [TeamCare] becomes more of a way of doing business, more physicians would come onboard. Because it is the way we practice then (PCN manager-TC).
|
I mean even though I was housed within the PCN, I didn’t always necessarily feel like I got the support from the other PCN team members… But I think it was easier rather than trying to understand [HEALD] and continuing to fail, I think it was almost easier for them to just pretend, oh they didn’t exist, right? Like pretend that “HEALD, oh that’s the research study and they’re doing their own thing”, rather than, “Okay how can we work as a team and how can – how can we help you?” (PCN provider-HD).
| |
As far as [the exercise specialist], that was completely outside of anything we were doing [at the PCN]. So she was really working with you guys. It didn’t affect us at all (PCN manager-HD).
| |
Theme 2: Challenges to conducting primary care implementation research | |
➢ Interventions presumed better than usual care ▪ Study design is not appropriate for this stage of research |
Honestly, the on-off [design] has been one of the major challenges. And I know in a pure research, laboratory, very scientific methods, it makes abundant sense to do that because that is the gold standard, right? But in reality, I think that’s why you see so few people-related studies that are having an [intervention and active-control] group – unless it’s like a pharmaceutical double blind (PCN manager-HD/TC).
|
The on-off design? What I would do differently primarily would be… to my perspective, this was an implementation study and so the reality is, we have existing services. So the [control group] would be reasonable to use that with our existing [PCN] services…Rather [than] coming in with the attitude of “You are not effectively managing patients, let us show you how to do it, and so we’re going to, on our island, tell you how things should be done” – I feel that that was a really unfortunate approach. Because the existing services are very similar, at least within our PCN – very, very similar… Rather than saying, you know, “on-off - we just don’t take care of these people”. [Instead, for the active-control arm], this is usual care, using our mental health coordinators, and wouldn’t it be nice if the results are actually equally as positive (PCN provider-TC).
| |
▪ Poor patient engagement and retention of control group patients, specific to TeamCare |
The challenge I’m having now is the [active-control group] people, they’re just not as engaged, of course. And to get them to come back in for their six-month [data collection appointment] is very challenging, you know? And we’ve had a few that have dropped out… But right now, my major challenge is to get those people in who are due for their six months who are [in the active-control group]. Yeah, it’s a lot of phone calls, you know, and follow-up with them and it is a challenge to get them. It’s almost like, “what’s in it for me”, you know? (PCN provider-TC).
|
[The care manager] and I were just talking about it, not too long ago, saying, well that [active-control] group – what’s the incentive for them to come back in six months? You know? And that’s probably where you lose a lot of people, the attrition is that “what’s the point of coming back? They didn’t do anything the first time for me, what are they going to do the second time”? (PCN manager-TC).
| |
Having an unblinded study where there’s the [intervention] group and the [active-control] group and then this [active-control] group doesn’t really get anything from us other than having to fill out questionnaires. So I think that in itself was a bit of a difficulty to keep people in the study, difficult to keep them interested if they were proposed that, “Okay, this is the consent. By the way, you’re in the [active-control] group”, and then they’ll be like “Well what’s in it for me?” Right? (PCN provider-TC).
| |
▪ Difficulty referring control group patients to usual care, specific to TeamCare |
We have to go back to the physician and the physician has to refer them [to the PCN]. And if that physician isn’t engaged with the PCN and what we have to offer, that patient doesn’t get seen, even though they may score high on their testing. So that’s been very difficult for us. Hands off (PCN manager-TC).
|
➢ Role conflict ▪ Experimental study design conflicts with professional commitment to provide care |
Yeah, ‘cause it’s against what we – like you’re here to help people and it kind of goes against your grain to not be able to say “Well we could help you with that”, or “the [PCN] program here is great” or [city name] mental health, here’s their number… because we’ve had other people who’ve done really well with some small changes to their medication [in TeamCare]. So you want that for everyone (PCN provider-TC).
|
I think one of the things that we’ve talked a little bit about here as one of the challenges is… it’s sort of the [intervention] and [active-control] group idea, right? Like ‘cause obviously it’s been really hard for the nurses to say, “Well you know, you’re demonstrating all these symptoms and maybe you’re not managing so well with your diabetes but you can’t be part of the program”. And so that’s been a bit of a struggle for us, for sure… I know it was hard for the nurses to say, “Sorry” (PCN manager-HD/TC).
| |
So that way you wouldn’t have this group of people that you feel some of them have been the people that needed it the worst, you know? Looking at their PHQ scores and talking with them even just a brief amount of time that you talk. Actually some of them I’ve spent quite a bit of time talking to because they’re a 20 [on the PHQ]. You can’t really offer them everything. You think, “Oh I’d really like to follow up this individual” and you can’t do it. So it’s kind of annoying – not right. Morally not right for them. So that’s one thing I would change (PCN provider-TC).
| |
▪ Discomfort with experimental study design (e.g., unethical or immoral), specific to TeamCare |
Well I still have a hard time with the [active-control] group. Like I have to say like yesterday I had a client who scored 21 [on the PHQ]. Like I would have maybe liked to see this as a pilot project and then evaluate rather than having the intervention and non-intervention. I don’t know. It just feels sometimes unethical to bring people in – especially if their score is high. Like if they’re ten – but this fellow yesterday, just as an example, he was 21, his A1c was 11.1, his lipids were all elevated, blood pressure – and then I had to say, “Hmmm. Well thanks for coming – thanks for coming out”. So that feels unethical to me somehow ‘cause we know we could offer something more. So … I have a hard time with that (PCN provider-TC).
|
It’s an ethical clinical issue, yes. There’s research protocols…that will always be less significant than somebody’s physical safety. And that’s not a negotiable for me…compared to “is someone appropriately being treated for the risk of suicide”…The issue was “here’s the bigger picture” – patient safety should, in my mind as a clinician, have to come [first] – and it didn’t feel like it was. And that’s hard to do as a nurse. Especially as a legally - you know, as the legal expectation for myself that I would not compromise in any other setting. (PCN provider-TC).
| |
That way you wouldn’t have this group of people that you feel that some of them have been the people that needed [the intervention] the worst. Like looking at their PHQ scores and talking with them even just a brief amount of time - actually some of them I’ve spent quite a bit of time talking – because they’re a twenty [score on PHQ]… you can’t really offer them everything. You think, “Oh I’d really like to follow up this individual” and you can’t do it. So it’s kind of annoying, not right. Morally not right for them (PCN provider-TC).
| |
We’ve all had a really hard time with those patients in the [active-control] group. And that’s so hard on our staff. That has been a very hard thing because we have a mental health liaison that would be the natural fit. That’s what we would naturally do. It is. But we can’t tell [patients] that. We have to go back to the physician and the physician has to refer them. And if that physician isn’t engaged with the PCN and what we have to offer, that patient doesn’t get seen, even though they may score high on their [PHQ]. That’s been very difficult for us. Hands off…And, what about ethics? Where does ethics play in something like that? Because, really, that’s not ethical (PCN manager-TC).
| |
▪ Recommendations to modify the study design |
Well, I guess my surprise was the on-off [design], right? I had some concerns about that initially. And I talked to [the Research Program Manager] about it and, I think as a team, you likely had some concerns about that as well. But we’re committed, that’s how the approval was given and so you get committed to certain research models and so on. So I do get that. I just I think it’s unfortunate that there couldn’t have been some reassessment of that somewhere along the line. Because I do think it’s quite a strong [intervention]. I think it has the potential to help a lot of people (PCN manager-TC).
|
○ Patients act as own controls using histories |
I’m not sure that you’re going to get better data or better results by having had this group of people that didn’t get the protocol. They could have been their own control group and then look at how many more people we could have offered it to (PCN manager-HD/TC).
|
Or if you would want to maybe have everybody be intervention and look at their past history to be the non-intervention. Like what were they like before? What were their two years prior. Their own controls, yeah… I’m not sure that you’re going to get better data, or better results of having had this [active-control] group of people that didn’t [get the intervention] – that the protocol could have given to them (PCN provider-TC).
| |
○ Carefully match patients |
I think people can be their own controls or you can carefully match (PCN manager-HD/TC).
|
○ Focus on qualitative rather than quantitative evidence | I know there’s other ways of doing research. I don’t know if you want it to be more qualitative than quantitative (PCN provider-TC).
|
➢ Administrative burden ▪ Examples: scheduling data collection appointments, data entry |
Initially and ongoing trying to fit into my schedule all the admin and the phone calls and the scheduling and rescheduling if people can’t make appointments and doing that on top of my other duties that I have to do has been quite a challenge (PCN provider-HD).
|
There’s more paperwork involved than I originally anticipated – a lot more tracking of stuff. Like I’ve got binders that – I’ve got so many binders, just like wow!…For example, we have a binder for writing down all the patient appointments. And I have that in my schedule so it’s a bit tedious to have to go into another thing and write them in there… And then we have the binder with informed consent and everybody’s information page. And then we’ve got another binder for, you know? There’s just lots of tracking and paperwork and lots of stuff on the computer. I didn’t realize that it would be that much (PCN provider-HD).
| |
I feel like I was way too busy doing my own admin work to really worry about what else was going on. So I feel like, you know, there were times where the workload was ridiculously heavy. But it’s more in those recruiting stages, right? And then it kind of tapers off and you see your patients and then that’s it, right? And then you have your next wave of recruiting (PCN provider-HD).
| |
A challenge that I found was actually keeping up with [data entry in the Care Manager Tracking System (CMTS)] because I was part-time and I felt like I was double charting, right?… I found it overwhelming sometimes… But the part that always hung me up was that extra charting in the CMTS so that was where I did actually appreciate, again, the support [from the research team]– just for the quirkiness of that actual database as well as just the accountability of actually getting it done (PCN provider-TC).
| |
▪ Preferred passive patient recruitment and voluntary or engaged patient population |
At times I felt it was [the PCN’s] responsibility to find these participants for the study. I think it could have been pre-arranged, do you know what I mean? …Maybe doing an ad in the newspaper to get more participants, maybe going to the doctors’ offices and advertising a little bit more probably would have increased the numbers in the study. But not me doing that. I think that should have been done before. So that the numbers were already there, the participants were already volunteering to participate (PCN provider-HD).
|
▪ Decreased satisfaction related to research role, specific to HEALD |
The paperwork was a bit… I mean, there’s a lot of paperwork with it, right? So that wasn’t my favourite part of it… Less paperwork and less stuff around “You gotta invite these people at this time”… Well you know, as exercise specialists, we’re all action and let’s – don’t bother me with the details kind of thing. Let’s go walking or something, you know? (PCN provider-HD).
|
Now I understand that with, like it’s research so it had to be the way that it is and I think that that probably impacted my level of satisfaction with it. I mean I did really enjoy working in the group setting and I really think that at the end of the day just working with the people is what I enjoy (PCN provider-HD).
| |
I would consider doing [this role] again in a different capacity, with making some of the modifications that we’ve already discussed. And then in fact just doing the model. And not all the other [research] things (PCN provider-HD).
| |
Well [I was] surprised by all the admin work because it’s not something I really enjoy doing (PCN provider-HD).
|
Theme 1: eager to participate in primary care implementation research
Number one, we’re a PCN that really does support research (PCN manager-HD/TC).It will, regardless of what the results show…, it’s going to inform business planning (PCN manager-HD/TC).
If it was just brought to the PCN and it wasn’t a study and things like that, I think I would be more engaged in it because I would have adopted it as our program that’s staying… (PCN manager-TC).
Theme 2: challenges to conducting primary care implementation research
Interventions presumed better than usual care
My way of thinking, we would have had a lot more buy-in, since this is already a protocol that has been proved in other places to have worked. We’re not really necessarily testing the protocol so much as the applicability to this population. Honestly, is there that much difference between Canadians and Americans as far as whether or not a protocol would work? (PCN provider-TC).
[The care manager] shared with me that getting people in the [control group]… they’re just not coming back in [for data collection appointments]. And really, why should they? There’s nothing in it for them (PCN manager-TC).
Is the client gonna really go back to the doctor? I have no control over that. He may say “Yeah I’ll go” but whether he actually goes – I’m not following him up in two weeks and saying “Hey, did you get over to your doctor yet?” (PCN provider-TC).
This fellow yesterday, just as an example, he was 21 [on the PHQ]… his A1c was 11.1, his lipids were all elevated, blood pressure. And then I had to say, “Hmmm. Well thanks for coming out”. So that feels unethical to me ‘cause we know we could offer something more. So I have a hard time with that. This fellow yesterday, he said, “I feel like my wife and I are falling through cracks”, and here he gave a hand out and I had to say, “Well thanks for coming out. Go back to your doctor” - you know? Just didn’t seem right…Can we make him in my [intervention] group? Could I switch him? And then when he says things like, “I feel like my wife and I have fallen through the cracks”, it’s like oh here’s another one, you’re falling through the cracks ‘cause – he’s so depressed, he feels like he’s on a treadmill. He feels like giving up. He feels like jumping off a bridge. So now I have to say “Hey wait a minute, go back to your doctor” rather than saying “Hey, why don’t you come back next week and see me and we’ll talk some more about this” (PCN provider-TC).
Role conflict
It may be easier if somebody is used to research, this is just the way it goes. So you pull somebody in who’s a primary caregiver and put them into a position where they’re used to solving problems, they’re used to making suggestions. That’s what you do and then you take that away from them. It’s like you strip that ability away from them (PCN manager-HD/TC).
We’ve all had a really hard time with those patients in the [active-control] group. And that’s so hard on our staff…That’s been very difficult for us. Hands off…And, what about ethics? Where does ethics play in something like that? Because, really, that’s not ethical (PCN manager-TC).
As far as ongoing challenges, I still think that the [control] group and the design methodology has been a major barrier to success. And in the future, I have a very strong recommendation that the design methodology be reconsidered (PCN manager-HD/TC).
Administrative burden
I just hate it. I hate making phone calls! That’s probably my least favourite is doing the follow-up calls and leaving the same message and asking the same questions… It’s a lot of calls! Boy. It is a bit tedious after a while. So I’ll be glad when I can teach the classes and not have to do the follow-up calls anymore! (PCN provider-HD).I was surprised by the amount of paperwork… it was entering it on to the paper and then into the computer as well. I don’t think there is a different way around it. It needs to be done and it’s just part of the study (PCN provider-TC).
I think another potential that we have promoted within the PCN is using our own [existing] patients. So when we identified a newly diagnosed [diabetic], we would screen them ourselves potentially for depression…And then the other thing I don’t think we have done is other means of recruitment. So going to the paper, looking at just the general population ‘cause there are a population of people out there that are interested in participating in studies (PCN provider-TC).
Working with patients is, for me, the satisfaction. Having to do the recruitment and the admin…that’s not what I want to do… I want to spend my time with the patients (PCN provider-HD).