Introduction
Methods
Multiple embedded case study
Case | Study objectives |
---|---|
1. Case costing of cancer | 1. Review case costing literature and programs using administrative databases. 2. Determine availability and quality of cost data in administrative databases in Ontario. 3. Test a case costing methodology using different patient cohorts (e.g., disease sites). 4. Determine disaggregated costs. 5. Validate the resources and costs extracted from administrative linkages. 6. Create a report on the cost of cancer in Ontario and other publications. |
2. Lung cancer surgery policy analysis | 1. Describe the trends in the distribution of lung cancer surgery in Ontario between April 2003 and March 2009. 2. Estimate the effect of the policy on surgical outcomes. 3. Analyze how structures and processes of care have been affected by changes in the organization and delivery of lung cancer surgery. 4. Explore how a policy to regionalize lung cancer surgery in Ontario was perceived by healthcare decision-makers, healthcare providers, and patients who received lung cancer surgery. |
3. Patient and provider reported outcomes | 1. To perform linkages between these symptom datasets and administrative data, thereby potentially creating an unparalleled cancer outcomes database. 2. To evaluate whether formal symptom assessment in the course of routine clinical care might be used to predict ER utilization. There were three other specific research aims (not included). |
4. Colorectal cancer screening | 1. To evaluate the proposed mailed invitations prior to dissemination. 2. To describe the perceptions of the recipients of mailed invitations regarding: screening to prevent CRC; the mailed invitation; and their screening experiences following receipt of the invitation. 3. To describe the perceptions of participating PCPs with respect to: a) the ColonCancerCheck program; b) the mailed invitation; and c) the Screening Performance Report. 4. To describe the proportion, characteristics and outcomes of eligible Ontarians who responded/did not respond to the mailed invitations. |
5. Inter-disciplinary team approach to women’s cancer survivorship | 1. Create an inter-disciplinary team to address clinically important research questions related to the interplay between cancer and other medical conditions using the best data and analytical methods. 2. Create a cadre of researchers with interest and expertise in cancer survivorship research through graduate education, supervision and mentorship of trainees. 3. Ensure that the results of the research have a positive effect on clinical practice and patient outcomes through the development of an integrated knowledge translation strategy. |
Sampling
Recruitment
Data collection
Document review
Semi-structured interviews
Coding and data analysis
Research ethics board review
Results
Description of participants
Within case analysis
Case 1
“…I think that [cancer system organization] needs a really robust integrated KT strategy which I don’t recall has ever actually been implemented.” (Knowledge User)
Case 2
“I would say that it [expectation for role] has not [been met] in the sense that I might have thought that there would be a somewhat higher level of engagement…” (Knowledge User)
Case 3
“… [name of cancer system organization] is notorious for having high rates of staff turnover. Having continuity of person and kind of that memory you know across the life of the project can be a problem…You know it is not unique to me or to my program but… I see that quite a lot. You know people move, they change portfolios, they leave for other jobs, all sorts of other things happen. It is hard to have the same person doing the same thing for any stretch of time that you can build a meaningful relationship with them.” (Researcher)
Case 4
Study Phase | Case 1 | Case 2 | Case 3 | Case 4 |
1. Involved in planning stages (i.e. defining research question, designing study) | Document Review (Proposala): - Not described Interviews: - Researcher did not indicate that knowledge users (KUs) were involved in the planning stages. - 2/3 KUs indicated they provided access to data but were not involved in planning stages; 1/3 KUs indicated they were involved in study design - All 3 KUs indicated they had an advisory role. | Document Review (Proposal): - “The decision-maker partners will be engaged in the research throughout the project, including the development and refinement of the research questions…” (pp.66) Interviews: - Researcher indicated that KUs were involved in planning stages, but had a limited role - 2 KUs indicated they were involved in the planning stages. - 1 KU could not recall their involvement. | Document Review (Proposal): - “[Names of two KUs] have been involved in the development of this project and articulation of its goals…” (pp.93) - “They have provided input on key methodological issues in the drafting of this proposal…” (pp.93) Interviews: - Researcher indicated that the research team did most of the project design - 1 KU described themselves as an adviser - 2 KUs said that they were involved in the research design stage with researchers. | Document Review (Proposal): - “Key stakeholders have been involved in the planning stages…of this research project.” (pp.55) Interviews: - Researcher indicated that KUs were involved in planning stages, which was built from a pilot done by the cancer system organization. - KUs indicated that the cancer system organization was involved in the “front end” and research approach was developed collaboratively with researchers. |
2. Involved in methods and/or analysis throughout the study | Document Review (Proposal): - Not described Interviews: - Researcher did not indicate that KUs were involved in methods or analysis throughout the study. Researcher indicated that they presented the results to KUs. - KUs confirmed the researcher’s views. | Document Review (Proposal): - “The entire study team will meet every 3 months to review analytic plans, interpret findings, and assign research tasks. The decision-maker partners will be engaged in the research throughout the project, including the development and refinement of the research questions, study methodology, and development of instruments and measures.” (pp.66) Interviews: - Researcher did not indicate KU involvement in methods or analysis - 2 KUs indicated there were minimal meetings and informal updates when the research was being conducted; - 1 KU did not recall their involvement. | Document Review (Proposal): - Not described Interviews: - Research team held regular meetings to keep KUs involved and informed throughout the project, and to obtain feedback from KUs. - 1 KU indicated they provided input throughout the project, and that there were regular meetings and email communication. - 1 KU indicated they were engaged “all the way along” - 1 KU left their organization role. | Document Review (Proposal): - “A working group comprised of [description of KUs] and study investigators will meet regularly throughout the project to ensure continued integrated KT [knowledge translation].” (pp.55) Interviews: - Researcher indicated one KU was involved in methods - 1 KU said they were engaged but had a role change and could no longer continue as a KU, but researcher kept them informed about study activities. |
3. Provided feedback on results | Document Review Proposal: -“To encourage this exchange, we will convene an advisory panel made up of representatives … to communicate findings and discuss implications of the data derived which may also lead to further sub-investigations.” (pp.101–102) Interviews: - Researcher and KUs indicated that, as advisers, they provided feedback on data and findings | Document Review Proposal: - “Regular reporting of results to the team will ensure timely progress and adjustments for further analyses and ultimately will result in findings which are relevant to the decision-makers.” (pp.66) Interviews: - Researcher indicated that more KU involvement came at the point when results were being reviewed. - 2 KUs indicated they provided feedback on findings - 1 KU did not recall their involvement. | Document Review Proposal: - “[KUs] will participate in reviewing and interpreting results of analysis.” (pp.93) Interviews: - Researcher indicated that KUs provided feedback on results and suggested future research directions. - 2 KUs indicated they provided feedback on results - 1 KU indicated that results were not available when they part of the project. | Document Review Proposal: - “The findings from each aim will be reviewed, analyzed and interpreted with input from this working group.” (pp.55) Interviews: - Researcher indicated KUs provided feedback on results then KUs used those findings to make programmatic changes. - 1 KU left before findings were available - 1 KU joined project to implement results once project was finished. |
4. a) Shared results with cancer system organization | Document Review Proposal: - Not described Interviews: - Researcher indicated that the research team shared results cancer system organization - 1 KU did not recall discussing results with organization. They were not sure if research team had sent them the results. - 2 KUs did not recall discuss results with the organization. | Document Review Proposal: - Not described Interviews: - Researcher was not aware if KUs had shared the results of the project with their organization - 1 KU indicated that they presented results at a cancer system organization meeting - 1 KU did not indicate they shared results with their organization - 1 KU did not recall their involvement. | Document Review Proposal: - Not described Interviews: - Researcher team was invited to cancer system organization to present results. - 1 KU indicated that they discussed project results with “lots of people” at the organization. - 2/3 KUs did not indicate that they shared results with organization | Document Review Proposal: - “The findings from each aim will be reviewed, analyzed and interpreted with input [from organization]…key findings will be communicated to other stakeholders integral to the conduct of the [name] program.” (pp.55) Interviews: - Researcher indicated that the team wrote a report for the KUs and the KUs used it to make program changes at cancer system organization. - 1 KU left role so was not involved in any sharing of results to cancer system organization - 1 KU joined project as a decision maker to implement results once project was finished. |
4. b) Shared results with other audiences | Document Review Proposal: - Not described Interviews: - Neither researcher nor KUs indicated that KUs shared results to audiences outside of cancer system organization. | Document Review Proposal: - Not described Interviews: - Researcher did not indicate that KUs had shared the results of the project with other audiences. - None of the KUs indicated that they shared the results with other audiences outside of organization. | Document Review Proposal - Not described Interviews: - 1 KU indicated that they shared the results to clinical audiences nationally and internationally - 1 KU changed roles during project but says they still reference the results in different presentations to other audiences outside of organization. | Document Review Proposal: - Not described Interviews: - Neither researcher nor KUs indicated that KUs shared results to audiences outside of organization. |
5. Implemented study results | Document Review Proposal (plan to implement): - Not described Interviews: - Researcher indicated that they are currently determining how to implement results and aligning with cancer system organization priorities. - None of the KUs said that they implemented results. | Document Review Proposal (plan to implement): - Not described Interviews: - Researcher did not indicate that KUs implemented the results. - 1 KU indicated that the researcher did not give them direction or tell them what to do next. - 1 KU was not aware if results were used by cancer system organization. - 1 KU did not recall their involvement. | Document Review Proposal (plan to implement): - Not described Interviews: - Researcher did not indicate that KUs implemented results, but continued to work with KUs and the collaboration led to additional research questions and projects. - 1 KU said they used the results to build a case around the validity of cancer system work and changes that should be made at a ‘clinical level’. - 1 KU left their role at cancer system organization; when they left, the KUs were working through how to implement results. - 1 KU left project and did not have a role in implementation. | Document Review Proposal (plan to implement): - “The objectives of our knowledge strategy are to improve [cancer system area] by: …translating the outcomes of our project to the broader research community.” (pp.55) - “With [KU] assistance, key findings will be communicated to other stakeholders integral to the conduct of the [cancer system organization program].” (pp.55) Interviews: - Researcher indicated that results were presented to KUs, which informed future cancer system organization activities, changes, and further research designs. - 1 KU indicated that program changes were made, but they themselves did not use results because they left the organization - 1 KU works with researcher often, and KU’s cancer system organization used project results to make cancer system process changes. |
Cross case analysis
Factor | Notes |
---|---|
1. Research team discusses roles at beginning of projecta | Participants recommended that in future health services research network projects an explicit discussion of researcher and knowledge user (KU) roles take place at the beginning of the project. |
2. KUs have decisional authority within the organization to implement project resultsa | While KU decisional authority was identified as important, not all KUs had such authority within their organization. |
3. Researchers engage KUs throughout duration of project (e.g., via regular contact, checking in, meetings)a | KUs preferred to be engaged throughout the study in contrast to collaborations where KUs are minimally engaged or engaged at the beginning and/or end of the project, but less throughout. No discussion by researchers or KUs that KUs would take initiative to engage with researchers throughout the project. |
4. Project goals align with organizational goals and prioritiesa | Project goals may be important but they must align with the organization’s goals and priorities for results to be considered for implementation by the cancer system organization. |
Factors | Notes |
---|---|
1. No explicit discussion of knowledge user (KU) rolea | Lack of an explicit discussion of roles may have led to ambiguity about KU role throughout the project including their role in sharing results in the KU’s organization. |
2. KUs do not have decisional authority within the organization to implement project resultsa | Not all KUs had decisional authority to implement the results at the organization. |
3. No clear implementation plan for results in KU’s organizationb | The goal of specific projects may have been to generate new knowledge, but there was no plan as to what exactly would be done with that knowledge in the KU’s organization. |
4. Lack of alignment between project goals and organizational goals and prioritiesb | Project goals must align with the organization’s goals and priorities for results to be considered for implementation by the cancer system organization. |
5. Lack of time for KU involvement in project and/or geographical distance from researchersa | KUs had multiple responsibilities and had to make time for involvement in project. Being a KU on a project was not part of their usual role in the organization. In some projects, KUs were located in another city and interactions occurred by teleconference. |
6. KU turnover at organizationa | KU turnover was perceived to significantly impact collaboration on projects. For example, project history was lost when a new KU joined the study team. |
7. Lack of knowledge translation framework or process in KU’s organizationb | KUs commented that without a knowledge translation framework within the organization, it was difficult to share results of research study with key decision-makers. While the KUs used the term “framework”, it is possible that other organizational features such as knowledge translation structures and processes may be needed. |