Using the CollaboraKTion framework to report on primary care practice recruitment and data collection: costs and successes in a cross-sectional practice-based survey in British Columbia, Ontario, and Nova Scotia, Canada

In Fraser East, BC, there were 164 providers (family physicians and nurse practitioners) working in 58 practices; in Eastern Ontario Health Unit, ON, there were 190 providers in 63 practices; and in Central Zone, NS, there were 190 providers in 123 practices. Once the list of practices and providers was verified by each region’s stakeholder advisor group, we recruited between 22 and 39 eligible primary care practices via individual primary care providers within the region. Our sample size calculation required a minimum of 15 practices per region to detect minimal differences of 2–5% across jurisdictions in two main patient-level performance indicators, access and continuity of care, using a Bonferroni-corrected two sided alpha level of 0.017 to account for multiple pair-wise comparisons. For these calculations, we used estimated standard deviations and intraclass correlation coefficients from data collected in the Comparison of Primary Care Models in Ontario study [4]. These calculations take into account that patients who share the same provider may be more similar to each other and may have similar primary care experiences, and that different providers from the same practice may not practice as independently from each other as providers in different practices. A maximum of five physicians/ nurse practitioners per independent practice were eligible to participate. Based on previous work [36], practices were considered independent if they did not share more than four of the following five criteria: 1) office space; 2) staff; 3) expenses; 4) patient records; and 5) on-call duties. Physicians and nurse practitioners were eligible to participate if they met the following criteria: 1) must be part of their current practice for at least 1 yr; 2) identify that practice as their “principal clinical practice”; and 3) practice comprehensive, all-ages primary care and not limit their practice to a special focus such as sports medicine, emergency medicine, palliative care, or psychotherapy (i.e., the special focus does not comprise more than 20% of their total practice time).

Providers were initially informed about the study through a letter or email sent by local decision-makers. Research coordinators in each region then mailed out a recruitment letter to physicians with an expression of interest mail-back card enclosed. Those who sent back expression of interest cards were contacted immediately by the research coordinators, whereas those practices who did not send back an expression of interest were contacted by phone after ten business days. We expected to send up to three follow-up reminder emails/phone calls but left our recruitment protocol open to more contacts if needed. For consenting practices, organizational leads were asked to complete one organizational survey, and one to five physicians or nurse practitioners completed a provider survey in each practice. To minimize burden on providers, this survey was kept as short as possible (e.g. number of years practicing, age range, gender). All staff were also asked to complete one Team Climate Survey [37, 38]. Once practices agreed to participate, we then recruited a consecutive sample of their patients (a minimum of 20 per practice) to fill out a paper-based patient experience survey. Patients were eligible to participate if the following criteria were met: 1) aged 18 years and over; 2) have been with their current provider for at least 1 yr; and 3) able to complete the survey in either English or French.

Data were collected between 2014 and 2016. Provider and organizational surveys were collected using REDCap (Research Electronic Data Capture) [39] or paper surveys. A survey administrator distributed patient surveys in primary care practice waiting rooms. Written informed consent was obtained from all clinicians and patients. Patients were specifically asked for written consent to: a) participate in the survey, b) have their survey data linked to the administrative data, c) be contacted again for further related studies. All procedures were approved by the Behavioural Research Ethics Boards at Fraser Health, University of British Columbia, Ottawa Health Science Network, Bruyère Continuing Care, and the Nova Scotia Health Authority.

Data for this report of study recruitment methods were derived from qualitative sources including: study team meeting minutes; memos/notes from survey administrators regarding their interactions with practice staff, providers, and patient participants; and stakeholder meeting minutes. Data were also derived from spreadsheets with numbers for participant eligibility, responses, and survey completions and from time and cost tracking for recruitment and survey administration.

All notes were electronically documented. We used the CollaboraKTion framework steps to guide our analysis. Notes were read several times and coded by two members of the research team according to the first three steps: contacting and connecting; deepening understandings; and adapting and applying the knowledge base. Codes were then organized into themes; initial coding and author agreement was reached through an iterative process of discussion and returning to the data.

We calculated participation and completion rates for the patient surveys using the Wong et al. [12] approach to calculating the different rates for comparability with existing literature on CBPHC performance measurement. This method tracked the participation process, allowing the elucidation of “steps” and their respective attrition rates. The project had one research coordinator per region, which represents our fixed costs of carrying out this work. We also calculated a detailed breakdown of variable costs required for practice recruitment and survey implementation in each province.

In recruiting practices, the first three CollaboraKTion activities, Contacting and Connecting, Deepening Understandings, and Adapting and Applying the Knowledge Base, and their associated processes were most pertinent to our participant and community engagement strategy. As CBPHC performance measurement and reporting continues beyond the end of the study and scales up in the study regions, the other two CollaboraKTion activities, Supporting and Evaluating Continued Action, and, Transitioning and Embedding, will become more relevant with continued re-iteration through the other three activities. Table 1 provides an overview of the specific actions we took in engagement during our recruitment and data collection phase. In keeping with the iterative nature of the CollaboraKTion framework, the described actions did not necessarily happen chronologically as presented. Extracting the qualitative data and our spreadsheets on recruitment training was key for our discussion of the costs associated with recruitment (Excel tracking sheets) and the different strategies for engagement in each region (meeting minutes and data collection notes).

Practice participation and completion rates for all three study regions are presented in Fig. 2. Participation rates was similar (average was 36%) across the study sites and completion rates were high (99%) once practices agreed to participate.

Patient participation and completion rates for all three study regions are presented in Fig. 3. Participation rates in BC were substantially lower than in the ON or NS sites. In part, this may have been due to the involvement of front office staff. In BC, the receptionist only introduced patients to the survey administrator (SA) if the patients were both interested and eligible, whereas the front office staff in the other provinces were encouraged to send all interested patients to the SA so that the SA could directly screen the patients for eligibility. While all provinces had intended on the receptionist screening for patient eligibility, we found that the staff in ON and NS were too busy to do this additional step. It is possible that the BC front office staff may have informally screened patients for eligibility in a way that some patients who could have been eligible and interested in participating were not told about the study. We learned that the front office staff appreciated being as minimally disrupted as possible, and in cases such as ours where there was an opportunity for research staff to be on site during patient survey recruitment and data collection, it may be advantageous to screen for patient eligibility directly.

The variable costs required to recruit practices and administer the patient surveys ranged from $1503 to $1792 and are summarized in Table 2. In BC and NS, the survey administrator was embedded within the community so less time (and expenses) was spent on travel. In ON, the survey administrator was based at the university and spent more time in travel. The lower practice recruitment costs in the areas of peer recruiter and community engagement in NS are likely due to the fact that they had the highest number of eligible practices in their study region and they were able to recruit more practices. Researchers in NS also had strong ties to decision maker leads and existing research relationships with many of the physicians in the region and the peer recruiter for NS stood out in that they were particularly capable of quickly recruiting practices, largely due to pre-existing relationships with physicians in the community.

Each region strategized practice recruitment expenditures differently. Although recruiting practices was the main concern for success of our study and was also the largest recruitment expense, we did funnel some funds into patient recruitment. BC and NS offered coffee cards to front office staff and participating patients. Given the lack of existing relationships between researchers and potential practice participants and the known time conflicts of providers in the ON study region, the research team in ON decided to offer practices a higher honorarium amount than the other provinces ($500 vs. $250). This meant that they did not have the funds to offer gift cards to patients. BC also used funds for community engagement and peer recruiters. Across all sites, recruitment funds were targeted in ways most appropriate to their given contexts.