Knowledge mobilization activities to support decision-making by youth, parents, and adults using a systematic and living map of evidence and recommendations on COVID-19: protocol for three randomized controlled trials and qualitative user-experience studies

In this study, adults are defined as individuals self-reported to be 21 years of age or older, youth are defined as individuals self-reported to be between the ages of 15 and 24 years [17], and parents are defined as individuals self-reported to be 18 years or above and are a parent or legal guardian of a child under 18 years old. Participants can be from any country. For all three populations, participants will need to be able to read and understand English. We expect variation in the English language proficiency and health literacy level among participants. Therefore, when collecting demographic information, questions related to language proficiency and familiarity with COVID-19 health information will be included to understand the participant’s level of health literacy. We may adjust for these variables in our analysis if needed, although we expect them to be reasonably balanced between the SLV and PLR groups due to randomization.

Each leading site will run recruitment for the different population groups separately yet with similar and overlapping strategies. We will recruit English-speaking (mother tongue or fluent) participants globally through our experienced RecMap investigator team (https://​covid19.​recmap.​org/​about), the Cochrane Consumer Network and various Cochrane networks, guideline co-authors who interact with youth (including the international Young Persons’ Advisory Groups), parents, and adult citizens. We plan to run public recruitment campaigns through identified youth, adult, and parent organizations’ social media platforms (e.g., organizations’ accounts on Twitter, Instagram, and Facebook) and newsletters. Each leading site will be responsible for monitoring its own recruitment, though all investigators will support overall recruitment for the trials (e.g., if recruitment falls behind for one of the populations, other sites will help support recruitment for that group). Surveys will be administered through SurveyMonkey, where participants are screened for eligibility prior to allocation. Previous studies suggest that the target recruitment rates for such trials are achievable in less than 3 months, and we will recruit at least 240 participants per group [18‐20].

The average survey completion time is estimated to be 15–25 min, based on stakeholder feedback and pilot testing. As suggested by our stakeholders in the preparation of this trial, we will be inviting participants who complete the survey to enter a draw for a chance to win one of five $50 CAD gift cards for the adult and parents’ populations. The youth group will invite youth participants to enter a draw for one of ten gift cards valued at $25 CAD. This is offered as a gesture and appreciation for completing the survey.

We will summarize participant baseline characteristics and outcomes using means and standard deviations (SD) for continuous variables and proportions for categorical variables.

We will perform a primary analysis including all randomized participants. We will exclude participants who take less than 6 min to complete the survey. For the outcome of understanding, we will use χ² tests and risk difference with 95% CIs to compare the proportion of correct responses between groups. For the outcomes of accessibility and satisfaction, we will use t-tests and mean differences 95% confidence intervals (95% CIs) to compare means and SDs between the intervention and control groups. Finally, we will present preference as mean (SD) overall and for both trial arms. Skewness, Shapiro-Wilk tests, and histograms will be used to evaluate whether the distribution was shifted toward the same preference in both groups. Levene’s test of equal variances will be used for all t-tests and degrees of freedom will be adjusted for p < 0.05. We will report all p-values to three decimal places, with values less than 0.001 reported as < 0.001. We will consider statistical significance at p < 0.05. Furthermore, we will conduct a meta-analysis across the three trials by combining results of the three trials. Our a priori hypothesis is that there is no difference across the three populations despite the differences in population characteristics and slight difference in the presentation formats for each population. However, we will explore differences between trials applying the GRADE approach to evaluating inconsistency which includes the chi-square test, the I² value, the overlap in confidence intervals, and the differences in the point estimates of effect for all outcomes [29].

At the end of each survey, participants will be invited to contribute to a virtual, one-on-one, semi-structured interview, conducted by the research coordinator for each leading site. Interested participants will voluntarily provide their email address at the end of the survey, and the research coordinator will follow up with them through email.

We will invite individuals from the list of participants who have agreed to participate in the qualitative interviews. For the parents’ population, we will use purposive sampling (inviting participants based on region, ethnicity, gender, and topic of recommendation). For the adult and youth populations, we will not use a purposive sample; however, we will collect demographic data from the participants at the time of the interview. All interview participants will be compensated for their time with a gift card valued at $25 CAD.

Prior to the interview, participants will be provided with the consent form for the semi-structured interviews. Following that, a Zoom invite will be shared with the participant. Prior to the start of the interview, the research coordinator conducting the interview will obtain verbal consent from the participant for participating in the interview and for the interview to be audio-recorded. If the participant chooses to leave their video camera on during the interview, consent will be acquired for video-recording the interview. Consent will be documented on a log by the coordinator. If participants decline recording of the interviews, we will not move forward with the interview. We will transcribe interview recordings verbatim and de-identify all interview transcripts.

Interviews will be conducted in English by trained interviewers and will take approximately 30–60 min. Interviews will be guided by open-ended questions that cover the seven facets of the honeycomb model to verify user’s experiences with the different recommendation formats: usefulness, usability, findability, accessibility, desirability, credibility, and value [30]. We will pilot test the interview guide within the three populations to refine the questions and language based on the populations’ unique needs prior to the interviews.

All recordings will be stored on a password-protected device and secure network and accessed only by members of the research team. The recordings will be kept for 5 years, at which point they will be destroyed. All transcribed recordings will be reviewed and verified by the interviewer or a second research team member prior to being de-identified. NVivo software (QSR, 2018) will be used for data coding and management.

Qualitative data analysis will follow a hybrid inductive/deductive method [31]. This method allows for flexibility to utilize an established framework during analysis (deductive) as well as identify codes that emerge from the data (inductive). The 7 elements of the honeycomb model for user experience (Fig. 4) informed the creation of the interview guide and were adapted and used as the initial model for analysis (see Supplementary Material 2 for the interview guide) [30]. The seven elements of the model will guide coding with inductive coding of data that extends beyond the model.

Data collection and analysis will occur iteratively [32]. The analysis will follow six steps: (1) data cleaning-comparing the transcript with the recording and doing the cleaning to ensure alignment, (2) generating initial codes using line-by-line coding, (3) searching for themes guided by the honeycomb model, (4) reviewing the themes and generating a “thematic map,” (5) defining and naming themes guided by the honeycomb model, and (6) writing the report. Data collection will occur until saturation is reached [33].

We will develop joint displays, explicitly merging the results from the three populations’ data sets through a side-by-side comparison to assess how findings are aligned among the three data sets [34].

Strategies for maintaining rigor [35] will be used such as utilize a detailed study log and audit trail to ensure transparency. A coding system will be utilized and refined throughout the iterative data collection and analysis stages and each site will have a primary coder and a secondary coder to code approximately 10% of the transcripts and compare to maintain intra-rater reliability [35]. We will use iterative member checking, reflecting back statements made during interviews to participants for clarity and understanding.

We will also use our trial survey data to triangulate our findings with survey responses. Our analysis team will reflect on and disclose our own digital experiences and consider this in our iterative analysis process.