Development of a core outcome set for clinical trials in basal cell carcinoma: study protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey

This COS is intended as the global/international standard for clinical trials examining the efficacy of various treatments for BCC. The COS to be developed will not be population specific and will encompass outcomes from all basal cell therapies.

Outcomes will be generated over four phases:

A systematic literature review using PubMed, Medline, and Embase will be conducted using “basal cell carcinoma” and “BCC” as search terms, with queries limited to the title or abstract fields. The systematic review will adhere to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [23]. Included studies will be randomized controlled trials with duplicate studies represented in the various databases included only once. Study characteristics, such as authors, year of publication, source of funding, methodology, number of study centers, treatment comparators, treatment vehicle, inclusion and exclusion criteria, and blinding and randomization status, will be documented. Study design characteristics, such as length of follow-up, treatment duration, results, outcomes, and outcome measures, will be noted. Data will be entered by eight data extractors from four different universities into a spreadsheet. Entries will be periodically reviewed by two senior data extractors for accuracy. The long-list of outcomes extracted from published studies will then be placed into appropriate domains by two Measurement of Priority Outcome Variables in Dermatologic Surgery (IMPROVED) investigators using a forced consensus method. Similar outcomes will be listed only once. Combining and collapsing of outcomes, while necessary, will be performed in moderation to prevent loss of content.

Semi-structured interviews will be conducted to explore patient-identified outcomes. Patients, both in the United States and worldwide, will be recruited from the current patients of practicing physicians and skin cancer advocacy groups via emails and phone calls. Open-ended questions will allow for patient expression of outcomes important to them. According to the research concept to theoretical saturation, approximately 10–15 patients with BCC will be interviewed. This refers to the number of differing opinions required to adequately represent qualitative research findings without further identification of new themes [24]. Past research reveals the requisite number to be 12 or more interviews, with even six interviews identifying most basic themes [25]. A global context will be provided by including participants both in the United States and internationally. Interviews will be audio-recorded, transcribed, and coded to allow complete capturing of outcomes.

Stakeholders, or those invested in the development of a COS in BCC, will also be included in the decision process (Table 1). Dermatologists, drug and device safety regulators (e.g., US Food and Drugs Administration (FDA), European Medicines Agency (EMA)), pharmacologists, pharmacists, and industry scientists are potential members that can provide input regarding what outcomes they feel should be represented. Nurses, physician assistants, and other health care practitioners may be included as well to enhance further discussion.

Outcomes obtained from the steps described above will then be examined by the Steering Committee, composed of four dermatologists: Drs. Murad Alam (Northwestern University), Ian A. Maher (Saint Louis University), Joseph F. Sobanko (University of Pennsylvania), and Todd V. Cartee (Pennsylvania State University). Members may add or remove outcomes prior to the Delphi process. The Steering Committee members will not join in the Delphi process but will be invited to participate in the final consensus meeting.

Delphi surveys have been used in prior COS research [26]. The process involves a series of rounds of data collection and analysis to condense the opinions of individuals into a group consensus. Surveys can be conducted online through the use of specialized software. Responses to each round are collected, analyzed, and then redistributed to participants in successive rounds. We plan to conduct two Delphi rounds prior to the consensus meeting.

Participants in the Delphi process will include patients and physicians in groups of approximately 30 individuals each. Group size was selected to provide a greater diversity of input and account for potential dropouts. A global context will be provided by including patients and physicians from both the United States and internationally. Prior to the exercise, details of the COS will be summarized and demographic/occupational information obtained, including years of experience, field of interest, and position. Consent will be assumed if participants complete the questionnaire. Participants will have 3 weeks to complete the online-survey with email reminders at the 1- and 2-week marks. For each round, the number of participants invited and those who completed the surveys will be documented.

In the first Delphi, the complete list of outcomes gathered from the aforementioned steps will be presented to participants for rating. Outcomes will be listed randomly after each round to avoid any influence of display order on the evaluation of outcomes. Scoring for each outcome will be performed using the scale devised by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group [27]. In this scale, participants rate outcomes numerically on a scale of 1 to 9 (7 to 9 being critical, 4 to 6 being important, and 1 to 3 being of limited importance). The first round will also include a score “U,” to signify uncertainty if the outcome merits inclusion in the set. As discussed by the GRADE working group, this scale will allow participants to focus on ranking the most valued outcomes high and excluding outcomes of lesser importance. Participants will also have the option to add outcomes to the list that they feel should be included. All outcomes will be carried to the subsequent round.

Results from the first round will be analyzed using descriptive statistics. Responses from both the patient and physician groups will be summarized and fed back to the corresponding groups. Participants will then be given the opportunity to use this information to change their score in light of others’ insights. New outcomes will be added if suggested by two or more participants, with any uncertainties addressed by the Steering Committee.

In round 2 of the Delphi exercise, participants will again score the outcomes on a scale from 1 to 9, following the same format as the previous Delphi exercise. The end result of the Delphi should consist of a more simplified set of outcomes that will be further examined at the consensus meeting.

Prior to solidifying a core set of outcomes, a consensus meeting will be held to discuss the results of the Delphi rounds. Physicians, patients, and other stakeholders will be invited to the meeting to provide insight on the process. Results from each round of the Delphi survey will be presented. In terms of consensus, if 70% of participants rank the outcome 7, 8, or 9 with less than 15% scoring it 1–3, the outcome will be retained in the consensus pool [28]. Outcomes will be removed from the consensus list if 70% or more of the participants rank the outcome 1–3 and less than 15% rank the outcome 7, 8, or 9.

Feedback regarding the consensus-derived set of outcomes will then be elicited with the assistance of a trained moderator. Using live polling software, items will anonymously be voted “yes” or “no” for inclusion into the final core set of outcomes. By the end of the meeting, the goal is to create a core set of outcomes which can be agreed upon by all stakeholders, patients, and physicians.

Once a COS has been developed, the Harmonizing Outcome Measures for Eczema (HOME) roadmap will be utilized for developing a core set of measures to track the outcomes selected [21]. Initial steps include identifying current instruments utilized through a systematic review covering at least two databases. Quality of the studies will be assessed by rating their validity, reliability, responsiveness to change, and interpretability.

In order to determine which measurements are suitable per outcome domain, a consensus meeting with key stakeholders, patients, and clinicians will be held [21]. Results from the systematic review will be provided to guide discussion. Attendees will then judge the measures based on how valid, reliable, and feasible they may be for assessing each core outcome domain. New instruments will be developed if there is inadequate evidence supporting existing methods. At the end of the consensus meeting, relevant stakeholders will vote to determine which measures should be included.

The development of the COS is active and ongoing in its initial phase of outcome extraction.