Utilizing a user-centered approach to develop and assess pharmacogenomic clinical decision support for thiopurine methyltransferase

Physicians from both healthcare systems were recruited for a 30-min, semi-structured interview conducted by a pharmacist researcher (KN). Physicians were eligible to participate if they had prescribed azathioprine or mercaptopurine in their clinical practice. Participating physicians were divided into two groups of specialty pracitce: gastroenterology and oncology with each group further divided into subspecialty of adult and pediatric practice. Each participant was verbally consented prior to the interview (the IRB approved the exemption for written consent for this phase).

Participants were interviewed individually. The full interview guide is available in the Additional file 1: S4. Interview questions. Examples of questions include:

Interviews were audio recorded and transcribed for analysis [20]. Two independent analysts, a pharmacist researcher, and a human-computer interaction expert analyzed transcripts through a combination of inductive and deductive qualitative analysis to identify initial theme [21]. A priori codes were derived from the interview questions. Analysts met regularly to discuss and agree upon themes that were identified inductively. Similar to other literature, analysts independently coded the transcripts then met weekly to discuss discrepancies until reaching consensus [22]. A third individual, a human factors expert, was consulted if discrepancies could not be resolved. Data were analyzed and managed using NVivo 10 (QSR International).

Data from qualitative interviews and available literature [23‐25] were used to inform the development of an electronic prototype of the PGx-CDS in a mock EHR (Additional file 1: S5.) The prototype was developed by a research pharmacist and a human-computer interaction expert. We created a list of key features to design a PGx-CDS for TPMT using both collected interview data and human factors principles (Table 1) These key features in Table 1 focused mainly on the display while interview data provided the content of the PGx-CDS prototype. In phase II of this study, we evaluated both the display and the content of the PGx-CDS prototype via usability testing. To facilitate iterative evaluation and redesign, the prototype was not implemented in a “live” EHR. This approach, which has been used in other studies [34, 35], allowed the PGx-CDS tool to be evaluated safely, separate from patient care, providing insight on how it can be improved prior to implementation. Instead, Axure RP 7 (Axure Software Solutions, Inc., San Diego, CA) was used to design the mock-up EHR as well as the prototype (Fig. 1) This was a cost-effective, rapid approach to help physicians visualize the potential design of a future PGx-CDS interface. Participants could interact with the prototype as if they were working with an actual PGx-CDS from the EHR. This prototype provided a realistic impression of what the PGx-CDS could look like in clinical practice. This also helped us identify early errors in the design. Finally, in clinical practice, a TPMT related PGx-CDS would need to support many types of prescribers for patients being treated for leukemia, IBD, or rheumatoid arthritis. Therefore, we chose to apply a universal design approach [36] for PGx-CDS with the goal of creating a CDS that can effectively support a wide range of primary care and specialty prescribers. A universal design approach is generally robust and often requires less training and education of end users [37].

Based on interview data, information that needs to be included in the PGx-CDS interface for TPMT decision-making are below, along with numbers of participants that expressed a need

Additionally, while pediatric oncologists (n = 2) expressed a preference to have a dose adjustment based on patients’ body surface area, gastroenterologists recommended that the PGx-CDS provide dose adjustment based on patients’ weight. Also, several physicians from both groups (n = 3) shared that since thiopurine medications are used for maintaining therapy when treating patients with either leukemia or IBD, the PGx-CDS should also recommend specific lab tests (e.g., white blood cells, hemoglobin) that should be ordered for monitoring medications. Physicians confirmed that lab values related to TPMT medications also are important information to present in the PGx-CDS. Additionally, physicians expressed a desire for the PGx-CDS to present the following options for medication management: neutrophil count as a lab value; ability to turn on/off certain options within the PGx-CDS content; and ability for the PGx-CDS to automatically generate recommendations for a new dosing regimen within the medication ordering screen of the EHR. Finally, if a patient presents with a TPMT homozygous mutation, gastroenterologist providers (n = 3) stated that they would be more likely to prescribe a different biologic agent rather than adjust the medication dose, therefore information like Tuberculosis test and vaccine information should be available to the prescriber through PGx-CDS so they can choose the alternate biologic medication more effectively.

Physicians (n = 3) from both specialties expressed the importance of having clear and concise information on the PGx-CDS: “less is more.” Also, since the TPMT mutation can cause serious side effects for patients, they envisioned an interruptive, PGx-CDS pop-up in the middle of the screen to help them pay more attention.

When participants were asked whether they want to have an option to send the supporting reference(s) for PGx-CDS recommendations to their email account, and most (n = 9) were interested, including all five gastroenterologists. In contrast, oncologists generally wanted more opportunity to review patient information in real-time since “pediatric oncology is different than the primary care setting, we have more time”. Therefore, they preferred to have all information, including reference content, presented on the PGx-CDS interface to help them make prescribing decisions.

Efficiency was measured by the time each physician spent to complete the medication orders (see Fig. 2) All participants (n = 7) successfully completed all assigned patient scenarios for the usability tasks without help from the moderator.

Satisfaction ratings were high across participants (Table 4). One participant (#5), however, rated the ‘interface quality’ relatively low (mean 3.3).

During the usability sessions, participants across specialties expressed high satisfaction with this prototype. They were especially satisfied with the visual design of the alert (n = 5). One adult GI specialist mentioned, “I like that the [warning] is red to alert me about the TPMT mutation. It catches my attention.” Similarly, an adult oncologist stated, “[the] alert is visually well designed, it gets your attention, things that [are] important are in red, it gives you some options [to make decision].”

We highlight four usability errors that were uncovered during usability testing. First, several participants (n = 3) mentioned they were unclear whether the “Accept” button approved the PGx-CDS recommendation or approved their original medication orders. A pediatric oncologist stated, “I assume ‘Accept’ will accept a new order but it is not clear. ‘Accept new order’ will be more clear – pediatric oncologist E.” Another participant suggested changing the label to “Accept recommendation. – Adult oncologist C” Also, the initial button labeled “Others” also was unclear for several participants (n = 3). An adult GI specialist mentioned, “I like the ‘Other’ button but it does not necessarily tell me what that is.- Adult GI C” Thus, we replaced it with “Immunization and contact info” to aid clarity during usability testing. Second, the earlier version of the PGx-CDS prototype did not have a cancel button to allow participants to get out of CDS without further action. This hinders the control and freedom of prescribers when ordering medications. We added this option to the prototype during the usability testing period after two participants completed usability testing. Third, when participants used the “Override” button from the CDS interface, they stated that it lacks a warning message to prompt them about their potential dangerous actions of overriding the recommendation. An adult GI specialist commented, “[PGx-CDS] should have a second level of warning such as, ‘This patient is at a very high risk of severe leukopenia, are you sure you want to expose patient to that risk?’- Adult GI D.” Another adult GI specialist stated, “For [the] override, you should build something like, ‘Are you sure you want to override this recommendation?’. It can be malpractice if overriden.- Adult GI D” Finally, two providers mentioned that the CDS interface did not provide a patient’s weight or body surface area (BSA) to help providers confirm the dosing calculation. A pediatric oncologist mentioned, “This CDS is based on weight, we used BSA to calculate the dose. – Pediatric oncologist F” This suggestion was noted for future improvement.