Evaluation of the Med Safety mobile app for reporting adverse events in Burkina Faso

This study had two components: (i) a cross-sectional online survey targeting users of the Med Safety mobile app from October 1, 2022, to February 10, 2023, and (ii) a retrospective, comparative analysis of the characteristics of Med Safety reports versus the paper form from January 1, 2021, to December 31, 2022, in Burkina Faso.

Burkina Faso is a Sahelian country located in the heart of West Africa with a population of 20,505,155 in 2019 [21]. The ratios of physicians, pharmacists, midwives and nurses per capita were 1/9,659, 1/65,351, 1/4,436 and 1/2,806, respectively in 2021 [22].

The national vigilance system for health products for human use in Burkina Faso was created according to Decree 2012 -1033/PRES/PM/MS of December 28, 2012[21]. The purpose of the system is to guarantee the safe use of health products. The vigilance system is integrated into the public health system and is responsible for monitoring AEs related to the use of health products. Burkina Faso has a national health products vigilance centre (CNVPS), 70 district pharmacovigilance focal points and 13 regional pharmacovigilance focal points, which act as the relays for the national pharmacovigilance centre.

HCPs are responsible for reporting all AEs to the CNVPS directly using the Med Safety mobile app or indirectly through the paper form. This form is submitted to the district pharmacovigilance focal point, who collects AE reports in their area and then sends them to the regional focal point, who in turn transmits the reports to the CNVPS of the National Agency for Pharmaceutical Regulation (ANRP). The recommended time for transmitting ICSRs to the ANRP, from the date of notification, is 15 days for cases of serious AEs and 90 days for non-serious cases [21].

Based on 400 downloads of the Med Safety mobile app in 2020 in Burkina Faso and an estimated frequency of use of 50%, an error rate of α = 0.05, a precision of 5%, and a 10% non-response rate, the minimum sample size required for our study was 217 individuals.

We used the systematic sampling method to select our sample. From the Med Safety app's user management tab in the vigilance hub [23], we extracted the email addresses and/or phone numbers of people who had downloaded the app and registered (n = 400). The list of addresses formed our survey sample base. We calculated the sampling interval k (k = 400/217), which was approximately 2, and we randomly selected a number, and then every kth address was chosen [24].

The online survey also targeted all national pharmacovigilance focal points in Burkina Faso (13 health regions and 70 health districts, 5 university hospitals, and 3 public health programs). Since the launch of the Med Safety app in 2017, some pharmacovigilance focal points have been trained with practice (adequate training) in adverse drug event reporting, but there is no documented information on this training). Our final sample size was 308 individuals.

The online survey was created using Google Forms. A preparatory information workshop on data collection was convened with the 13 regional pharmacovigilance focal points and the questionnaire was tested during this workshop. The Google Forms questionnaire was adapted into French and sent to the survey participants.

We shared the invitation link, the participant information sheet, and the informed consent form via email or social networks (WhatsApp) with the interviewees following the method used in other studies [7, 25, 26]. They were informed that the invitation link was personal and that signing the informed consent form was required to participate in the study. To avoid duplicate responses, we applied a restriction of one response per invitation link. Those who did not respond to the questionnaire were contacted again by phone or email one week later. Participants who did not respond after two reminders were considered non-respondents. Data collection was conducted from October 1, 2022, to February 10, 2023.

For the cross-sectional survey component, we collected sociodemographic and professional variables (age group, sex, profession, health region and practice structure), pharmacovigilance knowledge and practice (knowledge of the role of the pharmacovigilance focal point, reporting of an AE, preferred tool for reporting, estimated time interval for reporting), access to a mobile phone, type of phone, knowledge of the Med Safety app, Med Safety download status, and ability to report an AE through the Med Safety app. The full questionnaire is available as electronic supplementary material file 1. On the variables of acceptability, usefulness and feasibility (as defined by Proctor [29]) of the Med Safety app (Table 2), participants were asked to rate on a 4-point Likert scale (strongly disagree, disagree, agree and strongly agree) [30]. Acceptability is the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory [31]. Feasibility is defined as the extent to which a new treatment, or an innovation, can be successfully used or carried out within a given agency or setting [31]. Usefulness (appropriateness) is the perceived fit, relevance, or compatibility of the innovation or evidence-based practice for a given practice setting, provider, or consumer; and/or the perceived fit of the innovation to address a particular issue or problem [31]. We adapted Weiner et al.'s criteria of acceptability, usefulness and feasibility for our study [31].

We extracted all ICSRs from Burkina Faso from the WHO global pharmacovigilance database, VigiBase^®, using the Vigilyze^® tool. Burkina Faso has been sharing its pharmacovigilance data in VigiBase^® since November 2010 [27]. We included reports from January 1, 2021, to December 31, 2022. We excluded reports from 2017 to 2020 because we could not distinguish the ICSRs sent through the Med Safety app from those originating from a paper form. Data extraction was performed on February 10, 2023.

Data were processed in Microsoft Excel, and statistical analysis was conducted in Stata version 17. Categorical variables were described using frequencies and continuous variables using medians and interquartile ranges (IQRs). The Pearson Chi2 test was used to compare the reporting year, the type of reporter, AE seriousness, and drugs (therapeutic groups) between the Med Safety app and the paper form [5, 16]. The Wilcoxon rank-sum test was used to compare the VigiGrade^® median completeness score and the median time to submit a report to the CNVPS between the two notification tools. The significance level of the statistical tests was set at 5%.

Acceptability, usefulness, and feasibility are three essential measures for evaluating the success of the implementation of the Med Safety intervention [28]. Each measure is made up of a set of items (Table 2). For each item, we calculated the proportion of respondents who strongly disagreed, disagreed, agreed, or strongly agreed with the item (4-point Likert scale).

We calculated the proportion of participants for whom the Med Safety application was acceptable, useful, and feasible. To do this, we first calculated the total score for each participant by adding the scores corresponding to the responses to each item. Each participant's total score can range from 14 to 56. We then calculated the mean total score for all participants (mean = 35.8). We defined the acceptability, feasibility, and usefulness of the Med Safety app for a total score greater than 35.8 [29‐33]. Finally, we calculated the proportion of participants whose total score for all items was greater than 35.8.

For pharmacovigilance data, the variables used in our study were the ICSR identification number (Worldwide unique ID), the initial date of receipt of the report at the national pharmacovigilance centre, the initial date of entry into VigiBase®, the qualification of the reporter, the drug reported, the therapeutic indication, the start and end date of the AE, the type of AE, the seriousness and the VigiGrade^® completeness score of the safety report. The VigiGrade^® completeness score measures the informativeness of the time of onset of the AE, age, gender, indication, outcome of the AE, type of report, dose, country, reporter and case history [34]. The time to submit a report to the CNVPS was calculated as the difference between the date of initial receipt of the report to the CNVPS and the date of onset of the AE [35, 36].

Our study compared the number of ICSRs per year, the type of reporter, the AE seriousness, the drugs involved (therapeutic group), the VigiGrade^® completeness score and the time taken to submit ICSRs to the CNVPS between the two reporting tools.

A total of 160 out of 308 (51.9%) users completed the online survey. Half of the respondents were in the 35-44 age group (Table 1). The respondents were predominantly male (71.9%). The respondents were mainly pharmacists (35.6%), nurses (33.1%) and physicians (17.5%). Their practice structures were essentially primary health facilities (20.6%), medical centres with surgical units (15.0%), university teaching hospitals (13.8%) and health districts (13.1%). The Centre (29.0%), “Hauts-Bassins” (12.9%) and Centre-West (12.3%) health regions were the most represented regions.

Of the 160 respondents, 72.5% (116/160) preferred to report ICSR to the CNVPS via the Med Safety mobile app, 17.5% (28/160) via the paper form, and 10.0% (16/160) via a phone call (data not presented in a table or figure). As per the reporting tool, a duration of 10-20 minutes was the most frequent category (Fig. 1). Personal smartphone use was 96.9% (155/160).

Bild vergrößern

Of 81 people who reported downloading the Med Safety app, 61 to 74 (75.3–91.4%) responded to the questions about assessing acceptability, feasibility and usefulness. The Med Safety mobile app was acceptable, useful and feasible for 58.1% (43/74) of respondents. The Med Safety app was acceptable, useful and feasible for 93.8% of respondents who received training with practice on the application versus 32.5% of those who did not (p < 0.001). Responses to the various statements about the acceptability, usefulness and feasibility of the app are summarized in Table 2.

The most useful feature of the Med Safety app according to the respondents was “reporting” 53.1% (43/81), “news” 13.6% (11/81), “watch list” 12.4% (10/81); and 21.0% (17/81) no specific feature was “most useful”(data not presented in a table or figure).

The proportion of those who used the Med Safety app to report AEs was 23.1% (37/160), including 33.3% (19/57) of pharmacists, 25% (7/28) of physicians, 11.3% (6/53) of nurses, 14.3% (1/7) of specialized nurse and 4 others (3 pharmacy students and 1 health logistics specialist). Of these, 73.0% (27/37) reported using the Med Safety app at least 2 times to report AEs, while 27% (10/37) used the application only once (data not presented in a table or figure).

During the period, a total of 1472 ICSRs were received by the CNVPS, including 357 (24.3%) through the Med Safety app and 1,115 (75.7%) through the paper form.

The number of cases reported through the Med Safety app fell from 283 (35.6%) ICSRs in 2021 to 74 (10.9%) in 2022, while the number of cases reported through the paper form rose from 512 (64.4%) ICSRs in 2021 to 603 (89.1%) in 2022 (Table 3). Reports received from physicians were more frequent with the Med Safety app than with the paper form (15.1% vs. 6.7%). Reports sent through the Med Safety app were frequently more serious than those sent through the paper form (12.3% vs. 3.5%, p < 0.001). Adverse events following immunization (AEFIs) for COVID-19 vaccines were reported more frequently with the Med Safety app than with the paper form (71.7% vs. 64.4%).

The median VigiGrade® completeness score for report informativeness was 0.9 (IQR 0.7–0.9) for the Med Safety app. The median score was also 0.9 (IQR: 0.6–0.9) for the paper form.

The median time to submit the ICSR to the CNVPS was 159 days (IQR 59–187) using the Med Safety app, whereas the median time was 166 days (IQR 107–281) using the paper form (p < 0.001). The median time difference was 22 days (95% CI [10.0–40.0]).

Regarding the reporting of AEFIs, the median time of transmission to the CNVPS was 175 days (IQR 132–189) for the Med Safety app and 181 days (IQR 137–388) for the paper form (p < 0.001). The median time difference was 29 days (95% CI: [15.0–50.0]).

The median time of transmission for AE reports was 46 days (IQR 5–153) for the Med Safety app and 94 days (IQR 1–166) for the paper form (p = 0.30). The median time difference was 18 days (95% CI: [− 6.0 to 48.0]).

The 35–44 age group accounted for half the participants (50.6%). This may have been the group that downloaded and registered more in the Med Safety app for reporting AEs. The high representation of men (71.9%) also suggests that they may have downloaded and registered more in the Med Safety app. The central region was most represented in the survey; this could be explained by the fact that many HCPs have downloaded the Med Safety app because of the awareness raised during the COVID-19 vaccination campaigns.

We did not observe any consumers/patients participating in this study. This could be justified because they are not sensitized and encouraged to download and report AEs through the Med Safety app in Burkina Faso [37]. However, in Ghana, a study showed that patients and consumers (the general public) reported more AEs with the Med Safety mobile app than HCPs [38]. To increase the reporting rate in Burkina Faso, it would be important to train and sensitize patients, especially those suffering from chronic diseases, to report AEs [39] through the Med Safety app.

The Med Safety mobile app is a complementary tool to electronic reporting of AEs. We noted that for the majority of users surveyed in Burkina Faso (58.1%), the application was acceptable, useful, and feasible. However, for some users, this was not the case. This could be explained in our study by the fact that 56.2% of respondents had not received adequate training to use the Med Safety app. In Uganda, Kiguba et al. showed that training and practice of the Med Safety app would facilitate its adoption by HCPs [40]. Given that 72.5% of those surveyed would prefer to report AEs via the mobile app, it is important to establish a plan for ongoing training and awareness of the benefits of the Med Safety app for users [38].

For 43.1% of Med Safety users and 45.5% of paper form users, the reporting time was estimated between 10 and 20 min. This suggests that some reporters in Burkina Faso have not yet perceived the time-saving benefits of using the Med Safety app. However, the speed of reporting AEs with mobile applications was an advantage elsewhere [15, 19]. The most useful feature of the Med Safety app, according to users, was “reporting” (53.1%) followed by “news” (13.6%). In Ghana, the most useful feature of the Med Safety app was “news” (35.0%) followed by “reporting” (32%). The small sample size in the Ghana study (N = 33) could mask a difference in choice. The difference in appreciation between the two countries could be justified by the fact that awareness-raising in Burkina Faso concerned reporting. Also, in Ghana, the fact that the public was using the application meant that they needed information on drug safety. Conversely, in Burkina Faso, it was only HCPs who used the app, and they already had some knowledge of drug safety.

The Med Safety app contributed to the reporting of AEs with higher reporting figures in 2021. The difference in reporting between 2021 and 2022 could be explained by the fact that there was more awareness among HCPs about reporting through the app in the context of COVID-19 immunization, where physical contact had to be limited. The proportion of ICSRs with the mobile app was 24.3% during the 24 months. The reports were mostly related to COVID-19 vaccines. However, Montastruc and al. found in 2017 that reports sent by the VigiBIP^® mobile app accounted for 4.7% of reports in Toulouse, France [16]. This finding could be explained by the difference in the study period. Indeed, the study on the VigiBIP^® app was conducted during the first 25 months of the application's launch. However, our study on the Med Safety app was conducted approximately 42 months after the application was launched in the context of population immunization against COVID-19. The impact of mobile applications in terms of reporting will need to be observed in the longer term to draw more in-depth conclusions.

The quality of reports sent by the Med Safety app was similar to those sent by the paper form, whilst overall the reporting time was shorter for the mobile app. However, the time taken to submit the reports through the Med Safety app could still be further improved for rapid communication of drug risk as would be expected for a mobile reporting app [15]. The recommended time for submitting ICSRs to the ANRP, from the date of notification, is 15 days for cases of serious AEs and 90 days for non-serious cases [21]. The entry of paper reporting forms in the Med Safety app by the pharmacovigilance focal points would explain this fact. It is important to train HCPs and/or raise their awareness in reporting through the Med Safety app, especially given the context of insecurity in Burkina Faso [41], where it is sometimes difficult for certain peripheral health facilities to travel twenty kilometres or so to transmit reports to the health district's pharmacovigilance focal point.

The main limitation of our study is that all the evaluation statements of acceptability, usefulness and feasibility had the same weight for the calculation of the total score. It is therefore difficult to conclude which of these factors may play a more important role in the implementation of the use of Med Safety app longer term in our setting. Using a 4-point Likert scale can limit the subtlety of responses, thereby reducing the ability to capture intermediate opinions. In addition, we underestimated the non-participation rate, so we cannot generalize the results due to the low participation rate, however, we had respondents in all 13 health regions of Burkina Faso therefore providing broad geographical representation of the country [42]. Information on the health district or region and details of other health professionals were not available in the data extracted from VigiBase^®. As a result, we were unable to analyze geographical disparities in terms of reporting through the Med Safety app for targeted interventions. Further, we were unable to identify other groups of reporters using the Med Safety app apart from physicians and pharmacists, to tailor awareness-raising messages. Other studies will be necessary, particularly qualitative or mixed studies to further assess the value of the Med Safety app compared to the paper form for reporting AEs in Burkina Faso.