Social media posts and online search behaviour as early-warning system for MRSA outbreaks

We conducted an exploratory study in which we compared information about MRSA outbreaks derived from social media and online search behaviour to the official Dutch reference standard. MRSA specific searches were performed for the time period between January 1st, 2015 and March 31st, 2017. As reference standard for MRSA outbreaks, SO-ZI/AMR was used [8]. It provides information on official outbreaks including the date of the outbreak and the region and type of health care facility (e.g. hospital or nursing home). The geographical scope of the study was The Netherlands; therefore, we only searched using the Dutch language.

To capture social media posts about MRSA (all publicly shared social media posts by individuals or organizations), we used Coosto, a social media monitoring tool [9]. This tool has proven to be a valuable source of social media information and is currently in use by the Dutch government to monitor the quality of healthcare organizations [7]. It provides the exact time and, if available, the location of the message in various social media sources, including Facebook and Twitter. Presently, its database includes posts in the Dutch language.

In addition, Google Trends was used to assess online search behaviour [10]. This tool provides insight into the search behaviour based on specific searches performed in Google Search. It provides the relative frequency of searches for different countries and regions. Google Trends has been used for early detection of influenza outbreaks [6]. Although multiple search engines are being used in The Netherlands, we limited our searches to Google Trends since Google covers over 80% of Internet searches, capturing the overall majority of Internet searches [11].

We searched Coosto for publicly available social media posts about MRSA with the following search query: (“mrsa” OR “methicillin-resistant Staphylococcus aureus” OR “methicillin resistant Staphylococcus aureus” OR “meticilline-resistente Staphylococcus aureus” OR “meticilline resistente Staphylococcus aureus”). Several preliminary searches revealed that both Dutch and English names had to be included, since English terminology was sometimes used in Dutch social media posts and the combination maximized the number of hits. Furthermore, the word ‘outbreak’ was not included in the search query, since the preliminary searches showed that searching for ‘outbreak’ resulted in a large number of irrelevant hits, and that combining ‘outbreak’ with MRSA via Boolean search (“AND”) negatively affected the sensitivity of the search. Based on the results of the preliminary searches, we performed manual inspections of results with 25, 15 and 10 hits per day. In general, the lowest number of hits per day would result the highest number of potential outbreaks. However, from the above comparison, we concluded found that 10 posts per day was the minimum number of hits, or ‘critical mass’ [7] to identify a potential MRSA outbreak, Consequently, we set (≥10 hits) as criterion for a potential outbreak. For all posts on a specific day, that met this criterion, we identified whether a potential outbreak was discussed, meaning that MRSA was mentioned in relation to a Dutch healthcare institution and indicating a present or potential outbreak. The latter could consist of (but not limited to) patients, relatives or employees found or suspected with MRSA, hospital wards closed due to MRSA, any other information about a present or expected MRSA outbreak shared by the institution, its employees, government, or other any other individual (e.g., patients or relatives). Days meeting all criteria were marked as ‘representing a potential MRSA outbreak’ and all other days as ‘not representing a potential MRSA outbreak’. Dates, number of hits, institution and geographical area and outbreak (YES/NO) were subsequently stored in a research database.

Regarding the searches in Google Trends, we used similar terms, but with individual searches for each term. Google Trends presents search interest of topics on a scale from 0 to 100 per day instead of the absolute number of searches, thus every search will have at least one day with the maximum score of 100, even when absolute search numbers are low during a particular time period (e.g., a time period without any outbreaks). This system characteristic required a different way of defining days with potential outbreaks. Assuming that a Dutch MRSA outbreak occurred at least once every 3 months, we used the mean search interest of 3 months in our analyses. We extracted search interest per day, as well as geographical information (province) for days with potential outbreaks.

All statistical analyses were done in SPSS version 22. The database with search results from Coosto and Google Trends was compared to data from the official SO-ZI/AMR database. In case of consecutive days with the same potential outbreak, we defined this as a single outbreak, both for Coosto and Google Trends. To assess the validity of Coosto to detect potential MRSA outbreaks, we calculated the overall sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) with 95% confidence intervals (CIs). Furthermore, we stratified the analyses by type of healthcare institution affected.

For the Google Trends data, a search score was calculated for each day:

Google Trends Score = (relative search volume - mean relative search volume) / standard deviation.

with the mean relative search volume and standard deviation based on the preceding 3 months. Using SO-ZI/AMR as the reference standard, we calculated the area under the curve (AUC) for 7 days before until 7 days after an outbreak. Furthermore, as the optimal cut-off value to detect an outbreak based on online search behaviour is unknown, we determined the Google Trends Score that maximized the sum of sensitivity and specificity. For the present study, we used a cut-off value of 2*SD to detect a potential outbreak with Google Trends. Finally, we calculated the Pearson correlation coefficient to assess the association between the number of posts related to MRSA detected with Coosto and the Google Trends Score.

In this study, we compared information about potential MRSA outbreaks retrieved from social media posts and online search behaviour in The Netherlands to the national notification reference standard. We found that simple online (social media) searches do provide additional information about potential MRSA outbreaks in The Netherlands compared to the reference standard. These promising findings suggest that supervisory bodies such as SO-ZI/AMR may enrich their palette of data sources with more dynamic information from social media and other online sources such as search engine data. However, the validity of the online sources Coosto and Google trends needs further investigation. Some things need to be discussed.

The sensitivity of the social media monitoring tool Coosto to detect MRSA outbreaks was low and therefore a substantial number of true outbreaks will be missed when relying on this data source. However, its specificity was high, indicating a relatively small number of false positive outbreaks detected by Coosto. Interestingly, the opposite was observed for Google Trends, with a higher sensitivity and lower specificity, indicating that Google Trends detects more potential MRSA outbreaks but that many of these represent false positive signals. This difference between the two online data sources may be explained by their nature: social media posts provide more detailed information on MRSA outbreaks than online searches, but the patients and healthcare workers involved in MRSA outbreaks may be more reluctant to post a message about the outbreak on social media than to search for information online. In addition, it is impossible to distinguish between online searches for actual outbreaks and searches for random issues related to MRSA using Google Trends.

To the best of our knowledge, this was the first study using online search engines for social media posts and internet search behaviour on MRSA outbreak detection. A study by Lui et al. used search terms from the social media platform Baidu to identify Noro virus epidemics [12]. They concluded that several limitations exist to using Internet to monitor epidemics but that it still might have value as additional tool, particularly when other monitoring systems are lacking. Also, the importance of social media as an early warning system in addition to traditional slow reporting mechanisms has been emphasized [13]. In general, this might be the case for all outbreaks as notification is only done after firm confirmation of the outbreak. Consequently, the notification date in SO-ZI/AMR is “delayed” by several days.

Additional potential outbreaks were found via social media, which were not in the SO-ZI/AMR database. Most of these outbreaks occurred in nursing homes for which notification is not mandatory, but on occasion even hospitals were shown as non-compliant in reporting outbreaks. The fact that more dynamic data sources could have value compared to traditional slow reporting mechanisms has also been recognized in public health, where social media are used as an early warning system for disease outbreaks [14].

The main strength of this study is the use of dynamic online content from social media and search engine behaviour in combination with an official reference (SO-ZI/AMR). Using predefined selection criteria, this allowed us to efficiently study the value of social media and online search behaviour via both Coosto and Google Trends. Coosto on occasion is limited by the fact that it may not always be possible to determine whether a potential outbreak is actually a true outbreak. Google Trends has even more difficulty in this determination, for using this data source, it is hard to link potential outbreaks to specific organizations, since it does not provide specific names or locations. A refinement of search mechanism of the freely available default Google Trends software might allow an increase of its sensitivity and specificity.

The extent of information patients and caregivers get when confronted with an outbreak may influence their search behaviour. If the information is complete and offered right away, as it is customary in many Dutch hospitals, it may become more difficult to detect the outbreak using social media and search engine behaviour.