Does googling for preconception care result in information consistent with international guidelines: a comparison of information found by Italian women of childbearing age and health professionals

By means of separate interviews, we investigated the likely search strings for a web search on preconception care in two groups of individuals: 21 women planning a pregnancy and 18 health care professionals, including gynecologists, pediatricians, neonatologists and geneticists. Mean age of the interviewed women was 32.02 years, 80% of them reported to be employed while 65% had a University degree. The interviews were conducted by email. Interviewees were invited to provide at least two search strings that they have used to perform a web search on preconception topics. Respectively, the two groups generated a list of 35 and 38 search strings excluding double entries.

We then submitted each of the strings into Google Adwords (Keyword Tool), which is a tool that reports the actual web search volume for a specific country in a specific period of time (Table 1) [28]. Based on the obtained search volume ranks, we selected the 12 strings with the highest volume score for each group (women and health professionals: 24 total search strings).

We launched 24 searches on Google using each of the selected search strings in a single session on October 4th, 2011. For each search, the first search engine result page (SERP) for each search string was saved on a hard disk for off line review (list available on request). Each SERP showed a total of 10 websites. On the same day, the web pages were saved on a hard disk using an offline browser and limiting the sublink download to a depth of three levels. The sponsored websites appearing on the SERP were not included in the analysis.

In order to analyze the information quality on preconception care, we created a checklist of 19 items summarizing recommendations published by the American Journal of Obstetrics and Gynecology (AJOG) [2, 9].

The review process for each item was conducted to find out whether it corresponded to: a) absent, incorrect or incomplete recommendation; or b) correct and complete recommendation. The web pages resulting from the searches were reviewed by a single operator (EA) based on the checklist.

We recorded the type of website choosing among five categories, as follows: a) Medical/Public Agencies: web pages with a clear affiliation to a hospital, a University, a medical centre, or a national health agency; b) Communities and blogs: websites managed and maintained by independent groups, where an interactive discussion with external users is displayed (social networks were not included in this category) c) Magazines: news magazine websites; d) Wikipedia: web pages belonging to the web encyclopedia; and e) Others: web pages not classifiable in the previous categories, including YouTube and sites belonging to any brand or industry.

We estimated that 182 websites (91 per group) were needed for finding significant differences in consistent recommendations with guidelines between the simulated search performed by women and the one performed by health care professionals, with the following assumptions: 80% power, 5% significance level and an expected frequency of correct recommendations of 20% in searches performed by women and of 40% in searches performed by health care professionals. We took into account a potential number of double entries of 30%, due to website overlapping across searches conducted with different keywords, yielding a total sample size of 236.

We described the frequency of recommendations by group (women or health professionals) and we used the chi square test for comparing proportions. We performed logistic regression models to assess the effect of search strategy on presence of a recommendation consistent with AJOG guidelines [2], adjusted for website category and presence of commercial links. We considered results significant when p-values were below or equal to 0.05 for both the univariate and multivariate analyses.

We used STATA 11 statistical software for analysis.

The main strength of our study concerns the methodology: we investigated preconception information available on the Internet through a formal approach that helps avoiding the biases originating from arbitrary choices of keywords by study researchers. Several authors investigated quality of information found on the Internet regarding specific health topics, including cystic fibrosis [40], pediatric cancer [41], rheumatologic diseases [42], Mediterranean diet [43]. The search keywords used in these studies were chosen by the authors themselves, either simulating a search by a random user, or simulating a patient’s experience [44]. Weighing suggestions from a group of users against the frequency of Google searches should increase the likelihood to perform simulations closer to real life. We recommend this methodology for the evaluation of quality of medical information on the Internet.

The main limitation of our study is that it has a cross sectional design, and it analyses a static situation at the moment when the survey has been conducted. Presence of information on the Internet, on the contrary, is intrinsically dynamic. A periodic survey of the web would constitute a precious, alternative source of information on specific public health issues.

The analysis has been conducted using only Italian search strings, therefore the results may not be generalizable to other countries. Finally, we investigated presence of specific web information using generic search strings. This may have led to an underestimation of the presence of consistent recommendations on the web. Nevertheless, we can assume that this approach reflects the average search strategy most frequently chosen by the users.