http://orcid.org/0000-0002-6384-9162
Figures
Abstract
Introduction
Poor recall during investigations of foodborne outbreaks may lead to misclassifications in exposure ascertainment. We conducted a simulation study to assess the frequency and determinants of recall errors.
Methods
Lunch visitors in a cafeteria using exclusively cashless payment reported their consumption of 13 food servings available daily in the three preceding weeks using a self-administered paper-questionnaire. We validated this information using electronic payment information. We calculated associated factors on misclassification of recall according to time, age, sex, education level, dietary habits and type of servings.
Results
We included 145/226 (64%) respondents who reported 27,095 consumed food items. Sensitivity of recall was 73%, specificity 96%. In multivariable analysis, for each additional day of recall period, the adjusted chance for false-negative recall increased by 8% (OR: 1.1;95%-CI: 1.06, 1.1), for false-positive recall by 3% (OR: 1.03;95%-CI: 1.02, 1.05), for indecisive recall by 12% (OR: 1.1;95%-CI: 1.08, 1.15). Sex and education-level had minor effects.
Discussion
Forgetting to report consumed foods is more frequent than reporting food-items actually not consumed. Bad recall is strongly enhanced by delay of interviews and may make hypothesis generation and testing very challenging. Side dishes are more easily missed than main courses. If available, electronic payment data can improve food-history information.
Citation: Gertler M, Czogiel I, Stark K, Wilking H (2017) Assessment of recall error in self-reported food consumption histories among adults—Particularly delay of interviews decrease completeness of food histories—Germany, 2013. PLoS ONE 12(6): e0179121. https://doi.org/10.1371/journal.pone.0179121
Editor: Martyn Kirk, Australian National University, AUSTRALIA
Received: May 13, 2016; Accepted: May 24, 2017; Published: June 22, 2017
Copyright: © 2017 Gertler et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: At study conduct, participants had been expressly warranted that the results are used for public health research only. Data will be available upon request to all interested researchers. Data request can be send to Hendrik Wilking (WilkingH@rki.de) or info@rki.de.
Funding: The funder of the study is the Robert Koch Institute, Berlin, Germany, as all authors of the manuscript and other contributors are employees of the institute and provided contributions during their normal working time. IC is partially contracted by the European Centers for Disease Control. No other institutions contributed funds or personnel to the project.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Interviewing sick persons concerning their food history is probably the oldest and most important method used for hypothesis generation in investigations of outbreaks of foodborne infectious disease. In a next step, analytical studies comparing interview data from sick and healthy people (case control or cohort design) allows for hypothesis testing. This strategy is recommended by international guidelines [1–4]. Interviewees´ poor recall can lead to exposure misclassification of food items which is a frequent experience of any public health epidemiologist which can lead to problems in identifying and testing hypotheses. Misclassification may hinder identification of contaminated vehicles in food-borne outbreaks [5]. If a vehicle is poorly remembered it can hardly be detected. At the same time, uncontaminated food items which are associated with the actual vehicle but better recalled could be wrongly suspected. This is especially problematic for outbreaks of diseases with long incubation periods including listeriosis, Hepatitis A or during the outbreak of Shiga toxin–producing Escherichia coli (STEC) O104:H4 infection in Germany 2011 [6]. Additionally, interview-based investigations are even more difficult when the disease sets patients into a state in which they cannot be interviewed.
During the STEC outbreak in 2011 in Germany, studies designed independently from the human recall capability have been particularly successful [7]. In one of the case-control studies, the cashless payment system of a company cafeteria used by the investigators provided food histories of patients and controls in a short time [8]. Other similar experiences of use of electronic payment data to investigate foodborne outbreaks were reported [9–11].
Little information is available about the actual frequency and determinants of recall error and misclassification of food items. In a study from 1986 epidemiologists investigated food recall during a luncheon in their institute. The investigators videotaped 32 attendees at the buffet table and interviewed them afterwards concerning their food selection. Consumers failed more often to report selection of desserts and bread compared to other servings, but influence of recall period could not be studied [12]. Similarly, Mann et al. observed attendees of a luncheon documenting their selection. Then, they compared the observed food choice with reported food history of the attendees from questionnaire-based interviews five days after the meal. They estimated sensitivity of recall of 88% and specificity between 73% and 93% [5].
To better understand determinants of food history recall, we simulated an outbreak investigation and used electronic data from personal payment cards as gold standard for food history in a cafeteria in Berlin, Germany, to check the recall of the consumers as ascertained using a paper-based questionnaire.
Material and methods
Visitors of a company cafeteria in a bank in Berlin were approached and interviewed during the regular opening hours at lunchtime (11:45 AM to 2:30 PM). In the morning of the same day, all employees with access to the cafeteria received an information letter via email, informing them about the interviews, the simulative and anonymous nature of the study. In the cafeteria, employees of the Robert Koch Institute (RKI), the responsible public health agency for the control of infectious diseases in Germany, approached the cafeteria guests to further inform them about the study and invite them to participate.
Participants were asked to fill in a standardized questionnaire about daily cafeteria visits and their food consumption of 13 different regularly served items in the cafeteria during the preceding three weeks (15 opening days). Additionally, personal characteristics (year of birth, sex, education degree), information on dietary habits (eating vegetarian, low-calorie-diet and having any food intolerance) and the personal customer identifier code number (ID) displayed on the card of the cashless payment system were retrieved. The questionnaire was designed using the same layout as the normal weekly menu of the cafeteria to increase ability to remember as might have been done by the field epidemiology team in a real outbreak scenario. Every day, the canteen offers three different main courses which, like four of the five offered side dishes and like two of the three desserts, vary every day. In addition, consumers may choose from a salad bar and may choose to take bakery (a roll or a bread) with their lunch. For analysis, we grouped the varying categories together, into 8 food item categories: main courses, side dishes, boiled potatoes (the non-varying third side dish), vegetable side dishes, desserts, fruit-salad (the non-varying third dessert), salad-bar (available every day) and bakery (available every day). To visualise the questionnaire, it is provided in supportive information files “S1 Questionnaire German” and “S1 Questionnaire English“.
The management of the cafeteria provided printed copies of the canteen payment of each participant’s IDs. All paper records were digitalised with software EpiData Entry (http://www.epidata.dk/). Double data entry and checks were performed for all data to reduce data entry errors.
For analysis the electronic payment information was used as standard and misclassifications were categorised as false-positive (reported eaten, not paid), false-negative (reported not eaten, but paid) and indecisive (Don’t know-answer). We used multivariable logistic regression separately for each misclassification category as dependent variable. We used recall period, sex, age group, degree of education, dietary habits and food item categories in each model as independent variables, without selection of variables. Statistical analysis was performed with STATA version 12.1C.
In this study, anonymous data on food histories and demographic characteristics were retrieved. No information on disease, disease-related states or disease-relevant exposures were collected. In detail, we asked healthy volunteers to report anonymously about their food intake in their canteen—there was no outbreak, nobody was asked for symptoms or about his/her medical condition, nobody was treated or underwent biomedical diagnostic tests or similar.
Participants were informed before and at the beginning of the survey about the simulation character of the study and were only included after written informed consent. We compared the reported food histories with those registered by the electronic payment system (identification by canteen card ID number).
To guarantee the highest possible level of anonymity, we requested and received approval of the data safety office at the Robert Koch Institute (the German National Public Health Institute). Therefore we consider this study to be in accordance with the Declaration of Helsinki without having applied for a review of an institutional ethics committee prior to the interviews.
Results
Study population
Altogether, 241 visitors responded to our survey. We excluded 18 of whom payment information could not be read or ID was ambiguous, 39 who declared to have used another person´s payment card at least once and39 who did not respond to one third or more of the inquired food items. Overall, we analysed data from 145 participants. None-responders did not differ from participants regarding age (p = 0.142) and gender (p = 0.472). Altogether 84/145 (58%) participants were female. Median age was 41 years, range: 22–64 years; 80/145 (55%) stated they hold a university degree. Of 28,275 (13x15x145) possible food recalls 1,180 (0.04%) were excluded because of no response or single purchase data could not been read out from the database.
Overall sensitivity and specificity of recall
Altogether 27,095 recalls were analysed. Of 3,523 purchased items, participants reported eating 2,268 (overall sensitivity of 72.8%), denied 846 and indecisively (Don’t know-answer) answered for 409. Of 23,572 items, actually not purchased, participants reported 20,931 as not eaten, 872 as eaten (overall specificity of 96.0%) and indecisively answered for 1,769. Altogether, participants indecisively recalled 2,178 (8.0%) food items. Median number of errors per participant was 11 with a range of 1–46. There was no significant association between the number of foods selected from the 13 investigated items and the number of reporting errors (p = 0.429). To allow better interpretation and adjustment of the results of other investigations, measures of performance of interviews are provided for each associated variable in detail in supportive information Tables A-C in S1 File.
Influence of recall period
All participants together paid for between 155 (day 18) and 255 (day 13) food items per day. False-negative recall increased with recall period (Fig 1). There were remarkably few bad recalls on day 14 and day 17 interrupting a continuous decline. The chance of false negative recall was twice as high after 21 days compared to 7 days (OR: 2.04; 95%-CI: 1.21, 3.45), while differences in false-positive recall are less pronounced (Table 1). In multivariable analysis, for each additional day, the chance for false-negative recall increased by 8% (OR: 1.08; 95%-CI: 1.06, 1.1), for false-positive recall by 3% (OR: 1.03; 95%-CI: 1.02, 1.05), for indecisive food recall by 12% (OR: 1.12; 95%-CI: 1.08, 1.15).
Influence of type of food
Compared to the main courses, other food items were generally less accurately recalled. The use of the salad bar in the cafeteria was especially prone to false-negative recall (OR: 2.29; 95%-CI: 1.41, 3.71) as well as false-positive recall (OR: 2.23; 95%-CI: 1.49, 3.33) and indecisive recall (OR: 1.82; 95%-CI: 1.26, 2.62). Similarly, vegetables and potatoes, although less likely as food vehicles of outbreaks, were poorly recalled comparing to main courses in all three categories. False-positive recall was less likely in bakery products and fruit salad.
Influence of demographic characteristics
The 59 males paid for 1,420 food items (24 per person) while the 84 females paid for 1,668 items (20 per person). While false-negative recall did not differ between males and females, the chance for false-positive recall was higher in males (OR 1.46; 95%-CI 1.11, 1.91). False-positive recall was also higher in vegetarians (OR 1.66; 95%-CI 1.09 2.52). False-negative recall did not vary by age or education. However, indecisive recall was more likely in vegetarians (OR: 4.30; 95%-CI: 1.16, 15.92) and in 20–29 year old participants compared to those aged 50–65 years (OR: 3.67; 95%-CI: 1.34, 10.00). Level of education of participants was not associated significantly with false-negative (OR: 1.16; 95%-CI: 0.79, 1.69), false-positive (OR: 1.02; 95%-CI: 0.77, 1.35) and indecisive recall (OR: 1.03; 95%-CI: 0.49, 2.17).
Effort for data acquisition
Data collection based on the questionnaire required presence of 10 persons in the cafeteria for 3 hours to contact and inform visitors, receive interviewees´ informed consent, to distribute and receive the questionnaires. In comparison, to extract the data from the payment system required one staff for 2 hours.
Discussion
This study shows that exposure misclassification can be a significant problem in the investigation of foodborne infectious disease outbreaks using data from food history interviews. The misclassification can be differential regarding the inquired food items, leading to an underestimation of measures of association of the true outbreak vehicle and false incrimination of other vehicles. For example, this scenario happened during investigation of large outbreaks of STEC in Germany caused by sprouts [7,13] and Salmonella Saintpaul in the USA caused by jalapeño and serrano peppers [14,15]. In both outbreaks epidemiological association from early studies initially identified different products. We found that the proportion of false-negative recalls is higher than false-positive, indicating that forgetting to report consumed foods is more likely than reporting food-items actually not consumed. Higher specificity and lower sensitivity of recall were reported before in a similar experiment [12].
Influence of recall period
While false-negative recall and indecisive recall strongly increases with time, false-positive recall does not. After recall periods of two weeks or more, around 20% of all items do not get reported correctly which means lower power in epidemiological studies to detect outbreak vehicles. The high chance for false-negative recall is particularly problematic for hypothesis generation. Outbreak vehicles may be underestimated or overseen only because the exposure lies two weeks or more in the past.
Influence of type of food
Decker et al. reported more accurate recall of more complex or distinctive dishes compared to a range of relatively similar vegetable side dishes. This is supported by our findings suggesting better recall of main courses compared to all other dishes, particularly compared to unvarying daily offerings like fruit salad and bakery. Contrarily to Decker et al., we did not find indication of significant misclassification of desserts [12]. However, better recall of main courses needs to be taken into account when evaluating explorative findings, to avoid missing vehicles in side dishes. Particularly consumption at the salad bar is poorly recalled which is in accordance with observations from an outbreak in Germany [7,8]. Unfortunately, we could not obtain information on different salad bar items as this was not included in the billing data.
Influence of demographic characteristics
Altogether, respondent-related variables have a smaller impact than recall time and food item variables. Our findings confirm a higher chance for false-positive recall in men. This is in accordance with findings of Decker et al. [12]. Increasing age does not lead to poor recall in our study. Participants who declare being vegetarian have a higher chance for false-positive or indecisive recall despite the assumption that sensible diet leads to better recall of food consumption. However, this finding is based only on small numbers: only few study participants indicated being vegetarian (n = 9) or eating low-calorie food (n = 10).
Effort for data acquisition
The interviews of participants required 15-times more work compared to the extraction of the electronic information from the billing system. Therefore the latter provides potential to make data collection quicker, more accurate and allows for larger study populations. However, it’s only applicable if a large proportion of cases and non-diseased persons pay cashless. An electronic interface between billing systems and databases of public health agencies might accelerate investigations.
Limitations
Unfortunately, in our simulation study only printouts were available, demanding manual data entry. The bank as employer and the cafeteria allowed us only limited interview time. In a real scenario, such would be much longer and provide more detailed information especially regarding the different main courses and regarding individual food choices. Furthermore, the data from the payment system was only specific on the menu level and not on the choice of the visitor. Therefore, participants were not asked if they had eaten anything containing a specific ingredient and they did not have the possibility to report items which were not on the questionnaire. In a real-life scenario investigations on ingredients level might be complemented by interviews with the chefs and the kitchen staff.
One main limitation of recall-independent electronic data is that it cannot tell if paid food items were actually eaten by the participant. But we think that this misclassification is of minor importance compared with misclassification due to incorrect recall.
Conclusion
Our results show that earliness of interviews of patients during foodborne outbreaks is essential, particularly when the pathogen and disease have long incubation periods. At least, hypothesis generating exploratory interviews should be performed before failure of recall. If available, electronic payment data for food history collection can facilitate and accelerate investigations, especially if patients are very sick or even dead. Data from our study can be used for better interpretation and adjustment of the results of surveys, case-control studies and cohort studies in outbreaks.
Supporting information
S1 File.
Table A: False-negative food recalls by different groups for reported food selections, Berlin, Germany, 2013. Table caption: Univariable, Odds ratio and 95% confidence interval derived from logistic regression; CI, confidence interval; Recall period defined as the interval from the day of food consumption to the day of the interview in days. Table B: False-positive food recalls by different groups for reported food selections, Berlin, Germany, 2013. Table caption: Univariable, Odds ratio and 95% confidence interval derived from logistic regression; CI, confidence interval; Recall period defined as the interval from the day of food consumption to the day of the interview in days. Table C: Indecisive (Don’t know-answer) food recalls by different groups for reported food selections, Berlin, Germany, 2013. Table caption: Univariable, Odds ratio and 95% confidence interval derived from logistic regression; CI, confidence interval; Recall period defined as the interval from the day of food consumption to the day of the interview in days.
https://doi.org/10.1371/journal.pone.0179121.s001
(DOC)
S1 Questionnaire German. The questionnaire was designed using the same layout as the normal weekly menu of the cafeteria.
Each column represents a working day when the canteen was open. The lines represent the 13 different food categories from which participants could choose a different serving every day. For analysis, the varying categories were grouped together: Giving 8 food item categories: main courses, side dishes, boiled potatoes (the non-varying third side dish), vegetable side dishes, desserts, fruit-salad (the non-varying third dessert), salad-bar (available every day) and bakery (available every day).
https://doi.org/10.1371/journal.pone.0179121.s002
(PDF)
S1 Questionnaire English. The English questionnaire is a translation of the German original.
It was not used in the study but produced exclulsively to facilitate reading of this report.”
https://doi.org/10.1371/journal.pone.0179121.s003
(PDF)
Acknowledgments
We thank Ute Rexrodt, Marlene Kretschmer, Hannes Ulrich, Benjamin Blümel, Katja Alt, Elise Curo Gutierrez, Ariane Böttcher, Caterina Lindig, and Susanne Behnke for technical assistance with interviews and data entry. We thank Katharina Alpers, Manuel Dehnert and the European Programme for Intervention Epidemiology Training (EPIET) for advice on the study design and manuscript.
Author Contributions
- Conceptualization: HW KS MG.
- Data curation: HW MG.
- Formal analysis: HW IC MG.
- Funding acquisition: HW.
- Investigation: MG.
- Methodology: HW MG IC.
- Project administration: MG HW.
- Resources: KS.
- Supervision: HW KS.
- Validation: HW IC MG KS.
- Visualization: MG HW.
- Writing – original draft: MG.
- Writing – review & editing: MG HW KS IC.
References
- 1.
Centers for Disease Control and Prevention (CDC). Investigating Outbreaks. 2008:http://www.cdc.gov/foodsafety/outbreaks/investigating-outbreaks/ Accessed: 30th May 2017. 10.1007/s12082-008-0148-1.
- 2.
European Centre for Disease Prevention and Control (ECDC). Toolkit for investigation and response to Food and Waterborne Disease Outbreaks with an EU dimension. 2008:http://ecdc.europa.eu/en/healthtopics/food_and_waterborne_disease/toolkit/Pages/index.aspx Accessed: 30th May 2017. 10.1007/s12082-008-0148-1.
- 3.
Robert Koch Institute (RKI). Untersuchung von lebensmittelbedingten Ausbrüchen (in German). 2008:http://www.rki.de/DE/Content/Infekt/Ausbrueche/LM/lebensmittelbedingte_Ausbrueche_node.html#doc5225236bodyText2 Accessed: 30th May 2017. 10.1007/s12082-008-0148-1.
- 4.
World Health Organization (WHO). Foodborne disease outbreaks: Guidelines for investigation and control 2008:http://www.who.int/foodsafety/publications/foodborne_disease/outbreak_guidelines.pdf Accessed: 30th May 2017. 10.1007/s12082-008-0148-1.
- 5. Mann JM. A prospective study of response error in food history questionnaires: implications for foodborne outbreak investigation. American journal of public health. 1981;71(12):1362–6. Epub 1981/12/01. pmid:7316002;.
- 6. Werber D, King LA, Muller L, Follin P, Buchholz U, Bernard H, et al. Associations of age and sex with the clinical outcome and incubation period of Shiga toxin-producing Escherichia coli O104:H4 infections, 2011. American journal of epidemiology. 2013;178(6):984–92. pmid:23935124.
- 7. Buchholz U, Bernard H, Werber D, Bohmer MM, Remschmidt C, Wilking H, et al. German outbreak of Escherichia coli O104:H4 associated with sprouts. The New England journal of medicine. 2011;365(19):1763–70. Epub 2011/10/28. pmid:22029753.
- 8. Wilking H, Götsch U, Meier H, Thiele D, Askar M, Dehnert M, et al. Identifying risk factors for shiga toxin-producing Escherichia coli by payment information. Emerging infectious diseases. 2012;18(1):169–70. Epub 2012/01/21. pmid:22261344;
- 9. Ethelberg S, Smith B, Torpdahl M, Lisby M, Boel J, Jensen T, et al. Outbreak of non-O157 Shiga toxin-producing Escherichia coli infection from consumption of beef sausage. Clinical infectious diseases. 2009;48(8):e78–81. Epub 2009/03/11. pmid:19272017.
- 10. Fretz R, Sagel U, Ruppitsch W, Pietzka A, Stoger A, Huhulescu S, et al. Listeriosis outbreak caused by acid curd cheese Quargel, Austria and Germany 2009. Euro surveillance. 2010;15(5). Epub 2010/02/11. pmid:20144447.
- 11. Shah L, MacDougall L, Ellis A, Ong C, Shyng S, LeBlanc L. Challenges of investigating community outbreaks of cyclosporiasis, British Columbia, Canada. Emerging infectious diseases. 2009;15(8):1286–8. Epub 2009/09/16. pmid:19751593;
- 12. Decker MD, Booth AL, Dewey MJ, Fricker RS, Hutcheson RH Jr., Schaffner W. Validity of food consumption histories in a foodborne outbreak investigation. American journal of epidemiology. 1986;124(5):859–63. Epub 1986/11/01. pmid:3766519.
- 13. Frank C, Faber MS, Askar M, Bernard H, Fruth A, Gilsdorf A, et al. Large and ongoing outbreak of haemolytic uraemic syndrome, Germany, May 2011. Euro surveillance. 2011;16(21). pmid:21632020.
- 14. Barton Behravesh C, Mody RK, Jungk J, Gaul L, Redd JT, Chen S, et al. 2008 outbreak of Salmonella Saintpaul infections associated with raw produce. New England Journal of Medicine. 2011;364(10):918–27. pmid:21345092
- 15. Taylor E, Kastner J, Renter D. Challenges involved in the Salmonella Saintpaul outbreak and lessons learned. Journal of Public Health Management and Practice. 2010;16(3):221–31. pmid:20357608