Health Department Use of Social Media to Identify Foodborne Illness — Chicago, Illinois, 2013–2014

Jenine K. Harris, PhD1, Raed Mansour, MS2, Bechara Choucair, MD2, Joe Olson3, Cory Nissen, MS3, Jay Bhatt, DO2 (Author affiliations at end of text)

An estimated 55 million to 105 million persons in the United States experience acute gastroenteritis caused by foodborne illness each year, resulting in costs of $2–$4 billion annually (1). Many persons do not seek treatment, resulting in underreporting of the actual number of cases and cost of the illnesses (2). To prevent foodborne illness, local health departments nationwide license and inspect restaurants (3) and track and respond to foodborne illness complaints. New technology might allow health departments to engage with the public to improve foodborne illness surveillance (4). For example, the New York City Department of Health and Mental Hygiene examined restaurant reviews from an online review website to identify foodborne illness complaints (5). On March 23, 2013, the Chicago Department of Public Health (CDPH) and its civic partners launched FoodBorne Chicago (6), a website (https://www.foodbornechicago.org) aimed at improving food safety in Chicago by identifying and responding to complaints on Twitter about possible foodborne illnesses. In 10 months, project staff members responded to 270 Twitter messages (tweets) and provided links to the FoodBorne Chicago complaint form. A total of 193 complaints of possible foodborne illness were submitted through FoodBorne Chicago, and 133 restaurants in the city were inspected. Inspection reports indicated 21 (15.8%) restaurants failed inspection, and 33 (24.8%) passed with conditions indicating critical or serious violations. Eight tweets and 19 complaint forms to FoodBorne Chicago described seeking medical treatment. Collaboration between public health professionals and the public via social media might improve foodborne illness surveillance and response. CDPH is working to disseminate FoodBorne Chicago via freely available open source software

FoodBorne Chicago tracked Twitter messages using a supervised learning algorithm (7). The algorithm parsed tweets originating from Chicago that included "food poisoning" to identify specific instances of persons with complaints of foodborne illness. The geographic boundaries used by the algorithm also included some neighboring Chicago suburbs. However, follow-up inspections were conducted only at restaurant locations within the city limits. Tweets identified by the algorithm were reviewed by project staff members for indications of foodborne illness (e.g., stomach cramps, diarrhea, or vomiting) from food prepared outside the home. Project staff members provided feedback on whether each tweet fit the criteria, enabling the tweet identification algorithm to learn and become more effective over time.

For tweets meeting the criteria, project staff members used Twitter to reply. For example, Tweet: "Guess who's got food poisoning? This girl!" Reply: "That doesn't sound good. Help us prevent this and report where you ate here (link to Foodborne Chicago and a web form to report the illness)." The information in submitted forms went directly into the Chicago 311 system that handles all requests for nonemergency city services. Descriptive statistics were used to evaluate FoodBorne Chicago over its first 10 months of use and to compare the results of complaint-based health inspections of food establishments resulting from FoodBorne Chicago use with health inspections of food establishments based on complaints not submitted through FoodBorne Chicago. The comparisons did not include reinspections or routine inspections not based on a complaint.

During March 2013–January 2014, FoodBorne Chicago identified 2,241 "food poisoning" tweets originating from Chicago and neighboring suburbs. From these, project staff members identified 270 tweets describing specific instances of persons with complaints of foodborne illness. Eight of the 270 tweets (3.0%) mentioned a visit to a doctor or an emergency department. A total of 193 complaints of food poisoning were submitted through the FoodBorne Chicago web form. However, project staff members were not able to track how many of the 193 came from persons led to the form via Twitter and how many came from persons who visited the FoodBorne Chicago site on their own.

Of the 193 FoodBorne Chicago complaints, 19 (9.8%) persons indicated they sought medical care. The complaints identified 179 Chicago restaurant locations; at 133 (74.3%) locations, CDPH inspectors conducted unannounced health inspections. These 133 inspections amounted to 6.9% of the 1,941 health inspections of food establishments prompted by complaints during the study period. Of the 133 FoodBorne Chicago–prompted health inspections, 122 (91.7%) inspection reports identified at least one health violation, compared with 91.8% of inspection reports following complaints filed outside of FoodBorne Chicago during the same period.

Of the 133 FoodBorne Chicago–prompted health inspections 27 (20.3%) identified at least one critical violation, compared with 16.4% of the 1,808 inspections not prompted by FoodBorne Chicago. Critical violations indicate an "immediate health hazard" resulting in a high risk for foodborne illness. Critical violations must be fixed while the inspector is present or the restaurant fails inspection, has its license suspended, and is closed.* Twenty-nine restaurants (21.8%) reported via FoodBorne Chicago had at least one serious violation compared with 27.8% of restaurants not reported via FoodBorne Chicago. Serious violations indicate a "potential health hazard" that must be corrected within a timeframe determined by the health inspector, typically 5 days. If the serious violation is not fixed on re-inspection, the license is suspended, and the business is closed. Overall, at least one critical or serious violation was found in 37.6% of inspections prompted by FoodBorne Chicago and 37.2% of inspections from other complaints during the same period.

Some differences were noted in the distribution of specific violations between FoodBorne Chicago inspections and other complaint inspections. For example, 13.5% of FoodBorne Chicago inspections resulted in (critical) violation 3 (i.e., food not stored at appropriate temperatures), compared with 8.2% of other complaint inspections (Table). In addition, 14.3% of other complaint inspections reported (serious) violation 18 (i.e., food not protected from contamination), compared with 6% of FoodBorne inspections.

A total of 21 (15.8%) of the 133 restaurants reported through FoodBorne Chicago failed inspection and were closed; an additional 33 restaurants (24.8%) passed with conditions, indicating that serious or critical violations were identified and corrected during inspection or within a specified timeframe. Of the inspected restaurants with complaints not reported through FoodBorne Chicago, 25.8% failed and 14.2% passed with conditions. During the study period, among all restaurants inspected, FoodBorne Chicago–prompted inspections accounted for 4.3% of failed inspections and 11.4% of pass with conditions inspections.

Discussion

Foodborne illness is a serious and underreported public health problem with high health and financial costs. Emerging evidence on the effectiveness of social media for foodborne illness surveillance suggests mining tweets and restaurant reviews might aid in identifying and taking action on localized foodborne illness complaints that would otherwise go unreported (5,8,9). Using a new surveillance and response strategy, the CDPH identified and responded to 270 tweets about foodborne illness over 10 months in the Chicago area; 193 Chicago FoodBorne forms reporting foodborne illness were filed during this period. The majority of the 193 forms did not indicate that medical treatment was sought and so would likely not have been included in the usual surveillance numbers nor prompted inspections by the health department. Twenty-one of the reported restaurants failed inspection and were closed; 33 additional restaurants passed with conditions. Rates of critical and serious violations and failing inspections prompted by FoodBorne Chicago complaints were similar to those from inspections in response to other complaints during the same period.

The findings in this report are subject to at least two limitations. First, the Twitter application programming interface does not allow precise geographic filtering, and FoodBorne Chicago only used the keyword "food poisoning" to identify tweets. Second, it was not possible to determine how many of the 193 web form complaints were from persons directed to the form via Twitter. Project staff members were able to link 30 tweets directly to a corresponding complaint when report submitters clicked on the link in the "reply tweet" to access and complete the form. However, the number of persons who tweeted, did not click the link, and later accessed the Foodborne Chicago web form is unknown.

CDPH food inspectors and supervisors initially were concerned that use of Twitter would overburden them with increased inspections. However, by understanding the process better and seeing the success in finding violations, CDPH staff members have become supportive of obtaining potential foodborne illness information via Twitter.

CDPH and its partners are actively working to improve and disseminate the FoodBorne Chicago program. In an effort to increase the effectiveness of staff replies to complaints via Twitter, CDPH held four focus groups and plans an online survey. In addition, CDPH is currently working with the Boston Public Health Commission and the New York City Department of Health and Mental Hygiene to adapt FoodBorne Chicago for use in those two cities. FoodBorne Chicago also is available as open-source software on GitHub, an online host for sharing computer code with the public or a private audience.†

Acknowledgments

Margaret Eaglin, MPH, Chicago Department of Public Health; Kaitlyn Williams, Feinberg School of Medicine, Northwestern University, Chicago.

1Brown School, Washington University in St. Louis; 2Chicago Department of Public Health; 3FoodBorne Chicago (Corresponding author: Jenine K. Harris, harrisj@wustl.edu, 314-935-3522)

References

1. McCabe-Sellers BJ, Beattie SE. Food safety: emerging trends in foodborne illness surveillance and prevention. J Am Diet Assoc 2004;104:1708–17.
2. Mead PS, Slutsker L, Dietz V, et al. Food-related illness and death in the United States. Emerg Infect Dis 1999;5:607–25.
3. Campbell ME, Gardner CE, Dwyer JJ, Isaacs SM, Krueger PD, Ying JY. Effectiveness of public health interventions in food safety: a systematic review. Can J Public Health 1998;89:197–202.
4. Ordun C, Blake JW, Rosidi N, et al. Open source health intelligence (OSHINT) for foodborne illness event characterization. Online J Public Health Inform 2013;5.
5. Harrison C, Jorder M, Stern H, et al. Using online reviews by restaurant patrons to identify unreported cases of foodborne illness—New York City, 2012–2013. MMWR 2014;63:441–5.
6. Kuehn BM. Agencies use social media to track foodborne illness. JAMA 2014;312:117–8.
7. Feinerer I, Buchta C, Geiger W, Rauch J, Mair P, Hornik K. The textcat package for n-gram based text categorization in R. J Stat Softw 2013;52:1–17.
8. Sadilek A, Brennan S, Kautz H, Silenzio V. nEmesis: Which restaurants should you avoid today? Proceedings of the First AAAI Conference on Human Computation and Crowdsourcing. Palm Springs, CA, November 7–9, 2013.
9. Kang JS, Kuznetsova P, Luca M, Choi Y. Where not to eat? improving public policy by predicting hygiene inspections using online reviews. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Seattle, WA, October 18–21, 2013;22:58–77.