Assessing changes in the food retail environment during the COVID-19 pandemic: opportunities, challenges, and lessons learned

Consistent with previous studies that utilized health licensing and permits databases to enumerate restaurants and food retailers within a specified location [9, 19], we identified publicly available datasets from local and state public health agencies to enumerate the baseline sample of restaurants and food retailers in NYC. For restaurants, we used the NYC Department of Health and Mental Hygiene (DOHMH) Restaurant Inspections database [20]. The dataset contains names, locations, and inspection results for restaurants, cafeterias, and similar food outlets inspected by the DOHMH up to three years prior to the most recent inspection. We excluded duplicate entries. Though inspection results are usually updated daily, due to the COVID-19 pandemic, the DOHMH suspended restaurant inspections on March 16, 2020 [21]. Therefore, this list was the most comprehensive and up to date list of restaurants available for the NYC prior to the COVID-19 pandemic.

For food retailers, we used the New York State (NYS) Retail Food Stores list, provided by the NYS Department of Agriculture and Markets. The dataset contains names and addresses of food retailers – including convenience/corner stores (also commonly called “bodegas” in NYC), grocery stores, supermarkets, wholesale superstores, bakeries, seafood markets and more –licensed in NYS in 2019 [22]. Food retailers were sampled from this registry. Both datasets were cleaned manually and using Stata (v.15.0, StataCorp LLC, College Station, TX), to check for duplicate entries and errors in names or addresses.

Due to the constraints of COVID-19 crisis and the multitude of food outlets in NYC, we selected three types of neighborhoods, across two boroughs, to evaluate and compare how the COVID-19 pandemic has affected the food retail landscape. We first selected Chinese ethnic neighborhoods (Chinatown in Manhattan; Sunset Park in Brooklyn). Reporting from news outlets and community partners suggested that the pandemic was having a disproportionate effect on businesses in Chinatown due to decreased tourism and travel from China as well as general fears of contracting the virus, whereby losses in business was not comparable to the actual rates of COVID-19 infection [5, 6].

We selected one higher- and one lower-resourced comparison neighborhood in both Manhattan and Brooklyn based on objective and subjective measures. Indicators from the NYC Community Health Profiles – an in-depth, detailed data source on the 59 Community Districts (CDs) in NYC, developed and maintained by the DOHMH [23] – included in the consideration were key correlates of food access (supermarket to bodega ratio), demographics (population size, foreign born, limited English proficiency), socioeconomic factors (poverty, unemployment, rent burden, uninsured), health behaviors (physical activity, fruit & vegetable, and sugary drink consumption) and chronic conditions (obesity, diabetes, hypertension). No neighborhoods were unequivocally determined to be highest- or lowest-resourced based on aforementioned characteristics. Consequently, we additionally considered anecdotal evidence from research staff and community partner discussions, most of whom are longtime NYC residents, in addition to these factors. These neighborhoods have historically been used as examples of disparities, where the Upper East Side (UES) is often referenced as one of the overserved neighborhoods, in terms of food and medical access comparable to the needs of the residents, in Manhattan and NYC, overall. NYU Center for the Study of Asian American Health has established community partnerships through community health workers, volunteers, and member at organizations including but not limited to the Apicha Community Health Center, Charles B Wang Community Health Center, and the Chinese-American Planning Council [24]. A full list of partnerships can be found here: https://​med.​nyu.​edu/​departments-institutes/​population-health/​divisions-sections-centers/​health-behavior/​section-health-equity/​community-engagement-education/​partnerships-coalitions. The final selections of higher- and lower-resourced neighborhoods were the UES and East Harlem in Manhattan, and Park Slope and Brownsville in Brooklyn, respectively (Table 1).

Restaurant and food retailer databases included geographic information in the form of street addresses and latitude and longitude coordinates. We created maps defining NYC neighborhoods using neighborhood tabulation areas (NTAs) – geographic units created by the DOHMH – on top of which, coordinates of food outlets were plotted. NTAs were purposefully selected to define neighborhoods instead of CDs, referenced previously, for logistical and feasibility reasons. NTAs are smaller geographic regions than CDs, so they were easier to cover given the timeframe of COVID-19 and number of team members available. Because NTA boundaries were created for data purposes, they may not definitively represent neighborhoods. Given these constraints, we extended the NTA boundaries 50 m (m) in an attempt to capture neighborhoods more completely. Additionally, in Chinatown and Sunset Park, the two Chinese ethnic neighborhoods, we expanded boundaries further based on conversations with community partners. In Chinatown, the western NTA-defined border was Bowery Street and Canal Street. This was extended to Mulberry Street and Kenmare Street. In Sunset Park, the south-western NTA-defined border was 60th Street and 6th Avenue. This was extended to 50th Street and 5th Avenue. (Fig. 1) In Chinatown Manhattan, prior research had established the boundaries of the neighborhood during an in-person walk through [25]. However, due to limitations in resources and time-constraints, we did not physically check the boundaries of these neighborhoods, i.e., that Chinese stores were significantly lower across certain streets, prior to initiating data collection.

Using ArcGIS software (ArcGIS [GIS software]. Version 10.0. Redlands, CA), food retailers and restaurants were plotted overlaying the NTA boundaries, which were imported using a DOHMH provided shapefile. https://​data.​cityofnewyork.​us/​City-Government/​Neighborhood-Tabulation-Areas-NTA-/​cpf4-rkhq We first re-drew and extended the boundaries of the NTAs in ArcGIS. Next, we exported all restaurants and food retailers located within the new boundaries to create a census of restaurants and food retailers in the six neighborhoods. Data included in the new lists were the names of the restaurant or food retailers, locations, and phone numbers when available.

To develop the baseline lists of restaurants and food retailers open in 2019, we searched for evidence online that the food outlet was open prior to 2020. For the purposes of this study, we defined the pre-pandemic period as prior to January 2020. Though NYS On Pause, the statewide mandated closure of non-essential businesses, did not take effect until March 22, socioeconomic ramifications of COVID-19 began to take effect in Chinatown as early as January 2020 [5, 6]. Research staff searched for evidence that the food outlet was open in 2019, starting with GSV imagery, which contained images from fall of 2019. Additional methods were used if the status could not be ascertained using GSV because something was obstructing the view of the outlet or there were no recent images. Staff then searched for other available evidence with a calendar date in the form of reliable customer reviews, website information, online photos, or reliable news articles. Dates of evidence were recorded, and food outlets were classified as open pre-COVID-19 if they were open prior to 2020. If food outlets were found to be closed in 2019, they were marked as closed pre-COVID-19. Food outlets were marked as ‘not found’ if there was evidence that the food outlet had changed names or ownership in 2019. Stores marked ‘pre-COVID-19 closure’ and ‘not found’ were later excluded from the baseline database. Food outlets confirmed to be open during 2019 were included in the baseline database.

We created standardized restaurant and food retailer Excel spreadsheets to collect data on the current status of food outlets in the baseline database. Included in the retail food data collection spreadsheet was the tradename (doing business as, “DBA”), location (including the address and coordinates of the business), phone number, status of the store during the COVID-19 pandemic, pre-COVID-19 pandemic status of the store, and extra checks (phone call and in-person) (see Supplemental File). These columns were also included in the restaurant data collection spreadsheet in addition to columns to account for whether the store offered curbside-pickup, contactless delivery, take-out, or dine-in. Both restaurant and food retailer spreadsheets also included a notes column to allow research staff to comment on additional information that did not fit into one of the aforementioned columns.

We used three-step verification system to ascertain the status of all food outlets in this study. The first step was the Google Maps search and online verification. We employed manual web-scraping to identify two independent sources certifying the status of the outlet. If two sources could not confirm the status of the outlet, the next steps involved phone calls and/or in-person visits. Study team members entered data collected on restaurants and food retailers from these sources into the corresponding columns in the Excel spreadsheets. The timeline of the process is available in the Supplemental File. Each step is described in detail below.

To assess the status of food outlets during COVID-19, research staff first searched for the food outlet on Google Maps (https://​www.​google.​com/​maps) using the DBA name and address. Any information provided by Google Maps, such as the hours and days the store was open, live visit data (updated real-time information representing how active a location is compared to usual activity managed by Google) [26]; or a temporarily or permanently closed notice was extracted. For some restaurants, Google Maps also provided information about dine-in, take-out and delivery. This data was extracted when available. If Google Maps did not have information available on whether the food outlet was open, (e.g., listed the phone number but no hours; or the store was not recognized) then a note was made that no information was found. For all Google Maps checks, the date the check was conducted was also noted. As Google Maps has a feature enabling users to take notes, we were able to provide real time updates whether a food outlet was open or closed. An additional internet verification step was then conducted to confirm that the Google Maps data was accurate, through a manual data scrape of other online sources, including Google reviews, Yelp, Facebook, Seamless, Grub Hub, the food outlet’s website as well as other websites (e.g., blogs, newspapers). A food outlet was considered open during COVID-19 if there were recent posts or reviews (in the past four months: March – June 2020) about the food outlet. The date of the secondary verification confirming the food outlet as open or closed was also noted. Food outlets were considered ‘temporarily closed’ if they were marked as such on Google or the food outlets’ website. Food outlets were considered ‘indefinitely closed’ if Google had a red ‘permanently closed’ notice or Yelp had an alert that ‘Yelp users have reported this location as closed.’ Food outlets with no evidence online about their current operation status were identified as needing a phone call verification.

Phone numbers were provided in the original restaurant dataset and retrieved from online sources in the food retailer dataset. When calling, research staff verified the name of the food outlet and asked whether they were currently open. If possible, research staff ascertained if the food outlet had closed in the past 6 months and if restaurants were offering delivery, take-out or dine-in services. In the case of no answer, staff noted the time of the call (e.g. morning or afternoon) and called back at a different day and/or time – i.e., if the call was made in the morning, the follow-up call was made in the afternoon. This was especially important for restaurants, which may be open during lunch and evening hours but not in the morning and mid-afternoon. If two calls went unanswered, the food outlet was marked as needing an in-person check. In-person checks were also required for food outlets that had no working phone number (i.e., no longer in service, wrong number, or not available). Some food outlets needed in-language follow-up, particularly those in the ethnic or more racially/ethnically diverse neighborhoods (Chinatown, Sunset Park, East Harlem). Two research staff fluent in Mandarin Chinese and one research staff with conversational Spanish ability conducted these additional calls.

Location data about food outlets needing an in-person check were uploaded to Google Maps using a Google login that was shared with the CoClo team. We utilized the Lists and Notes functions of Google Maps, which allowed individuals in the field wrote notes directly in Google Maps. These notes synched in real-time enabling researchers to split up in the field and keep updated on which outlets had been checked and which remained. We also provided the option of updating data on paper lists, which individuals later uploaded into Google Maps. The in-person check required marking whether food outlets were open, closed, or no longer the same business on the date of the check. If businesses appeared closed at the time of the in-person check, researchers observed the surrounding area and checked whether the hours of operation were posted on the business to judge the status. For example, businesses that appeared closed but had hours available indicating they were open later or on a different day were marked as open. Businesses that were closed with a for sale or rent sign in the window were marked as indefinitely closed while those that appeared vacant but no evidence of permanent closure (e.g., for rent sign) were marked as temporary closed. Some stores required an in-language follow-up when the business sign was only available in Chinese. Photos were taken of these stores and reviewed by research staff with ability to read written Chinese.

Based on the three-step verification process, restaurants and food retailers were classified as ‘Open,’ ‘Temporarily Closed’ or ‘Indefinitely Closed.’ The first round of data collection was completed summer of 2020. We conducted another round of data collection during spring of 2021 to evaluate longitudinal changes. Person-hours, in terms of time spent on each task, were not measured, and depended on the size of the neighborhood and number of food outlets. The online checks and in-person visits tended to take the longest, as all outlets were checked online initially, and some neighborhoods spanned a larger geographic area. The second data collection round was finished faster than the first, suggesting that additional training and experience could improve efficiency.

Building on prior work, the North American Industry Classification System (NAICS) was adapted to classify food outlets [27]. The NAICS is the standard used to classify business establishments by type of economic activity [28]. Rather than assigning businesses the NAICS codes, the definitions of the NAICS codes were used to guide categorizations of food retailers and restaurants, with some definitions aggregated into a single category (e.g., specialized food store). Definitions of categories used in this project can be found in the Supplemental File.

For restaurants, two new variables were created corresponding to the category of service - limited service (NAICS 722513); full service (NAICS 722511); cafeterias, grills buffets, and buffets (NAICS 722514); snack and nonalcoholic beverage bars (NAICS 722515); and drinking places (NAICS 722410) – and service types – e.g., chain vendors, takeout, casual, fine dining, bakeries, bars. Full service restaurants were then further categorized into 1) Casual-Dining or 2) Fine-Dining, based on definitions outlined by Hwong & Ok (2013) [29].

Stores on the food retailer database were classified into the following groups: liquor stores (NAICS code 445310), smoke shops (NAICS 453991), convenience stores (NAICS 447110 and 445,120), grocery stores (NAICS 445110), specialized food stores (NAICS 445200, 445,210, 445,230, 445,292) and pharmacies (NAICS 446110). Categorization of food retailers was done using the DBA name. Well-known (e.g., Rite Aid, 7-Eleven, Food Stop) and straightforward DBA names (e.g., Fortune Meat Market, Stanley’s Pharmacy, Dubai Smoke Shop) were used to classify food retailers. After this was completed, an online search was used to confirm the type of business for the remaining stores with ambiguous names. Another variable was created to represent whether the food retailer was part of a larger chain. Names were scanned to identify easily recognizable chain food outlets (e.g., Whole Foods Market, Gristedes). Those that could not be identified by name alone were searched online to determine whether they met the DOHMH’s definition of a chain outlet (i.e., 15 locations or more nationally) [30].

To quantify differences in the data collection steps required across neighborhood, we compared percentages of stores needing call checks, and in-person checks using chi-squared and post-hoc Bonferroni analyses in Stata (v.15.0, StataCorp LLC, College Station, TX). Maps of food outlet closures were created using ArcGIS.

We found that ascertaining the status of food outlets was more difficult in certain neighborhoods compared to others. Of the 1832 restaurants across Manhattan and Brooklyn, 78% were verified using the online data extraction process only. There were 404 restaurants (22% of total) that required a phone-call follow-up and 169 (9% of total) that required an in-person follow-up. Only one restaurant required an in-person check without a prior phone call, as all other phone numbers were available from the initial restaurants database. Aggregate neighborhood data is available in Table 2. The UES required the lowest percentage of call checks (12%) and in-person checks (4%), while Sunset Park required the greatest percentage of call checks (50%) and in-person checks (17%). (Supplemental Table 1).

Of the total 753 food retailers across all neighborhoods, 417 stores (55% of total) were completed using the online data extraction process only. A greater percentage of Brooklyn stores required follow-ups (50%) than Manhattan stores (38%). Overall, 218 stores (29% of total) required a call check. There were 221 stores (29% of total) – 103 stores identified during call checks and 118 stores that had no phone number available – that required an in-person follow-up. The UES required the lowest percentage of call checks (15%) and in-person checks (3%). Sunset Park required the greatest percentage of call checks (61%) Brownsville required the greatest percentage of in-person checks (47%). (Supplemental Table 1).

In post-hoc analyses, three patterns emerged. First, lower-resourced neighborhoods had significantly fewer call checks when compared to Chinese ethnic enclave neighborhoods among food retailers and restaurants. Second, higher-resourced neighborhoods required significantly fewer call and in-person checks when compared to lower-resourced and ethnic enclave neighborhoods among food retailers and restaurants. Lastly, for restaurants only, Sunset Park required significantly more call and in-person checks compared to Manhattan Chinatown.

Important lessons were learned when developing this protocol. We found that the harmonization of in-person checks data within the Google Maps platform, through the Lists and Notes functions, allowed researchers to make independent field visits while keeping up to date on which food outlets had been checked by other team members and which were remaining. An advantage of the recent scale-up in delivery and takeout from restaurants was that some secondary verifications could be completed using third-party delivery services – e.g., UberEats, Grub Hub, Seamless and DoorDash. Additionally, when completing phone call checks, we found that some calls needed to be done in-language, i.e., the native language of the outlet owner or employee. When assessing neighborhood food environments, research teams should understand the language requirements within communities and ensure that they have staff or community partners willing and able to conduct checks in-language if needed. Lastly, during our second round of data collection, we found that the online checks were completed more efficiently, as research team members were more comfortable with the extraction process, suggesting that training could reduce the time required to conduct data collection.

Recent anecdotal evidence has revealed that food retailers have had to adapt to changing circumstances by scaling up their online presence and introducing curbside pickup or contactless delivery options [37]. The resulting shifts in the food retail environment (e.g., increased online grocery shopping) and food purchasing patterns (e.g., decreased intake of food away from home) have major implications for food security, dietary behavior, and health outcomes [38, 39]. The purpose of the CoClo study was to create longitudinal data to describe changes to the food retail environment during the COVID-19 pandemic. Data from this project can be used in future systems science efforts to simulate the effect of food outlet closures on health behaviors and outcomes. Techniques such as network analysis, systems dynamics and agent-based modeling will allow us to understand the complex relationships between environmental factors (e.g., food outlet status), food security, health behaviors, risks and outcomes in the era of the COVID-19 pandemic and post-pandemic periods.

In addition to health concerns related to changes in food retail environment, the economic ramifications among smaller, local restaurants and food retailers as well as the cascading effects on the food system have been largely unexplored. Smaller restaurants and food retailers may lack the financial infrastructure to endure lengthy closures or scale-up take-out options compared to chain stores and established franchises. These small businesses are a vital part of the US food system; in April of 2020, the New York Times reviewed of how initial closures left farmers with no buyers, resulting in staggering amounts of food waste [40]. The Paycheck Protection Program (PPP) of Coronavirus Aid, Relief and Economic Security (CARES) Act was designed to help struggling small businesses during the COVID-19 pandemic-related government mandated closures. Though businesses with 500 employees or more were ineligible to receive funds, loopholes in the program guidelines led to larger food corporations receiving over $10 million in loans [41]. Without assistance, these small food retailers may permanently shut their doors. The absence of these retailers from the US food system has unknown but potentially devastating effects. Data from CoClo may be used in future advocacy work for government relief programs and other economic assistance.

Given the resource constraints of the pandemic, we were unable to evaluate further business aspects, i.e., online sales and changes in food item prices, or test the validity of this three-step data collection process. Novel methods to assess patronage include mobile phone data that can enable researchers to identify the frequency of visits to locations. This data has been important during the pandemic for contract tracing and monitoring migration patterns. We are unaware of current research efforts to use mobile phone data to evaluate changes in patronage of food businesses, but this presents an opportunity that might be explored in future work. Furthermore, future data collection efforts of business closures using the three-step process should validate the method against in-person only checks. Ground-truthing or on-site verification of food outlets has been previously established as the gold standard for food retail environment surveillance [42‐44]. Another opportunity could be to re-order the phone call and internet checks to test how robust the original method was. A validation study could support the use of this method in other settings and contexts.

Despite our efforts to adapt to the circumstances, we report some shortcomings to our methods. There may be instances where stores were incorrectly categorized as indefinitely closed when they were temporarily closed during COVID-19 mandates. However, temporary closures may lead to lasting closures, as seen already in some cities where restaurants closed during the stay-at-home order were forced to close permanently [45, 46]. Moreover, there may be misclassification whereby businesses were marked as temporarily closed during in-person checks but were open and operating under extremely limited hours or on a takeout basis only. In future work, if these methods were applied to different settings in which operations were not able to be checked, then an ‘uncertain’ category may be applicable. Because the objective of this project was to determine changes in the food retail environment during the COVID-19 pandemic, we felt that accounting for any closure was sufficient, especially as the initial documentation was done when restaurants and stores were beginning to reopen. Additionally, we are hesitant to attribute all of the changes directly to COVID-19 mandates. Some closures may be due to construction. Moreover, Black Lives Matter protests, which started May 2020 and continued through August 2020, may have contributed to some changes. Food retailers that opened since June 2019 were not accounted for, as this was the last date the food retailer database was updated at the time of data extraction and analysis. It was not known when the database would be updated, and given the pressing nature of this project, the 2019 data were used. In some instances, food outlets underwent a name change, which made it more challenging to determine the status, i.e. if the food outlet had closed or remained open under a different name.