March 7, 2012
Improve Restaurant Report Cards—A Commentary by Daniel E. Ho ’05
The following commentary was posted on newyorktimes.com on March 7, 2012.
Improve Restaurant Report Cards
By Daniel E. Ho ’05
In July 2010, New York City began requiring restaurants to publicly post letter grades summarizing their sanitation reports. The highly publicized system was well intentioned, but it’s broken.
We know from decades of social science research that mandated information disclosure often fails; think of the byzantine disclosures contained in mortgage agreements, drug labels or software licenses. But the emerging consensus is that simplified and “targeted” disclosures — like the restaurant sanitation grades — are a promising avenue to inform consumers and help them make better decisions. Evidence from Los Angeles County, for example, suggests that grading restaurants can reduce hospitalizations for food-borne illness.
The New York City Department of Health and Mental Hygiene deserves substantial credit for leading the charge for targeted disclosure and, more important, making detailed inspection data publicly available.
But the current system is ineffective. Along with researchers at New York University, Stanford and Yale law schools, I have studied data from more than 500,000 inspections of more than 100,000 restaurants from the last few years in nine jurisdictions: Austin, Tex.; Catawba County, N.C.; Chicago; El Paso; Florida; Louisville, Ky.; New York City; San Diego; and Seattle. Our research examined the process for tallying violations and the consistency of inspections across repeat, unannounced visits to the same restaurant. In a critical dimension, New York performed the worst of the nine.
At their core, the inspections work similarly across the jurisdictions. From once to a few times a year, a health inspector shows up unannounced to tally health code violations, like failure to wash hands or to maintain food temperatures. If violations amount to a public health hazard, the restaurant may be shut down until they are resolved.
Our examination found key deficiencies in New York’s inspection system.
First, the score a restaurant gets in New York says little about how it will perform in the future. Grades are based on a point system: in New York, 0 to 13 points yields an A, 14 to 27 points a B, and 28 or more points a C. In other jurisdictions, numerical scores substantively predict future scores. In San Diego, for example, prior scores account for roughly 25 percent of the variation in future scores. But New York is an outlier: Prior scores predict less than 2 percent of the variation in future scores. New York City’s posted restaurant grades therefore fail the most basic criterion: they communicate little about future cleanliness.
Why such inconsistency? Although the jurisdictions share broad similarities, the details of New York’s inspection process are far more complex. There are more inspectors (some 180, not all of whom necessarily specialize full time in restaurant inspections), more violations to score and far wider point ranges for each violation.
While San Diego, for example, has a single violation for vermin, New York records separate violations for evidence of rats or live rats; evidence of mice or live mice; live roaches; and flies — each scored at 5, 6, 7, 8 or 28 points, depending on the evidence. Thirty “fresh mice droppings in one area” result in 6 points, but 31 droppings result in 7 points.
Not surprisingly, inspectors appear to implement the New York system in sharply different ways. A 2009 audit by the city comptroller’s office, which also found that the department did not adequately monitor and supervise inspectors, showed that of 67 inspectors who conducted at least 100 inspections in the 2008 fiscal year, the average ratings ranged from as low as 15 to as high as 50.
But a second, and perhaps more disconcerting, flaw of the inspection system goes beyond whether the grades themselves offer useful information. Grading appears to shift inspection resources away from the worst offenders. Previously, the Health Department would follow up with restaurants posing the greatest risks — those now in the C range. Although the department still conducts some of these inspections, the grading system introduced a new type of inspection for the purpose of resolving a restaurant’s grade. Roughly a week to a month after an initial inspection that does not result in an A, restaurants are reinspected for re-grading. In the first year and a half of the grading system, some 14,000 inspections have been devoted to grade resolution for B-range restaurants. And that diverts attention from the most delinquent.
In short, our study shows that grades are not just imprecise, but also inefficient. To reduce imprecision, the city should apply the key insight of grading — simplification — not only for information consumers, but also for information producers — i.e., the inspectors. To increase efficiency, the city should abandon inspections for the purpose of resolving grades and instead redeploy those resources to focus on the worst offenders. It’s time for grade reform.
Daniel E. Ho, a professor of law at Stanford, is a visiting professor of law at Yale this spring.