American politics
Democracy in America

Ricci's wrinkles

The impact of Ricci v DiStefano

"DISPARATE IMPACT" is the phrase on every pundit's lips since the announcement of the Supreme Court's verdict in Ricci v DiStefano, which found the city of New Haven erred in discarding the results of an exam that was to help determine firefighters' eligibility for promotion after it turned out that none of the black candidates had made the cut. One thing is clear from reviewing both the decision and the reactions to it: Quite apart from the difficult legal and moral questions involved, the empirical problem of identifying "disparate impact" is a thorny one.

Suppose ability were evenly and randomly distributed across ethnic and racial groups, and you had a perfect test—some kind of brain scanner that could unerringly predict future performance in a leadership role. If you ran the test a million times on random pools proportioned like New Haven's—118 candidates, 27 of them black—then statistically (and assuming I've remembered my high school maths right) you'd expect that in about 1.5% of the runs, you'd find your 15 top scorers didn't include any black candidates. Of course, ability is not evenly distributed. The reason we're so concerned about these results is precisely that we're coming off several centuries of systematic, state-enforced racism that have left minority groups with fewer opportunities, more poverty, and less social capital. So the question, given that a systematic gap is predictable, is which differences in ability are job-related and would be measured even by our ideal brain-scanning test, and which are more like plaintiff Frank Ricci's dyslexia—apt to distort test performance more than they would really impair job performance. Given that every real-world test will have some fudge factor relative to our magic scanner, the goal is, first, that it be as small as possible, because we want to accurately identify the best fire captains, and second, that it not be a systematic fudge that magnifies the disadvantage of historically disfavoured groups.

If we had that magic scanner, then we could use it to establish a benchmark that would establish the fudge factor for each test—assuming, for some reason, we didn't just want to use the magic scanner! The problem, of course, is that we don't: What we have are a variety of different tests (necessarily, since a firefighter in Manhattan will need to know different things than one in Tempe) as applied to many different pools of firefighters across the country. With a lot of time and a huge research budget, you might try to correlate test scores with the future performance of promoted candidates, though this would clearly be complicated by both regional differences and the impossibility of determining how the lower scorers who weren't promoted in each city would have fared. In the interim, you can say that this test seems to have generated bigger average score gaps between racial groups than that one, but without a benchmark, there's no way to know which is correct: A test with a low gap might be ignoring irrelevant differences or failing to measure relevant ones. For any particular test outcome, you have a double problem of ignorance: You don't know what sort of disparity an ideal test would show on average, and you don't know whether any particular deviation from the ideal average is the result of a bad test or an outlier candidate pool.

Given these problems, it's not hard to see the rationale behind the high court's insistence that there be a "strong basis in evidence" for thinking there was something amiss with a particular test, beyond the unlovely statistical shape of a particular result. Writing in Slate, however, Richard Thompson Ford worries that the finding will "burn down civil rights law"—which seems awfully tendentious if one is prepared to allow that the white and Hispanic plaintiffs in Ricci are also raising genuine "civil rights" claims, whether or not one thinks they should prevail. Still, Mr Ford raises an interesting quandary:


The problem becomes a bit more tractable if one takes care to separate the varied rationales for race-conscious policy. "Disparate impact" analysis is an extension of the more intuitive bar on formal, intentional discrimination. One may not deliberately disfavour racial groups, nor may one use non-racial criteria that tend to disproportionately exclude minorities unless those criteria are manifestly job related. This is distinct from affirmative action, which may be remedial in nature—here the court typically wants evidence that the specific institution employing it is trying to compensate for its own past discrimination—or in limited circumstances aim at achieving diversity. In the latter case, the point is to advantage historically disfavored groups, whereas in the former, the idea is that sometimes you have to be race conscious in order to achieve genuine neutrality, because formally neutral standards may be biased in subtle ways. It shouldn't be terribly difficult to drive a wedge between these two types of analysis—not least because the court pointedly refrained from grounding its Ricci holding in the 14th amendment.

More generally, it probably makes sense to treat employment and public education by somewhat different standards. No test is perfect, but there are at least roughly objective criteria for what makes the "best" firefighter, and the state has a powerful interest in having a fire department that functions as well as possible, which means trying to promote the objectively best firefighters. Education is different. We speak casually about "better" and "worse" students, but at the margin, especially at the most selective institutions, this is to some extent a matter of aesthetics. Students are compared along a whole array of incommensurable dimensions, and there's no obviously "correct" way of weighting them. That makes it much harder to claim that a student is entitled to be evaluated in some particular way, with deviations from that standard prima facie unfair. Moreover, it may be enormously important to the community that the slightly better fire captain be promoted, whereas marginal college admissions choices don't make much difference to anyone but the student. If you swap a dozen kids from the top decile at Georgetown with a dozen of Harvard's marginal admits, one group will be rather happier with their resume and networking opportunities four years hence, and another somewhat less happy, but they'll both get fine educations, and the world at large won't be much altered for it.

Reuse this contentThe Trust Project
The Economist Today

Hand-picked stories, in your inbox

A daily email with the best of our journalism