Bryan Caplan

The Truth Hurts: What Harford Didn't Say About Statistical Discrimination

The Cause of Everything... Taking the Initiative on Healt...

I'm a fan of Tim Harford's Logic of Life, and I'm a big promoter of the explanatory power of statistical discrimination (see here, here, and here for starters; also check out my lecture notes). So naturally I'm thrilled that Tim's new book devotes a full chapter to statistical discrimination - or as he prefers to call it, "rational racism." However, in the process of trying to make the idea palatable to his audience, I'm afraid that Tim failed to communicate some of the most important implications of the theory of statistical discrimination, and emphasized striking experiments at the expense of some basic facts about the labor market. Perhaps more importantly, Tim doesn't mention some simple solutions to genuine problems he laments.


1. A key building block of statistical discrimination is the assumption of stereotype accuracy. For statistical discrimination to be stable, the stereotypes that market participants rely upon must be accurate statistical generalizations. But the average person could easily read Tim's chapter without realizing that he is building upon this radically politically incorrect claim. Tim really should have spent some time poking fun at the idea that people make up stereotypes out of thin air, and pointing out some of the obvious but offensive evidence that stereotypes have been falsely stereotyped.

To be fair, several sentences in Tim's chapter imply what I'm saying. For example, he tell us that "Statistical discrimination happens when employers use the average performance of the applicant's racial group as a piece of information to help them decide whether to make the hire." But the idea that racial groups have different average levels of performance is so objectionable, and Tim's persona so likeable, that most people probably didn't get the point. And that's a sad missed opportunity.

2. An important implication of stereotype accuracy that Tim neglected is that the fundamental social conflict is not between groups, but within groups.* Take auto insurance. Young males are riskier drivers, so insurance companies charge them more. But if each young male were judged as an individual, the result would not be lower rates for young males. The result would be that young males who are more reckless than the average young male would pay even more, while young males who are less reckless than the average young male would pay less. The cause of the suffering of the cautious young male driver is not the "evil" insurance company, but the reckless young male drivers who make all young male drivers look bad.

This is a counter-intuitive claim that would probably offend a lot of readers. It is also one of the key implications of statistical discrimination. I wish Tim had stuck his neck out more.

3. In response, Tim could object that I've overlooked a subtler way for statistical discrimination to harm a group. After all, he heavily emphasizes a few experiments showing that statistical discrimination could be a "self-fulfilling prophesy." For example, he describes a resume experiment where otherwise identical fake resumes with "black names" were less likely to get a response. "High-quality applicants were more likely to be invited for an interview, but only if they were white. Employers didn't seem to notice whether black applicants had extra skills or experience." If that is how employers treat black applicants, what's the point of trying? As Tim asks, "Why bother to get a degree or work experience if you are young, gifted, and black?"

But is it really true that the market fails to reward blacks for getting more education? Is it even true that the market rewards them less? I tested these claims using one of the world's best labor data sets, the NLSY. The results directly contradict Tim's self-fulfilling prophesy story. Blacks actually get a substantially larger return to education than non-blacks! The same goes for experience, though the result is not statistically significant. The real lesson of the data is that if you are young, gifted, and black, you should get a ton of education, because it has an exceptionally large pay-off.

Why would this be so? I'm not sure, but one simple story is that counter-stereotypical behavior stands out. When my sons were young, my wife was working a lot, so I often took my kids places on my own. Funny thing: Time and again, strangers came up and said, "Wow, you're such a great dad!" But there were moms of young kids doing the same thing in plain sight, and the strangers rarely praised them. Why not? Because a dad taking care of two babies is counter-stereotypical, which grabs people's attention.

Purely anecdotal, yes. But it is consistent with the small academic literature on counter-stereotypical behavior. If you clearly violate expectations, people not only notice; they often over-react.

The upshot is that stereotypes may actually be self-reversing rather than self-fulfilling. The marginal payoff of distinguishing yourself from the pack is high if people think poorly of the typical member of the pack.

4. Tim seems pretty fatalistic about statistical discrimination. He shouldn't be. Are you disturbed by the fact that the market fails to judge people as individuals? Then allow employers to write contracts to solve the problem.

Example: Some young women are 100% focused on their careers, and don't want kids. Most young women, however, do want kids, and intend to strike a balance between work and family. That balance often involves receiving expensive job training from a firm, then quiting before the firm can recoup its expenses.

Under current law, an employer isn't even allowed to ask about a female applicant's child-bearing plans. If you wanted to blow up the glass ceiling, though, you should not only allow employers to ask; you should allow them to offer deals like "We'll hire you, but your health insurance doesn't cover pregnancy." The career woman would be happy to sign, reassuring the employer.

How will that help women? It won't! On average, it's a wash: It will help career-minded women, and hurt the rest. And if you want to judge female workers on the basis of individual productivity, that is exactly what should happen.

Once again, I expect that my proposed legal reform would appall many of Tim's readers. From Tim's point of view, it was probably a rational decision not to mention it. But the logic is compelling nonetheless.

Don't get me wrong - on balance I'm glad that Tim raised the profile of the theory of statistical discrimination. And I'm glad to live in a world where he is one of the world's best-selling writers of popular economics. But on balance, I'm afraid that Tim sugar-coated some hard lessons about discrimination and the market that most people are not yet ready to hear.

* To be fair, Tim did discuss black-on-black social sanctions for "acting white," but that has no intrinsic connection to statistical discrimination.

Comments and Sharing

TRACKBACKS (12 to date)
TrackBack URL:
The author at The Undercover Economist in a related article titled Bryan Caplan on me on race writes:
    Bryan Caplan has an excellent post criticising the discussion of racial discrimination in The Logic of Life. I'm going to read it a few more times but I think I'm sticking to my guns for the most part. But it [Tracked on February 14, 2008 2:15 AM]
The author at Vivre La DiffÃ©rence in a related article titled Statistical Discrimination writes:
    As we go through the numbers about the gender pay gap, something I’ve promised to do here, we need to make a distinction between two different types of discrimination. [Read More]... [Tracked on February 14, 2008 1:36 PM]
The author at Cobb in a related article titled Statistical Morality: The Logic of Life writes:
    Tim Harford has written an interesting book called The Logic of Life. I may check it out. It is, after all, intrinsic to my interests in BI and Knowledge Management, which is for those of you who may not be [Tracked on February 14, 2008 7:18 PM]
The author at The Undercover Economist in a related article titled More on returns to education for victims of discrimination writes:
    Yesterday I reported on Bryan Caplan's thoughtful criticisms of chapter six of The Logic of Life. One of the things that worried me most in that chapter is the possibility that employers don't value qualifications or experience of minority groups, [Tracked on February 15, 2008 3:59 AM]
COMMENTS (12 to date)
Buzzcut writes:

Best. Post. Ever.

eric writes:

In the same way that people are generally for health care reform only if it involves more redistribution from rich to poor, I suspect the political advocates against stereotypes are only interested if their group is given an aggregate boost. You can argue that making each male pay his fair share for insurance makes for a more efficient allocation, and in competitive markets, this cost savings will be shared with consumers, but I doubt that is sufficient.

Most egalitarians operate under the assumption that everyone is the same--statistically, by race, ethnicity, gender, sexual orientation, etc.--and so differences between groups must be due to ephemeral and arbitrary starting values--there can't be differences between men and women because they are the same in everything (see Gene Expression on the book Apes or Angels). Until the axiom of equality (all humans are the same, across all dimensions of human groupings or talents) can be safely rejected without being called a reactionary, stupid, mean, etc. your solution can't happen.

a student of economics writes:


Dan Weber writes:

The "black names don't get interviews" story was debunked in Freakonomics, I thought. The big point was that the names used to signify race were more accurately signals of class.

Still, even if I were to accept the theory that, say, "whites are better than blacks at software development on average," it doesn't do me any good as a filter for employment. Because all applicants have already passed through other screens, such as college, that filtered out the poor performers from both groups.

(You can legitimately say that a degree isn't a good test, but you could also use other filters, like previous experience on their resume.)

Steve Sailer writes:

Excellent post.

Steve Sailer writes:

Steven Levitt and Steve Dubner wrote in Slate in an excerpt from Freakonomics called "Would a Roshanda by Any Other Name Smell as Sweet?" about those super-black names that black mothers started giving their babies during the Black Pride era:

"The typical baby girl born in a black neighborhood in 1970 was given a name that was twice as common among blacks than among whites. By 1980 she received a name that was twenty times more common among blacks."

Levitt and Dubner show strikingly little sympathy toward blacks who have a harder time getting called in for a job interview because, as shown by numerous "audit studies", employers are dubious of DeShawns and Darnells.

Levitt and Dubner scoff:

"Was he rejected because the employer is a racist and is convinced that DeShawn Williams is black? Or did he reject him because ‘DeShawn’ sounds like someone from a low-income, low-education family?"

Sure, as the authors imply, a boy named DeShawn may indeed be, on average, more likely to goldbrick or to rip off his employer than a boy named, say, "Dov" (the male name with the most educated parents according to the book).

Following their Naturist inclinations, Levitt and Dubner conclude:

"And that's why, on average, a boy named Jake [the whitest common male name] will tend to earn more money and get more education than a boy named DeShawn. A DeShawn is more likely to have been handicapped by a low-income, low-education, single-parent background. His name is an indicator—not a cause—of his outcome. Just as a child with no books in his home isn't likely to test well in school, a boy named DeShawn isn't likely to do as well in life."

There aren't too many people who make _me_ sound like a diversity-sensitive multi-cultist, but sometimes Levitt is one of them! The authors could at least have a little compassion for the poor kid. DeShawn didn't ask to be given his name.

And I must point out that a recent study by economist David Figlio calls into question Levitt's assumption that DeShawns aren't hurt by prejudice. Figlio cleverly looked at siblings, and found that the ones with the blacker names tended to get rated more poorly by their schoolteachers even when their test scores were the same.

James A. Donald writes:

Similarly, I find that I am tremendously impressed by a black guy in a well tailored business suit, whereas I am apt to assume a white guy in well tailored business suit is a pointy haired idiot.

Cobb writes:

Black on black discrimination for 'acting-white' does have an intrinsic link to statistical discrimination, you may just be unaware of the statistics. Roland Fryer did the study several years ago.

In any case I tend to look at all matters of this type of research to exhibit a sort of confirmation bias and the bits about self-fulfillment or self-reversal to be post-hoc rationalizations. Stereotypes are, after all, a shorthand way of thinking and in the interests of generating policy, so are statistics. There are very few aspects of human behavior that can be accounted for on the basis of key performance indicators, nor are statistics likely to be taken on the rationale. And so we tend to focus on the few statistics that can be collected reliably and then, post-hoc, try to make sense of them.

Take the example of the federal consent decree set for the LAPD in the wake of the CRASH scandal and general public concern about the beating of Rodney King. There was a stereotypical assumption that white police officers were stopping black motorists because of racist reasons. And so in order to monitor this, a set of new reporting regime & requirements involving racial checkboxes was established. At some point, we would have a fairly representative sample of which officers were citing which people by race, but none of that statistical information gives any outsider a better understanding of the reasons why police do what they do. In otherwords we created a statistical abstract, an incomplete model of officer behavior, which is much more complex than can ever be statistaclly represented - especially on moral questions like discrimination.

So we can generate statistics on discriminations, but can we generate statistics on the rationales behind those discriminations?

It seems to me, that for the purposes of policy and remedy, we are always going to be at a loss with our abstractions, and that those who are close enough to the transactions in question cannot be both efficient at their jobs and effective in generating documentation bringing the layman close enough. We will always use some stereotypical and reverse-stereotypical thinking. It's how human mind copes with realtime performance without crashing.

Sindney Clare writes:

Bryan, this is a good post, but you need to cite your sources.


Citation? If this isn't published, Hartford can't be faulted.



Steve Sailer writes:

There are about a half dozen federal longitudinal tracking data bases going back to 1967, for which interviews are conducted every year or two to see what people are up to now. The most famous is NLSY79, for which the U.S. military paid to have the entire nationally representative sample of about 12,000 15-23 year olds given the military's AFQT IQ test, its entrance exam. The military gave the test scores to Charles Murray and Richard Hernnstein and they matched it up with what the panel was doing in 1990. We now have another 15 or so years of data, and, amazingly enough, we even have data on about 6,000 children of the original panelists, including the kids' IQ scores.

Just Google NLSY and you can download it for free from the Bureaa of Labor. It's all out there for any social scientist to use. It's been used in countless studies, but they don't get much publicity because, on the whole, they just keep finding the same kind of thing Herrnstein and Murray found in The Bell Curve. And nobody wants to hear about that!

Scott Wood writes:

Wouldn't allowing employers to ask about child rearing plans (assuming they can get accurate information) help women overall by making them less risky? Some women may be helped and other hurt, but by the logic that you employ I would think that the overall effect would increase average female income.

Is the return on education for blacks significant for every level of education? For instance, it could be the case that there is not a significant return to a four year degree but that there is for post-doc work.

Public relations drives companies to recruit minorities for the top positions in a way that it does not for middle management.

Comments for this entry have been closed
Return to top