Mixing Memory

Implicit Racial Bias in NBA Referees

i-adf5e0d35e4cb101a5764c74812d3c06-refplayer.jpgEven though study after study has shown that implicit race bias is pretty much ubiquitous in American society, I’m still occasionally surprised when a study comes out demonstrating it in an area I hadn’t previously thought about. That was the case when a friend emailed me a New York Times article on a study of racial bias in NBA referees (the working paper on the study is here. Why did the study authors, Joseph Price and Justin Wolfers, choose NBA referees? Here’s their explanation, from the paper:

Our setting provides intriguing insights into own-race bias; relative to social, judicial, or labor market settings, the evaluators in our sample (NBA referees) are a particularly expert group, with substantial experience, continual feedback, and they face robust career incentives to beaccurate. Indeed, NBA Commissioner Stern has claimed that these referees “are the most ranked, rated, reviewed, statistically analyzed and mentored group of employees of any company in any place in the world.”

In other words, all the factors that you think might control for racial bias are in place. Furthermore, NBA referees provide a particularly good study group because their assignment to games is essentially random.

What Price and Wolfers did is pretty simple. They first collected three types of data: player and game statistics using box scores from every regular season NBA game played from the 1991-1992 season to the 2003-2004 season. During that period, there were 29 NBA teams (there are 30 now, but the Charlotte Bobcats were added in 2004), two of which started playing in the 1995-1996 season (the Toronto Raptors and Memphis Grizzlies), with each team playing 82 regular season games. According to Price and Wolfers, this yielded more than 250,000 “player-game observations.” Next, they classified the race of NBA players using several sources, and NBA referees by using press photographs. As an initial analysis, they then looked at the number of fouls white and black players get as a function of whether the majority of officials (there are 3 in every regular season game) were white and black. They found that black players were called for about the same number of fouls per 48 minutes (the length of an NBA game) regardless of the race of the majority of the officials, while white players were called for significantly fewer fouls when the majority of the officials were white than when the majority was black.

Since they’d found a difference in this initial analysis, they decided to do a more complex analysis using regression. What regression analysis does, in essence, is measure the relationship between several variables and one the measure you’re interested in (in this case, the number of fouls called on players). This allows you to “control” for the influence of some variables so that you can focus a few target variables (in this case, the race of players and officials) on that measure. For their analysis, Price and Wolfers included control variables like the position of the players (guard or forward), the players’ height and weight, whether the player was an all star, whether the player was a starter, whether the team was in contention to make the playoffs, along with the stadium where the game was played (including a variable for whether a player’s team was the home or away team). This analysis showed that controlling for several potentially relevant factors, the number of fouls called on black players increased between 0.12 to 0.21 (or 2.5-4.5%) when the the number of white referees increased from 0 out of 3 to 3 out of 3.

Subsequent regression analyses focusing on statistics like points scored found that as the number of “opposite race” referees increased, players’ scoring, blocks, and steals per game decreased, while their number of turnovers increased. How important are these differences? Based on their player-level analyses, Price and Wolfers calculate that “a team’s winning margin would rise by up to half a point if they could simply change the race of a player so that it matched that of the refereeing crew” (p. 14). Over the course of an 82 game season, that difference could mean a change in the win-loss outcome of a game or two, which, in a tight playoff race, could have a big impact on the outcome of a team’s season.

Once again, then, we learn that racism is everywhere. It shouldn’t be surprising, I know, but judging by the fact that this as of yet unpublished study made it into the New York Times, I’m not the only one who was surprised by it. Unfortunately, it’s impossible to know how the implicit biases demonstrated by Price and Wolfers analyses (presumably referees are consciously focusing on race) correlate with referees explicit attitudes toward race, because the referees were not studied directly. However, this study does suggest that explicit biases are not necessary to produce real-world effects — in this case, effects on players’ performance and even the outcome of games and seasons. It also shows that it’s very difficult, without using fairly sophisticated statistical techniques, to observe the effects of implicit bias in real world situations, because the NBA’s referee evaluation procedures presumably failed to discover it.

UPDATE: Statisticians discuss the study.


  1. #1 Sara
    May 2, 2007

    Sounds like the refs’ races could also be affecting the players’ performance.


    Thus, even if you controlled for the refs’ implicit and explicit racial attitudes, you might be likely to still get the same result.

    This was an interesting article, and even though some people think it’s beating a dead horse, I think it’s important for reserachers to continue to study this topic. Results like these show how it’s important for us to continue to consider affirmative action programs, because we don’t even realize how much unconscious racial bias is out there.

  2. #2 Chris
    May 2, 2007

    Sara, I agree. It’s important to keep doing research like this if for no other reason than the fact that some people have to be beaten over the head with example after example before they’ll believe it.

  3. #3 ed k? />
    May 3, 2007

    Basketball stats geeks also discuss the study. My conclusions: the foul effect size is tiny, and the other, larger effects in the paper are not persuasive.

  4. #4 Clark Goble
    May 3, 2007

    I have no real opinion. My inclination, given other cognitive studies, is that we all have unconscious racist judgments at times even if we hate racism and do everything we can to avoid it.

    But I tend to agree with Ed K. Isn’t the effect like 3%? That seems lost in the noise. Someone on one of the sports stations here mentioned an other issue. Some refs do appear to have issues with certain players. If one of those players was black (as, given the makeup of the NBA is highly probable) wouldn’t that intrinsically bias these results?

    I think looking for bias against individual players and accounting for individual players might be something to look for.

    Once again I’m not denying the study in the least. I’ve not looked at it closely enough to be able to draw a firm conclusion. But that really seems like a small figure.