Archive for the ‘numerical analysis’ Category

Michael Gerson, Sabermetrics, and Election Results

November 10, 2012

In Michael Gerson’s November 5 column in the Washington Post he takes on people who use math models to predict election results.  He says that problem with this method that it is not only ” pseudo-scientific but that it is trivial.”  He uses Nate Silver’s method as an example calling it ” little more than weighting and aggregating state polls and combining them with various historical assumptions to project a future outcome.”  He argues that this sort of analysis reduces political science to the prediction of election results without considering the things that motivate people to form opinions and make decisions.  He calls political science a “division of the humanities.”

Gerson’s argument reminds me of the baseball establishment’s reaction to the application of statistical techniques to baseball.  Baseball stats have been around since the dawn of the professional game.  In the 1940s and early 1950s general manager Branch Rickey used some predictive techniques that weren’t widely publicized.  In the early 1960s a storm broke when retired metallurgist Earnshaw Cook published a book called “Percentage Baseball,” which was the first attempt to subject the game to mathematical analysis.  Cook, like many pioneers, made some large mistakes.  He also was right about many things.  Baseball managers and executives railed against Cook’s ideas on baseball strategy.  One area where he was undeniably correct was that the bunt is generally a bad idea except under certain conditions when a team needs to score a single run late in a game.  Cook’s big mistakes were a method of  setting  a batting order that doesn’t maximize scoring and a system for using pitchers that, while similar to the way teams use pitchers today, is not compatible with the human body or the amount of pitching talent available on most teams.  The common theme in the reaction of the baseball establishment was that baseball transcended numbers.  Gerson said something very similar, “An election is not a mathematical equation; it is a nation making a decision.”

In the 1970s Bill James, Pete Palmer and others developed better methods of analysis and prediction for baseball.  James coined the term “sabermetrics” from the acronym SABR for the Society for American Baseball Research, a group that started out with a historical emphasis and broadened its approach to include the statistical analysis that was becoming more popular.  The book Moneyball, by Michael Lewis, and the recent movie of the same name chronicled the application of sabermetric techniques to running a baseball team.  The baseball establishment reacted to Billy Beane’s use of  math the same way it did to Cook’s book.  Beane’s Oakland Athletics became very successful because Beane used analytical conclusions to try to find the best bargains in player talent.  Other teams weren’t doing that.  Today most teams use some sabermetric analysis in assessing talent, considering the value of trades, etc.

Gerson laments the popularity of numerical analysis in political science today.  He talks about scholarly journals being filled with “a profusion of numbers and formulas more suited to the study of physics.”  He believes that politics is “mainly the realm of ethics — the study of justice, human nature, moral philosophy and the common good.”  One problem with the ethical focus is that people don’t necessarily consider ethics or justice in forming their political opinions.  Some vote their wallet.  Others vote the candidates’ appearance.  The Nixon Kennedy debate is a reminder that the electorate, as a group, isn’t quite as high minded as the average scholar.

It’s certainly possible that some of the journal articles attempt to quantify things that are difficult to quantify, especially when the subject at hand doesn’t have a precise, readily agreed upon definition.  That’s a common flaw in psychological research.  Where Gerson is wrong is in downplaying the significance of numerical analysis.  The outcome of elections is something people find interesting.  If the best results so far have come from black box techniques that ignore the actual decision making process, the predictive value of the techniques is both interesting and useful.  The canonical black box approach is the use of input to predict output without being concerned with what happens in between.  Stuff goes into the black box.  We don’t know what the black box does with it.  We do know that something comes out.  If we can predict the output of the black box on the basis of the input we know something.   Sometimes we can use that knowledge to open the lid of the box.

The Society for American Baseball Research has never abandoned its historical and biographical work.  There’s still a great deal of interest in the human side of baseball.  Members write papers and books on baseball history and biography all the time.  The Society also has many members whose primary interest is statistical.  The  Society recognizes both approaches as interesting.  Gerson wants political science to ignore those whose interest is in quantifying and prediction and stick with the philosophical approach that he favors.  It’s reminiscent of the old mathematicians’ toast, “Here’s to pure mathematics.  May she never be of use to anyone.”  Of course, eventually someone almost always finds an application for the most esoteric area of mathematical abstraction.  If political science is defined the way Gerson wants to define it, there’s no place for numerical analysis or the prediction of political events, whether they’re election results or long term trends.  Admitting numerical analysis broadens the scope of the field.  It may not make pundits obsolete but it just might be able to illuminate punditry in ways that are different from the age old scholastic speculation that’s characterized the field.  Numerical analysis can never tell us everything we want or need to know.  It can tell us some things that other methods can’t.

Although Gerson probably doesn’t think so, this election was an occasion for celebration for pollsters and math modelers.  Almost all of the polls predicted the outcome correctly.  The math modelers did well too.   It’s true, the people who were concerned with predicting the outcome didn’t spend much time on the why.  That doesn’t preclude others from examining those questions.