G20 summit: How the bandwagon wrecked the wisdom of market crowds

Politicians, take note: the answer may not be to use markets less, but to make sure the players are many and the opinions diverse

On the eve of the G20 summit there is one question that everyone is thinking about the financial crisis: how could we have been so stupid? To be sure, there is one explanation of the crisis that insists that people weren't stupid, that instead they were responding rationally to a flood of cheap money from the US Federal Reserve and China. But while there's something to this, it's ultimately not enough of an explanation. Even if money was cheap, all (or nearly all) of it didn't have to flow into housing. It could have gone into other, more productive investment. And while you expect borrowing to expand when money is cheap, you also expect there to be some connection between the risks people take and the rewards they reap. Yet looking back on the bubble and the crash, it's hard to argue that risk and reward were in sync.

Bubbles always have at their heart some big idea, and in this case, obviously, that idea was twofold: housing prices could keep going up indefinitely, and our new instruments for managing risk meant that investing in housing was effectively risk-free. But while this veneration of real estate was central to the bubble, its effects were amplified by specific institutional and cultural factors that led the financial industry to make what amounted, ultimately, to the worst bet in history. In the US, at least, most big financial institutions in effect wagered the value of their entire companies on the housing market. It's when those bets started going bad that the current crisis began.

So what drove these bets? First, at most companies the link between pay and performance, risk and reward, was severed - or, to be more accurate, the link between pay and long-term performance was nonexistent. Crowds are wise when the individuals in them have an incentive to get the right answer. On Wall Street, though, the enormous amounts of money that one could make in the short term meant that the long-term consequences of failure mattered much less than they once did. Similarly, the fact that mortgages were packaged and sold as securities to outside investors changed incentives dramatically. Mortgage brokers were rewarded not on the ultimate performance of the mortgages, but on how many deals they made. As a result, it's not surprising they were not interested in meaningfully evaluating potential borrowers' riskiness.

The problem was compounded by what you might call the outsourcing of responsibility. Again, markets work best when people in them are thinking for themselves. But during the bubble, an enormous amount of authority was given to the ratings agencies, whose evaluations of housing assets had a tremendous impact on the securities that investors chose to buy and the interest rates they were willing to accept. These agencies were either lacking or uninterested in the kind of granular information that would have allowed them to offer meaningful judgments about the assets, and relied too often on broad analyses and historical trends to make their judgments.

Still, that doesn't quite explain why seemingly savvy investors, who knew that the rating agencies had conflicts of interest, simply went along with their judgments. To explain that, I think you have to accept something that isn't necessarily easy to believe, which is that Wall Street to some extent drank its own Kool-Aid. In a way, the story of the crisis would be simpler if it were a story of corrupt financiers foisting off worthless assets on hapless investors. But the truth is that it's those corrupt financiers' firms that ended up being destroyed by the bubble, precisely because they did not just foist off these assets, but instead invested heavily in them. Merrill Lynch, Citigroup, Lehman Brothers: the reason these companies went to the brink (and beyond) is because their balance sheets were, and in some cases are, piled high with toxic assets. If they were fooling anyone, they were fooling themselves.

That shouldn't be too surprising, because human beings are remarkable imitators. We often take our cues from those around us, and in times of uncertainty - whether good or bad - we tend to herd together. The key to a group making wise decisions is that its members be diverse in their opinions and relatively independent of each other. But this impulse toward imitation - or what sociologists call "social learning" - can make that difficult. It isn't simply that we are mimics. It's that the people around us often do have good information. They often have come up with good answers. So there is a powerful bandwagon tendency in humans generally, and in markets specifically; and during this bubble it exerted a powerful influence.

So if the folly of crowds got us here, is there any hope for using the wisdom of crowds to get us out? I'm hesitant to offer any forecast that would be too optimistic, given the enormous amount of uncertainty that's still out there. But the answer may not be necessarily to use markets less, but rather to do a better job of ensuring that they are working the way they should: that there are many players in them rather than few, that diverse opinions can be easily registered, and that incentives and goals are well aligned. The collective intelligence of markets can be a startling and useful thing. It's just too bad that it's been years since we have seen much of it.

James Surowiecki is a staff writer at the New Yorker and author of The Wisdom of Crowds