As I write this, the dow is down 430 points, below 10,000 for the first time in four years.  Tyler Cowen has published his list of what went wrong, with which I largely concur.  Another way to think about it is as a series of cognitive errors that afflicted everyone:  investors, home buyers, lenders, regulators.  They're standard cognitive errors that so far, no one has a very good way of eradicating:

  • Bandwagon effect:  People tend to believe that something is a better idea if a lot of other people are doing it.  In essence, they are trying to free ride on other people's analysis, on the assumption that someone must have thought this thing out.  This has its uses--we don't all need to learn every hard lesson for itself.  But in markets it produces herding behavior, which makes outcomes more extreme on both the upside and the downside.  In other words, we get booms and busts.  In the professional world, this is exacerbated by the fact that you're less likely to get fired if you fail at the same time everyone else does.
  • Availability heuristic:  People tend to overweight data that comes easily to mind, which is to say vivid (extreme) and recent examples.
  • Beneffectance:  People tend to view success as a result of their own actions, while they view failures as having been due to factors largely outside of their control.
  • Confirmation bias:  The tendency to look for data that confirms your theory, rather than data that falsifies it.  Yes, we all know how this works in politics, but it's a much broader problem.  People will repeatedly devise tests that give positive proofs of their theories, but much less often devise tests to falsify them.
  • Hyperbolic discounting:  People value small short-term payoffs more than much higher long-term payoffs.
  • Overconfidence bias:  Relatedly, people are too confident in the accuracy of their predictions.  When asked to estimate a range of possibilities where the true outcome is 95% likely to fall within that range, people's guesses are wrong 40% of the time.

As I see it, these affected everyone who was involved in the markets.

Homebuyers looked at 50 years of basically steadily rising home prices, with few and small declines, and concluded that rising home prices were some kind of natural law. In fact, the run-up in home prices was the idiosyncratic result of a lot of factors:  the move to long-term self-amortizing mortgages; the home mortgage interest tax deduction, which became steadily more valuable as income tax rates rose; the steady decline in nominal interest rates, which lowered monthly house payments for any given home price; severe regulatory contraction of supply in a few areas.

As capital flooded into the US debt markets in the wake of the Asian financial crisis--and I think that Asian savers and central banks bear much more of the responsibility for the credit bubble than can be plausibly pinned on either Alan Greenspan or Ben Bernanke--real interest rates also fell, which made housing an even better deal.  The recency effect kicked in:  as the bubble grew, it began to seem more, not less, likely that home prices would continue to rise.  The bandwagon effect also reared its ugly head.  Everyone else was buying a house on potentially catastrophic terms, so it must be safe! 

Any self-introspection on the safety of the housing market was also plagued by bias.  Examples of falling house prices were not available to memory; spectacular coups reaped by coworkers on a two bedroom fixer-upper were.  And in testing their theory of future house prices, people looked for reasons that housing prices might rise--the many amenities of homeownership wherever they happened to live--rather than thinking of reasons that it might fall.  Besides, the memory of the stock market crash was extremely recent and vivid.  People began to see a home less as a place to live than an investment, a safe alternative to the risky securities market.

Once they had decided that it was likely to rise, they were overconfident in this assessment, which made them rather careless about the terms under which they signed mortgages.

Lenders went through much the same pattern.  The revolution in credit scoring that took place in the 1990s actually did make lenders better at predicting default risk.  As we moved into the current decade, however, the steady rise of home prices started to skew the numbers.  Borrowers who historically might have defaulted simply sold their house into a rising market, recouping at least the value of their mortgage.  Or they refinanced, getting much better terms because the loan now represented a smaller proportion of the overall value of the house, meaning that lenders were more likely to get all their money back.

As default rates fell, lenders went through the same process borrowers had.  They too, overweighted recent events.  They, too, looked at other people lending and concluded that it was probably safe.  As I think I've mentioned before, several years ago, I had a conversation with an investment banker who did a lot of debt deals.  As a general thought, he stated that we'd gotten much better at managing credit risk in the last ten years.

"Have we actually gotten better?"  I asked.  "Or do we just think we've gotten better."

"Oh no, we've really gotten better," he assured me.  Oops.

Lenders built their risk models around pre-payment risk--the risk that buyers would refinance their mortgages, making your investment suddenly much less profitable.  That had historically been the main risk of mortgage lending.  As defaults stayed low, year after year, they revised down their expectations of default, viewing the current situation as a "new normal" rather than an unusually rosy time that might eventually regress to the mean.  They tended to be overconfident about the probability that they were right and the boom would continue, making little provision for an eventual rise in defaults, much less what Nassim Taleb calls a "black swan" event, the kind of broad crisis that we see today.  And they attributed the profits thus made to their own brilliance, rather than luck.

Lenders did have another set of pressures working on them.  For any individual lender, it was better to all fail together than to underperform separately, and not merely because a massive failure might bring on a bailout.  Any individual loan officer or risk manager would be much less likely to be blamed for incompetence if everyone else in the industry was having the same problem.  Moreover, the drive for the year-end bonus led people to hyperbolic discounting--to get it while the getting was good, and worry about the future later.  And, of course, if they did underperform, they would soon have no money to lend, because the investors would take it elsewhere.

So, too, the lenders confirmation bias and availability heuristics led them to construct stories about why the new situation was normal, and would not regress to the mean.  Credit scoring had gotten better, house prices had never suffered a broad and sustained decline since the Great Depression, and of course, our old favorite "Real estate is the only thing they're not making any more of". 

Investors--the people who bought the mortgage backed securities--participated in much the same madness.  Just as lenders had seen declining default rates as evidence that they could safely make riskier loans, investors looked at the recent performance of mortgage-backed securities as something of a natural law.  Hedge funds and investment houses ran more of their tests against historical data than against a broad range of individually unlikely scenarios.  Most importantly, they drastically underweighted systemic risk.  That is, they looked hard at the likelihood that an individual security would underperform.  But they didn't look at what might happen if house prices suddenly dropped ten percent, or credit markets violently contracted.  They chased high returns while underweighting risk, which put pressure on lenders to do the same--if they didn't, the investors wouldn't give them any money to lend.

Just like the lenders, investors saw their excellent performance in a boom market as evidence of their investing acumen, their ability to pick good securities or good sectors, rather than evidence that they had gotten lucky in a time of easy money.  And because many investors, like hedge funds, pension plans, and mutual funds, have their own investors chasing returns, they were subject to the same pressures to take on more more risk that the lenders were under.  In particular, they started taking on a lot more leverage, borrowing money in order to lend it.  This is the practice that brought down Long Term Capital Management.

Perhaps the most important point is that regulators, the bonny chaps who are supposed to save us from all this madness, displayed the same pattern.  So did the politicians behind them.  Over the past few weeks, much has been made of the various regulatory actions that enabled this mess.  Though some are wrong, two are not:  the Democrats protected OFHEO's shockingly loose regulation of Fannie and Freddie against the White House's attempt to toughen it; and the Republican-appointed SEC loosened the capital requirements for the five largest banks.

But why did they do this?  Democrats seem to believe that the Republicans and the SEC simply did this out of wanton greed and a blind faith in markets; Republicans seem to believe that OFHEO, the Democrats, and Fannie/Freddie did this because of political corruption and a blind belief in homeownership for poor people.  But neither side was simply accepting the risk that the whole thing might come crashing down leaving the economy in tatters and the taxpayers on the hook.  The regulators, too, were misled by recent history.  In recent history, lending had been safer, and risk models did seem to be performing better.  Both groups genuinely believed that improvements in both computer models, and in economic theory of regulation, would allow them to identify and halt any crisis before it occurred.  And just like everyone else, when no disaster occurred, they became ever more confident in their own genius.

What we need, fundamentally, is not simply stricter regulation or less greedy bankers.  What we need is better economic theory of how these things play out, so that the regulators have better tools to assess and prevent systemic risk.  But that's not how we're thinking right now.  What we're looking for is not better tools, but someone to blame.

Because, after all, we know that it wasn't us who was at fault.  We're just the victim of broad market forces outside our control.