My views on the market, tech, and everything else

Why is it so Hard to Forecast the Future?


For most of human history the life experiences of people have been overwhelmingly linear. Human are accustomed to encountering situations that reflect a simple proportional relationship between cause and effect. People expect that when they do X that Y will happen if Y was what happened in the past. This type of linear relationship is comforting to people since it is familiar. Ray Kurzweil believes: “our intuition about the future is linear because that is the way the world worked for most of history. Prey animals did not get exponentially faster for example.” Except for a virus or bacterial infection multiplying inside your body, few things in ordinary life are nonlinear.

The rise of modern science combined with modern distribution and other processes developed by businesses has resulted in people increasingly encountering nonlinear change. The economist Paul Romer explains one common reaction: “People are reasonably good at estimating how things add up, but for compounding, which involves repeated multiplication, we fail to appreciate how quickly things grow. As a result, we often lose sight of how important even small changes in the average rate of growth can be.” When something is sufficiently nonlinear, a phenomenon can seem almost magical. Especially when the outcome of a nonlinear change is negative, tendencies like loss aversion can kick in and people can have a strong tendency to react in highly emotional ways. Even people who are otherwise rational may not think clearly in this situation. Nobel Prize winner Daniel Kahneman describes one important reaction: “People talk of the new economy and of reinventing themselves in the workplace, and in that sense most of us are less secure.” People who feel less secure can feel confused and even angry. In one of his famous memos sent to readers just this week Howard Marks frames the current challenge:

“I realized recently that in my early decades in the investment business, change came so slowly that people tended to think of the environment as a fixed context in which cycles played out regularly and dependably.  But starting about twenty years ago – keyed primarily by the acceleration in technological innovation – things began to change so rapidly that the fixed-backdrop view may no longer be applicable.  Now forces like technological developments, disruption, demographic change, and political instability and media trends give rise to an ever-changing environment, as well as to cycles that no longer necessarily resemble those of the past.  That makes the job of those who dare to predict the macro more challenging than ever.”

We all must make decisions including about events that will happen in a future that is risky, uncertain and may involve unknown unknowns. What’s the right response to this reality? Howard Marks describes an approach that works for him: “We can’t predict, but we can prepare…. the key to dealing with the future lies in knowing where you are, even if you can’t know precisely where you’re going.  Knowing where you are in a cycle and what that implies for the future is very different from predicting the timing, extent and shape of the next cyclical move.” When change is both inevitable and gaining speed a person’s ability to adapt to the environment based on what he sees in the present is far more useful than trying predict the future. Steering as you react to signals being generated in the present is very different from trying to predict events that may happen in the future. The changes we are seeing in society and business effectively mean that evolution, as broadly defined, has accelerated. Evolution favors individuals who can adapt the fastest and most effectively to that accelerated change.

One way to be smarter about the ways in which we adapt is to change the way we think. One of many important ideas that have resulted from the work of Kahneman and Amos Tversky is the “distinction between two profoundly different approaches to forecasting, (1) the inside view and (2) the outside view. In his book Thinking, Fast and Slow Kahneman describes a view he and Tversky followed when they tried to predict when they would finish a project:


In his book Think Twice, Michael Mauboussin describes alternative to the inside view: “The outside view asks if there are similar situations that can provide a statistical basis for making a decision. Rather than seeing a problem as unique, the outside view wants to know if others have faced comparable problems and, if so, what happened. The outside view is an unnatural way to think, precisely because it forces people to set aside all the cherished information they have gathered.”

In addition to adopting a better viewpoint so to increase the probability of making a successful forecast, a person can also try to avoid making predictions that by their nature are particularly hard to make successfully. Howard Marks describes the objective best: “The more we concentrate on smaller-picture things, the more it’s possible to gain a knowledge advantage. With hard work and skill, we can consistently know more than the next person about individual companies and securities, but that’s much less likely with regard to markets and economies. Thus, I suggest people try to ‘know the knowable.’” Charlie Munger has a similar view: “Micro-economics is what we do and macro-economics is what we put up with.” Warren Buffett describes the objective of people like Marks and Munger in this way: “I don’t look to jump over 7-foot bars: I look around for 1-foot bars that I can step over.” Munger makes a similar point by joking that he wants to know where he will die, so he can just not go there.

What categories of phenomenon are less knowable? Grace Hopper famously said: “Life was simple before World War II. After that, we had systems.” When systems are involved it can get especially hard to make forecasts. What is a system? Nobel Prize winner Murray Gell-Mann has said that a scientist would rather use someone else’s toothbrush than another scientist’s definitions. Nevertheless, one dictionary definition a “system” is: a regularly interacting or interdependent group of items forming a unified whole. There are many types of systems, but the system that creates the greatest challenges for people in today’s world is a specific type known as a complex system. A complex system is a system composed of many interacting independent agents or elements that can lead to outcomes that are either difficult or impossible to predict by looking at the components. Capital markets, ecosystems, ant colonies and the human immune system are all example of complex systems. Michael Mauboussin describes a new reality: “Increasingly, professionals are forced to confront decisions related to complex systems, which are by their very nature nonlinear. Complex adaptive systems effectively obscure cause and effect.  You can’t make predictions in any but the broadest and vaguest terms.  Complexity doesn’t lend itself to tidy mathematics in the way that some traditional, linear financial models do.” Nassim Taleb identifies key ideas about complex adaptive systems in this way:


A small system like a business is not easy to make predictions about, but on a relative basis these predictions bout small systems are easier than making other more macro predictions. In other words, understanding enough about a single business like a hot dog stand, a grocery or Ford in order to make predictions with some reasonable degree of accuracy is far more possible to do than making predictions about the future state of an economy on a given date. Even with a focus individual businesses it is still hard to value a business. If it was easy to do so or if there was a simple formula to follow, everyone would be rich. Munger says anyone who thinks investing is easy is stupid.  Howard Marks writes: “investing can’t be reduced to an algorithm and turned over to a computer. Even the best investors don’t get it right every time. No rule always works. The environment isn’t controllable, and circumstances rarely repeat exactly.” Since the future is uncertain you must think probabilistically.

Until the Internet made facts as transparent and retrievable as they are today many people took comfort that experts from branded institutions understood what was going on and could reliably predict the future. The situation today is that people are able to easily “showroom” the predictive performance of experts against what actually happened and they do not like what they see. Sometimes you will hear someone say: “people have lost faith in our institutions.” That may be true, but no small part of what they may actually be saying is that they have lost faith in experts who claim to be able to predict the future. As people recognize that “experts” can’t predict the future any better than chance so they are doing things like flocking to index funds. Only when experts stop predicting the unpredictable will their credibility return. The expert may forget about their prediction failure due to hindsight bias, but the public can now easily fact check the record of the pundit. This reality will not change. Ever.

The number of and magnitude of systems that impact our lives is increasing as digitization of the economy proliferates and systems are to a far greater degree interconnected by networks. Mauboussin writes in Think Twice: “Unintended system-level consequences arise from even the best-intentioned individual-level actions has long been recognized. But the decision-making challenge remains for a couple of reasons. First, our modern world has more interconnected systems than before. So we encounter these systems with greater frequency and, most likely, with greater consequence. Second, we still attempt to cure problems in complex systems with a naïve understanding of cause and effect.” A networked and always connected world is an environment where people encounter complex adaptive systems in many more situations than in the past. Andrew Haldane, Chief Economist, Bank of England believes: “These systems are even more unpredictable because they nest within each other creating even more turbulence. Modern economic and financial systems are not classic complex, adaptive networks.  Rather, they are perhaps better characterised as a complex, adaptive ‘system of systems.’  In other words, global economic and financial systems comprise a nested set of sub-systems, each one themselves a complex web.”

Whether we like it or not, the economy is currently changing in ways that are increasingly nonlinear. Feedback of all kinds is being amplified in new ways and that increases the magnitude of nonlinearity. That crazy event you watch on a video is like the screeching of a microphone at a concert feeding back on itself. When nonlinear change happens the aggregate behavior of systems is vastly more complicated than would have been predicted by summing the inputs into the system. With a nonlinear system when we do X sometimes instead of Y happening Z can happen which we did not expect at all or had never even conceived of as a possible outcome. For example, in a nonlinear world you can train for profession X that has been around for many years and it can disappear over a very short period of time.

Volatility in the prosperity of businesses and professions produced by exponential change can be seen virtually everywhere. For example, many giant corporations are losing the competitive advantages that come with greater economies of scale and are under assault by smaller more nimble competitors. Smaller nimble businesses spend as much time competing with each other as attacking more established businesses, creating unprecedented levels of competition. Jobs disappear in one profession and are yet are being created in new industries, if you have the right skills. People are comparing prices of virtually everything using their mobile phones, which means profits that once were made possible by taking advantage of information asymmetry between producers and consumers are disappearing. The CEO of Costco told the CEO of American Express that the credit cards they provide are no different than ketchup. Shopping malls are being decommissioned as e-commerce rises. New markets are proliferating, value chains are breaking up, and profit pools are shifting. Industry boundaries are blurring and barriers to entry are disappearing. Sources of competitive advantage are fundamentally changing at unprecedented speed.

Cars and trucks driving themselves is a nonlinear change, especially if your job is to drive for a living. Stores that do not need checkers is a nonlinear change, especially if you do that or a living. People being able to get news for free instead of buying a newspapers is a nonlinear change, especially if you are a reporter. If you make your living selling any of the devices that a modern smartphone replaces that is nonlinear change.  As was noted above, since we are not accustomed to nonlinear change, when it happens it can be confusing. When humans get confused they start telling stories to make the world make sense again. These stories may or may not have any tie to reality, but they make us feel better. Daniel Kahneman has said: “What we have is a storytelling system and the coherence of the story determines how much faith we have in it.”

I have written a book on Native American legends entitled Ah Mo that captures how people in the Northwest part of what is now the United States used stories to make sense of the world before Columbus arrived in the Americas. These stories explained simple things like where fire came from and why salmon appear in the streams in the fall to spawn. A different sort of storytelling about complex systems is happening today, most notably as politicians or promoters try to use stories to explain things like why someone’s job has disappeared. As these stories proliferate being able to tell the difference between the truth, what we don’t know and what we can’t know, will be increasingly important.

On the subject of forecasting, in addition to reading what Howard Marks has written on the subject, I suggest that you read Philip Tetlock and Mauboussin, who writes in this cautionary note:

“The predictions of the average expert were ‘little better than guessing,’ which is a polite way to say that ‘they were roughly as accurate as a dart-throwing chimpanzee.’ When confronted with the evidence of their futility, the experts did what the rest of us do: they put up their psychological defense shields. They noted that they almost called it right, or that their prediction carried so much weight that it affected the outcome, or that they were correct about the prediction but simply off on timing.” https://doc.research-and-analytics.csfb.com/docView?language=ENG&format=PDF&source_id=em&document_id=1053681521&serialid=gRAGx5o9KjpeAGBLPq7bpyJRa6r6fj06KjHB6PGBbGU%3d


Think for yourself.



Howard Marks:  https://www.oaktreecapital.com/docs/default-source/memos/expert-opinion.pdf

Thinking, Fast and Slow http://www.mckinsey.com/business-functions/strategy-and-corporate-finance/our-insights/daniel-kahneman-beware-the-inside-view

Think Twice:  https://www.amazon.com/Think-Twice-Harnessing-Power-Counterintuition/dp/1422187381

Haldane: http://www.bankofengland.co.uk/publications/Documents/speeches/2015/speech812.pdf

Tetlock: https://www.amazon.com/Superforecasting-Science-Prediction-Philip-Tetlock/dp/0804136718

Ah Mo:  https://www.amazon.com/Ah-Mo-Indian-Legends-Northwest/dp/0888392443

Categories: Uncategorized