The Age of Prediction In a world where data is growing exponentially, increasing complexity may overwhelm those who don’t have the technology to try to make sense of it.

When I started trading, more than 20 years ago, I had just one alpha and I guarded it with my life. As I write this, my firm will soon pass 6 million alphas — i.e., algorithms or models that seek to predict the prices of securities — and I no longer have to worry about any of them like I used to.

The exponential growth in the number of alphas at WorldQuant, LLC is representative of a much larger trend. Each day the world gives birth to 2.5 quintillion bytes of data, created by everything from satellites and social media to credit cards and smartphones. Research firm IDC forecasts that the total volume of data will grow from 44 zettabytes in 2020 to 180 zettabytes by 2025.

In a 2012 report titled “Big Data, Big Impact: New Possibilities for International Development,” the World Economic Forum outlined how government organizations and companies could turn data into an economic development tool. Five years later Big Data is old news. What’s new is prediction.

The exponential growth in data is making the world increasingly complex, as the frequency of events accelerates and as the world gets more and more chaotic geopolitically.

For anything you’re trying to predict — securities prices, retail sales, unemployment, the weather — the amount of data available is growing exponentially. As a result, following certain mathematical laws, you get a linear increase in predictability as long as the data continues to grow at a nonlinear rate. All things held constant, everything will become more and more predictable from the perspective of the data.

But there are countervailing forces at work. The exponential growth in data is making the world increasingly complex, as the frequency of events accelerates and as the world gets more and more chaotic geopolitically. Even though people can predict events with greater accuracy, everything is getting more complicated. Unless you’re able to predict things better, the complexity is going to overwhelm you.

Technology is critical for success in the age of prediction. Organizations that can effectively employ machine learning and artificial intelligence to analyze all this data will have an advantage over their less sophisticated peers. People will be vital to the process. You need lots of people with lots of different ideas and opinions, because the key to better prediction is combining many different models. Once you have the people, you need the ability to test ideas quickly and easily. Here simulation technology plays a key role. Machine learning and AI can help develop more models, amplifying the humans at the center of the process. Instead of producing one model, a human aided by technology can produce thousands of them.

The age of prediction is not some far-off idea. It’s already having a big impact. For companies, it is improving manufacturing efficiency. For WorldQuant, it enhances our ability to deliver quality returns to investors. For predictive medicine, like the research initiative we recently launched in partnership with Weill Cornell Medicine, it is saving lives.

Welcome to the future.


Thought Leadership articles are prepared by and are the property of WorldQuant, LLC, and are circulated for informational and educational purposes only. This article is not intended to relate specifically to any investment strategy or product that WorldQuant or any of its affiliates offers or may offer in the future, nor does this article constitute investment, accounting, legal or tax advice or convey an offer to sell, or the solicitation of an offer to buy, any securities or other financial products. WorldQuant makes no representations, express or implied, regarding the accuracy or adequacy of this information, and you accept all risks in relying on the above information for any purposes whatsoever. The views and opinions expressed herein are those solely of the author, as of the date of this article, and are subject to change without notice, and do not necessarily reflect the views of WorldQuant, its affiliates or their respective employees. No assurances can be given that any aims, assumptions, expectations and/or goals described in this article will be realized or that the activities described in the article did or will continue at all or in the same manner as they were conducted during the period covered by this article.