Velocity A second dimension of the Big Data revolution is its greater frequency and timeliness. Higher-frequency data can give new or timelier insights into trends in financial markets and the economy. It can also sometimes help with thorny identification problems, which otherwise plague both Big Data (as the Google flu example demonstrated) and classic econometric methods (as the DSGE example demonstrated). The crisis revealed that, in situations of stress, some of the world’s largest and deepest financial markets could be drained of liquidity. This resulted in some of these markets seizing up. In response, as one of their first acts, the G20 agreed in 2009 to collect a far greater amount of data on transactions in these markets, to 39  help better understand their dynamics in situations of stress. These data are stored in Trade Repositories. Over recent years, these trade repositories have begun collecting data on a highly granular, trade-by-trade basis. This has meant they have quickly accumulated a large stockpile of data. For example, around 11 million reports are collected in the foreign exchange market each working business day. These provide a rich data source when making sense of high-frequency financial market dynamics and dislocations. One example of such a dislocation came when the Swiss franc was de-pegged in January 2015. This unexpected move caused large shifts in asset prices. The franc exhibited a sharp V-shaped movement in  36  Farrell and Wheat (2015). Ali, Haldane and Nahai-Williamson (2012). 38 For example, Bean (2016). 39 See, for example, FSB (2010).
Traditional surveys of market participants or the general public tend to be biased in their sampling and framed in their responses. As in quantum physics, the act of observing can itself alter behaviour. These realities may call for exploring non-traditional means of revealing people’s preferences and sentiments. To give one recent example, data on music downloads from Spotify has been used, in tandem with semantic search techniques applied to the words of songs, to provide an indicator of people’s sentiment. Intriguingly, the resulting index of sentiment does at least as well in tracking consumer spending as the Michigan survey of consumer confidence. 51  And why stop at music? People’s tastes in books, TV and radio may also offer a window on their soul. So too might their taste in games. Indeed, I am interested in the potential for using gaming techniques, not just to extract data on people’s preferences, but as a means of generating data on preferences and actions. Existing models, empirical and theoretical, often make strong assumptions about agent behaviour. Theoretical models are based on axiomatic assumptions. Empirical models are based on historical  49  Rotemberg (1984), for example, discusses the statistical rejection of rational expectations models for consumption and labour demand. 50 Tuckett and Nyman (2017), Shiller (2017) and Nyman et al (2018). 51 Sabouni (2018). 12  All speeches are available online at www.bankofengland.co.uk/speeches  12  behaviours. These restrictions may, or may not, be borne out in future behaviour.