---
abstract: 'Rank size plots of very different systems are usually fitted with Zipf’s law, however, one often observes strong deviations at large sizes. We show that these deviations contain essential and general information on the evolution and the intrinsic cutoffs of the system. In particular, if the first ranks show deviations from Zipf’s law, the empirical maximum represents the intrinsic upper cutoff of the physical system. Moreover, pure Zipf’s law is always present whenever the underlying power-law size distribution is undersampled.'
author:
- Giordano De Marzo$^1$
- 'Andrea Gabrielli$^{2,3}$'
- Andrea Zaccaria$^3$
- 'Luciano Pietronero$^{1,3}$'
title: 'Deviations from Zipf’s law contain more information than Zipf’s law itself'
---

Introduction
============

Zipf’s law [@Language; @Jaguar] is one of the most important topics in complex systems. This law is ubiquitous in nature: it has been observed in the size distribution of cities [@Batty], of firms [@Firms] and of GDPs [@Batty], but also in natural language [@Language], in web page visits [@Web], in scientific citations [@Citations; @Newman] and many natural systems, such as earthquakes and lunar craters [@Newman]. There are currently numerous approaches to explain Zipf’s law based on different mechanisms including multiplicative processes [@Multiplicative], adjacent possible framework [@Loreto; @Kauffman], sample space reducing processes [@SSR], and information theory arguments [@Marsili]. However, while all these models give insights on the upset of Zipf’s scaling, they fail to provide a general explanation of the phenomenon. In particular a common feature, which is necessary but not sufficient, to get Zipf’s law is the presence of an underlying power-law size distributions [@Multiplicative; @Batty].

Here we show that the presence of cutoffs and the level of sampling are subtly connected to deviations from pure Zipf’s law, and these can be used to gather fundamental information on the system under investigation and its dynamics.

![**Rank-size plots.** (a) Rank-size plot of the $180$ most populous Italian cities and of the $180$ most used words in Melville’s Moby Dick. Both sets do not show deviations from Zipf’s law. (b) Rank-size plot of the $180$ most populous Chinese cities and of the $180$ rivers longer than $1000$km. Both systems show consistent deviations from Zipf’s law.[]{data-label="fig:various_rank_size"}](sizes.pdf){width="\linewidth"}

Let us consider $N$ objects, which could be for instance either the number of occurrences of N different words in a book or the populations of $N$ cities in a country. They are said to follow the generalized Zipf’s law with exponent $\gamma$ if the rank-size functional relation is of the form $$S(k)=\frac{S(1)}{k^{\gamma}}
      \label{eq:zipf's_law}$$ where the rank $k$ spans from 1 to $N$ moving from the largest to the smallest object. The easiest way to visualize Zipf’s law is through a rank-size plot in log-log scale. A quite underestimated phenomenon is that it is very common to find deviations from the expected linear behavior: the largest elements are often smaller than expected. In order to show this, we plotted in Fig. \[fig:various\_rank\_size\] the rank-size relation of Italian cities [@Cities], of Moby Dick words, of Chinese cities [@Cities], and of rivers longer than $1000$ km [@Rivers] [^1]. While the first two sets do not show any deviation, the second ones display a clear distortion for the first ranks. Such deviations are not due to the exceptional conditions of the specific system under analysis, being observed in many systems, see for instance [@Q_cities] for cities, [@Q_companies] for companies or [@Mandelbrot] for language. As shown below this phenomenon, affecting the scaling of the first ranks, is of great statistical relevance. First, it is important to note that similar systems, e.g. Italian and Chinese cities, can present or not this kind of deviations from the scaling behavior, while systems belonging to different domains can show not only a similar scaling regime, but also similar deviations. The universality itself of the deviations shows that their sources are often not system dependent, but, similarly to the Zipf’s scaling, derive from universal statistical mechanisms. More precisely we are going to show that the presence of deviations is directly related to essential dynamical properties of the system, independent of Zipf’s exponent, concluding that **deviations from Zipf’s law contain more information on the system than Zipf’s law itself.**

In order to quantify deviations from Zipf’s scaling, we can use the simple expression [@Mandelbrot] $$S(k)=\frac{\bar{S}}{(k+Q)^{\gamma}}
        \label{eq:Q}$$ where $\bar{S}$, $Q$ and $\gamma$ are fit parameters. For $Q\ll 1$ we recover the generalized Zipf’s law, while deviations in the first ranks appear as soon as $Q\approx1$; we can thus use this parameter to measure deviations. Fitting this curve to the four rank-size plots of figure \[fig:various\_rank\_size\] we obtain $Q_{\text{Italy}}=0$, $Q_{\text{Moby}}=0.14$, $Q_{\text{China}}=1.60$, and $Q_{\text{rivers}}=7.59$.

Zipf’s law: power laws and beyond
=================================

![image](earthquakes_MobyDick_Cities.pdf){width="1\linewidth"}

In this section we show how to relate deviations from pure Zipf’s law to the properties of the inherent Probability Density Function (PDF) of sizes. The two key elements are the eventual cutoff in the size PDF and the sampling rate. These allow to see deviations from scaling under a new perspective in which they play the role of useful indicators about the system statistical and dynamical properties.

It is well known [@Batty] that a necessary condition for a set of objects to follow Zipf’s law is that the size PDF is a power law, that is $$p(s)=\frac{c}{s^{\alpha}}$$ where $c$ is the normalization constant and $s>1$. To compute it we have to specify the upper and lower limits of the PDF $s_{m}$ and $s_{M}$, which correspond to natural cutoffs always present in real systems. These cutoffs are connected to $c$ by the normalization condition $$c\int\limits_{s_m}^{s_M}\frac{ds}{s^{\alpha}}=1 \ \rightarrow \ c=\frac{\alpha-1}{s_m^{1-\alpha}-s_M^{1-\alpha}}
\label{eq:c}$$ In order to derive the rank-size relation from the above PDF, we use the the fact that given the PDF $p(s)$ of a continuous variable $S$, the values of its Cumulative Distribution Function (CDF) $P(s)$, associated to the different values of S, are equiprobable [^2]. This implies that, given $N$ values of $S$ independently extracted from $p(s)$, with good approximation they can be taken as uniformly spaced in the corresponding variable $P$. Thus, the $k^{\mbox{th}}$ size ranked value $S(k)$ approximately corresponds to the CDF value $\frac{N+1-k}{N+1}$. In formulas $$\int\limits_{s_m}^{S(k)}p(s)ds=c\int\limits_{s_m}^{S(k)}\frac{ds}{s^{\alpha}}\simeq\frac{N+1-k}{N+1}\,,$$ which, together to Eq. , gives $$\frac{S(k)^{1-\alpha}-s_m^{1-\alpha}}{s_M^{1-\alpha}-s_m^{1-\alpha}}\simeq\frac{N+1-k}{N+1} \,.$$ By assuming $N+1\approx N$, $s_M\gg s_m$, and introducing $\gamma=\frac{1}{\alpha-1}$, we end up with the final rank-size formula $$S(k)=\qua*{\frac{Ns_m^{\frac{1}{\gamma}}s_M^{\frac{1}{\gamma}}}{Ns_m^{\frac{1}{\gamma}}+ks_M^{\frac{1}{\gamma}}}}^{\gamma}=\frac{N^{\gamma}s_m}{\qua*{k+N\ton*{\frac{s_m}{s_M}}^{\frac{1}{\gamma}}}^{\gamma}}\,.
            \label{eq:rank_size}$$

Eqs.  and imply the following relations $$\begin{cases}
                \gamma=\frac{1}{\alpha-1}\\
                \bar{S}=N^{\gamma}s_m\\
                Q=N\ton*{\frac{s_m}{s_M}}^{\frac{1}{\gamma}}\,
            \end{cases}
            \label{eq:bar_s_Q}$$ between the number of values$/$objects and the parameters of the PDF $p(s)$ on one side, and the Zipf’s scaling exponent $\gamma$ and the deviation parameter $Q$ on the other. In particular, $Q$ is connected to both the upper and lower cutoffs of the PDF and the number of objects in the system, i.e. the sampling from the size PDF.

In order to better understand this connection let us focus on the largest element in the system. From Eq.  we get $$S(1)=\frac{N^{\gamma}s_m}{\qua*{1+N\ton*{\frac{s_m}{s_M}}^{\frac{1}{\gamma}}}^{\gamma}}\,.
        \label{eq6}$$ As aforementioned, negligible deviations correspond to $Q\ll 1$, which is equivalent to $$N\ton*{\frac{s_m}{s_M}}^{\frac{1}{\gamma}}\ll1 \ \longleftrightarrow\ N^{\gamma}s_m\ll s_M\,.$$ From Eq. , this condition translates into $$S(1)\ll s_M
            \label{eq:condition_s(1)_s_M}$$ Condition has a very simple interpretation: a set of objects whose sizes are power law distributed follows Zipf’s law with negligible deviations at small ranks if and only if the population of objects is sufficiently small not to explore the tail of the PDF, so that the largest object is much smaller than the upper cutoff.

Zipf’s law and evolution
========================

We have just shown that deviations from Zipf’s scaling at first ranks can be measured through the parameter $Q=N\ton*{s_m/s_M}^{1/\gamma}$ and that they are directly related to the extent the underlying PDF has been explored by the system. In general, in an evolving system neither the number of elements nor the bounds of the PDF are fixed. This implies that, at least in principle, it can follow Zipf’s law at a certain moment of its evolution and show deviations later or, vice-versa, increase its adherence to pure Zipf’s law with time. Both kinds of evolution give important insights about the dynamics of the inner maximal scale of the system. This is well shown by considering three experimental sets which are usually told to follow Zipf’s law: earthquakes, language and metropolitan areas. In particular, we choose Italian earthquakes [@Earthquakes], Moby Dick book [@Moby_Dick] and US metropolitan areas [@Metro_areas]. For each set we fit the value of $Q$ at three different times and plot the corresponding rank-size relation in Fig. \[fig:earthquakes\_MobyDick\_Cities\]. All systems present a non trivial behavior [^3]: for earthquakes \[from (a) to (c)\] $Q$ grows with time, and clearly also the cardinality of the set increases; on the contrary in the case of Moby Dick \[from (d) to (f)\] the larger portion of the text, i.e. the more words we consider, the greater the agreement to pure Zipf’s law; finally, the rank-size plot of US metro areas \[from (g) to (i)\] is characterized by a more complex trend, in fact $Q$ decreased from $1790$ (g) to $1900$ (h), while deviations are again slightly present nowadays \[$2010$, graph (i)\].

A central question is whether Zipf’s law is a dynamical attractor for a given system with power law size PDF with evolving cut-offs, or not. We therefore introduce the concept of *coherent evolution*: a system is said to be evolving coherently if its dynamics prevents the proliferation of large objects, where “large" means close to the upper cutoff.

In order to visualize this evolution, we introduce the *Zipf’s plane*, defined by the axes $Ns_m^{1/\gamma}$ and $s_M^{1/\gamma}$. In Fig. \[fig:zipf\_plane\] we represent the motion in this plane of the three system introduced above. The color gradients from blue to yellow correspond to increasing values of $\log Q$. A motion from the yellow region to the blue one corresponds to coherent evolution, while a trajectory which goes in the opposite direction indicates an incoherent dynamics, i.e. an increase of deviations from Zipf’s scaling.

Meaning and evolution of coherence
==================================

![image](zipf_plane_24_10_19.jpg){width="1\linewidth"}

As aforementioned, a power-law distribution of sizes is only a necessary condition to produce Zipf’s scaling: there must be also a mechanism which prevents overpopulation of objects with a size too close to the upper cut-off. Earthquakes and language provide a framework to understand this mechanism. It is well known that earthquakes follow the Gutenberg-Richter law [@Gutenberg] - i.e. the energy released is power-law distributed - and it is reasonable to assume that, in a given seismic zone, the upper cutoff of this PDF can vary only over geological times. By using Eq. , we thus deduce that increasing the numerosity of the set, by considering larger and larger time windows (but always much smaller than geological scales), results in a growth of $Q$. This is confirmed by the corresponding trajectory in the Zipf’s plane, whose points accumulates in correspondence of the *maximum possible size for an earthquake occurring in Italy*. In the case under analysis the loss of coherence derives from the fact that earthquakes are by a good extent independent over long periods, being energy always injected into the system. It is therefore clear that power-law distributed objects cannot tend to Zipf’s law if they evolve independently as long as the cutoff of the inherent PDF is fixed. By looking at Fig.  \[fig:earthquakes\_MobyDick\_Cities\], it is then possible to conclude that no future Italian earthquake will be substantially stronger than the largest event already recorded, which is a rather interesting and non-trivial result, being obtained by simple statistical considerations.

Conversely, the trajectory in the Zipf’s plane described by words occurrences in Moby Dick is indicative of a coherent dynamics. We remind that now the objects are the different words, the sizes are their numbers of occurrence, and time can be seen as the progressively increasing fraction of the novel we consider. In this case, occurrences of different words are not independent events, being them constrained by grammar and semantic rules, which force a high degree of coherence in the text: thus the upper limit of the number of occurrences grows faster than the product $N^{\gamma}\cdot s_m$, where $N$ is the number of different words accumulated by progressively reading the text. For instance in order to increase $N$ new meaningful sentences, semantically coordinated with the previous text, are to be composed. However these sentences must contain, on average, many occurrences of the article “the”, which is the most common word in the book we considered. This leads $s_M$ to grow faster than $N^{\gamma}\cdot s_m$ and thus the deviations from pure Zipf’s scaling decrease with the number of pages.

The last system we consider is the set of US metropolitan areas. Here the behavior is more complex, being the trajectory in Zipf’s plane characterized by three different sections. Up to 1776 the States were, by a good extent, independent entities and each of them used its resources to make its own capital grow. As soon as interaction became relevant and the USA turned into a single nation, resources were centralized, flowing into only some of those cities and allowing them to reach a population that would have been impossible to sustain for a single State. This process, which corresponds to the emergence of New York as the driving city of the USA, is represented by the first part of the trajectory in the Zipf’s plane (Fig. \[fig:zipf\_plane\]), along which $s_M^{1/\gamma}$ increases very fast with respect to $N\cdot s_m^{1/\gamma}$. As already suggested by [@Batty], coherence derives from a screening mechanism produced by big cities over smaller ones. The second section of the dynamics in the Zipf’s plane, evolution at constant and approximately null $Q$, coincides with the almost unbounded growth of New York, while the third one corresponds to the slowing down of this growth and to the catching of Los Angeles metropolitan areas. We can interpret this in terms of New York reaching the upper limit of the PDF.

Conclusions {#conclusions .unnumbered}
===========

Zipf’s law is one of the most common and relevant scaling relations characterizing complex systems and, consequently, it has been studied from many different points of view. However, it has always been analyzed statically. In order to introduce a *dynamical evolution* in the problem, we focused on the deviations at first ranks/largest objects. A probabilistic argument permitted us to quantify these deviations by a single parameter $Q$ which is linked to the intrinsic cut-offs of the underlying power law PDF and the sampling density through Eq. . Deviations are substantially absent if $Q\ll 1$ and this can be expressed through the condition $$S(1)\ll s_M\,,$$ i.e., Zipf’s law holds also for first ranks only if biggest objects are much smaller than the intrinsic upper limit of the truncated power law. This equation also permits to understand why a dynamical approach is crucial: for real systems, $N$, $s_m$ and $s_M$ can vary during evolution and consequently deviations can appear, as in the case of earthquakes, or disappear, as happens with cities. For this reason, we introduce the concept of *coherent evolution*, which drives the system to exact Zipf’s law, and *Zipf’s plane*, a tool which allows studying deviations dynamically. We then showed that earthquakes, being independent and characterized by a fixed upper limit, can evolve only incoherently, while natural language is intrinsically coherent due to grammar rules. Moreover, we also found out that US metropolitan areas evolved coherently from the Declaration of Independence, due to screening effect between cities, up to the first half of the last century, when New York got close to the upper limit of the PDF slowing its growth.

In short, the main implications and applications of our work are:

-   Deviations from Zipf’s law contain important information unobtainable from Zipf’s exponent and they allow to study Zipf’s scaling under a dynamical point of view;

-   If the first ranks show deviations from Zipf’s law the empirical maximum is a good estimator of the intrinsic upper cutoff of the physical system. For instance, this can be used to predict the most powerful possible earthquake in a given region, that for what concerns Italy is $s_M\approx 7.4 M$;

-   Any power-law distributed system can show Zipf’s law spuriously during its evolution if the PDF is undersampled, while only systems characterized by internal coherence are asymptotically attracted toward Zipf’s law. Models should therefore distinguish between the two situations because they correspond to different dynamical properties.

We thank Michael Batty for his helpful comments.

[24]{}ifxundefined \[1\][ ifx[\#1]{} ]{}ifnum \[1\][ \#1firstoftwo secondoftwo ]{}ifx \[1\][ \#1firstoftwo secondoftwo ]{}““\#1””@noop \[0\][secondoftwo]{}sanitize@url \[0\][‘\
12‘\$12 ‘&12‘\#12‘12‘\_12‘%12]{}@startlink\[1\]@endlink\[0\]@bib@innerbibempty @noop [**]{} (, ) @noop [**]{} (, ) @noop [****,  ()]{} @noop [****,  ()]{} @noop [**]{},  (, ) @noop [ ()]{} @noop [****,  ()]{} @noop [****,  ()]{} @noop [****,  ()]{} @noop [**]{} (, ) @noop [****,  ()]{} @noop [****,  ()]{} [“,” ](http://www.citypopulation.de) () [“,” ](https://en.wikipedia.org/w/index.php?title=List_of_rivers_by_length&oldid=902925727) () @noop [****,  ()]{} @noop [****,  ()]{} @noop [****,  ()]{} @noop [  ()]{} @noop [**]{} (, ) @noop [ ()]{} @noop [****,  ()]{}

[^1]: For cities the size is the population, for rivers it is the length, while in the case of Moby Dick the size of a word is number of occurrences in the book

[^2]: If $p(s)$ is the PDF of the variable $S$ defined in the interval $[s_m,s_M]$, then $P(s)=\int_{s_m}^s ds'\,p(s')$. By performing the change of variables from $S$ to $P=P(S)$, and calling $f(P)$ its PDF, we get by definition of PDF and CDF $f(P)=\frac{ds(P)}{dP}p(s)|_{s=s(P)}=1$ for $0\le P\le 1$.

[^3]: In the case of earthquakes the size is given by the exponential of the magnitude, while for metropolitan areas we used population (in thousands)
