Statistical problems, but not enough to warrant a rejection

“Evidence for a limit to human lifespan” contains two analysis: one on supercentenarians and one survival data. Experts think both are doubtful.

The article Evidence for a limit to human lifespan contains two different analyses. One is about the very oldest elderly people since the second half of the twentieth century. Are they getting ever older or not? The article contains two graphs on the subject. The most important (shown on the left) is based on the International Database on Longevity. In the appendix is a similar graph (shown on the right), based on the database of the Gerontology Research Group. Jeanne Calment, who died in 1997 at the age of 122, is the highest data point in both charts.

Jan Vijg’s team argues that there was a turning point around 1995: the very oldest people were no longer getting older. The data points before the turning point are coloured blue, those after are coloured red. (Source of graphs: Nature.) The maximum in the IDL is approximately 115 years of age.

Statisticians criticise the fact that the authors assumed in advance that a turning point occurred around 1995. Jan Vijg earlier told NRC Handelsblad this is clear from one look at the graph. “You don’t really need statistics if you can see it with your own eyes.” The authors adjusted their statistical trend analysis in accordance with that conclusion.

Critics argue that you must first infer a turning point statistically from the data. Various specialists in the field actually see in the data a continuing upward trend, albeit with large outliers (such as Jeanne Calment). Professor of Healthy Ageing Rudi Westendorp told NRC: “I see a continuous line, one that moves upward.”

Then there is the limited amount of data – for example, there are only 33 super-elderly in the left-hand chart. Reviewer Jean-Marie Robine believes you cannot draw far-reaching conclusions about the lifespan of humans from an analysis of only record-breakers. “That way, you are looking too much at the details,” he said in Utrecht.

There are also some inaccuracies. For instance, the right-hand chart does not show the annual Maximum Reported Age at Death (MRAD) (which is why there are years missing). Colleagues have also questioned why the mortality data have been rounded down. In two places, the article says ‘age’ where ‘calendar year’ is meant.

Survival in 41 countries

The other analysis is about the survival of people in 41 countries from the Human Mortality database. Vijg focused on France in his article; the other 40 countries are listed in the data appendix.

France first. The left chart is about the period between 1914 and 2014. It shows that during that period the biggest gain was in survival from birth to age 103 years (for women) and from birth to age 101 years (for men). That is where the peak in the graph is situated. After that peak (i.e. from age 104 to 110), the graph turns downwards. This means that the gain over time in the probability of survival from birth to an even higher age than 101 or 103 years, is lower than the maximum.

On the right-hand graph, there are about a hundred of these peaks in a row. The final data point in the right-hand graph therefore relates to the peak in the period 1914-2014, which we see in the left-hand graph. The right-hand graph has been reasonably flat since the 1970s. According to Vijg, this means that the period of increasing life expectancy for humans is over.

Demographer John Wilmoth (United Nations) points out that the gain in the probability of surviving to the very highest ages is indeed less than the maximum in the left-hand graph, but that there is nevertheless still a gain (values above 0). “So long as probabilities of survival from birth to each individual age are increasing, life expectancy is rising as well. It is impossible to argue on this basis that a ceiling has been reached.” According to Joop de Beer (NIDI, The Hague), someone who is 115 years old, still has a 40 percent chance of becoming 116.

There is also a substantive objection. Before 1950, the elderly barely contributed to the increase in life expectancy at birth: back then it was more about combating child mortality and infectious diseases. If you want to draw conclusions about the contemporary increase in the age of the elderly, the picture becomes distorted if you start in 1914.

Perhaps even more important is methodological criticism. “There are just too many statistical problems with this paper”, demographer Adam Lenart (Max Planck Instituut Odense) says. Lenart made several points, among which: the shape of the left-hand graph (the one with the peak) is highly dependent on an arbitrary assumption. This is a figure you need to neatly convert survival figures which have the value zero (which means that everyone is dead) into logarithms.

This presents a problem for this analysis, because rather a lot of survival figures are zero. For example, until the sixties, no one in France reached the age of 110. Lenart reconstructed the calculations of the peaks, to illustrate his point. Vijg’s conclusions about the lifespan of human beings are built on those peaks. Demographer Joop de Beer (Netherlands Interdisciplinary Demographic Institute, The Hague) confirms this problem. However, author Brandon Milholland denies. ”If I choose another value, I see pretty much the same results.”

Demographers also have difficulty with the fact that all 41 countries from the Human Mortality Database have been analysed in the same way and given the same weight. John Wilmoth, one of the coordinators of the HMD: “Many countries in the HMD have data problems. We have made that very clear, and authors need to take that into account.” Some countries have had very different histories from the western countries. The Eastern European countries, for example, still had higher mortality at young ages than Western Europe for a long time. A final problem relates to the length of the data series: data for Chile, for instance, begin only in 1992.

As a result, and due to the problems with the calculation method, the graphs for many of the countries look improbable.

Milholland says he deliberately showed everything. “It would be dishonest to say: these and those don’t fit the pattern, and I take them out. It’s best to show it all, and you can evaluate yourself.”

This summary is based on conversations with Adam Lenart, Joop de Beer, Brandon Milholland and John Wilmoth. It further incorporates comments by Peter van der Heijden, Jean-Marie Robine, Jonas Schöley, Jim Vaupel, Jan Vijg, Daniel Wells, Philipp Berens & Tom Wallis, and Rudi Westendorp. More can be found on the Publons and Pubpeer web forums, and on the Ask a Swiss blog.

    • Hester van Santen