The surprisingly weak case for global warming


Fast summaries


Introduction and definitions


  1. The earth has been getting warmer.


The data and the test (for GW1)





Testing GW2




Initial results



Testing assumptions Part 1





Correcting for correlation




Correcting for correlation Part 2 (A better way?)




Here’s what the plot looks like subject to the correlation constraint:


One more assumption to test




Beyond falsifiability





Invoking Pascal’s Wager







Beware the anthropic principle











Final analysis and caveats




Thanks for reading all the way through! Here’s the code:

Code in R

# There has to be a more elegant way to do this
# Get a single vector of Year over Year changes
rawChanges = diff(theData$means, 1)
# SD on yearly changes
changes = rawChanges - mean(rawChanges)
# Find the total range, 1881 to 2011
y1a = theData$means[1]/100 + 14
y131a = theData$means[131]/100 + 14
netChange = (y131a - y1a)*100 
# First simulation, with plotting
trials = 1000
finalResults = rep(0,trials)
for(i in 1:trials) {
	jumps = sample(changes, 130, replace=T)
	finalResults[i] = sum(jumps)
		jumps = sample(changes, 130, replace=T)
	# Add lines to plot for this
	finalResults[i] = sum(jumps)
# Plot of YoY changes over time
# Is there a trend?
absRawChanges = abs(rawChanges)
pts = 1:130

Tags: , ,


  1. Matt:

    This concerned me, so I too went to the NASA data that you linked to and conducted a Bayesian moving average time series analysis. The goal was to determine the distribution in the trend rate of annual changes in temperature. I allowed for the trend rate to change over time and also allowed for the standard deviation in the trend rate to change. The model that I used is:
    [Temp in year t] = [temp in year t-1] + c + b(t + (n-1)/2) + theta * (change in temperature in year t-1)


    c= annual trend in change in temperature (the Trend Factor)

    b= annual change in Trend Factor

    theta = regression to the mean factor

    t=1,…, 131

    I also assume that the change in temperature has a standard deviation that can change over time (parameter d).

    Based on my analysis, the value, c, the Trend Factor, has a 97% chance of being greater than zero. This suggests that the NASA data is strongly supportive of Global Warming.

    The full results of the analysis and the code that I used can be found here:

    In summary, using the same data that you used, but adopting a Bayesian approach, I get to very different results. Perhaps this is due to the way you model correlations, where the correlation between years gets “broken” after two or three years.

  2. “To be as clear as possible, many of the GW claims are highly extreme, in terms of estimated temperature changes (all of the extreme predictions made 10 years ago have failed) and how we should change our lives. Do the data provide strong enough evidence to support this?”

    Well, extreme predictions usually fail. But maybe you’d like to indicate which extreme predictions you are talking about? Is it the IPCC on atmospheric temperatures? Or sea level rise? And by saying “in the last 10 years”, aren’t you falling into the usual trap of drawing conclusions from not enough data?

    “At this point, none of the proponents of the consensus views on GW have been willing to state their degree of belief in the different GW claims.”

    Quite a few scientists have given likely bounds on climate sensitivity, and this is the key claim in AGW. The “skeptics” like to pretend it is 0.7C (or even 0C) for a doubling of CO2. The scientists generally go for around 2 – 3C for a doubling of CO2. There’s also some distinction between the transient response (which only includes “quick” feedbacks like humidity) and the equilibrium response that includes the long term feedbacks like changing albedo with loss of ice.

    “None have pointed to an alternative data set that’s more convincing (or any other data set). None have posited an alternative method that doesn’t start with an assumption of a trend then require that it be disproved.”

    But it is an almost trivial exercise to show that the assumption of no trend in the temperature data is wrong. However, if you were give the data in isolation, you would only conclude that it had increased – you would have no idea about its future behaviour. But the data is not in isolation, it is accompanied by a convincing physical rationale for a rising temperature. Put the two together, and its very hard to see anything other than a continued increase in temperatures.

    Of course you should feel free to look for other explanations, but “skeptics” have been doing that for quite a few years now, and they have yet to come up with anything even remotely as convincing as AGW. What is worse, many of their attempts contradict each other, and seem mainly to exist to mislead people, rather than having any substance to them. Its almost as though the “skeptics” know that they are wrong, but are trying to convince the general public that they aren’t.

  3. Now this is just getting silly. A ridiculous model, whose implications about long-term climate stability are utterly disastrous, ‘can’t be ruled out’ as a competitor with GW2 because it allows large departures from initial ‘temperatures’ without any basis in physics, and this is insisted on as a serious objection, despite the fundamental untenability of this model being pointed out in multiple comments?

    Then, to pile Pelion on Ossa, the author complains that there are deep problems of falsifiability, ‘Pascal’s wager arguments’ and the new suggestion that greater year to year variance could explain the strong trend in global temperatures over the second half of the 1881 to present record.

    Falsifiability is not a problem here: show that CO2 isn’t a greenhouse gas, or that the evidence for rapidly climbing CO2 levels is somehow misleading, or that the basic radiative physics is wrong, or that measures of change in outgoing LW radiation are wrong, or that … the list is nearly endless but I don’t see any chance of the challenge being taken up seriously here, where predictive models are dismissed as unsupported because non-physical tinker-toy models are “alternatives” we should consider.

    Pascal’s wager starts out from the assumption that we have no evidence at all for Xianity, and argues we should believe it anyway. No one is using that kind of argument to support taking action to reduce GHG emissions. Instead, real evidence, from the basic physics to measurements confirming the impact of GHGs on downwelling LW radiation to the agreement of serious physical models that substantial warming is a very real risk, is what motivates acting now to avoid the worst-case outcomes of continued, unrestrained emissions.

    Finally, if you have a credible model instead of a statistical tinker toy that gives a substantial probability to the observed trend given only increased variance, let’s see it.

  4. Matt Asher: “The advantage of my ignorance of the science is that I can offer an independent analysis of the data. ”

    OK, Matt, read that. Now, read that again. If you do not understand the context of the data–and that includes the physical system that produced it–you cannot hope to develop a meaningful analysis.

    You also misunderstand the point of the graphic supplied by Alfonso. It is that even with the stakes as high as they are, that only 0.17% of articles in climate change question anthropogenic causation of the current warming epoch. If there were any controversy over the role of CO2 in climate, you would expect that to be far higher.

    As to falsifiability, climate models have a very good record of successful prediction. See, for example:

    Finally, as a Toronto graduate, might I commend to you the work by Jim Prall:


    • That would be a great response iff (if and only if) climatologists’ predictions turn out to be true.

      The easiest predictions to test against reality are those made at IPCC’s first assessment report (FAR), the report that got the ball rolling. Let’s take the predictions from the FAR and compare to the data.

      It can be found here :http://www.ipcc.ch/ipccreports/1992%20IPCC%20Supplement/IPCC_1990_and_1992_Assessments/English/ipcc_90_92_assessments_far_overview.pdf (this is the document that the IPCC saw fit to present to “policymakers” and clearly intends as a guide to any decisions)

      The claims made here now have accrued ~ 22 years of data (and given they’re 100 years-out predictions, even 22 years seems far too little data).

      “A n average rate o f increase o f globa l mean
      temperature during the next century of about 0.3°C per
      decade (with an uncertainty range of 0.2—0.5°C per
      decade)” (uncertainty range is generally meant to mean 95%)
      Second prediction – not applicable since action was not taken, CO2 usage has grown as assumed for “Business as usual scenario”
      “an average rate of global mean sea-level
      rise of about 6 cm per decade over the next century
      (with an uncertainty range of 3—10 cm per decade)”

      Ok now let’s connect the actual data.

      A) temperature change:
      Temperature rise per decade for 1990-2012 interval:
      (data from http://www.wolframalpha.com/input/?i=global+climate+studies+from+1990+to+2012 )
      14.2 to 14.25 or 0.05 degrees. Split over the decades we get 0.1 and -0.05 degrees.

      ALL these values fall outside of the 95% certainty interval that is presented as the scientific consensus.

      Now I have had this course “philosophy of science” that I distinctly recall teaching me that if predictions don’t pan out, the theory they’re based on is flawed. Granted I hated the course, but still.

      B) sea level rise
      data from http://en.wikipedia.org/wiki/File:Trends_in_global_average_absolute_sea_level,_1870-2008_(US_EPA).png

      I read this graph as slightly under 1 inch per decade, or ~2.5 cm per decade.

      Again this value is outside of the 95% interval that the IPCC’s scientific consensus gave.

      So, frankly, the way I see your comment is “people who live in glass houses shouldn’t throw stones”. Maybe the statistics are wrong. But the scientific consensus is wrong, by scientific rules (wrong prediction made -> your theory is wrong).

  5. “Is it possible that, on a small enough scale, the changes in direction of your boat are essentially random?”

    And, if true, you’d then conclude that it’s possible that the engine, right there in front of your eyes, doesn’t exist, and that the boat quite likely will never cross the atlantic?

    CO2 forcing is real. If you think you can disprove this basic physical fact with statistical analysis, I invite you to stare into the business end of a CO2 laser, hit the “on” switch, and report back afterwards …

    Meanwhile, I have a perpetual motion machine for sale. Make me an offer …

  6. This post wasn’t even making that strong of a statement, just saying that a random walk can’t be ruled out-

    has ignited the true believers.

    Apparently, infidels (like Mr. Asher) must be purged if they do not accept chapter and verse-

    well done.

    • Scharfy, a random walk can certainly be ruled out on physical grounds unless you want to throw out conservation of energy. Several other posters have also pointed out a variety of other problems with the analysis–including that violates its own assumptions. Are the concepts of coservation of energy and self-consistency too advanced for you?

  7. Interesting exercise and all, but the problem is that the data you’re looking at are a summary statistic of a huge physical system that follows physical laws like any other. This isn’t some particle bouncing around in a dish where we’re trying to figure out if it’s moving randomly or has a center point it’s attracted to. We KNOW there are physical laws that create a baseline expected temperature, and that if all the inputs that affect temperature don’t change, we CAN’T be observing a random walk with paths leading to wide dispersion from the expected mean equally likely as other paths that don’t deviate from the expected as much.

    Think statistical mechanics. States that lead to temperatures far from the expected mean have a high free energy and are unstable and unlikely. Just taking the auto-correlation into account is a weak model for this fact.

    • Matt, I’m very much afraid the negative auto correlations don’t tell you very much. I did the calculation based on the data I get a value of -0.30. If you had nothing but random normal deviates to start with you would expect a value of -0.50 for the corelation. Do the math. It is the correlation between x1-x2 and x2-x3.

      • Hi Larry,

        Yes the correlation is -0.30 or -0.32 depending on whether you compare the vector of data to itself offset by one:


        or compare the odd and even entries of x.

        I’m not sure how you got such a high amount for the expected correlation. Random noise tends to give a much lower number, try running this a few times:

        x = rnorm(130)

        Also, I saw the post on your blog, please note that I never claimed this method could be extended to an indefinite number of years. In general, models (or simulations) that are good over specific ranges are the norm for what we do, not the exception, no?

        If you did a study of some students and found that the scores they got on their test could be modeled well with a straight regression line, should you reject that model because extending it to a student who studies 50 hours would predict the nonsensical result of 120% on their test?

  8. Hmm, you ran your random walk model for 130 years… We have temperature data sets going back for millenia, or longer depending on what proxies you use.

    What would your random walk model show for the temperature evolution of the earth over 2000 years? 200K years? 2 B years?

    According to your simple statistics, shouldn’t the temperature have “walked off” one way or another by a pretty large amount by now?

    I think you have shown that no one expects the earth’s temperature to follow a random walk. Good job. Now go learn some of the science and try again.

  9. Hi Matt
    I have a few comments to your code.

    Why don’t you just use column 15 in the datafile you link to? That is the annual average from December to November, and it is easy to see that it is equal to the average of the seasonal numbers you use.

    But I would actually recommend you to use the Januar to December average from column 14, since then will have data for1880 also.

    To get reproducible results of simulation it is a good idea to set the seed of your random number generator. I added this line before the first simulation:

    I then get the fraction of trials that were more extreme than the original data to 53.7%. You should be able to reproduce that value if you set the same seed.

    Btw, a lot of your code is on the same line above, so that line is extremely long, makes it a bit confusing to rerun.

    • Hi SRJ,

      Thanks for your recommendations about the code! I’ll try to remember to set seeds and keep my line lengths reasonable. I wish R had an easy way to do multi-line comments.

      • Hi Matt
        Another question
        In the part of the code that does the Maxent stuff, why do you use the value 74 in this line:
        ( length(finalResults[finalResults>74]) + length(finalResults[finalResults<(-74)]) ) / trials

        Shouldn’t it be the netChange that you calculate earlier?

        • Sorry about that, that line was old code that hard-coded the temp change from an initial (slightly different calculation) I did. Use netChange (which is 75.75).

          ( length(finalResults[finalResults>netChange]) + length(finalResults[finalResults<(-netChange)]) ) / trials

  10. ad Sharfy: We’re making the point that this little exercise in statistics is not relevant to a serious discussion of climate change. If you think it is relevant, then let’s hear why. ‘True belief’ isn’t the issue– having a serious discussion is. Are you up for that?

  11. A couple people have argued that it’s invalid to use a model for 130 years of data if that model becomes meaningless when extended to a million years. Note that if this is your argument, than you are saying that most regression analyses are invalid because they cannot be extended much beyond the existing data while still retaining meaning.

    Think for a moment, do you really believe that the same simulation must work for 130 years and a million? If so, then why do we need climate change models at all when we have perfectly good weather forecasts? Why not just extend these out for the next 10 years?

    A lot of the confusion stems from the thinking that I’m trying to “model the climate.” I’m not. I’m doing a simulation based on the real data, to see how extreme the observed results are relative to the yearly changes and what might happen if this empirical data represented a distribution of possible changes. If that’s not clear, please re-read the post.

    Note that I’ve asked a number of questions in the comments to try and understand where those who disagree are coming from, but I can even get basic quantitative estimates of how strongly you believe in the different GW claims (thanks to Joshua for providing a qualitative estimate). Nor has anyone given any indication that failed predictions or periods of non-warming matter (thanks to Anne R. for providing a list).

    One final note, there’s an inherent tradeoff between mistaking noise for signal, and missing an existing signal. Some of the complaints about the piece take the form of “If you look at it like this you can see the signal.” I don’t doubt that true, but you may also see something where there’s nothing (or only a very small signal). See my post about testing your model with fake data.


      Every stat course I took that touched on regression also stated one of the required limitations: Thou shalt not estimate beyond the data. Worth remembering.

    • “…you are saying that most regression analyses are invalid because they cannot be extended much beyond the existing data while still retaining meaning.”

      As Robert notes, this is indeed a standard caveat in the very first lecture in regression analysis.

      Show me a person who extrapolates regression data beyond the range of the independent variable and I’ll show you someone who is misapplying statistics. Moreover, I’ll show you someone who is misapplying statistics in ignorance of the complexities of the physical world.

      For those who don’t understand the point, in the world of regression “as within” an independent variable does not equal “as beyond” an independent variable. Ignore this dictum at your peril.

  12. Irrelevant distractions here from Matt. Your exercise is pointless if the alternative view of the (recent) climate record it offers is not a real alternative. No one needs the temperature record itself to rule your random walk out– it’s ruled out physically: absent substantial changes in the energy flows involved, large excursions of temperature (secular trends like the one actually observed) are not possible. Your model ignores that constraint, so it’s not a candidate and not a serious competitor for GW2. So it doesn’t show that there’s a significant probability that the climate change we’ve observed is the product of some kind of random process and not evidence of a substantial change in the forces that drive the climate.

    And we know you’re not trying to model the climate. That’s the problem: what we’re talking about is the climate!

    You want a response re. your various hypotheses– there’s already a pretty good one above– but I’ll bite, too:

    1: yes 2: yes 3: yes (re. my understanding ‘radical’: the rate at which we’re forcing the climate is extreme, and the consequences of a 4 degree Celsius rise would be very serious; with a bit of bad luck a mass extinction on the order of the paleocence-eocene seems possible) 4: yes 5: yes (well understood radiative physics by itself makes this likely) 6: yes 7: yes (a combination of energy efficiency and aggressive deployment of low carbon energy technologies– not as big an effort as a major war, with much better payoffs) 8: yes (most side effects are improvements on present practice: health benefits from reduced air pollution, higher population densities and less sprawl occupying good farmland, …) 9: yes (I thought, from 8, that you were worried about unintended consequences? Global engineering of the kinds proposed looks very risky to me– and requires a long term, continuing effort to sustain, so long as GHG levels remain elevated, i.e. for much longer than the median lifespan of any political order the world has ever seen).

    • “And we know you’re not trying to model the climate. That’s the problem: what we’re talking about is the climate!”

      Wow are you a troll or just really dense? I’m only in second year (biostats) and I get it. He did a simulation of global temperature and found it was indistinguishable from a random noise!

      PROTIP: “quantitative means” numerical. Since you don’t give your probability for the warming claims, you must be saying “yes” I have 100% confidence in them. That’s not how science works! You don’t even care about the data or predictions. It’s like if a drug is tested vs placebo and the statistician finds no real difference only natural variation then the company says “oh our theory says it has to work because we understand the chemistry and you don’t we know there must be an effect you need to look at the data with our theory.” They say go ahead FDA approve it. We are 100% sure it works no doubt! You don’t even consider what is significant effect size. Basic stuff!

  13. You’ve put the AGW case in point form, and I’ll say that I believe in 1 – 5, and hope that the remainder are true. But as an exercise, I’ve constructed what I believe the skeptic position to be:

    1) Its not warming, and anyway, its been warmer in the past
    2) If it is, its natural, or it will only be small
    3) If it will be big, it will be good. Warm is better.
    4) CO2 is not to blame, it is plant food.
    5) CO2 levels are not going up, and they’ve been higher in the past.
    6) If CO2 levels are going up, its from undersea volcanoes
    7) If CO2 levels are going up, and it is our fault, there is nothing we can do about it, because civilisation would collapse if we stopped burning fossil fuel

  14. “Think for a moment, do you really believe that the same simulation must work for 130 years and a million?”

    Yes, of course. The physics haven’t changed. The major problem is getting good data on solar output at the time, etc. If major forcings can be pinned down, geographical location of the continent[s], etc, then yes, we’d expect a good simulation to work over any timeframe.


    Well, weather forecasting models don’t have to deal with changing forcings due to decadal fluctuations in solar output, increased forcing due to increased GHGs, Milankovic cycles, etc. On the other hand, climate models don’t have to be as concerned with precise and fine-grained data on current conditions.

    You’re not making a lot of sense. And my perpetual motion machine is still for sale.

  15. ““Think for a moment, do you really believe that the same simulation must work for 130 years and a million?”

    So your position is that paleoclimatologists who do use GCMs to model paleoclimate are on a fool’s errand? Because they do, you know.

    A moment in google reveals that much work along these lines is being done, just one example:


    Perhaps you’d like to suggest they model climate as a random walk instead?

  16. Take a good look at the plot and play with Matt’s code and it is clear that the random walk model does not do a good job of reflecting reality.

    Yes some of us have looked at it over a million year time period. Clearly it fails there. But the type of changes implied by the typical simulation over the 131 year time span imply much large temperature variations than have been seen over that time span at least since the end of the last ice age.

    In addition the individual simulations imply a much greater variability even within the 131 year plots. Change he variable trial to something like 10 and change the code in the line command to something like
    col=rgb(0, 0, 1, alpha = .9) so that the individual simulations show up much better. It then becomes readily apparent that every single simulation shows a much greater variability than does the red temperature line. This is clear indication that the model does not reflect what is happening in the real world.

    Change trials to 1 and you can see the individual simulations. Run the code a hundred times if you want. When none of those simulations, which claim to represent what was going on with the real data, show stability in the temperatures equal to or better than the actual temperature record that is clear evidence for the failure of the model.

    With the failure of the model the claims about the weak evidence for global warming cannot be substantiated.

    The problem Matt has is that he has proposed a model but has failed to justify why it is a good model. He has made no attempt to validate that the model reflects the real world. And then goes on to make claims about what the failed model says about global warming.

    • Larry, I found your comment to be the most disappointing:


      Matt has proposed that the current trend in global temperature can be explained by random chance. Using this model would almost never predict the actual outcome because it is nearly impossible to predict chance. We cannot model the outcome of 100 coin flips, no matter how well we understand the coin or the mechanics of flipping it. But we can calculate that 51 heads and 49 tails does not a biased coin make.

  17. Matt – I actually think it is an interesting academic exercise to see if you can tell if that data was randomly generated (without any understanding of the physics) – there was a few arguments that I thought were interesting that I don’t think you have addressed

    1) the ADF test that alfesin mentioned
    2) the Bayesian moving average time series analysis that Howard mentioned
    3) the Eduardo Zorita paper that I mentioned


  18. (I didn’t mean to post this as a reply to an earlier comment, but as a new comment. Sorry for the double post)

    1) Matt, now you are inventing new terminology. “[A] structured random process with no trend?” The term random walk is not only used in the context of independent increments. What you are simulating IS a random walk. The fact that the increments are correlated does not change its asymptotics (limsup = +infty, liminf = -infty), as you can guess from looking at your plot. And again, I am not just raising this objection because of the asymptotics- but because a random walk is the wrong kind of model to use and you are effectively cheating by using it.

    *** I challenge you to explain your reasoning for choosing a random walk for your simulation, instead of something like linear regression or splines.

    2) “As with all of my posts here (see the website motto or the manifesto page), I don’t so much model (models try to see if the data conforms to platonic distributional forms) as simulate. … [The] data … provides no evidence to reject this hypothesis.”

    What you are simulating from is a model- a random walk model with non-independent increments sampled from an empirical cdf. Your hypothesis is that the data are well-represented by this model, and you failed to reject that hypothesis. Now, leaving aside the point above that this is the wrong kind of model and that makes it unduly difficult for the data to cause rejection of the hypothesis, as others have pointed out we could still reject this hypothesis using a different statistic. The statistic you chose was just the temperature difference at the end of 130 years. That statistic is not a sufficient statistic for this model (by far- it loses a huge amount of information).

    *** I challenge you to consider some other statistics and provide p-values. For example, consider the integral of the square of the sample path, and report a p-value for the probability of observing a smaller square integral than the data.

    (As an aside: I think you undervalue models. Of course they are platonic, but they serve a purpose for which we don’t yet have anything better. Language serves to represent things so that we can reason about them and communicate, but nobody would mistake a definition of a chair in a dictionary for an actual chair. Similarly, careful modeling allows us to reason about things and understand them. And understanding is vitally important, it is the difference between a scientist who actually knows stuff about the real climate and a hobbyist blindly writing code and producing graphs that have no connection whatsoever to the thing he thinks he is analyzing)


    Okay, I’ll bite harder this time:

    1-4 all greater than 99%. I would say 100*(1-epsilon)%, but I hesitate to be that certain about a topic I don’t know much about based solely on the expertise of others, and I guess the climate may be a sufficiently complex system so that it’s possible the experts have missed something.

    The remaining claims are stated in ways where I am highly certain about some aspect of them and uncertain about another. For 5, I would say greater than 95% that humans are an important cause, and I have no clue about whether they are the MOST important cause. For 6 I would say 100% that we can (hypothetically) stop the trend and maybe 80% or so that we could hypothetically reverse it (this is an off-the-cuff guess).

    For 7, I would say less than 20% if “realistic” is interpreted to mean “without drastic change from the current political system.” But this is entirely a failure of politics, not economics or science or will of the people.

    For 8, I guess less than 10% for unintended negative consequences being more damaging than warming itself. And I think this guess is conservative- not a tight upper bound.

    For 9, say greater than 80% (again just a ball-park).

    These are all very very subjective degree of belief statements and I would probably say different numbers if I were to answer again at another time.

    *** I challenge you to state your own degrees of belief in the list of GW claims or any other GW-skeptic claims that you prefer over them.

    4) “None have pointed to an alternative data set that’s more convincing (or any other data set). None have posited an alternative method that doesn’t start with an assumption of a trend then require that it be disproved. … None of the proponents have addressed the points I raised about degrees of falsifiability, biases (including our tendency to see patterns and the anthropic principle)”

    I guess you haven’t read all the comments or something (there are quite a few of them). Of course people have suggested other data, the berkeley link for one. And other models, like the ones I mentioned above, were also brought up before (regression). The fact that regression involves a mean function does not mean it is assuming a *nonzero* trend. The question is whether the regression function is increasing over time, and you could conceivably answer “no” if the data allows that.

    I specifically mentioned falsifiability, along with other commentors above. The theory of GW makes many predictions, all of which can be falsified. Or you could try to falsify any of the many intermediary scientific results that GW rests upon, such as the laws of thermodynamics- that would call into question all the models built using those laws. I also specifically mentioned biases, asking if anyone has ever demonstrated whether or not the scientific community is more likely to accept studies which confirm its existing theories or ones which disprove or establish other theories. My hypothesis is that there might actually be a bias toward constantly changing things, even unnecessarily, because of the “publish or perish” imperative.


      Historically, more the other way. Three points.
      1) Einstein’s theories were not rapidly accepted, since they conflicted with Newton, and everyone else.
      2) Einstein resisted quantum mechanics, because he didn’t believe in the premise (God playing at dice, and all).
      3) Standard dictum: data doesn’t invalidate a theory, but another theory does.

      3) may sound odd to the youngsters, but it’s been an article of faith for hundreds of years. Science doesn’t toss a theory based on some contradictory data. A theory gets tossed when a proposed theory explains both the data explained by previous theory as well as data only explained by the proposed theory. This is why it’s the “theory” of evolution. Saying “God did it” for either evolution or climate change doesn’t cut it.

      Publish or perish has led, given the explosion in population which includes Ph.Ds, to a lot of dancing on pinheads. Fundamental theory disputes largely ignore all of that.

      • Your first point is not correct.
        According to Wikipedia. special relativity was widely accepted within 6 years of publication:
        Eventually, around 1911 most mathematicians and theoretical physicists accepted the results of special relativity. …

        And experimental confirmation of general relativity was coming in as early as 1919, only four years after Einstein’s final version in 1915.

        Your second point is irrelevant, a physicist is not physics, and so what if Einstein wouldn’t accept QM?

      • “1) Einstein’s theories were not rapidly accepted, since they conflicted with Newton, and everyone else.”

        On the contrary, within five years of publishing his paper on special relativity, “most mathematicians and theoretical physicists accepted the results of special relativity”. (http://en.wikipedia.org/wiki/History_of_special_relativity#Early_reception)

        Physicists knew Newton’s theories were wrong long before then due to experimental evidence and Einstein’s work was the culmination of much effort to resolve that problem.


        He did, however, receive the Nobel Prize for his 1905 paper on the photoelectric effect, which effectively *established* quantum theory. What he *resisted* was the premise that quantum theory is complete and that there are no local hidden variables that, if known, would explain away the apparent randomness. It wasn’t until 1981 that experiments convincingly proved John Bell’s 1964 theory that there are no (local) hidden variables, both well after Einstein’s death. His “resistance” is therefore a lot more subtle than you suggest.

        “3) Standard dictum: data doesn’t invalidate a theory, but another theory does.”

        That depends on both the data and the theory in question. A well-established theory that has stood the test of time and made countless correct predictions isn’t going to be tossed overnight because of one experiment that contradicts it. Remember “cold fusion”, an experimental result that seemingly overturned known physics?

        Furthermore, even though a theory is known to be wrong — as both the Standard Model and Relativity are known to be — doesn’t mean the theory isn’t *useful*. We just have to know under what circumstances each is valid, and tread very carefully when dealing with situations where both apply (e.g. singularities).

  19. Conclusion that statistical analysis with random walk fails to find evidence of global warming is correct but seems incomplete. How strong is this conclusion?

    What is missing is an experiment that applies the same method on data with known trend and same variance.

    For instance, if we measure the temperature in a thermostat that keeps it between T1 and T2, the temperatures will go up and down over time. Random walk permutations will go far beyond T1 and T2. If T1 and T2 are changing over time how extreme the trend should be for this analysis to detect it? One can use synthetic data generated in the same R program with the same variance. I am afraid that for all but a few extreme trends this analysis will come to the same conclusion – no evidence of a trend. But again – such experiment is missing.

  20. John Rogers (December 6, 2012 at 6:26 am)

    “He did a simulation of global temperature and found it was indistinguishable from a random noise!”

    No he didn’t John. He made 1000 runs of a random walk model that produced a large envelope of outputs one of which (or a subgroup of which) could be selected as being coincidentally similar to the actual progression of surface temperature.

    Your “PROTIP” example is not a good analogy ‘though it does perhaps have an ironic relationship to Matt’s analysis. The FDA wouldn’t approve a drug based on lack of efficacy relative to a placebo. On the other hand (and we know this happens occasionally), a drug might be approved under circumstances that the company was aware of a problem with respect to side effects, or had given the pretence of efficacy by using trials with flawed designs. The key point in assessing drug efficacy is Hard Unbiased Information.

    The same applies to understanding the Earth temperature evolution. If we’re interested in attribution/causality, we obviously take into account the fact that the troposphere has warmed while the stratosphere has cooled (indicating enhanced greenhouse effect), that the surface warming is associated with the vast and progressive increase in ocean heat content (i.e. not a “random walk” at all), and all of the other physical signatures that informs our understanding.

    No doubt one could also model the progression of a quantitative parameter of a person’s cancer (for example) as a “random walk” but you’d be unlikely to convince knowledgeable people that cancers are the products of the accumulation of random fluctuations of cell mass without causality.

  21. It seems to me that “average yearly surface temperature” is not just an abstract philosophical notion, but rather a physical property. Any physical property is dependent on other physical properties, and therefore cannot be analyzed in isolation.
    How does this “random walk” model of yours account for other related properties? Energy in the system comes to mind, as energy and temperature are very closely related. If the temperature increase is just a random walk, where did the required energy come from? Was it the energy going on a random walk, which was the actual basis of the temperature change? Is it possible for energy to randomly change?

  22. Matt,

    Thanks for the comments you posted over on my blog. I responded directly to those there, and if someone is interested in that exchange I trust they can find their way of there.

    But I wanted to expand on some of the what has been said and to say I think the issue of going beyond the original 131 is getting in the way of understanding the deficiencies in your model.

    Others have taken the original 131 year data set and applied a CADF test and found that it rejects a claim that the series is a random walk. I would think that would also make it clear that modeling using a random walk to see if the trend is “significant” would be ruled out.

    Let me take it from the other side. You have created 1000 simulations of the time series based on the random walk model. So you have a distribution of potential outcomes if in fact the random walk was a good model. So I ask the question is it reasonable to conclude that the actual time series arose from the simulated distribution.

    In very simplistic terms if I assumed that a variable was from a N(0,1) distribution(your simulation) than obtained an observation x(the actual time series) can I conclude that x came from that distribution. If x=1.2 then certainly it could have come from there, but if x=1000 it is very unlikely to have come from there.

    Your set of simulations provide a base distribution of outcomes. As an evaluator I want to focus on the variability of the time series. My actual measure is the difference between the maximum and the minimum temperature during the 131 year period.

    For the base data this is


    which yields a value of 107.

    Then I calculate this same value for each of the 1000 simulations. To do this I add the following code to your program.


    This way you can duplicate my numbers if you wish.

    Just prior to the for loop add the two lines:

    jump.max <- rep(0,trials)
    jump.min <- rep(0,trials)

    I use these variables to capture the spread of the temperature range for each simulation.

    Inside the for loop add the lines:

    running.sum <- cumsum(c(0,jumps))
    jump.max[i] <- max(running.sum)
    jump.min[i] max(theData$means)-min(theData$means))

    This will give a count of the number of cases in the simulation where the difference between the minimum and maximum temperature is greater than 107. The range in the original data. The result I got is 963.

    That is over 96% of the simulations had a greater variability than does the original time series, using this measure, is pretty conclusive to me that the original time series does not follow the distribution that results from your model.

    Again I am force to reject the hypothesis that a random walk models the temperature over the last 131 years.

    • I see I mixed up my code a bit. Must have cut somewhere at the wrong time.

      In the for loop the code is:

      running.sum <- cumsum(c(0,jumps))
      jump.max[i] <- max(running.sum)

      It is this last line that gives the value of 963.

      • Hi Larry,

        I think the comment parser at wordpress messes up some of the code.

        Did you try adding your code to the MaxEnt version, which bakes the covariance back into the data? I just did this and the number of trials which exceeded the observed range was about 80%, still perhaps uncomfortably large, but much less concerning than 96%. I used set.seed(345) and 1000 trials for this.

        Your point about the range of the trials being similar to the original is interesting though and worth more consideration. Because the original data dips so little before heading up, the total range isn’t much above the final result (0.76). Thus, anytime the simulation yields a more extreme result, it’s likely to also have a larger range, no?

        I must admit that the CADF details are new to me. At this point it’s clear that I shouldn’t have used the “random walk” term, since right away people start thinking AR and ARMA and Gaussian noise and all these other things and then get upset with the idea of applying this to climate, even over (relatively) short periods of time. Unfortunately, calling the method a “bootstrap of centered empirical temperature changes adjusted for observed data structures” probably wouldn’t have helped with comprehension.

        • Sorry should have begun by saying that I understood what you were doing with the code and implemented it. Here’s what I used:

          running.sum = cumsum(c(0,jumps))
          jump.range[i] = max(running.sum) – min(running.sum)

          inside the loop, with:

          jump.range <- rep(0,trials)

          outside the loop.

        • Matt, I did all my coding and testing on your first set of data before you played with the adjustments for the negative correlations.

          I had two reasons for that. First you had said the adjustment did not make any fundamental differences in your conclusions. And second I never saw the need for doing the adjustment due in the problems I saw with the data model. I also expected a negative correlation. You are actually calculation the correlation between x(1)-x(2) and x(2)-x(3). If the observations are random then the correlation will be negative.

          Yes the 80% number is not quite the concern as is the 96% figure I got. Keep in mind thought that if there is a trend in the data then the range in the actual data is going to be higher than in a stationary climate situation. The way you build your simulations there is no “real trend” except what a random walk tends to create. I view the 96% and the 80% to be biased downward due to what I see as a real trend in the actual data set.

  23. Frustration. I know I cut and pasted right the second time. One last try. The jump.min code is the same as the jump .max code except that is used the min function

    The last line of code is outside of the for loop. It subtracts jump.min from jump.max to get the range and computes the number of times that values is greater than 107.

    Hope that works….

  24. Firstly, for an R post nice code and graphs and everything.

    However, for a GW post – shame on you. The title is incredibly misleading, you redeem yourself by pointing out that YoY global averages are meaningless yet everything prior to that can be used by ‘GW deniers’ (read: big oil companies, governments not wanting to spend money etc. etc.) as ‘evidence’.

    You already state that you don’t have the required background to be claiming that there is only a weak basis for global warming, yet make that claim anyway – shame. How can you honestly expect to get a decent idea if what is going on using only an average temperature for the whole planet per year (“I’m only considering the yearly average temperature”), that average is almost meaningless on its own, not to mention the data from the early years is far less reliable.

    GW causes among other things more extremes in temperature, eg. colder winters, warmer summers and more intense storms, these things are lost when taking an average.

    The scientific evidence (that you admit to not even knowing) is overwhelming, it is not a bunch of people following the pack, that is not how science works. A real science article would reference the rising ocean levels, increasing CO2 levels and all the other evidence that shows global warming is real. Cherry picking one small peice of a very large puzzle is not real science.

    In addition your mention of Pascal’s wager is not apt here, in that wager what you lose is a life time of pleasure (a pretty big deal to most people). In this argument if we act according to GW being real and man made and it turns out to either not be real or not man made then what do we actually lose? Nothing, money gets spent making the planet much greener, less cancer causing pollution etc. To claim that would be a waste of time and money is incredibly short sighted, criminal almost.


      As the canard goes, “one foot in a lava stream, the other on a glacier; comfortable on average”.

    • The mention of Pascal’s Wager is valid, but lacks the broader context of the precautionary principle. This is the logic of the environmental movement. See my comment on this from Dec 5th.

      You’re argument is flawed in that it assumes “doing something” has no costs and no risks (no black swans). You’re assuming “doing something” can only have good outcomes. You’re position is the environment trumps everything else, and use the precautionary principle to smuggle in your desire for “doing something” (whatever that may be).

      • So, as you see a bad collision approaching, you would refuse to step on the brakes because you couldn’t be sure that wouldn”t somehow make it worse?

        We have (if you’re prepared to actually look at the evidence seriously) powerful evidence that our GHG emissions pose a terrible threat to the climate and oceans we depend on. And you think doing something about it is a bad idea because something or other just might go wrong? That’s some serious crazy in my book.

        • You’re assuming you know the climate as well as you know your automobile. You’re assuming you know what are the “brakes” and this will only stop or slow you down with no other effect. You’ve not even suggested what this “brake” would be. So it’s impossible to assess the risk or costs of “doing something.”

          • I made no such assumption– maybe you’re unfamiliar with the use of metaphor? There are plenty of obvious things to do, beginning with rapid, ongoing reductions in our use of fossil fuels. Energy efficiency (for vehicles, buildings, industry), development of non-fossil fuel based energy sources (already competitive in many applications) and quite possibly development of next-generation nuclear energy systems (thorium, for instance, is a safer fuel cycle, very difficult to divert to weapons production). All these things are well within current technology, and the easy ones are economically superior to current practice already (even ignoring the externalities associated with fossil fuels).

            Instead, we continue to insist on subsidizing fossil fuel development (a mature and very profitable industry with reserves large enough to commit us to 5 or 6 degrees C of warming, perhaps as early as the end of the century. Our agricultural systems won’t survive that in anything like their current form– and acidification of the oceans threatens them as a food source too.

            Impossible to assess? The only alternative to burning fossil fuels until they’re all gone is to jump down a dark well of mystery? Why not actually look at the plans and proposals out there? The ‘wedge’ strategy (combining conservation with multiple new energy sources over time); proposals to cut vehicle fuel consumption by 50% in the next 10-15 years (well within current technology); proposals for thorium fueled reactors; continued improvement in wind power systems… the notion that it’s all just unknowable is pure fear-mongering.

  25. Once again, you did not make any mistake by calling it a random walk. IT IS A RANDOM WALK (with correlated increments). Everything that is true generally about random walks (without assuming independence of increments) is true about your model.

    Also, the max-min statistic gets more information but still is not even close to being a sufficient statistic. Try the integrated square, or integrated absolute value. Just sum(x^2)/131 where x[i] = temp at time i.

    • Joshua,

      The sum of the squared deviations looks to me to be a better measure than the range statistic I used. It better captures what is going on for the entire time series.

  26. As noted, this analysis is of the land and sea surface records, which have been averaged together. This is the flaw of one foot in a lava stream, the other on a glacier where on average one is comfortable. But it’s actually worse than that.

    Averaging them together is like averaging the temperature of a balloon and a bowling ball. They have the same volume, but they’re not equal. The density of water is much greater than air. The temperature in my bathroom is not the average of my cold bath water and the air coming from my hair drier. Air requires much less energy to warm than water. The problem is assuming temperatures are equal measurements of energy.

    I have a distrust of the land-based temperature records. First, these records are effected by land management, urban development, tall buildings, etc. Knowing the “normal” temperature from these areas is unlikely. We also have a alternative state of the art system (USCRN) unaffected by such factors and it’s shown the commonly used land based data you’ve used is significantly effected with a warming bias.

    Secondly, these land based records have a history of being quietly adjusted (at least I haven’t found an explanation). These adjustments would create a warming trend even if the source were random noise. The past has been made cooler, and the more recent years have been made warmer.

    Simply compare the land and the sea surface record from 1880 to the present. Presumably they should be similar. Yet you’ll find the land temperatures from 1980 have been significantly warming that over the sea. That alone causes me to question the reliability of land based temperature data.

    I prefer to examine the sea surface temperatures in isolation. It’s not effected by factors that occur on land. Plus the air can more more freely, making it homogenous. It’s less prone to variability. More importantly the oceans are the planet’s main store of energy. The temperatures of the ocean waters and air are not equal, and cannot be average together. Unfortunately we don’t have long-term records of the ocean’s temperature, but we do have air temperatures at the sea’s surface (which is at least something better than our land temperature records).

    I’m not a statistician and I cannot apply your code to the available data. The data is also provided in more detail. You can get monthly temperatures of the sea surface going back to 1880, and even for specific long/lat grids.

    In my amateurish analysis of sea surface temperature I find a warming trend that began in 1909 rising 0.68 degrees C in 35 years. Then temps dropped dramatically, for some inexplicable reason, in just four years erasing nearly half of that warming. Presumably this was natural and not due to AGW since the level of CO2 was not yet significant.

    To be generous, if we say the next warming trend begins at the coldest point in 1948 and continues to the present. There’s been less warming and over a longer period of time (50 years to climb 0.6 degrees C). In other words, the warming that began in 1909 shows a naturally occurring phenomena that exceeds what is claimed by the AGW theory. The conclusion is current warming isn’t unusual, even with the significant increase of CO2 in recent years (1/3rd has been released since 1998).

    Hopefully someone with the statistical skills I lack can better analyze the data. It can be found here:



  27. Quick question:
    Since you used the observed YoY changes over the whole dataset to set your parameters, aren’t you just basically just re-creating the dataset with some stochasticity? You’re essentially saying that the observed trend is no different from a randomized version of the past 130 years.
    You should probably use the first 50 years or so to set your parameters and then see if the rest of the dataset deviates from that model.

  28. The analyses you present would make some sort of sense for a purely observational study of an unknown phenomenon. I’m an epidemiologist and I do this sort of analysis all the time.

    In my spare time I do a bit of demography, where the ‘physics’ of the system are quite well understood. People are born, they migrate, and they die. All my work on demography is based on this well understood model.

    This is, I gather, how people who know the subject analyse climatology data. Please understand that you can no more analyse climate data in splendid isolation, than I can anlyse demographic data, ignoring the processes that lead to my populations.





















Leave a comment