Header 1

Our future, our universe, and other weighty topics

Saturday, April 5, 2014

Double-Fudging Their Way to the BICEP2 “Breakthrough”

I like a double-fudged ice cream sundae, but I don't like double-fudged scientific studies, particularly when they claim to be of epic importance.

The BICEP2 study was released a few weeks ago to great fanfare. The press release for the study announced breathlessly that it was evidence for the theory of cosmic inflation, the theory that the universe underwent exponential expansion during a fraction of its first second.

But the study involved at least two big fudges – cases in which curves were squashed or stretched unnaturally and unreasonably for the sake of getting observations to fit in with the favored storyline that evidence had been found  for cosmic inflation. Before discussing each of the fudges, let me give a little background information.

The Difficulties of Looking for Primordial B-Mode Polarizations

The idea behind the BICEP2 study is to look for a particular type of radiation called b-mode polarization. Scientists predict that if a period of cosmic inflation had occurred in the universe's first second, it would have produced this type of radiation. But before the BICEP2 study many scientists commented on the extreme difficulty of finding evidence for cosmic inflation through such a process. The main problem is that quite a few other astronomical phenomena can produce this same type of b-mode polarization radiation. Among these other phenomena are: various types of dust, synchrotron radiation and gravitational lensing. 

The problem is illustrated by the graph below. The red line shows the b-mode polarization predicted to occur from synchrotron radiation. The blue lines shows the b-mode polarization predicted to occur from dust. The dotted green line shows the b-mode polarization predicted to occur from gravitational lensing. The solid green line shows the the b-mode polarization predicted to occur from cosmic inflation, using a version of that theory compatible from the most recent findings from the Planck satellite.

From the scientific paper here

The problem is that any lines that are higher-up in this graph are stronger signals that will drown out any signals that are lower in the graph (just as a 100-decibel sound of a passing motorcycle will completely drown out the 30-decibel sound of a child whispering). So if we use the curves in the chart above, there would seem to be basically zero chance of ever being able to confirm a theory of cosmic inflation by measuring b-mode polarizations (the technique used by the BICEP2 study).

To try to overcome such problems, the BICEP2 study resorted to some fudges I will now list.

Outrageous Fudge Number 1: Shrinking the Gravitational Lensing Model

The BICEP2 study has a graph showing a projection of the expected amount of b-mode polarization from gravitational lensing. But the projection is a shrunken, low-ball projection. It is nowhere near as high as the projections made by some previous scientists.

Here is the BICEP2 graph in which they project gravitational lensing (the solid red line):

You have to look closely at the little lines on this logarithmic graph to figure out two things: the assumption being made about where gravitational lensing starts, and the assumption being made about where it peaks. The assumptions being made by the BICEP2 study are these:

Starting point for gravitational lensing: 50 multipole ( l )
Peak of gravitational lensing: .05 (close to 10-1).

The problem is that this is a shrunken estimate, an extreme low-ball projection. The first graph in this post (in which l is the multipole) gives a very different, much-larger projection:

Starting point for gravitational lensing: 2 multipole
Peak of gravitational lensing: .2

In fact, most of the estimates that you will find (made prior to the BICEP2 study) match this much larger estimate for gravitational lensing. Indeed, the most recent POLARBEAR observations support these larger estimates, by showing the peak of the gravitational lensing at a much higher point than the peak in the BICEP2 graph.

Why is this important? If the larger estimate of gravitational lensing is correct, then all of the BICEP2 observations can be explained by assuming gravitational lensing (not cosmic inflation) as the source of the radiation. For example, if the larger projection of gravitational lensing is correct, then we have a model of gravitational lensing similar to the green line below, which can explain all of the BICEP2 observations without requiring any cosmic inflation in the universe's first second.


To force their observations to fit a theory of cosmic inflation, the BICEP2 study chose to present a shrunken, low-ball estimate of gravitational lensing.

Outrageous Fudge Number 2: Shrinking the Dust Projection

When they estimated the amount of b-mode polarization produced by cosmic dust, the BICEP2 team almost admitted that they didn't have what they needed to make an accurate projection:

The main uncertainty in foreground modeling is currently the lack of a polarized dust map. (This will be alleviated soon by the next Planck data release.) In the meantime we have therefore investigated a number of existing models and have formulated two new ones.

The team then present a graph showing some models they selected, models that minimize the amount of dust, and suggest that dust is no big problem when trying to measure signals from cosmic inflation. The low-ball models selected are inconsistent with some previous estimates, which estimate that dust should be blocking all or most of any b-mode polarization produced by cosmic inflation.

See, for example, this scientific paper, which on page 2 predicts a level of dust polarization many times greater than the amount projected by the BICEP2 study (as does this graph from a scientific conference). The relevant graph is shown at the beginning of this blog post.
It was almost rather like this:

Previous scientists: How on earth can we find a signal from cosmic inflation with all this cosmic dust all over the places we're looking for the signal, dust that blocks what we're looking for?
BICEP2 scientists: Dust? What dust?

Of course, the BICEP2 study needed to shrink and low-ball the dust projections, to clear the field for their triumphant announcement of evidence for cosmic inflation, and to try to rule out dust as the source of their observations. This was another biased case of artificially stomping on a data curve to get observations to fit a favored explanation of the data.

In an attempt to rule out dust as the source of their observations, and bolster their case for cosmic inflation (in the universe's first second) as the source of their observations, the BICEP2 team ran some simulations (using lots of subjective, hand-picked inputs) that they say show that their observations have characteristics “atypical” of dust. That is very lame and unconvincing reasoning – rather like arguing that a particular light seen in the sky is an alien spaceship because it has characteristics “atypical” of an airplane. The graph of their simulations (Figure 8 in the study) still shows a perfectly decent chance that dust or synchrotron radiation is the source of their observations, not cosmic inflation.

The End Result: A Half Fit

We might expect with these two examples of curve fiddling that the end results would match the favored model exactly. But no: even with these heroic efforts, the BICEP2 graph below only shows 5 out of 9 data points matching the favored model, with several of the data points far off of the model. 


Can We Trust the Claimed Data Points?

When a study is based on simple data observations, you can trust the observer to record the observations correctly, unless you think he might be careless or prone to fraud. For example, if a scientist measures the temperature on a particular day, you pretty much have to trust him, unless you think he might be faking it. But in a case such as the BICEP2 study we have a very different situation. The scientists took raw data, and subjected it to an extremely complicated process of transformations, summaries, and modeling. The process was almost like the process shown in the visual below:

Source: wikiuniversity, Howard Community College

Can we be confident that the BICEP2 team got this extremely complicated process right, and that the data points shown in their final graph are correct? No, we cannot be. This is because it is rather clear from these examples I have shown that the BICEP2 team had a strong experimental bias. Evidently they wanted very much to make their observations match a storyline that the observations came from primordial cosmic inflation. Given this very strong partiality, there are any number of ways in which things could have gone wrong because of experimental bias. At any number of points in the incredibly complicated data transformation process, the scientists may have made decisions influenced by their desire to end up with results favoring a theory of cosmic inflation, decisions that more objective and impartial scientists would not have made.

We need studies like the BICEP2 study to be performed by objective, non-biased scientists without any favored agenda (scientists with an attitude of “let the chips fall where they may”), rather than scientists who seem determined to hammer square pegs into round holes, in order to fit some desired preconceived storyline.

Postscript: See this link for a National Geographic story on how the BICEP2 results may be due to dust, not cosmic inflation. I'm not accusing any on the BICEP2 team of deliberately misleading anyone. I merely think that their desire to have an inflation-related result (good for their own careers) has influenced their paper, leading to some presentation and interpretation decisions that might not have been made by a more impartial set of writers. 

Post-postscript: I didn't originally mention the pathetic way the BICEP2 paper handled the issue of synchrotron radiation.  Synchrotron radiation is a widespread phenomenon that produces the same b-mode polarization observed by BICEP2.  This type of radiation can be produced by many types of high-energy violent events inside and outside of our galaxy. Rather than making any substantial attempt to show that synchrotron radiation is not the source of their observations (which would require many pages), the BICEP2 paper (in sections 9.2 and 9.3) has the skimpiest treatment of the topic, using only 6 sentences to address it. This discussion calls section 9 of their paper "the most ridiculous handwave of all time in the whole history of physics."

Post-post-postscript:  Today a physicist has on his blog a post that says the BICEP2 team made some big error in their dust projection (an error along the same lines as I insinuate in this post). He says, "However, at this point, there seems to be no statistically significant evidence for the primordial B-modes of inflationary origin in the CMB [cosmic microwave background]."

Yet another postscript: see this post for a discussion of a talk at Princeton University in which a scientist gives a presentation that gives a devastating blow to the inflated claims of the BICEP2 study. The scientist gives projections of dust and gravitational lensing which show how such common phenomena (not from the Big Bang or cosmic inflation) can explain the BICEP2 observations.