The failure of science museums (and some radio shows)

Three years ago, I posted my ambivalence about science museums. I recently accompanied my seven year old on a field trip to one and now I am of the firm belief that they have very little utility for educating children about science.  Current science museums strive to be as interactive as possible. Many of the exhibits perform some simple experiment where the user participates by pulling or pushing some buttons or knobs. However, unless you are patient enough to read the information placard, the exhibits are more like toys or video games. I’m sure there are seven year olds out there that do read all the information and are enriched by the exhibits but not the ones I chaperoned. Their level of engagement with each exhibit did vary but if any scientific information was transferred, I would be shocked. The juxtaposition of interactivity and passive reading is just a bad idea. If you want to be interactive then all the information must be presented interactively.  No child and probably most adults won’t bother to read a sign before randomly hitting some buttons to see what happens. It may work better if there were some sort of gate that prevented access to the exhibit until the introductory information was read. I don’t know what the optimal format would be.

While I’m in the ranting mood, I’m also going to criticize my  favourite childhood radio show Quirks and Quarks on CBC. The problem I have with the show these days is that it basically only covers astronomy, dinosaurs, and animal behavior. Occasionally, it will also cover high energy physics or climate change. It pays scant attention to the rest of biology, physics, chemistry, computer science, or mathematics. The show does a very poor job of giving the public an idea of what most scientists really do and what constitutes scientific breakthroughs. I think it is more important now than ever that science shows try to educate the public on how the scientific method really works, to get across how difficult it can be to come up with experiments to test hypotheses and how long it takes to get from breakthroughs in the lab to applications. They should also better convey the sense of how it is impossible to predict what will become useful in the future and how lots and lots of failure is a prerequisite for progress. I hope Quirks and Quarks will become more serious because it’s migrating its way to the bottom of my podcast stack.


Wiener on robots

An essay by Norbert Wiener written in 1949 intended for the New York Times was recently uncovered.  He pretty much had it right 64 years ago. Below is the rather serious last section. Earlier in the piece, we find out that programming was  called “taping” at that time.

New York Times:

The Genie and the Bottle







Most of neuroscience is wrong

John Ioannidis has a recent paper in Nature Reviews Neuroscience arguing that many results in neuroscience are wrong. The argument follows his previous papers of why most published results are wrong (see here and here) but emphasizes the abundance of studies with small sample sizes in neuroscience. This both reduces the chances of finding true positives and increases the chances of obtaining false positives. Under powered studies are also susceptible to what is called the “winner’s curse” where the effect sizes of true positives are artificially amplified. My take is that any phenomenon with a small effect should be treated with caution even if it is real. If you really wanted to find what causes a given disease then you probably want to find something that is associated with all cases, not just in a small percentage of them.

Bayesian model comparison Part 2

In a previous post, I summarized the Bayesian approach to model comparison, which requires the calculation of the Bayes factor between two models. Here I will show one computational approach that I use called thermodynamic integration borrowed from molecular dynamics. Recall, that we need to compute the model likelihood function

P(D|M)=\int P((D|M,\theta)P(\theta|M) d\theta     (1)

for each model where P(D|M,\theta) is just the parameter dependent likelihood function we used to find the posterior probabilities for the parameters of the model.

The integration over the parameters can be accomplished using the Markov Chain Monte Carlo, which I summarized previously here. We will start by defining the partition function

Z(\beta) = \int P(D|M,\theta)^\beta P(\theta| M) d\theta    (2)

where \beta is an inverse temperature. The derivative of the log of the partition function gives

\frac{d}{d\beta}\ln Z(\beta)=\frac{\int d\theta \ln[P(D |\theta,M)] P(D | \theta, M)^\beta P(\theta|M)}{\int d\theta \ P(D | \theta, M)^\beta P(\theta | M)}    (3)

which is equal to the ensemble average of \ln P(D|\theta,M). However, if we assume that the MCMC has reached stationarity then we can replace the ensemble average with a time average \frac{1}{T}\sum_{i=1}^T \ln P(D|\theta, M).  Integrating (3) over \beta from 0 to 1 gives

\ln Z(1) = \ln Z(0) + \int \langle \ln P(D|M,\theta)\rangle d\beta

From (1) and (2), we see that  Z(1)=P(D|M), which is what we want to compute  and Z(0)=\int P(\theta|M) d\theta=1.

Hence, to perform Bayesian model comparison, we simply run the MCMC for each model at different temperatures (i.e. use P(D|M,\theta)^\beta as the likelihood in the standard MCMC) and then integrate the log likelihoods Z(1) over \beta at the end. For a Gaussian likelihood function, changing temperature is equivalent to changing the data “error”. The higher the temperature the larger the presumed error. In practice, I usually run at seven to ten different values of \beta and use a simple trapezoidal rule to integrate over \beta.  I can even do parameter inference and model comparison in the same MCMC run.

Erratum, 2013-5-2013,  I just fixed an error in the final formula