

Oct 25 2007 Published by under Bad Statistics

causative risk factor for breast cancer, which he posted about
this morning
. When the person who motivated me to start what's turned out to be a shockingly
successful blog asks for something, how could I possibly say no? Especially when it's such a great example
of the misuse of mathematics for political purposes?

The paper is "The Breast Cancer Epidemic: Modeling and Forecasts Based on Abortion and Other Risk
Factors", by Patrick S. Carroll, published in the Journal of American Physicians and Surgeons (JPANDS).
Before getting to the meat of the paper, there's a couple of preliminary things to say about it.

In an ideal world, every time you read a paper, you'd study every bit of it in great, absorbing detail. But in the real world, you can't do that. There are too many papers; if you tried to give every paper a full and carefully detailed reading, even if you never stopped to eat and sleep,

what kind of paper it is.

If a paper is just piling on more evidence, you look at the data that it presents - and don't pay a lot attention to much else, because they're rehashing what's already been said. The only really interesting thing in a paper like that is the data. So you focus your attention on the data, how it

gathered and analyzed, to see if the new information that they're adding is valid.

and which haven't been carefully looked at by many different people yet, so they require the most
careful attention and analysis. This paper is a member of that last class: it's claiming to find
a statistical link which many careful studies have not found. So it's in line for a very
careful reading.

So what's the source? Well, it's published in JPANDS. JPANDS in a terrible journal. In fact, the first post on Good Math/Bad Math was a critique of a JPANDS paper that used some of the worst statistics that I've ever seen published. That's bad - the paper is appearing in a very low-credibility journal with a history of not carefully reviewing statistical analysis. That's certainly not enough
to justify ignoring the paper - but the quality of the journal is a valid consideration. A paper about
to mind the question "Why publish this work in a non-MEDLINE indexed, low quality journal?" Like I said, it's not enough to ignore the paper, but it does raise red flags right away: this is a paper where you're going to have to give the data and its analysis a very careful read.

breast cancer in those same populations. That's a sort-of strange approach. At best, that approach can
show a statistical correlation, but it's going to be a weak one - because it doesn't maintain any link
a correlative study like that when you can't associate risk factors and incidences with specific
individuals. The author does address this point: he says that it's difficult for epidemiologists to
the fact remains that it's going to be much harder to establish a causal link rather than a correlative link using this methodology.

race. (Orac has more to say about that.) But what's also important to notice is that these factors are
number who've had children. The women who have a large number of children are much more likely to have
and higher rates of breastfeeding. This list looks fishy.

and using information from public health services, does a linear regression of risk factor versus cancer incidence over time. If the linear regression doesn't produce a strong positive correlation, he throws it away. The fact that this means that he's asserting that well-known and well-supported
correlations should be discarded as invalid isn't even mentioned. But what's worse is, it's clearly quite

On page two, he shows a graph of the data for "mean age of first live birth" plotted against breast
ridiculous. It is a completely unacceptable and invalid use of statistics. Anyone who's
even taken a college freshman course in stats should know that that is absolutely ridiculous. It's very
deliberately ignoring independence from other variables, in obviously foolish ways. I just don't even
know how to mock this, because it's so off-the-wall ridiculous.

There's another obvious problem with the whole methodology, which pales in comparison to
the dreadful way that they selected data. But I'll mention it anyway. Linear regression and correlation
coefficient measures how well a linear relationship matches the data. It doesn't test for
anything else. But there are numerous correlative and/or causal relationships that don't show a
various diseases, there's often an initial decrease in risk, which bottoms out and is followed by a large increase in risk. There are often threshold effects, where something doesn't start
to have an impact until beyond a minimum threshold. And so on. There's a lot more to things that
for that, and makes no attempt to justify it. It's just presented as if it's beyond question.

as being irrelevant factors that don't need to be included in the model. That

Yi = a + b1x1i + b2x2i + ei

ranging from -0.0135 to _0.0041. In other words, according to their measure, fertility - the rate
of live birth - is not a significant factor in breast cancer rates compared to

the known data, you throw the model right out the window, because it's worthless. The notion that
childbirth - when we know that having children causes a dramatic decrease in the risk of
breast cancer - is unquestionably wrong. If it were true, what it would mean is that the
of breast cancer rates - you'd have a small sub-population causing an inordinately huge
portion of the breast cancer rates. We know that things like that are easily visible: that's how
we discovered the so-called "breast cancer genes" - a small group of women were

So we've got a model which doesn't fit reality. What a real scientist does when this happens
is to say "Damn, I was wrong. Back to the ol' drawing board", and try to find a new model that
does fit with reality.

wrong, by saying "The coefficient of fertility is rather small, with the 95% confidence interval straddling zero. Some improvement in breastfeeding may be offsetting fertility decline." No, sorry, you can't say "My mathematical model has absolutely no relation with reality, but that's probably because one of the factors that I excluded is probably important, and so now I'm going to go on pretending that the model works."


crap that's just attempting to justify a political position using incredibly sloppy math; math so
bad that a college freshman should be able to see what's wrong with it. But for the "reviewers" at JPANDS, apparently a college freshman level of knowledge of statistics isn't necessary for reviewing
a paper on statistical epidemiology.

No responses yet

  • spudbeach says:

    I guess this paper just gives another example of that.

  • gg says:

    MarkCC wrote: "So what's the source? Well, it's published in JPANDS. JPANDS in a terrible journal."

  • bsci says:

    The JPANDS mission statement includes:
    "...a commitment to publishing scholarly articles in defense of the practice of private medicine, the pursuit of integrity in medical research...Political correctness, dogmatism and orthodoxy will be challenged with logical reasoning, valid data and the scientific method."
    They same link also says they have a new focus and name (from Medical Sentinal) and "We have eliminated the news capsules that tend to be old news by the time a quarterly journal is published, and reduced the political commentary in favor of more articles of a scientific nature, particularly if they are relevant to contemporary policy debates."
    They do have double-blind peer review so just because you criticize them here doesn't rule out your chances of getting published. 🙂

  • Jeb, FCD says:

    I had to stop reading here:

    When a woman is nulliparous, an induced abortion has a greater
    carcinogenic effect because it leaves breast cells in a state of
    interrupted hormonal development in which they are more

    Seems to me one round of birth control pills after an abortion should, given the logic in the quote, "reset" the breast cells to their quiescent state.

  • Mark C. Chu-Carroll says:

    The problem with the paper isn't that they're examining that hypothesis. The problem is *how* they examine it.
    There've been other studies that examine that question, and their results have been, pretty uniformly, that there's no statistically significant difference.

  • other bill says:


  • other bill says:


  • John Johnson says:


  • PalMD says:

    "I just don't even know how to mock this"
    I'll help you..."it's not even wrong".

  • Jim says:

    Thank you for posting this information. Unfortunately, the paper will be quoted by those groups who want to show "proof" that you should "be fruitful and multiply" and "not have abortions". I can respect moral arguments for not having abortions, but the paper doesn't provide scientific support for it.

  • Rev.Enki says:

    Given the proposed model (leaving the breast cells in a state of "interrupted hormonal development) I can't see what the difference would be between an abortion and a miscarriage, except that the latter would generally be far more common, at least during early pregnancy. That wouldn't necessarily make his argument false (though it's pretty obvious it is, for other reasons) but it would suggest that focusing on the politically hot abortion issue would be a bit suspect when the real effect would be considerably more general than that. Why isn't he out there terrorizing women who just had miscarriages instead, or as well?
    Now *that* is a rhetorical question.

  • Glen says:

    Hey Mark,
    "It's very deliberately ignoring independence from other variables"
    Did you mean a different word than "independence" there?
    Analysis problems aside, when you get a result that flies in the face of accepted understanding, you have to say "well, either I'm completely wrong, or I've really found something worth investigating", and the next place to go is to try and figure out where you went wrong (since that's much more likely).
    Only when you can't find any alternative explanation for what you find do you start looking at what has gone before and try to figure out why it might be wrong.
    In either case (you results don't fit because you're wrong or they don't fit because you're right and everyone else is wrong), you have a lot of further work to do before you can publish.
    The first response to a weird result should not be "publish!".

  • Doug says:

    I'd say a high schooler shouldn't make this mistake. He wants to imply something about causality from an average? Averages come as mathematical information, not physical causes as basically everyone knows.
    [In other words, according to their measure, fertility - the rate of live birth - is not a significant factor in breast cancer rates compared to abortion.]
    Never mind that live birth causes massive chemical changes in women's bodies. Especially, in the few days/week after children leave the womb.
    [The notion that abortion as a risk factor for breast cancer completely dwarfs the reduction in risk after childbirth - when we know that having children causes a dramatic decrease in the risk of breast cancer - is unquestionably wrong. ]
    Strange, I pre-thought it would work the other way around. Nonetheless, I did at least think that having children would change the risk in breast cancer... as obviously having children yields massive chemical and biological and phyiscal changes. How can anyone think otherwise?

  • I just don't even know how to mock this, because it's so off-the-wall ridiculous.
    The top three states in home ownership are West Virginia, Mississippi, and Alabama. Clearly, poverty is a risk factor for home ownership.
    Not absurd enough?
    Okay. To return to topic of the paper being discussed, there's also a negative correlation between UFO sightings and abortion. Clearly, the saucer people are pro-life.

  • other bill says:

    Some passing technical notes: (statistical) independence is not required for regression analysis. That's part of what the inv(X`X)X` term adjusts for in a regression formula. The coefficients are partial coefficients, adjusted for the other regressors. It also means his marginal tests are irrelevant to the partial contribution of a regressor (the coefficient in the presence of all other regressors). What he should have looked at was the partial coefficients for his seven predictors. Of course with a sample size of 15, very little is going to be "significant". I'd guess that he originally tried that, went "oops" and started hunting for something publishable.
    There is nothing unusual or wrong about looking at cohort data in a scatter plot. That's how you pick up non-linear relationships. He's trying to cram three plots into one in the paper (X vs. Y, X vs Cohort Year, and Y vs Cohort Year). A scatterplot matrix would have been more informative here, as well as highlighting the shifting sample sizes.

  • SLC says:

    As I stated in a comment on Dr. Oracs' blog, the notion that the anti-abortionists have any interest in scientific integrity is piffle. Their only interest is in making abortion illegal and lying to achieve that objective is considered perfectly acceptable. It's called lying for Joshua of Nazareth.

    • Philip Larson says:

      You might be taken seriously if you weren't committed to propaganda yourself.

      At a minimum, don't refer to people using designations they would not accept.

  • MoBo says:

    Before we talk about who's being biased and selective, I think it's worth actually looking into what breast cancer is at the molecular level.
    That's an easy-to-understand, recent paper by one of the giants in breast cancer research - Dr. Russo et al., and a fact sheet by Breast Cancer Prevention Institute:

  • Monado, FCD says:

    Thanks for publishing this. I don't have even second-year statistics so every little bit helps. Conclusion: the author and the journal have an axe to grind.
    I think you mean "JPANDS is a terrible journal."

Leave a Reply