Thursday, May 9, 2013

Causality vs. correlation - statistical reasoning is not enough - NY Times Interview with Dave Ferrucci

Dave Ferrucci, who was until several months ago an IBM Fellow  and was known as the father of Watson, was interviewed by the NY Times in his new working place at Bridgewater Associates.

In the interview Ferrruci somewhat continues the line of thought of Noam Chomsky,  saying that AI has concentrated around statistical reasoning based on correlations, but the drawback is that one cannot understand why the prediction made by the statistical reasoning is correct.  While Chomsky bluntly stated that statistical reasoning does not create a solid model of the universe, Ferruci claims that a complementary approach is required -  understanding causality.    This is a rather old issue, in symbolic logic, there is a distinction between "material implication"  which states that  IF A is true then B is true, and the meaning is that always when A is true then B is also true, which makes a sentence like  "If the week has seven days than  the capital city of France is Paris" - a valid statement in logic.    Entailment, on the other hand, said that "A ENTAILS B" if it is necessary and relevant, in other word, there is a causality among them.  Thus, Ferruci concentrates now on building causality models to model the world economy.      I concur with the assertion that understanding causalities give better abilities of reasoning and prediction.   As David Luckham already noted, causality among events is one of the major abstraction of event processing models.   Here is a rather old discussion about causality of events.  

1 comment:

Rainer von Ammon said...

I love your link and to remember these times when we tried to start the CEP community and EP-TS.

If we use CEP or U-CEP or edBPM for not relatively simple applications, the problem of the causality would be unsolvable or better "unprovable" what we know from Kurt Gödel's Incompleteness Theorems

So, it seems to me that theories about causalities of events are a nice occupation for philosophers, theoretical physicists and mathematicans in order to explain the causality of events in a 10 or 11 dimensional universe or multiverse - so far, until we have a new theory beyond all the String Theories.

In the meantime, we should simply use CEP or U-CEP or edBPM for _relatively simple applications_ and _domain-specifically_, in order to restrict the complexity of world knowledge and the causalities of events (types), which we need to model such an application domain. Afterwards we could try to abstract from CEP to Ubiquitous CEP, according to David Luckham's forecast I like so much

I remember our nice try of the "Working Group Business Value" in the EP-TS and what we have submitted to the Cambridge DEBS 2010. There we wanted to discuss David's comment: "What can we measure? – Start with the right granularity"

If we try more, we would probably always fail and get such judgements from - perhaps not-expert - reviewers like "... The concept itself is largely over-speculative and the key issue is whether the planned objectives would lead to meaningful research." Sorry when I repeat this great sentence from time to time.

But anyway, I also like such discussions about the causality of the Big Bang as the - perhaps not first - event and the Big Crunch as the - perhaps not last - event