A few words on Science

As a (somewhat) frequent reader of the Sequences on lesswrong.com, I was always wondering about the seemingly-abusive usage of the word “Bayesian”. Being curious as a proper wannabe-scientist has to be, I tried to investigate the issue. And oh dear, why did I even do this.

After half an hour worth of research, I have found out that there are two competing views in scientific methodology that can be traced back to two different views on inference models. Long story short, if you compute P( Data | Model ), you are a frequentist. If you compute P( Model | Data ), you may proudly label yourself as a Bayesian.

Okay, but seriously, can it be a serious scientific issue? Apparently, yes. (Google: 265K hits on “frequentist versus bayesian”) Most Google links lead to blog posts in which authors argue for their favorite toy model by giving examples in different kinds of weirdness. Frequentists give obvious toy examples, Bayesians reply with taking the examples and showing that with advanced probability-fu, one can get better results (if the example behaves well), then a debate on the philosophical implications of iteratees inference begins…

Don’t get me wrong: I somewhat like the Bayesian point of view, since computing the probability/likelihood of the model seems to be more straightforward in terms of scientific reasoning, which is about testing models, not data. But in the end, you still have to justify your statistical model used in inference (contrary to: “yo dawg, I heard you like probability distributions, so I took the Beta distribution: and put it into your model so you can compute more funny numbers!”). This is especially relevant since there are MANY inference models around (all of which are pretty much awful in terms of computational tractability, but that’s a different issue).