when a frequentist algorithm works, there’s supposed to be a Bayesian explanation of why it works. I have said this
before many times but it seems to be a “resistant concept” which simply cannot sink in for many people.
Perhaps the reason this is not sinking in for many people is because it is not true.
Bayes assumes you can write down your prior, your likelihood and your posterior. That is what we need to get Bayes theorem to work. If you are working with a statistical model where this is not possible*, you cannot really use the standard Bayesian story, yet there still exist ways of attacking the problem.
(*) Of course, “not possible in principle” is different from “we don’t know how to yet.” In either case, I am not really sure what the point of an official Bayesian epistemology explanation would be.
This idea that there is a standard Bayesian explanation for All The Things seems very strange to me. Andrew Gelman has a post on his blog about how to define “identifiability” if you are a Bayesian:
This is apparently a tricky (or not useful) concept to define within that framework. Which is a little weird, because it is both a very useful concept, and a very clear one to me.
Gelman is a pretty prominent Bayesian. Either he is confused, or I am confused, or his view on the stuff causal folks like me work on is so alien that it is not illuminating. The issue to me seems to me to be cultural differences between frameworks.
I wouldn’t say “no Bayesian explanation,” but perhaps “a Bayesian explanation is unknown to me, nor do I see how this explanation would illuminate anything.” But yes, I gave an example elsewhere in this thread. The FCI algorithm for learning graph structure in the non-parametric setting with continuous valued variables, where the correct underlying model has the following independence structure:
A is independent of B and C is independent of D (and nothing else is true).
Since I (and to my knowledge everyone else) do not know how to write the likelihood for this model, I don’t know how to set up the standard Bayesian story here.
Perhaps the reason this is not sinking in for many people is because it is not true.
Bayes assumes you can write down your prior, your likelihood and your posterior. That is what we need to get Bayes theorem to work. If you are working with a statistical model where this is not possible*, you cannot really use the standard Bayesian story, yet there still exist ways of attacking the problem.
(*) Of course, “not possible in principle” is different from “we don’t know how to yet.” In either case, I am not really sure what the point of an official Bayesian epistemology explanation would be.
This idea that there is a standard Bayesian explanation for All The Things seems very strange to me. Andrew Gelman has a post on his blog about how to define “identifiability” if you are a Bayesian:
(http://andrewgelman.com/2014/02/12/think-identifiability-bayesian-inference/)
This is apparently a tricky (or not useful) concept to define within that framework. Which is a little weird, because it is both a very useful concept, and a very clear one to me.
Gelman is a pretty prominent Bayesian. Either he is confused, or I am confused, or his view on the stuff causal folks like me work on is so alien that it is not illuminating. The issue to me seems to me to be cultural differences between frameworks.
Do you have a handy example of a frequentist algorithm that works, for which there is no Bayesian explanation?
I wouldn’t say “no Bayesian explanation,” but perhaps “a Bayesian explanation is unknown to me, nor do I see how this explanation would illuminate anything.” But yes, I gave an example elsewhere in this thread. The FCI algorithm for learning graph structure in the non-parametric setting with continuous valued variables, where the correct underlying model has the following independence structure:
A is independent of B and C is independent of D (and nothing else is true).
Since I (and to my knowledge everyone else) do not know how to write the likelihood for this model, I don’t know how to set up the standard Bayesian story here.