Thank you for the chronological suggestion; I think that will help it to make sense.
I would comment that the consistency theory for posteriors only guarantees convergence to “the truth” WHEN the likelihood terms are capable of making distinctions between the different possible models. If data is unavailable that distinguishes them… no convergence, just stabilization to potentially different subjective posteriors supported on the indistinguishable set. Since all we care about is making decisions, this is usually a distinction with no difference, except if there is a future event to come where they will make different predictions.
Accordingly my “God hypothesis” posterior probability has not updated in a very long time. I am, for example, not aware of what evidence there exists AT ALL with which to update this prior, but I suppose this is just because my posterior mass, conditional on the God hypothesis, came to rest, long ago, on a very non-falsifiable, (arguably science-compatible) version of belief in God. I.e. belief in a God who “operates all in accord with His will”, but who also does this exactly through the naturalistic mechanisms that we see operating all the time and not through miraculous interventions. Or, to be more precise, another indistinguishable possibility is that God is perpetrating miraculous interventions continuously, and just consistently wills to do them in exactly the way that the true but unknown “laws of nature” predict. It remains possible that God could revoke this policy at some future time: this does not require much Kolmogorov complexity. I can rule out the ordinary miraculous part of the mass (“why did it happen? Because God wanted it that special way in this special case”) because it’s not predictive. Accordingly my posterior does not give me “the mathematical right” to assert God’s non-existence.
Indeed, unless posterior mass is actually 0, I don’t think this is a matter of “mathematical rights” anyway. We’re making decisions under uncertainty, and we have some model for what the loss is if we make a mistake. You have, for example, the right to, say, not believe in God but not want to talk about it anymore, and otherwise act in accord with your belief, but I don’t see how it gives you the right to assert God’s non-existence. Sometimes we assign mass 0 to some possibilities, effectively, because we are not perfectly rational Bayesians, either we’d never thought of those possibilities, or it was computationally expedient to ignore them. In either of these cases we have no such “right”.
If I had the misfortune to have my prior “initialized” by my environment somewhere with a “vengeful God” hypothesis, who waits in Secret to test if you are really “one of the chosen” faithful, I’d be in a more difficult predicament. Hopefully, if I’d been born in such a sect, they’d have had some other clearly falsifiable beliefs that come along with the dogmatic ride, so that I’d have a way to identify the error. The worrisome part here, I notice, is that my thinking is rather driven by how it was initialized.
Well, you are certainly a lot better at Bayesian Statistics than I am. But if I am to base my “physics-defying, benevolent, superintelligent sky wizard” hypothesis on evidence such as badly written holy books that look spuriously like hodge-podge culture dumps, the general religious disagreement, the continued non-answering of prayers, failure to divine simple mathematical or physical truths, and how science is significantly more productive, well… Every time a prayer goes unanswered I can theoretically update on it for a lower credence. Every conflicting holy scripture allegedly bequeathed by the divine, claiming to be the one truth is a major hit.
Meanwhile the hypothesis that what most people call religious experiences is located in the temporal lobe gains credence from every cognitive neuro-scientific study of it.
If you haven’t updated your credence of immortal physics defying super-intelligent friendly sky wizards in a while, I am going to tell you to look harder. The evidence is there. From what you state it seems that you might suffer from motivated cognition and belief hysteresis, so take this opportunity to get a feel on those so you might recognize them in the future.
Now, for the hypothesis of “immortal physics-defying wizard doing whatever makes the world seem normal” is strictly dominated by the hypothesis of “whatever makes the world seem normal”. That is Minimum Message Length/Solomonoff Induction right there.
But given that you already have Bayes down to a science, I don’t see why you should become a fine Lesswrongian in just a short time. Welcome to lesswrong.
It’s a straw dummy to say that I haven’t updated my priors about insert ridiculous thing here. Those are not the things I described having residual posterior mass on; my “sky wizard” is currently content to play by very different rules. As for the epi-phenomena of how a society embraces religion, or does not, I agree, this does suggest that it would be foolish to base one’s beliefs SOLELY on the “evidence” of the crowd as you grow up.
Also, incidentally, MML/Kolmogorov complexity and Solomonoff are not the same here. The former do not give a posterior on models, they select one model. The later could and would easily put posterior mass on multiple programs that were observationally the same up to the amount of data we have currently observed.
Consider the sequence of numbers: 1,2,3.
MML would (for most Turing machines anyway) select the program: next=old+1.
Solomonoff would have posterior mass on many other sequences (e.g. do a query on http://oeis.org/). It’d put most of it’s mass on that program, but leave nonzero mass on other programs, like one that computes this sequence, say: http://oeis.org/A006530, and if the next numbers reveals the sequence:
1, 2, 3, 2, 5, 3, 7, 2, 3, 5
Then you can bet that’s where a lot of the mass will shift.
… And the really subtle bit, that I don’t want you to miss, is that it doesn’t all shift there, it’s still not a point posterior. There’s still mass on:
That mass won’t go away for a LONG time (that is if the data keeps coming from http://oeis.org/A006530 for at least Googol/2 steps.
And there are many possibilities like this out there. Sure, they have less mass individually, but there ARE many of them. And some don’t even have appreciably less prior mass (under a Solomonoff prior and a “reasonable” Turing machine). Computability is a whole ’nother issue.
Hee! I sort of like the idea of immortal physics defying super-intelligent friendly sky wizards, actually. ”Hamlet, in love with the old man’s daughter, the old… man… thinks.”
Accordingly my “God hypothesis” posterior probability has not updated in a very long time. I am, for example, not aware of what evidence there exists AT ALL with which to update this prior, but I suppose this is just because my posterior mass, conditional on the God hypothesis, came to rest, long ago, on a very non-falsifiable, (arguably science-compatible) version of belief in God. I.e. belief in a God who “operates all in accord with His will”, but who also does this exactly through the naturalistic mechanisms that we see operating all the time and not through miraculous interventions. Or, to be more precise, another indistinguishable possibility is that God is perpetrating miraculous interventions continuously, and just consistently wills to do them in exactly the way that the true but unknown “laws of nature” predict. It remains possible that God could revoke this policy at some future time: this does not require much Kolmogorov complexity.
What evidence was there in the first place to promote to your attention the hypothesis of a naturalistic god over no god at all? If you don’t have any particular evidence that favors a naturalistic god over no god, surely the hypothesis of no god requires less complexity.
A Bayesian might never abandon possibilities to which he or she assigns prior mass without new evidence, but in addition to evidence that shifts the posterior, one can also revise a belief on information that suggests that an inappropriate prior was assigned to begin with.
Well, I was raised on it. If one day your Mom says, “don’t touch the stove, it’ll hurt”, and voila she’s right, you start to think maybe you ought to pay attention to what they’re telling you some times, including when they talk about “God.” Theres no way to distinguish one form of advice from the other until you get more experience. On this basis many things are acquired by making inferences based on the actions of people around us as we are growing up. “Everyone is wearing pants. Hmm. I guess I should too” is a pretty good heuristic Bayesian argument for many things, and keeps us out of trouble in unfamiliar experiences more often than not [cite some darwin page on here].
If I hadn’t been raised that way, probably nothing would have promoted it to my attention.
Knowing more about the processes that actually gave rise to your parents’ pronouncements on religion, do you think you were right to assign as much weight of evidence to them as you originally did?
Ah. Well, you’ve got me there. I’ll think about it. Your comment makes me think, though, about a more general issue. Is there a name for a bias that can happen if you think about an issue multiple times and get more and more convinced by, what actually, is essentially only one piece of evidence?
Thank you for the chronological suggestion; I think that will help it to make sense.
I would comment that the consistency theory for posteriors only guarantees convergence to “the truth” WHEN the likelihood terms are capable of making distinctions between the different possible models. If data is unavailable that distinguishes them… no convergence, just stabilization to potentially different subjective posteriors supported on the indistinguishable set. Since all we care about is making decisions, this is usually a distinction with no difference, except if there is a future event to come where they will make different predictions.
Accordingly my “God hypothesis” posterior probability has not updated in a very long time. I am, for example, not aware of what evidence there exists AT ALL with which to update this prior, but I suppose this is just because my posterior mass, conditional on the God hypothesis, came to rest, long ago, on a very non-falsifiable, (arguably science-compatible) version of belief in God. I.e. belief in a God who “operates all in accord with His will”, but who also does this exactly through the naturalistic mechanisms that we see operating all the time and not through miraculous interventions. Or, to be more precise, another indistinguishable possibility is that God is perpetrating miraculous interventions continuously, and just consistently wills to do them in exactly the way that the true but unknown “laws of nature” predict. It remains possible that God could revoke this policy at some future time: this does not require much Kolmogorov complexity. I can rule out the ordinary miraculous part of the mass (“why did it happen? Because God wanted it that special way in this special case”) because it’s not predictive. Accordingly my posterior does not give me “the mathematical right” to assert God’s non-existence.
Indeed, unless posterior mass is actually 0, I don’t think this is a matter of “mathematical rights” anyway. We’re making decisions under uncertainty, and we have some model for what the loss is if we make a mistake. You have, for example, the right to, say, not believe in God but not want to talk about it anymore, and otherwise act in accord with your belief, but I don’t see how it gives you the right to assert God’s non-existence. Sometimes we assign mass 0 to some possibilities, effectively, because we are not perfectly rational Bayesians, either we’d never thought of those possibilities, or it was computationally expedient to ignore them. In either of these cases we have no such “right”.
If I had the misfortune to have my prior “initialized” by my environment somewhere with a “vengeful God” hypothesis, who waits in Secret to test if you are really “one of the chosen” faithful, I’d be in a more difficult predicament. Hopefully, if I’d been born in such a sect, they’d have had some other clearly falsifiable beliefs that come along with the dogmatic ride, so that I’d have a way to identify the error. The worrisome part here, I notice, is that my thinking is rather driven by how it was initialized.
Well, you are certainly a lot better at Bayesian Statistics than I am. But if I am to base my “physics-defying, benevolent, superintelligent sky wizard” hypothesis on evidence such as badly written holy books that look spuriously like hodge-podge culture dumps, the general religious disagreement, the continued non-answering of prayers, failure to divine simple mathematical or physical truths, and how science is significantly more productive, well… Every time a prayer goes unanswered I can theoretically update on it for a lower credence. Every conflicting holy scripture allegedly bequeathed by the divine, claiming to be the one truth is a major hit.
Meanwhile the hypothesis that what most people call religious experiences is located in the temporal lobe gains credence from every cognitive neuro-scientific study of it.
If you haven’t updated your credence of immortal physics defying super-intelligent friendly sky wizards in a while, I am going to tell you to look harder. The evidence is there. From what you state it seems that you might suffer from motivated cognition and belief hysteresis, so take this opportunity to get a feel on those so you might recognize them in the future.
Now, for the hypothesis of “immortal physics-defying wizard doing whatever makes the world seem normal” is strictly dominated by the hypothesis of “whatever makes the world seem normal”. That is Minimum Message Length/Solomonoff Induction right there.
But given that you already have Bayes down to a science, I don’t see why you should become a fine Lesswrongian in just a short time. Welcome to lesswrong.
Thanks for mentioning belief hysteresis.
It’s a straw dummy to say that I haven’t updated my priors about insert ridiculous thing here. Those are not the things I described having residual posterior mass on; my “sky wizard” is currently content to play by very different rules. As for the epi-phenomena of how a society embraces religion, or does not, I agree, this does suggest that it would be foolish to base one’s beliefs SOLELY on the “evidence” of the crowd as you grow up.
Also, incidentally, MML/Kolmogorov complexity and Solomonoff are not the same here. The former do not give a posterior on models, they select one model. The later could and would easily put posterior mass on multiple programs that were observationally the same up to the amount of data we have currently observed.
Consider the sequence of numbers: 1,2,3. MML would (for most Turing machines anyway) select the program: next=old+1. Solomonoff would have posterior mass on many other sequences (e.g. do a query on http://oeis.org/). It’d put most of it’s mass on that program, but leave nonzero mass on other programs, like one that computes this sequence, say: http://oeis.org/A006530, and if the next numbers reveals the sequence: 1, 2, 3, 2, 5, 3, 7, 2, 3, 5 Then you can bet that’s where a lot of the mass will shift.
… And the really subtle bit, that I don’t want you to miss, is that it doesn’t all shift there, it’s still not a point posterior. There’s still mass on:
Use http://oeis.org/A006530 for the first Googol entries, and then, all zeros.
That mass won’t go away for a LONG time (that is if the data keeps coming from http://oeis.org/A006530 for at least Googol/2 steps.
And there are many possibilities like this out there. Sure, they have less mass individually, but there ARE many of them. And some don’t even have appreciably less prior mass (under a Solomonoff prior and a “reasonable” Turing machine). Computability is a whole ’nother issue.
If I understand what you mean, put a hyphen between “physics” and “defying”, because I had mis-parsed that the first time.
Hee!
I sort of like the idea of immortal physics defying super-intelligent friendly sky wizards, actually.
”Hamlet, in love with the old man’s daughter, the old… man… thinks.”
What evidence was there in the first place to promote to your attention the hypothesis of a naturalistic god over no god at all? If you don’t have any particular evidence that favors a naturalistic god over no god, surely the hypothesis of no god requires less complexity.
A Bayesian might never abandon possibilities to which he or she assigns prior mass without new evidence, but in addition to evidence that shifts the posterior, one can also revise a belief on information that suggests that an inappropriate prior was assigned to begin with.
Well, I was raised on it. If one day your Mom says, “don’t touch the stove, it’ll hurt”, and voila she’s right, you start to think maybe you ought to pay attention to what they’re telling you some times, including when they talk about “God.” Theres no way to distinguish one form of advice from the other until you get more experience. On this basis many things are acquired by making inferences based on the actions of people around us as we are growing up. “Everyone is wearing pants. Hmm. I guess I should too” is a pretty good heuristic Bayesian argument for many things, and keeps us out of trouble in unfamiliar experiences more often than not [cite some darwin page on here].
If I hadn’t been raised that way, probably nothing would have promoted it to my attention.
Knowing more about the processes that actually gave rise to your parents’ pronouncements on religion, do you think you were right to assign as much weight of evidence to them as you originally did?
Ah. Well, you’ve got me there. I’ll think about it. Your comment makes me think, though, about a more general issue. Is there a name for a bias that can happen if you think about an issue multiple times and get more and more convinced by, what actually, is essentially only one piece of evidence?
Well, there are various ways to double-count evidence, but that sounds a lot like the idea discussed in this post.
Thanks.