Ah, your comment, and those of jmh and Dagon, have made me realise that I didn’t make it clear that I was taking a Bayesian/subjectivist interpretation of probability as a starting assumption (probably because I wrote this quickly and I know LessWrong leans that way). My intention was not really to engage in a Bayesian vs frequentist debate, as I feel that’s been adequately done elsewhere, but instead to say, “Let’s assume the Bayesian interpretation, and then try put it up against what seem like particularly challenging cases for the idea that someone can arrive at a meaningful probability estimate, and think about how one might arrive at an estimate even then, and what that might mean.”
And by “what that might mean”, I don’t mean just “Kyle thinks there’s a 0.001% chance a god exists”, but rather something like how we should interpret why Kyle gave that number rather than something orders of magnitude higher or lower (but that still matches the fuzzy, intuitive notion of “very low odds” which is perhaps all Kyle’s introspection on his gut feeling would give him), and how meaningful it is that he gave that particular number, rather than some other number.
The broader context is that I was working on a post about what the distinction between risk and uncertainty (now posted here), and came across Chris Smith’s example of Kyle the atheist. And I wanted to sort of take up the challenge implicit in what Smith wrote, and then take up the further challenge of a version of that claim that might never affect the world at all, such that we’d never get any data on it or, arguably, everything in its most obvious reference class. In that case, let’s still agree we’re sticking with the Bayesian interpretation, but then ask, does one’s specific choice of subjective credence really mean?
But I definitely should’ve made it more explicit that I was assuming the Bayesian interpretation and that those were the purposes of my post. I’ve now edited the intro to hopefully fix those issues.
(In case anyone’s for some reason interested in reading the comments in context, here’s what the post was originally like.)
Ah, your comment, and those of jmh and Dagon, have made me realise that I didn’t make it clear that I was taking a Bayesian/subjectivist interpretation of probability as a starting assumption (probably because I wrote this quickly and I know LessWrong leans that way). My intention was not really to engage in a Bayesian vs frequentist debate, as I feel that’s been adequately done elsewhere, but instead to say, “Let’s assume the Bayesian interpretation, and then try put it up against what seem like particularly challenging cases for the idea that someone can arrive at a meaningful probability estimate, and think about how one might arrive at an estimate even then, and what that might mean.”
And by “what that might mean”, I don’t mean just “Kyle thinks there’s a 0.001% chance a god exists”, but rather something like how we should interpret why Kyle gave that number rather than something orders of magnitude higher or lower (but that still matches the fuzzy, intuitive notion of “very low odds” which is perhaps all Kyle’s introspection on his gut feeling would give him), and how meaningful it is that he gave that particular number, rather than some other number.
The broader context is that I was working on a post about what the distinction between risk and uncertainty (now posted here), and came across Chris Smith’s example of Kyle the atheist. And I wanted to sort of take up the challenge implicit in what Smith wrote, and then take up the further challenge of a version of that claim that might never affect the world at all, such that we’d never get any data on it or, arguably, everything in its most obvious reference class. In that case, let’s still agree we’re sticking with the Bayesian interpretation, but then ask, does one’s specific choice of subjective credence really mean?
But I definitely should’ve made it more explicit that I was assuming the Bayesian interpretation and that those were the purposes of my post. I’ve now edited the intro to hopefully fix those issues.
(In case anyone’s for some reason interested in reading the comments in context, here’s what the post was originally like.)