Sorry, I don’t feel like completely understanding your POV is worth the time. But I did read you reply 2-3 times. In roughly the same order as your writing.
Yes, so if you observe no sabotage, then you do update about the existence of a fifth column that would have, with some probability, sabotaged (an infinite possibility). But you don’t update about the existence of the fifth column that doesn’t sabotage, or wouldn’t have sabotaged YET, which are also infinite possibilities.
I’m not sure why infinity matters here, many things have infinite possibilities (like any continuous random variable) and you can still apply a rough estimate on the probability distribution.
I guess it’s a general failure of Bayesian reasoning. You can’t update 1 confidence beliefs, you can’t update 0 confidence beliefs, and you can’t update undefined beliefs.
I think this is an argument similar to an infinite recursion of where do priors come from? But Bayesian updates usually produces better estimate than your prior (and always better than your prior if you can do perfect updates, but that’s impossible), and you can use many methods to guestimate a prior distribution.
You have a pretty good model about what might cause the sun to rise tomorrow, but no idea, complete uncertainty (not 0 with certainty nor 1 with certainty, nor 50⁄50 uncertainty, just completely undefined certainty) about what would make the sun NOT rise tomorrow, so you can’t (rationally) Bayesian reason about it. You can bet on it, but you can’t rationally believe about it.
Unknown Unknowns are indeed a thing. You can’t completely rationally Bayesian reason about it, and that doesn’t mean you can’t try to Bayesian reason about it. Eliezer didn’t say you can become a perfect Bayesian reasoner either, he always said you can attempt to reason better, and strive to approach Bayesian reasoning.
you can still apply a rough estimate on the probability distribution.
No, you cannot. For things you have no idea, there is no way to (rationally) estimate their probabilities.
Bayesian updates usually produces better estimate than your prior (and always better than your prior if you can do perfect updates, but that’s impossible), and you can use many methods to guestimate a prior distribution.
No. There are many many things that have “priors” of 1, 0, or undefined. These are undefined. You can’t know anything about their “distribution” because they aren’t distributions. Everything is either true or false, 1 or 0. Probabilities only make sense when talking about human (or more generally, agent) expectations/uncertainties.
You can’t completely rationally Bayesian reason about it, and that doesn’t mean you can’t try to Bayesian reason about it.
That’s not what I mean, and it’s not even what I wrote. I’m not saying “completely”. I said you can’t Bayesian reason about it. I mean you are completely irrational when you even try to Bayesian reason about undefined, 1, or 0 things. What would trying to Bayesian reason about an undefined thing even look like to you?
Do you admit that you have no idea (probability/certainty/confidence-wise) about what might cause the sun not to rise tomorrow? Like is that a good example to you of a completely undefined thing, for which there is no “prior”? It’s one of the best to me, because the sun rising tomorrow is such a cornerstone example for introducing Bayesian reasoning. But to me it’s a perfect example of why Bayesianism is utterly insane. You’re not getting more certain that anything will happen again just because something like it happened before. You can never prove a hypothesis/theory/belief right (because you can’t prove a negative). We can only disprove hypotheses/theories/beliefs. So, with the sun, we have no idea what might cause it to not-rise tomorrow, so we can’t Bayesian ourselves into any sort of “confidence” or “certainty” or “probability” about it. A Bayesian alive but isolated from things that have died would believe itself immortal. This is not rational. Rationality is to just fail to disprove the null hypothesis, not believe ever-stronger in the null just because disconfirming evidence hasn’t yet been encountered.
Back to the blog post, there are cases in which absence of evidence is evidence of absence, but this isn’t it. If you look for something expected by a theory/hypothesis/belief, and you fail to find what it predicts, then that is evidence against it. But, “The 5th column exists” doesn’t predict anything (different from “the 5th column doesn’t exist”), so “the 5th column hasn’t attacked (yet)” isn’t evidence against it.
Sorry, I don’t feel like completely understanding your POV is worth the time. But I did read you reply 2-3 times. In roughly the same order as your writing.
I’m not sure why infinity matters here, many things have infinite possibilities (like any continuous random variable) and you can still apply a rough estimate on the probability distribution.
I think this is an argument similar to an infinite recursion of where do priors come from? But Bayesian updates usually produces better estimate than your prior (and always better than your prior if you can do perfect updates, but that’s impossible), and you can use many methods to guestimate a prior distribution.
Unknown Unknowns are indeed a thing. You can’t completely rationally Bayesian reason about it, and that doesn’t mean you can’t try to Bayesian reason about it. Eliezer didn’t say you can become a perfect Bayesian reasoner either, he always said you can attempt to reason better, and strive to approach Bayesian reasoning.
No, you cannot. For things you have no idea, there is no way to (rationally) estimate their probabilities.
No. There are many many things that have “priors” of 1, 0, or undefined. These are undefined. You can’t know anything about their “distribution” because they aren’t distributions. Everything is either true or false, 1 or 0. Probabilities only make sense when talking about human (or more generally, agent) expectations/uncertainties.
That’s not what I mean, and it’s not even what I wrote. I’m not saying “completely”. I said you can’t Bayesian reason about it. I mean you are completely irrational when you even try to Bayesian reason about undefined, 1, or 0 things. What would trying to Bayesian reason about an undefined thing even look like to you?
Do you admit that you have no idea (probability/certainty/confidence-wise) about what might cause the sun not to rise tomorrow? Like is that a good example to you of a completely undefined thing, for which there is no “prior”? It’s one of the best to me, because the sun rising tomorrow is such a cornerstone example for introducing Bayesian reasoning. But to me it’s a perfect example of why Bayesianism is utterly insane. You’re not getting more certain that anything will happen again just because something like it happened before. You can never prove a hypothesis/theory/belief right (because you can’t prove a negative). We can only disprove hypotheses/theories/beliefs. So, with the sun, we have no idea what might cause it to not-rise tomorrow, so we can’t Bayesian ourselves into any sort of “confidence” or “certainty” or “probability” about it. A Bayesian alive but isolated from things that have died would believe itself immortal. This is not rational. Rationality is to just fail to disprove the null hypothesis, not believe ever-stronger in the null just because disconfirming evidence hasn’t yet been encountered.
Back to the blog post, there are cases in which absence of evidence is evidence of absence, but this isn’t it. If you look for something expected by a theory/hypothesis/belief, and you fail to find what it predicts, then that is evidence against it. But, “The 5th column exists” doesn’t predict anything (different from “the 5th column doesn’t exist”), so “the 5th column hasn’t attacked (yet)” isn’t evidence against it.