You could choose to single out a single alternative hypothesis that says the sun won’t rise some day in the future. The ratio between P(sun rises until day X) and P(sun rises every day) will not change with any evidence before day X. If initially you believed a 99% chance of “the sun rises every day until day X” and a 1% chance of Solomonoff induction’s prior, you would end up assigning more than a 99% probability to “the sun rises every day until day X”.
Solomonoff induction itself will give some significant probability mass to “induction works until day X” statements. The Kolmogorov complexity of “the sun rises until day X” is about the Kolmogorov complexity of “the sun rises every day” plus the Kolmogorov complexity of X (approximately log2(x)+2log2(log2(x))). Therefore, even according to Solomonoff induction, the “sun rises until day X” hypothesis will have a probability approximately proportional to P(sun rises every day) / (X log2(X)^2). This decreases subexponentially with X, and even slower if you sum this probability for all Y >= X.
In order to get exponential change in the odds, you would need to have repeatable independent observations that distinguish between Solomonoff induction and some other hypothesis. You can’t get that in the case of “sun rises every day until day X” hypotheses.
If you only assign significant probability mass to one changeover day, you behave inductively on almost all the days up to that point, and hence make relatively few epistemic errors. To put it another way, unless you assign superexponentially-tiny probability to induction ever working, the number of anti-inductive errors you make over your lifespan will be bounded.
If you only assign significant probability mass to one changeover day, you behave inductively on almost all the days up to that point, and hence make relatively few epistemic errors.
But even one epistemic error is enough to cause an arbitrarily large loss in utility. Suppose you think that with 99% probability, unless you personally join a monastery and stop having any contact with the outside world, God will put everyone who ever existed into hell on 1/1/2050. So you do that instead of working on making a positive Singularity happen. Since you can’t update away this belief until it’s too late, it does seem important to have “reasonable” priors instead of just a non-superexponentially-tiny probability to “induction works”.
But even one epistemic error is enough to cause an arbitrarily large loss in utility.
This is always true.
Since you can’t update away this belief until it’s too late, it does seem important to have “reasonable” priors instead of just a non-superexponentially-tiny probability to “induction works”.
I’d say more that besides your one reasonable prior you also need to not make various sorts of specifically harmful mistakes, but this only becomes true when instrumental welfare as well as epistemic welfare are being taken into account. :)
Do you think it’s useful to consider “epistemic welfare” independently of “instrumental welfare”? To me it seems that approach has led to a number of problems in the past.
Solomonoff Induction was historically justified a way similar to your post: you should use the universal prior, because whatever the “right” prior is, if it’s computable then substituting the universal prior will cost you only a limited number of epistemic errors. I think this sort of argument is more impressive/persuasive than it should be (at least for some people, including myself when I first came across it), and makes them erroneously think the problem of finding “the right prior” or “a reasonable prior” is already solved or doesn’t need to be solved.
Thinking that anthropic reasoning / indexical uncertainty is clearly an epistemic problem and hence ought to be solved within epistemology (rather than decision theory), leading for example to dozens of papers arguing over what is the right way to do Bayesian updating in the Sleeping Beauty problem.
Yep! And for the record, I agree with your above paragraphs given that.
I would like to note explicitly for other readers that probability goes down proportionally to the exponential of Kolmogorov complexity, not proportional to Kolmogorov complexity. So the probability of the Sun failing to rise the next day really is going down at a noticeable rate, as jacobt calculates (1 / x log(x)^2 on day x). You can’t repeatedly have large likelihood ratios against a hypothesis or mixture of hypotheses and not have it be demoted exponentially fast.
You could choose to single out a single alternative hypothesis that says the sun won’t rise some day in the future. The ratio between P(sun rises until day X) and P(sun rises every day) will not change with any evidence before day X. If initially you believed a 99% chance of “the sun rises every day until day X” and a 1% chance of Solomonoff induction’s prior, you would end up assigning more than a 99% probability to “the sun rises every day until day X”.
Solomonoff induction itself will give some significant probability mass to “induction works until day X” statements. The Kolmogorov complexity of “the sun rises until day X” is about the Kolmogorov complexity of “the sun rises every day” plus the Kolmogorov complexity of X (approximately log2(x)+2log2(log2(x))). Therefore, even according to Solomonoff induction, the “sun rises until day X” hypothesis will have a probability approximately proportional to P(sun rises every day) / (X log2(X)^2). This decreases subexponentially with X, and even slower if you sum this probability for all Y >= X.
In order to get exponential change in the odds, you would need to have repeatable independent observations that distinguish between Solomonoff induction and some other hypothesis. You can’t get that in the case of “sun rises every day until day X” hypotheses.
If you only assign significant probability mass to one changeover day, you behave inductively on almost all the days up to that point, and hence make relatively few epistemic errors. To put it another way, unless you assign superexponentially-tiny probability to induction ever working, the number of anti-inductive errors you make over your lifespan will be bounded.
But even one epistemic error is enough to cause an arbitrarily large loss in utility. Suppose you think that with 99% probability, unless you personally join a monastery and stop having any contact with the outside world, God will put everyone who ever existed into hell on 1/1/2050. So you do that instead of working on making a positive Singularity happen. Since you can’t update away this belief until it’s too late, it does seem important to have “reasonable” priors instead of just a non-superexponentially-tiny probability to “induction works”.
This is always true.
I’d say more that besides your one reasonable prior you also need to not make various sorts of specifically harmful mistakes, but this only becomes true when instrumental welfare as well as epistemic welfare are being taken into account. :)
Do you think it’s useful to consider “epistemic welfare” independently of “instrumental welfare”? To me it seems that approach has led to a number of problems in the past.
Solomonoff Induction was historically justified a way similar to your post: you should use the universal prior, because whatever the “right” prior is, if it’s computable then substituting the universal prior will cost you only a limited number of epistemic errors. I think this sort of argument is more impressive/persuasive than it should be (at least for some people, including myself when I first came across it), and makes them erroneously think the problem of finding “the right prior” or “a reasonable prior” is already solved or doesn’t need to be solved.
Thinking that anthropic reasoning / indexical uncertainty is clearly an epistemic problem and hence ought to be solved within epistemology (rather than decision theory), leading for example to dozens of papers arguing over what is the right way to do Bayesian updating in the Sleeping Beauty problem.
Ok, I agree with this interpretation of “being exposed to ordered sensory data will rapidly promote the hypothesis that induction works”.
Yep! And for the record, I agree with your above paragraphs given that.
I would like to note explicitly for other readers that probability goes down proportionally to the exponential of Kolmogorov complexity, not proportional to Kolmogorov complexity. So the probability of the Sun failing to rise the next day really is going down at a noticeable rate, as jacobt calculates (1 / x log(x)^2 on day x). You can’t repeatedly have large likelihood ratios against a hypothesis or mixture of hypotheses and not have it be demoted exponentially fast.