You’re only “calculating a probability through Solomonoff Induction” if the probability is only affected by complexity. If there are other reasons that could reduce the probability, they can reduce it by more. For instance, a lying mugger can increase his probability of being able to extort money from a naive rationalist by increasing the size of the purported payoff, so a large payoff is better evidence for a lying mugger than a small payoff.
Additional factors very well may reduce the probability. The question is whether they reduce it by enough. Given how enormously large 3^^^^3 is, I’m practically certain they won’t. And even if you somehow manage to come up with a way to reduce the probability by enough, there’s nothing stopping the mugger from simply adding another up-arrow to his claim: “Give me five dollars, or I’ll torture and kill 3^^^^^3 people!” Then your probability reduction will be rendered pretty much irrelevant. And then, if you miraculously find a way to reduce the probability again to account for the enormous increase in utility, the mugger will simply add yet another up-arrow. So we see that ad hoc probability reductions don’t work well here, because the mugger can always overcome those by making his number bigger; what’s needed is a probability penalty that scales with the size of the mugger’s claim: a penalty that can always reduce the expected utility of his offer down to ~0. Factors independent of the size of his claim, such as the probability that he’s lying (since he could be lying no matter how big or how small his number actually is), are unlikely to accomplish this.
such as the probability that he’s lying (since he could be lying no matter how big or how small his number actually is)
He could be lying regardless of the size of the number, but the probability that he is lying would still be affected by the size of the number. A larger number is more likely to convince a naive rationalist than a smaller number, precisely because believing the larger number means believing there is more utility. This makes larger numbers more beneficial to fake muggers than smaller numbers. So the larger the number, the lower the chance that the mugger is telling the truth. This means that changing the size of the number can decrease the probability of truth in a way that keeps pace with the increase in utility that being true would provide.
(Actually, there’s an even more interesting factor that nobody ever brings up: even genuine muggers must have a distribution of numbers they are willing to use. This distribution must have a peak at a finite value, since it is impossible to have an even distribution over all numbers. If the fake mugger keeps adding arrows, he’s going to go over this peak and a rationalist’s estimate that he is telling the truth should go down because of that as well.)
You’re only “calculating a probability through Solomonoff Induction” if the probability is only affected by complexity. If there are other reasons that could reduce the probability, they can reduce it by more. For instance, a lying mugger can increase his probability of being able to extort money from a naive rationalist by increasing the size of the purported payoff, so a large payoff is better evidence for a lying mugger than a small payoff.
Additional factors very well may reduce the probability. The question is whether they reduce it by enough. Given how enormously large 3^^^^3 is, I’m practically certain they won’t. And even if you somehow manage to come up with a way to reduce the probability by enough, there’s nothing stopping the mugger from simply adding another up-arrow to his claim: “Give me five dollars, or I’ll torture and kill 3^^^^^3 people!” Then your probability reduction will be rendered pretty much irrelevant. And then, if you miraculously find a way to reduce the probability again to account for the enormous increase in utility, the mugger will simply add yet another up-arrow. So we see that ad hoc probability reductions don’t work well here, because the mugger can always overcome those by making his number bigger; what’s needed is a probability penalty that scales with the size of the mugger’s claim: a penalty that can always reduce the expected utility of his offer down to ~0. Factors independent of the size of his claim, such as the probability that he’s lying (since he could be lying no matter how big or how small his number actually is), are unlikely to accomplish this.
He could be lying regardless of the size of the number, but the probability that he is lying would still be affected by the size of the number. A larger number is more likely to convince a naive rationalist than a smaller number, precisely because believing the larger number means believing there is more utility. This makes larger numbers more beneficial to fake muggers than smaller numbers. So the larger the number, the lower the chance that the mugger is telling the truth. This means that changing the size of the number can decrease the probability of truth in a way that keeps pace with the increase in utility that being true would provide.
(Actually, there’s an even more interesting factor that nobody ever brings up: even genuine muggers must have a distribution of numbers they are willing to use. This distribution must have a peak at a finite value, since it is impossible to have an even distribution over all numbers. If the fake mugger keeps adding arrows, he’s going to go over this peak and a rationalist’s estimate that he is telling the truth should go down because of that as well.)