such as the probability that he’s lying (since he could be lying no matter how big or how small his number actually is)
He could be lying regardless of the size of the number, but the probability that he is lying would still be affected by the size of the number. A larger number is more likely to convince a naive rationalist than a smaller number, precisely because believing the larger number means believing there is more utility. This makes larger numbers more beneficial to fake muggers than smaller numbers. So the larger the number, the lower the chance that the mugger is telling the truth. This means that changing the size of the number can decrease the probability of truth in a way that keeps pace with the increase in utility that being true would provide.
(Actually, there’s an even more interesting factor that nobody ever brings up: even genuine muggers must have a distribution of numbers they are willing to use. This distribution must have a peak at a finite value, since it is impossible to have an even distribution over all numbers. If the fake mugger keeps adding arrows, he’s going to go over this peak and a rationalist’s estimate that he is telling the truth should go down because of that as well.)
He could be lying regardless of the size of the number, but the probability that he is lying would still be affected by the size of the number. A larger number is more likely to convince a naive rationalist than a smaller number, precisely because believing the larger number means believing there is more utility. This makes larger numbers more beneficial to fake muggers than smaller numbers. So the larger the number, the lower the chance that the mugger is telling the truth. This means that changing the size of the number can decrease the probability of truth in a way that keeps pace with the increase in utility that being true would provide.
(Actually, there’s an even more interesting factor that nobody ever brings up: even genuine muggers must have a distribution of numbers they are willing to use. This distribution must have a peak at a finite value, since it is impossible to have an even distribution over all numbers. If the fake mugger keeps adding arrows, he’s going to go over this peak and a rationalist’s estimate that he is telling the truth should go down because of that as well.)