Can you explain how such a preference can be consistent? The total incidence of both torture and dust specks is unknown in either case. On what basis would an agent that trades one torture for avoiding 3^^^3*10^-80 dust specks refuse the same deal a second time? Or the 10^80th time? Given that 3^^^3*10^-80 people are involved it seems astronomically unlikely that the rate of torture changed noticeably even only assuming knowledge available to the agent. In any case 10^80 separate instances of the agent with no knowledge of each other would make the same deal 10^80 times, and can’t complain about being deceived since no information about the incidence of torture was assumed. Even assuming the agent only makes the deal only a single time consistency would then require that the agent prefer trading 3^^^3 dust specks for avoiding 10^80 instances of torture over trading 3^^^3*(1+10^-80) dust specks for 10^80 +1 instances of torture, which seems implausible.
The total incidence of both torture and dust specks is unknown in either case.
Where was this declared? (Not that it matters for the purpose of this point.) The agent has prior probabilities distributed over the number of possible incidence of torture and dustspecks. It is impossible not to. And after taking one such deal those priors will be different. Sure, restricting the access to information about the current tortured population will make it harder for an agent to implement preferences that are not linear with respect to additional units but it doesn’t make those preferences inconsistent and it doesn’t stop the agent doing its best to maximise utility despite the difficulty.
There is no information on the total incidence of either included in the problem statement (other than the numbers used), and I have seen no one answer conditionally based on the incidence of either.
The agent has prior probabilities distributed over the number of possible incidence of torture and dustspecks.
Yes, of course, I thought my previous comment clearly implied that?
And after taking one such deal those priors will be different.
Infinitesimally. I thought I addressed that? The problem implies the existence of an enormous number of people. Conditional on there actually being that many people the expected number of people tortured shifts by the tiniest fraction of the total. If the agent is sensitive to such a tiny shift we are back to requiring extraordinary precision.
Can you explain how such a preference can be consistent? The total incidence of both torture and dust specks is unknown in either case. On what basis would an agent that trades one torture for avoiding 3^^^3*10^-80 dust specks refuse the same deal a second time? Or the 10^80th time? Given that 3^^^3*10^-80 people are involved it seems astronomically unlikely that the rate of torture changed noticeably even only assuming knowledge available to the agent. In any case 10^80 separate instances of the agent with no knowledge of each other would make the same deal 10^80 times, and can’t complain about being deceived since no information about the incidence of torture was assumed. Even assuming the agent only makes the deal only a single time consistency would then require that the agent prefer trading 3^^^3 dust specks for avoiding 10^80 instances of torture over trading 3^^^3*(1+10^-80) dust specks for 10^80 +1 instances of torture, which seems implausible.
Where was this declared? (Not that it matters for the purpose of this point.) The agent has prior probabilities distributed over the number of possible incidence of torture and dustspecks. It is impossible not to. And after taking one such deal those priors will be different. Sure, restricting the access to information about the current tortured population will make it harder for an agent to implement preferences that are not linear with respect to additional units but it doesn’t make those preferences inconsistent and it doesn’t stop the agent doing its best to maximise utility despite the difficulty.
There is no information on the total incidence of either included in the problem statement (other than the numbers used), and I have seen no one answer conditionally based on the incidence of either.
Yes, of course, I thought my previous comment clearly implied that?
Infinitesimally. I thought I addressed that? The problem implies the existence of an enormous number of people. Conditional on there actually being that many people the expected number of people tortured shifts by the tiniest fraction of the total. If the agent is sensitive to such a tiny shift we are back to requiring extraordinary precision.