If you have a nonzero probability that the mugger can produce arbitrary amounts of utility, the mugger just has to offer you enough to outweigh the smallness of this probability, which is fixed.
Without invoking complexity, one can say that an agent is immune to this form of Pascal’s mugging if, for fixed I, the quantity P(x amount of utility | I) goes to zero as x grows.
If the agent’s utility function is such that “x amount of utility” entails “f(x) amount of complexity,” f(x) --> infinity, then this will hold for priors that are sensitive to complexity.
See Sewing-Machine’s comment. The smallness of the probability isn’t fixed, if the probability is controlled by complexity, and complexity controls utility.
More precisely, the probability that the mugger can produce arbitrary amounts of utility is dominated by (the probability that the mugger can produce more than N units of utility), for every N; and as the latter is arbitrarily small for N sufficiently large, the former must be zero.
Don’t see how your idea defeats this:
Having a bounded utility function defeats that.
Without invoking complexity, one can say that an agent is immune to this form of Pascal’s mugging if, for fixed I, the quantity P(x amount of utility | I) goes to zero as x grows.
If the agent’s utility function is such that “x amount of utility” entails “f(x) amount of complexity,” f(x) --> infinity, then this will hold for priors that are sensitive to complexity.
See Sewing-Machine’s comment. The smallness of the probability isn’t fixed, if the probability is controlled by complexity, and complexity controls utility.
More precisely, the probability that the mugger can produce arbitrary amounts of utility is dominated by (the probability that the mugger can produce more than N units of utility), for every N; and as the latter is arbitrarily small for N sufficiently large, the former must be zero.