Bob: Sure, if you specify a disutility function that mandates lots-o’-specks to be worse than torture, decision theory will prefer torture. But that is literally begging the question, since you can write down a utility function to come to any conclusion you like. On what basis are you choosing that functional form? That’s where the actual moral reasoning goes. For instance, here’s a disutility function, without any of your dreaded asymptotes, that strictly prefers specks to torture:
U(T,S) = ST + S
Freaking out about asymptotes reflects a basic misunderstanding of decision theory, though. If you’ve got a rational preference relation, then you can always give a bounded utility function. (For example, the function I wrote above can be transformed to U(T,S) = (ST + S)/(ST + S + 1), which always gives you a function in [0,1], and gives rise to the same preference relation as the original.) If you absolutely require unbounded utilities in utility functions, then you become subject to a Dutch book (see Vann McGee’s “An Airtight Dutch Book”). Attempts to salvage unbounded utility pretty much always end up accepting certain Dutch books as rational, which means you’ve rejected the whole decision-theoretic justification of Bayesian probability theory. Now, the existence of bounds means that if you have a monotone utility function, then the limits will be well-defined.
So asymptotic reasoning about monotonically increasing harms is entirely legit, and you can’t rule it out of bounds without giving up on either Bayesianism or rational preferences.
Bob: Sure, if you specify a disutility function that mandates lots-o’-specks to be worse than torture, decision theory will prefer torture. But that is literally begging the question, since you can write down a utility function to come to any conclusion you like. On what basis are you choosing that functional form? That’s where the actual moral reasoning goes. For instance, here’s a disutility function, without any of your dreaded asymptotes, that strictly prefers specks to torture:
U(T,S) = ST + S
Freaking out about asymptotes reflects a basic misunderstanding of decision theory, though. If you’ve got a rational preference relation, then you can always give a bounded utility function. (For example, the function I wrote above can be transformed to U(T,S) = (ST + S)/(ST + S + 1), which always gives you a function in [0,1], and gives rise to the same preference relation as the original.) If you absolutely require unbounded utilities in utility functions, then you become subject to a Dutch book (see Vann McGee’s “An Airtight Dutch Book”). Attempts to salvage unbounded utility pretty much always end up accepting certain Dutch books as rational, which means you’ve rejected the whole decision-theoretic justification of Bayesian probability theory. Now, the existence of bounds means that if you have a monotone utility function, then the limits will be well-defined.
So asymptotic reasoning about monotonically increasing harms is entirely legit, and you can’t rule it out of bounds without giving up on either Bayesianism or rational preferences.