Stephen, no problem. Incidentally, I share your doubt about the optimality of optimizing expected utility (though I wonder whether there might be a theorem that says anything coherent can be squeezed into that form).
CC, indeed there are many infinities (not merely infinitely many, not merely more than we can imagine, but more than we can describe), but so what? Any sort of infinite utility, coupled with a nonzero finite probability, leads to the sort of difficulty being contemplated here. Higher infinities neither help with this nor make it worse, so far as I can see. (I suppose it’s worth considering that it might conceivably make sense for an agent’s utilities to live in some structure “richer” than the usual real numbers, like Conway’s surreal numbers, where there are infinities and infinitesimals aplenty. But I think there are technical difficulties with this sort of scheme; for instance, doing calculus over the surreals is problematic. And of course we actually only have finite brains, so whatever utilities we have are presumably representable in finite terms even if they feature incommensurabilities of the sort that might be modelled in terms of something like the surreal numbers. But all this is a separate issue.)
Stephen, no problem. Incidentally, I share your doubt about the optimality of optimizing expected utility (though I wonder whether there might be a theorem that says anything coherent can be squeezed into that form).
CC, indeed there are many infinities (not merely infinitely many, not merely more than we can imagine, but more than we can describe), but so what? Any sort of infinite utility, coupled with a nonzero finite probability, leads to the sort of difficulty being contemplated here. Higher infinities neither help with this nor make it worse, so far as I can see. (I suppose it’s worth considering that it might conceivably make sense for an agent’s utilities to live in some structure “richer” than the usual real numbers, like Conway’s surreal numbers, where there are infinities and infinitesimals aplenty. But I think there are technical difficulties with this sort of scheme; for instance, doing calculus over the surreals is problematic. And of course we actually only have finite brains, so whatever utilities we have are presumably representable in finite terms even if they feature incommensurabilities of the sort that might be modelled in terms of something like the surreal numbers. But all this is a separate issue.)