The problem goes away if you add finiteness in any of a bunch of different places: restrict agents to only output decisions of bounded length, or to only follow strategies of bounded length, or expected utilities are constrained to finitely many distinct levels. (Making utility a bounded real number doesn’t work, but only because there are infinitely many distinct levels close to the bound).
The problem also goes away if you allow agents to output a countable sequence of successively better decisions, and define an optimal sequence as one such that for any possible decision, a decision at least that good appears somewhere in the sequence. This seems like the most promising approach.
The problem goes away if you add finiteness in any of a bunch of different places: restrict agents to only output decisions of bounded length, or to only follow strategies of bounded length, or expected utilities are constrained to finitely many distinct levels. (Making utility a bounded real number doesn’t work, but only because there are infinitely many distinct levels close to the bound).
The problem also goes away if you allow agents to output a countable sequence of successively better decisions, and define an optimal sequence as one such that for any possible decision, a decision at least that good appears somewhere in the sequence. This seems like the most promising approach.
Except that isn’t the problem and this post isn’t intended to address practicalities, so the criticism that this is unrealistic is irrelevant.