Your comment gave me pause, and certainly makes me lean away from finite-support probability distributions somewhat.
However, if the problem is that you can actively generate more and more plausible universes without stop, then it does seem at some level like your belief structure is a sequence of finite-support probability distributions, doesn’t it? As you mentally generate more and more plausible universes, your belief gets updates to a distribution with larger and larger support. The main problem is just that “sequence of distributions” is a much uglier mathematical object than a single distribution.
Another thought: if you can actively mentally generate more and more possible universes, and if, in addition, the universes you generate have such large utilities that they become “more and more important” to consider (i.e. even after multiplying by their diminishing probabilities, the absolute value of probability*utility is increasing), then you are screwed. This was shown nicely by your examples. So in some sense, we have to restrict to situations where the possible universes you mentally generate are diminishing in importance (i.e. even if their utility is increasing, their probability is diminishing fast enough to make the sequence absolutely convergent).
Your comment gave me pause, and certainly makes me lean away from finite-support probability distributions somewhat.
However, if the problem is that you can actively generate more and more plausible universes without stop, then it does seem at some level like your belief structure is a sequence of finite-support probability distributions, doesn’t it? As you mentally generate more and more plausible universes, your belief gets updates to a distribution with larger and larger support. The main problem is just that “sequence of distributions” is a much uglier mathematical object than a single distribution.
Another thought: if you can actively mentally generate more and more possible universes, and if, in addition, the universes you generate have such large utilities that they become “more and more important” to consider (i.e. even after multiplying by their diminishing probabilities, the absolute value of probability*utility is increasing), then you are screwed. This was shown nicely by your examples. So in some sense, we have to restrict to situations where the possible universes you mentally generate are diminishing in importance (i.e. even if their utility is increasing, their probability is diminishing fast enough to make the sequence absolutely convergent).