Instead of thinking of the proposition H = the mugger is honest, and trying to calculate E(U|AH)P(H) + E(U|A~H)P(~H) where A is an action such as handing over your wallet and U is utility, you consider the hypotheses you’re really applying, T, that your general theory about muggers is sufficient to understand the situation, and ~T, that you just don’t have a handle on the situation. Then instead of directly using the mugger’s stated utility for the value you try to appeal to a simpler and more general theory to find a value for E(U|A~T). The more general “an offer from a stranger” reference class should suffice; you buy only a tiny minority of the things that you’re offered. Beyond that you have the “don’t know” reference class, but that has to have expected zero utility.
This argument doesn’t apply to any possibility that you are in a position to properly think about. You are in a position to assess the probability that Miriam Achaba wishes to entrust you with 25M USD, so you’re best advised not to reach for the “other” column on that one.
Instead of thinking of the proposition H = the mugger is honest, and trying to calculate E(U|AH)P(H) + E(U|A~H)P(~H) where A is an action such as handing over your wallet and U is utility, you consider the hypotheses you’re really applying, T, that your general theory about muggers is sufficient to understand the situation, and ~T, that you just don’t have a handle on the situation. Then instead of directly using the mugger’s stated utility for the value you try to appeal to a simpler and more general theory to find a value for E(U|A~T). The more general “an offer from a stranger” reference class should suffice; you buy only a tiny minority of the things that you’re offered. Beyond that you have the “don’t know” reference class, but that has to have expected zero utility.
This argument doesn’t apply to any possibility that you are in a position to properly think about. You are in a position to assess the probability that Miriam Achaba wishes to entrust you with 25M USD, so you’re best advised not to reach for the “other” column on that one.