Stuart_Armstrong comments on Best utility normalisation method to date?

[Stuart_Armstrong]

Er, this normalisation system way well solve that problem entirely. If $U_i$ prefers option $o_i$ (utility $1$), with second choice $o_0$ (utility $1/2$), and all the other options as third choice (utility $0$), then the expected utility of the random dictator is $1/n$ for all $U_i$ (as $p_i^{\ast }$ gives utility $1$, and $p_j^{\ast }$ gives utility $0$ for all $j\ne i$), so the normalised weighted utility to maximise is:

• $U=\frac{1}{n-1}(U_1+U_2+\dots U_n)$.

Using $(n-1)U$ (because scaling doesn’t change expected utility decisions), the utility of any $o_i$, $i>0$, is $1$, while the utility of $o_0$ is $n/2$. So if $n>2$, the compromise option $o_0$ will get chosen.

Don’t confuse the problems of the random dictator, with the problems of maximising the weighted sum of the normalisations that used the random dictator (and don’t confuse the other way, either; the random dictator is immune to players’ lying, this normalisation is not).

[Gurkenglas]

I was aware, but addressing his objection as though it were justified, which it would be if this were the only place where the agent’s preferences matter. This counterfactual is supported by my fondness for linear logic.