axioman comments on When Goodharting is optimal: linear vs diminishing returns, unlikely vs likely, and other factors

[axioman]

I was thinking about normalisation as linearly rescaling every reward to $[0,1]$ when I wrote the comment. Then, one can always look at $[0,1{]}^2$, which might make it easier to graphically think about how different beliefs lead to different policies. Different scales can then be translated to a certain reweighting of the beliefs (at least from the perspective of the optimal policy), as maximizing $P\left(R_1\right)S_1{R}_1+P\left(R_2\right)S_2{R}_2$ is the same as maximizing $\frac{P\left(R_1\right)S_1}{P\left(R_1\right)S_1+P\left(R_2\right)S_2}{R}_1+\frac{P\left(R_2\right)S_2}{P\left(R_1\right)S_1+P\left(R_2\right)S_2}{R}_2$