I’m not sure I understand what you’re getting at.
If is the target utility function and is the proxy, and is then we calculated expected value of assuming is normally distributed over .
See this other comment.
It seems people have already explored this in depth. I like John Wentworth’s slogan that Goodheart is about generalization, not approximation.
Likely it’s not great news, like I said in the original post I am not sure how to interpret what I had noticed. But right now, after the reflection that’s come from people’s comments, I don’t think it’s bad news, and it might (maybe) even be slightly good news.[1]
Could you explain to me where the single step / multiple steps aspect comes in? I don’t see an assumption of only a single step anywhere, but maybe this comes from a lack of understanding.
Instead of a world where we need our proxy to be close to 100% error-free (this seems totally unrealistic), we just need the error to have ~ no tails (this might also be totally unrealistic but is a weaker requirement).