TheManxLoiner comments on Visual demonstration of Optimizer’s curse

[TheManxLoiner]

What do we mean by $U-V$?

I think the setting is:

• We have a true value function $V$

• We have a process to learn an estimate of $V$. We run this process once and we get $U$

• We then ask an AI system to act so as to maximize $U$ (its estimate of human values)

So in this context, $U-V$ is just a fixed function measuring the error between the learnt values and true values.

I think confusion could be using the term $U$ to represent both a single instance or the random variable/​process.

[Roman Malov]

So, $U\left(x\right)$ is a random variable in the sense that it is drawn from a distribution of functions, and the expected value of those functions at each point $x$ is equal to $V\left(x\right)$. Am I understanding you correctly?

[TheManxLoiner]

Sounds sensible to me!