I agree with you that “optimal all the time, no exceptions” is not a realistic goal or criterion.
Indeed I believe it’s provably impossible even without needing to add the fuzziness and confusion of real life into the mix. Even if we limit ourselves to simple bounded systems.
Which kind of puts a hole in EY’s thesis that it should be possible to have a decision theory which always wins.
Eliezer has conceded that it is impossible in principle to have a decision theory which always wins. He says he wants one that will always win except when an adversary is deliberately making it lose. In other words, he hopes that your scenario is sufficiently complicated that it wouldn’t happen in reality unless someone arranges things to cause the decision theory to lose.
Even if we limit ourselves to simple bounded systems.
If the “simple bounded systems” are, basically, enumerable and the definition of “win” is fixed, F(P) can be a simple lookup table which does always win.
It’s the same thing as saying that given a dataset I can always construct a model with zero error for members of this dataset. That does not mean that the model will perform well on out-of-sample data.
I am also not sure to which degree EY intended this statement to be a “hard”, literal claim.
I agree with you that “optimal all the time, no exceptions” is not a realistic goal or criterion.
Indeed I believe it’s provably impossible even without needing to add the fuzziness and confusion of real life into the mix. Even if we limit ourselves to simple bounded systems.
Which kind of puts a hole in EY’s thesis that it should be possible to have a decision theory which always wins.
Eliezer has conceded that it is impossible in principle to have a decision theory which always wins. He says he wants one that will always win except when an adversary is deliberately making it lose. In other words, he hopes that your scenario is sufficiently complicated that it wouldn’t happen in reality unless someone arranges things to cause the decision theory to lose.
If the “simple bounded systems” are, basically, enumerable and the definition of “win” is fixed, F(P) can be a simple lookup table which does always win.
It’s the same thing as saying that given a dataset I can always construct a model with zero error for members of this dataset. That does not mean that the model will perform well on out-of-sample data.
I am also not sure to which degree EY intended this statement to be a “hard”, literal claim.