I understand all that, but I still think it’s impossible to operationalize an admonition to Win. If
Omega says that Box B is empty if you try to win what’s inside it.
then you simply cannot implement a strategy that will give you the proceeds of Box B (unless you’re using some definition of “try” that is inconsistent with “choose a strategy that has a particular expected result”).
I think that falls under the “ritual of cognition” exception that Eliezer discussed for a while: when Winning depends directly on the ritual of cognition, then of course we can define a situation in which rationality doesn’t Win. But that is perfectly meaningless in every other situation (which is to say, in the world), where the result of the ritual is what matters.
I understand all that, but I still think it’s impossible to operationalize an admonition to Win. If
then you simply cannot implement a strategy that will give you the proceeds of Box B (unless you’re using some definition of “try” that is inconsistent with “choose a strategy that has a particular expected result”).
I think that falls under the “ritual of cognition” exception that Eliezer discussed for a while: when Winning depends directly on the ritual of cognition, then of course we can define a situation in which rationality doesn’t Win. But that is perfectly meaningless in every other situation (which is to say, in the world), where the result of the ritual is what matters.