The problem specifies that something will be revealed to you, which will program you to believe it, even though false. It doesn’t explicitly limit what can be injected into the information stream. So yes, assuming you would value the existence of a Friendly AI, yes, that’s entirely valid as optimal false information. Cost: you are temporarily wrong about something, and realize your error soon enough.
I think we have a good contender for the optimal false information here.
The problem specifies that something will be revealed to you, which will program you to believe it, even though false. It doesn’t explicitly limit what can be injected into the information stream. So yes, assuming you would value the existence of a Friendly AI, yes, that’s entirely valid as optimal false information. Cost: you are temporarily wrong about something, and realize your error soon enough.