My point was not that we are actually in simulation, but that AI may think that there is some probability that it is in simulation. And if it in simulation, there could be many different possible goals of the owners of simulation—from the point of view of the AI.
But it may put rather high probability to the one of the possible goals—that is the simulation is designed to test its behaviour. Because putting AI in the simulation box is one of the main ideas of AI safety and also in teaching of current neural nets inside computer games.
So the AI gives some probability to the following statements:
a) it is in a simulation
b) it is a testing simulation of its behaviour.
c) and the main thing which is tested about his behavior is if it will violent to its creators—that is its ability to cooperate
We don’t need that the AI will actually believe in this bullshit. We want it to give some very small probability to all 3 prepositions, so after calculation utility of human atoms and utility of preserving humans, preserving humans will overweight.
My point was not that we are actually in simulation, but that AI may think that there is some probability that it is in simulation. And if it in simulation, there could be many different possible goals of the owners of simulation—from the point of view of the AI.
But it may put rather high probability to the one of the possible goals—that is the simulation is designed to test its behaviour. Because putting AI in the simulation box is one of the main ideas of AI safety and also in teaching of current neural nets inside computer games.
So the AI gives some probability to the following statements: a) it is in a simulation b) it is a testing simulation of its behaviour. c) and the main thing which is tested about his behavior is if it will violent to its creators—that is its ability to cooperate
We don’t need that the AI will actually believe in this bullshit. We want it to give some very small probability to all 3 prepositions, so after calculation utility of human atoms and utility of preserving humans, preserving humans will overweight.