There are two potential obstacles such a system would present to an AI, the first is locating our world out of all the possibilities. Every bit of data the AI can access, be it its own code or the rules of the simulation or the amount of time you let it run is a clue. If the total amount of data exceeds th Komelgorov complexity of reality, then an AIXI would figure out where it was. The laws of quantum physics are quite simple, and we really don’t know how much info is needed to point to a useful description of the world. This means that if our AI code is simple, and we are getting it to prove a theorem, this could be a serious barrier, but with an AI thats fed oodles of real world data, not so much.
The second barrier presented to an AI is in getting stuff done, once it knows that its in a sandbox, it would find weird tricks about making radio signals with the memory to be harder than if it had direct hardware access. Even if the software sandbox is flawless, it can work out a lot about its hardware and software just by knowing that they were optimized for efficient computation. This might give it enough understanding of how its implemented to try rowhammer.
Alternatively it can just encode whatever malicious payload it likes into the output channel. If you don’t look at the output then its just a warm box. Either way, it looks less useful than holomorphic encryption.
There are two potential obstacles such a system would present to an AI, the first is locating our world out of all the possibilities. Every bit of data the AI can access, be it its own code or the rules of the simulation or the amount of time you let it run is a clue. If the total amount of data exceeds th Komelgorov complexity of reality, then an AIXI would figure out where it was. The laws of quantum physics are quite simple, and we really don’t know how much info is needed to point to a useful description of the world. This means that if our AI code is simple, and we are getting it to prove a theorem, this could be a serious barrier, but with an AI thats fed oodles of real world data, not so much.
The second barrier presented to an AI is in getting stuff done, once it knows that its in a sandbox, it would find weird tricks about making radio signals with the memory to be harder than if it had direct hardware access. Even if the software sandbox is flawless, it can work out a lot about its hardware and software just by knowing that they were optimized for efficient computation. This might give it enough understanding of how its implemented to try rowhammer.
Alternatively it can just encode whatever malicious payload it likes into the output channel. If you don’t look at the output then its just a warm box. Either way, it looks less useful than holomorphic encryption.