This makes me wonder if you could get a safe and extremely useful oracle if you only allow it to output a few bits (eg buy/sell some specific stock).
Yes, they call it a low-bandwidth oracle.
If the AI is omniscient, it brings out whichever of the two timelines it likes better. In the worst case, this doubles the chance that, say, an AI aligned with the boxed one arises.
This makes me wonder if you could get a safe and extremely useful oracle if you only allow it to output a few bits (eg buy/sell some specific stock).
Yes, they call it a low-bandwidth oracle.
If the AI is omniscient, it brings out whichever of the two timelines it likes better. In the worst case, this doubles the chance that, say, an AI aligned with the boxed one arises.