The AIs motivations can be precisely controlled. In fact such an AI can be limited to purely prediction. It would have no agency. No motivations or goals whatsoever. It just tries to predict what the price of each stock will be the next day.
For such a task, the AI doesn’t need any of the reduced impact stuff described here. That stuff becomes relevant in more complicated domains, like controlling a robot body in the real world. Say, to do something simple like collect paperclips that have fallen on the floor.
In such a domain you might want to limit it to just predicting what a human would do if they were controlling the robot. Not find the absolute optimal sequence of actions. Which might involve running away, building more robots, and taking over the world. Then building as many paperclip factories as possible.
AIXI is controllable in this way. Or at least the Solomonoff induction part, which just predicts the future. You could just use it to see what the future will be. The dangerous optimization only comes in later. When you put another program on top of it that searches for the optimal sequence of actions to get a certain outcome. An outcome we might not want.
As far as I can tell, all the proposals for AI control require the ability to use the AI like this. As an optimizer or predictor for an arbitrary goal. Which we can control, if only in a restricted sense. If the AI is fundamentally malicious and uncontrollable, there is no way to get useful work out of it. Let alone use it to build FAI.
The AIs motivations can be precisely controlled. In fact such an AI can be limited to purely prediction. It would have no agency. No motivations or goals whatsoever. It just tries to predict what the price of each stock will be the next day.
For such a task, the AI doesn’t need any of the reduced impact stuff described here. That stuff becomes relevant in more complicated domains, like controlling a robot body in the real world. Say, to do something simple like collect paperclips that have fallen on the floor.
In such a domain you might want to limit it to just predicting what a human would do if they were controlling the robot. Not find the absolute optimal sequence of actions. Which might involve running away, building more robots, and taking over the world. Then building as many paperclip factories as possible.
AIXI is controllable in this way. Or at least the Solomonoff induction part, which just predicts the future. You could just use it to see what the future will be. The dangerous optimization only comes in later. When you put another program on top of it that searches for the optimal sequence of actions to get a certain outcome. An outcome we might not want.
As far as I can tell, all the proposals for AI control require the ability to use the AI like this. As an optimizer or predictor for an arbitrary goal. Which we can control, if only in a restricted sense. If the AI is fundamentally malicious and uncontrollable, there is no way to get useful work out of it. Let alone use it to build FAI.