Well, there’s the systems that simply can’t process your wishes (AIXI for instance), but which you can use to e.g. cure cancer if you wish (you could train it to do what you tell it to but all it is looking for is sequence that leads to reward button press, which is terminal—no value for button being held). Just as there is a system, screwdriver, which I can use to unscrew screws, if I wish, but it’s not a screw unscrewing genie.
Well, there’s the systems that simply can’t process your wishes (AIXI for instance), but which you can use to e.g. cure cancer if you wish (you could train it to do what you tell it to but all it is looking for is sequence that leads to reward button press, which is terminal—no value for button being held). Just as there is a system, screwdriver, which I can use to unscrew screws, if I wish, but it’s not a screw unscrewing genie.