It seems like you’re relying on the existence of exponentially hard problems to mean that taking over the world is going to BE an exponentially hard problem. But you don’t need to solve every problem. You just need to take over the world.
Like, okay, the three body problem is ‘incomputable’ in the sense that it has chaotically sensitive dependence on initial conditions in many cases. So… don’t rely on specific behavior in those cases on long time horizons without the ability to do small adjustments to keep things on track.
If the AI can detect most of the hard cases and avoid relying on them, and include robustness by having multiple alternate mechanisms and backup plans, even just 94% success on arbitrary problems could translate into better than that on an overall solution.
This was specifically responding to the claim that an AI could solve problems without trial and error by perfectly simulating them, which I think it does a pretty reasonable job of shooting down.
It seems like you’re relying on the existence of exponentially hard problems to mean that taking over the world is going to BE an exponentially hard problem. But you don’t need to solve every problem. You just need to take over the world.
Like, okay, the three body problem is ‘incomputable’ in the sense that it has chaotically sensitive dependence on initial conditions in many cases. So… don’t rely on specific behavior in those cases on long time horizons without the ability to do small adjustments to keep things on track.
If the AI can detect most of the hard cases and avoid relying on them, and include robustness by having multiple alternate mechanisms and backup plans, even just 94% success on arbitrary problems could translate into better than that on an overall solution.
This was specifically responding to the claim that an AI could solve problems without trial and error by perfectly simulating them, which I think it does a pretty reasonable job of shooting down.