I see — perhaps I did misinterpret your earlier comment. It sounds like the transition you are more interested in is closer to (AI has ~free rein over the internet) ⇒ (AI invents nanotech). I don’t think this is a step we should expect to be able to model especially well, but the best story/analogy I know of for it is probably the end part of That Alien Message. i.e., what sorts of approaches would we come up with, if all of human civilization was bent on solving the equivalent problem from our point of view?
If instead you’re thinking more about a transition like (AI is superintelligent but in a box) ⇒ (AI has ~free rein over the internet), then I’d say that I’d expect us to skip the “in a box” step entirely.
I see — perhaps I did misinterpret your earlier comment. It sounds like the transition you are more interested in is closer to (AI has ~free rein over the internet) ⇒ (AI invents nanotech). I don’t think this is a step we should expect to be able to model especially well, but the best story/analogy I know of for it is probably the end part of That Alien Message. i.e., what sorts of approaches would we come up with, if all of human civilization was bent on solving the equivalent problem from our point of view?
If instead you’re thinking more about a transition like (AI is superintelligent but in a box) ⇒ (AI has ~free rein over the internet), then I’d say that I’d expect us to skip the “in a box” step entirely.