If you’d asked me in 1995 how many people it would take for the world to develop a fast, distributed system for moving films and TV episodes to people’s homes on an ‘when you want it, how you want it’ basis, internationally, without ads, I’d have said hundreds of thousands. In practice it took one guy with the right algorithm, depending on whether you pick napster or bittorrent as the magic that solves the problem without the need for any new physical technologies.
The thing about self-improving AI, is that we only need to get the algorithm right (or wrong :-() once.
We know with probability 1 it’s possible to create self-improving intelligence. After all, that’s what most humans are. No doubt other solutions exist. If we can find an algorithm or heuristic to implement any one of these solutions, or if we can even find any predecessor of any one of them, then we’re off—and given the right approach (be that algorithm , machine, heuristic, or whatever) it should be simply a matter of throwing computer power (or moore’s law) at it to speed up the rate of self-improvement. Heck, for all I know it could be a giant genetically engineered brain in a jar that cracks the problem.
Put it this way. Imagine you are a parasite. For x billion years you’re happy, then some organism comes up with sexual reproduction and suddenly it’s a nightmare. But eventually you catch up again. Then suddenly, in just 100 years, human society basically eradicates you completely out of the blue. The first 50 years of that century are bad. The next 20 are hideous. The next 10 are awful. The next 5 are disastrous… etc.
Similarly useful powerplant-scale nuclear fusion has always been 30 years away. But at some point, I suspect it will suddenly be only 2 years away, completely out of the blue....
Not sure I see your point. All the high speed connections were built long before bittorrent came along, and they were being used for idiotic point-to-point centralised transfers.
All that potential was achieving not much, before the existance of the right algorithm or approach to exploit it. I suspect a strong analogy here with future AI.