You can’t know the difficulty of a problem until you’ve solved it. Look at Hilbert’s problems. Some were solved immediately while others are still open today. Proving the you can color a map with five colors is easy and only takes up half a page. Proving that you can color a map with four colors is hard and takes up hundreds of pages. The same is true of science—a century ago physics was thought to soon be a dead field until the minor glitches with blackbody radiation and Mercury’s orbit turned out to be more than minor and actually dictated by mathematically complex theories who’s interaction with each other is still well beyond our best minds today. That’s why trying to predict the growth of intelligence is exactly as silly as trying to predict the number of Hilbert’s problems that will be solved over time. It has much less to do with how smart we are and much more to do with how hard the problems are, and that we won’t know until we solve them.
Contrary to everything said in (1), I think the software problem of AI is already solved. Simply note that (a) When people think programming an AI to be impossible it’s because they think of hardcoding and how no one understands the mind even remotely well enough to do this. But do we hardcode neural nets? No, in fact neural nets are magical in that no one can hardcode a facial recognition program as effective as a trained neural net. Suppose a sufficiently large neural net can be as smart as a human. Then what we would expect from smaller neural nets is exactly what we see now, namely non-rigid intelligence similar to our own but more limited. It would be absurd to expect more of them given our current hardware. (b) There are two forms of signalling in the body—electrical via action potentials and chemical via diffusion. Since the chemical sets up the electrical and diffusion is rather imprecise there are fundamental limits on how refined the brain’s macroscopic architecture can be. At the molecular scale biology is extremely complex. Enzymatic proteins are machines of profound sophistication. But none of it matters when it comes to understanding how the brain computes in real time because the only fast form of signalling is between neurons through electrical signals (chemical at the synapses but that’s a tiny distance). So the issue comes down to how the neurons are arranged to give rise to intelligence. But how they are arranged is relatively rough in its precision because that’s how chemical diffusion works.
With (1) and (2) in mind let’s address what the AI problem is really about—hardware. Moore’s law is going to hit the atomic barrier much earlier than even Kurzweil would expect computers to facilitate AI. The simple fact of the matter is that there is no clear way beyond this point. Neither parallel programming nor quantum computing is going to save the day without massive unprecedented breakthroughs. It’s a hard ware problem, and we won’t know how hard until we solve it.
~ a bioinformatics student and ex-singularitarian
Sorry the way worded it makes me look silly. I just meant that even if we had the perfect software we simply wouldn’t get a big enough speedup to bridge the gap.