Firstly, if we are talking actual computational complexity, then the mathematical background is already implicitly talking about the fastest possible algorithm to do X.
That the problem is NP-hard means that it will be difficult, no matter how intelligent the AI is.
Whether or not P=NP is an unsolved problem.
Predicting how a protien will fold is in BQP, which might be easier than NP. (another unsolved problem)
Computational complexity classes often don’t matter in practice. If you are solving the travelling salesman problem, you rarely need the shortest path, a short path is good enough. Secondly, the P vs NP is worst case. There are some special cases of the travelling salesman that are easy to solve. Taking an arbitrary protein and predicting how it will fold might be computationally intractable, but the work here is done by the word “arbitrary”. There are some protiens that are really hard to predict, and some that are easier. Can molecular nanotech be made using only the easily predicted protiens?
(Also, an algorithm doesn’t have an intelligence level, it has an intelligence to compute relation. Once you have invented minmax, increasing the recursion depth takes next to no insight into intelligence. Given a googleplex flop computer, your argument obviously fails, because any fool could bootstrap intelligence on that.)
I have an intuition that there should be some “best architecture”, at least for any given environment, and that this architecture should be relatively “simple”.
I agree. I think that AIXI, shows that there is a simple optimal design with unlimited compute. There being no simple optimal design with finite compute would be somewhat surprising. (I think logical induction is something like only exponentially worse than any possible mathematical reasoner in use of compute)
But this is a different argument than “as soon as artificial intelligence surpasses human intelligence, recursive self-improvement will take place, creating an entity we can’t hope to comprehend, let alone oppose.”
A model in which both are true. Suppose that there was a design of AI that was optimal for its compute. And suppose this design was reasonably findable, ie a bunch of smart humans could find this design with effort. And suppose this design was really, really smart.
(Humans often get the answer wrong even on the cases where the exact maths takes a trivial amount of compute, like the doctors with a disease that has prevalence 1 in 1000, and the 90% reliable test) The gap between humans and optimal use of compute is likely huge.
So either humans figure out the optimal, and implement it. Or humans hack together something near human level. The near human level AI might fiddle with its own workings, trying to increase its intelligence, and then it figures out the optimal design.
In this world the prediction that vastly superhuman AI arrives not long after AI reaches human level. Its just that the self improvement isn’t that recursive.
Firstly, if we are talking actual computational complexity, then the mathematical background is already implicitly talking about the fastest possible algorithm to do X.
Whether or not P=NP is an unsolved problem.
Predicting how a protien will fold is in BQP, which might be easier than NP. (another unsolved problem)
Computational complexity classes often don’t matter in practice. If you are solving the travelling salesman problem, you rarely need the shortest path, a short path is good enough. Secondly, the P vs NP is worst case. There are some special cases of the travelling salesman that are easy to solve. Taking an arbitrary protein and predicting how it will fold might be computationally intractable, but the work here is done by the word “arbitrary”. There are some protiens that are really hard to predict, and some that are easier. Can molecular nanotech be made using only the easily predicted protiens?
(Also, an algorithm doesn’t have an intelligence level, it has an intelligence to compute relation. Once you have invented minmax, increasing the recursion depth takes next to no insight into intelligence. Given a googleplex flop computer, your argument obviously fails, because any fool could bootstrap intelligence on that.)
I agree. I think that AIXI, shows that there is a simple optimal design with unlimited compute. There being no simple optimal design with finite compute would be somewhat surprising. (I think logical induction is something like only exponentially worse than any possible mathematical reasoner in use of compute)
A model in which both are true. Suppose that there was a design of AI that was optimal for its compute. And suppose this design was reasonably findable, ie a bunch of smart humans could find this design with effort. And suppose this design was really, really smart.
(Humans often get the answer wrong even on the cases where the exact maths takes a trivial amount of compute, like the doctors with a disease that has prevalence 1 in 1000, and the 90% reliable test) The gap between humans and optimal use of compute is likely huge.
So either humans figure out the optimal, and implement it. Or humans hack together something near human level. The near human level AI might fiddle with its own workings, trying to increase its intelligence, and then it figures out the optimal design.
In this world the prediction that vastly superhuman AI arrives not long after AI reaches human level. Its just that the self improvement isn’t that recursive.