What is productive cannot be judged in advance if you are facing unknown unknowns. And the very nature of scientific advances is an evolutionary process, not one of deliberate design but discovery. We may very well be able to speed up certain weak computational problems by sheer brute force but not solve problems by creating an advance problem solving machine.
That we do not ask ourselves what we are trying to achieve is a outstanding feature of our ability to learn and change our habits. If we would all be the productive machines that you might have in mind, then the religious people would stay religious and the bushman would keep striving for a successful chase even after they build a supermarket in front of his village. Our noisy and bloated nature is fundamental to our diversity and ability to discover the unknown unknowns that the highly focused, productive and change adverse autistic mind would never come across or care about. Pigeons outperform humans at the Monty Hall Dilemma because they are less methodical.
What I’m trying to say is that the idea of superhuman intelligence is not as clear as it is portrayed here. Greg Egan may very well be right. That is not to say that once we learnt about a certain problem there isn’t a more effective way to solve it than using the human mind. But I wouldn’t bet my money on the kind of god-like intelligence that is somehow about to bootstrap itself out of the anthropocentric coding it emerged from.
So, what Greg Egan is saying is that the methods of epistemic rationality and creativity are all mostly known by humans, all we lack is memory space.
I sincerely doubt it. I genuinely believe Anna Salamon’s statement , that humans are only on the cusp of general intelligence, is closer to the truth.
EDITED : to add hyperlink to Anna Salamon’s article
What is productive cannot be judged in advance if you are facing unknown unknowns. And the very nature of scientific advances is an evolutionary process, not one of deliberate design but discovery. We may very well be able to speed up certain weak computational problems by sheer brute force but not solve problems by creating an advance problem solving machine.
That we do not ask ourselves what we are trying to achieve is a outstanding feature of our ability to learn and change our habits. If we would all be the productive machines that you might have in mind, then the religious people would stay religious and the bushman would keep striving for a successful chase even after they build a supermarket in front of his village. Our noisy and bloated nature is fundamental to our diversity and ability to discover the unknown unknowns that the highly focused, productive and change adverse autistic mind would never come across or care about. Pigeons outperform humans at the Monty Hall Dilemma because they are less methodical.
What I’m trying to say is that the idea of superhuman intelligence is not as clear as it is portrayed here. Greg Egan may very well be right. That is not to say that once we learnt about a certain problem there isn’t a more effective way to solve it than using the human mind. But I wouldn’t bet my money on the kind of god-like intelligence that is somehow about to bootstrap itself out of the anthropocentric coding it emerged from.