I don’t care about that specific formulation of the idea; maybe Robin Hanson’s formulation that there exists no “grand unified theory of intelligence” is clearer? (link)
After all, we seem to have little reason to expect there is a useful grand unified theory of betterness to discover, beyond what we already know. “Betterness” seems mostly a concept about us and what we want – why should it correspond to something out there about which we can make powerful discoveries?
...but the answer seems simple. A big part of “betterness” is the ability to perform inductive inference, which is not a human-specific concept. We do already have a powerful theory about that, which we discovered in the last 50 years. It doesn’t immediately suggest implementation strategy—which is what we need. So: more discoveries relating to this seem likely.
To me it seems a lot like the question of whether general, computationally tractable methods of compression exist.
Provided you are allowed to assume that the expected inputs obey some vaguely-sensible version of Occam’s razor, I would say that the answer is just “yes, they do”.
I don’t care about that specific formulation of the idea; maybe Robin Hanson’s formulation that there exists no “grand unified theory of intelligence” is clearer? (link)
Clear—but also clearly wrong. Robin Hanson says:
...but the answer seems simple. A big part of “betterness” is the ability to perform inductive inference, which is not a human-specific concept. We do already have a powerful theory about that, which we discovered in the last 50 years. It doesn’t immediately suggest implementation strategy—which is what we need. So: more discoveries relating to this seem likely.
Clearly, I do not understand how this data point should influence my estimate of the probablity that general, computationally tractable methods exist.
To me it seems a lot like the question of whether general, computationally tractable methods of compression exist.
Provided you are allowed to assume that the expected inputs obey some vaguely-sensible version of Occam’s razor, I would say that the answer is just “yes, they do”.