I agree with your entire first paragraph. It doesn’t seem to me that you have addressed my question though. You are claiming that this hypothesis “implies that machine learning alone is not a complete path to human-level intelligence.” I disagree. If I try to design an ML model which can identify primes, is it fair for me to give it some information equivalent to the definition (no more information than a human who has never heard of prime numbers has)?
If you allow that it is fair for me to do so, I think I can probably design an ML model which will do this. If you do not allow this, then I don’t think this hypothesis has any bearing on whether ML alone is “a complete path to human-level intelligence.” (Unless you have a way of showing that humans who have never received any sensory data other than a sequence of “number:(prime/composite)label” pairs would do well on this.)
Does any ML model that tells cats from dogs get definitions thereof? I think the only input it gets is “picture:(dog/cat)label”. It does learn to tell them apart, to some degree, at least. One would expect the same approach here. Otherwise you can ask right away for the sieve of Eratosthenes as a functional and inductive definition, in which case things get easy …
In that case, I believe your conjecture is trivially true, but has nothing to do with human intelligence or Bengio’s statements. In context, he is explicitly discussing low dimensional representations of extremely high dimensional data, and the things human brains learn to do automatically (I would say analogously to a single forward pass).
If you want to make it a fair fight, you either need to demonstrate a human who learns to recognize primes without any experience of the physical world (please don’t do this) or allow an ML model something more analogous to the data humans actually receive, which includes math instruction, interacting with the world, many brain cycles, etc
Regarding your remark on finding low-dimensional representations, I have added a section on physical intuitions for the challenge. Here I explain how the prime recognition problem corresponds to reliably finding a low-dimensional representation of high-dimensional data.
I agree with your entire first paragraph. It doesn’t seem to me that you have addressed my question though. You are claiming that this hypothesis “implies that machine learning alone is not a complete path to human-level intelligence.” I disagree. If I try to design an ML model which can identify primes, is it fair for me to give it some information equivalent to the definition (no more information than a human who has never heard of prime numbers has)?
If you allow that it is fair for me to do so, I think I can probably design an ML model which will do this. If you do not allow this, then I don’t think this hypothesis has any bearing on whether ML alone is “a complete path to human-level intelligence.” (Unless you have a way of showing that humans who have never received any sensory data other than a sequence of “number:(prime/composite)label” pairs would do well on this.)
Does any ML model that tells cats from dogs get definitions thereof? I think the only input it gets is “picture:(dog/cat)label”. It does learn to tell them apart, to some degree, at least. One would expect the same approach here. Otherwise you can ask right away for the sieve of Eratosthenes as a functional and inductive definition, in which case things get easy …
In that case, I believe your conjecture is trivially true, but has nothing to do with human intelligence or Bengio’s statements. In context, he is explicitly discussing low dimensional representations of extremely high dimensional data, and the things human brains learn to do automatically (I would say analogously to a single forward pass).
If you want to make it a fair fight, you either need to demonstrate a human who learns to recognize primes without any experience of the physical world (please don’t do this) or allow an ML model something more analogous to the data humans actually receive, which includes math instruction, interacting with the world, many brain cycles, etc
I also believe my conjecture is true, however non-trivially. At least, mathematically non-trivially. Otherwise, all is trivial when the job is done.
I also believe my conjecture is true, however non-trivially. At least, mathematically non-trivially. Otherwise, all is trivial while the job is done.
Regarding your remark on finding low-dimensional representations, I have added a section on physical intuitions for the challenge. Here I explain how the prime recognition problem corresponds to reliably finding a low-dimensional representation of high-dimensional data.