So, Chollet is the author of the important deep-learning library Keras ( https://keras.io/), so at first glance there’s reason to take him seriously. But I don’t think this is a good essay. Some counter-arguments:
The No-Free-Lunch Theorem doesn’t imply that relatively general reasoners can’t exist; humans and animals are much more multipurpose than today’s machine learning algorithms.
“High-IQ humans aren’t ultra-powerful” doesn’t seem to generalize well at all to AI. I don’t know that IQ (a measure derived from psychometric tests) is even analogous to the kind of accuracy and performance metrics used in AI, though there’s probably some correlation (high-IQ humans and good ML algorithms both perform well at chess and go). Humans are affected by all kinds of phenomena like socioeconomic class and motivation which either aren’t relevant to computers or would be classified as a type of “intelligence” if they were implemented in computers. And humans are much more powerful than animals, which is a more relevant scale of “differences in brain capacity” to artificial intelligence than variation between humans.
″ Clearly, the intelligence of a single human, over a single lifetime, cannot design intelligence, or else, over billions of trials, it would have already occurred.” This is a fully general argument against any unsolved problem ever being solved in the future; it’s vacuous.
“no complex real-world system can be modeled as `X(t + 1) = X(t) * a, a > 1”—unless “complex” is doing a lot of work there, this is obviously false. There are plenty of exponential-growth phenomena visible in the real world. (Population growth and nuclear chain reactions, for instance.)
“The number of software developers has been booming exponentially for decades, and the number of transistors on which we are running our software has been exploding as well, following Moore’s law. Yet, our computers are only incrementally more useful to us than they were in 2012, or 2002, or 1992.” This is a subjective claim and not justified; but even if it’s true, it doesn’t mean that computer performance among various metrics can’t grow exponentially; it has! But there’s also declining marginal utility of any good (to humans) and Weber’s law of logarithmic perception scales; exponential improvements in computer performance will by default seem like only linear improvements in “usefulness” to us.
Basically, the only thing here which is a valid argument against an intelligence explosion is the last section, which mentions bottlenecks and antagonistic processes (like the fact that collaboration among more people is more difficult, so more workers doesn’t mean proportionately more progress.) This is basically Robin Hanson’s argument against FOOM, and is several years old, and Chollet doesn’t really add anything new here. As an argument considered in a vacuum, I don’t think this article provides any new reason to update away from believing in an intelligence explosion.
The fact that Chollet believes there won’t be an intelligence explosion is, of course, an update both against Chollet’s credibility on AI futurism (if you already think intelligence explosions are likely) and against the likelihood of an intelligence explosion (if you’re impressed with Chollet’s achievements), but that doesn’t tell you where belief propagation is going to converge, or even whether it will.
I really liked Chollet’s earlier essay, The Future of Deep Learning, which is more technical and agrees with a lot of the conclusions that I came to independently. I’m inclined to believe that Chollet writing for a general audience on Medium may be practicing propaganda, while his more concrete futurist predictions seem very credible.
In the above article, he says, “As such, this perpetually-learning model-growing system could be interpreted as an AGI—an Artificial General Intelligence. But don’t expect any singularitarian robot apocalypse to ensue: that’s a pure fantasy, coming from a long series of profound misunderstandings of both intelligence and technology. This critique, however, does not belong here.”
You could take this to mean something like “Yes, AGI is possible and I have just laid out a rough path to getting there; but I want to strongly disaffiliate with science-fiction geeks.”
I agree with all your criticisms. I also think the article is wrong and didn’t update except against Chollet, but I found the article educational.
Things I learned.
No free lunch theorem. I’m very grateful for this, and it made me start learning more about optimisation.
From the above: there is no general intelligence. I had previously believed that finding a God’s algorithm for optimisation would be one of the most significant achievements of the century. I discovered that’s impossible.
Exponential growth does not imply exponential progress as exponential growth may meet exponential bottlenecks. This was also something I didn’t appreciate. Upgrading from a level n intelligence to a level n+1 intelligence may require more relative intelligence than upgrading from a level n-1 to a level n. Exponential bottlenecks may result in diminishing marginal growth of intelligence.
The article may have seemed of significant pedagogical vale to me, because I hadn’t met these ideas before. For example, I have just started reading the Yudkowsky-Hanson AI foom debate.
So, Chollet is the author of the important deep-learning library Keras ( https://keras.io/), so at first glance there’s reason to take him seriously. But I don’t think this is a good essay. Some counter-arguments:
The No-Free-Lunch Theorem doesn’t imply that relatively general reasoners can’t exist; humans and animals are much more multipurpose than today’s machine learning algorithms.
“High-IQ humans aren’t ultra-powerful” doesn’t seem to generalize well at all to AI. I don’t know that IQ (a measure derived from psychometric tests) is even analogous to the kind of accuracy and performance metrics used in AI, though there’s probably some correlation (high-IQ humans and good ML algorithms both perform well at chess and go). Humans are affected by all kinds of phenomena like socioeconomic class and motivation which either aren’t relevant to computers or would be classified as a type of “intelligence” if they were implemented in computers. And humans are much more powerful than animals, which is a more relevant scale of “differences in brain capacity” to artificial intelligence than variation between humans.
″ Clearly, the intelligence of a single human, over a single lifetime, cannot design intelligence, or else, over billions of trials, it would have already occurred.” This is a fully general argument against any unsolved problem ever being solved in the future; it’s vacuous.
“no complex real-world system can be modeled as `X(t + 1) = X(t) * a, a > 1”—unless “complex” is doing a lot of work there, this is obviously false. There are plenty of exponential-growth phenomena visible in the real world. (Population growth and nuclear chain reactions, for instance.)
“The number of software developers has been booming exponentially for decades, and the number of transistors on which we are running our software has been exploding as well, following Moore’s law. Yet, our computers are only incrementally more useful to us than they were in 2012, or 2002, or 1992.” This is a subjective claim and not justified; but even if it’s true, it doesn’t mean that computer performance among various metrics can’t grow exponentially; it has! But there’s also declining marginal utility of any good (to humans) and Weber’s law of logarithmic perception scales; exponential improvements in computer performance will by default seem like only linear improvements in “usefulness” to us.
Basically, the only thing here which is a valid argument against an intelligence explosion is the last section, which mentions bottlenecks and antagonistic processes (like the fact that collaboration among more people is more difficult, so more workers doesn’t mean proportionately more progress.) This is basically Robin Hanson’s argument against FOOM, and is several years old, and Chollet doesn’t really add anything new here.
As an argument considered in a vacuum, I don’t think this article provides any new reason to update away from believing in an intelligence explosion.
The fact that Chollet believes there won’t be an intelligence explosion is, of course, an update both against Chollet’s credibility on AI futurism (if you already think intelligence explosions are likely) and against the likelihood of an intelligence explosion (if you’re impressed with Chollet’s achievements), but that doesn’t tell you where belief propagation is going to converge, or even whether it will.
I really liked Chollet’s earlier essay, The Future of Deep Learning, which is more technical and agrees with a lot of the conclusions that I came to independently. I’m inclined to believe that Chollet writing for a general audience on Medium may be practicing propaganda, while his more concrete futurist predictions seem very credible.
In the above article, he says, “As such, this perpetually-learning model-growing system could be interpreted as an AGI—an Artificial General Intelligence. But don’t expect any singularitarian robot apocalypse to ensue: that’s a pure fantasy, coming from a long series of profound misunderstandings of both intelligence and technology. This critique, however, does not belong here.”
You could take this to mean something like “Yes, AGI is possible and I have just laid out a rough path to getting there; but I want to strongly disaffiliate with science-fiction geeks.”
I agree with all your criticisms. I also think the article is wrong and didn’t update except against Chollet, but I found the article educational.
Things I learned.
No free lunch theorem. I’m very grateful for this, and it made me start learning more about optimisation.
From the above: there is no general intelligence. I had previously believed that finding a God’s algorithm for optimisation would be one of the most significant achievements of the century. I discovered that’s impossible.
Exponential growth does not imply exponential progress as exponential growth may meet exponential bottlenecks. This was also something I didn’t appreciate. Upgrading from a level n intelligence to a level n+1 intelligence may require more relative intelligence than upgrading from a level n-1 to a level n. Exponential bottlenecks may result in diminishing marginal growth of intelligence.
The article may have seemed of significant pedagogical vale to me, because I hadn’t met these ideas before. For example, I have just started reading the Yudkowsky-Hanson AI foom debate.