tl;dr: It seems noteworthy that “deepware” has strong connotations with “it involves magic”, while the same is not true for AI in general.
I would like to point out one thing regarding the software vs AI distinction that is confusing me a bit. (I view this as complementing, rather than contradicting, your post.)
As we go along the progression “Tools > Machines > Electric > Electronic > Digital”, most[1] of the examples can be viewed as automating a reasonably-well-understood process, on a progressively higher level of abstraction.[2] [For example: A hammer does basically no automation. > A machine like a lawn-mower automates a rigidly-designed rotation of the blades. > An electric kettle does-its-thingy. > An electronic calculator automates calculating algorithms that we understand, but can do it for much larger inputs than we could handle. > An algorithm like Monte Carlo tree search automates an abstract reasoning process that we understand, but can apply it to a wide range of domains.]
But then it seems that this progression does not neatly continue to the AI paradigm. Or rather, some things that we call AI can be viewed as a continuation of this progression, while others can’t (or would constitute a discontinuous jump). [For example, approaches like “solving problems using HCH” (minus the part where you use unknown magic to obtain a black box that imitates the human) can be viewed as automating a reasonably-well-understood process (of solving tasks by decomposing & delegating them). But there are also other things that we call AI that are not well described as a continuation of this progression—or perhaps they constitute a rather extreme jump. On the other hand, deep learning automates the not-well-understood process of “stare at many things, then use magic to generalise”. And the other example is abstract optimisation, which automates the not-well-understood process of “search through many potential solutions and pick the one that scores the best according to an objective function”. And there are examples that lie somewhere inbetween—for example, AlphaZero is mostly a quite well-understood process, but it does involve some opaque deep learning.]
I suppose we could refer to the distinction as “does it involve magic?”. It then seems noteworthy that “deepware” has strong connotations with magic, while the same isn’t true for all types of AI.[2]
An interesting open question is how large hit to competitiveness would we suffer if we restricted ourselves to systems that only involve a small amount of magic.
Yeah, this is a pretty interesting twist in the progression, and one which I failed to see coming as a teenager learning about AI. I looked at the trend from concrete to abstract—from machine-code to structured programming to ever-more-abstract high-level programming languages—and I thought AI would look like the highest-level programming language one could imagine.
In some sense this is not wrong. Telling the machine what to do in plain natural language is the highest-level programming language one could imagine.
However, naive extrapolation of ever-more-sophisticated programming languages might lead one to anticipate convergence between compilers and computational linguistics, such that computers would be understanding natural language with sophisticated but well-understood parsing algorithms, converting natural-language statements to formal representations resembling logic, and then executing the commands via similarly sophisticated planning algorithms.
The reality is that computational linguistics itself has largely abandoned the idea that we can make a formal grammar which captures natural language; the best way to parse a bunch of English is, instead, to let machine learning “get the idea” from a large number of hand-parsed examples! Rather than bridging the formal-informal divide by fully formalizing English grammar, it turns out to be easier to formalize informality itself (ie, mathematically specify a model of messy neural network learning) and then throw the formalized informality at the problem!
Weird stuff.
However, at some point I did get the idea and make the update. I think it was at the 2012 AGI conference, where someone was presenting a version of neural networks which was supposed to learn interpretable models, due to the individual neurons implementing interpretable functions of their inputs, rather than big weighted sums with a nonlinear transform thrown in. It seemed obvious that the approach would be hopeless, because as the models got larger and larger, it would be no more interpretable than any other form of neural network. I had the startling realization that this same argument seems to apply to anything, no matter how logic-like the underlying representation: it will become an opaque mess as it learns the high complexity of the real world.
It seems relevant, to the progression, that a lot of human problem solving—though not all—is done by the informal method of “getting exposed to examples and then, somehow, generalising”. (And I likewise failed to appreciate this, not sure until when.) This suggests that if we want to build AI that solves things in similar ways that humans solve them, “magic”-involving “deepware” is a natural step. (Whether building AI in the image of humans is desirable, that’s a different topic.)
tl;dr: It seems noteworthy that “deepware” has strong connotations with “it involves magic”, while the same is not true for AI in general.
I would like to point out one thing regarding the software vs AI distinction that is confusing me a bit. (I view this as complementing, rather than contradicting, your post.)
As we go along the progression “Tools > Machines > Electric > Electronic > Digital”, most[1] of the examples can be viewed as automating a reasonably-well-understood process, on a progressively higher level of abstraction.[2]
[For example: A hammer does basically no automation. > A machine like a lawn-mower automates a rigidly-designed rotation of the blades. > An electric kettle does-its-thingy. > An electronic calculator automates calculating algorithms that we understand, but can do it for much larger inputs than we could handle. > An algorithm like Monte Carlo tree search automates an abstract reasoning process that we understand, but can apply it to a wide range of domains.]
But then it seems that this progression does not neatly continue to the AI paradigm. Or rather, some things that we call AI can be viewed as a continuation of this progression, while others can’t (or would constitute a discontinuous jump).
[For example, approaches like “solving problems using HCH” (minus the part where you use unknown magic to obtain a black box that imitates the human) can be viewed as automating a reasonably-well-understood process (of solving tasks by decomposing & delegating them). But there are also other things that we call AI that are not well described as a continuation of this progression—or perhaps they constitute a rather extreme jump. On the other hand, deep learning automates the not-well-understood process of “stare at many things, then use magic to generalise”. And the other example is abstract optimisation, which automates the not-well-understood process of “search through many potential solutions and pick the one that scores the best according to an objective function”. And there are examples that lie somewhere inbetween—for example, AlphaZero is mostly a quite well-understood process, but it does involve some opaque deep learning.]
I suppose we could refer to the distinction as “does it involve magic?”. It then seems noteworthy that “deepware” has strong connotations with magic, while the same isn’t true for all types of AI.[2]
Or perhaps just “many”? I am not quite sure, this would require going through more examples, and I was intending for this to be a quick comment.
To be clear, I am not super-confident that this progression is a legitimate phenomenon. But for the sake of argument, let’s say it is.
An interesting open question is how large hit to competitiveness would we suffer if we restricted ourselves to systems that only involve a small amount of magic.
Yeah, this is a pretty interesting twist in the progression, and one which I failed to see coming as a teenager learning about AI. I looked at the trend from concrete to abstract—from machine-code to structured programming to ever-more-abstract high-level programming languages—and I thought AI would look like the highest-level programming language one could imagine.
In some sense this is not wrong. Telling the machine what to do in plain natural language is the highest-level programming language one could imagine.
However, naive extrapolation of ever-more-sophisticated programming languages might lead one to anticipate convergence between compilers and computational linguistics, such that computers would be understanding natural language with sophisticated but well-understood parsing algorithms, converting natural-language statements to formal representations resembling logic, and then executing the commands via similarly sophisticated planning algorithms.
The reality is that computational linguistics itself has largely abandoned the idea that we can make a formal grammar which captures natural language; the best way to parse a bunch of English is, instead, to let machine learning “get the idea” from a large number of hand-parsed examples! Rather than bridging the formal-informal divide by fully formalizing English grammar, it turns out to be easier to formalize informality itself (ie, mathematically specify a model of messy neural network learning) and then throw the formalized informality at the problem!
Weird stuff.
However, at some point I did get the idea and make the update. I think it was at the 2012 AGI conference, where someone was presenting a version of neural networks which was supposed to learn interpretable models, due to the individual neurons implementing interpretable functions of their inputs, rather than big weighted sums with a nonlinear transform thrown in. It seemed obvious that the approach would be hopeless, because as the models got larger and larger, it would be no more interpretable than any other form of neural network. I had the startling realization that this same argument seems to apply to anything, no matter how logic-like the underlying representation: it will become an opaque mess as it learns the high complexity of the real world.
Agreed.
It seems relevant, to the progression, that a lot of human problem solving—though not all—is done by the informal method of “getting exposed to examples and then, somehow, generalising”. (And I likewise failed to appreciate this, not sure until when.) This suggests that if we want to build AI that solves things in similar ways that humans solve them, “magic”-involving “deepware” is a natural step. (Whether building AI in the image of humans is desirable, that’s a different topic.)