You’re entirely right that taking Herbert’s views on most specific subjects isn’t helpful. He was wrong about genetics, about education, and about a lot of things besides. (Though like moridinamael, I’m also not clear on whether he personally believed in things like genetic memory, though I would be interested to see sources if you have them. I assumed that it was an element he included for fictional/allegorical purposes.) But I think he was a clever guy who spent a lot of time thinking about problems we’re interested in, even if he often got it wrong.
I think it’s a little harsh to say that I commit the error of reading-in my beliefs about the Butlerian Jihad, given that I quote Reverend Mother Gaius Helen Mohiam as saying, “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them,” and Leto II as saying, “The target of the Jihad was a machine-attitude as much as the machines.” I’m aware that there are a lot of textual clues that the Jihad wasn’t a war against autonomous machines themselves. Though autonomous machines were certainly involved; the glossary to the original book defines the Butlerian Jihad as, “the crusade against computers, thinking machines, and conscious robots”, and the OC Bible’s commandment is, “Thou shalt not make a machine in the likeness of a human mind.”
More generally, I was using the Jihad as a metaphor to make a point about automation in general.
It’s clear that Strong AI is illegal under the prohibition of “thinking machines”, but it had always puzzled me why lesser devices — like calculators and recording devices — were included. I had passed it off as another mistake on Herbert’s part. But when I read Nabil’s comment it reminded me strongly of the Jihad, and I realized that if taken to an extreme conclusion it would lead to a proscription against almost all automation, like the one we find in Dune. Consider it a steelman of the position, if you would like.
Just because I quote Samuel Butler at the end, doesn’t mean I think the Jihad was named after him! It’s just an amusing coincidence.
Looking forward to reading your essay on the Genetics of Dune!
Though like moridinamael, I’m also not clear on whether he personally believed in things like genetic memory, though I would be interested to see sources if you have them. I assumed that it was an element he included for fictional/allegorical purposes.
Yes, we shouldn’t assume a SF author endorsed any speculative proto/pseudo-science he includes. But in the case of genetic memory, we can be fairly sure that he ‘believed in it’ in the sense that he took it way more seriously than you or I and considered it a live hypothesis because he says so explicitly in an interview I quote in the essay: he thinks genetic memory and pheromones, or something much like them, is necessary to explain things like the cohesion of mob & social groups like aristocracies without explicit obvious status markers, or the supposed generational patterns of warfare ‘spasms’ (this is a reference to the obscure crankery of The Sexual Cycle of Human Warfare† which apparently deeply influenced Herbert and you won’t understand all the references/influences unless you at least look at an overview of it because it’s so lulzy).
Reading back, I see I got sidetracked and didn’t resolve your main point about why the Butlerian Jihad targeted all software. The one-line explanation is: permitting any software is an existential risk because it is a crutch which will cripple humanity’s long-term growth throughout the universe, leaving us vulnerable to the inevitable black swans (not necessarily AI).
First, you should read my essay, and especially that Herbert interview and the Spinrad democracy footnote and if you have the time, Herbert’s attitude towards computers & software is most revealed in Without Me You’re Nothing, which is a very strange artifact: his 1980 technical guide/book on programming PCs of that era—leaving aside the wildly outdated information which you can skip over, the interesting parts are his essays or commentaries on PCs in general, which convey his irascible humanist libertarian attitude on PCs as being a democratizing and empowering force for independent-human growth. Herbert was quite a PC enthusiast: beyond writing a whole book about how to use them, his farmstead apparently had rigged up all sorts of gadgets and ‘home automation’ he had made as a hobby to help him farm and, at least in theory, be more independent & capable & a Renaissance man. (Touponce is also well worth reading.) There’s a lot of supporting information in those I won’t try to get into here which I think support my generalizations below.
So, your basic error is that you are wrong about the BJ not being about AI or existential-risk per se. The BJ here is in fact about existential-risk from Herbert’s POV; it’s just that it’s much more indirect than you are thinking. It has nothing to do with signaling or arms-races. Herbert’s basic position is that machines (like PCs), ‘without me [the living creative human user], they are nothing’: they are dead, uncreative, unable to improvise or grow, and constraining. (At least without a level of strong AI he considered centuries or millennia away & to require countless fundamental breakthroughs.) They lock humans into fixed patterns. And to Herbert, this fixedness is death. It is death, sooner or later, perhaps many millennium later, but death nevertheless; and [human] life is jazz:
In all of my universe I have seen no law of nature, unchanging and inexorable. This universe presents only changing relationships which are sometimes seen as laws by short-lived awareness. These fleshly sensoria which we call self are ephemera withering in the blaze of infinity, fleetingly aware of temporary conditions which confine our activities and change as our activities change. If you must label the absolute, use its proper name: “Temporary”.
Or
The person who takes the banal and ordinary and illuminates it in a new way can terrify. We do not want our ideas changed. We feel threatened by such demands. ‘I already know the important things!’ we say. Then Changer comes and throws our old ideas away.
And
Odrade pushed such thoughts aside. There were things to do on the crossing. None of them more important than gathering her energies. Honored Matres could be analyzed almost out of reality, but the actual confrontation would be played as it came—a jazz performance. She liked the idea of jazz although the music distracted her with its antique flavors and the dips into wildness. Jazz spoke about life, though. No two performances ever identical. Players reacted to what was received from the others: jazz. Feed us with jazz.
(‘Muad’dib’s first lesson was how to learn’/‘the wise man shapes himself, the fool lives only to die’ etc etc)
Whether it’s some space plague or space aliens or sterility or decadence or civil war or spice running out or thinking machines far in the future, it doesn’t matter, because the universe will keep changing, and humans mentally enslaved to, and dependent on, their thinking machines, would not. Their abilities will be stunted and wither away, they will fail to adapt and evolve and grow and gain capabilities like prescience. (Even if the thinking-machines survive whatever doomsday inevitably comes, who cares? They aren’t humans. Certainly Herbert doesn’t care about AIs, he’s all about humanity.) And sooner or later—gambler’s ruin—there will be something and humanity will go extinct. Unless they strengthen themselves and enter into the infinite open universe, abandoning delusions about certainty or immortality or reducing everything to simple rules.
That is why the BJ places the emphasis on banning anything that serves as a crutch for humans, mechanizing their higher life.* It’s fine to use a forklift or a spaceship, humans were never going to hoist a 2-ton pallet or flap their wings to fly the galaxy and those tools extend their abilities; it’s not fine to ask a computer for an optimal Five-Year Plan for the economy or to pilot the space ship because now it’s replacing the human role. The strictures force the development of mentats, Reverend Mothers, Navigators, Face Dancers, sword-masters, and so on and so force, all of which eventually merge in the later books, evolving super-capable humans who can Scatter across the universe, evading ever new and more dangerous enemies, ensuring that humanity never goes extinct, never gets lazy, and someday will become, as the Bene Gesserit put it, ‘adults’, who presumably can discard all the feudal trippery and stand as mature independent equals in fully democratic societies.
As you can see, this has little to do with Confucianism or the stasis being intrinsically desirable or it being a good thing to remove all bureaucracy (bureaucracy is just a tool, like any other, to be used skillfully) or indeed all automation etc.
You’re entirely right that taking Herbert’s views on most specific subjects isn’t helpful. He was wrong about genetics, about education, and about a lot of things besides. (Though like moridinamael, I’m also not clear on whether he personally believed in things like genetic memory, though I would be interested to see sources if you have them. I assumed that it was an element he included for fictional/allegorical purposes.) But I think he was a clever guy who spent a lot of time thinking about problems we’re interested in, even if he often got it wrong.
I think it’s a little harsh to say that I commit the error of reading-in my beliefs about the Butlerian Jihad, given that I quote Reverend Mother Gaius Helen Mohiam as saying, “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them,” and Leto II as saying, “The target of the Jihad was a machine-attitude as much as the machines.” I’m aware that there are a lot of textual clues that the Jihad wasn’t a war against autonomous machines themselves. Though autonomous machines were certainly involved; the glossary to the original book defines the Butlerian Jihad as, “the crusade against computers, thinking machines, and conscious robots”, and the OC Bible’s commandment is, “Thou shalt not make a machine in the likeness of a human mind.”
More generally, I was using the Jihad as a metaphor to make a point about automation in general.
It’s clear that Strong AI is illegal under the prohibition of “thinking machines”, but it had always puzzled me why lesser devices — like calculators and recording devices — were included. I had passed it off as another mistake on Herbert’s part. But when I read Nabil’s comment it reminded me strongly of the Jihad, and I realized that if taken to an extreme conclusion it would lead to a proscription against almost all automation, like the one we find in Dune. Consider it a steelman of the position, if you would like.
Just because I quote Samuel Butler at the end, doesn’t mean I think the Jihad was named after him! It’s just an amusing coincidence.
Looking forward to reading your essay on the Genetics of Dune!
Yes, we shouldn’t assume a SF author endorsed any speculative proto/pseudo-science he includes. But in the case of genetic memory, we can be fairly sure that he ‘believed in it’ in the sense that he took it way more seriously than you or I and considered it a live hypothesis because he says so explicitly in an interview I quote in the essay: he thinks genetic memory and pheromones, or something much like them, is necessary to explain things like the cohesion of mob & social groups like aristocracies without explicit obvious status markers, or the supposed generational patterns of warfare ‘spasms’ (this is a reference to the obscure crankery of The Sexual Cycle of Human Warfare† which apparently deeply influenced Herbert and you won’t understand all the references/influences unless you at least look at an overview of it because it’s so lulzy).
Reading back, I see I got sidetracked and didn’t resolve your main point about why the Butlerian Jihad targeted all software. The one-line explanation is: permitting any software is an existential risk because it is a crutch which will cripple humanity’s long-term growth throughout the universe, leaving us vulnerable to the inevitable black swans (not necessarily AI).
First, you should read my essay, and especially that Herbert interview and the Spinrad democracy footnote and if you have the time, Herbert’s attitude towards computers & software is most revealed in Without Me You’re Nothing, which is a very strange artifact: his 1980 technical guide/book on programming PCs of that era—leaving aside the wildly outdated information which you can skip over, the interesting parts are his essays or commentaries on PCs in general, which convey his irascible humanist libertarian attitude on PCs as being a democratizing and empowering force for independent-human growth. Herbert was quite a PC enthusiast: beyond writing a whole book about how to use them, his farmstead apparently had rigged up all sorts of gadgets and ‘home automation’ he had made as a hobby to help him farm and, at least in theory, be more independent & capable & a Renaissance man. (Touponce is also well worth reading.) There’s a lot of supporting information in those I won’t try to get into here which I think support my generalizations below.
So, your basic error is that you are wrong about the BJ not being about AI or existential-risk per se. The BJ here is in fact about existential-risk from Herbert’s POV; it’s just that it’s much more indirect than you are thinking. It has nothing to do with signaling or arms-races. Herbert’s basic position is that machines (like PCs), ‘without me [the living creative human user], they are nothing’: they are dead, uncreative, unable to improvise or grow, and constraining. (At least without a level of strong AI he considered centuries or millennia away & to require countless fundamental breakthroughs.) They lock humans into fixed patterns. And to Herbert, this fixedness is death. It is death, sooner or later, perhaps many millennium later, but death nevertheless; and [human] life is jazz:
Or
And
(‘Muad’dib’s first lesson was how to learn’/‘the wise man shapes himself, the fool lives only to die’ etc etc)
Whether it’s some space plague or space aliens or sterility or decadence or civil war or spice running out or thinking machines far in the future, it doesn’t matter, because the universe will keep changing, and humans mentally enslaved to, and dependent on, their thinking machines, would not. Their abilities will be stunted and wither away, they will fail to adapt and evolve and grow and gain capabilities like prescience. (Even if the thinking-machines survive whatever doomsday inevitably comes, who cares? They aren’t humans. Certainly Herbert doesn’t care about AIs, he’s all about humanity.) And sooner or later—gambler’s ruin—there will be something and humanity will go extinct. Unless they strengthen themselves and enter into the infinite open universe, abandoning delusions about certainty or immortality or reducing everything to simple rules.
That is why the BJ places the emphasis on banning anything that serves as a crutch for humans, mechanizing their higher life.* It’s fine to use a forklift or a spaceship, humans were never going to hoist a 2-ton pallet or flap their wings to fly the galaxy and those tools extend their abilities; it’s not fine to ask a computer for an optimal Five-Year Plan for the economy or to pilot the space ship because now it’s replacing the human role. The strictures force the development of mentats, Reverend Mothers, Navigators, Face Dancers, sword-masters, and so on and so force, all of which eventually merge in the later books, evolving super-capable humans who can Scatter across the universe, evading ever new and more dangerous enemies, ensuring that humanity never goes extinct, never gets lazy, and someday will become, as the Bene Gesserit put it, ‘adults’, who presumably can discard all the feudal trippery and stand as mature independent equals in fully democratic societies.
As you can see, this has little to do with Confucianism or the stasis being intrinsically desirable or it being a good thing to remove all bureaucracy (bureaucracy is just a tool, like any other, to be used skillfully) or indeed all automation etc.
* I suspect that there’s a similar idea behind ‘BuSab’ in his ConSentiency universe, but TBH, I find those novels/stories too boring to read carefully.
† 183MB color scan:
https://www.gwern.net/docs/sociology/1950-walter-thesexualcycleofhumanwarfare.pdf