This reminds me how some people notice that superintelligent AI is just another version of Golem… but the same people fail to notice that the ordinary computers around us are already just another version of Golem.
Students of popular science [...] are always insisting that Christianity and Buddhism are very much alike [...] The reasons were of two kinds: resemblances that meant nothing because they were common to all humanity, and resemblances which were not resemblances at all. The author solemnly explained that the two creeds were alike in things in which all creeds are alike, or else he described them as alike in some point in which they are quite obviously different. [...] it was gravely urged that [Christ and Buddha], by a singular coincidence, both had to do with the washing of feet. You might as well say that it was a remarkable coincidence that they both had feet to wash.
Is there actually a version of the Golem tale where AI-risk is a theme? I had a look once and I couldn’t actually find a version where the Golem fastidiously follows its instructions beyond their intended meaning. Perhaps people are just confusing it with The Sorcerer’s Apprentice?
Until I actually looked into this so was I. In my case I think it’s Terry Pratchett’s fault. In Feet of Clay he describes Golems as being prone to continue with tasks forever unless told to stop.
Let us call this precise, instruction-following genie a Golem Genie. (A golem is a
creature from Jewish folklore that would in some stories do exactly as told [Idel 1990],
often with unintended consequences, for example polishing a dish until it is as thin as
paper [Pratchett 1996].)
(The “Idel” reference goes to Idel, Moshe. 1990. Golem: Jewish Magical and Mystical Traditions on the Artificial Anthropoid. SUNY Series
in Judaica. Albany: State University of New York Press.)
This reminds me how some people notice that superintelligent AI is just another version of Golem… but the same people fail to notice that the ordinary computers around us are already just another version of Golem.
Which further reminds me of Chesterton writing:
Is there actually a version of the Golem tale where AI-risk is a theme? I had a look once and I couldn’t actually find a version where the Golem fastidiously follows its instructions beyond their intended meaning. Perhaps people are just confusing it with The Sorcerer’s Apprentice?
Quite possibly; in which case I would also belong to the set of confused people.
Until I actually looked into this so was I. In my case I think it’s Terry Pratchett’s fault. In Feet of Clay he describes Golems as being prone to continue with tasks forever unless told to stop.
From the MIRI paper “Intelligence Explosion and Machine Ethics”:
(The “Idel” reference goes to Idel, Moshe. 1990. Golem: Jewish Magical and Mystical Traditions on the Artificial Anthropoid. SUNY Series in Judaica. Albany: State University of New York Press.)