If you want to define “a shoggoth” as “a pile of contextually activated masks”, then I think we’re in agreement about what an LLM is. But I worry that what people usually hear when we talk about “a shoggoth” is “a single cohesive mind that is alien to ours in a straightforward way (like wanting to maximize paperclips)”. For instance, in the Eliezer Yudkowsky tweet “can I please speak to the shoggoth” I think the tweet makes more sense if you read it as “can I please speak to [the true mind in the LLM]” instead of “can I please speak to [the pile of contextually activated masks]”.
The whole point of “shoggoth” as a metaphors is that shoggoths are really fucking weird, and if you understood them you’d go insane. This is compatible with most ways a LLM’s true nature could turn out to be, I think.
I agree that this is a conceptually weird thing that is likely to get reduced to something simple-enough-for-people-to-easily-understand, but, the problem there is that people don’t know what shoggoths are, not that shoggoths were a bad metaphor.
But I’m asking if an LLM even has a “true nature”, if (as Yudkowsky says here) there’s an “actual shoggoth” with “a motivation Z”. Do we have evidence there is such a true nature or underlying motivation? In the alternative “pile of masks” analogy its clear that there is no privileged identity in the LLM which is the “true nature”, whereas the “shoggoth wearing a mask” analogy makes it seem like there is some singular entity behind it all.
To be clear, I think “shoggoth with a mask” is still a good analogy in that it gets you a lot of understanding in few words, I’m just trying to challenge one implication of the analogy that I don’t think people have actually debated.
I think you’re still not really getting what a shoggoth is and why it was used in the metaphor. (like “a pile of masks” is totally a thing that a shoggoth might turn out to be, if we could understand it, which we can’t).
I do think “most people don’t understand lovecraftian mythology and are likely to be misunderstanding this meme” is totally a reasonable argument.
And “stop anthropomorphizing the thing-under-the-mask” is also totally reasonable.
(I talked to someone today about how shoggoths are alien intelligences that are so incomprehensible they’d drive humans insane if they tried to understand the,”, and the person said “oh, okay I thought it was just, like, a generic monster.” No, it’s not a generic monster. The metaphor was chosen for a specific reason, which is that it’s a type of monster that doesn’t have motivations the way humans have motivations)
((I do wanna flag, obviously the nuances of lovecraftian mythology aren’t actually important,. It just happened that other people in the comments had covered the real technical arguments pretty well already and my marginal contribution here was “be annoyed someone was wrong on the internet about what-a-shoggoth-is”))
In the alternative “pile of masks” analogy its clear that there is no privileged identity in the LLM which is the “true nature”
I do think this feels like it’s still getting something confused. Like, see Ronny Fernandez’s description.. There is a “true nature” – it’s “whatever processes turn out to be predicting the next token”. It’s nature is just not anything like a human, or even a simulation of human. It’s just a process.
There is a “true nature” – it’s “whatever processes turn out to be predicting the next token”.
I’d agree to this in the same way the true nature of our universe is the laws of physics. (Would you consider the laws of physics a shoggoth?) My concern when people jump from that to “oh so there’s a God (of Physics)”.
I think the crux of my issue is what our analogy says to answer the question “a powerful LLM could coexist with humanity” (I lean towards yes). When people read shoggoths in the non-canonical way as “a weird alien” I think they conclude no. But if its more like a physics simulator or a pile of masks, then as long as you have it simulating benign things or wearing friendly masks, the answer is yes. I’ll leave it to someone who actually read lovecraft to say whether humanity could coexist with canonical shoggoths :)
If you want to define “a shoggoth” as “a pile of contextually activated masks”, then I think we’re in agreement about what an LLM is. But I worry that what people usually hear when we talk about “a shoggoth” is “a single cohesive mind that is alien to ours in a straightforward way (like wanting to maximize paperclips)”. For instance, in the Eliezer Yudkowsky tweet “can I please speak to the shoggoth” I think the tweet makes more sense if you read it as “can I please speak to [the true mind in the LLM]” instead of “can I please speak to [the pile of contextually activated masks]”.
The whole point of “shoggoth” as a metaphors is that shoggoths are really fucking weird, and if you understood them you’d go insane. This is compatible with most ways a LLM’s true nature could turn out to be, I think.
I agree that this is a conceptually weird thing that is likely to get reduced to something simple-enough-for-people-to-easily-understand, but, the problem there is that people don’t know what shoggoths are, not that shoggoths were a bad metaphor.
But I’m asking if an LLM even has a “true nature”, if (as Yudkowsky says here) there’s an “actual shoggoth” with “a motivation Z”. Do we have evidence there is such a true nature or underlying motivation? In the alternative “pile of masks” analogy its clear that there is no privileged identity in the LLM which is the “true nature”, whereas the “shoggoth wearing a mask” analogy makes it seem like there is some singular entity behind it all.
To be clear, I think “shoggoth with a mask” is still a good analogy in that it gets you a lot of understanding in few words, I’m just trying to challenge one implication of the analogy that I don’t think people have actually debated.
I think you’re still not really getting what a shoggoth is and why it was used in the metaphor. (like “a pile of masks” is totally a thing that a shoggoth might turn out to be, if we could understand it, which we can’t).
I do think “most people don’t understand lovecraftian mythology and are likely to be misunderstanding this meme” is totally a reasonable argument.
And “stop anthropomorphizing the thing-under-the-mask” is also totally reasonable.
(I talked to someone today about how shoggoths are alien intelligences that are so incomprehensible they’d drive humans insane if they tried to understand the,”, and the person said “oh, okay I thought it was just, like, a generic monster.” No, it’s not a generic monster. The metaphor was chosen for a specific reason, which is that it’s a type of monster that doesn’t have motivations the way humans have motivations)
((I do wanna flag, obviously the nuances of lovecraftian mythology aren’t actually important,. It just happened that other people in the comments had covered the real technical arguments pretty well already and my marginal contribution here was “be annoyed someone was wrong on the internet about what-a-shoggoth-is”))
I do think this feels like it’s still getting something confused. Like, see Ronny Fernandez’s description.. There is a “true nature” – it’s “whatever processes turn out to be predicting the next token”. It’s nature is just not anything like a human, or even a simulation of human. It’s just a process.
I think I’ll retreat to this since I haven’t actually read the original lovecraft work. But also, once enough people have a misconception, it can be a bad medium for communication. (Shoggoths are also public domain now, so don’t force my hand.)
I’d agree to this in the same way the true nature of our universe is the laws of physics. (Would you consider the laws of physics a shoggoth?) My concern when people jump from that to “oh so there’s a God (of Physics)”.
I think the crux of my issue is what our analogy says to answer the question “a powerful LLM could coexist with humanity” (I lean towards yes). When people read shoggoths in the non-canonical way as “a weird alien” I think they conclude no. But if its more like a physics simulator or a pile of masks, then as long as you have it simulating benign things or wearing friendly masks, the answer is yes. I’ll leave it to someone who actually read lovecraft to say whether humanity could coexist with canonical shoggoths :)