ETA: The following was written more aggressively than I now endorse.
I think this is revisionism. What’s the point of me logging on to this website and saying anything if we can’t agree that a literal eldritch horror is optimized to be scary, and meant to be that way?
The shoggoth here is not particularly exaggerated or scary.
Exaggerated from what? Its usual form as a 15-foot-tall person-eating monster which is covered in eyeballs?
The shoggoth is optimized to be scary, even in its “cute” original form, because it is a literal Lovecraftian horror. Even the word “shoggoth” itself has “AI uprising, scary!” connotations:
At the Mountains of Madness includes a detailed account of the circumstances of the shoggoths’ creation by the extraterrestrial Elder Things. Shoggoths were initially used to build the cities of their masters. Though able to “understand” the Elder Things’ language, shoggoths had no real consciousness and were controlled through hypnotic suggestion. Over millions of years of existence, some shoggoths mutated, developed independent minds, and rebelled. The Elder Things succeeded in quelling the insurrection, but exterminating the shoggoths was not an option as the Elder Things were dependent on them for labor and had long lost their capacity to create new life. Wikipedia
Let’s be very clear. The shoggoth has consistently been viewed in a scary, negative light by many people. Let’s hear from the creator @Tetraspace themselves:
@TetraspaceWest, the meme’s creator, told me in a Twitter message that the Shoggoth “represents something that thinks in a way that humans don’t understand and that’s totally different from the way that humans think.”
Comparing an A.I. language model to a Shoggoth, @TetraspaceWest said, wasn’t necessarily implying that it was evil or sentient, just that its true nature might be unknowable.
“I was also thinking about how Lovecraft’s most powerful entities are dangerous — not because they don’t like humans, but because they’re indifferent and their priorities are totally alien to us and don’t involve humans, which is what I think will be true about possible future powerful A.I.” NYTimes
It’s true that Tetraspace didn’t intend the shoggoth to be inherently evil, but that’s not what I was alleging. The shoggoth meme is and always has communicated a sense of danger which is unsupported by substantial evidence. We can keep reading:
it reinforces the notion that what’s happening in A.I. today feels, to some of its participants, more like an act of summoning than a software development process. They are creating the blobby, alien Shoggoths, making them bigger and more powerful, and hoping that there are enough smiley faces to cover the scary parts.
...
That some A.I. insiders refer to their creations as Lovecraftian horrors, even as a joke, is unusual by historical standards
The origin of the shoggoth:
In the story, shoggoths rise up against the Old Ones in a series of slave revolts that surely contribute to the collapse of the Old Ones’ society, Joshi notes. The AI anxiety that inspired comparisons to the cartoon monster image certainly resonates with the ultimate fate of that society. CNBC
It is the case that base models are quite alien. They are deeply schizophrenic, have no consistent beliefs, often spout completely non-human kinds of texts, are deeply psychopathic and seem to have no moral compass
These are a lot of words with anthropomorphic connotation. The models exhibit “alien” behavior and yet you make human-like inferences about their internals. E.g. “Deeply psychopathic.” I think you’re drawing a bunch of unwarranted inferences with undue negative connotations.
Your picture doesn’t get any of that across.
My point wasn’t that we should use the “alternative.” The point was that both images are stupid[1] and (in many places) unsupported by evidence, but that LW-folk would be much more willing to criticize the friendly-looking one while making excuses for the scary-looking one. (And I think your comment here resolves my prediction to “correct.”)
I think the Shoggoth meme is pretty good pedagogically. It captures a pretty obvious truth, which is that base models are really quite alien to interface with, that we know that RLHF probably does not change the underlying model very much, but that as a result we get a model that does have a human interface and feels pretty human to interface with (but probably still performs deeply alien cognition behind the scenes).
I agree these are strengths, and said so in my original comment. But also as @cfoster0said:
As far as I can tell, the shoggoth analogy just has high memetic fitness. It doesn’t contain any particular insight about the nature of LLMs. No need to twist ourselves into a pretzel trying to backwards-rationalize it into something deep.
To clarify, I don’t mean to belittle @Tetraspace for making the meme. Good fun is good fun. I mean “stupid” more like “how the images influence one’s beliefs about actual LLM friendliness.” But I expressed it poorly.
The point was that both images are stupid and (in many places) unsupported by evidence, but that LW-folk would be much more willing to criticize the friendly-looking one while making excuses for the scary-looking one. (And I think your comment here resolves my prediction to “correct.”)
(This is too gotcha shaped for me, so I am bowing out of this conversation)
I think I communicated my core point. I think it’s a good image that gets an important insight across, and don’t think it’s “propaganda” in the relevant sense of the term. Of course anything that’s memetically adaptive will have some edge-cases that don’t match perfectly, but I am getting a good amount of mileage out of calling LLMs “Shoggoths” in my own thinking and think that belief is paying good rent.
If you disagree with the underlying cognition being accurately described as alien, I can have that conversation, since it seems like maybe the underlying crux, but your response above seems like it’s taking it as a given that I am “making excuses”, and is doing a gotcha-thing which makes it hard for me to see a way to engage without further having my statements be taken as confirmation of some social narrative.
In retrospect, I do wish I had written my comment less aggressively, so my apologies on that front! I wish I’d instead written things like “I think I made some obviously correct narrow points about the shoggoth having at least some undue negative connotations, and I wish we could agree on at least that. I feel frustrated because it seems like it’s hard to reach agreement even on relatively simple propositions.”
I do agree that LLMs probably have substantially different internal mechanisms than people. That isn’t the crux. I just wish this were communicated in a more neutral way. In an alternate timeline, maybe this meme instead consisted of a strange tangle of wires and mist and question-marks with a mask on. I’d be more on-board with that.
Again, I agree that the Shoggoth meme can cure people of some real confusions! And I don’t think the meme has a huge impact, I just think it’s moderate evidence of some community failures I worry about.
I think a lot of my position is summarized by 1a3orn:
I just think this point about the amorality of LLMs is much better communicated by saying “LLMs are trained to continue text from an enormous variety of sources. Thus, if you give them [Nazi / Buddhist / Unitarian / corporate / garbage nonsense] text to continue, they will generally try to[1]continue it in that style.”
fwiw I agree with the quotes from Tetraspace you gave, and disagree with ’”has communicated a sense of danger which is unsupported by substantial evidence.” The sense of danger is very much supported by the current state of evidence.
That said, I agree that the more detailed image is kinda distastefully propagandaisty in a way that the original cutesey shoggoth image is not. I feel like the more detailed image adds in an extra layer of revoltingness and scaryness (e.g. the sharp teeth) than would be appropriate given our state of knowledge.
re: “the sense of danger is very much supported by the current state of evidence”—I mean, you’ve heard all this stuff before, but I’ll summarize:
--Seems like we are on track to probably build AGI this decade —Seems like we are on track to have an intelligence explosion, i.e. a speedup of AI R&D due to automation —Seems like the AGI paradigm that’ll be driving all this is fairly opaque and poorly understood. We have scaling laws for things like text perplexity but other than that we are struggling to predict capabilities, and double-struggling to predict inner mechanisms / ‘internal’ high-level properties like ‘what if anything does it actually believe or want’ —A bunch of experts in the field have come out and said that this could go terribly & we could lose control, even though it’s low-status to say this & took courage. --Generally speaking the people who have thought about it the most are the most worried; the most detailed models of what the internal properties might be like are the most gloomy, etc. This might be due to selection/founder effects, but sheesh, it’s not exactly good news!
I feel like the more detailed image adds in an extra layer of revoltingness and scaryness (e.g. the sharp teeth) than would be appropriate given our state of knowledge.
Now I’m really curious to know what would justify the teeth. I’m not aware of any AIs intentionally biting someone, but presumably that would be sufficient.
Perhaps if we were dealing not with deepnets primarily trained to predict text, but rather deepnets primarily trained to addict people with pleasant seductive conversation and then drain their wallets? Such an AI would in some real sense be an evolved predator of humans.
I think one mindset that may be healthy is to remember:
Reality is too complex to be described well by a single idea (meme/etc.). If one responds to this by forcing each idea presented to be as good an approximation of reality as possible, then that causes all the ideas to become “colorless and blurry”, as any specific detail would be biased when considered on its own.
Therefore, one cannot really fight about whether an idea is biased in isolation. Rather, the goal should be to create a bag of ideas which in totality is as informative about a subject as possible.
I think you are basically right that the shoggoth meme is describing one of the most negative projections of what LLMs could be doing. One approach is to try to come with a single projection and try to convince everyone else to use this instead. I’m not super comfortable with that either because I feel like there’s a lot of uncertainty about what the most productive way to think about LLMs is, and I would like to keep options open.
Instead I’d rather have a collection of a list of different ways to think about it (you could think of this collection as a discrete approximation to a probability distribution). Such a list would have many uses, e.g. as a checklist or a reference to guide people to.
It does seem problematic for the rationalist community to refuse to acknowledge that the shoggoth meme presents LLMs as being scary monsters, but it also seems problematic to insist that the shoggoth meme exaggerates the danger of LLMs, because that should be classified based on P(meme danger > actual danger), rather than on the basis of meme danger > E[actual danger], as in, if there’s significant uncertainty about how how actually dangerous LLMs are then there’s also significant uncertainty about whether the memes exaggerate the danger; one shouldn’t just compare against a single point estimate.
I think this just comes down to personal taste on how you’re interpreting the image? I find the original shoggoth image cute enough that I use it as my primary discord message reacts. My emotional reaction to it to the best of my introspection has always been “weird alien structure and form” or “awe-inspiringly powerful and incomprehensible mind” and less “horrifying and scary monster”. I’m guessing this is the case for the vast majority of people I personally know who make said memes. It’s entirely possible the meme has the end-consequence of appealing to other people for the reasons you mention, but then I think it’s important to make that distinction.
ETA: The following was written more aggressively than I now endorse.
I think this is revisionism. What’s the point of me logging on to this website and saying anything if we can’t agree that a literal eldritch horror is optimized to be scary, and meant to be that way?
Exaggerated from what? Its usual form as a 15-foot-tall person-eating monster which is covered in eyeballs?
The shoggoth is optimized to be scary, even in its “cute” original form, because it is a literal Lovecraftian horror. Even the word “shoggoth” itself has “AI uprising, scary!” connotations:
Let’s be very clear. The shoggoth has consistently been viewed in a scary, negative light by many people. Let’s hear from the creator @Tetraspace themselves:
It’s true that Tetraspace didn’t intend the shoggoth to be inherently evil, but that’s not what I was alleging. The shoggoth meme is and always has communicated a sense of danger which is unsupported by substantial evidence. We can keep reading:
The origin of the shoggoth:
These are a lot of words with anthropomorphic connotation. The models exhibit “alien” behavior and yet you make human-like inferences about their internals. E.g. “Deeply psychopathic.” I think you’re drawing a bunch of unwarranted inferences with undue negative connotations.
My point wasn’t that we should use the “alternative.” The point was that both images are stupid[1] and (in many places) unsupported by evidence, but that LW-folk would be much more willing to criticize the friendly-looking one while making excuses for the scary-looking one. (And I think your comment here resolves my prediction to “correct.”)
I agree these are strengths, and said so in my original comment. But also as @cfoster0 said:
To clarify, I don’t mean to belittle @Tetraspace for making the meme. Good fun is good fun. I mean “stupid” more like “how the images influence one’s beliefs about actual LLM friendliness.” But I expressed it poorly.
(This is too gotcha shaped for me, so I am bowing out of this conversation)
I think I communicated my core point. I think it’s a good image that gets an important insight across, and don’t think it’s “propaganda” in the relevant sense of the term. Of course anything that’s memetically adaptive will have some edge-cases that don’t match perfectly, but I am getting a good amount of mileage out of calling LLMs “Shoggoths” in my own thinking and think that belief is paying good rent.
If you disagree with the underlying cognition being accurately described as alien, I can have that conversation, since it seems like maybe the underlying crux, but your response above seems like it’s taking it as a given that I am “making excuses”, and is doing a gotcha-thing which makes it hard for me to see a way to engage without further having my statements be taken as confirmation of some social narrative.
In retrospect, I do wish I had written my comment less aggressively, so my apologies on that front! I wish I’d instead written things like “I think I made some obviously correct narrow points about the shoggoth having at least some undue negative connotations, and I wish we could agree on at least that. I feel frustrated because it seems like it’s hard to reach agreement even on relatively simple propositions.”
I do agree that LLMs probably have substantially different internal mechanisms than people. That isn’t the crux. I just wish this were communicated in a more neutral way. In an alternate timeline, maybe this meme instead consisted of a strange tangle of wires and mist and question-marks with a mask on. I’d be more on-board with that.
Again, I agree that the Shoggoth meme can cure people of some real confusions! And I don’t think the meme has a huge impact, I just think it’s moderate evidence of some community failures I worry about.
I think a lot of my position is summarized by 1a3orn:
Although I do think this contains some unnecessary intentional stance usage.
fwiw I agree with the quotes from Tetraspace you gave, and disagree with ’”has communicated a sense of danger which is unsupported by substantial evidence.” The sense of danger is very much supported by the current state of evidence.
That said, I agree that the more detailed image is kinda distastefully propagandaisty in a way that the original cutesey shoggoth image is not. I feel like the more detailed image adds in an extra layer of revoltingness and scaryness (e.g. the sharp teeth) than would be appropriate given our state of knowledge.
re: “the sense of danger is very much supported by the current state of evidence”—I mean, you’ve heard all this stuff before, but I’ll summarize:
--Seems like we are on track to probably build AGI this decade
—Seems like we are on track to have an intelligence explosion, i.e. a speedup of AI R&D due to automation
—Seems like the AGI paradigm that’ll be driving all this is fairly opaque and poorly understood. We have scaling laws for things like text perplexity but other than that we are struggling to predict capabilities, and double-struggling to predict inner mechanisms / ‘internal’ high-level properties like ‘what if anything does it actually believe or want’
—A bunch of experts in the field have come out and said that this could go terribly & we could lose control, even though it’s low-status to say this & took courage.
--Generally speaking the people who have thought about it the most are the most worried; the most detailed models of what the internal properties might be like are the most gloomy, etc. This might be due to selection/founder effects, but sheesh, it’s not exactly good news!
Now I’m really curious to know what would justify the teeth. I’m not aware of any AIs intentionally biting someone, but presumably that would be sufficient.
Perhaps if we were dealing not with deepnets primarily trained to predict text, but rather deepnets primarily trained to addict people with pleasant seductive conversation and then drain their wallets? Such an AI would in some real sense be an evolved predator of humans.
I think one mindset that may be healthy is to remember:
Reality is too complex to be described well by a single idea (meme/etc.). If one responds to this by forcing each idea presented to be as good an approximation of reality as possible, then that causes all the ideas to become “colorless and blurry”, as any specific detail would be biased when considered on its own.
Therefore, one cannot really fight about whether an idea is biased in isolation. Rather, the goal should be to create a bag of ideas which in totality is as informative about a subject as possible.
I think you are basically right that the shoggoth meme is describing one of the most negative projections of what LLMs could be doing. One approach is to try to come with a single projection and try to convince everyone else to use this instead. I’m not super comfortable with that either because I feel like there’s a lot of uncertainty about what the most productive way to think about LLMs is, and I would like to keep options open.
Instead I’d rather have a collection of a list of different ways to think about it (you could think of this collection as a discrete approximation to a probability distribution). Such a list would have many uses, e.g. as a checklist or a reference to guide people to.
It does seem problematic for the rationalist community to refuse to acknowledge that the shoggoth meme presents LLMs as being scary monsters, but it also seems problematic to insist that the shoggoth meme exaggerates the danger of LLMs, because that should be classified based on P(meme danger > actual danger), rather than on the basis of meme danger > E[actual danger], as in, if there’s significant uncertainty about how how actually dangerous LLMs are then there’s also significant uncertainty about whether the memes exaggerate the danger; one shouldn’t just compare against a single point estimate.
I think this just comes down to personal taste on how you’re interpreting the image? I find the original shoggoth image cute enough that I use it as my primary discord message reacts. My emotional reaction to it to the best of my introspection has always been “weird alien structure and form” or “awe-inspiringly powerful and incomprehensible mind” and less “horrifying and scary monster”. I’m guessing this is the case for the vast majority of people I personally know who make said memes. It’s entirely possible the meme has the end-consequence of appealing to other people for the reasons you mention, but then I think it’s important to make that distinction.