This does not seem like the obvious reading of the thread to me.
Obviously, Eliezer is saying that there is a plausible but extremely upsetting idea that could be learned by studying neural networks sufficiently competently.
I think Eliezer is saying that if you understood on a gut level how messy deep networks are, you’d realize how doomed prosaic alignment is. And that would be horrible news. And that might make you scream, although perhaps not constantly.
After all, Eliezer is known to use… dashes… of colorful imagery. Do you really think he is literally constantly screaming silently to himself? No? Then he was probably also being hyperbolic about how he truly thinks a person would respond to understanding a deep network in great detail.
That’s why I feel that your interpretation is grasping really hard at straws. This is a standard “we’re doomed by inadequate AI alignment” thread from Eliezer.
Even though it’s an exaggeration, Eliezer is, with this exaggeration, trying to indicate an extremely high level of fear, off the charts compared with what people are normally used to, as a result of really taking in the information. Such a level of fear is not clearly lower than the level of fear experienced by the psychotic people in question, who experienced e.g. serious sleep loss due to fear.
I strong-upvoted both of Jessica’s comments in this thread despite disagreeing with her interpretation in the strongest possible terms; I did so because I think it is important to note that, for every “common-sense” interpretation of a community leader’s words, there will be some small minority who interpret it in some other (possibly more damaging) way—and while I think (importantly) this does not imply it is the community leader’s responsibility to manage their words in such a way that no misinterpretation is possible (which I think is simply completely unfeasible), I am nonetheless in favor of people sharing their (non-standard) interpretations, given the variation in potential responses.
As Eliezer once said (I’m paraphrasing from memory here, so the following may not be word-for-word accurate, but I am >95% confident I’m not misremembering the thrust of what he said), “The question I have to ask myself is, will this drive more than 5% of my readers insane?”
EDIT: I have located the text of the original comment. I note (with some vindication) that once again, it seems that Eliezer was sensitive to this concern way ahead of when it actually became a thing.
This does not seem like the obvious reading of the thread to me.
I think Eliezer is saying that if you understood on a gut level how messy deep networks are, you’d realize how doomed prosaic alignment is. And that would be horrible news. And that might make you scream, although perhaps not constantly.
After all, Eliezer is known to use… dashes… of colorful imagery. Do you really think he is literally constantly screaming silently to himself? No? Then he was probably also being hyperbolic about how he truly thinks a person would respond to understanding a deep network in great detail.
That’s why I feel that your interpretation is grasping really hard at straws. This is a standard “we’re doomed by inadequate AI alignment” thread from Eliezer.
Even though it’s an exaggeration, Eliezer is, with this exaggeration, trying to indicate an extremely high level of fear, off the charts compared with what people are normally used to, as a result of really taking in the information. Such a level of fear is not clearly lower than the level of fear experienced by the psychotic people in question, who experienced e.g. serious sleep loss due to fear.
I strong-upvoted both of Jessica’s comments in this thread despite disagreeing with her interpretation in the strongest possible terms; I did so because I think it is important to note that, for every “common-sense” interpretation of a community leader’s words, there will be some small minority who interpret it in some other (possibly more damaging) way—and while I think (importantly) this does not imply it is the community leader’s responsibility to manage their words in such a way that no misinterpretation is possible (which I think is simply completely unfeasible), I am nonetheless in favor of people sharing their (non-standard) interpretations, given the variation in potential responses.
As Eliezer once said (I’m paraphrasing from memory here, so the following may not be word-for-word accurate, but I am >95% confident I’m not misremembering the thrust of what he said), “The question I have to ask myself is, will this drive more than 5% of my readers insane?”
EDIT: I have located the text of the original comment. I note (with some vindication) that once again, it seems that Eliezer was sensitive to this concern way ahead of when it actually became a thing.