I’m sorry to hear that your health is poor and you feel that this is all on you. Maybe you’re right about the likelihood of doom, and even if I knew you were, I’d be sorry that it troubles you this way.
I think you’ve done an amazing job of building the AI safety field and now, even when the field has a degree of momentum of its own, it does seem to be less focused on doom than it should be, and I think you continuing to push people to focus on doom is valuable.
I don’t think its easy to get people to take weird ideas seriously. I’ve had many experiences where I’ve had ideas about how people should change their approach to a project that weren’t particularly far out and (in my view) were right for very straightforward reasons, and yet for the most part I was ignored altogether. What you’ve accomplished in building the AI safety field is amazing because AI doom ideas seemed really crazy when you started talking about them.
Nevertheless, I think some of the things you’ve said in this post are counterproductive. Most of the post is good, but insulting people who might contribute to solving the problem is not, nor is demanding that people acknowledge that you are smarter than they are. I’m not telling you that people don’t deserve to be insulted, nor that you have no right to consider yourself smarter than them—I’m telling you that you shouldn’t say it in public.
My concrete suggestion is this: if you are criticising or otherwise passing pessimistic judgement on people or a group of people —Give more details about what it is they’ve done to merit this criticism (“pretend plan that can fool EAs too ‘modest’ to trust their own judgments”—what are modest EAs actually doing that you think is wrong? Paying not enough attention to AI doom?) - Avoid talking about yourself (“So most organizations don’t have plans, because I haven’t taken the time to personally yell at them”)
Many people are proud, including me. If working in AI safety means I have to be regularly reminded that the fact that I didn’t go into the field sooner will be held as a mark against me, then that is a reason for me not to do it. Maybe not a decisive reason, but it is a reason. If working in AI safety means that you are going to ask me to publicly acknowledge that you’re smarter than me, that’s a reason for me not to do it. Maybe not decisive, but it’s a reason. I think there might be others who feel similarly.
If you want people to accept what you’re saying, it helps let people change their minds without embarrassing them. There are plenty of other things to do—many of which, as I’ve said, you seem to be much better at doing than me—but this one is important too. I wonder if you might say something like “anyone turned off by these comments can’t be of any value to the project”. If you think that—I just don’t. There are many, many smart people with dumb motivations, and many of them can do valuable work if they can be motivated to do it. This includes thinking deeply about things they were previously motivated not to think about.
You are a key, maybe the key, person in the AI safety field. What you say is attended to people in, around and even disconnected from the field. I don’t think you can reasonably claim that you shouldn’t be this important to the field. I think you should take this fact seriously, and that means exercising discipline in the things you say.
I say all this because I think that a decent amount of EA/AI safety seems to neglect AI doom an unreasonable amount, and certainly the field of AI in general neglects it. I find statements of the type I pointed out above off-putting, and I suspect I’m not alone.
There’s a point here about how fucked things are that I do not know how to convey without saying those things, definitely not briefly or easily. I’ve spent, oh, a fair number of years, being politer than this, and less personal than this, and the end result is that people nod along and go on living their lives.
I expect this won’t work either, but at some point you start trying different things instead of the things that have already failed. It’s more dignified if you fail in different ways instead of the same way.
Do whatever you want, obviously, but I just want to clarify that I did not suggest you avoid personally criticising people (only that you avoid vague/hard to interpret criticism) or saying you think doom is overwhelmingly likely. Some other comments give me a stronger impression than yours that I was asking you in a general sense to be nice, but I’m saying it to you because I figure it mostly matters that you’re clear on this.
I vehemently disagree here, based on my personal and generalizable or not history. I will illustrate with the three turning points of my recent life.
First step: I stumbled upon HPMOR, and Eliezer way of looking straight into the irrationality of all our common ways of interacting and thinking was deeply shocking. It made me feel like he was in a sense angrily pointing at me, who worked more like one of the PNJ rather than Harry. I heard him telling me you’re dumb and all your ideals of making intelligent decisions, being the gifted kid and being smarter than everyone are all are just delusions. You’re so out of touch with reality on so many levels, where to even start.
This attitude made me embark on a journey to improve myself, read the sequences, pledge on Giving What we can after knowing EA for many years, and overall reassess whether I was striving towards my goal of helping people (spoiler: I was not).
Second step: The April fools post also shocked me on so many levels. I was once again deeply struck by the sheer pessimism of this figure I respected so much. After months of reading articles on LessWrong and so many about AI alignment, this was the one that made me terrified in the face of the horrors to come.
Somehow this article, maybe by not caring about not hurting people, made me join an AI alignment research group in Berlin. I started investing myself into the problem, working on it regularly, diverting my donations towards effective organizations in the field. It even caused me to publish my first bit of research on preference learning.
Third step: Today this post, by not hiding any reality of the issue and striking a lot of ideas down that I was relying on for hope, made me realize I was becoming complacent. Doing a bit of research in the weekend is the way to be able to say “Yeah I participated in solving the issue” once it’s solved, not making sure it is in fact solved.
Therefore, based on my experience, not a lot of works made me significantly alter my life decisions. And those who did are all strangely ranting, smack-in-your-face works written by Eliezer.
Maybe I’m not the audience to optimize for to solve the problem, but on my side, I need even more smacks in the face, breaking you fantasy style posts.
I disagree strongly. To me it seems that AI safety has long punched below its weight because its proponents are unwilling to be confrontational, and are too reluctant to put moderate social pressure on people doing the activities which AI safety proponents hold to be very extremely bad. It is not a coincidence that among AI safety proponents, Eliezer is both unusually confrontational and unusually successful.
This isn’t specific to AI safety. A lot of people in this community generally believe that arguments which make people feel bad are counterproductive because people will be “turned off”.
This is false. There are tons of examples of disparaging arguments against bad (or “bad”) behavior that succeed wildly. Such arguments very frequently succeed in instilling individual values like e.g. conscientiousness or honesty. Prominent political movements which use this rhetoric abound. When this website was young, Eliezer and many others participated in an aggressive campaign of discourse against religious ideas, and this campaign accomplished many of its goals. I could name many many more large and small examples. I bet you can too.
Obviously this isn’t to say that confrontational and insulting argument is always the best style. Sometimes it’s truth-tracking and sometimes it isn’t. Sometimes it’s persuasive and sometimes it isn’t. Which cases are which is a difficult topic that I won’t get into here (except to briefly mention that it matters a lot whether the reasons given are actually good). Nor is this to say that the “turning people off” effect is completely absent; what I object to is the casual assumption that it outweighs any other effects. (Personally I’m turned off by the soft-gloved style of the parent comment, but I would not claim this necessarily means it’s inappropriate or ineffective—it’s not directed at me!) The point is that this very frequent claim does not match the evidence. Indeed, strong counterevidence is so easy to find that I suspect this is often not people’s real objection.
Deliberately phrasing things in confrontational or aggressive ways, in the hope that this makes your conversation partner “wake up” or something.
Choosing not to hide real, potentially-important beliefs you have about the world, even though those beliefs are liable to offend people, liable to be disagreed with, etc.
Either might be justifiable, but I’m a lot more wary of heuristics like “it’s never OK to talk about individuals’ relative proficiency at things, even if it feels very cruxy and important, because people just find the topic too triggering” than of heuristics like “it’s never OK to say things in ways that sound shouty or aggressive”. I think cognitive engines can much more easily get by self-censoring their tone than self-censoring what topics are permissible to think or talk about.
This kind of post scares away the person who will be the key person in the AI safety field if we define “key person” as the genius main driver behind solving it, not the loudest person. Which is rather unfortunate, because that person is likely to read this post at some point.
I don’t believe this post has any “dignity”, whatever weird obscure definition dignity has been given now. It’s more like flailing around in death throes while pointing fingers and lauding yourself than it is a solemn battle stance against an oncoming impossible enemy.
For context, I’m not some Eliezer hater, I’m a young person doing an ML masters currently who just got into this space and within the past week have become a huge fan of Eliezer Yudkowsky’s earlier work while simultaneously very disappointed in the recent, fruitless, output.
It seems worth doing a little user research on this to see how it actually affects people. If it is a net positive, then great. If it is a net negative, the question becomes how big of a net negative it is and whether it is worth the extra effort to frame things more nicely.
I think this was excellently worded, and I’m glad you said it. I’m also glad to have read all the responses, many of which seem important and on point to me. I strong upvoted this comment as well as several of the responses.
I’m leaving this comment, because I want to give you some social reinforcement for saying what you said, and saying it as clearly and tactfully as you did.
I’m sorry to hear that your health is poor and you feel that this is all on you. Maybe you’re right about the likelihood of doom, and even if I knew you were, I’d be sorry that it troubles you this way.
I think you’ve done an amazing job of building the AI safety field and now, even when the field has a degree of momentum of its own, it does seem to be less focused on doom than it should be, and I think you continuing to push people to focus on doom is valuable.
I don’t think its easy to get people to take weird ideas seriously. I’ve had many experiences where I’ve had ideas about how people should change their approach to a project that weren’t particularly far out and (in my view) were right for very straightforward reasons, and yet for the most part I was ignored altogether. What you’ve accomplished in building the AI safety field is amazing because AI doom ideas seemed really crazy when you started talking about them.
Nevertheless, I think some of the things you’ve said in this post are counterproductive. Most of the post is good, but insulting people who might contribute to solving the problem is not, nor is demanding that people acknowledge that you are smarter than they are. I’m not telling you that people don’t deserve to be insulted, nor that you have no right to consider yourself smarter than them—I’m telling you that you shouldn’t say it in public.
My concrete suggestion is this: if you are criticising or otherwise passing pessimistic judgement on people or a group of people
—Give more details about what it is they’ve done to merit this criticism (“pretend plan that can fool EAs too ‘modest’ to trust their own judgments”—what are modest EAs actually doing that you think is wrong? Paying not enough attention to AI doom?)
- Avoid talking about yourself (“So most organizations don’t have plans
, because I haven’t taken the time to personally yell at them”)Many people are proud, including me. If working in AI safety means I have to be regularly reminded that the fact that I didn’t go into the field sooner will be held as a mark against me, then that is a reason for me not to do it. Maybe not a decisive reason, but it is a reason. If working in AI safety means that you are going to ask me to publicly acknowledge that you’re smarter than me, that’s a reason for me not to do it. Maybe not decisive, but it’s a reason. I think there might be others who feel similarly.
If you want people to accept what you’re saying, it helps let people change their minds without embarrassing them. There are plenty of other things to do—many of which, as I’ve said, you seem to be much better at doing than me—but this one is important too. I wonder if you might say something like “anyone turned off by these comments can’t be of any value to the project”. If you think that—I just don’t. There are many, many smart people with dumb motivations, and many of them can do valuable work if they can be motivated to do it. This includes thinking deeply about things they were previously motivated not to think about.
You are a key, maybe the key, person in the AI safety field. What you say is attended to people in, around and even disconnected from the field. I don’t think you can reasonably claim that you shouldn’t be this important to the field. I think you should take this fact seriously, and that means exercising discipline in the things you say.
I say all this because I think that a decent amount of EA/AI safety seems to neglect AI doom an unreasonable amount, and certainly the field of AI in general neglects it. I find statements of the type I pointed out above off-putting, and I suspect I’m not alone.
There’s a point here about how fucked things are that I do not know how to convey without saying those things, definitely not briefly or easily. I’ve spent, oh, a fair number of years, being politer than this, and less personal than this, and the end result is that people nod along and go on living their lives.
I expect this won’t work either, but at some point you start trying different things instead of the things that have already failed. It’s more dignified if you fail in different ways instead of the same way.
FWIW you taking off the Mr. Nice guy gloves has actually made me make different life decisions. I’m glad you tried it even if it doesn’t work.
Do whatever you want, obviously, but I just want to clarify that I did not suggest you avoid personally criticising people (only that you avoid vague/hard to interpret criticism) or saying you think doom is overwhelmingly likely. Some other comments give me a stronger impression than yours that I was asking you in a general sense to be nice, but I’m saying it to you because I figure it mostly matters that you’re clear on this.
You might not have this ability, but surely you know at least one person who does?
I vehemently disagree here, based on my personal and generalizable or not history. I will illustrate with the three turning points of my recent life.
First step: I stumbled upon HPMOR, and Eliezer way of looking straight into the irrationality of all our common ways of interacting and thinking was deeply shocking. It made me feel like he was in a sense angrily pointing at me, who worked more like one of the PNJ rather than Harry. I heard him telling me you’re dumb and all your ideals of making intelligent decisions, being the gifted kid and being smarter than everyone are all are just delusions. You’re so out of touch with reality on so many levels, where to even start.
This attitude made me embark on a journey to improve myself, read the sequences, pledge on Giving What we can after knowing EA for many years, and overall reassess whether I was striving towards my goal of helping people (spoiler: I was not).
Second step: The April fools post also shocked me on so many levels. I was once again deeply struck by the sheer pessimism of this figure I respected so much. After months of reading articles on LessWrong and so many about AI alignment, this was the one that made me terrified in the face of the horrors to come.
Somehow this article, maybe by not caring about not hurting people, made me join an AI alignment research group in Berlin. I started investing myself into the problem, working on it regularly, diverting my donations towards effective organizations in the field. It even caused me to publish my first bit of research on preference learning.
Third step: Today this post, by not hiding any reality of the issue and striking a lot of ideas down that I was relying on for hope, made me realize I was becoming complacent. Doing a bit of research in the weekend is the way to be able to say “Yeah I participated in solving the issue” once it’s solved, not making sure it is in fact solved.
Therefore, based on my experience, not a lot of works made me significantly alter my life decisions. And those who did are all strangely ranting, smack-in-your-face works written by Eliezer.
Maybe I’m not the audience to optimize for to solve the problem, but on my side, I need even more smacks in the face, breaking you fantasy style posts.
I disagree strongly. To me it seems that AI safety has long punched below its weight because its proponents are unwilling to be confrontational, and are too reluctant to put moderate social pressure on people doing the activities which AI safety proponents hold to be very extremely bad. It is not a coincidence that among AI safety proponents, Eliezer is both unusually confrontational and unusually successful.
This isn’t specific to AI safety. A lot of people in this community generally believe that arguments which make people feel bad are counterproductive because people will be “turned off”.
This is false. There are tons of examples of disparaging arguments against bad (or “bad”) behavior that succeed wildly. Such arguments very frequently succeed in instilling individual values like e.g. conscientiousness or honesty. Prominent political movements which use this rhetoric abound. When this website was young, Eliezer and many others participated in an aggressive campaign of discourse against religious ideas, and this campaign accomplished many of its goals. I could name many many more large and small examples. I bet you can too.
Obviously this isn’t to say that confrontational and insulting argument is always the best style. Sometimes it’s truth-tracking and sometimes it isn’t. Sometimes it’s persuasive and sometimes it isn’t. Which cases are which is a difficult topic that I won’t get into here (except to briefly mention that it matters a lot whether the reasons given are actually good). Nor is this to say that the “turning people off” effect is completely absent; what I object to is the casual assumption that it outweighs any other effects. (Personally I’m turned off by the soft-gloved style of the parent comment, but I would not claim this necessarily means it’s inappropriate or ineffective—it’s not directed at me!) The point is that this very frequent claim does not match the evidence. Indeed, strong counterevidence is so easy to find that I suspect this is often not people’s real objection.
I think there’s an important distinction between:
Deliberately phrasing things in confrontational or aggressive ways, in the hope that this makes your conversation partner “wake up” or something.
Choosing not to hide real, potentially-important beliefs you have about the world, even though those beliefs are liable to offend people, liable to be disagreed with, etc.
Either might be justifiable, but I’m a lot more wary of heuristics like “it’s never OK to talk about individuals’ relative proficiency at things, even if it feels very cruxy and important, because people just find the topic too triggering” than of heuristics like “it’s never OK to say things in ways that sound shouty or aggressive”. I think cognitive engines can much more easily get by self-censoring their tone than self-censoring what topics are permissible to think or talk about.
How is “success” measured among AI safety proponents?
This kind of post scares away the person who will be the key person in the AI safety field if we define “key person” as the genius main driver behind solving it, not the loudest person. Which is rather unfortunate, because that person is likely to read this post at some point.
I don’t believe this post has any “dignity”, whatever weird obscure definition dignity has been given now. It’s more like flailing around in death throes while pointing fingers and lauding yourself than it is a solemn battle stance against an oncoming impossible enemy.
For context, I’m not some Eliezer hater, I’m a young person doing an ML masters currently who just got into this space and within the past week have become a huge fan of Eliezer Yudkowsky’s earlier work while simultaneously very disappointed in the recent, fruitless, output.
It seems worth doing a little user research on this to see how it actually affects people. If it is a net positive, then great. If it is a net negative, the question becomes how big of a net negative it is and whether it is worth the extra effort to frame things more nicely.
I think this was excellently worded, and I’m glad you said it. I’m also glad to have read all the responses, many of which seem important and on point to me. I strong upvoted this comment as well as several of the responses.
I’m leaving this comment, because I want to give you some social reinforcement for saying what you said, and saying it as clearly and tactfully as you did.
Strongly agree with this, said more eloquently than I was able to :)