I agree with the first point of “dying with dignity” doesn’t seem like it accomplishes the goal it was intending to accomplish. I’m not yet sold on “play to your outs” accomplishing the right thing (my cruxes have to do with how people end up interpreting / acting-on / being-motivated-by the phrase).
I interpreted the die-with-dignity post to be saying something like:
P(doom) is very high. (I’m personally less confident of this than Eliezer but I’d be willing to substitute “P(doom) is likely enough to be high that we should start thinking about how we’d engage psychologically with that)”
When faced with doom, people’s epistemics tend to get slippery, avoiding confronting the doom and coming up with plans that feel vaguely good without engaging with the hard problem. (I agree with this)
One way their epistemics get slippery is that they condition on being in worlds where the plans-they-could-think-of-might work, and once you’ve done that a couple times you’re just living in a fantasy world. (I’m slightly philosophically confused about this, but I recall Eliezer once saying ‘you get to do this approximately once without it being a bad idea’, which seems plausible to me)
When facing doom, people also tend to look for extreme strategies that burn the coordination commons, while not even really solving the problem. (I’m a bit confused about this but it seems true-ish)
Thinking in terms of “raising log-odds of survival while continuing to tackle the hard parts of the problem” is a useful reframe for motivation. (I’m confused about this. I’m not sure how much the “log-odds” part is supposed to be helping. I think there was also supposed to be some flavor of where Eliezer thought people’s strategic plan-making thoughts should be going, but I don’t think I could summarize it succinctly in a way he endorses)
Calling #6 “Die with dignity” is a good slogan. (I disagree with this)
“Die with dignity” seems overdeterminedly like a bad slogan because of landfish’s aforementioned google of “dying with dignity”, which is about terminally people voluntarily killing themselves. Seems like totally the wrong vibe, and not worth trying to win a memetic war on.
I think Eliezer meant something like “die with dignity the way a soldier would” (where you keep fighting to end). I think “die fighting with dignity” is at least a marginal improvement (and even fits within the 5 word limit for slogans!). But, still doesn’t feel quite right.
But… “Play to your outs”… doesn’t seem at first glance to me like it really solves the problems in the middle? (#2, #3, #4, #5?). I feel like it encourages #5 (and Jeff’s vague Q&A on “don’t do that” doesn’t seem reassuring to me). It also seems to encourage #3 (and again the vague admonishment to “not do that” doesn’t seem that reassuring to me.)
I don’t think “play to your outs” scales. When I imagine 100 people trying to do that who aren’t part of a single company with shared leadership, I imagine them doing a bunch of random stuff that is often at cross-purposes.
Right now I feel like “play to your outs” gives me the illusion of having a strategy-and-life-philosophy, but doesn’t feel like it directs my attention in particularly useful ways.
I don’t have a great alternate slogan at the time, but I’m not sure the thing exactly bottlenecked personally on a slogan exactly.
(I’m not strongly confident it’s not a good phrase here – If it turned out everyone ended up naturally doing sensible things when following it as a strategy/life-philosophy, coolio I guess. I see some people have responded positively to the post so far. I’m worried those people are responding more to the fact that someone gave them something less uncomfortable/depressing than ‘die with dignity’ to orient around, rather than actually getting directed in a useful way)
“It also seems to encourage #3 (and again the vague admonishment to “not do that” doesn’t seem that reassuring to me.)”
I just pointed to Eleizer’s warning which I thought was sufficient. I could write more about why I think it’s not a good idea, but I currently think a bigger portion of the problem is people not trying to come up with good plans rather than people coming up with dangerous plans which is why my emphasis is where it is.
Eliezer is great at red teaming people’s plans. This is great for finding ways plans don’t work, and I think it’s very important he keep doing this. It’s not great for motivating people to come up with good plans, though. And I think that shortage of motivation is a real threat to our chances to mitigate AI existential risk. I was talking to a leading alignment researcher yesterday who said their motivation had taken a hit from Eliezer’s constant “all your plans will fail” talk, so I’m pretty sure this is a real thing, even though I’m unsure of the magnitude.
I largely agree with that, but I think there’s an important asymmetry here: it’s much easier to come up with a plan that will ‘successfully’ do huge damage, than to come up with a plan that will successfully solve the problem.
So to have positive expected impact you need a high ratio of [people persuaded to come up with good plans] to [people persuaded that crazy dangerous plans are necessary].
I’d expect your post to push a large majority of readers in a positive direction (I think it does for me—particularly combined with Eliezer’s take). My worry isn’t that many go the other way, but that it doesn’t take many.
I think that’s a legit concern. One mitigating factor is that people who seem inclined to rash destructive plans tend to be pretty bad at execution, e.g. Aum Shinrikyo
I think Eliezer meant something like “die with dignity the way a soldier would” (where you keep fighting to end). I think “die fighting with dignity” is at least a marginal improvement (and even fits within the 5 word limit for slogans!). But, still doesn’t feel quite right.
Just a note on confidence, which seems especially important since I’m making a kind of normative claim:
I’m very confident “dying with dignity” is a counterproductive frame for me. I’m somewhat confident that “playing to your outs” is a really useful frame for me and people like me. I’m not very confident “playing to your outs” is a good replacement to “dying with dignity” in general, because I don’t know how much people will respond to it like I do. Seeing people’s comments here is helpful.
So, in my mind, the thing that “dying with dignity” is supposed to do is that when you look at plan A and B, you ask yourself: “which of these is more dignified?” instead of “which of these is less likely to lead to death?”, because your ability to detect dignity is more sensitive than your ability to detect likelihood of leading to death on the present margin. [This is, I think, the crux; if you don’t buy this then I agree the framing doesn’t seem sensible.]
This lets you still do effective actions (that, in conjunction with lots of other things, can still lead to less likelihood of death), even if when you look at any plan in isolation, the result is “yep, still 0% chance of survival”, because maybe some plans lead to 3 units of dignity and other plans lead to 4 units of dignity, and you’d rather have 4 than 3 units of dignity.
[Going back to the crux—I think if we actually had outs, and knew how to play to them, this plan would make lots of sense. “Ok guys, we should do X because when we draw Fireball, we’ll then be able to win.” But the plans I’m presently most optimistic about look more like “when you don’t know where the bugs are, tidy up your codebase”, which seems like a pretty different approach, while lining up with ‘dignity’.]
Of course, you also need a sense of dignity that’s more like “we did things that were sane and cared about whether or not we made it” instead of “we didn’t make a fuss or look too strange” or something like that.
Would you say that this constitutes using dignity as a proxy for indirect increase in survival odds and/or increase in broad preparation to execute on outs such that “dignity” is expected to have easier-to-grasp scaling properties and better emotional binding?
I agree with the first point of “dying with dignity” doesn’t seem like it accomplishes the goal it was intending to accomplish. I’m not yet sold on “play to your outs” accomplishing the right thing (my cruxes have to do with how people end up interpreting / acting-on / being-motivated-by the phrase).
I interpreted the die-with-dignity post to be saying something like:
P(doom) is very high. (I’m personally less confident of this than Eliezer but I’d be willing to substitute “P(doom) is likely enough to be high that we should start thinking about how we’d engage psychologically with that)”
When faced with doom, people’s epistemics tend to get slippery, avoiding confronting the doom and coming up with plans that feel vaguely good without engaging with the hard problem. (I agree with this)
One way their epistemics get slippery is that they condition on being in worlds where the plans-they-could-think-of-might work, and once you’ve done that a couple times you’re just living in a fantasy world. (I’m slightly philosophically confused about this, but I recall Eliezer once saying ‘you get to do this approximately once without it being a bad idea’, which seems plausible to me)
If people do fully accept high likelihood of failure, they have a hard time staying motivated (both individuals, and as a group). (I agree with this)
When facing doom, people also tend to look for extreme strategies that burn the coordination commons, while not even really solving the problem. (I’m a bit confused about this but it seems true-ish)
Thinking in terms of “raising log-odds of survival while continuing to tackle the hard parts of the problem” is a useful reframe for motivation. (I’m confused about this. I’m not sure how much the “log-odds” part is supposed to be helping. I think there was also supposed to be some flavor of where Eliezer thought people’s strategic plan-making thoughts should be going, but I don’t think I could summarize it succinctly in a way he endorses)
Calling #6 “Die with dignity” is a good slogan. (I disagree with this)
“Die with dignity” seems overdeterminedly like a bad slogan because of landfish’s aforementioned google of “dying with dignity”, which is about terminally people voluntarily killing themselves. Seems like totally the wrong vibe, and not worth trying to win a memetic war on.
I think Eliezer meant something like “die with dignity the way a soldier would” (where you keep fighting to end). I think “die fighting with dignity” is at least a marginal improvement (and even fits within the 5 word limit for slogans!). But, still doesn’t feel quite right.
But… “Play to your outs”… doesn’t seem at first glance to me like it really solves the problems in the middle? (#2, #3, #4, #5?). I feel like it encourages #5 (and Jeff’s vague Q&A on “don’t do that” doesn’t seem reassuring to me). It also seems to encourage #3 (and again the vague admonishment to “not do that” doesn’t seem that reassuring to me.)
I don’t think “play to your outs” scales. When I imagine 100 people trying to do that who aren’t part of a single company with shared leadership, I imagine them doing a bunch of random stuff that is often at cross-purposes.
Right now I feel like “play to your outs” gives me the illusion of having a strategy-and-life-philosophy, but doesn’t feel like it directs my attention in particularly useful ways.
I don’t have a great alternate slogan at the time, but I’m not sure the thing exactly bottlenecked personally on a slogan exactly.
(I’m not strongly confident it’s not a good phrase here – If it turned out everyone ended up naturally doing sensible things when following it as a strategy/life-philosophy, coolio I guess. I see some people have responded positively to the post so far. I’m worried those people are responding more to the fact that someone gave them something less uncomfortable/depressing than ‘die with dignity’ to orient around, rather than actually getting directed in a useful way)
“It also seems to encourage #3 (and again the vague admonishment to “not do that” doesn’t seem that reassuring to me.)”
I just pointed to Eleizer’s warning which I thought was sufficient. I could write more about why I think it’s not a good idea, but I currently think a bigger portion of the problem is people not trying to come up with good plans rather than people coming up with dangerous plans which is why my emphasis is where it is.
Eliezer is great at red teaming people’s plans. This is great for finding ways plans don’t work, and I think it’s very important he keep doing this. It’s not great for motivating people to come up with good plans, though. And I think that shortage of motivation is a real threat to our chances to mitigate AI existential risk. I was talking to a leading alignment researcher yesterday who said their motivation had taken a hit from Eliezer’s constant “all your plans will fail” talk, so I’m pretty sure this is a real thing, even though I’m unsure of the magnitude.
I largely agree with that, but I think there’s an important asymmetry here: it’s much easier to come up with a plan that will ‘successfully’ do huge damage, than to come up with a plan that will successfully solve the problem.
So to have positive expected impact you need a high ratio of [people persuaded to come up with good plans] to [people persuaded that crazy dangerous plans are necessary].
I’d expect your post to push a large majority of readers in a positive direction (I think it does for me—particularly combined with Eliezer’s take).
My worry isn’t that many go the other way, but that it doesn’t take many.
I think that’s a legit concern. One mitigating factor is that people who seem inclined to rash destructive plans tend to be pretty bad at execution, e.g. Aum Shinrikyo
“Die with honor” might be a good phrasing?
I find something compelling about “with a shield or on it”
Just a note on confidence, which seems especially important since I’m making a kind of normative claim:
I’m very confident “dying with dignity” is a counterproductive frame for me. I’m somewhat confident that “playing to your outs” is a really useful frame for me and people like me. I’m not very confident “playing to your outs” is a good replacement to “dying with dignity” in general, because I don’t know how much people will respond to it like I do. Seeing people’s comments here is helpful.
So, in my mind, the thing that “dying with dignity” is supposed to do is that when you look at plan A and B, you ask yourself: “which of these is more dignified?” instead of “which of these is less likely to lead to death?”, because your ability to detect dignity is more sensitive than your ability to detect likelihood of leading to death on the present margin. [This is, I think, the crux; if you don’t buy this then I agree the framing doesn’t seem sensible.]
This lets you still do effective actions (that, in conjunction with lots of other things, can still lead to less likelihood of death), even if when you look at any plan in isolation, the result is “yep, still 0% chance of survival”, because maybe some plans lead to 3 units of dignity and other plans lead to 4 units of dignity, and you’d rather have 4 than 3 units of dignity.
[Going back to the crux—I think if we actually had outs, and knew how to play to them, this plan would make lots of sense. “Ok guys, we should do X because when we draw Fireball, we’ll then be able to win.” But the plans I’m presently most optimistic about look more like “when you don’t know where the bugs are, tidy up your codebase”, which seems like a pretty different approach, while lining up with ‘dignity’.]
Of course, you also need a sense of dignity that’s more like “we did things that were sane and cared about whether or not we made it” instead of “we didn’t make a fuss or look too strange” or something like that.
Would you say that this constitutes using dignity as a proxy for indirect increase in survival odds and/or increase in broad preparation to execute on outs such that “dignity” is expected to have easier-to-grasp scaling properties and better emotional binding?