Eliezar- I love the content, but similar to some other commenters, I think you are missing the value (and rationality) of positivity. Specifically, when faced with an extremely difficult challenge, assume that you (and the other smart people who care about it) have a real shot at solving it! This is the rational strategy for a simple reason: if you don’t have a real shot at solving it then you haven’t lost anything anyway. But if you do have a real shot at solving it, then let’s all give it our 110%!
I’m not proposing being unrealistic about the challenges we face—I’m as concerned as you are. But I believe thinking this way and inviting the community and our broader society to work together on this challenge is part of Good Strategy
I’m not proposing being unrealistic about the challenges we face
Well yes, you are. You can’t both say “let’s assume we have a real shot at success regardless of factual beliefs” and “let’s be realistic about the challenges we face”. If the model says that the challenges are so hard that we don’t have a real shot (which is in fact the case here, for Eliezer’s model), then these two things are a straight-forward contradiction.
Which is also the problem with your argument. Pretending as if we have a real shot requires lying. However, I think lying is really bad idea. Your argument implicitly assumes that the optimal strategy is independent of the odds of success, but I think that assumption is false—I want to know if Eliezer thinks the current approach is doomed, so that we can look for something else (like a policy approach). If Elliezer had chosen to lie about P(doom-given-alignment), we may keep working on alignment rather than policy, and P(overall-doom) may increase!
Eliezar- I love the content, but similar to some other commenters, I think you are missing the value (and rationality) of positivity. Specifically, when faced with an extremely difficult challenge, assume that you (and the other smart people who care about it) have a real shot at solving it! This is the rational strategy for a simple reason: if you don’t have a real shot at solving it then you haven’t lost anything anyway. But if you do have a real shot at solving it, then let’s all give it our 110%!
I’m not proposing being unrealistic about the challenges we face—I’m as concerned as you are. But I believe thinking this way and inviting the community and our broader society to work together on this challenge is part of Good Strategy
Well yes, you are. You can’t both say “let’s assume we have a real shot at success regardless of factual beliefs” and “let’s be realistic about the challenges we face”. If the model says that the challenges are so hard that we don’t have a real shot (which is in fact the case here, for Eliezer’s model), then these two things are a straight-forward contradiction.
Which is also the problem with your argument. Pretending as if we have a real shot requires lying. However, I think lying is really bad idea. Your argument implicitly assumes that the optimal strategy is independent of the odds of success, but I think that assumption is false—I want to know if Eliezer thinks the current approach is doomed, so that we can look for something else (like a policy approach). If Elliezer had chosen to lie about P(doom-given-alignment), we may keep working on alignment rather than policy, and P(overall-doom) may increase!