Interesting, thank you. I think that all makes sense, and I’m sure it plays at least some part in their strategy. I’ve wondered about this possibility a little bit.
Yudkowsky has been consistent in his belief that doom is near certain without a lot more time to work on alignment. He’s publicly held that opinion, and spent a huge amount of effort explaining and arguing for it since well before the current wave of success with deep networks. So I think for him at least, it’s a sincerely held belief.
Your point about the stated belief changing the reality is important. Everything is safer if you think it’s dangerous—you’ll take more precautions.
With that in mind, I think it’s pretty important for even optimists to heavily sprinkle in the message “this will probably go well IF everyone involved is really careful”.
Interesting, thank you. I think that all makes sense, and I’m sure it plays at least some part in their strategy. I’ve wondered about this possibility a little bit.
Yudkowsky has been consistent in his belief that doom is near certain without a lot more time to work on alignment. He’s publicly held that opinion, and spent a huge amount of effort explaining and arguing for it since well before the current wave of success with deep networks. So I think for him at least, it’s a sincerely held belief.
Your point about the stated belief changing the reality is important. Everything is safer if you think it’s dangerous—you’ll take more precautions.
With that in mind, I think it’s pretty important for even optimists to heavily sprinkle in the message “this will probably go well IF everyone involved is really careful”.