Curated. This was perhaps the most detailed yet informative story I’ve read about how failure will go down. As you say at the start it’s making several key assumptions, it’s not your ‘mainline’ failure story. Thx for making the assumptions explicit, and discussing how to vary them at the end. I’d like to see more people write stories written under different assumptions.
The sorts of stories Eliezer has told in the past have focused on 10-1000x faster takeoffs than discussed here, so those stories are less extended (you kinda just wake up one day then everyone dies). This slower one is helpful in seeing many of the relevant dynamics happen in more detail (although many of these issues wouldn’t quite apply in the 1000x faster world).
Failure stories seem to me especially helpful in focusing research on what the actual problems will be. I also like this post in the context of Critch’s post.
Curated. This was perhaps the most detailed yet informative story I’ve read about how failure will go down. As you say at the start it’s making several key assumptions, it’s not your ‘mainline’ failure story. Thx for making the assumptions explicit, and discussing how to vary them at the end. I’d like to see more people write stories written under different assumptions.
The sorts of stories Eliezer has told in the past have focused on 10-1000x faster takeoffs than discussed here, so those stories are less extended (you kinda just wake up one day then everyone dies). This slower one is helpful in seeing many of the relevant dynamics happen in more detail (although many of these issues wouldn’t quite apply in the 1000x faster world).
Failure stories seem to me especially helpful in focusing research on what the actual problems will be. I also like this post in the context of Critch’s post.