I think https://scottaaronson.blog/?p=7266 might be a good overview. It is kind of pointless to focus on specific ways everyone dies, since it is easy to argue with each specific one. The whole point is that Doom is disjunctive. Like trying to dam or plug a single channel of a river’s delta, there will be another stream flowing somewhat differently to flood the basin, and the process is adversarial and anti-inductive.
I don’t think it is pointless to focus on specific ways everyone dies, unless there is a single strategy that addresses every possible way everyone dies.
If FOOM isn’t likely but something like thisis likely, it seems really unlikely to me that the approach of “continue to focus on strategies that rely on a single agent having a high level of control over the world” is still optimal (or, more accurately, it’s probably still a good idea to have some people working on that but not all the people).
I think https://scottaaronson.blog/?p=7266 might be a good overview. It is kind of pointless to focus on specific ways everyone dies, since it is easy to argue with each specific one. The whole point is that Doom is disjunctive. Like trying to dam or plug a single channel of a river’s delta, there will be another stream flowing somewhat differently to flood the basin, and the process is adversarial and anti-inductive.
I don’t think it is pointless to focus on specific ways everyone dies, unless there is a single strategy that addresses every possible way everyone dies.
If FOOM isn’t likely but something like this is likely, it seems really unlikely to me that the approach of “continue to focus on strategies that rely on a single agent having a high level of control over the world” is still optimal (or, more accurately, it’s probably still a good idea to have some people working on that but not all the people).