This look like a great list of risk factors leading to AI lethalities, why making AI safe is a hard problem and why we are failing. But this post is also not what I would have expected by taking the title at face value. I thought that the post would be about detailed and credible scenarios suggesting how AI could lead to extinction, where for example each scenario could represent a class of AI X-risks that we want to reduce. I suspect that such an article would also be really helpful because we probably have not been so good at generating very detailed and credible scenarios of doom so far. There are risks of info hazard associated with that for sure. Also I am sympathetic to the argument that “AI does not think like you do” and that AI is likely to lead to doom in ways we cannot think of because of its massive strategic advantage. But still I think it might be very helpful to write some detailed and credible stories of doom so that a large part of the AI community take extinction risks from AI really seriously and approaches AI capability research more like working at high security bio hazard lab. Perverse incentives might still lead lots of people in the AI community to not take these concerned seriously. Also, it is true that there are some posts going in that direction for ex What failure looks like, It looks like you’re trying to take over the world, but I don’t think we have done enough on that front and that probably hinders our capacity to have X-risks from AI be taken seriously.
This look like a great list of risk factors leading to AI lethalities, why making AI safe is a hard problem and why we are failing. But this post is also not what I would have expected by taking the title at face value. I thought that the post would be about detailed and credible scenarios suggesting how AI could lead to extinction, where for example each scenario could represent a class of AI X-risks that we want to reduce. I suspect that such an article would also be really helpful because we probably have not been so good at generating very detailed and credible scenarios of doom so far. There are risks of info hazard associated with that for sure. Also I am sympathetic to the argument that “AI does not think like you do” and that AI is likely to lead to doom in ways we cannot think of because of its massive strategic advantage. But still I think it might be very helpful to write some detailed and credible stories of doom so that a large part of the AI community take extinction risks from AI really seriously and approaches AI capability research more like working at high security bio hazard lab. Perverse incentives might still lead lots of people in the AI community to not take these concerned seriously. Also, it is true that there are some posts going in that direction for ex What failure looks like, It looks like you’re trying to take over the world, but I don’t think we have done enough on that front and that probably hinders our capacity to have X-risks from AI be taken seriously.