I agree with some of this. I admit that I’ve been surprised several times by leading AI safety community orgs outperforming my expectations, from Openphil to MIRI to OpenAI. However, considering the rate that the world has been changing, I thing that the distance between 2023 and 2033 is more like the distance between 2023 and 2003, and the whole point of this post is taking a step back and looking at the situation which is actually pretty bad.
I think that between the US/China AI competition, and the AI companies also competing with each other under the US umbrella, as well as against dark AI companies like Facebook and companies that might be started indigenously by Microsoft or Apple or Amazon under their full control, and the possibility of the US government taking a treacherous turn and becoming less democratic more broadly (e.g. due to human behavior manipulation technology), I’m still pessimistic that the 2020s have more than a 50% chance of going well for AI safety. For example, the AI safety community might theoretically be forced to choose between rallying behind a pause vs. leaving humanity to die, and if they were to choose the pause in that hypothetical, then it’s reasonable to anticipate a 40% chance of conflict.
I agree with some of this. I admit that I’ve been surprised several times by leading AI safety community orgs outperforming my expectations, from Openphil to MIRI to OpenAI. However, considering the rate that the world has been changing, I thing that the distance between 2023 and 2033 is more like the distance between 2023 and 2003, and the whole point of this post is taking a step back and looking at the situation which is actually pretty bad.
I think that between the US/China AI competition, and the AI companies also competing with each other under the US umbrella, as well as against dark AI companies like Facebook and companies that might be started indigenously by Microsoft or Apple or Amazon under their full control, and the possibility of the US government taking a treacherous turn and becoming less democratic more broadly (e.g. due to human behavior manipulation technology), I’m still pessimistic that the 2020s have more than a 50% chance of going well for AI safety. For example, the AI safety community might theoretically be forced to choose between rallying behind a pause vs. leaving humanity to die, and if they were to choose the pause in that hypothetical, then it’s reasonable to anticipate a 40% chance of conflict.