he is worried about catastrophic disruptions of the political, economic, and environmental systems
Ok, but how is this any different in practice? Or preventable via “corporate law”? It feels to me like people make too much of a distinction between slow and fast take offs scenarios, as if somewhat, if humans appear to be in the loop a bit more, this makes the problem less bad or less AI-related.
Essentially, if your mode of failure follows almost naturally from introducing AI system in current society and basic economic incentives, to the point that you can’t really look at any part of the process and identify anyone maliciously and intentionally setting it up to end the world, yet it does end the world, then it’s an AI problem. It may be a weird, slow, cyborg-like amalgamation of AI and human society that caused the catastrophe instead of a singular agentic AI taking everything over quickly, but the AI is still the main driver, and the only way to avoid the problem is to make AI extremely robust not just to intentional bad use but also to unintentional bad incentives feedback loops, essentially smart and moral enough to stop its own users and creators when they don’t know any better. Or alternatively, to just not make the AI at all.
Honestly, given what Facebook’s recommender systems have already caused, it’s disheartening that the leader of AI research at Meta doesn’t get something like this.
Ok, but how is this any different in practice? Or preventable via “corporate law”? It feels to me like people make too much of a distinction between slow and fast take offs scenarios, as if somewhat, if humans appear to be in the loop a bit more, this makes the problem less bad or less AI-related.
Essentially, if your mode of failure follows almost naturally from introducing AI system in current society and basic economic incentives, to the point that you can’t really look at any part of the process and identify anyone maliciously and intentionally setting it up to end the world, yet it does end the world, then it’s an AI problem. It may be a weird, slow, cyborg-like amalgamation of AI and human society that caused the catastrophe instead of a singular agentic AI taking everything over quickly, but the AI is still the main driver, and the only way to avoid the problem is to make AI extremely robust not just to intentional bad use but also to unintentional bad incentives feedback loops, essentially smart and moral enough to stop its own users and creators when they don’t know any better. Or alternatively, to just not make the AI at all.
Honestly, given what Facebook’s recommender systems have already caused, it’s disheartening that the leader of AI research at Meta doesn’t get something like this.