The linked tweet doesn’t actually make this claim, or even imply it. Is Yann on the record saying “zero threat” somewhere else? Or even claiming that the level of risk is <0.1%? Failing that, is there somewhere he has implied that the risk is very small?
It seems like his position is more like: this problem looks likely to be manageable in a variety of ways, so let’s push ahead and try to manage it. The probability of doom today is very low, so we can reconsider later if things look bleak.
He does say “the ‘hard takeoff scenario’ is utterly impossible.” I’m a bit sympathetic to this, since Eliezer’s description of super hard takeoff feels crazy to me as well. That said: (i) “utterly impossible” is too strong even for super hard takeoffs, which seem very unlikely but possible, (ii) based on his rhetoric about adaptation Yann seems to be implicitly rejecting even months-long takeoffs, which I think have >10% probability depending on how you define them.
I believe I have seen him make multiple statements on Twitter over months that express this point of view, and I see his statements herein as reinforcing that, but in the interests of this not being distracting to the main point I am editing this line.
Also including an additional exchange from yesterday he had with Elon Musk.
Mods, please reimport.
EDIT: Also adding his response to Max Tegmark from yesterday, and EY’s response to that, at the bottom, but raising the bar for further inclusions substantially.
Includes this quote: But regardless, until we have a semi credible design, we are discussing the sex of angels. The worst that can happen is that we won’t figure out a good way to make them safe and we won’t deploy them.
Which, logically, says zero risk, since the worst that can happen is non-deployment?
(I still have no idea where the button is that’s supposed to let me switch to a WYSIWYG editor)
I agree that “the worst that can happen is...” suggests an unreasonably low estimate of risk, and technically implies implies either zero threat or zero risk of human error.
That said, I think it’s worth distinguishing the story “we will be able to see the threat and we will stop,” from “there is no threat.” The first story makes it clearer that there is actually broad support for measurement to detect risk and institutional structures that can slow down if risk is large.
It also feels like the key disagreement isn’t about corporate law or arguments for risk, it’s about how much warning we get in advance and how reliably institutions like Meta would stop building AI if they don’t figure out a good way to make AI safe. I think both are interesting, but the “how much warning” disagreement is probably more important for technical experts to debate—my rough sense is that the broader intellectual world already isn’t really on Yann’s page when he says “we’d definitely stop if this was unsafe, nothing to worry about.”
And the idea that intelligent systems will inevitably want to take over, dominate humans, or just destroy humanity through negligence is preposterous. They would have to be specifically designed to do so. Whereas we will obviously design them to not do so.
He specifically told me when I asked this question that his views were the same as Geoff Hinton and Scott Aaronson and neither of them hold the view that smarter than human AI poses zero threat to humanity.
From a Facebook discussion with Scott Aaronson yesterday:
Yann: I think neither Yoshua nor Geoff believe that AI is going kill us all with any significant probability.
Scott: Well, Yoshua signed the pause letter, and wrote an accompanying statement about what he sees as the risk to civilization (I agree that there are many civilizational risks short of extinction). In his words: “No one, not even the leading AI experts, including those who developed these giant AI models, can be absolutely certain that such powerful tools now or in the future cannot be used in ways that would be catastrophic to society.”
Geoff said in a widely-shared recent video that it’s “not inconceivable” that AI will wipe out humanity, and didn’t offer any reassurances about it being vanishingly unlikely.
Yann: Scott Aaronson he is worried about catastrophic disruptions of the political, economic, and environmental systems.
I don’t want to speak for him, but I doubt he worries about a Yuddite-style uncontrollable “hard takeoff”
On the one hand, “Yuddite” is (kinda rude but) really rather clever.
On the other hand, the actual Luddites were concerned about technological unemployment which makes “Yuddite” a potentially misleading term, given that there’s something of a rift between the “concerned about ways in which AI might lead to people’s lives being worse within a world that’s basically like the one we have now” and “concerned about the possibility that AI will turn the world so completely upside down that there’s no room for us in it any more” camps and Yudkowsky is very firmly in the latter camp.
On the third hand, the Luddites made a prediction about a bad (for them) outcome, and were absolutely correct. They were against automatic looms because they thought the autolooms would replace their artisan product with lower quality goods and also worsen their wages and working conditions. They were right: https://www.smithsonianmag.com/history/what-the-luddites-really-fought-against-264412/
he is worried about catastrophic disruptions of the political, economic, and environmental systems
Ok, but how is this any different in practice? Or preventable via “corporate law”? It feels to me like people make too much of a distinction between slow and fast take offs scenarios, as if somewhat, if humans appear to be in the loop a bit more, this makes the problem less bad or less AI-related.
Essentially, if your mode of failure follows almost naturally from introducing AI system in current society and basic economic incentives, to the point that you can’t really look at any part of the process and identify anyone maliciously and intentionally setting it up to end the world, yet it does end the world, then it’s an AI problem. It may be a weird, slow, cyborg-like amalgamation of AI and human society that caused the catastrophe instead of a singular agentic AI taking everything over quickly, but the AI is still the main driver, and the only way to avoid the problem is to make AI extremely robust not just to intentional bad use but also to unintentional bad incentives feedback loops, essentially smart and moral enough to stop its own users and creators when they don’t know any better. Or alternatively, to just not make the AI at all.
Honestly, given what Facebook’s recommender systems have already caused, it’s disheartening that the leader of AI research at Meta doesn’t get something like this.
The linked tweet doesn’t actually make this claim, or even imply it. Is Yann on the record saying “zero threat” somewhere else? Or even claiming that the level of risk is <0.1%? Failing that, is there somewhere he has implied that the risk is very small?
It seems like his position is more like: this problem looks likely to be manageable in a variety of ways, so let’s push ahead and try to manage it. The probability of doom today is very low, so we can reconsider later if things look bleak.
He does say “the ‘hard takeoff scenario’ is utterly impossible.” I’m a bit sympathetic to this, since Eliezer’s description of super hard takeoff feels crazy to me as well. That said: (i) “utterly impossible” is too strong even for super hard takeoffs, which seem very unlikely but possible, (ii) based on his rhetoric about adaptation Yann seems to be implicitly rejecting even months-long takeoffs, which I think have >10% probability depending on how you define them.
I believe I have seen him make multiple statements on Twitter over months that express this point of view, and I see his statements herein as reinforcing that, but in the interests of this not being distracting to the main point I am editing this line.
Also including an additional exchange from yesterday he had with Elon Musk.
Mods, please reimport.
EDIT: Also adding his response to Max Tegmark from yesterday, and EY’s response to that, at the bottom, but raising the bar for further inclusions substantially.
Includes this quote: But regardless, until we have a semi credible design, we are discussing the sex of angels. The worst that can happen is that we won’t figure out a good way to make them safe and we won’t deploy them.
Which, logically, says zero risk, since the worst that can happen is non-deployment?
(I still have no idea where the button is that’s supposed to let me switch to a WYSIWYG editor)
I agree that “the worst that can happen is...” suggests an unreasonably low estimate of risk, and technically implies implies either zero threat or zero risk of human error.
That said, I think it’s worth distinguishing the story “we will be able to see the threat and we will stop,” from “there is no threat.” The first story makes it clearer that there is actually broad support for measurement to detect risk and institutional structures that can slow down if risk is large.
It also feels like the key disagreement isn’t about corporate law or arguments for risk, it’s about how much warning we get in advance and how reliably institutions like Meta would stop building AI if they don’t figure out a good way to make AI safe. I think both are interesting, but the “how much warning” disagreement is probably more important for technical experts to debate—my rough sense is that the broader intellectual world already isn’t really on Yann’s page when he says “we’d definitely stop if this was unsafe, nothing to worry about.”
this was posted after your comment, but i think this is close enough:
@ylecun
I’m most convinced by the second sentence:
Which definitely seems to be dismissing the possibility of alignment failures.
My guess would be that he would back off of this claim if pushed on it explicitly, but I’m not sure. And it is at any rate indicative of his attitude.
He specifically told me when I asked this question that his views were the same as Geoff Hinton and Scott Aaronson and neither of them hold the view that smarter than human AI poses zero threat to humanity.
From a Facebook discussion with Scott Aaronson yesterday:
Yann: I think neither Yoshua nor Geoff believe that AI is going kill us all with any significant probability.
Scott: Well, Yoshua signed the pause letter, and wrote an accompanying statement about what he sees as the risk to civilization (I agree that there are many civilizational risks short of extinction). In his words: “No one, not even the leading AI experts, including those who developed these giant AI models, can be absolutely certain that such powerful tools now or in the future cannot be used in ways that would be catastrophic to society.”
Geoff said in a widely-shared recent video that it’s “not inconceivable” that AI will wipe out humanity, and didn’t offer any reassurances about it being vanishingly unlikely.
https://yoshuabengio.org/2023/04/05/slowing-down-development-of-ai-systems-passing-the-turing-test/
https://twitter.com/JMannhart/status/1641764742137016320
Yann: Scott Aaronson he is worried about catastrophic disruptions of the political, economic, and environmental systems. I don’t want to speak for him, but I doubt he worries about a Yuddite-style uncontrollable “hard takeoff”
On the one hand, “Yuddite” is (kinda rude but) really rather clever.
On the other hand, the actual Luddites were concerned about technological unemployment which makes “Yuddite” a potentially misleading term, given that there’s something of a rift between the “concerned about ways in which AI might lead to people’s lives being worse within a world that’s basically like the one we have now” and “concerned about the possibility that AI will turn the world so completely upside down that there’s no room for us in it any more” camps and Yudkowsky is very firmly in the latter camp.
On the third hand, the Luddites made a prediction about a bad (for them) outcome, and were absolutely correct. They were against automatic looms because they thought the autolooms would replace their artisan product with lower quality goods and also worsen their wages and working conditions. They were right: https://www.smithsonianmag.com/history/what-the-luddites-really-fought-against-264412/
Ok, but how is this any different in practice? Or preventable via “corporate law”? It feels to me like people make too much of a distinction between slow and fast take offs scenarios, as if somewhat, if humans appear to be in the loop a bit more, this makes the problem less bad or less AI-related.
Essentially, if your mode of failure follows almost naturally from introducing AI system in current society and basic economic incentives, to the point that you can’t really look at any part of the process and identify anyone maliciously and intentionally setting it up to end the world, yet it does end the world, then it’s an AI problem. It may be a weird, slow, cyborg-like amalgamation of AI and human society that caused the catastrophe instead of a singular agentic AI taking everything over quickly, but the AI is still the main driver, and the only way to avoid the problem is to make AI extremely robust not just to intentional bad use but also to unintentional bad incentives feedback loops, essentially smart and moral enough to stop its own users and creators when they don’t know any better. Or alternatively, to just not make the AI at all.
Honestly, given what Facebook’s recommender systems have already caused, it’s disheartening that the leader of AI research at Meta doesn’t get something like this.