This video is based on this article. @jai has written both the original article and the script for the video.
Script:
The ACM Turing Award is the highest distinction in computer science, comparable to the Nobel Prize. In 2018 it was awarded to three pioneers of the deep learning revolution: Geoffrey Hinton, Yoshua Bengio, and Yann LeCun.
In May 2023, Geoffrey Hinton left Google so that he could speak openly about the dangers of advanced AI, agreeing that “it could figure out how to kill humans” and saying “it’s not clear to me that we can solve this problem.”
Later that month, Yoshua Bengio wrote a blog post titled “How Rogue AIs may Arise”, in which he defined a “rogue AI” as “an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere.”
Yann LeCun continues to refer to thoseanyone suggesting that we’re facing severe and imminent risk as “professional scaremongers” and says it’s a “simple fact” that “the people who are terrified of AGI are rarely the people who actually build AI models.”
LeCun is a highly accomplished researcher, but in light of Bengio and Hinton’s recent comments it’s clear that he’s misrepresenting the field whether he realizes it or not. There is not a consensus among professional researchers that AI research is safe. Rather, there is considerable and growing concern that advanced AI could pose extreme risks, and this concern is shared by not only both of LeCun’s award co-recipients, but the headsleaders of all three leading AI labs (OpenAI, Anthropic, and Google DeepMind):
Demis Hassabis, CEO of DeepMind, said in an interview with Time Magazine: “When it comes to very powerful technologies—and obviously AI is going to be one of the most powerful ever—we need to be careful. Not everybody is thinking about those things. It’s like experimentalists, many of whom don’t realize they’re holding dangerous material.”
Anthropic, in their public statement “Core Views on AI Safety”, says: “One particularly important dimension of uncertainty is how difficult it will be to develop advanced AI systems that are broadly safe and pose little risk to humans. Developing such systems could lie anywhere on the spectrum from very easy to impossible.”
And OpenAI, in their blog post “Planning for AGI and Beyond”, says “Some people in the AI field think the risks of AGI (and successor systems) are fictitious; we would be delighted if they turn out to be right, but we are going to operate as if these risks are existential.” Sam Altman, the current CEO of OpenAI, once said “Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity. ”
There are objections one could raise to the idea that advanced AI poses significant risk to humanity, but “it’s a fringe idea that actual AI experts do not take seriously” is no longer among them. Instead, a growing share of experts are echoing the conclusion reached by Alan Turing, considered by many to be the father of computer science and artificial intelligence, back in 1951: “[I]t seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers. [...] At some stage therefore we should have to expect the machines to take control.”
Sometimes I wonder if the best case of assuring that the world will collaborate on AGI safety would be some sort of Watchmen-like hoax with a fake hyper aggressive AGI-mimicking system/event that scares the world into collaborating in lockstep until safe AGI can be deployed. Without real urgency to avoid existential risks, the greatest risk of AGI doom IMO is competition between nation states who see deployment of AGI as a means to fully realize their nationalistic aims, under-appreciating the risks of their AGI war machine going rogue.
The thing that made AI risk “real” for me was a report of an event that turned out not to have happened (seemingly just a miscommunication). My brain was already very concerned, but my gut had not caught up until then. That said, I do not think this should be taken as a norm, for three reasons:
Creating hoaxes in support of a cause is a good way to turn a lot of people against a cause
In general, if you feel a need to fake evidence for your position, that is itself is weak evidence against your position
I don’t like dishonesty
If AI capabilities continue to progress and if AI x-risk is a real problem (which I think it is, credence ~95%), then I hope we get a warning shot. But I think a false flag “warning shot” has negative utility.
May I remind you how that is implied to end in Watchmen? That’s just one of the reasons why this kind of thing isn’t just horribly unethical, but also probably ineffectual. That’s the problem with the supervillain brand of consequentialism in general.
As I recall, it ends with the world successfully united against the alien threat and the imminent nuclear apocalypse averted. The Rorschach notebook with his expose of Ozymandias is successfully mailed before he can be killed, but in line with the failure of the rest of his life and the powerlessness of the truth in the cynical Watchmen universe where no one else cares about truth & justice as much as the autistic Rorschach, it is implied that it will be published (and some will know the truth of Ozymandias’s hoax) but that it will be ignored because it is published in the in-universe equivalent of the John Birch Society’s magazine (or possibly Larouchites, I was never quite sure) and Ozymandias has killed or co-opted or driven away everyone who might confirm the claims (such as the researchers who created the monster, or Night Owl or Doctor Manhattan) while remaining the smartest & richest man in the world.
I would also note that “hey, what if your consequentialism didn’t lead to good consequences, how about that? betcha never thought of that smartiepants” would be a pretty poor critique of consequentialism—it is, in fact, the standard Hollywood-movie-level critique, and if your interpretation of Alan Moore’s Watchmen reduces his work to something as simple-minded as a Hollywood movie, it’s probably wrong.
I mean, if your consequentialist plan requires killing millions it better have one damn high chance of succeeding. I don’t see a point in focusing on Rorschach’s journal in the end if not to suggest that, as the peak of dramatic irony, after dying a complete loser ignored by everyone and failing to stop Ozymandias, Rorschach’s last action will in fact trigger a chain of events that ruins even the one good thing to come out of his plan. The thing is that this kind of false flag plan is fragile; it doesn’t take much to find out it’s actually fake, which ends up accomplishing the exact opposite effect.
If the alternative to your consequentialist plan is an all-out thermonuclear war between the USSR and USA within months which will end civilization with the deaths of hundreds of millions and perhaps billions of people, it doesn’t particularly need a ‘damn high chance of succeeding’. How many nukes were targeted at Manhattan Times Square under the Russian equivalent of their SIOP? Probably >1...
It certainly is ironic, but as I said, to emphasize the powerlessness and futility of the ‘truth and justice and the American Way’, as Rorschach worships the naive superhero ideals. Who watches the watchmen? No one.
Not really? Can you name three examples of false flags where they accomplished their initial political goals and triggered the intended war or other major event, but then were exposed, and everyone went ‘oh my gosh, we were tricked! it wasn’t true!’ and immediately undid everything they had already done? I can’t.
False flags generally only fail if exposed early on before anything can happen, while there is still a public ‘choice’ in the matter. (The Russian false flags in the runup to Ukraine, for example—minimal propaganda/political effect because exposed so quickly, so Putin invaded ‘naked’.) History is filled with false flags which were exposed not terribly long after, to no real effect. The Reichstag Fire, the Manchurian Incident, the Gulf of Tonkin incident… (The Maine, WWI ‘German atrocities in Belgium’...) Sure, they got exposed, but the Nazis were still in power, Imperial Japan remained invaded, the Vietnam War remained the Vietnam War, etc.
I mean, there’s a big difference between a false flag to cause a war, and a false flag to prevent one. If you successfully started a war, after a while people have given each other plenty of reasons for hate, and removing the casus belli doesn’t do much. If you successfully averted one, the hatred will only be rekindled tenfold upon discovering you’ve all been duped.
That’s a rather extreme idea, even if humanity was on the brink of extinction deceit is hard to justify.
We haven’t even scratched the surface of possible practical solutions, once those are exhausted there are many more possible paths.