Widening Overton Window—Open Thread
Major news outlets have published articles about the Future of Life Institute’s Open Letter. Time Magazine published an opinion piece by Elizer Yudkowsky. Lex Friendman featured EY on his podcast. Several US Members of Congress have spoken about the risks of AI. And a Fox News reporter asked a what the US President is doing to combat AI x-risk at a White House Press Conference.
Starting an Open Thread to discuss this, and how best to capitalize on this sudden attention.
Links:
WH Press Conference: https://twitter.com/therecount/status/1641526864626720774
Time Magazine: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/
FLI Letter: https://futureoflife.org/open-letter/pause-giant-ai-experiments/
The public early Covid-19 conversation (in like Feb-April 2020) seemed pretty hopeful to me—decent arguments, slow but asymmetrically correct updating on some of those arguments, etc. Later everything became politicized and stupid re: covid.
Right now I think there’s some opportunity for real conversation re: AI. I don’t know what useful thing follows from that, but I do think it may not last, and that it’s pretty cool. I care more about the “an opening for real conversation” thing than for the changing overton window as such, although I think the former probably follows from the latter (first encounters are often more real somehow).
Sundar Pichai was on the Hard Fork podcast today and was asked directly by Kevin Roose and Casey Newton about the FLI letter as well as long-term AI risk. I pulled out some of Pichai’s answers here:
The FLI letter proposal:
Game theory:
Long term AI x-risk:
A reason for optimism:
The increased public attention towards AI Safety risk is probably a good thing. But, when stuff like this is getting lumped in with the rest of AI Safety, it feels like the public-facing slow-down-AI movement is going to be a grab-bag of AI Safety, AI Ethics, and AI… privacy(?). As such, I’m afraid that the public discourse will devolve into “Woah-there-Slow-AI” and “GOGOGOGO” tribal warfare; from the track record of American politics, this seems likely—maybe even inevitable?
More importantly, though, what I’m afraid of is that this will translate into adversarial relations between AI Capabilities organizations and AI Safety orgs (more generally, that capabilities teams will become less inclined to incorporate safety concerns in their products).
I’m not actually in an AI organization, so if someone is in one and has thoughts on this dynamic happening/not happening, I would love to hear.
Yeah, since the public currently doesn’t have much of an opinion on it, trying to get the correct information out seems critical. I fear some absolutely useless legislation will get passed, and everyone will just forget about it once the shock-value of GPT wears off.
I’m currently thinking that if there are any political or PR resources available to orgs (AI-related or EA) now is the time to use them. Public interest is fickle, and currently most people don’t seem to know what to think, and are looking for serious-seeming people to tell them whether or not to see this as a threat. If we fail to act, someone else will likely hijack the narrative, and push it in a useless or even negative direction. I don’t know how far we can go, or how likely it is, but we can’t assume we’ll get another chance before the public falls back asleep or gets distracted (the US has an election next year, so most discourse will then likely become poisoned). This is especially important for those in the community who are viewed as “serious people” or “serious organizations” (lots of academic credentials, etc.)
I agree that AI absolutely needs to be regulated ASAP to mitigate the many potential harms that could arise from its use. So, even though the FLI letter is flimsy and vague, I appreciate its performance of concern.
Yudkowsky’s worry about runaway intelligence is, I think, an ungrounded distraction. It is ungrounded because Yudkowsky does not have a coherent theory of intentionality that makes sense of the idea of an algorithm gaining a capacity to engage in its own goal directed activity. It is a distraction from the public discourse about the very real, immediate, tangible risks of harms caused AI systems we have today.
The independent red-teaming organization ARC Evals that OpenAI partnered with to evaluate GPT-4 seems to disagree with this. While they don’t use the term “runaway intelligence”, they have flagged similar dangerous capabilities that they think will possibly be in reach for the next models beyond GPT-4:
Thanks for sharing the link to ARC. It seems to me the kinds of things they are testing for and worried about are analogous to the risks of self-driving cars: when you incorporate ML systems into a range of human activities, their behaviour is unpredictable and can be dangerous. I am glad ARC is doing the work they are doing. People are using unpredictable tools and ARC is investigating the risks. That’s great.
I don’t think these capabilities ARC is looking at are “similar” to runaway intelligence, as you suggest. They clearly do not require it. They are far more mundane (but dangerous nonetheless, as you rightly point out).
At one point in the ARC post, they hint vaguely at being motivated by Yudkowsky-like worries: “As AI systems improve, it is becoming increasingly difficult to rule out that models might be able to autonomously gain resources and evade human oversight – so rigorous evaluation is essential.” They seem to be imagining a system giving itself goals, such that it is motivated to engage in tactical deception to carry out its goals—a behaviour we find in a range of problem-solving non-human animals. It strikes me as a worry that is extraneous to the good work ARC is doing. And the end of the quote is odd, since rigorous evaluation is clearly essential regardless of autonomous resource gains or oversight evasion.