You haven’t commented much on Eliezer’s views on the social approach to slow down the development of AGI—the blocks starting with
I don’t know how to effectively prevent or slow down the “next competitor” for more than a couple of years even in plausible-best-case scenarios.
and
I don’t want to sound like I’m dismissing the whole strategy, but it sounds a lot like the kind of thing that backfires because you did not get exactly the public reaction you wanted
On slowing down, I’d say strong inside view agreement, I don’t see a way either, not without something far more universal. There’s too many next competitors. Could have been included, probably excluded due to seeming like it followed from other points and was thus too obvious.
On the likelihood of backfire, strong inside view agreement. Not sure why that point didn’t make it into the post, but consider this an unofficial extra point (43?), of something like (paraphrase, attempt 1) “Making the public broadly aware of and afraid of these scenarios is likely to backfire and result in counterproductive action.”
On the object level it looks like there are a spectrum of society-level interventions starting from “incentivizing research that wouldn’t be published” (which is supported by Eliezer) and all the way to “scaring the hell out of general public” and beyond. For example, I can think of removing $FB and $NVDA from ESGs, disincentivizing publishing code and research articles in AI, introducing regulation of compute-producing industry. Where do you think the line should be drawn between reasonable interventions and ones that are most likely to backfire?
On the meta level, the whole AGI foom management/alignment starts not some abstract 50 years in the future, but right now, with the managing of ML/AI research by humans. Do you know of any practical results produced by alignment research community that can be used right now to manage societal backfire / align incentives?
You haven’t commented much on Eliezer’s views on the social approach to slow down the development of AGI—the blocks starting with
and
What’s your take on this?
On slowing down, I’d say strong inside view agreement, I don’t see a way either, not without something far more universal. There’s too many next competitors. Could have been included, probably excluded due to seeming like it followed from other points and was thus too obvious.
On the likelihood of backfire, strong inside view agreement. Not sure why that point didn’t make it into the post, but consider this an unofficial extra point (43?), of something like (paraphrase, attempt 1) “Making the public broadly aware of and afraid of these scenarios is likely to backfire and result in counterproductive action.”
What particular counterproductive actions by the public are we hoping to avoid?
On the object level it looks like there are a spectrum of society-level interventions starting from “incentivizing research that wouldn’t be published” (which is supported by Eliezer) and all the way to “scaring the hell out of general public” and beyond. For example, I can think of removing $FB and $NVDA from ESGs, disincentivizing publishing code and research articles in AI, introducing regulation of compute-producing industry. Where do you think the line should be drawn between reasonable interventions and ones that are most likely to backfire?
On the meta level, the whole AGI foom management/alignment starts not some abstract 50 years in the future, but right now, with the managing of ML/AI research by humans. Do you know of any practical results produced by alignment research community that can be used right now to manage societal backfire / align incentives?