On slowing down, I’d say strong inside view agreement, I don’t see a way either, not without something far more universal. There’s too many next competitors. Could have been included, probably excluded due to seeming like it followed from other points and was thus too obvious.
On the likelihood of backfire, strong inside view agreement. Not sure why that point didn’t make it into the post, but consider this an unofficial extra point (43?), of something like (paraphrase, attempt 1) “Making the public broadly aware of and afraid of these scenarios is likely to backfire and result in counterproductive action.”
On the object level it looks like there are a spectrum of society-level interventions starting from “incentivizing research that wouldn’t be published” (which is supported by Eliezer) and all the way to “scaring the hell out of general public” and beyond. For example, I can think of removing $FB and $NVDA from ESGs, disincentivizing publishing code and research articles in AI, introducing regulation of compute-producing industry. Where do you think the line should be drawn between reasonable interventions and ones that are most likely to backfire?
On the meta level, the whole AGI foom management/alignment starts not some abstract 50 years in the future, but right now, with the managing of ML/AI research by humans. Do you know of any practical results produced by alignment research community that can be used right now to manage societal backfire / align incentives?
On slowing down, I’d say strong inside view agreement, I don’t see a way either, not without something far more universal. There’s too many next competitors. Could have been included, probably excluded due to seeming like it followed from other points and was thus too obvious.
On the likelihood of backfire, strong inside view agreement. Not sure why that point didn’t make it into the post, but consider this an unofficial extra point (43?), of something like (paraphrase, attempt 1) “Making the public broadly aware of and afraid of these scenarios is likely to backfire and result in counterproductive action.”
What particular counterproductive actions by the public are we hoping to avoid?
On the object level it looks like there are a spectrum of society-level interventions starting from “incentivizing research that wouldn’t be published” (which is supported by Eliezer) and all the way to “scaring the hell out of general public” and beyond. For example, I can think of removing $FB and $NVDA from ESGs, disincentivizing publishing code and research articles in AI, introducing regulation of compute-producing industry. Where do you think the line should be drawn between reasonable interventions and ones that are most likely to backfire?
On the meta level, the whole AGI foom management/alignment starts not some abstract 50 years in the future, but right now, with the managing of ML/AI research by humans. Do you know of any practical results produced by alignment research community that can be used right now to manage societal backfire / align incentives?