I think that it’s likely sometimes the case that this is just a miscommunication, but I think often the disagreement is just straightforwardly about epistemology / how much Knightian uncertainty is justified. Like, considering Tyler Cowen’s recent post, I see the primary content as being (paraphrased) “our current civilization is predicated on taking an optimistic view to Knightian uncertainty / letting people act on their own judgment, and we should keep doing what worked in the past.” The issue isn’t that Eliezer said “>90% doom” instead of “at least 10% doom”, it’s that Eliezer thinks this isn’t the right way to reason about AI as compared to other technologies.
[To the extent you’re just saying “look, this is an avoidable miscommunication that’s worth paying the price to avoid”, I think I agree; I’m just suspecting the disagreement isn’t just surface-level.]
I guess this depends on what you mean by “surface-level”. I do think there are deep philosophical disagreements here about Knightian uncertainty. I just don’t know whether they matter much, conditional on actually getting people to sit down and talk details about AI. (I don’t know if the miscommuncation with Tyler specifically is avoidable or not, but it seems like we agree that it is in many other cases.)
Maybe one case you might make is: look, MIRI thinks the people working on ML safety at big labs aren’t doing anything useful, and they know all the arguments, and maybe getting them to converge on the Knightian uncertainty stuff is important in bridging the remaining gap.
But even from that perspective, that sure seems like very much “step 2”, where step 1 is convincing people “hey, this AI risk stuff isn’t crazy”—which is the step I claim this post helps with.
(And I strong disagree with that perspective regardless, I think that the key step here is just making the arguments more extensively, and the fact that all the unified cases for AI risk have been written by more ML-safety-sympathetic people like me, Ajeya, and Joe (with the single exception of “AGI ruin” EDIT: and of course Superintelligence) is indicative that that strategy mostly hasn’t been tried.)
the fact that all the unified cases for AI risk have been written by more ML-safety-sympathetic people like me, Ajeya, and Joe (with the single exception of “AGI ruin”) is indicative that that strategy mostly hasn’t been tried.
I’m not sure what you mean by this, but here’s half-a-dozen “unified cases for AI risk” made by people like Eliezer Yudkowsky, Nick Bostrom, Stuart Armstrong, and myself:
There’s a type signature that I’m trying to get at with the “unified case” description (which I acknowledge I didn’t describe very well in my previous comment), which I’d describe as “trying to make a complete argument (or something close to it)”. I think all the things I was referring to meet this criterion; whereas, of the things you listed, only Superintelligence seems to, with the rest having a type signature more like “trying to convey a handful of core intuitions”. (CFAI may also be in the former category, I haven’t read it, but it was long ago enough that it seems much less relevant to questions related to persuasion today.)
It seems to me that this is a similar complaint as Eliezer’s when he says in List of Lethalities:
“The fact that, twenty-one years into my entering this death game, seven years into other EAs noticing the death game, and two years into even normies starting to notice the death game, it is still Eliezer Yudkowsky writing up this list, says that humanity still has only one gamepiece that can do that.”
except that I’m including a few other pieces of (ML-safety-sympathetic) work in the same category.
I think that it’s likely sometimes the case that this is just a miscommunication, but I think often the disagreement is just straightforwardly about epistemology / how much Knightian uncertainty is justified. Like, considering Tyler Cowen’s recent post, I see the primary content as being (paraphrased) “our current civilization is predicated on taking an optimistic view to Knightian uncertainty / letting people act on their own judgment, and we should keep doing what worked in the past.” The issue isn’t that Eliezer said “>90% doom” instead of “at least 10% doom”, it’s that Eliezer thinks this isn’t the right way to reason about AI as compared to other technologies.
[To the extent you’re just saying “look, this is an avoidable miscommunication that’s worth paying the price to avoid”, I think I agree; I’m just suspecting the disagreement isn’t just surface-level.]
I guess this depends on what you mean by “surface-level”. I do think there are deep philosophical disagreements here about Knightian uncertainty. I just don’t know whether they matter much, conditional on actually getting people to sit down and talk details about AI. (I don’t know if the miscommuncation with Tyler specifically is avoidable or not, but it seems like we agree that it is in many other cases.)
Maybe one case you might make is: look, MIRI thinks the people working on ML safety at big labs aren’t doing anything useful, and they know all the arguments, and maybe getting them to converge on the Knightian uncertainty stuff is important in bridging the remaining gap.
But even from that perspective, that sure seems like very much “step 2”, where step 1 is convincing people “hey, this AI risk stuff isn’t crazy”—which is the step I claim this post helps with.
(And I strong disagree with that perspective regardless, I think that the key step here is just making the arguments more extensively, and the fact that all the unified cases for AI risk have been written by more ML-safety-sympathetic people like me, Ajeya, and Joe (with the single exception of “AGI ruin” EDIT: and of course Superintelligence) is indicative that that strategy mostly hasn’t been tried.)
I’m not sure what you mean by this, but here’s half-a-dozen “unified cases for AI risk” made by people like Eliezer Yudkowsky, Nick Bostrom, Stuart Armstrong, and myself:
2001 - https://intelligence.org/files/CFAI.pdf
2014 - https://smarterthan.us/
2014 - Superintelligence
2015 - https://intelligence.org/2015/07/24/four-background-claims/
2016 - https://intelligence.org/2016/12/28/ai-alignment-why-its-hard-and-where-to-start/
2017 - https://intelligence.org/2017/04/12/ensuring/
There’s a type signature that I’m trying to get at with the “unified case” description (which I acknowledge I didn’t describe very well in my previous comment), which I’d describe as “trying to make a complete argument (or something close to it)”. I think all the things I was referring to meet this criterion; whereas, of the things you listed, only Superintelligence seems to, with the rest having a type signature more like “trying to convey a handful of core intuitions”. (CFAI may also be in the former category, I haven’t read it, but it was long ago enough that it seems much less relevant to questions related to persuasion today.)
It seems to me that this is a similar complaint as Eliezer’s when he says in List of Lethalities:
except that I’m including a few other pieces of (ML-safety-sympathetic) work in the same category.