Firstly, I think I should say that this post was very much not intended as anything like a scathing takedown of Ord and Yudkowsky’s claims or evidence. Nor did I mean to imply I’m giving definitive arguments that these cases provide no evidence for the claims made. I mean this to have more of a collaborative than combative spirit in relation to Ord and Yudkowsky’s projects.
My aim was simply to “prod at each suspicious plank on its own terms, and update incrementally.” And my key conclusion is that the authors, “in my opinion, imply these cases support their claims more clearly than they do”—not that the cases provide no evidence. It seems to me healthy to question evidence we have—even for conclusions we do still think are right, and even when our questions don’t definitively cut down the evidence, but rather raise reasons for some doubt.
It’s possible I could’ve communicated that better, and I’m open to suggestions on that front. But from re-reading the post again, especially the intro and conclusion, it does seem I repeatedly made explicit statements to this effect. (Although I did realise after going to bed last night that the “And I don’t think we should update much...” sentence was off, so I’ve now made that a tad clearer.)
I’ve split my response about the Rutherford and Fermi cases into different comments.
Of the 4 case studies you criticize, your claim actually supports them in the first one, you agree the second one is accurate, and you provide only speculations and no actual criticisms in the third and fourth.
Again, I think this sentence may reflect interpreting this post as much more strident and critical than it was really meant to be. I may be wrong about the “direct connection” thing (discussed in a separate comment), but I do think I raise plausible reasons for at least some doubt about (rather than outright dismissal of) the evidence each case provides, compared to how a reader might initially interpret them.
I’m also not sure what “only speculations and no actual criticisms” would mean. If you mean e.g. that I don’t have evidence that a lot of Americans would’ve believed nuclear weapons would exist someday, then yes, that’s true. I don’t claim otherwise. But I point out a potentially relevant disanalogy between nuclear weapons development and AI development. And I point give some evidence that “the group of people who did know about nuclear weapons before the bombing of Hiroshima, or who believed such weapons may be developed soon, was (somewhat) larger than one might think from reading Yudkowsky’s essay.” And I do give some evidence for that, as well as pointing out that I’m not aware of evidence either way for one relevant point.
Also, I don’t really claim any of this post to be “criticism”, at least in the usual fairly negative sense, just “prod[ding] at each suspicious plank on its own terms”. I’m explicitly intending to make only relatively weak claims, really.
And then the “Sample size and representativeness” section provides largely separate reasons why it might not make much sense to update much on these cases (at least from a relatively moderate starting point) even ignoring those reasons for doubt. (Though see the interesting point 3 in Daniel Kokotajlo’s comment.)
Firstly, I think I should say that this post was very much not intended as anything like a scathing takedown of Ord and Yudkowsky’s claims or evidence. Nor did I mean to imply I’m giving definitive arguments that these cases provide no evidence for the claims made. I mean this to have more of a collaborative than combative spirit in relation to Ord and Yudkowsky’s projects.
My aim was simply to “prod at each suspicious plank on its own terms, and update incrementally.” And my key conclusion is that the authors, “in my opinion, imply these cases support their claims more clearly than they do”—not that the cases provide no evidence. It seems to me healthy to question evidence we have—even for conclusions we do still think are right, and even when our questions don’t definitively cut down the evidence, but rather raise reasons for some doubt.
It’s possible I could’ve communicated that better, and I’m open to suggestions on that front. But from re-reading the post again, especially the intro and conclusion, it does seem I repeatedly made explicit statements to this effect. (Although I did realise after going to bed last night that the “And I don’t think we should update much...” sentence was off, so I’ve now made that a tad clearer.)
I’ve split my response about the Rutherford and Fermi cases into different comments.
Again, I think this sentence may reflect interpreting this post as much more strident and critical than it was really meant to be. I may be wrong about the “direct connection” thing (discussed in a separate comment), but I do think I raise plausible reasons for at least some doubt about (rather than outright dismissal of) the evidence each case provides, compared to how a reader might initially interpret them.
I’m also not sure what “only speculations and no actual criticisms” would mean. If you mean e.g. that I don’t have evidence that a lot of Americans would’ve believed nuclear weapons would exist someday, then yes, that’s true. I don’t claim otherwise. But I point out a potentially relevant disanalogy between nuclear weapons development and AI development. And I point give some evidence that “the group of people who did know about nuclear weapons before the bombing of Hiroshima, or who believed such weapons may be developed soon, was (somewhat) larger than one might think from reading Yudkowsky’s essay.” And I do give some evidence for that, as well as pointing out that I’m not aware of evidence either way for one relevant point.
Also, I don’t really claim any of this post to be “criticism”, at least in the usual fairly negative sense, just “prod[ding] at each suspicious plank on its own terms”. I’m explicitly intending to make only relatively weak claims, really.
And then the “Sample size and representativeness” section provides largely separate reasons why it might not make much sense to update much on these cases (at least from a relatively moderate starting point) even ignoring those reasons for doubt. (Though see the interesting point 3 in Daniel Kokotajlo’s comment.)