It seems odd to insist that someone explicitly working on benevolence should consider themselves to be in the same reference class as someone who thinks they just need to take care of the AGI and the benevolence will pretty much take care of itself.
I wasn’t intending to use “AGI researchers” as a reference class to show that Eliezer’s work is likely to have net negative consequences, but to show that people whose work can reasonably be expected to have net negative consequences (of whom AGI researchers is a prominent class) still tend not to believe such claims, and therefore Eliezer’s failure to be convinced is not of much evidential value to others.
The reference class I usually do have in mind when I think of Eliezer is philosophers who think they have the right answer to some philosophical problem (virtually all of whom end up being wrong or at least incomplete even if they are headed in the right direction).
ETA: I’ve written a post that expands on this comment.
It seems odd to insist that someone explicitly working on benevolence should consider themselves to be in the same reference class as someone who thinks they just need to take care of the AGI and the benevolence will pretty much take care of itself.
I wasn’t intending to use “AGI researchers” as a reference class to show that Eliezer’s work is likely to have net negative consequences, but to show that people whose work can reasonably be expected to have net negative consequences (of whom AGI researchers is a prominent class) still tend not to believe such claims, and therefore Eliezer’s failure to be convinced is not of much evidential value to others.
The reference class I usually do have in mind when I think of Eliezer is philosophers who think they have the right answer to some philosophical problem (virtually all of whom end up being wrong or at least incomplete even if they are headed in the right direction).
ETA: I’ve written a post that expands on this comment.