A bit of anecdotal impressions, yes, but mainly I just think that in humans being smart, conscientious, reflective, etc. enough to be the brightest researcher a big AI lab is actually pretty correlated with being Good (and also, that once you actually solve the technical problems, it doesn’t take that much Goodness to do the right thing for the collective and not just yourself).
Or, another way of looking at it, I find Scott Aaronson’s perspective convincing, when it is applied to humans. I just don’t think it will apply at all to the first kinds of AIs that people are actually likely to build, for technical reasons.
I think there are way more transhumanists and post-humanists at AGI labs than you imagine. Richard Sutton is a famous example (btw, I’ve just discovered that he moved from DeepMind to Keen Technologies, John Carmack’s venture), but I believe there are many more of them, but they disguise themselves for political reasons.
A bit of anecdotal impressions, yes, but mainly I just think that in humans being smart, conscientious, reflective, etc. enough to be the brightest researcher a big AI lab is actually pretty correlated with being Good (and also, that once you actually solve the technical problems, it doesn’t take that much Goodness to do the right thing for the collective and not just yourself).
Or, another way of looking at it, I find Scott Aaronson’s perspective convincing, when it is applied to humans. I just don’t think it will apply at all to the first kinds of AIs that people are actually likely to build, for technical reasons.
I think there are way more transhumanists and post-humanists at AGI labs than you imagine. Richard Sutton is a famous example (btw, I’ve just discovered that he moved from DeepMind to Keen Technologies, John Carmack’s venture), but I believe there are many more of them, but they disguise themselves for political reasons.