If I had to guess, the lack of discussion on this seems likely due to a founder effect. The people pulling the alarm in the early days of AGI safety concerns were disproportionately to the technical/philosophical side rather than to the policy/outreach/activism side.
Also, the people pulling the alarm in the early days of AGI safety concerns, are also people interested in AGI. They find it cool. I get the impression that some of them think aligned people should also try to win the AGI race, so doing capabilities research and being willing to listen to alignment concerns is good. (I disagree with this position and I don’t think it’s a strawman, but it might be a bit unfair.)
Many of the people that got interested in AGI safety later on also find AGI cool, or have done some capabilities research (e.g. me), so thinking that what we’ve done is evil is counterintuitive.
Yeah most people in the AGI safety concern group are also technological progress enthusiasts (with good reason, technological progress is generally awesome). And the recent string of “alignement startup” fondations points toward an unhealthy mix of capacity and safety research.
Also, the people pulling the alarm in the early days of AGI safety concerns, are also people interested in AGI. They find it cool. I get the impression that some of them think aligned people should also try to win the AGI race, so doing capabilities research and being willing to listen to alignment concerns is good. (I disagree with this position and I don’t think it’s a strawman, but it might be a bit unfair.)
Many of the people that got interested in AGI safety later on also find AGI cool, or have done some capabilities research (e.g. me), so thinking that what we’ve done is evil is counterintuitive.
Yeah most people in the AGI safety concern group are also technological progress enthusiasts (with good reason, technological progress is generally awesome). And the recent string of “alignement startup” fondations points toward an unhealthy mix of capacity and safety research.