Does X agree that there is at least one concern such that we have not yet solved it and we should not build superintelligent AGI until we do solve it?
What counts as building superintelligent AGI?
This could mean anything from working on foundational theory which could be used to facilitate building an AGI, to finishing the final phase of training on a fully functional AGI implementation.
In the former case you’re going to get close to 0% agreement. In the latter, well over 50% already (I hope!).
I don’t see any natural/clear in-the-spirit-of-the-question interpretation. E.g. if we set the bar as low as: “build superintelligent AGI” = “do non-safety work related to AGI”, then you’ll never get above 50% agreement, since anyone who fulfils (1) must deny (3) or admit they’re already being irresponsible.
I don’t think it’s a useful question without clarification of this.
As things stand, it’ll be pretty easy for a researcher to decide that they’re not building superintelligent AGI, almost regardless of what they’re doing. It’s then easy to concede that safety problems should be solved first, since that can always mean later.
On the other hand, requiring that a researcher agrees with Rohin on interpretation of “build superintelligent AGI” in order to say that they “agree with safety concerns” seems a high bar.
I think the following is underspecified:
What counts as building superintelligent AGI?
This could mean anything from working on foundational theory which could be used to facilitate building an AGI, to finishing the final phase of training on a fully functional AGI implementation.
In the former case you’re going to get close to 0% agreement. In the latter, well over 50% already (I hope!).
I don’t see any natural/clear in-the-spirit-of-the-question interpretation. E.g. if we set the bar as low as: “build superintelligent AGI” = “do non-safety work related to AGI”, then you’ll never get above 50% agreement, since anyone who fulfils (1) must deny (3) or admit they’re already being irresponsible.
I don’t think it’s a useful question without clarification of this.
As things stand, it’ll be pretty easy for a researcher to decide that they’re not building superintelligent AGI, almost regardless of what they’re doing. It’s then easy to concede that safety problems should be solved first, since that can always mean later.
On the other hand, requiring that a researcher agrees with Rohin on interpretation of “build superintelligent AGI” in order to say that they “agree with safety concerns” seems a high bar.
I was thinking more like