I answered the following subquestion to help me answer the overall question: “How likely is it that the condition Rohin specified would already be met (if he went out and talked to the researchers today)?”
Considerations that make it more likely:
1. The considerations identified in ricaz’s and Owain’s comments and their subcomments
2. The bar for understanding safety concerns (question 2 on the “survey”) seems like it may be quite low. It seems to me that researchers entirely unfamiliar with safety could gain the required level of understanding in just 30 minutes of reading (depends on how Rohin would interpret his conversation with the researcher in deciding whether to mark “Yes” or “No”)
Considerations that make it less likely:
1. I’d guess that currently, most AI researchers have no idea what any of the concrete safety concerns are, i.e. they’d be “No”s on question 2
2. The bar for question 3 on the “survey” (“should we wait to build AGI”) might be pretty high. If someone thinks that some safety concerns remain but that we should cautiously move forward on building things that look more and more like AGI, does that count as a “Yes” or a “No”?
3. I have the general impression that many AI researchers really dislike the idea that safety concerns are serious enough that we should in any way slow down AI research
I thought about this subquestion before reading the comments or looking at Rohin’s distribution. Based on that thinking, I thought that there was a 25% chance that the condition Rohin specified would already be met.
Note: I work at Ought, so I’m ineligible for the prizes
It seems to me that researchers entirely unfamiliar with safety could gain the required level of understanding in just 30 minutes of reading
I might have said an hour, but that seem qualitatively right. But that requires them 1. having the motivation to do so and 2. finding and reading exactly the right sources in a field with a thousand blog posts and not much explicit prioritization around them. I think both of these are huge considerations against this condition already being met.
If someone thinks that some safety concerns remain but that we should cautiously move forward on building things that look more and more like AGI, does that count as a “Yes” or a “No”?
Hmm, good question. Probably a Yes? I might try to push on more clear hypotheticals (e.g. a team believes that such-and-such training run would produce AGI, should they do it?) to get a clearer answer.
I answered the following subquestion to help me answer the overall question: “How likely is it that the condition Rohin specified would already be met (if he went out and talked to the researchers today)?”
Considerations that make it more likely:
1. The considerations identified in ricaz’s and Owain’s comments and their subcomments
2. The bar for understanding safety concerns (question 2 on the “survey”) seems like it may be quite low. It seems to me that researchers entirely unfamiliar with safety could gain the required level of understanding in just 30 minutes of reading (depends on how Rohin would interpret his conversation with the researcher in deciding whether to mark “Yes” or “No”)
Considerations that make it less likely:
1. I’d guess that currently, most AI researchers have no idea what any of the concrete safety concerns are, i.e. they’d be “No”s on question 2
2. The bar for question 3 on the “survey” (“should we wait to build AGI”) might be pretty high. If someone thinks that some safety concerns remain but that we should cautiously move forward on building things that look more and more like AGI, does that count as a “Yes” or a “No”?
3. I have the general impression that many AI researchers really dislike the idea that safety concerns are serious enough that we should in any way slow down AI research
I thought about this subquestion before reading the comments or looking at Rohin’s distribution. Based on that thinking, I thought that there was a 25% chance that the condition Rohin specified would already be met.
Note: I work at Ought, so I’m ineligible for the prizes
I might have said an hour, but that seem qualitatively right. But that requires them 1. having the motivation to do so and 2. finding and reading exactly the right sources in a field with a thousand blog posts and not much explicit prioritization around them. I think both of these are huge considerations against this condition already being met.
Hmm, good question. Probably a Yes? I might try to push on more clear hypotheticals (e.g. a team believes that such-and-such training run would produce AGI, should they do it?) to get a clearer answer.