We therefore conclude that we should assign very low credence to the spontaneous emergence of scheming in future AI systems— perhaps 0.1% or less.
I personally think there is a moderate gap (perhaps factor of 3) between “world is ended by serious[1] spontaneous scheming” and “serious spontaneous scheming”. And, I could imagine updating to a factor of 10 if the world seemed better prepared etc. So, it might be good to clarify this in the post. (Or clarify your comment.)
(I think perhaps spontaneous scheming (prior to human obsolence) is ~25% likely and x-risk conditional on being in one of those worlds which is due to this scheming is about 30% likely for an overall 8% on “world is ended by serious spontaneous scheming” (prior to human obsolence).)
The exact language you use in the post is:
I personally think there is a moderate gap (perhaps factor of 3) between “world is ended by serious[1] spontaneous scheming” and “serious spontaneous scheming”. And, I could imagine updating to a factor of 10 if the world seemed better prepared etc. So, it might be good to clarify this in the post. (Or clarify your comment.)
(I think perhaps spontaneous scheming (prior to human obsolence) is ~25% likely and x-risk conditional on being in one of those worlds which is due to this scheming is about 30% likely for an overall 8% on “world is ended by serious spontaneous scheming” (prior to human obsolence).)
serious = somewhat persistant, thoughtful, etc