It seems like you think CICERO and Sydney are bigger updates than I do. Yes, there’s a continuum of cases of catching deception where it’s reasonable for the ML community to update on the plausibility of AI takeover. Yes, it’s important that the ML community updates before AI systems pose significant risk, and there’s a chance that they won’t do so. But I don’t see the lack of strong update towards p(doom) from CICERO as good evidence that the ML community won’t update if we get evidence of systematic scheming (including trying to break out of the lab when there was never any training signal incentivising that behaviour). I think that kind of evidence would be much more relevant to AI takeover risk than CICERO.
To clarify my position in case i’ve been misunderstood. I’m not saying the ML community will definitely update in time. I’m saying that if there is systematic scheming and we catch it red-handed (as I took Buck to be describing) then there will likely be a very significant update. And CICERO seems like a weak counter example (but not zero evidence)
It seems like you think CICERO and Sydney are bigger updates than I do. Yes, there’s a continuum of cases of catching deception where it’s reasonable for the ML community to update on the plausibility of AI takeover. Yes, it’s important that the ML community updates before AI systems pose significant risk, and there’s a chance that they won’t do so. But I don’t see the lack of strong update towards p(doom) from CICERO as good evidence that the ML community won’t update if we get evidence of systematic scheming (including trying to break out of the lab when there was never any training signal incentivising that behaviour). I think that kind of evidence would be much more relevant to AI takeover risk than CICERO.
To clarify my position in case i’ve been misunderstood. I’m not saying the ML community will definitely update in time. I’m saying that if there is systematic scheming and we catch it red-handed (as I took Buck to be describing) then there will likely be a very significant update. And CICERO seems like a weak counter example (but not zero evidence)