To be a bit blunt, I don’t take it for granted that an arbitrarily smart AI would be able to manipulate a human into developing a supervirus or nanomachines in a risk-free fashion.
How did you reach that conclusion? What does that ontology look like?
The fast takeoff doom scenarios seem like they should be subject to Drake equation-style analyses to determine P(doom). Even if we develop malevolent AIs, I’d say that P(doom | AGI tries to harm humans) is significantly less than 100%… obviously if humans detect this it would not necessarily prevent future incidents but I’d expect enough of a response that I don’t see how people could put P(doom) at 95% or more.
What is your p(doom)? Is that acceptable? If yes, why is it acceptable? If no, what is the acceptable p(doom)?
How did you reach that conclusion? What does that ontology look like?
What is your p(doom)? Is that acceptable? If yes, why is it acceptable? If no, what is the acceptable p(doom)?