That’s not how I wrote the essay at all—and I don’t mean to imply that the situation is good (I find 95% chance of human extinction in my lifetime credible! This is awful!). Hope is a attitude to the facts, not a claim about the facts; though “high confidence in [near-certain] doom is unjustified” sure is. But in the spirit of your fair question, here are some infelicitous points:
I find P(doom)≈0.1 is less credible than P(doom)≈0.9 (in the essay above). Conditional on no attempts at alignment, my inside view might be about P(doom)≈0.8 - haven’t thought hard about that though and it’s a counterfactual anyway.
I think it’s pretty unlikely (credible interval 0--40%) that we’ll have good enough interpretability tools [to avoid an otherwise existential catastrophe] by the time we really really need them. (via this comment)
[short timelines, civilizational inadequacy, etc. I’ve given up on precision for this point but the vibe is not good.]
Writing up my views on x-risk from AI in detail and with low public miscommunication risk would be a large and challenging project, and honestly I expect that I’ll choose to focus on work, open source, and finishing my PhD thesis instead.
That’s not how I wrote the essay at all—and I don’t mean to imply that the situation is good (I find 95% chance of human extinction in my lifetime credible! This is awful!). Hope is a attitude to the facts, not a claim about the facts; though “high confidence in [near-certain] doom is unjustified” sure is. But in the spirit of your fair question, here are some infelicitous points:
I find P(doom)≈0.1 is less credible than P(doom)≈0.9 (in the essay above). Conditional on no attempts at alignment, my inside view might be about P(doom)≈0.8 - haven’t thought hard about that though and it’s a counterfactual anyway.
I think it’s pretty unlikely (credible interval 0--40%) that we’ll have good enough interpretability tools [to avoid an otherwise existential catastrophe] by the time we really really need them. (via this comment)
[short timelines, civilizational inadequacy, etc. I’ve given up on precision for this point but the vibe is not good.]
Writing up my views on x-risk from AI in detail and with low public miscommunication risk would be a large and challenging project, and honestly I expect that I’ll choose to focus on work, open source, and finishing my PhD thesis instead.