I think a substantial fraction of ML researchers probably agree with Yann LeCun that AI safety will be solved “by default” in the course of making the AI systems useful. The crux is probably related to questions like how competent society’s response will be, and maybe the likelihood of deceptive alignment.
Two points of disagreement though:
I don’t think setting P(doom) = 10% indicates lack of engagement or imagination; Toby Ord in the Precipice also gives a 10% estimate for AI-derived x-risk this century, and I assume he’s engaged pretty deeply with the alignment literature.
I don’t think P(doom) = 10% or even 5% should be your threshold for “taking responsibility”. I’m not sure I like the responsibility frame in general, but even a 1% chance of existential risk is big enough to outweigh almost any other moral duty in my mind.
Hey! Thanks for sharing the debate with LeCun, I found it very interesting and I’ll do more research on his views.
Thanks for pointing out that even a 1% existential risk is worth worrying about, I imagine it’s true even in my moral system, if I just realize that ie 1% probability that humanity wipes = 70 million expected deaths (1% of 7 billions) plus all the expected humans that wouldn’t come to be.
That’s logically.
Emotionally, I find it WAY harder to care for a 1% X-risk. Scope insensitivity. I want to think about where else in my thinking this is causing output errors.
I think a substantial fraction of ML researchers probably agree with Yann LeCun that AI safety will be solved “by default” in the course of making the AI systems useful. The crux is probably related to questions like how competent society’s response will be, and maybe the likelihood of deceptive alignment.
Two points of disagreement though:
I don’t think setting P(doom) = 10% indicates lack of engagement or imagination; Toby Ord in the Precipice also gives a 10% estimate for AI-derived x-risk this century, and I assume he’s engaged pretty deeply with the alignment literature.
I don’t think P(doom) = 10% or even 5% should be your threshold for “taking responsibility”. I’m not sure I like the responsibility frame in general, but even a 1% chance of existential risk is big enough to outweigh almost any other moral duty in my mind.
Hey! Thanks for sharing the debate with LeCun, I found it very interesting and I’ll do more research on his views.
Thanks for pointing out that even a 1% existential risk is worth worrying about, I imagine it’s true even in my moral system, if I just realize that ie 1% probability that humanity wipes = 70 million expected deaths (1% of 7 billions) plus all the expected humans that wouldn’t come to be.
That’s logically.
Emotionally, I find it WAY harder to care for a 1% X-risk. Scope insensitivity. I want to think about where else in my thinking this is causing output errors.