FWIW my ass numbers have for a while been: 50% we die because nobody will know a workable technical solution to keeping powerful AI under control, by the time we need it, plus another (disjunctive) 40% that we die despite knowing such a solution, thanks to competition issues, other coordination issues, careless actors, bad actors, and so on (see e.g. here), equals 90% total chance of extinction or permanent disempowerment.
When you say “We just have to not be idiots about it” that’s an ambiguous phrase, because people say that phrase in regards to both very easy problems and very hard problems.
People say “We can move the delicate items without breaking them, we just have to not be idiots about it—y’know, make sure our shoes are tied, look where we’re going, etc.” That’s example is actually easy for a group of non-idiots. We would bet on success “by default” (without any unusual measures).
But people also say “Lowering rents in San Francisco is easy, we just have to not be idiots about it—y’know, build much much more housing where people want to live.” But that’s actually hard. There are strong memetic and structural forces arrayed against that. In other words, idiots do exist and are not going away anytime soon! :)
So anyway, I’m not sure what you were trying to convey in that comment.
FWIW my ass numbers have for a while been: 50% we die because nobody will know a workable technical solution to keeping powerful AI under control, by the time we need it, plus another (disjunctive) 40% that we die despite knowing such a solution, thanks to competition issues, other coordination issues, careless actors, bad actors, and so on (see e.g. here), equals 90% total chance of extinction or permanent disempowerment.
When you say “We just have to not be idiots about it” that’s an ambiguous phrase, because people say that phrase in regards to both very easy problems and very hard problems.
People say “We can move the delicate items without breaking them, we just have to not be idiots about it—y’know, make sure our shoes are tied, look where we’re going, etc.” That’s example is actually easy for a group of non-idiots. We would bet on success “by default” (without any unusual measures).
But people also say “Lowering rents in San Francisco is easy, we just have to not be idiots about it—y’know, build much much more housing where people want to live.” But that’s actually hard. There are strong memetic and structural forces arrayed against that. In other words, idiots do exist and are not going away anytime soon! :)
So anyway, I’m not sure what you were trying to convey in that comment.