Absolutely. But also, I think it strains credulity to believe that rationalists or EAs have a monopoly on the first concept—the sharp AI people already know, and everyone else will know in a few years due to people who don’t care about infohazards.
I have a hard time seeing the reason not to at least raise the salience of alignment. (That said, being unopinionated about timelines publicly while letting your knowledge guide your actions is undoubtedly the better choice.)
Respectfully, I disagree with the assertion that “the sharp AI people already know”
I feel that the idea of superhuman AI is a mental block, a philosophical blind spot for many people. It’s not about infohazards—the information is already out there. Nick Bostrom wrote “Superintelligence: Paths, Dangers, Strategies” in 2014. And EY has been talking about it for longer than that. But not everyone is capable of seeing that conclusion as true.
Possibly many of the sharpest AI people do already know, or are at least aware of the possibiity (“but the AI I’m working on wouldn’t do that, because...” [thought sequence terminated]). But I don’t think that information has become general knowledge, at all. It is still very much up in the air.
Ok, is raising the salience of alignment the strategy you are proposing? I guess it seems sensible?
I interpreted the “Do not make it easier for more people to build such systems. Do not build them yourself.” as “Let’s trying to convince people that AI is dangerous and they should stop building AI”. which is seems to me as either unlikely to work, or something that might backfire (“let’s build superhuman AI first before our enemies do”)
(I’m not 100% certain of it. Maybe mass advocacy for AI risks would work. But I have a… bad feeling about it).
My current, unfinished idea for a solution would be:
reach to out existing AI projects. Align them towards AI safety, as much as possible anyway.
solve the AI alignment problem
launch FAI before someone launches UFAI
(and yes, time crunch and arms race is probably not good for the kind of wisdom, restraint and safety culture required to build FAI)
(but if the alternative is “we all die”, this is the best take I have for now, even if this strategy in its current form may only have < 0.000000000000001% chance of succeeding)
Oh yeah 100%, “don’t unilaterally defect even though you might get lucky if you don’t defect” is not a good persuasion strategy.
“If you unilaterally defect you will definitely die, don’t do that to yourself, all the experts endorse” is a much more coherent argument. Idiots might not follow this advice, but if it can take $100M and a team of programmers, governments might be able to effectively prevent such a dumb accumulation of resources. Might. We have to try.
Absolutely. But also, I think it strains credulity to believe that rationalists or EAs have a monopoly on the first concept—the sharp AI people already know, and everyone else will know in a few years due to people who don’t care about infohazards.
I have a hard time seeing the reason not to at least raise the salience of alignment. (That said, being unopinionated about timelines publicly while letting your knowledge guide your actions is undoubtedly the better choice.)
epistemic status: uncertain.
Respectfully, I disagree with the assertion that “the sharp AI people already know”
I feel that the idea of superhuman AI is a mental block, a philosophical blind spot for many people. It’s not about infohazards—the information is already out there. Nick Bostrom wrote “Superintelligence: Paths, Dangers, Strategies” in 2014. And EY has been talking about it for longer than that. But not everyone is capable of seeing that conclusion as true.
Possibly many of the sharpest AI people do already know, or are at least aware of the possibiity (“but the AI I’m working on wouldn’t do that, because...” [thought sequence terminated]). But I don’t think that information has become general knowledge, at all. It is still very much up in the air.
Ok, is raising the salience of alignment the strategy you are proposing? I guess it seems sensible?
I interpreted the “Do not make it easier for more people to build such systems. Do not build them yourself.” as “Let’s trying to convince people that AI is dangerous and they should stop building AI”. which is seems to me as either unlikely to work, or something that might backfire (“let’s build superhuman AI first before our enemies do”)
(I’m not 100% certain of it. Maybe mass advocacy for AI risks would work. But I have a… bad feeling about it).
My current, unfinished idea for a solution would be:
reach to out existing AI projects. Align them towards AI safety, as much as possible anyway.
solve the AI alignment problem
launch FAI before someone launches UFAI
(and yes, time crunch and arms race is probably not good for the kind of wisdom, restraint and safety culture required to build FAI)
(but if the alternative is “we all die”, this is the best take I have for now, even if this strategy in its current form may only have < 0.000000000000001% chance of succeeding)
Oh yeah 100%, “don’t unilaterally defect even though you might get lucky if you don’t defect” is not a good persuasion strategy.
“If you unilaterally defect you will definitely die, don’t do that to yourself, all the experts endorse” is a much more coherent argument. Idiots might not follow this advice, but if it can take $100M and a team of programmers, governments might be able to effectively prevent such a dumb accumulation of resources. Might. We have to try.