For what it’s worth, I interpreted Yitz’s words as having the subtext “and no one, at present, as sufficiently educated, because no good solution is known” and not the subtext “so it’s OK because all we have to do is educate people”.
(Also and unrelatedly: I don’t think it’s right to say “The recklessness is not the source of the problem”. It seems to me that the recklessness is a problem potentially sufficient to kill us all, and not knowing a solution to the alignment problem is a problem potentially sufficient to kill us all, and both of those problems are likely very hard to solve. Neither is the source of the problem; the problem has multiple sources all potentially sufficient to wipe us out.)
I fully agree with your last point, btw. If I remember correctly (could be misremembering though), EY has stated in the past that it doesn’t matter if you can convince everyone alignment is hard, but I don’t think that’s fully true. If you really can convince a sufficient number of people to take alignment seriously, and not be reckless, you can affect governance, and simply prevent (or at least delay) AGI from being built in the first place.
Delay it for a few years, sure. Maybe. If you magically convince our idiotic governments of a complex technical fact that doesn’t fit the prevailing political narratives.
But if there are some people who are convinced they have a magic alignment solution…
Someone is likely to run some sort of AI sooner or later. Unless some massive effort to restrict access to computers or something.
Well then, imagine a hypothetical in which the world succeeds at a massive effort to restrict access to compute. That would be a primarily social challenge, to convince the relatively few people at the top to take the risk seriously enough to do that, and then you’ve actually got a pretty permanent solution...
Is it primarily a social challenge? Humanity now relies relatively heavily on quick and easy communications, CAD[1], computer-aided data processing for e.g. mineral prospecting, etc, etc.
(One could argue that we got along without this in the early-to-mid 1900s, but at the same time we now have significantly more people. Ditto, it wasn’t exactly sustainable.)
For what it’s worth, I interpreted Yitz’s words as having the subtext “and no one, at present, as sufficiently educated, because no good solution is known” and not the subtext “so it’s OK because all we have to do is educate people”.
(Also and unrelatedly: I don’t think it’s right to say “The recklessness is not the source of the problem”. It seems to me that the recklessness is a problem potentially sufficient to kill us all, and not knowing a solution to the alignment problem is a problem potentially sufficient to kill us all, and both of those problems are likely very hard to solve. Neither is the source of the problem; the problem has multiple sources all potentially sufficient to wipe us out.)
Thanks for the charitable read :)
I fully agree with your last point, btw. If I remember correctly (could be misremembering though), EY has stated in the past that it doesn’t matter if you can convince everyone alignment is hard, but I don’t think that’s fully true. If you really can convince a sufficient number of people to take alignment seriously, and not be reckless, you can affect governance, and simply prevent (or at least delay) AGI from being built in the first place.
Delay it for a few years, sure. Maybe. If you magically convince our idiotic governments of a complex technical fact that doesn’t fit the prevailing political narratives.
But if there are some people who are convinced they have a magic alignment solution…
Someone is likely to run some sort of AI sooner or later. Unless some massive effort to restrict access to computers or something.
Well then, imagine a hypothetical in which the world succeeds at a massive effort to restrict access to compute. That would be a primarily social challenge, to convince the relatively few people at the top to take the risk seriously enough to do that, and then you’ve actually got a pretty permanent solution...
Is it primarily a social challenge? Humanity now relies relatively heavily on quick and easy communications, CAD[1], computer-aided data processing for e.g. mineral prospecting, etc, etc.
(One could argue that we got along without this in the early-to-mid 1900s, but at the same time we now have significantly more people. Ditto, it wasn’t exactly sustainable.)
Computer-aided design