I agree with pretty much all of this, thanks for the clarification. I think the problem is that we need both a laser and a shotgun. We need a laser shotgun. It’s analogous to missile defense, where each individual reentry vehicle is very difficult to shoot down, and we have to shoot down every one or else we’ve failed. I appreciate now why it’s such a hard problem.
I definitely think there needs to be a better introductory material for alignment, particularly regarding what approaches have been investigated and what names they go by. I know the community already knows there is a problem with public communication so I’m not saying anything novel here.
I’d like to see a kind of “choose your own adventure” format webpage where lay people can select by multiple choice how they would go about addressing the problem, and the webpage will explain in not-too-technical language how much progress has been made and where to go for more information. I’m thinking about putting something like this together, I’ll use a wiki format since I don’t have all the answers of course.
Agreed on the introductory material (and various other things in that general direction).
I’m not clear on the case for public communication. We don’t want lay people to have strong and wildly inaccurate opinions, but it seems unachievable to broadly remedy the “wildly inaccurate” part. I haven’t seen a case made that broad public engagement can help (but I haven’t looked hard—has such a case been made?).
I don’t see what the lay public is supposed to do with the information—even supposing they had a broadly accurate high-level picture of the situation. My parents fit this description, and thus far it’s not clear to me what I’d want them to do. It’s easy to imagine negative outcomes (e.g. via politicization), and hard to imagine realistic positive ones (e.g. if 80% of people worldwide had an accurate picture, perhaps that would help, but it’s not going to happen).
It does seem important to communicate effectively to a somewhat narrower audience. The smartest 1% of people being well informed seems likely to be positive, but even here there are pitfalls in aiming to achieve this—e.g. if you get many people as far as [AGI will soon be hugely powerful] but they don’t understand the complexities of the [...and hugely dangerous] part, then you can inadvertently channel more resources into a race. (still net positive though, I’d assume EDIT: the being-well-informed part, that is)
I’ll grant that there’s an argument along the lines of: Perhaps we don’t know what we’d want the public to do, but there may come a time when we do know. If time is short it may then be too late to do the public education necessary. Therefore we should start now.
I don’t think I buy this. The potential downsides of politicization seem great, and it’s hard to think of many plausible ”...and then public pressure to do X saved the day!” scenarios.
This is very much not my area, so perhaps there are good arguments I haven’t seen. I’d just warn against reasoning of the form “Obviously we want to get the public engaged and well-informed, so...”. The “engaged” part is far from obvious (in particular we won’t be able to achieve [engaged if and only if well-informed]).
I’ll have to think about this more, but if you compare this to other things like climate change or gain of function research, it’s a very strange movement indeed that doesn’t want to spread awareness. Note that I agree that regulation is probably more likely to be harmful than helpful, and also note that I personally despise political activists, so I’m not someone normally amenable to “consciousness raising.” It’s just unusual to be against it for something you care about.
Absolutely—but it’s a strange situation in many respects.
It may be that spreading awareness is positive, but I don’t think standard arguments translate directly. There’s also irreversibility to consider: err on the side of not spreading info, and you can spread it later (so long as there’s time); you can’t easily unspread it.
More generally, I think for most movements we should ask ourselves, “How much worse than the status-quo can things plausibly get?”.
For gain-of-function research, we’d need to consider outcomes where the debate gets huge focus, but the sensible side loses (e.g. through the public seeing gof as the only way to prevent future pandemics). This seems unlikely, since I think there are good common-sense arguments against gof at most levels of detail.
For climate change, it’s less clear to me: there seem many plausible ways for things to have gotten worse. Essentially because the only clear conclusion is “something must be done”, but there’s quite a bit less clarity about what—or at least there should be less clarity. (e.g. to the extent that direct climate-positive actions have negative economic consequences, to what extent are there downstream negative-climate impacts; I have no idea, but I’m sure it’s a complex situation)
For AGI, I find it easy to imagine making-things-worse and hard to see plausible routes to making-things-better.
Even the expand-the-field upside needs to be approached with caution. This might be better thought of as something like [expand the field while maintaining/improving the average level of understanding]. Currently, most people who bump into AI safety/alignment will quickly find sources discussing the most important problems. If we expanded the field 100x overnight, then it becomes plausible that most new people don’t focus on the real problems. (e.g. it’s easy enough only to notice the outer-alignment side of things)
Unless time is very short, I’d expect doubling the field each year works out better than 5x each year—because all else would not be equal. (I have no good sense what the best expansion rate or mechanism is—just that it’s not [expand as fast as possible])
But perhaps I’m conflating [aware of the problem] with [actively working on the problem] a bit much. Might not be a bad idea to have large amounts of smart people aware of the problem overnight.
I agree with pretty much all of this, thanks for the clarification. I think the problem is that we need both a laser and a shotgun. We need a laser shotgun. It’s analogous to missile defense, where each individual reentry vehicle is very difficult to shoot down, and we have to shoot down every one or else we’ve failed. I appreciate now why it’s such a hard problem.
I definitely think there needs to be a better introductory material for alignment, particularly regarding what approaches have been investigated and what names they go by. I know the community already knows there is a problem with public communication so I’m not saying anything novel here.
I’d like to see a kind of “choose your own adventure” format webpage where lay people can select by multiple choice how they would go about addressing the problem, and the webpage will explain in not-too-technical language how much progress has been made and where to go for more information. I’m thinking about putting something like this together, I’ll use a wiki format since I don’t have all the answers of course.
Agreed on the introductory material (and various other things in that general direction).
I’m not clear on the case for public communication. We don’t want lay people to have strong and wildly inaccurate opinions, but it seems unachievable to broadly remedy the “wildly inaccurate” part. I haven’t seen a case made that broad public engagement can help (but I haven’t looked hard—has such a case been made?).
I don’t see what the lay public is supposed to do with the information—even supposing they had a broadly accurate high-level picture of the situation. My parents fit this description, and thus far it’s not clear to me what I’d want them to do. It’s easy to imagine negative outcomes (e.g. via politicization), and hard to imagine realistic positive ones (e.g. if 80% of people worldwide had an accurate picture, perhaps that would help, but it’s not going to happen).
It does seem important to communicate effectively to a somewhat narrower audience. The smartest 1% of people being well informed seems likely to be positive, but even here there are pitfalls in aiming to achieve this—e.g. if you get many people as far as [AGI will soon be hugely powerful] but they don’t understand the complexities of the [...and hugely dangerous] part, then you can inadvertently channel more resources into a race. (still net positive though, I’d assume EDIT: the being-well-informed part, that is)
I’ll grant that there’s an argument along the lines of:
Perhaps we don’t know what we’d want the public to do, but there may come a time when we do know. If time is short it may then be too late to do the public education necessary. Therefore we should start now.
I don’t think I buy this. The potential downsides of politicization seem great, and it’s hard to think of many plausible ”...and then public pressure to do X saved the day!” scenarios.
This is very much not my area, so perhaps there are good arguments I haven’t seen.
I’d just warn against reasoning of the form “Obviously we want to get the public engaged and well-informed, so...”. The “engaged” part is far from obvious (in particular we won’t be able to achieve [engaged if and only if well-informed]).
I’ll have to think about this more, but if you compare this to other things like climate change or gain of function research, it’s a very strange movement indeed that doesn’t want to spread awareness. Note that I agree that regulation is probably more likely to be harmful than helpful, and also note that I personally despise political activists, so I’m not someone normally amenable to “consciousness raising.” It’s just unusual to be against it for something you care about.
Absolutely—but it’s a strange situation in many respects.
It may be that spreading awareness is positive, but I don’t think standard arguments translate directly. There’s also irreversibility to consider: err on the side of not spreading info, and you can spread it later (so long as there’s time); you can’t easily unspread it.
More generally, I think for most movements we should ask ourselves, “How much worse than the status-quo can things plausibly get?”.
For gain-of-function research, we’d need to consider outcomes where the debate gets huge focus, but the sensible side loses (e.g. through the public seeing gof as the only way to prevent future pandemics). This seems unlikely, since I think there are good common-sense arguments against gof at most levels of detail.
For climate change, it’s less clear to me: there seem many plausible ways for things to have gotten worse. Essentially because the only clear conclusion is “something must be done”, but there’s quite a bit less clarity about what—or at least there should be less clarity. (e.g. to the extent that direct climate-positive actions have negative economic consequences, to what extent are there downstream negative-climate impacts; I have no idea, but I’m sure it’s a complex situation)
For AGI, I find it easy to imagine making-things-worse and hard to see plausible routes to making-things-better.
Even the expand-the-field upside needs to be approached with caution. This might be better thought of as something like [expand the field while maintaining/improving the average level of understanding]. Currently, most people who bump into AI safety/alignment will quickly find sources discussing the most important problems. If we expanded the field 100x overnight, then it becomes plausible that most new people don’t focus on the real problems. (e.g. it’s easy enough only to notice the outer-alignment side of things)
Unless time is very short, I’d expect doubling the field each year works out better than 5x each year—because all else would not be equal. (I have no good sense what the best expansion rate or mechanism is—just that it’s not [expand as fast as possible])
But perhaps I’m conflating [aware of the problem] with [actively working on the problem] a bit much. Might not be a bad idea to have large amounts of smart people aware of the problem overnight.