I think OpenAI has been a net-positive influence for reducing x-risk from AI, mainly by releasing products in a sufficiently helpful-yet-fallible form that society is now able to engage in less-abstract more-concrete public discourse to come to grips with AI and (soon) AI-risk.
I have a similar feeling: I think that ChatGPT has been, by far, the best thing to happen to AI x-risk discussion since the original Sequences. Suddenly a vast number of people have had their intuitions about AI shifted from “pure science fiction” to “actually a thing”, and the various failure modes that ChatGPT has are a concrete demonstration both about the general difficulty of aligning AI and some of the specific issues more specifically. And now we are seeing serious calls for AI regulation as well as much more wide-spread debate about things that weren’t previously seen much outside LW.
There used to be a plausible-sounding argument there’s no way of convincing the public about the risks of AGI in time to enact any serious societal response until we were already close enough for AGI that it wouldn’t be enough. Now it looks like that might be incorrect, at least assuming that timelines aren’t very short (but even if they are, I expect that we still have more time in this window than in a counterfactual world where someone developed the equivalent of ChatGPT later when we had more cheap compute lying around). So I’m generally very happy about OpenAI’s impact on x-risk.
the various failure modes that ChatGPT has are a concrete demonstration both about the general difficulty of aligning AI and some of the specific issues more specifically
By this logic, wouldn’t Microsoft be even more praiseworthy, because Bing Chat / Sidney was even more misaligned, and the way it was released (i.e. clearly prioritizing profit and bragging rights above safety) made AI x-risk even more obvious to people?
My assumption has been that Bing was so obviously rushed and botched that it’s probably less persuasive of the problems with aligning AI than ChatGPT is. To the common person, ChatGPT has the appearance of a serious product by a company trying to take safety seriously, but still frequently failing. I think that “someone trying really hard and doing badly” looks more concerning than “someone not really even trying and then failing”.
I haven’t actually talked to any laypeople to try to check this impression, though.
The majority of popular articles also seem to be talking specifically about ChatGPT rather than Bing, suggesting that ChatGPT has vastly more users. Regular use affects people’s intuitions much more than a few one-time headlines.
Though when I said “ChatGPT”, I was actually thinking about not just ChatGPT, but also the steps that led there—GPT-2 and GPT-3 as well. Microsoft didn’t contribute to those.
I agree that ChatGPT was positive for AI-risk awareness. However from my perspective being very happy about OpenAI’s impact on x-risk does not follow from this. Releasing powerful AI models does have a counterfactual effect on the awareness of risks, but also a lot of counterfactual hype and funding (such as the vast current VC investment in AI) which is mostly pointed at general capabilities rather than safety, which from my perspective is net negative.
Is there a scenario where you could get the public concern without the hype and funding? (The hype seems to be a big part of why people are getting concerned and saying we should stop the rush and get better regulation in place, in fact.)
It seems to me that the hype and funding is inevitable once you hit a certain point in AI research; we were going to get it sooner or later, and it’s better to have it sooner, when there’s still more time to rein it in.
I agree that some level public awareness would not have been reached without accessible demos of SOTA models.
However, I don’t agree with the argument that AI capabilities should be released to increase our ability to ‘rein it in’ (I assume you are making an argument against a capabilities ‘overhang’ which has been made on LW before). This is because text-davinci-002 (and then 3) were publicly available but not accessible to the average citizen. Safety researchers knew these models existed and were doing good work on them before ChatGPT’s release. Releasing ChatGPT results in shorter timelines and hence less time for safety researchers to do good work.
To caveat this: I agree ChatGPT does help alignment research, but it doesn’t seem like researchers are doing things THAT differently based on its existence. And secondly I am aware that OAI did not realise how large the hype and investment would be from ChatGPT, but nevertheless this hype and investment is downstream of a liberal publishing culture which is something that can be blamed.
Thanks for sharing this! Because of strong memetic selection pressures, I was worried I might be literally the only person posting on this platform with that opinion.
I have a similar feeling: I think that ChatGPT has been, by far, the best thing to happen to AI x-risk discussion since the original Sequences. Suddenly a vast number of people have had their intuitions about AI shifted from “pure science fiction” to “actually a thing”, and the various failure modes that ChatGPT has are a concrete demonstration both about the general difficulty of aligning AI and some of the specific issues more specifically. And now we are seeing serious calls for AI regulation as well as much more wide-spread debate about things that weren’t previously seen much outside LW.
There used to be a plausible-sounding argument there’s no way of convincing the public about the risks of AGI in time to enact any serious societal response until we were already close enough for AGI that it wouldn’t be enough. Now it looks like that might be incorrect, at least assuming that timelines aren’t very short (but even if they are, I expect that we still have more time in this window than in a counterfactual world where someone developed the equivalent of ChatGPT later when we had more cheap compute lying around). So I’m generally very happy about OpenAI’s impact on x-risk.
By this logic, wouldn’t Microsoft be even more praiseworthy, because Bing Chat / Sidney was even more misaligned, and the way it was released (i.e. clearly prioritizing profit and bragging rights above safety) made AI x-risk even more obvious to people?
My assumption has been that Bing was so obviously rushed and botched that it’s probably less persuasive of the problems with aligning AI than ChatGPT is. To the common person, ChatGPT has the appearance of a serious product by a company trying to take safety seriously, but still frequently failing. I think that “someone trying really hard and doing badly” looks more concerning than “someone not really even trying and then failing”.
I haven’t actually talked to any laypeople to try to check this impression, though.
The majority of popular articles also seem to be talking specifically about ChatGPT rather than Bing, suggesting that ChatGPT has vastly more users. Regular use affects people’s intuitions much more than a few one-time headlines.
Though when I said “ChatGPT”, I was actually thinking about not just ChatGPT, but also the steps that led there—GPT-2 and GPT-3 as well. Microsoft didn’t contribute to those.
I agree that ChatGPT was positive for AI-risk awareness. However from my perspective being very happy about OpenAI’s impact on x-risk does not follow from this. Releasing powerful AI models does have a counterfactual effect on the awareness of risks, but also a lot of counterfactual hype and funding (such as the vast current VC investment in AI) which is mostly pointed at general capabilities rather than safety, which from my perspective is net negative.
Is there a scenario where you could get the public concern without the hype and funding? (The hype seems to be a big part of why people are getting concerned and saying we should stop the rush and get better regulation in place, in fact.)
It seems to me that the hype and funding is inevitable once you hit a certain point in AI research; we were going to get it sooner or later, and it’s better to have it sooner, when there’s still more time to rein it in.
I agree that some level public awareness would not have been reached without accessible demos of SOTA models.
However, I don’t agree with the argument that AI capabilities should be released to increase our ability to ‘rein it in’ (I assume you are making an argument against a capabilities ‘overhang’ which has been made on LW before). This is because text-davinci-002 (and then 3) were publicly available but not accessible to the average citizen. Safety researchers knew these models existed and were doing good work on them before ChatGPT’s release. Releasing ChatGPT results in shorter timelines and hence less time for safety researchers to do good work.
To caveat this: I agree ChatGPT does help alignment research, but it doesn’t seem like researchers are doing things THAT differently based on its existence. And secondly I am aware that OAI did not realise how large the hype and investment would be from ChatGPT, but nevertheless this hype and investment is downstream of a liberal publishing culture which is something that can be blamed.
Thanks for sharing this! Because of strong memetic selection pressures, I was worried I might be literally the only person posting on this platform with that opinion.