I was discouraged from writing a blog post estimating when AI would be developed, on the basis that a real conversation about this topic among rationalists would cause AI to come sooner, which would be more dangerous
Does anyone actually believe and/or want to defend this? I have a strong intuition that public-facing discussion of AI timelines within the rationalist and AI alignment communities is highly unlikely to have a non-negligible effect on AI timelines, especially in comparison to the potential benefit it could have for the AI alignment community being better able to reason about something very relevant to the problem they are trying to solve. (Ditto for probably most but not all topics regarding AGI that people interested in AI alignment may be tempted to discuss publicly.)
I kind of believe this, but it’s not a huge effect. I do think that the discussion around short timelines had some effect on the scaling laws research, which I think had some effect on OpenAI going pretty hard on aggressively scaling models, which accelerated progress by a decent amount.
My guess is the benefits of public discussion are still worth more, but given our very close proximity to some of the world’s best AI labs, I do think the basic mechanism of action here is pretty plausible.
Your comment makes sense to me as a consideration for someone writing on LW in 2017. It doesn’t really make sense to me as a consideration for someone writing on LW in 2021. (The horse has left the barn.) Do you agree?
No, I think the same mechanism of action is still pretty plausible, even in 2021 (attracting more researchers and encouraging more effort to go into blindly-scaling-type research), so I think additional research here could have similar effects. As Gwern has written about extensively, for some reason the vast majority of AI companies are still not taking the scaling hypothesis seriously, so there is lots of room for more AI companies going in on it.
I also think there is a broader reference class of “having important ideas about how to build AGI” (of which the scaling hypothesis is one), that due to our proximity to top AI labs, does seem like it could have a decently sized effect.
As in my comment, I think saying “Timelines are short because the path to AGI is (blah blah)” is potentially problematic in a way that saying “Timelines are short” is not problematic. In particular, it’s especially problematic (1) If “(blah blah)” an obscure line of research, or (2) if “(blah blah)” is a well-known but not widely-accepted line of research (e.g. scaling hypothesis) AND the post includes new concrete evidence or new good arguments in favor of it.
If neither of those is applicable, then I want to say there’s really no problem. Like, if some AI Company Leader is not betting on the scaling hypothesis, not after GPT-2, not after GPT-3, not after everything that Gwern and OpenAI etc. have said about the topic … well, I have a hard time imagining that yet another LW post endorsing the scaling hypothesis would be what tips the balance for them.
I have updated over the years on how many important people in AI read and follow LessWrong and the associated meme-space. I agree marginal discussion does not make a big difference. I also think overall all discussion still probably didn’t make enough of a difference to make it net-negative, but it was substantial enough to cause me to think for quite a while on whether it was worth it overall.
I agree with you that the future costs seem marginally lower, but not low enough to make me not think hard and want to encourage others to think hard about the tradeoff. My estimate of the tradeoff came out on the net-positive side, but I wouldn’t think it would be crazy for someone’s tradeoff to come out on the net-negative side.
Does anyone actually believe and/or want to defend this?
I believe this. For example, one of my benign beliefs in ~2014 was “songs in frequency space are basically just images; you can probably do interesting things in the music space by just taking off-the-shelf image stuff (like style transfer) and doing it on songs.”
The first paper doing something similar that I know of came out in 2018. If I had posted about it in 2014, would it have happened sooner? Maybe—I think there’s a sort of weird thing going on in the music space where all the people with giant libraries of music want to maintain their relationships with the producers of music, and so there’s not much value for them in doing research like this, and so there might be unusually little searching for fruit in that corner of the orchard. But also maybe my idea was bad, or wouldn’t really help all of that much, or no one would have done it just because they read it. (I don’t think that paper worked in wavelet space, but didn’t look too closely.)
I’m much less certain that the net effect is “you shouldn’t talk about such things.” The more important the consequences of sharing a belief seem to you (“oh, if you just put together X and Y you can build unsafe AGI”), the more important for your models that you’re right (“oh, if that doesn’t work I think we have five more years”).
It’s possible for someone to believe “Timelines are short because the path to AGI is (blah blah)”, in which case they might hesitate to publicly justify their timelines, and this might indirectly bleed into a hesitation to bring it up in the first place. I agree that merely stating a belief about timelines publicly on LW, per se, seems pretty harmless right now, unless there’s something I’m not thinking of.
(Update: if you’re a famous AI person or politician publishing a high-profile op-ed that it’s feasible for a focused project to make AGI today, that would be a bit different, that would require some thought about whether you’re contributing to a worldwide competitive sprint to AGI. But a LW post today wouldn’t move the needle on that, I think.)
Timelines are short because the path to AGI is (blah blah)
This requires a high degree of precision about your knowledge of the path to AGI, which makes it seem not that plausible, unless timelines are very short no matter what you say because others will stumble their way through the path you’ve identified soon anyway.
Does anyone actually believe and/or want to defend this? I have a strong intuition that public-facing discussion of AI timelines within the rationalist and AI alignment communities is highly unlikely to have a non-negligible effect on AI timelines, especially in comparison to the potential benefit it could have for the AI alignment community being better able to reason about something very relevant to the problem they are trying to solve. (Ditto for probably most but not all topics regarding AGI that people interested in AI alignment may be tempted to discuss publicly.)
I kind of believe this, but it’s not a huge effect. I do think that the discussion around short timelines had some effect on the scaling laws research, which I think had some effect on OpenAI going pretty hard on aggressively scaling models, which accelerated progress by a decent amount.
My guess is the benefits of public discussion are still worth more, but given our very close proximity to some of the world’s best AI labs, I do think the basic mechanism of action here is pretty plausible.
Your comment makes sense to me as a consideration for someone writing on LW in 2017. It doesn’t really make sense to me as a consideration for someone writing on LW in 2021. (The horse has left the barn.) Do you agree?
No, I think the same mechanism of action is still pretty plausible, even in 2021 (attracting more researchers and encouraging more effort to go into blindly-scaling-type research), so I think additional research here could have similar effects. As Gwern has written about extensively, for some reason the vast majority of AI companies are still not taking the scaling hypothesis seriously, so there is lots of room for more AI companies going in on it.
I also think there is a broader reference class of “having important ideas about how to build AGI” (of which the scaling hypothesis is one), that due to our proximity to top AI labs, does seem like it could have a decently sized effect.
As in my comment, I think saying “Timelines are short because the path to AGI is (blah blah)” is potentially problematic in a way that saying “Timelines are short” is not problematic. In particular, it’s especially problematic (1) If “(blah blah)” an obscure line of research, or (2) if “(blah blah)” is a well-known but not widely-accepted line of research (e.g. scaling hypothesis) AND the post includes new concrete evidence or new good arguments in favor of it.
If neither of those is applicable, then I want to say there’s really no problem. Like, if some AI Company Leader is not betting on the scaling hypothesis, not after GPT-2, not after GPT-3, not after everything that Gwern and OpenAI etc. have said about the topic … well, I have a hard time imagining that yet another LW post endorsing the scaling hypothesis would be what tips the balance for them.
I have updated over the years on how many important people in AI read and follow LessWrong and the associated meme-space. I agree marginal discussion does not make a big difference. I also think overall all discussion still probably didn’t make enough of a difference to make it net-negative, but it was substantial enough to cause me to think for quite a while on whether it was worth it overall.
I agree with you that the future costs seem marginally lower, but not low enough to make me not think hard and want to encourage others to think hard about the tradeoff. My estimate of the tradeoff came out on the net-positive side, but I wouldn’t think it would be crazy for someone’s tradeoff to come out on the net-negative side.
There could be more than one horse.
I believe this. For example, one of my benign beliefs in ~2014 was “songs in frequency space are basically just images; you can probably do interesting things in the music space by just taking off-the-shelf image stuff (like style transfer) and doing it on songs.”
The first paper doing something similar that I know of came out in 2018. If I had posted about it in 2014, would it have happened sooner? Maybe—I think there’s a sort of weird thing going on in the music space where all the people with giant libraries of music want to maintain their relationships with the producers of music, and so there’s not much value for them in doing research like this, and so there might be unusually little searching for fruit in that corner of the orchard. But also maybe my idea was bad, or wouldn’t really help all of that much, or no one would have done it just because they read it. (I don’t think that paper worked in wavelet space, but didn’t look too closely.)
I’m much less certain that the net effect is “you shouldn’t talk about such things.” The more important the consequences of sharing a belief seem to you (“oh, if you just put together X and Y you can build unsafe AGI”), the more important for your models that you’re right (“oh, if that doesn’t work I think we have five more years”).
It’s possible for someone to believe “Timelines are short because the path to AGI is (blah blah)”, in which case they might hesitate to publicly justify their timelines, and this might indirectly bleed into a hesitation to bring it up in the first place. I agree that merely stating a belief about timelines publicly on LW, per se, seems pretty harmless right now, unless there’s something I’m not thinking of.
(Update: if you’re a famous AI person or politician publishing a high-profile op-ed that it’s feasible for a focused project to make AGI today, that would be a bit different, that would require some thought about whether you’re contributing to a worldwide competitive sprint to AGI. But a LW post today wouldn’t move the needle on that, I think.)
This requires a high degree of precision about your knowledge of the path to AGI, which makes it seem not that plausible, unless timelines are very short no matter what you say because others will stumble their way through the path you’ve identified soon anyway.