I’m fairly scared that if “the EA community” attempted to pivot to running the fire-alarm, that nothing real would happen, we’d spend our chips, and we’d end up in some complicated plot that had no real chance of working whilst maybe giving up our ability to think carefully any more. Like, there’s no plan stated in the post. If someone has a plan that has a chance of doing any particular thing that’d be more interesting.
I spend various amounts of time in close proximity to a bunch of parts of “EA leadership”, and if you convince me that a strategy will work I could advocate for it.
(Also happy to receive DMs if you want to keep specifics private.)
This was pointed out by someone working in ML—but I can’t find major conference deadlines that this would have been in response to, so maybe it’s not a useful explanation.
But NeurIPS submissions open this week—these may have been intended for that, or precede that to claim priority in case someone else submits something similar. I’d be interested in looking at release dates / times for previous big advances compared to conferences.
My understanding is that when the post was written, Anthropic had already had the first Claude, so the knowledge was available to the community.
A month after this post was retracted, ChatGPT was released.
Plausibly, “the EA community” would’ve been in a better place if it started to publicly and privately use its chips for AI x-risk advocacy and talking about the short timelines.
Looking back at the parameters of the bet, it’s interesting to me that the benchmark and math components have all fallen, but that the two “real world” components of the bet are still standing.
No disagreements here; I just want to note that if “the EA community” waits too long for such a pivot, at some point AI labs will probably be faced with people from the general population protesting because even now a substantial share of the US population views the AI progress in a very negative light. Even if these protests don’t accomplish anything directly, they might indirectly affect any future efforts. For example, an EA-run fire alarm might be compromised a bit because the memetic ground would already be captured. In this case, the concept of “AI risk” would, in the minds of AI researchers, shift from “obscure overconfident hypotheticals of a nerdy philosophy” to “people with different demographics, fewer years of education, and a different political party than us being totally unreasonable over something that we understand far better”.
I’m not sure I would agree. The post you linked to is titled “A majority of the public supports AI development.” Only 10% of the population is strongly opposed to. You’re making an implicit assumption that the public is going to turn against the technology in the next couple of years but I see no reason to believe that.
In the past, public opinion really only turns against technology dolloping a big disaster. But we may not see a big AI induced disaster before a change in public opinion will be irrelevant to AGI
I think that we might want to encourage people to contribute what they can towards safety even if they only think they can make small contributions. I think that most people could find something useful to do if they really thought about it and were willing to put any ego aside.
Look at the sidebar here? Is this anywhere near optimal? I don’t think so. Surely it should be encouraging people to undertake logical first steps towards becoming involved in alignment (ie. AGI safety fundamentals course, 80,000 hours coaching or booking a call with AI Safety Support).
In a few weeks, I’ll probably be spending a few hours setting up a website for AI Safety Australia and NZ (a prospective org to do local movement-building). Lots of people have web development capabilities, but you don’t even need that with things like Wordpress.
I’ve been spending time going through recent threads and encouraging people who’ve expressed interest in doing something about this, but are unsure what to do, to consider a few logical next steps.
Or maybe just reading about safety and answering questions on the Stampy Wiki (https://stampy.ai)?
Or failing everything else, just do some local EA movement building and make sure to run a few safety events.
I don’t know, it just seems like there’s low-hanging fruit all over the place. Not claiming these are huge impacts, but beats doing nothing.
It seems like you’re pointing at a model where society can make progress on safety by having a bunch of people put some marginal effort towards it. That seems insane to me—have I misunderstood you?
Sorry, I don’t quite understand your objection? Is it that you don’t think these are net-positive, that you think all of these little bits will merely add up to a rounding error or that you think timelines are too short for them to make a difference?
I think the impact of little bits of “people engage with the problem” is not significantly positive. Maybe it rounds to zero. Maybe it is negative, if people engaging lightly flood serious people with noisy requests.
Hard research problems just don’t get solved by people thinking for five minutes. There are some people who can make real contributions [0] by thinking for ~five hours per week for a couple of months, but they are quite rare.
(This is orthogonal to the current discussion, but: I had not heard of stampy.ai before your comment. Probably you should refer to it as stampy.ai, because googling “stampy wiki” give sit as the ~fifth result, behind some other stuff that is kind of absurd.)
[0] say, write a blog post that gets read and incorporated into serious people’s world models
I’m not suggesting that they contribute towards research, just that if they were able to reliably get things done they’d be able to find someone who’d benefit from a volunteer. But I’m guessing you think they’d waste people’s time by sending them a bunch of emails asking if they need help? Or that a lot of people who volunteer then cause issues by being unreliable?
The question isn’t so much whether a contribution toward safety is small or big but whether you can actually find a contribution that’s certain to have a small contribution toward safety. If you think there are a bunch of small things toward safety that can be done, what do you have in mind?
I’ve found this week’s progress pretty upsetting.
I’m fairly scared that if “the EA community” attempted to pivot to running the fire-alarm, that nothing real would happen, we’d spend our chips, and we’d end up in some complicated plot that had no real chance of working whilst maybe giving up our ability to think carefully any more. Like, there’s no plan stated in the post. If someone has a plan that has a chance of doing any particular thing that’d be more interesting.
I spend various amounts of time in close proximity to a bunch of parts of “EA leadership”, and if you convince me that a strategy will work I could advocate for it.
(Also happy to receive DMs if you want to keep specifics private.)
I find it slightly meta-upsetting that we are already measuring progress in weeks.
Announcements of progress tend to clump together before the major AI conferences.
How much do you think that was a factor in the recent releases happening in short succession? Is there a conference happening soon that this was for?
This was pointed out by someone working in ML—but I can’t find major conference deadlines that this would have been in response to, so maybe it’s not a useful explanation.
But NeurIPS submissions open this week—these may have been intended for that, or precede that to claim priority in case someone else submits something similar. I’d be interested in looking at release dates / times for previous big advances compared to conferences.
Three years later, I think the post was right, and the pushback was wrong.
People who disagreed with this post lost their bets.
My understanding is that when the post was written, Anthropic had already had the first Claude, so the knowledge was available to the community.
A month after this post was retracted, ChatGPT was released.
Plausibly, “the EA community” would’ve been in a better place if it started to publicly and privately use its chips for AI x-risk advocacy and talking about the short timelines.
Looking back at the parameters of the bet, it’s interesting to me that the benchmark and math components have all fallen, but that the two “real world” components of the bet are still standing.
I agree that the update was correct. But you didn’t state a concrete action to take?
No disagreements here; I just want to note that if “the EA community” waits too long for such a pivot, at some point AI labs will probably be faced with people from the general population protesting because even now a substantial share of the US population views the AI progress in a very negative light. Even if these protests don’t accomplish anything directly, they might indirectly affect any future efforts. For example, an EA-run fire alarm might be compromised a bit because the memetic ground would already be captured. In this case, the concept of “AI risk” would, in the minds of AI researchers, shift from “obscure overconfident hypotheticals of a nerdy philosophy” to “people with different demographics, fewer years of education, and a different political party than us being totally unreasonable over something that we understand far better”.
I’m not sure I would agree. The post you linked to is titled “A majority of the public supports AI development.” Only 10% of the population is strongly opposed to. You’re making an implicit assumption that the public is going to turn against the technology in the next couple of years but I see no reason to believe that.
In the past, public opinion really only turns against technology dolloping a big disaster. But we may not see a big AI induced disaster before a change in public opinion will be irrelevant to AGI
And that’s really more like 6% after you take in account the lizardman constant.
Reflecting on this and other comments, I decided to edit the original post to retract the call for a “fire alarm”.
Positive reinforcement for being able to do a retraction! Even when it’s the right thing to do it can be a hard thing to do.
DMed.
I think that we might want to encourage people to contribute what they can towards safety even if they only think they can make small contributions. I think that most people could find something useful to do if they really thought about it and were willing to put any ego aside.
Sounds like a nice thing to think, but I don’t put much stock in it.
Look at the sidebar here? Is this anywhere near optimal? I don’t think so. Surely it should be encouraging people to undertake logical first steps towards becoming involved in alignment (ie. AGI safety fundamentals course, 80,000 hours coaching or booking a call with AI Safety Support).
In a few weeks, I’ll probably be spending a few hours setting up a website for AI Safety Australia and NZ (a prospective org to do local movement-building). Lots of people have web development capabilities, but you don’t even need that with things like Wordpress.
I’ve been spending time going through recent threads and encouraging people who’ve expressed interest in doing something about this, but are unsure what to do, to consider a few logical next steps.
Or maybe just reading about safety and answering questions on the Stampy Wiki (https://stampy.ai)?
Or failing everything else, just do some local EA movement building and make sure to run a few safety events.
I don’t know, it just seems like there’s low-hanging fruit all over the place. Not claiming these are huge impacts, but beats doing nothing.
I think it is good to do things if you have traction. I think it is good to grow the things you can do.
It seems like you’re pointing at a model where society can make progress on safety by having a bunch of people put some marginal effort towards it. That seems insane to me—have I misunderstood you?
Sorry, I don’t quite understand your objection? Is it that you don’t think these are net-positive, that you think all of these little bits will merely add up to a rounding error or that you think timelines are too short for them to make a difference?
I think the impact of little bits of “people engage with the problem” is not significantly positive. Maybe it rounds to zero. Maybe it is negative, if people engaging lightly flood serious people with noisy requests.
Hard research problems just don’t get solved by people thinking for five minutes. There are some people who can make real contributions [0] by thinking for ~five hours per week for a couple of months, but they are quite rare.
(This is orthogonal to the current discussion, but: I had not heard of stampy.ai before your comment. Probably you should refer to it as stampy.ai, because googling “stampy wiki” give sit as the ~fifth result, behind some other stuff that is kind of absurd.)
[0] say, write a blog post that gets read and incorporated into serious people’s world models
I’m not suggesting that they contribute towards research, just that if they were able to reliably get things done they’d be able to find someone who’d benefit from a volunteer. But I’m guessing you think they’d waste people’s time by sending them a bunch of emails asking if they need help? Or that a lot of people who volunteer then cause issues by being unreliable?
The question isn’t so much whether a contribution toward safety is small or big but whether you can actually find a contribution that’s certain to have a small contribution toward safety. If you think there are a bunch of small things toward safety that can be done, what do you have in mind?
See this comment.