TOMORROW: the largest AI Safety protest ever!
Tomorrow, PauseAI and collaborators are putting on the largest AI Safety protest to date, across 7 locations in 6 countries. All are eagerly welcomed!
Your presence at this protest is a rare impact opportunity when in-person volunteering is not fungible with money or intellectual support—showing up is how we show the public our concern in a way that humans instinctively understand. Plus, we think it will be fun, with camaraderie and drinks afterward. It feels great to pluck new low-hanging fruit and flex different muscles in the fight for AI Safety.
October 21st (Saturday), in multiple countries
US, California, San Francisco (facebook)
US, Massachusetts, Boston (facebook)
Netherlands, Den Haag (Sign up)
Australia, Melbourne (Sign up)
Canada, Ottawa (Organized by Align the World, sign up on facebook or EventBrite)
Denmark, Copenhagen (Facebook)
Your country here? Discuss on discord!
Why we protest
AI is rapidly becoming more powerful, far faster than virtually any AI scientist has predicted. Billions are being poured into AI capabilities, and the results are staggering. New models are outperforming humans in a lot of domains. As capabilities increase, so do the risks. Scientists are even warning that AI might end up destroying humanity. This dire outcome not only seems possible, but also likely, as the average probability estimates for these outcomes range from 14% to 40%.
We need our leaders to listen to these warnings, but they are not taking this topic remotely as seriously as they should. There is AI safety legislation being drafted, but not a single measure would actually prevent or delay superintelligent AI. Over 70% wants to slow down AI, and over 60% want regulation to actively prevent superintelligent AI. Why is there no legislation draft that actually does this? The answer is lobbying: our politicians are mostly meeting AI company CEOs, and they will push policy measures that are in their interest.
On November 1st and 2nd, the very first AI Safety Summit will be held in the UK. The perfect opportunity to set the first steps towards sensible international AI safety regulation.
What we ask
Policy makers: Don’t allow companies to build a superintelligence. Regulations and hardware restrictions should apply before training has started as it is very difficult to control dissemination once a new capability has been achieved. . We cannot allow companies to train potentially world-ending AI models. Writing legislation is hard, and it takes time, but we may not have that long, so work as if your life depends on it. Because it does.
Companies: Many of you are scared of what AI can do, but you’re locked in a race. So be vocal about supporting a pause in principle. If you sign statements that this technology could kill us all, show the world that you’d prefer not to build it if it was a viable option.
Summit invitees: Prioritize safety over economic growth. We know AI can make our countries richer, but that’s not why you’re summoned here. Be the adult in the room
I’ll be there!
Can we get an update? How did this go?
I went. I had a decent time. About 30-40 people showed up.
It’s hard to judge, but random people passing by seemed to have a decently high rate of sympathy, with a decent number of people taking fliers (1/4 groups of people?) that were being handed out, as well as an occasional car driving by with people honking in support of the protest. The protest definitely felt reasonably within the Overton window of protests.
To be clear, the participants were still like 80% people I knew in-person from the Bay Area Rationality/EA/AI-Alignment community, and I don’t think the pause AI stuff has so far succeeded at sparking some kind of independent protest movement. I also don’t know whether the protest did anything. We were seen by a decent number of random people in SF (my guess is around 2000-3000 people walked by or drove by and noticed us during the two hours of the protest), but it’s not super clear what that does. My guess is most of the impact is from growing into something much larger, which does seem at least plausible.
Here’s a 2-min edited video of the protest.
Most people who hear our message do so well after the protest, via sharing of this kind of media.
The SF one went great! Here’s a first batch of pics. A lot of the impact will come from sharing the pics and videos.
It’s hard to say what the true impact of the events will be at this time, but they went well! I’m going to write a post-mortem for the SF PauseAI protest yesterday and the Meta protest in September and post it on EAF/LW that will cover the short-term outcomes.
Considering they are financially cheap to do (each around $2000 if you don’t count my salary), I’d call them pretty successful already. Meta protest got good media coverage, and it remains to be seen how this one will be covered since most of the coverage happened in the two following weeks last time.
I mean, to be clear, most of the cost is born by the protesters, so I don’t think this argument goes through. I would value the time of many people attending that protest at $100/hr+, which moves the more realistic cost more into the $10k+ range.
Is that really the right way to value such things? Like, is it actually true that if you had decided “eh, this protest is actually a bad idea and I won’t go”, then you would instead have spent the day working and earning $100/hr (or whatever amount)? In other words, did you actually in fact forego
\$(yourHourlySalary * protestDurationInHours)by attending? (And, by extension, the same question for all other protesters?)I think that somewhat straightforwardly applies to at least me. Don’t know about other people. I found it more exhausting than my average hour of work and probably traded off against more than one per hour of attendance.
This doesn’t seem right to me. We aren’t contractors paid by the hour, and the marginal hour of work on the weekend doesn’t translate linearly into likelihood of getting fired / a raise, so I don’t think it should be approximated as whatever one’s hourly average salary is.
Not sure what you mean. I am pretty sure most people attending worked less on other stuff (roughly by the amount of hours they attended) than they would have if they had not attended. You can quibble a bit over the multiplier, but it’s IMO obvious that you need to take into account the time cost of the attendees somehow.
For people who work during the weekends, those marginal hours of work are not as productive as the first chunk of 40 hours they do in a week, so my guess is that it should be modeled as like a factor of 2x less valuable as other work hours.
For people who don’t work during the week and this came out of social time, I think it was kind of a fun adventure, and personally I would have (in retrospect) paid quite a bit of money to go. It was a very important experience to me and I would be less without having attended. I feel stronger for having gone out to the world and said what I think is true in my heart about something I’m really worried about. My guess is that it was a positive sum event for many other people attending too, including just for people for whom it was a better bonding experience and memory than whatever else they would have done with their weekend.
That’s why I said “financially cheap”. They are expensive for the organizer in terms of convincing people to volunteer and to all attendees as far as their time and talents, and getting people to put in sweat equity is what makes it an effective demonstration. But per dollar invested they are very effective.
I would venture that the only person who was seriously prevented from doing something else by being involved in this protest was me. Of course there is some time and labor cost for everyone involved. I hope it was complementary to whatever else they do, and, as Ben said, perhaps even allowing them to flex different muscles in an enriching way.
Any reason to protest scaling instead of hardware or algorithmic development? (IDK if a comment on this post is the best place to say this, but I couldn’t think of a better place.)
I’d probably be in favor of slower scaling at current margins, but I don’t feel super confident in this. However, I’m strongly in favor of stopping hardware developement which seems more clearly bad. It’s probably harder to stop/regulate algorithmic development than either scaling or hardware.
I suppose frontier scaling is more obviously connected to danger which makes it a better political message?
Also, stopping frontier scaling is probably best for reducing short term risk (<2 year out) per unit of economic damage to the existing economy. Quite strong hardware pauses can also prevent short term risk and have other positive effects, but they also have stronger and more obvious economic damages. (In fact, I think very strong hardware pauses are probably more robust than scaling pauses for reducing short term risk.)
As far as what I mean by “very strong hardware pauses”:
Stop all H100 manufacturing (and ideally A100 manufacturing)
(Ideally) goverment buys back H100 and A100s and locks up this chips (or destroys them)
Improving hardware technology is banned, maybe nationalize nvidia and similar companies and mostly dismantle them.
This is obviously quite expensive, but slowing down hardware in other weaker ways still seems robustly good to me (ideally, slowing development globally instead of just in the US).
Personally, I’m interested in targeting hardware development and that will be among my future advocacy directions. I think it’ll be a great issue for corporate campaigns pushing voluntary agreements and for pushing for external regulations simultaneously. This protest is aimed more at governments (attending the UK Summit) and their overall plans for regulating AI, so we’re pushing compute governance as way to most immediately address the creation of frontier models. Imo hardware tracking at the very least is going to have to be part of enforcing such a limit if it is adopted, and slowing the development of more powerful hardware will be important to keeping an acceptable compute threshold high enough that we’re not constantly on the verge of someone illegally getting together enough chips to make something dangerous.
I strong downvoted, because I think public protest are not a good way of pushing for change.
They are a symmetric weapon.
They lock you in certain positions. There is a lot of momentum in a social movement that is carried through such public displays, which makes it difficult to change or reverse your position (for example, if we learned that for reason X it is much better to speed up the development of AI, which I don’t think is that improbable a priori).
They promote tribalistic, collective mindset. Protests like this are antithetic to the deep, 1-1 dialogue that LW stands for. I feel that the primary motivation for attending a protest is building camadarie and letting out emotions, which has more downsides than upsides, especially long-term. It also suports us-vs-them mentality.
Even if they change anyone’s mind, it is for wrong reasons. Public protests by necessity have to dumb down the message to a point that you can write on a poster. They lump people together present a unified front, and by doing that, lose nuance and diversity of opinions. If anyone changes their mind, it is because of reasons other than its merit.
They are an ineeffective way of using resources. The marginal value of spending time at a protest is negative for most of the people with any background in AI safety. It is much better to think, read papers, write papers, do experiments, chat with people around you, attend research seminars etc., than to picket on a street. Protests signal that you do not have anything more to offer than your presence.
There are some rare situations in which protests are a good choice, but mostly as the option of last resort. A possible counterpoint, that you are mostly advocating for awareness as opssosed to specific points is null, since pretty much everyone is aware of the problem now—both society as a whole, policymakers in particular, and people in AI research and alignment.
I think this specific point is false, especially outside of tech circles. My experience has been that while people are concerned about AI in general, and very open to X-risk when they hear about it, there is zero awareness of X-risk beyond popular fiction. It’s possible that my sample isn’t representative here, but I would expect that to swing in the other direction, given that the folks I interact with are often well-educated New-York-Times-reading types, who are going to be more informed than average.
Even among those aware, there’s also a difference between far-mode “awareness” in the sense of X-risk as some far away academic problem, and near-mode “awareness” in the sense of “oh shit, maybe this could actually impact me.” Hearing a bunch of academic arguments, but never seeing anybody actually getting fired up or protesting, will implicitly cause people to put X-risk in the first bucket. Because if they personally believed it to be big a near-term risk, they’d certainly be angry and protesting, and if other people aren’t, that’s a signal other people don’t really take it seriously. People sense a missing mood here and update on it.
I’ll be at the San Francisco protest!
We have Pause AI T-shirts, costumes, signs and other fun stuff. In addition to being a historic event, it’s a great day to make sane friends and we’ll grab some food/drinks after.
I wish I were in San Francisco so I could attend. Any way people can help make this go well outside one of those regions?
You could share the events with your friends and family who may be near, and signal boost media coverage of the events after! If you want to donate to keep me organizing events, I have a GoFundMe (and if anyone wants to give a larger amount, I’m happy to talk about how to do that :D). If you want to organize future events yourself, please DM me. Even putting the pause emoji ⏸️ in your twitter name helps :)
Here are the participating cities and links:
October 21st (Saturday), in multiple countries
US, California, San Francisco (facebook)
US, Massachusetts, Boston (facebook)
UK, Parliament Square, London (Sign up, facebook)
Netherlands, Den Haag (Sign up)
Australia, Melbourne (Sign up)
Canada, Ottawa (Organized by Align the World, sign up on facebook or EventBrite)
Denmark, Copenhagen (Facebook)
oh man I have such mixed feelings about this. at first strong downvoted, but then thought for 30 seconds, switched to strong upvote. I hope it goes well, and I worry it will not, but that doesn’t mean I really want to downvote it. I just.. worry that it will not go well. Hope it does. Unfortunately can’t make it myself.
In what sense do you think it will (might) not go well? My guess is that it will not go at all—some people will show up in the various locations, maybe some local news outlets will pick it up, and within a week it will be forgotten
whether its impact is net good for the world. my impression that protests don’t work may have come from anti-protest propaganda, I certainly don’t have a clear sense of how I got this sense; but at the same time, I do have this sense.
I think the impact will be pretty significant:
It’s one of those things where a lot of people—the majority according to some polls—already agree with it, so it’s building mutual knowledge and unlocking some tailwinds
It’s interesting and polarizing. People who think the movement is crazy are having fun with it on social media, which also keeps it top of mind.
I feel hesitant sharing this call to action in circles that would go to protests because they feel “stronger than gpt4” is a ridiculously lightweight ask and I’ll get heavily criticized for endorsing it with a share.
not to mention that the non-lesswrong link doesn’t explain why, and lesswrong links are generally frowned upon hard in most such contexts, as the site is considered to be classical liberal and unwilling to reject status quo. why not also ask for gpt4 to be deleted, I would expect them to say. it’s like, why bother going to a protest that is only asking for a pause? isn’t the whole point that we don’t want capital’s ais at all? why are we protesting for a pause rather than a stop? or so. and yet, I also expect any possible demand to not be honored. I do agree with all of those points, though I worry the backfire from #2 will be far stronger than the first-order impact. I’ll share it anyhow, but I feel like these protests are ignoring feedback like https://www.youtube.com/watch?v=AaU6tI2pb3M that would need to be considered and incorporated in at least an acknowledging way in order to get movement in the existing political circles that would push for action.edit: oh, those are facebook links I managed to not mentally parse as links. I’ll copy and paste the body, then. edit #2: I copy and pasted it into a “give me the raw markdown from a paste” site, so that I can paste it formatting-included into discord. maybe this will help others https://www.pastetomarkdown.com/
The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.
Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?