peterslattery
Thanks—FYI, I see: Sorry, the map could not be loaded. Please check the setup or contact support.
Thank you for this. Much appreciated.
I think you should make a new version of this for your website. You are now becoming more of a public figure, and better communicating your forecasting record will help make you, and outputs like AI 2027, more credible.
It’s not my paper — I just wanted to share the link on LW as it seemed likely to be of interest to some people here. Thanks for the feedback, and sorry for the confusion!
In the future, there should be some organization or some group of individuals in the LW community who raise awareness about these sorts of opportunities and offer content and support to ensure submissions from the most knowledgeable and relevant actors. This seems like a very low-hanging fruit and is something several groups I know are doing.
As I understand it, there are several studies showing that it works for depression. Of course, it may be the case that this is only for people for whom anxiety caused the depression.
E.g.,
https://pubmed.ncbi.nlm.nih.gov/26718792/https://link.springer.com/article/10.1007/s00406-024-01783-2
I recommend anyone considering SSRIs to consider Silexan. See this write-up from Scott Alexander which covers the research. Someone I know has replaced their SSRI with Silexan. Several people I know have had good experiences. The main, and not always common, side effect is lavender flavored burps. Not everyone gets it. I don’t, and it wasn’t bad when I did. No-one I know has had a bad experience. Overall, the downside risk appears to be relatively low.
I also strongly endorse this based on my experience. I was a research consultant who created evidence summaries for decision makers in industry and government. This usually involved searching for published content. Anything that wasn’t indexed by Google Scholar/publication repositories was almost always excluded.
Yeah, I found this helpful. In general, I’d like to see more of these dialogues. I think that they do a good job of synthesing different arguments in an accessible way. I feel that’s increasingly important as more arguments emerge.
As an aside, I like the way that the content goes from relatively accessible high level discussion and analogy into more specific technical detail. I think this makes it much more accessible to novice and non-technical readers.
[Reposting from a Facebook thread discussing the article because my thoughts may be of interest]
I woke to see this shared by Timnit Gebru on my Linkedin and getting 100s of engagements. https://twitter.com/xriskology/status/1642155518570512384It draws a lot of attention to the airstrikes comment which is unfortunate.
Stressful to read
A quick comment on changes that I would probably make to the article:Make the message less about EY so it is harder to attack the messenger and undermine the message.
Reference other supporting authorities and sources of evidence, so this seems like a more evidenced backed view point. Particularly more conventional ones because EY has no conventional credentials (AFAIK)
Make it clear that more and more people (ideally like/admired by the target audience, perhaps policymakers/civil servants in this case) are starting to worry about AI/act accordingly (leverage social proof/dynamic norms)
Make the post flow a little better to increase fluency and ease of understanding (hard to be precise about what to do here but I didn’t think that it read as well as it could have)
Make the post more relatable by choosing examples that will be more familiar to relevant readers (e.g., not stockfish).
Don’t mention the airstrikes—keep the call for action urgent and strong but vague so that you aren’t vulnerable to people taking a quote out of context.
Finish with some sort of call to action or next steps for the people who were actually motivated.
Anonymous submission: I have pretty strong epistemics against the current approach of “we’ve tried nothing and we’re all out of ideas”. It’s totally tedious seeing reasonably ideas get put forward, some contrarian position gets presented, and the community reverts to “do nothing”. That recent idea of a co-signed letter about slowing down research is a good example of the intellectual paralysis that annoys me. In some ways it feels built on perhaps a good analytical foundation, but a poor understanding of how humans and psychology and policy change actually work.
Thanks for this.
Is anyone working on understanding LLM Dynamics or something adjacent? Is there early work that I should read? Are there any relevant people whose work I should follow?
Hey Hoagy, thanks for replying, I really appreciate it!
I fixed that link, thanks for pointing it out.
Here is a quick response to some of your points:
My feeling with the posts is that given the diversity of situations for people who are currently AI safety researchers, there’s not likely to be a particular key set of understandings such that a person could walk into the community as a whole and know where they can be helpful.
I tend to feel that things could be much better with little effort. As an analogy, consider the difference between trying to pick a AI safety project to work on now, versus before we had curation and evaluation posts like this.
I’ll note that those posts seem very useful but they are now almost a year out of date and were only ever based on a small set of opinions. It wouldn’t be hard to have something much better.
Similarly, I think that there is room for a lot more of this “coordination work’ here and lots of low-hanging fruit in general.
It’s going to be more like here are the groups and organizations which are doing good work, what roles or other things do they need now, and what would help them scale up their ability to produce useful work.
This is exactly what I want to know! From my perspective effective movement builders can increase contributors, contributions, and coordination within the AI Safety community, by starting, sustaining, and scaling useful projects.
Relatedly, I think that we should ideally have some sort of community consensus gathering process to figure out what is good and bad movement building (e.g., who are the good/bad groups, and what do the collective set of good groups need).
The shared language stuff and all of what I produced in my post is mainly a means to that end. I really just want to make sure that before I survey the community to understand who wants what and why, there is some sort of standardised understanding and language about movement building so that people don’t just write it off as a particular type of recruitment done without supervision by non-experts.
Anonymous submission:
I only skimmed your post so I very likely missed a lot of critical info. That said, since you seem very interested in feedback, here are some claims that are pushing back against the value of doing AI Safety field building at all. I hope this is somehow helpful.- Empirically, the net effects of spreading MIRI ideas seems to be squarely negative, both from the point of view of MIRI itself (increasing AI development, pointing people towards AGI), and from other points of views.
- The view of AI safety as expounded by MIRI, Nick Bostrom, etc is essentially an unsolvable problem. To put it in words that they would object it, they believe at some point humanity is going to invent a Godlike machine and this Godlike machine will then shape the future of the universe as it sees fit; perhaps according to some intensely myopic goal like maximizing paperclips. To prevent this from happening, we need to somehow make sure that AI does what we want it to do by formally specifying what we really want in math terms.
The reason MIRI have given up on making progress on this and don’t see any way forward is because this is an unsolvable situation.
Eliezer sometimes talks about how the textbook from the future would have simple alignment techniques that work easily but he is simply imagining things. He has no idea what these techniques might be, and simply assumes there must be a solution to the problem as he sees it.
- There are many possibilities of how AI might develop that don’t involve MIRI-like situations. The MIRI view essentially ignores economic and social considerations of how AI will be developed. They believe that the economic advantages of a super AI will lead to it eventually happening, but have never examined this belief critically, or even looked at the economic literature on this very big, very publicly important topic that many economists have worked on.
- A lot of abuse and bad behavior has been justified or swept under the rug in the name of ‘We must protect unaligned AGI from destroying the cosmic endowment’. This will probably keep happening for the foreseeable future.
- People going into this field don’t develop great option value.
I just want to say that this seems like a great idea, thanks for proposing it.
I have a mild preference for you to either i) do this in collaboration with a project like Stampy or ii) plan how to integrate what you do into with another existing project in the future.
In general, I think that we should i) minimise the number of education providers and ii) maximise uniformity of language and understanding within the AI existential risk educational ecosystem.
Also, just as feedback (which probably doesn’t warrant any changes being made unless similar feedback provided), I will flag that it would be good to be able to see posts that this is mentioned in ranked by recency rather than total karma.
- Part 3: A Proposed Approach for AI Safety Movement Building: Projects, Professions, Skills, and Ideas for the Future [long post][bounty for feedback] by (EA Forum; Mar 22, 2023, 12:54 AM; 22 points)
- A Proposed Approach for AI Safety Movement Building: Projects, Professions, Skills, and Ideas for the Future [long post][bounty for feedback] by (Mar 22, 2023, 1:11 AM; 14 points)
Is there a plan to review and revise this to keep it up to date? Or is there something similar that I can look at which is more updated? I have this saved as something to revisit, but I worry not that it could be out of date and inaccurate given the speed of progress.
Thanks! Quick responses:
I think these results, and the rest of the results from the larger survey that this content is a part of, have been interesting and useful to people, including Collin and I. I’m not sure what I expected beforehand in terms of helpfulness, especially since there’s a question “helpful with respect to /what/”, and I expect we may have different “what”s here.
Good to know. When discussing some recent ideas I had for surveys, several people told me that their survey results underperformed their expectations, so I was curious if you would say the same thing.
Yeah, I think this is currently mostly done informally—when Collin and I were choosing materials, we had a big list, and were choosing based on shared intuitions that EAs / ML researchers / fieldbuilders have, in addition to applying constraints like “shortness”. Our full original plan was also much longer and included testing more readings—this was a pilot survey. Relatedly, I don’t think these results are very surprising to people (which I think you’re alluding to in this comment) -- somewhat surprising, but we have a fair amount of information about researcher preferences already.
Thanks for explaining. I realise that the point of that part of my comment was unclear, sorry. I think that using these sorts of surveys to test if best practice contrasts with current practice could make the findings clearer and spur improvement/innovation if needed.
For instance, doing something like this: “We curated the 10 most popular public communication paper from AI Safety organisations and collected predictions from X public AI Safety communicators about which of thse materials would be most effective at persuading existing ML researchers to care about AI Safety. We tested these materials with a random sample of X ML researchers and [supported/challenged existing beliefs/practices]… etc.”
I am interested to hear what you think of the idea of testing using these sorts of surveys to test if best practice contrasts with current practice, but ok if you don’t have time to explain! I imagine that it does add some extra complexity and challenge to the research process, so may not be worth it.
I hope you can do the larger study eventually. If you do, I would also like to see how sharing readings compares against sharing podcasts or videos etc. Maybe some modes of communication perform better on average etc.Instead of contacting a random subset of people who had papers accepted at ML conferences? I think it sort of depends on one’s goals here, but could be good. A few thoughts: I think this may already exist informally, I think this becomes more important as there’s more people doing surveys and not coordinating with each other, and this doesn’t feel like a major need from my perspective / goals but might be more of a bottleneck for yours!
Thanks, that’s helpful. Yeah, I think that the panel idea is one for the future. My thinking is something like this: Understanding why and how AI Safety related materials (e.g., arguments, research agendas, recruitment type messages etc) influence ML researchers is going to become increasingly important to a growing number of AI Safety community actors (e.g., researchers, organisations, recruiters and movement builders).
Whenever an audience becomes important to some social/business actor (e.g., government/academics/companies), this usually creates sufficient demand to justify setting up a panel/database to service those actors. Assuming the same trend, it may be important/useful to create a panel of ML researchers that AI Safety actors can access.
Does that seem right?
I mention the above in part because I think that you are one of the people who might be best-placed to set something like this up if it seemed like a good idea. Also, because I think that there is a reasonable chance that I would use a service like this within the next two years and end up referring several other people (e.g., those producing choosing educational materials for relevant AI Safety courses) to use it.
Yeah, I agree with Kaj here. We do need to avoid the risk of using misleading or dishonest communication. However it also seems fine and important to optimise relevant communication variables (e.g., tone, topic, timing, concision, relevance etc) to maximise positive impact.
Yeah, probably an issue with Brave Browser or similar. I don’t have time to look into it more, but I am sure you can figure it out regardless. Might be worth adding a related message to the page (not sure)