Something I’m unsure about (commenting from my mod-perspective but not making a mod pronouncement) is how LW should relate to posts that lay out ideas that may advance AI capabilities.
My current understanding is that all major AI labs have already figured out the chinchilla results on their own, but that younger or less in-the-loop AI orgs may have needed to run experiments that took a couple months of staff time. This post was one of the most-read posts on LW this month, and shared heavily around twitter. It’s plausible to me that spreading these arguments plausibly speeds up AI timelines by 1-4 weeks on average.
It seems important to be able to talk about that and model the world, but I’m wondering if posts like this should live behind a “need to log-in” filter, maybe with a slight karma-gate, so that the people who end up reading it are at least more likely to be plugged into the LW ecosystem and are also going to get exposed to arguments about AI risk.
nostalgiabraist, I’m curious how you would feel about that.
so that the people who end up reading it are at least more likely to be plugged into the LW ecosystem and are also going to get exposed to arguments about AI risk.
There’s also the chance that if these posts are not gated, people who previously weren’t plugged into the LW ecosystem but are interested in AI find LW through articles such as this one. And then eventually also start reading other articles here and become more interested in alignment concerns.
There’s also a bit of a negative stereotype among some AI researchers as alignment people being theoretical philosophers doing their own thing and being entirely out of touch about what real AI is like. They might take alignment concerns a bit more seriously if they find it easy to actually find competent AI discussion on LW / Alignment Forum.
My current understanding is that all major AI labs have already figured out the chinchilla results on their own, but that younger or less in-the-loop AI orgs may have needed to run experiments that took a couple months of staff time. This post was one of the most-read posts on LW this month, and shared heavily around twitter. It’s plausible to me that spreading these arguments plausibly speeds up AI timelines by 1-4 weeks on average.
What is the mechanism you’re imagining for this speedup? What happens that would not have happened without this post?
Consider that
The Chinchilla paper was released over four months ago, on 3/29/22.
It did not take long for the paper to get noticed among people interested in ML scaling, including here on LW.
I’m struggling to imagine a situation where a relevant AI org is doing Chinchilla-like scaling experiments, yet somehow has managed to miss this paper (or to ignore/misunderstand it) for 4+ months. The paper is not exactly a secret, and it’s not even especially difficult to read as these things go.
More broadly, I doubt LW has significant leverage to decrease the overall supply of these kinds of conversations. There are lots of venues for cutting-edge ML discussion, and the conversation is going to happen somewhere. (See Connor’s comments here.)
I don’t have a strong opinion on hiding nostalgebraist’s post behind a login gate. But as a data point, I’m not affiliated with any major labs and I don’t currently work on LLMs but in other AI fields and I still read the Chinchilla paper before seeing this post (and I wasn’t surprised by its results), so hiding the post wouldn’t have made much of a difference for me.
However, I am very surprised that a report like ELK is publicly available for any web crawl to access. I think that if you query a future LLM that has this report in its training data and prompt it with a query related to hiding its intentions, you’ll get much better results. Is this desirable? It seems to provide a shortcut to a lot of knowledge that an LLM would need to reason about by itself without access.
This is very tricky. On one hand, this may actually Streisand effect these results to greater prominence. On the other hand, at the point where people were specifically working around this to gain access to log-in gated LW resources, this would probably enhance our community status/prestige which might actually increase our influence.
Overall, I’d lean towards carefully experimenting with a log-in filter, with the notion of abandoning this idea if it doesn’t seem to be achieving its goals.
Something I’m unsure about (commenting from my mod-perspective but not making a mod pronouncement) is how LW should relate to posts that lay out ideas that may advance AI capabilities.
My current understanding is that all major AI labs have already figured out the chinchilla results on their own, but that younger or less in-the-loop AI orgs may have needed to run experiments that took a couple months of staff time. This post was one of the most-read posts on LW this month, and shared heavily around twitter. It’s plausible to me that spreading these arguments plausibly speeds up AI timelines by 1-4 weeks on average.
It seems important to be able to talk about that and model the world, but I’m wondering if posts like this should live behind a “need to log-in” filter, maybe with a slight karma-gate, so that the people who end up reading it are at least more likely to be plugged into the LW ecosystem and are also going to get exposed to arguments about AI risk.
nostalgiabraist, I’m curious how you would feel about that.
There’s also the chance that if these posts are not gated, people who previously weren’t plugged into the LW ecosystem but are interested in AI find LW through articles such as this one. And then eventually also start reading other articles here and become more interested in alignment concerns.
There’s also a bit of a negative stereotype among some AI researchers as alignment people being theoretical philosophers doing their own thing and being entirely out of touch about what real AI is like. They might take alignment concerns a bit more seriously if they find it easy to actually find competent AI discussion on LW / Alignment Forum.
What is the mechanism you’re imagining for this speedup? What happens that would not have happened without this post?
Consider that
The Chinchilla paper was released over four months ago, on 3/29/22.
It did not take long for the paper to get noticed among people interested in ML scaling, including here on LW.
On 3⁄29, the same day it was released, the paper was linked on r/mlscaling.
On 3⁄31, I heard about it through the EleutherAI discord, and immediately made an LW linkpost.
On 4⁄1, 1a3orn posted a more detailed explainer.
I’m struggling to imagine a situation where a relevant AI org is doing Chinchilla-like scaling experiments, yet somehow has managed to miss this paper (or to ignore/misunderstand it) for 4+ months. The paper is not exactly a secret, and it’s not even especially difficult to read as these things go.
More broadly, I doubt LW has significant leverage to decrease the overall supply of these kinds of conversations. There are lots of venues for cutting-edge ML discussion, and the conversation is going to happen somewhere. (See Connor’s comments here.)
I don’t have a strong opinion on hiding nostalgebraist’s post behind a login gate. But as a data point, I’m not affiliated with any major labs and I don’t currently work on LLMs but in other AI fields and I still read the Chinchilla paper before seeing this post (and I wasn’t surprised by its results), so hiding the post wouldn’t have made much of a difference for me.
However, I am very surprised that a report like ELK is publicly available for any web crawl to access. I think that if you query a future LLM that has this report in its training data and prompt it with a query related to hiding its intentions, you’ll get much better results. Is this desirable? It seems to provide a shortcut to a lot of knowledge that an LLM would need to reason about by itself without access.
Yeah a few people have also brought up this concern recently. Will think about it.
This is very tricky. On one hand, this may actually Streisand effect these results to greater prominence. On the other hand, at the point where people were specifically working around this to gain access to log-in gated LW resources, this would probably enhance our community status/prestige which might actually increase our influence.
Overall, I’d lean towards carefully experimenting with a log-in filter, with the notion of abandoning this idea if it doesn’t seem to be achieving its goals.