Can you say a bit more about Eleuther’s involvement in the last of these papers? I thought that this was mostly done by people working at Apollo. EleutherAI is credited for providing compute (at the same level as OpenAI for providing GPT-4 credits), but I am surprised that you would claim it as work produced by EleutherAI?
Ooops. It appeared that I deleted my comment (deeming it largely off-topic) right as you were replying. I’ll reproduce the comment below, and then reply to your question.
I separately had a very weird experience with them on the Long Term Future Fund where Conor Leahy applied for funding for Eleuther AI. We told him we didn’t want to fund Eleuther AI since it sure mostly seemed like capabilities-research but we would be pretty interested in funding AI Alignment research by some of the same people.
He then confusingly went around to a lot of people around EleutherAI and told them that “Open Phil is not interested in funding pre-paradigmatic AI Alignment research and that that is the reason why they didn’t fund Eleuther AI”.
This was doubly confusing and misleading because Open Phil had never evaluated a grant to Eleuther AI (Asya who works at Open Phil was involved in the grant evaluation as a fund member, but nothing else), and of course the reason he cited had nothing to do with the reason we actually gave. He seems to have kept saying this for a long time even after I think someone explicitly corrected the statement to
I had no vibes along the lines of “oh we don’t like EleutherAI” or “we don’t fund pre-paradigmatic research.” It was a surprise to some people at Open Phil that we had areas of overlapping interest, but we spent like half an hour clarifying our research agenda and half an hour talking about what we wanted to do next and people were already excited.
We ended up talking about this in DMs, but to gist of it is:
Back in June Hoagy opened a thread in our “community research projects” channel and the work migrated there. Three of the five authors of the [eventual paper](https://arxiv.org/abs/2309.08600) chose to have EleutherAI affiliation (for any work we organize with volunteers, we tell them they’re welcome to use an EleutherAI affiliation on the paper if they like) and we now have an entire channel dedicated to future work. I believe Hoagy has two separate paper ideas currently in the works and over a half dozen people working on them.
Yeah, seems like a reasonable thing to mention.
Can you say a bit more about Eleuther’s involvement in the last of these papers? I thought that this was mostly done by people working at Apollo. EleutherAI is credited for providing compute (at the same level as OpenAI for providing GPT-4 credits), but I am surprised that you would claim it as work produced by EleutherAI?
Ooops. It appeared that I deleted my comment (deeming it largely off-topic) right as you were replying. I’ll reproduce the comment below, and then reply to your question.
While this anecdote is largely orthogonal to the broader piece, I remembered that this existed randomly today and wanted to mention that Open Phil has recommended a 2.6 M/3 years grant to EleutherAI to pursue interpretability research. It was a really pleasant and very easy experience: Nora Belrose (head of interpretability) and I (head of everything) talked with them about some of our recent and on-going work such as Eliciting Latent Predictions from Transformers with the Tuned Lens, Eliciting Latent Knowledge from Quirky Language Models, and Sparse Autoencoders Find Highly Interpretable Features in Language Models very interesting and once they knew we had shared areas of interest it was a really easy experience.
I had no vibes along the lines of “oh we don’t like EleutherAI” or “we don’t fund pre-paradigmatic research.” It was a surprise to some people at Open Phil that we had areas of overlapping interest, but we spent like half an hour clarifying our research agenda and half an hour talking about what we wanted to do next and people were already excited.
We ended up talking about this in DMs, but to gist of it is:
Back in June Hoagy opened a thread in our “community research projects” channel and the work migrated there. Three of the five authors of the [eventual paper](https://arxiv.org/abs/2309.08600) chose to have EleutherAI affiliation (for any work we organize with volunteers, we tell them they’re welcome to use an EleutherAI affiliation on the paper if they like) and we now have an entire channel dedicated to future work. I believe Hoagy has two separate paper ideas currently in the works and over a half dozen people working on them.