TheBayesian
ACX Montreal Meetup August 17th 2024
I posted our submission in your twitter DMs and as a standalone post on LW the other day, but thought it wise to send it here as well: https://alignmentsearch.up.railway.app/
Mirroring other comments, we plan to get in contact with the team behind Stampy and possibly integrate some of the functionality of our project into their conversational agent.
There already exists a bunch of projects which do something similar. This technique is known as Retrieval-Augmented Generation, as described in this paper from May 2020. Tools like Langchain and openAI tutorials have been used to build similar projects quickly, and the tech (cheap openAI embeddings, separating the dataset into ~200 token chunks and chatGPT) have all existed and been used together for many months. A few projects I’ve seen that do something akin to what we do include HippocraticAI, Trevor Hubbard, and ChatLangChain. This could and will be applied more widely, like people adding Q&A abilities to their library’s documentation, to blogs, etc., but a key limitation is that, since it uses LLMs, it is pricier, slower and less reliable at inference time, without tricks that attempt to go around these limitations.
’Tis I. Didn’t intend bad incentives, the stakes on that market are imo pretty tiny. But I N/Aed, I don’t want anyone suspecting that had affected the final outcome.