[...]Search uses a dataset about [...] to construct a prompt for ChatGPT to answer [...]-related questions, while citing established sources… with results either on par or better than those given by much stronger LLMs (GPT-4 and Bing Search).
You’ve done this for the specific topic of AI alignment, but it seems like you have a methodology here, for getting good answers from ChatGPT about a specialized topic, that could be applied much more widely. Is anyone else doing this? Did you come up with all this yourselves?
There already exists a bunch of projects which do something similar. This technique is known as Retrieval-Augmented Generation, as described in this paper from May 2020. Tools like Langchain and openAI tutorials have been used to build similar projects quickly, and the tech (cheap openAI embeddings, separating the dataset into ~200 token chunks and chatGPT) have all existed and been used together for many months. A few projects I’ve seen that do something akin to what we do include HippocraticAI, Trevor Hubbard, and ChatLangChain. This could and will be applied more widely, like people adding Q&A abilities to their library’s documentation, to blogs, etc., but a key limitation is that, since it uses LLMs, it is pricier, slower and less reliable at inference time, without tricks that attempt to go around these limitations.
You’ve done this for the specific topic of AI alignment, but it seems like you have a methodology here, for getting good answers from ChatGPT about a specialized topic, that could be applied much more widely. Is anyone else doing this? Did you come up with all this yourselves?
There already exists a bunch of projects which do something similar. This technique is known as Retrieval-Augmented Generation, as described in this paper from May 2020. Tools like Langchain and openAI tutorials have been used to build similar projects quickly, and the tech (cheap openAI embeddings, separating the dataset into ~200 token chunks and chatGPT) have all existed and been used together for many months. A few projects I’ve seen that do something akin to what we do include HippocraticAI, Trevor Hubbard, and ChatLangChain. This could and will be applied more widely, like people adding Q&A abilities to their library’s documentation, to blogs, etc., but a key limitation is that, since it uses LLMs, it is pricier, slower and less reliable at inference time, without tricks that attempt to go around these limitations.