Three years later, and we actually got LLMs with visible thoughts, such as Deepseek, QwQ, and (although partially hidden from the user) o1-preview.
I (Nate) find it plausible that there are capabilities advances to be had from training language models on thought-annotated dungeon runs.
Good call!
But I don’t think these came about through training on synthetic thought-annotated texts.
Three years later, and we actually got LLMs with visible thoughts, such as Deepseek, QwQ, and (although partially hidden from the user) o1-preview.
Good call!
But I don’t think these came about through training on synthetic thought-annotated texts.