1.4Q tokens (ignoring where the tokens come from for the moment), am I highly confident it will remain weak and safe?
I’m pretty confident that if all those tokens relate to cooking, you will get a very good recipe predictor.
Hell, I’ll give you 10^30 tokens about cooking and enough compute and your transformer will just be very good at predicting recipes.
Next-token predictors are IMO limited to predicting what’s in the dataset.
In order to get a powerful, dangerous AI from a token-predictor, you need a dataset where people are divulging the secrets of being powerful and dangerous. And in order to scale it, you need more of that.
So we cannot “ignore where the tokens come from” IMO. It actually matters a lot; in fact it’s kind of all that matters.
Based on my reading of the article, “Ignore where the tokens come from” is less about “Ignore the contents of the tokens” and more about “Pretend we can scale up our current approach to 1.4Q tokens by magic.” So we would assume that, similar to current LLM datasets, there would be a very broad set of topics featured, since we’re grabbing large quantities of data without specifically filtering for topic at any point.
Empirically, I don’t think it’s true that you’d need to rely on superhuman intelligence. The latest paper from the totally anonymous and definitely not google team suggests PaL- I mean an anonymous 540B parameter model- was good enough to critique itself into better performance. Bootstrapping to some degree is apparently possible.
I don’t think this specific instance of the technique is enough by itself to get to spookyland, but it’s evidence that token bottlenecks aren’t going to be much of a concern in the near future. There are a lot of paths forward.
I’d also argue that it’s very possible for even current architectures to achieve superhuman performance in certain tasks that were not obviously present in its training set. As a trivial example, these token predictors are obviously superhuman at token predicting without having a bunch of text about the task of token predicting provided. If some technique serves the task of token prediction and can be represented within the model, it may arise as a result of helping to predict tokens better.
It’s hard to say exactly what techniques fall within this set of “representable techniques which serve token predicting.” The things an AI can learn from the training set isn’t necessarily the same thing as what a human would say the text is about. Even current kinda-dumb architectures can happen across non-obvious relationships that grow into forms of alien reasoning (which, for now, remain somewhat limited).
I’m pretty confident that if all those tokens relate to cooking, you will get a very good recipe predictor.
Hell, I’ll give you 10^30 tokens about cooking and enough compute and your transformer will just be very good at predicting recipes.
Next-token predictors are IMO limited to predicting what’s in the dataset.
In order to get a powerful, dangerous AI from a token-predictor, you need a dataset where people are divulging the secrets of being powerful and dangerous. And in order to scale it, you need more of that.
So we cannot “ignore where the tokens come from” IMO. It actually matters a lot; in fact it’s kind of all that matters.
Based on my reading of the article, “Ignore where the tokens come from” is less about “Ignore the contents of the tokens” and more about “Pretend we can scale up our current approach to 1.4Q tokens by magic.” So we would assume that, similar to current LLM datasets, there would be a very broad set of topics featured, since we’re grabbing large quantities of data without specifically filtering for topic at any point.
Even if you did that, you might need a superhuman intelligence to generate tokens of sufficient quality to further scale the output.
(Jay’s interpretation was indeed my intent.)
Empirically, I don’t think it’s true that you’d need to rely on superhuman intelligence. The latest paper from the totally anonymous and definitely not google team suggests PaL- I mean an anonymous 540B parameter model- was good enough to critique itself into better performance. Bootstrapping to some degree is apparently possible.
I don’t think this specific instance of the technique is enough by itself to get to spookyland, but it’s evidence that token bottlenecks aren’t going to be much of a concern in the near future. There are a lot of paths forward.
I’d also argue that it’s very possible for even current architectures to achieve superhuman performance in certain tasks that were not obviously present in its training set. As a trivial example, these token predictors are obviously superhuman at token predicting without having a bunch of text about the task of token predicting provided. If some technique serves the task of token prediction and can be represented within the model, it may arise as a result of helping to predict tokens better.
It’s hard to say exactly what techniques fall within this set of “representable techniques which serve token predicting.” The things an AI can learn from the training set isn’t necessarily the same thing as what a human would say the text is about. Even current kinda-dumb architectures can happen across non-obvious relationships that grow into forms of alien reasoning (which, for now, remain somewhat limited).