The bandwidth hypothesis is an interesting one, but I’m not sure the conclusion follows from it. Increasing I/O bandwidth by a few orders of magnitude doesn’t necessarily require increasing compute by the same ratio.
The dataset concerns are already being addressed, but one thing I noticed was the point “it’s going to require real-life data of e.g. of how to interact with humans”. That does indeed seem like something that we might want to have lots of training data on, but mostly regarding alignment rather than capability. I don’t see any way in which lack of human-AI interaction data reduces capabilities, except indirectly by humans coordinating on not improving capabilities until such data exists.
The bandwidth hypothesis is an interesting one, but I’m not sure the conclusion follows from it. Increasing I/O bandwidth by a few orders of magnitude doesn’t necessarily require increasing compute by the same ratio.
The dataset concerns are already being addressed, but one thing I noticed was the point “it’s going to require real-life data of e.g. of how to interact with humans”. That does indeed seem like something that we might want to have lots of training data on, but mostly regarding alignment rather than capability. I don’t see any way in which lack of human-AI interaction data reduces capabilities, except indirectly by humans coordinating on not improving capabilities until such data exists.