Yes, but you were the one who started talking about it as something you can “run into”, together with terms like “as fast as physically possible” and “ideally could from the data”—that last term in particular has previously been used in conversations like this to refer to Solomonoff induction on an infinitely powerful computer.
My point is that at any given moment an awful lot of things will be bottlenecks, including real-world data. The curve of capability is already observed in cases where you are free to optimize whatever variable is the lowest hanging fruit at the moment.
In other words, you are already “into” the current data limit; if you could get better performance by using less data and substituting e.g. more computation, you would already be doing it.
As time goes by, the amount of data you need for a given degree of performance will drop as you obtain more computing power, better algorithms etc. (But of course, better performance still will be obtainable by using more data.) However,
The amount of data needed won’t drop below some lower bound,
More to the point, the rate at which the amount needed drops, is itself bound by the curve of capability.
Yes, but you were the one who started talking about it as something you can “run into”, together with terms like “as fast as physically possible” and “ideally could from the data”—that last term in particular has previously been used in conversations like this to refer to Solomonoff induction on an infinitely powerful computer.
My point is that at any given moment an awful lot of things will be bottlenecks, including real-world data. The curve of capability is already observed in cases where you are free to optimize whatever variable is the lowest hanging fruit at the moment.
In other words, you are already “into” the current data limit; if you could get better performance by using less data and substituting e.g. more computation, you would already be doing it.
As time goes by, the amount of data you need for a given degree of performance will drop as you obtain more computing power, better algorithms etc. (But of course, better performance still will be obtainable by using more data.) However,
The amount of data needed won’t drop below some lower bound,
More to the point, the rate at which the amount needed drops, is itself bound by the curve of capability.