I think it’s a mistake to think of “productivity is linear in effort” as the “no diminishing returns” model, and to consider it a degenerate extreme case. Linear returns is the model where doubling inputs leads to doubled outputs. A priori, it’s nearly as natural for constant additional effort leads to doubling of efficiency, so we need to actually look at the data to distinguish.
(It seems more theoretically natural—and more common in practice—for each clever trick to lead to a 10% increase in efficiency, then for each clever trick to lead to an absolute increase of 1 unit of efficiency.)
In semiconductors, as you point out, output has increased exponentially over time. Research investment has also increased exponentially, but with a significantly smaller exponent. So on your model the curve appears to be xα for α>1.
The performance curves database contains many interesting time series, and you’ll note that the y-axis is typically exponential. They don’t track inputs, so it’s a bit hard to draw conclusions, but comparing to overall increases in R&D investment it looks like superlinear returns are probably quite common.
A few years ago Katja looked into the rate of algorithmic progress, and found that it was very approximately comparable to the rate of progress in hardware (though it’s hard to know how much of that comes from realizing increasing economies of scale w.r.t. compute), across a range of domains. Algorithms seem like a particularly relevant domain to the current discussion.
I think it’s a mistake to think of “productivity is linear in effort” as the “no diminishing returns” model, and to consider it a degenerate extreme case. Linear returns is the model where doubling inputs leads to doubled outputs. A priori, it’s nearly as natural for constant additional effort leads to doubling of efficiency, so we need to actually look at the data to distinguish.
(It seems more theoretically natural—and more common in practice—for each clever trick to lead to a 10% increase in efficiency, then for each clever trick to lead to an absolute increase of 1 unit of efficiency.)
In semiconductors, as you point out, output has increased exponentially over time. Research investment has also increased exponentially, but with a significantly smaller exponent. So on your model the curve appears to be xα for α>1.
The performance curves database contains many interesting time series, and you’ll note that the y-axis is typically exponential. They don’t track inputs, so it’s a bit hard to draw conclusions, but comparing to overall increases in R&D investment it looks like superlinear returns are probably quite common.
A few years ago Katja looked into the rate of algorithmic progress, and found that it was very approximately comparable to the rate of progress in hardware (though it’s hard to know how much of that comes from realizing increasing economies of scale w.r.t. compute), across a range of domains. Algorithms seem like a particularly relevant domain to the current discussion.