Papers about LLM scaling law projections don’t make an attempt to estimate base rate of technological progress.
In particular, they don’t make an attempt to estimate the base rate of false alarms when inventing new science and technology. If you study any hard science (not computer science) you realise how high this rate is. False alarms are the norm and older scientists grow cynical partly because of how many false alarms by newer scientists they need to debunk. By false alarm I mean someone claiming a new technology will be world-changing, only to later find out the result was faked, or had experimental error, or won’t work outside of lab conditions, or won’t scale, or will be too expensive, or any other reason.
Anyone remember LK99? That is the norm for people inside the space, it’s just not the norm for normies on twitter hence it blew up.
AI timelines don’t account for base rate of tech progress
Papers about LLM scaling law projections don’t make an attempt to estimate base rate of technological progress.
In particular, they don’t make an attempt to estimate the base rate of false alarms when inventing new science and technology. If you study any hard science (not computer science) you realise how high this rate is. False alarms are the norm and older scientists grow cynical partly because of how many false alarms by newer scientists they need to debunk. By false alarm I mean someone claiming a new technology will be world-changing, only to later find out the result was faked, or had experimental error, or won’t work outside of lab conditions, or won’t scale, or will be too expensive, or any other reason.
Anyone remember LK99? That is the norm for people inside the space, it’s just not the norm for normies on twitter hence it blew up.