Sorry for not responding to this. Examples do seem great, though digging up the exact charts I remember has turned out to be a bit of a longer time investment.
Some quick things I remembered feeling not that informative:
Go performance measured in ELO felt pretty hard to forecast from this kind of graph
Things like “When does chain-of-thought reasoning work?” for LLMs
LLM performance on various arithmetic tasks
Things like Alphafold, where I feel like there was basically no precursor. I remember there being forecasts about DL and protein folding, and I feel like none of them were very informative about when it would actually fall.
Sorry again for not linking to things. I might get around writing a post on this, since I do think it really deserves more exploration, but time is short these days.
Sorry for not responding to this. Examples do seem great, though digging up the exact charts I remember has turned out to be a bit of a longer time investment.
Some quick things I remembered feeling not that informative:
Go performance measured in ELO felt pretty hard to forecast from this kind of graph
Things like “When does chain-of-thought reasoning work?” for LLMs
LLM performance on various arithmetic tasks
Things like Alphafold, where I feel like there was basically no precursor. I remember there being forecasts about DL and protein folding, and I feel like none of them were very informative about when it would actually fall.
Sorry again for not linking to things. I might get around writing a post on this, since I do think it really deserves more exploration, but time is short these days.