Anthropic estimates its frontier model will require on the order of 10^25 FLOPs, or floating point operations — several orders of magnitude larger than even the biggest models today.
This doesn’t make sense. GPT-4 used around 2*10^25 FLOP, someone estimated.
My guess is “today” was supposed to refer to some date when they were doing the investigation prior to the release of GPT-4, not the date the article was published.
Minerva (from June 2022) used 3e24; there’s no way “several orders of magnitude larger” was right when the article was being written. I think the author just made a mistake.
This doesn’t make sense. GPT-4 used around 2*10^25 FLOP, someone estimated.
Got a source for this estimate?
Epoch says 2.2e25. Skimming that page, it seems like a pretty unreliable estimate. They say their 90% confidence interval is about 1e25 to 5e25.
My guess is “today” was supposed to refer to some date when they were doing the investigation prior to the release of GPT-4, not the date the article was published.
Minerva (from June 2022) used 3e24; there’s no way “several orders of magnitude larger” was right when the article was being written. I think the author just made a mistake.