like I’m surprised if a clever innovation does more good than spending 4x more compute
I worry that I’m misunderstanding this assertion because, as it stands, it sounds extremely likely that I’d win. Would transformers vs. CNNs/RNNs have won this the year that the transformers paper came out?
Hm. I wonder if there’s a bet to be extracted from this. Like: Eliezer says that Alphafold 2 beats [algorithms previous to Alphafold 2, but with 10x compute], and Paul says the latter beats the former? Or replace Alphafold 2 with anything that Eliezer thinks contains some amount of secret sauce over previous things (whether or not its performance is “on trend”).
Can I have a Bayes point for this comment ?
Quoting it: