My understanding is that the 2020 algorithms in Ajeya Cotra’s draft report refer to algorithms that train a neural network on a given architecture (rather than algorithms that search for a good neural architecture etc.). So the only “special sauce” that can be found by such algorithms is one that corresponds to special weights of a network (rather than special architectures etc.).
My understanding is that the 2020 algorithms in Ajeya Cotra’s draft report refer to algorithms that train a neural network on a given architecture (rather than algorithms that search for a good neural architecture etc.). So the only “special sauce” that can be found by such algorithms is one that corresponds to special weights of a network (rather than special architectures etc.).
Huh, that’s not how I interpreted it. I should reread the report. Thanks for raising this issue.