gwern comments on Matt Botvinick on the spontaneous emergence of learning algorithms

[gwern]

Learning still happening after weights are frozen? That’s crazy. I think it’s a big deal because it is evidence for mesa-optimization being likely and hard to avoid.

Sure. We see that elsewhere too, like Dactyl. And of course, GPT-3.

[Daniel Kokotajlo]

Huh, thanks.