gwern comments on What exactly is GPT-3′s base objective?

[gwern]

This paper today might be relevant: “An Explanation of In-context Learning as Implicit Bayesian Inference”, Xie et al 2021.