I do not currently know of any way to do the kinds of calculations, approximately-Bayesian updates, and splintered model validity bounds-checking that I have been describing by using solely a deep neural net trained by any current deep learning techniques
There is a recent result suggesting that for sufficiently-deep neural nets, in-context learning approaches Bayesian optimality, which (with sufficiently large contexts) might form a basis for this. If so, then that would motivate improving our Interpretability for in-context learning.
There is a recent result suggesting that for sufficiently-deep neural nets, in-context learning approaches Bayesian optimality, which (with sufficiently large contexts) might form a basis for this. If so, then that would motivate improving our Interpretability for in-context learning.