2) The “larger models are simpler” happens only after training to zero loss (at least if you’re using the double descent explanation for it, which is what I was thinking of).
3) Fair point; though note that for that you should also count up all the other things the brain has to do (e.g. motor control)
4) If “redoing evolution” produces AGI; I would expect that a mesa optimizer would “come from” the evolution, not the individual level; so to the extent you want to argue “double descent implies simple large models implies mesa optimization”, you have to apply that argument to evolution.
(Probably you were asking about the question independently of the mesa optimization point; I do still hold this opinion more weakly for generic “AI systems of the future”; there the intuition comes from humans being underparameterized and from an intuition that AI systems of the future should be able to make use of more cheap, diverse / noisy data, e.g. YouTube.)
2) The “larger models are simpler” happens only after training to zero loss (at least if you’re using the double descent explanation for it, which is what I was thinking of).
3) Fair point; though note that for that you should also count up all the other things the brain has to do (e.g. motor control)
4) If “redoing evolution” produces AGI; I would expect that a mesa optimizer would “come from” the evolution, not the individual level; so to the extent you want to argue “double descent implies simple large models implies mesa optimization”, you have to apply that argument to evolution.
(Probably you were asking about the question independently of the mesa optimization point; I do still hold this opinion more weakly for generic “AI systems of the future”; there the intuition comes from humans being underparameterized and from an intuition that AI systems of the future should be able to make use of more cheap, diverse / noisy data, e.g. YouTube.)
Sounds like we agree :)