Stan van Wingerden comments on Stan van Wingerden’s Shortform

[Stan van Wingerden]

Here’s the plot, which is very similar to Experience Machine’s:

My conclusion from this is that the LLC and the L2 norm measure basically the same thing in this setup. They don’t always: for further comparison with more unprincipled metrics in more complex setups, see comparisons with weight norm /​ Hessians in fig 22, 23, and 25 here and comparisons with Hessian-based methods and ablations here.