Adam Jermyn comments on Multi-Component Learning and S-Curves

[Adam Jermyn]

It’s a good caution, but I do see more bumps with Adam than with SGD across a number of random initializations.

[Adam Jermyn]

(with the caveat that this is still “I tried a few times” and not any quantitative study)