Learning dynamics of neural networks: the good, the bad and the ugly Deep neural networks are almost always trained using a simple gradient based method such as Adam or SGD. Interestingly, in deep learning optimization and generalization are entangled. For […]