Deep Learning Theory

Understanding Incremental Learning of Gradient Descent -- A Fine-grained Analysis of Matrix Sensing
It is believed that Gradient Descent (GD) induces an implicit bias towards good generalization in training machine learning models. …
Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon S. Du, Jason D. Lee
PDF Cite ArXiv
Understanding Incremental Learning of Gradient Descent -- A Fine-grained Analysis of Matrix Sensing
Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power
We provide theoretical evidence that the hardness of robust generalization may stem from the expressive power of deep neural networks. Even when standard generalization is easy, robust generalization provably requires the size of DNNs to be exponentially large.
Binghui Li, Jikai Jin, Han Zhong, John E. Hopcroft, Liwei Wang
PDF Cite ArXiv
Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power