Understanding deep learning requires rethinking generalization Understanding deep learning requires rethinking generalization
Paper summary The authors investigate the generalisation properties of several well-known image recognition networks. https://i.imgur.com/km0mrVs.png They show that these networks are able to overfit to the training set with 100% accuracy even if the labels on the images are random, or if the pixels are randomly generated. Regularisation, such as weight decay and dropout, doesn’t stop overfitting as much as expected, still resulting in ~90% accuracy on random training data. They then argue that these models likely make use of massive memorization, in combination with learning low-complexity patterns, in order to perform well on these tasks.
arxiv.org
arxiv-sanity.com
scholar.google.com
Understanding deep learning requires rethinking generalization
Chiyuan Zhang and Samy Bengio and Moritz Hardt and Benjamin Recht and Oriol Vinyals
arXiv e-Print archive - 2016 via Local arXiv
Keywords: cs.LG

more

Summary by Martin Thoma 1 year ago
Loading...
Your comment:
Summary by Léo Paillier 11 months ago
Loading...
Your comment:
Summary by Marek Rei 10 months ago
Loading...
Your comment:


ShortScience.org allows researchers to publish paper summaries that are voted on and ranked!
About

Sponsored by: and