Second Order Derivatives for Network Pruning: Optimal Brain SurgeonSecond Order Derivatives for Network Pruning: Optimal Brain SurgeonHassibi, Babak and Stork, David G.1992
Paper summarymartinthomaOptimal Brain Surgeon (OBS) is about pruning neural networks to minimize the amount of parameters, training time and overfitting. It is very similar to [Optimal Brain Damage](http://www.shortscience.org/paper?bibtexKey=conf/nips/CunDS89#martinthoma), but claims to choose better weights. However, it does require to compute the inverse hessian. The hessian matrix of a neural network is a $n \times n$ matrix, where $n$ is the number of parameters of the network. Typically, $n > 10^6$. This makes the approach unusable.
Optimal Brain Surgeon (OBS) is about pruning neural networks to minimize the amount of parameters, training time and overfitting. It is very similar to [Optimal Brain Damage](http://www.shortscience.org/paper?bibtexKey=conf/nips/CunDS89#martinthoma), but claims to choose better weights. However, it does require to compute the inverse hessian. The hessian matrix of a neural network is a $n \times n$ matrix, where $n$ is the number of parameters of the network. Typically, $n > 10^6$. This makes the approach unusable.