Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
Denton, Emily L. and Zaremba, Wojciech and Bruna, Joan and LeCun, Yann and Fergus, Rob
2014
Paper summary
nipsreviews
The paper addresses the problem of speeding up the evaluation of pre-trained image classification ConvNets. To this end, a number of techniques are proposed, which are based on the tensor representation of the conv. layer weight matrix. Namely, the following techniques are considered (Sect. 3.2-3.5):
1. SVD decomposition of the tensor
2. outer product decomposition of the tensor
3. monochromatic approximation of the first conv. layer - projecting RGB colors to a 1-D space, followed by clustering
4. biclustering tensor approximation - clustering input and output features to split the tensor into a number of sub-tensors, each of which is then separately approximated
5. fine-tuning of approximate models to (partially) recover the lost accuracy
