Exploiting Linear Structure Within Convolutional Networks for Efficient EvaluationExploiting Linear Structure Within Convolutional Networks for Efficient EvaluationDenton, Emily L. and Zaremba, Wojciech and Bruna, Joan and LeCun, Yann and Fergus, Rob2014
Paper summarynipsreviewsThe paper addresses the problem of speeding up the evaluation of pre-trained image classification ConvNets. To this end, a number of techniques are proposed, which are based on the tensor representation of the conv. layer weight matrix. Namely, the following techniques are considered (Sect. 3.2-3.5):
1. SVD decomposition of the tensor
2. outer product decomposition of the tensor
3. monochromatic approximation of the first conv. layer - projecting RGB colors to a 1-D space, followed by clustering
4. biclustering tensor approximation - clustering input and output features to split the tensor into a number of sub-tensors, each of which is then separately approximated
5. fine-tuning of approximate models to (partially) recover the lost accuracy
The paper addresses the problem of speeding up the evaluation of pre-trained image classification ConvNets. To this end, a number of techniques are proposed, which are based on the tensor representation of the conv. layer weight matrix. Namely, the following techniques are considered (Sect. 3.2-3.5):
1. SVD decomposition of the tensor
2. outer product decomposition of the tensor
3. monochromatic approximation of the first conv. layer - projecting RGB colors to a 1-D space, followed by clustering
4. biclustering tensor approximation - clustering input and output features to split the tensor into a number of sub-tensors, each of which is then separately approximated
5. fine-tuning of approximate models to (partially) recover the lost accuracy