[link]
This paper presents a model inspired by the SAGE (Sparse Additive GEnerative) model of Eisenstein et al. The authors use a different approach for modeling the "background" component of the model. SAGE uses the same background model for all; the authors allow different backgrounds for different topics/classification labels/etc., but try to keep the background matrix low rank. To make inference faster when using this low rank constraint, they use a bound on the likelihood function that avoids the logsumexp calculations from SAGE. Experimental results are positive for a few different tasks. Sparse additive models represent sets of distributions over large vocabularies as loglinear combinations of a dense, shared background vector and a sparse, distributionspecific vector. The paper presents a modification that allows distributions to have distinct background vectors, but requires that the matrix of background vectors be lowrank. This method leads to better predictive performance in a labeled classification task and in a mixedmembership LDAlike setting. Previous work on SAGE introduced a new model for text. It built a lexical distribution by adding deviation components to a fixed background. The model presented in this paper SAMLRB, builds on SAGE and claims to improve it by two additions. First, providing a unique background for each class/topic. Second, providing an approximation of loglikelihood so as to provide a faster learning and inference algorithm in comparison to SAGE.
Your comment:
