A Differentiable Transition Between Additive and Multiplicative NeuronsA Differentiable Transition Between Additive and Multiplicative NeuronsKöpp, Wiebke and van der Smagt, Patrick and Urban, Sebastian2016
Paper summaryopenreviewThe paper suggests using a differentiable function which can smoothly interpolate between multiplicative and additive gates in neural networks. It is an intriguing idea and the paper is well written. The mathematical ideas introduced are perhaps not novel (a cursory search seems to indicate that Abel's functional equation with f=exp is called the tetration equation and its solution called the iterated logarithm), but their use in machine learning seem to be.
The paper suggests using a differentiable function which can smoothly interpolate between multiplicative and additive gates in neural networks. It is an intriguing idea and the paper is well written. The mathematical ideas introduced are perhaps not novel (a cursory search seems to indicate that Abel's functional equation with f=exp is called the tetration equation and its solution called the iterated logarithm), but their use in machine learning seem to be.