Neural Architecture Search with Reinforcement LearningNeural Architecture Search with Reinforcement LearningBarret Zoph and Quoc V. Le2016
Paper summaryleopaillier_Objective:_ Design a network that will itself find the best architecture for a given task.
_Dataset:_ [CIFAR10](https://www.cs.toronto.edu/%7Ekriz/cifar.html) and [PTB](https://catalog.ldc.upenn.edu/ldc99t42).
The meta-network (a RNN) generates a string specifying the child network parameters. Such a child network is then trained for 35-50 epochs and its accuracy is used as the reward to train the meta-network with Reinforcement Learning.
The RNN first generates networks with few layers (6) then this number is increased as training progresses.
They develop one architecture for CNN where they predict each layers characteristic plus it's possible skip-connection:
[![screen shot 2017-05-24 at 8 13 01 am](https://cloud.githubusercontent.com/assets/17261080/26389176/d807de42-4058-11e7-942a-8a129558e126.png)](https://cloud.githubusercontent.com/assets/17261080/26389176/d807de42-4058-11e7-942a-8a129558e126.png)
And one specific for LTSM-style:
[![screen shot 2017-05-24 at 8 13 26 am](https://cloud.githubusercontent.com/assets/17261080/26389190/e2bfd506-4058-11e7-9168-62abd040156e.png)](https://cloud.githubusercontent.com/assets/17261080/26389190/e2bfd506-4058-11e7-9168-62abd040156e.png)
## Distributed setting:
Bellow is the distributed setting that they use with parameter servers connected to replicas (GPUs) that trained child networks.
[![screen shot 2017-05-24 at 8 09 05 am](https://cloud.githubusercontent.com/assets/17261080/26389084/5e354456-4058-11e7-83a9-089cb2c115b7.png)](https://cloud.githubusercontent.com/assets/17261080/26389084/5e354456-4058-11e7-83a9-089cb2c115b7.png)
Overall they trained 12800 networks on 800 GPUs but they achieve state of the art results which not human intervention except the vocabulary selection (activation type, type of cells, etc). Next step, transfer learning from one task to another for the meta-network?
First published: 2016/11/05 (2 years ago) Abstract: Neural networks are powerful and flexible models that work well for many
difficult learning tasks in image, speech and natural language understanding.
Despite their success, neural networks are still hard to design. In this paper,
we use a recurrent network to generate the model descriptions of neural
networks and train this RNN with reinforcement learning to maximize the
expected accuracy of the generated architectures on a validation set. On the
CIFAR-10 dataset, our method, starting from scratch, can design a novel network
architecture that rivals the best human-invented architecture in terms of test
set accuracy. Our CIFAR-10 model achieves a test error rate of 3.84, which is
only 0.1 percent worse and 1.2x faster than the current state-of-the-art model.
On the Penn Treebank dataset, our model can compose a novel recurrent cell that
outperforms the widely-used LSTM cell, and other state-of-the-art baselines.
Our cell achieves a test set perplexity of 62.4 on the Penn Treebank, which is
3.6 perplexity better than the previous state-of-the-art.