[link]
This paper presents a multitask Bayesian optimization approach to hyperparameter setting in machine learning models. In particular, it leverages previous work on multitask GP learning with decomposable covariance functions and Bayesian optimization of expensive cost functions. Previous work has shown that decomposable covariance functions can be useful in multitask regression problems (e.g. \cite{conf/nips/BonillaCW07}) and that Bayesian optimization based on responsesurfaces can also be useful for hyperparameter tuning of machine learning algorithms \cite{conf/nips/SnoekLA12} \cite{conf/icml/BergstraYC13}. The paper combines the decomposable covariance assumption \cite{conf/nips/BonillaCW07} and Bayesian optimization based on expected improvement \cite{journals/jgo/Jones01} and entropy search \cite{conf/icml/BergstraYC13} to show empirically that it is possible to : 1. Transfer optimization knowledge across related problems, addressing e.g. the coldstart problem 2. Optimize an aggregate of different objective functions with applications to speedingup cross validation 3. Use information from a smaller problem to help optimize a bigger problem faster Positive experimental results are shown on synthetic data (BraninHoo function), optimizing logistic regression hyperparameters and optimizing hyperparameters of online LDA on real data.
Your comment:
