Skip to Main content Skip to Navigation
Conference papers

Hyperparameter optimization of deep neural networks: combining Hperband with Bayesian model selection

Abstract : One common problem in building deep learning architectures is the choice of the hyper-parameters. Among the various existing strategies, we propose to combine two complementary ones. On the one hand, the Hyperband method formalizes hyper-parameter optimization as a resource allocation problem, where the resource is the time to be distributed between many configurations to test. On the other hand, Bayesian optimization tries to model the hyper-parameter space as efficiently as possible to select the next model to train. Our approach is to model the space with a Gaussian process and sample the next group of models to evaluate with Hyperband. Preliminary results show a slight improvement over each method individually, suggesting the need and interest for further experiments.
Complete list of metadatas

https://hal.telecom-paris.fr/hal-02412262
Contributor : Telecomparis Hal <>
Submitted on : Sunday, December 15, 2019 - 12:54:18 PM
Last modification on : Friday, July 31, 2020 - 11:28:04 AM

Identifiers

  • HAL Id : hal-02412262, version 1

Citation

Hadrien Bertrand, Roberto Ardon, Matthieu Perrot, Isabelle Bloch. Hyperparameter optimization of deep neural networks: combining Hperband with Bayesian model selection. CAp, 2017, Grenoble, France. ⟨hal-02412262⟩

Share

Metrics

Record views

29