Abstract—This paper proposes a new hyperparameter
search method involving elliptical grid transformations and
rotations of a grid of probe points. This technique is termed
“rotated grid search”. We begin by motivating the method by
discussing the limitations of random search. A new formalism
for more efficiently probing a hyperparameter search space is
then proposed. Next, we build a theoretical framework to
compare hyperparameter optimization performance of rotated
grid search against random search. We then evaluate both
search methods empirically to quantify the marginal benefit of
using one over the other. Monte-Carlo simulations on various
synthetic objective functions show that rotated grid search
outperforms random search over the full range of anisotropy
explored in this study. Finally, we conduct a case study on a real
dataset, rectangles-images, and show that rotated grid search
outperforms random search in a high dimensional space.
Index Terms—Random search, grid search, global
optimization, model selection, rotated grid search, neural
networks, deep learning
The authors are with the MRG Machine Learning Center of Excellence
at J.P.Morgan Chase & Co. in Manhattan, NY 10016, USA (e-mail: altan.allawala@gmail.com, krr2125@columbia.edu, pavan.wadhwa@jpmorgan.com)
Cite: Altan Allawala, Killian Rutherford, and Pavan Wadhwa, "Rotated Grid Search for Hyperparameter Optimization," International Journal of Machine Learning and Computing vol. 12, no. 5, pp. 266-271, 2022.
Copyright @ 2022 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).