Gradient-based Sample Selection for Faster Bayesian Optimization

Abstract

Standard Bayesian Optimization (BO) typically relies on Gaussian Process (GP) surrogate models. Because fitting a GP has $O(n^3)$ computational complexity (where $n$ is the number of data samples), BO becomes difficult to scale to large-budget scenarios. In this work, we propose a gradient-based sample selection (GSSBO) method designed to improve the computational efficiency of BO. Instead of using the entire dataset to fit the GP, the model is constructed using only a selected subset of samples. These samples are chosen by leveraging gradient information to eliminate redundancy while maintaining the diversity and representativeness of the dataset. Theoretical analysis in the paper provides explicit sublinear regret bounds for the framework. Experimental results on both synthetic and real-world tasks indicate that the method significantly reduces GP fitting costs while maintaining performance comparable to baseline methods.

Qiyu Wei 魏启宇

Ph.D student.

My research interests include AI in Semiconductor, LLM agents and Bayesian Optimisation.