Counterfactual Credit Guided Bayesian Optimization

Abstract

While traditional Bayesian Optimization (BO) often focuses on building a globally accurate surrogate model (e.g., via Gaussian processes) to represent the entire objective function, practitioners are frequently more interested in identifying the global optimum as quickly as possible. The authors argue that it is often inefficient to treat all historical observations as equally important for this specific goal. To address this, we introduce Counterfactual Credit Guided Bayesian Optimization (CCGBO), a framework that uses ‘counterfactual credit’ to explicitly quantify the contribution of individual historical observations to the discovery of the optimum. By incorporating these counterfactual credit scores into the acquisition function, the model can more effectively allocate sampling resources toward regions where the global optimum is most likely to reside. We prove that CCGBO maintains sublinear regret and demonstrate through empirical evaluations on both synthetic and real-world benchmarks that the method accelerates convergence to the global optimum compared to standard baselines.

Qiyu Wei 魏启宇

Ph.D student.

My research interests include AI in Semiconductor, LLM agents and Bayesian Optimisation.