Adaptively Incorporating Directional Hints into Zeroth-Order Optimization

This paper introduces Control-Variate Zeroth-Order Descent, or CV-ZOD, for nonconvex optimization when gradients are unavailable but low-dimensional directional hints are accessible. Instead of biasing the search toward a surrogate, CV-ZOD uses the hint subspace to construct a control-variate estimator that remains unbiased for the Gaussian-smoothed gradient, reducing variance when the hint aligns with the true gradient and reverting safely to isotropic zeroth-order descent as alignment deteriorates. An oracle version interpolates between first-order O(1/T) and zeroth-order O(d/T) convergence, while the fully adaptive method matches that guarantee up to logarithmic factors without knowing hint quality in advance, using O(k + log T) additional function queries per iteration. In a fluid inverse problem with dimension 8192, a budget of 16 function evaluations per iteration, and 100 optimization steps, CV-ZOD reached a final objective of 0.0119, compared with 0.0260 for unguided ZOD and 0.0466 for Guided Evolutionary Strategies. The method also performed well in molecular optimization using MACE-OFF23 as the black-box energy model and MMFF94 as the directional surrogate, with NVIDIA Warp providing the differentiable fluid simulator. The central result is robustness: directional hints improve sampling efficiency without distorting optimization toward the surrogate objective.