There are some techniques that can be used with simulated annealing to deal with noise in the evaluation of the objective function. See Section 3 of Branke et al (2008) for a quick overview of proposed methods (they also propose new techniques in that paper). Most of these techniques come with the usual convergence guarantees that are associated with simulated annealing (but there are of course performance penalties in dealing with noise).
What is the dimensionality of your parameter space? What do you know about the noise? (e.g., if you know that the noise is mostly homoscedastic or if you can parameterize it, then you can probably use this to push the performance of some of the simulated annealing algorithms.)
The parameter space is only two dimensional here, so it’s not hard to eyeball roughly where the minimum is if I sample enough. I can say very little about the noise. I’m more interested being able to approximate the optimum quickly (since simulation time adds up) than hitting it exactly. The approach taken in this paper based on a non-parametric tau test looks interesting.
There are some techniques that can be used with simulated annealing to deal with noise in the evaluation of the objective function. See Section 3 of Branke et al (2008) for a quick overview of proposed methods (they also propose new techniques in that paper). Most of these techniques come with the usual convergence guarantees that are associated with simulated annealing (but there are of course performance penalties in dealing with noise).
What is the dimensionality of your parameter space? What do you know about the noise? (e.g., if you know that the noise is mostly homoscedastic or if you can parameterize it, then you can probably use this to push the performance of some of the simulated annealing algorithms.)
Thanks for the SA paper!
The parameter space is only two dimensional here, so it’s not hard to eyeball roughly where the minimum is if I sample enough. I can say very little about the noise. I’m more interested being able to approximate the optimum quickly (since simulation time adds up) than hitting it exactly. The approach taken in this paper based on a non-parametric tau test looks interesting.