Noisyopt: A Python library for optimizing noisy functions.

Summary

Optimization problems have great practical importance across many fields. Sometimes a precise evaluation of the function to be optimized is either impossible or exceedingly computationally expensive. An example of the former case is optimization based on a complex simulation, an example of the latter arises in machine learning where evaluating a loss function over the complete data set can be to expensive. Optimization based on noisy evaluations of a function is thus an important problem.

Noisyopt provides optimization algorithms tailored to noisy problems with a call syntax compatible with scipy.optimize (Jones et al. 2001–2001--) routines. It implements an derivative-free algorithm robust to noise -- adaptive pattern search (Mayer et al. 2016), and a stochastic approximation algorithm using simultaneous perturbations (Spall 1998). Bound constraints on variables are supported. The library also has methods for finding a root of a noisy function by an adaptive bisection algorithm.

References

Jones, Eric, Travis Oliphant, Pearu Peterson, and others. 2001–2001--. “SciPy: Open Source Scientific Tools for Python.” http://www.scipy.org/.

Mayer, Andreas, Thierry Mora, Olivier Rivoire, and Aleksandra M Walczak. 2016. “Diversity of immune strategies explained by adaptation to pathogen statistics.” Proceedings of the National Academy of Sciences 113 (31): 8630–5.

Spall, James C. 1998. “Implementation of the Simulteaneous Perturbation Algorithm for Stochastic Optimization.” IEEE Transactions on Aerospace and Electronic Systems 34 (3): 817–23.