Stochastic Three Points Method for Unconstrained Smooth Minimization

Abstract

In this paper we consider the unconstrained minimization problem of a smoothfunction in ${\mathbb{R}}^n$ in a setting where only function evaluations arepossible. We design a novel randomized derivative-free algorithm --- thestochastic three points (STP) method --- and analyze its iteration complexity.At each iteration, STP generates a random search direction according to acertain fixed probability law. Our assumptions on this law are very mild:roughly speaking, all laws which do not concentrate all measure on anyhalfspace passing through the origin will work. For instance, we allow for theuniform distribution on the sphere and also distributions that concentrate allmeasure on a positive spanning set. Given a current iterate $x$, STP compares the objective function at threepoints: $x$, $x+\alpha s$ and $x-\alpha s$, where $\alpha>0$ is a stepsizeparameter and $s$ is the random search direction. The best of these threepoints is the next iterate. We analyze the method STP under several stepsizeselection schemes (fixed, decreasing, estimated through finite differences,etc). We study non-convex, convex and strongly convex cases.