Simulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Schedule

Handle URI:
http://hdl.handle.net/10754/599621
Title:
Simulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Schedule
Authors:
Liang, Faming; Cheng, Yichen; Lin, Guang
Abstract:
Simulated annealing has been widely used in the solution of optimization problems. As known by many researchers, the global optima cannot be guaranteed to be located by simulated annealing unless a logarithmic cooling schedule is used. However, the logarithmic cooling schedule is so slow that no one can afford to use this much CPU time. This article proposes a new stochastic optimization algorithm, the so-called simulated stochastic approximation annealing algorithm, which is a combination of simulated annealing and the stochastic approximation Monte Carlo algorithm. Under the framework of stochastic approximation, it is shown that the new algorithm can work with a cooling schedule in which the temperature can decrease much faster than in the logarithmic cooling schedule, for example, a square-root cooling schedule, while guaranteeing the global optima to be reached when the temperature tends to zero. The new algorithm has been tested on a few benchmark optimization problems, including feed-forward neural network training and protein-folding. The numerical results indicate that the new algorithm can significantly outperform simulated annealing and other competitors. Supplementary materials for this article are available online.
Citation:
Liang F, Cheng Y, Lin G (2014) Simulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Schedule. Journal of the American Statistical Association 109: 847–863. Available: http://dx.doi.org/10.1080/01621459.2013.872993.
Publisher:
Informa UK Limited
Journal:
Journal of the American Statistical Association
KAUST Grant Number:
KUS-C1-016-04
Issue Date:
3-Apr-2014
DOI:
10.1080/01621459.2013.872993
Type:
Article
ISSN:
0162-1459; 1537-274X
Sponsors:
Faming Liang is Professor (E-mail: fliang@stat.tamu.edu), and Yichen Cheng is Graduate Student (E-mail: ycheng@stat.tamu.edu), Department of Statistics, Texas A&M University, College Station, TX 77843. Guang Lin is Research Scientist, Pacific Northwest National Laboratory, 902 Battelle Boulevard, P.O. Box 999, MSIN K7-90, Richland, WA 99352 (E-mail: guang.lin@pnnl.gov). Liang's research was partially supported by grants from the National Science Foundation (DMS-1106494 and DMS-1317131) and the award (KUS-C1-016-04) made by King Abdullah University of Science and Technology (KAUST). The authors thank the editor, associate editor, and three referees for their constructive comments, which have led to significant improvement of this article.
Appears in Collections:
Publications Acknowledging KAUST Support

Full metadata record

DC FieldValue Language
dc.contributor.authorLiang, Famingen
dc.contributor.authorCheng, Yichenen
dc.contributor.authorLin, Guangen
dc.date.accessioned2016-02-28T06:06:00Zen
dc.date.available2016-02-28T06:06:00Zen
dc.date.issued2014-04-03en
dc.identifier.citationLiang F, Cheng Y, Lin G (2014) Simulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Schedule. Journal of the American Statistical Association 109: 847–863. Available: http://dx.doi.org/10.1080/01621459.2013.872993.en
dc.identifier.issn0162-1459en
dc.identifier.issn1537-274Xen
dc.identifier.doi10.1080/01621459.2013.872993en
dc.identifier.urihttp://hdl.handle.net/10754/599621en
dc.description.abstractSimulated annealing has been widely used in the solution of optimization problems. As known by many researchers, the global optima cannot be guaranteed to be located by simulated annealing unless a logarithmic cooling schedule is used. However, the logarithmic cooling schedule is so slow that no one can afford to use this much CPU time. This article proposes a new stochastic optimization algorithm, the so-called simulated stochastic approximation annealing algorithm, which is a combination of simulated annealing and the stochastic approximation Monte Carlo algorithm. Under the framework of stochastic approximation, it is shown that the new algorithm can work with a cooling schedule in which the temperature can decrease much faster than in the logarithmic cooling schedule, for example, a square-root cooling schedule, while guaranteeing the global optima to be reached when the temperature tends to zero. The new algorithm has been tested on a few benchmark optimization problems, including feed-forward neural network training and protein-folding. The numerical results indicate that the new algorithm can significantly outperform simulated annealing and other competitors. Supplementary materials for this article are available online.en
dc.description.sponsorshipFaming Liang is Professor (E-mail: fliang@stat.tamu.edu), and Yichen Cheng is Graduate Student (E-mail: ycheng@stat.tamu.edu), Department of Statistics, Texas A&M University, College Station, TX 77843. Guang Lin is Research Scientist, Pacific Northwest National Laboratory, 902 Battelle Boulevard, P.O. Box 999, MSIN K7-90, Richland, WA 99352 (E-mail: guang.lin@pnnl.gov). Liang's research was partially supported by grants from the National Science Foundation (DMS-1106494 and DMS-1317131) and the award (KUS-C1-016-04) made by King Abdullah University of Science and Technology (KAUST). The authors thank the editor, associate editor, and three referees for their constructive comments, which have led to significant improvement of this article.en
dc.publisherInforma UK Limiteden
dc.titleSimulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Scheduleen
dc.typeArticleen
dc.identifier.journalJournal of the American Statistical Associationen
dc.contributor.institutionDepartment of Statistics, Texas A&M University, College Station, TX 77843en
dc.contributor.institutionPacific Northwest National Laboratory, 902 Battelle Boulevard, P.O. Box 999, MSIN K7-90, Richland, WA 99352en
kaust.grant.numberKUS-C1-016-04en
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.