Accelerated Optimization in the PDE Framework: Formulations for the Active Contour Case

Handle URI:
http://hdl.handle.net/10754/626461
Title:
Accelerated Optimization in the PDE Framework: Formulations for the Active Contour Case
Authors:
Yezzi, Anthony; Sundaramoorthi, Ganesh ( 0000-0003-3471-6384 )
Abstract:
Following the seminal work of Nesterov, accelerated optimization methods have been used to powerfully boost the performance of first-order, gradient-based parameter estimation in scenarios where second-order optimization strategies are either inapplicable or impractical. Not only does accelerated gradient descent converge considerably faster than traditional gradient descent, but it also performs a more robust local search of the parameter space by initially overshooting and then oscillating back as it settles into a final configuration, thereby selecting only local minimizers with a basis of attraction large enough to contain the initial overshoot. This behavior has made accelerated and stochastic gradient search methods particularly popular within the machine learning community. In their recent PNAS 2016 paper, Wibisono, Wilson, and Jordan demonstrate how a broad class of accelerated schemes can be cast in a variational framework formulated around the Bregman divergence, leading to continuum limit ODE's. We show how their formulation may be further extended to infinite dimension manifolds (starting here with the geometric space of curves and surfaces) by substituting the Bregman divergence with inner products on the tangent space and explicitly introducing a distributed mass model which evolves in conjunction with the object of interest during the optimization process. The co-evolving mass model, which is introduced purely for the sake of endowing the optimization with helpful dynamics, also links the resulting class of accelerated PDE based optimization schemes to fluid dynamical formulations of optimal mass transport.
KAUST Department:
Electrical Engineering Program
Publisher:
arXiv
Issue Date:
27-Nov-2017
ARXIV:
arXiv:1711.09867
Type:
Preprint
Additional Links:
http://arxiv.org/abs/1711.09867v1; http://arxiv.org/pdf/1711.09867v1
Appears in Collections:
Other/General Submission; Other/General Submission; Electrical Engineering Program

Full metadata record

DC FieldValue Language
dc.contributor.authorYezzi, Anthonyen
dc.contributor.authorSundaramoorthi, Ganeshen
dc.date.accessioned2017-12-28T07:32:11Z-
dc.date.available2017-12-28T07:32:11Z-
dc.date.issued2017-11-27en
dc.identifier.urihttp://hdl.handle.net/10754/626461-
dc.description.abstractFollowing the seminal work of Nesterov, accelerated optimization methods have been used to powerfully boost the performance of first-order, gradient-based parameter estimation in scenarios where second-order optimization strategies are either inapplicable or impractical. Not only does accelerated gradient descent converge considerably faster than traditional gradient descent, but it also performs a more robust local search of the parameter space by initially overshooting and then oscillating back as it settles into a final configuration, thereby selecting only local minimizers with a basis of attraction large enough to contain the initial overshoot. This behavior has made accelerated and stochastic gradient search methods particularly popular within the machine learning community. In their recent PNAS 2016 paper, Wibisono, Wilson, and Jordan demonstrate how a broad class of accelerated schemes can be cast in a variational framework formulated around the Bregman divergence, leading to continuum limit ODE's. We show how their formulation may be further extended to infinite dimension manifolds (starting here with the geometric space of curves and surfaces) by substituting the Bregman divergence with inner products on the tangent space and explicitly introducing a distributed mass model which evolves in conjunction with the object of interest during the optimization process. The co-evolving mass model, which is introduced purely for the sake of endowing the optimization with helpful dynamics, also links the resulting class of accelerated PDE based optimization schemes to fluid dynamical formulations of optimal mass transport.en
dc.publisherarXiven
dc.relation.urlhttp://arxiv.org/abs/1711.09867v1en
dc.relation.urlhttp://arxiv.org/pdf/1711.09867v1en
dc.rightsArchived with thanks to arXiven
dc.titleAccelerated Optimization in the PDE Framework: Formulations for the Active Contour Caseen
dc.typePreprinten
dc.contributor.departmentElectrical Engineering Programen
dc.eprint.versionPre-printen
dc.contributor.institutionSchool of Electrical and Computer Engineering, Georgia Institute of Technologyen
dc.identifier.arxividarXiv:1711.09867en
kaust.authorSundaramoorthi, Ganeshen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.