Show simple item record

dc.contributor.authorXu, Zhiqiang
dc.contributor.authorCao, Xin
dc.contributor.authorGao, Xin
dc.date.accessioned2018-09-03T13:18:15Z
dc.date.available2018-09-03T13:18:15Z
dc.date.issued2018-07-05
dc.identifier.citationXu Z, Cao X, Gao X (2018) Convergence Analysis of Gradient Descent for Eigenvector Computation. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. Available: http://dx.doi.org/10.24963/ijcai.2018/407.
dc.identifier.doi10.24963/ijcai.2018/407
dc.identifier.urihttp://hdl.handle.net/10754/628358
dc.description.abstractWe present a novel, simple and systematic convergence analysis of gradient descent for eigenvector computation. As a popular, practical, and provable approach to numerous machine learning problems, gradient descent has found successful applications to eigenvector computation as well. However, surprisingly, it lacks a thorough theoretical analysis for the underlying geodesically non-convex problem. In this work, the convergence of the gradient descent solver for the leading eigenvector computation is shown to be at a global rate O(min{ (lambda_1/Delta_p)^2 log(1/epsilon), 1/epsilon }), where Delta_p=lambda_p-lambda_p+1>0 represents the generalized positive eigengap and always exists without loss of generality with lambda_i being the i-th largest eigenvalue of the given real symmetric matrix and p being the multiplicity of lambda_1. The rate is linear at (lambda_1/Delta_p)^2 log(1/epsilon) if (lambda_1/Delta_p)^2=O(1), otherwise sub-linear at O(1/epsilon). We also show that the convergence only logarithmically instead of quadratically depends on the initial iterate. Particularly, this is the first time the linear convergence for the case that the conventionally considered eigengap Delta_1= lambda_1 - lambda_2=0 but the generalized eigengap Delta_p satisfies (lambda_1/Delta_p)^2=O(1), as well as the logarithmic dependence on the initial iterate are established for the gradient descent solver. We are also the first to leverage for analysis the log principal angle between the iterate and the space of globally optimal solutions. Theoretical properties are verified in experiments.
dc.description.sponsorshipThis research is supported in part by the funding from King Abdullah University of Science and Technology (KAUST).
dc.publisherInternational Joint Conferences on Artificial Intelligence
dc.relation.urlhttps://www.ijcai.org/proceedings/2018/407
dc.rightsArchived with thanks to Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
dc.subjectConstraints and SAT: Constraint Optimisation
dc.subjectHeuristic Search and Game Playing: Combinatorial Search and Optimisation
dc.titleConvergence Analysis of Gradient Descent for Eigenvector Computation
dc.titleConvergence analysis of gradient descent for top-k eigenspace computation
dc.typeConference Paper
dc.contributor.departmentComputational Bioscience Research Center (CBRC)
dc.contributor.departmentComputer Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.identifier.journalProceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
dc.conference.dateJuly 2018
dc.conference.nameThe 27th International Joint Conference on Artificial Intelligence and The 23rd European Conference on Artificial Intelligence (IJCAI-ECAI-18)
dc.conference.locationStockholm, Sweden
dc.eprint.versionPost-print
dc.contributor.institutionUNSW, Australia
kaust.personXu, Zhiqiang
kaust.personGao, Xin
refterms.dateFOA2018-09-12T07:51:23Z
dc.date.published-online2018-07-05
dc.date.published-print2018-07


Files in this item

Thumbnail
Name:
0407.pdf
Size:
231.7Kb
Format:
PDF
Description:
Accepted Manuscript

This item appears in the following Collection(s)

Show simple item record