Show simple item record

dc.contributor.authorFattah, Esmail Abdul
dc.contributor.authorNiekerk, Janet Van
dc.contributor.authorRue, Haavard
dc.date.accessioned2021-06-16T06:39:28Z
dc.date.available2021-06-16T06:39:28Z
dc.date.issued2021-06-14
dc.identifier.urihttp://hdl.handle.net/10754/669609
dc.description.abstractComputing the gradient of a function provides fundamental information about its behavior. This information is essential for several applications and algorithms across various fields. One common application that require gradients are optimization techniques such as stochastic gradient descent, Newton's method and trust region methods. However, these methods usually requires a numerical computation of the gradient at every iteration of the method which is prone to numerical errors. We propose a simple limited-memory technique for improving the accuracy of a numerically computed gradient in this gradient-based optimization framework by exploiting (1) a coordinate transformation of the gradient and (2) the history of previously taken descent directions. The method is verified empirically by extensive experimentation on both test functions and on real data applications. The proposed method is implemented in the R package smartGrad and in C++.
dc.publisherarXiv
dc.relation.urlhttps://arxiv.org/pdf/2106.07313.pdf
dc.rightsArchived with thanks to arXiv
dc.subjectAdaptive Technique
dc.subjectGradient Estimation
dc.subjectNumerical Gradient
dc.subjectOptimization
dc.subjectVanilla Gradient Descent
dc.titleSmart Gradient -- An Adaptive Technique for Improving Gradient Estimation
dc.typePreprint
dc.contributor.departmentDepartment of Statistics King Abdullah University of Science and Technology Thuwal, 23955, Makkah
dc.contributor.departmentStatistics Program
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.eprint.versionPre-print
dc.identifier.arxivid2106.07313
kaust.personFattah, Esmail Abdul
kaust.personNiekerk, Janet Van
kaust.personRue, Haavard
refterms.dateFOA2021-06-16T06:40:06Z


Files in this item

Thumbnail
Name:
Preprintfile1.pdf
Size:
989.1Kb
Format:
PDF
Description:
Pre-print

This item appears in the following Collection(s)

Show simple item record