Near-optimal parameter selection methods for l<inf>2</inf> regularization
KAUST DepartmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Electrical Engineering Program
Online Publication Date2018-03-12
Print Publication Date2017-11
Permanent link to this recordhttp://hdl.handle.net/10754/630488
MetadataShow full item record
AbstractThis paper focuses on the problem of selecting the regularization parameter for linear least-squares estimation. Usually, the problem is formulated as a minimization problem with a cost function consisting of the square sum of the l norm of the residual error, plus a penalty term of the squared norm of the solution multiplied by a constant. The penalty term has the effect of shrinking the solution towards the origin with magnitude that depends on the value of the penalty constant. By considering both squared and non-squared norms of the residual error and the solution, four different cost functions can be formed to achieve the same goal. In this paper, we show that all the four cost functions lead to the same closed-form solution involving a regularization parameter, which is related to the penalty constant through a different constraint equation for each cost function. We show that for three of the cost functions, a specific procedure can be applied to combine the constraint equation with the mean squared error (MSE) criterion to develop approximately optimal regularization parameter selection algorithms. Performance of the developed algorithms is compared to existing methods to show that the proposed algorithms stay closest to the optimal MSE.
CitationBallal T, Suliman M, Al-Naffouri TY (2017) Near-optimal parameter selection methods for l<inf>2</inf> regularization. 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP). Available: http://dx.doi.org/10.1109/GlobalSIP.2017.8309170.
Conference/Event name5th IEEE Global Conference on Signal and Information Processing, GlobalSIP 2017