Risk Convergence of Centered Kernel Ridge Regression with Large Dimensional Data
Type
Conference PaperKAUST Department
Communication Theory LabComputer Science Program
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Electrical Engineering Program
Machine Intelligence & kNowledge Engineering Lab
Date
2020-04-09Preprint Posting Date
2019-04-19Online Publication Date
2020-04-09Print Publication Date
2020-05Permanent link to this record
http://hdl.handle.net/10754/660647
Metadata
Show full item recordAbstract
This paper carries out a large dimensional analysis of a variation of kernel ridge regression that we call centered kernel ridge regression (CKRR), also known in the literature as kernel ridge regression with offset. This modified technique is obtained by accounting for the bias in the regression problem resulting in the old kernel ridge regression but with centered kernels. The analysis is carried out under the assumption that the data is drawn from a Gaussian distribution and heavily relies on tools from random matrix theory (RMT). Under the regime in which the data dimension and the training size grow infinitely large with fixed ratio and under some mild assumptions controlling the data statistics, we show that both the empirical and the prediction risks converge to a deterministic quantities that describe in closed form fashion the performance of CKRR in terms of the data statistics and dimensions. A key insight of the proposed analysis is the fact that asymptotically a large class of kernels achieve the same minimum prediction risk. This insight is validated with synthetic data.Citation
Elkhalil, K., Kammoun, A., Zhang, X., Alouini, M.-S., & Al-Naffouri, T. Y. (2020). Risk Convergence of Centered Kernel Ridge Regression with Large Dimensional Data. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). doi:10.1109/icassp40776.2020.9053349Publisher
IEEEConference/Event name
ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)ISBN
978-1-5090-6632-2arXiv
1904.09212Additional Links
https://ieeexplore.ieee.org/document/9053349/https://ieeexplore.ieee.org/document/9053349/
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9053349
http://arxiv.org/pdf/1904.09212
ae974a485f413a2113503eed53cd6c53
10.1109/ICASSP40776.2020.9053349