Show simple item record

dc.contributor.advisorAlouini, Mohamed-Slim
dc.contributor.authorSifaou, Houssem
dc.date.accessioned2021-04-27T09:18:18Z
dc.date.available2021-04-27T09:18:18Z
dc.date.issued2021-04
dc.identifier.citationSifaou, H. (2021). Discriminant Analysis and Support Vector Regression in High Dimensions: Sharp Performance Analysis and Optimal Designs. KAUST Research Repository. https://doi.org/10.25781/KAUST-7TA37
dc.identifier.doi10.25781/KAUST-7TA37
dc.identifier.urihttp://hdl.handle.net/10754/668982
dc.description.abstractMachine learning is emerging as a powerful tool to data science and is being applied in almost all subjects. In many applications, the number of features is com- parable to the number of samples, and both grow large. This setting is usually named the high-dimensional regime. In this regime, new challenges arise when it comes to the application of machine learning. In this work, we conduct a high-dimensional performance analysis of some popular classification and regression techniques. In a first part, discriminant analysis classifiers are considered. A major challenge towards the use of these classifiers in practice is that they depend on the inverse of covariance matrices that need to be estimated from training data. Several estimators for the inverse of the covariance matrices can be used. The most common ones are estimators based on the regularization approach. In this thesis, we propose new estimators that are shown to yield better performance. The main principle of our proposed approach is the design of an optimized inverse covariance matrix estimator based on the assumption that the covariance matrix is a low-rank perturbation of a scaled identity matrix. We show that not only the proposed classifiers are easier to implement but also, outperform the classical regularization-based discriminant analysis classifiers. In a second part, we carry out a high-dimensional statistical analysis of linear support vector regression. Under some plausible assumptions on the statistical dis- tribution of the data, we characterize the feasibility condition for the hard support vector regression and, when feasible, derive an asymptotic approximation for its risk. Similarly, we study the test risk for the soft support vector regression as a function of its parameters. The analysis is then extended to the case of kernel support vector regression under generalized linear models assumption. Based on our analysis, we illustrate that adding more samples may be harmful to the test performance of these regression algorithms, while it is always beneficial when the parameters are optimally selected. Our results pave the way to understand the effect of the underlying hyper- parameters and provide insights on how to optimally choose the kernel function.
dc.language.isoen
dc.subjectHigh dimensions
dc.subjectDiscriminant analysis
dc.subjectSupport vector regression
dc.subjectAsymptotic analysis
dc.titleDiscriminant Analysis and Support Vector Regression in High Dimensions: Sharp Performance Analysis and Optimal Designs
dc.typeDissertation
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.rights.embargodate2022-04-25
thesis.degree.grantorKing Abdullah University of Science and Technology
dc.contributor.committeememberShihada, Basem
dc.contributor.committeememberZhang, Xiangliang
dc.contributor.committeememberKammoun, Abla
dc.contributor.committeememberMcKay, Matthew
thesis.degree.disciplineElectrical Engineering
thesis.degree.nameDoctor of Philosophy
dc.rights.accessrightsAt the time of archiving, the student author of this dissertation opted to temporarily restrict access to it. The full text of this dissertation will become available to the public after the expiration of the embargo on 2022-04-25.
kaust.request.doiyes


Files in this item

Thumbnail
Name:
Final_Dissertation_Houssem_Sifaou.pdf
Size:
1.129Mb
Format:
PDF
Description:
Final Dissertation
Embargo End Date:
2022-04-25

This item appears in the following Collection(s)

Show simple item record