KAUST Grant NumberKUS-CI-016-04
Online Publication Date2015-06-16
Print Publication Date2015-04-03
Permanent link to this recordhttp://hdl.handle.net/10754/599682
MetadataShow full item record
Abstract© 2015, © American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America. Recent years have seen active developments of various penalized regression methods, such as LASSO and elastic net, to analyze high-dimensional data. In these approaches, the direction and length of the regression coefficients are determined simultaneously. Due to the introduction of penalties, the length of the estimates can be far from being optimal for accurate predictions. We introduce a new framework, regression by projection, and its sparse version to analyze high-dimensional data. The unique nature of this framework is that the directions of the regression coefficients are inferred first, and the lengths and the tuning parameters are determined by a cross-validation procedure to achieve the largest prediction accuracy. We provide a theoretical result for simultaneous model selection consistency and parameter estimation consistency of our method in high dimension. This new framework is then generalized such that it can be applied to principal components analysis, partial least squares, and canonical correlation analysis. We also adapt this framework for discriminant analysis. Compared with the existing methods, where there is relatively little control of the dependency among the sparse components, our method can control the relationships among the components. We present efficient algorithms and related theory for solving the sparse regression by projection problem. Based on extensive simulations and real data analysis, we demonstrate that our method achieves good predictive performance and variable selection in the regression setting, and the ability to control relationships between the sparse components leads to more accurate classification. In supplementary materials available online, the details of the algorithms and theoretical proofs, and R codes for all simulation studies are provided.
CitationQi X, Luo R, Carroll RJ, Zhao H (2015) Sparse Regression by Projection and Sparse Discriminant Analysis. Journal of Computational and Graphical Statistics 24: 416–438. Available: http://dx.doi.org/10.1080/10618600.2014.907094.
SponsorsCarroll’s research was supported by a grant from the National Cancer Institute (R37-CA057030). This publication is based in part on work supported by Award Number KUS-CI-016-04, made by King Abdullah University of Science and Technology (KAUST). Zhao’s research was supported in part by NIH R01 GM59507, P01 CA154295, and NSF DMS 1106738.
PublisherInforma UK Limited
PubMed Central IDPMC4560121
CollectionsPublications Acknowledging KAUST Support
- Benefits of dimension reduction in penalized regression methods for high-dimensional grouped data: a case study in low sample size.
- Authors: Ajana S, Acar N, Bretillon L, Hejblum BP, Jacqmin-Gadda H, Delcourt C, BLISAR Study Group
- Issue date: 2019 Oct 1
- Confidence Intervals for Sparse Penalized Regression with Random Designs.
- Authors: Yu G, Yin L, Lu S, Liu Y
- Issue date: 2020
- Sparse partial least squares regression for simultaneous dimension reduction and variable selection.
- Authors: Chun H, Keleş S
- Issue date: 2010 Jan
- Fitting and Cross-Validating Cox Models to Censored Big Data With Missing Values Using Extensions of Partial Least Squares Regression Models.
- Authors: Bertrand F, Maumy-Bertrand M
- Issue date: 2021
- An iterative penalized least squares approach to sparse canonical correlation analysis.
- Authors: Mai Q, Zhang X
- Issue date: 2019 Sep