KAUST Grant NumberKUS-CI-016-04
MetadataShow full item record
Abstract© 2015, © American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America. Recent years have seen active developments of various penalized regression methods, such as LASSO and elastic net, to analyze high-dimensional data. In these approaches, the direction and length of the regression coefficients are determined simultaneously. Due to the introduction of penalties, the length of the estimates can be far from being optimal for accurate predictions. We introduce a new framework, regression by projection, and its sparse version to analyze high-dimensional data. The unique nature of this framework is that the directions of the regression coefficients are inferred first, and the lengths and the tuning parameters are determined by a cross-validation procedure to achieve the largest prediction accuracy. We provide a theoretical result for simultaneous model selection consistency and parameter estimation consistency of our method in high dimension. This new framework is then generalized such that it can be applied to principal components analysis, partial least squares, and canonical correlation analysis. We also adapt this framework for discriminant analysis. Compared with the existing methods, where there is relatively little control of the dependency among the sparse components, our method can control the relationships among the components. We present efficient algorithms and related theory for solving the sparse regression by projection problem. Based on extensive simulations and real data analysis, we demonstrate that our method achieves good predictive performance and variable selection in the regression setting, and the ability to control relationships between the sparse components leads to more accurate classification. In supplementary materials available online, the details of the algorithms and theoretical proofs, and R codes for all simulation studies are provided.
CitationQi X, Luo R, Carroll RJ, Zhao H (2015) Sparse Regression by Projection and Sparse Discriminant Analysis. Journal of Computational and Graphical Statistics 24: 416–438. Available: http://dx.doi.org/10.1080/10618600.2014.907094.
SponsorsCarroll’s research was supported by a grant from the National Cancer Institute (R37-CA057030). This publication is based in part on work supported by Award Number KUS-CI-016-04, made by King Abdullah University of Science and Technology (KAUST). Zhao’s research was supported in part by NIH R01 GM59507, P01 CA154295, and NSF DMS 1106738.
PublisherInforma UK Limited
PubMed Central IDPMC4560121
CollectionsPublications Acknowledging KAUST Support
- Regularized estimation of large-scale gene association networks using graphical Gaussian models.
- Authors: Krämer N, Schäfer J, Boulesteix AL
- Issue date: 2009 Nov 24
- Sparse partial least squares regression for simultaneous dimension reduction and variable selection.
- Authors: Chun H, Keleş S
- Issue date: 2010 Jan
- Elastic SCAD as a novel penalization method for SVM classification tasks in high-dimensional data.
- Authors: Becker N, Toedt G, Lichter P, Benner A
- Issue date: 2011 May 9
- Applying stability selection to consistently estimate sparse principal components in high-dimensional molecular data.
- Authors: Sill M, Saadati M, Benner A
- Issue date: 2015 Aug 15
- A flexible framework for sparse simultaneous component based data integration.
- Authors: Van Deun K, Wilderjans TF, van den Berg RA, Antoniadis A, Van Mechelen I
- Issue date: 2011 Nov 15