Handle URI:
http://hdl.handle.net/10754/597653
Title:
Bayesian nonlinear regression for large small problems
Authors:
Chakraborty, Sounak; Ghosh, Malay; Mallick, Bani K.
Abstract:
Statistical modeling and inference problems with sample sizes substantially smaller than the number of available covariates are challenging. This is known as large p small n problem. Furthermore, the problem is more complicated when we have multiple correlated responses. We develop multivariate nonlinear regression models in this setup for accurate prediction. In this paper, we introduce a full Bayesian support vector regression model with Vapnik's ε-insensitive loss function, based on reproducing kernel Hilbert spaces (RKHS) under the multivariate correlated response setup. This provides a full probabilistic description of support vector machine (SVM) rather than an algorithm for fitting purposes. We have also introduced a multivariate version of the relevance vector machine (RVM). Instead of the original treatment of the RVM relying on the use of type II maximum likelihood estimates of the hyper-parameters, we put a prior on the hyper-parameters and use Markov chain Monte Carlo technique for computation. We have also proposed an empirical Bayes method for our RVM and SVM. Our methods are illustrated with a prediction problem in the near-infrared (NIR) spectroscopy. A simulation study is also undertaken to check the prediction accuracy of our models. © 2012 Elsevier Inc.
Citation:
Chakraborty S, Ghosh M, Mallick BK (2012) Bayesian nonlinear regression for large small problems. Journal of Multivariate Analysis 108: 28–40. Available: http://dx.doi.org/10.1016/j.jmva.2012.01.015.
Publisher:
Elsevier BV
Journal:
Journal of Multivariate Analysis
KAUST Grant Number:
KUS-CI-016-04
Issue Date:
Jul-2012
DOI:
10.1016/j.jmva.2012.01.015
Type:
Article
ISSN:
0047-259X
Sponsors:
The research of Bani K Mallick was supported by National Science Foundation grant DMS 0914951 and by award KUS-CI-016-04 made by King Abdullah University of Science and Technology (KAUST).
Appears in Collections:
Publications Acknowledging KAUST Support

Full metadata record

DC FieldValue Language
dc.contributor.authorChakraborty, Sounaken
dc.contributor.authorGhosh, Malayen
dc.contributor.authorMallick, Bani K.en
dc.date.accessioned2016-02-25T12:43:46Zen
dc.date.available2016-02-25T12:43:46Zen
dc.date.issued2012-07en
dc.identifier.citationChakraborty S, Ghosh M, Mallick BK (2012) Bayesian nonlinear regression for large small problems. Journal of Multivariate Analysis 108: 28–40. Available: http://dx.doi.org/10.1016/j.jmva.2012.01.015.en
dc.identifier.issn0047-259Xen
dc.identifier.doi10.1016/j.jmva.2012.01.015en
dc.identifier.urihttp://hdl.handle.net/10754/597653en
dc.description.abstractStatistical modeling and inference problems with sample sizes substantially smaller than the number of available covariates are challenging. This is known as large p small n problem. Furthermore, the problem is more complicated when we have multiple correlated responses. We develop multivariate nonlinear regression models in this setup for accurate prediction. In this paper, we introduce a full Bayesian support vector regression model with Vapnik's ε-insensitive loss function, based on reproducing kernel Hilbert spaces (RKHS) under the multivariate correlated response setup. This provides a full probabilistic description of support vector machine (SVM) rather than an algorithm for fitting purposes. We have also introduced a multivariate version of the relevance vector machine (RVM). Instead of the original treatment of the RVM relying on the use of type II maximum likelihood estimates of the hyper-parameters, we put a prior on the hyper-parameters and use Markov chain Monte Carlo technique for computation. We have also proposed an empirical Bayes method for our RVM and SVM. Our methods are illustrated with a prediction problem in the near-infrared (NIR) spectroscopy. A simulation study is also undertaken to check the prediction accuracy of our models. © 2012 Elsevier Inc.en
dc.description.sponsorshipThe research of Bani K Mallick was supported by National Science Foundation grant DMS 0914951 and by award KUS-CI-016-04 made by King Abdullah University of Science and Technology (KAUST).en
dc.publisherElsevier BVen
dc.subjectBayesian hierarchical modelen
dc.subjectEmpirical Bayesen
dc.subjectGibbs samplingen
dc.subjectMarkov chain Monte Carloen
dc.subjectMetropolis-Hastings algorithmen
dc.subjectNear infrared spectroscopyen
dc.subjectRelevance vector machineen
dc.subjectReproducing kernel Hilbert spaceen
dc.subjectSupport vector machineen
dc.subjectVapnik's ε-insensitive lossen
dc.titleBayesian nonlinear regression for large small problemsen
dc.typeArticleen
dc.identifier.journalJournal of Multivariate Analysisen
dc.contributor.institutionUniversity of Missouri-Columbia, Columbia, United Statesen
dc.contributor.institutionUniversity of Florida, Gainesville, United Statesen
dc.contributor.institutionTexas A and M University, College Station, United Statesen
kaust.grant.numberKUS-CI-016-04en
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.