Handle URI:
http://hdl.handle.net/10754/597780
Title:
Classification algorithms using adaptive partitioning
Authors:
Binev, Peter; Cohen, Albert; Dahmen, Wolfgang; DeVore, Ronald
Abstract:
© 2014 Institute of Mathematical Statistics. Algorithms for binary classification based on adaptive tree partitioning are formulated and analyzed for both their risk performance and their friendliness to numerical implementation. The algorithms can be viewed as generating a set approximation to the Bayes set and thus fall into the general category of set estimators. In contrast with the most studied tree-based algorithms, which utilize piecewise constant approximation on the generated partition [IEEE Trans. Inform. Theory 52 (2006) 1335.1353; Mach. Learn. 66 (2007) 209.242], we consider decorated trees, which allow us to derive higher order methods. Convergence rates for these methods are derived in terms the parameter - of margin conditions and a rate s of best approximation of the Bayes set by decorated adaptive partitions. They can also be expressed in terms of the Besov smoothness β of the regression function that governs its approximability by piecewise polynomials on adaptive partition. The execution of the algorithms does not require knowledge of the smoothness or margin conditions. Besov smoothness conditions are weaker than the commonly used Holder conditions, which govern approximation by nonadaptive partitions, and therefore for a given regression function can result in a higher rate of convergence. This in turn mitigates the compatibility conflict between smoothness and margin parameters.
Citation:
Binev P, Cohen A, Dahmen W, DeVore R (2014) Classification algorithms using adaptive partitioning. The Annals of Statistics 42: 2141–2163. Available: http://dx.doi.org/10.1214/14-AOS1234.
Publisher:
Institute of Mathematical Statistics
Journal:
The Annals of Statistics
KAUST Grant Number:
KUS-C1-016-04
Issue Date:
Dec-2014
DOI:
10.1214/14-AOS1234
Type:
Article
ISSN:
0090-5364
Sponsors:
Supported by the Office of Naval Research Contracts ONR-N00014-08-1-1113, ONR-N0001409-1-0107; the AFOSR Contract FA95500910500; the ARO/DoD Contract W911NF-07-1-0185; the NSF Grants DMS-09-15231, DMS-12-22390 and DMS-09-15104; the Special Priority Program SPP 1324, funded by DFG; the French German PROCOPE contract 11418YB; the Agence Nationale de la Recherche (ANR) project ECHANGE (ANR-08-EMER-006); the excellence chair of the Foundation "Sciences Mathematiques de Paris" held by Ronald DeVore. This publication is based on work supported by Award No. KUS-C1-016-04, made by King Abdullah University of Science and Technology (KAUST).
Appears in Collections:
Publications Acknowledging KAUST Support

Full metadata record

DC FieldValue Language
dc.contributor.authorBinev, Peteren
dc.contributor.authorCohen, Alberten
dc.contributor.authorDahmen, Wolfgangen
dc.contributor.authorDeVore, Ronalden
dc.date.accessioned2016-02-25T12:56:35Zen
dc.date.available2016-02-25T12:56:35Zen
dc.date.issued2014-12en
dc.identifier.citationBinev P, Cohen A, Dahmen W, DeVore R (2014) Classification algorithms using adaptive partitioning. The Annals of Statistics 42: 2141–2163. Available: http://dx.doi.org/10.1214/14-AOS1234.en
dc.identifier.issn0090-5364en
dc.identifier.doi10.1214/14-AOS1234en
dc.identifier.urihttp://hdl.handle.net/10754/597780en
dc.description.abstract© 2014 Institute of Mathematical Statistics. Algorithms for binary classification based on adaptive tree partitioning are formulated and analyzed for both their risk performance and their friendliness to numerical implementation. The algorithms can be viewed as generating a set approximation to the Bayes set and thus fall into the general category of set estimators. In contrast with the most studied tree-based algorithms, which utilize piecewise constant approximation on the generated partition [IEEE Trans. Inform. Theory 52 (2006) 1335.1353; Mach. Learn. 66 (2007) 209.242], we consider decorated trees, which allow us to derive higher order methods. Convergence rates for these methods are derived in terms the parameter - of margin conditions and a rate s of best approximation of the Bayes set by decorated adaptive partitions. They can also be expressed in terms of the Besov smoothness β of the regression function that governs its approximability by piecewise polynomials on adaptive partition. The execution of the algorithms does not require knowledge of the smoothness or margin conditions. Besov smoothness conditions are weaker than the commonly used Holder conditions, which govern approximation by nonadaptive partitions, and therefore for a given regression function can result in a higher rate of convergence. This in turn mitigates the compatibility conflict between smoothness and margin parameters.en
dc.description.sponsorshipSupported by the Office of Naval Research Contracts ONR-N00014-08-1-1113, ONR-N0001409-1-0107; the AFOSR Contract FA95500910500; the ARO/DoD Contract W911NF-07-1-0185; the NSF Grants DMS-09-15231, DMS-12-22390 and DMS-09-15104; the Special Priority Program SPP 1324, funded by DFG; the French German PROCOPE contract 11418YB; the Agence Nationale de la Recherche (ANR) project ECHANGE (ANR-08-EMER-006); the excellence chair of the Foundation "Sciences Mathematiques de Paris" held by Ronald DeVore. This publication is based on work supported by Award No. KUS-C1-016-04, made by King Abdullah University of Science and Technology (KAUST).en
dc.publisherInstitute of Mathematical Statisticsen
dc.subjectAdaptive methodsen
dc.subjectBinary classificationen
dc.subjectSet estimatorsen
dc.subjectTree-based algorithmsen
dc.titleClassification algorithms using adaptive partitioningen
dc.typeArticleen
dc.identifier.journalThe Annals of Statisticsen
dc.contributor.institutionUniversity of South Carolina, Columbia, United Statesen
dc.contributor.institutionUniversite Pierre et Marie Curie, Paris, Franceen
dc.contributor.institutionRheinisch-Westfalische Technische Hochschule Aachen, Aachen, Germanyen
dc.contributor.institutionTexas A and M University, College Station, United Statesen
kaust.grant.numberKUS-C1-016-04en
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.