Is Attribute-Based Zero-Shot Learning an Ill-Posed Strategy?

Handle URI:
http://hdl.handle.net/10754/622148
Title:
Is Attribute-Based Zero-Shot Learning an Ill-Posed Strategy?
Authors:
Alabdulmohsin, Ibrahim ( 0000-0002-9387-5820 ) ; Cisse, Moustapha; Zhang, Xiangliang ( 0000-0002-3574-5665 )
Abstract:
One transfer learning approach that has gained a wide popularity lately is attribute-based zero-shot learning. Its goal is to learn novel classes that were never seen during the training stage. The classical route towards realizing this goal is to incorporate a prior knowledge, in the form of a semantic embedding of classes, and to learn to predict classes indirectly via their semantic attributes. Despite the amount of research devoted to this subject lately, no known algorithm has yet reported a predictive accuracy that could exceed the accuracy of supervised learning with very few training examples. For instance, the direct attribute prediction (DAP) algorithm, which forms a standard baseline for the task, is known to be as accurate as supervised learning when as few as two examples from each hidden class are used for training on some popular benchmark datasets! In this paper, we argue that this lack of significant results in the literature is not a coincidence; attribute-based zero-shot learning is fundamentally an ill-posed strategy. The key insight is the observation that the mechanical task of predicting an attribute is, in fact, quite different from the epistemological task of learning the “correct meaning” of the attribute itself. This renders attribute-based zero-shot learning fundamentally ill-posed. In more precise mathematical terms, attribute-based zero-shot learning is equivalent to the mirage goal of learning with respect to one distribution of instances, with the hope of being able to predict with respect to any arbitrary distribution. We demonstrate this overlooked fact on some synthetic and real datasets. The data and software related to this paper are available at https://mine. kaust.edu.sa/Pages/zero-shot-learning.aspx. © Springer International Publishing AG 2016.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Citation:
Alabdulmohsin I, Cisse M, Zhang X (2016) Is Attribute-Based Zero-Shot Learning an Ill-Posed Strategy? Lecture Notes in Computer Science: 749–760. Available: http://dx.doi.org/10.1007/978-3-319-46128-1_47.
Publisher:
Springer Nature
Journal:
Machine Learning and Knowledge Discovery in Databases
Conference/Event name:
15th European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2016
Issue Date:
3-Sep-2016
DOI:
10.1007/978-3-319-46128-1_47
Type:
Conference Paper
ISSN:
0302-9743; 1611-3349
Sponsors:
Research reported in this publication was supported by King Abdullah University of Science and Technology (KAUST) and the Saudi Arabian Oil Company (Saudi Aramco).
Additional Links:
http://link.springer.com/chapter/10.1007%2F978-3-319-46128-1_47
Appears in Collections:
Conference Papers; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorAlabdulmohsin, Ibrahimen
dc.contributor.authorCisse, Moustaphaen
dc.contributor.authorZhang, Xiangliangen
dc.date.accessioned2017-01-02T08:10:21Z-
dc.date.available2017-01-02T08:10:21Z-
dc.date.issued2016-09-03en
dc.identifier.citationAlabdulmohsin I, Cisse M, Zhang X (2016) Is Attribute-Based Zero-Shot Learning an Ill-Posed Strategy? Lecture Notes in Computer Science: 749–760. Available: http://dx.doi.org/10.1007/978-3-319-46128-1_47.en
dc.identifier.issn0302-9743en
dc.identifier.issn1611-3349en
dc.identifier.doi10.1007/978-3-319-46128-1_47en
dc.identifier.urihttp://hdl.handle.net/10754/622148-
dc.description.abstractOne transfer learning approach that has gained a wide popularity lately is attribute-based zero-shot learning. Its goal is to learn novel classes that were never seen during the training stage. The classical route towards realizing this goal is to incorporate a prior knowledge, in the form of a semantic embedding of classes, and to learn to predict classes indirectly via their semantic attributes. Despite the amount of research devoted to this subject lately, no known algorithm has yet reported a predictive accuracy that could exceed the accuracy of supervised learning with very few training examples. For instance, the direct attribute prediction (DAP) algorithm, which forms a standard baseline for the task, is known to be as accurate as supervised learning when as few as two examples from each hidden class are used for training on some popular benchmark datasets! In this paper, we argue that this lack of significant results in the literature is not a coincidence; attribute-based zero-shot learning is fundamentally an ill-posed strategy. The key insight is the observation that the mechanical task of predicting an attribute is, in fact, quite different from the epistemological task of learning the “correct meaning” of the attribute itself. This renders attribute-based zero-shot learning fundamentally ill-posed. In more precise mathematical terms, attribute-based zero-shot learning is equivalent to the mirage goal of learning with respect to one distribution of instances, with the hope of being able to predict with respect to any arbitrary distribution. We demonstrate this overlooked fact on some synthetic and real datasets. The data and software related to this paper are available at https://mine. kaust.edu.sa/Pages/zero-shot-learning.aspx. © Springer International Publishing AG 2016.en
dc.description.sponsorshipResearch reported in this publication was supported by King Abdullah University of Science and Technology (KAUST) and the Saudi Arabian Oil Company (Saudi Aramco).en
dc.publisherSpringer Natureen
dc.relation.urlhttp://link.springer.com/chapter/10.1007%2F978-3-319-46128-1_47en
dc.subjectAttribute-based classificationen
dc.subjectMultilabel classificationen
dc.subjectZero-shot learningen
dc.titleIs Attribute-Based Zero-Shot Learning an Ill-Posed Strategy?en
dc.typeConference Paperen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.identifier.journalMachine Learning and Knowledge Discovery in Databasesen
dc.conference.date2016-09-19 to 2016-09-23en
dc.conference.name15th European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2016en
dc.conference.locationRiva del Garda, ITAen
dc.contributor.institutionFacebook Artificial Intelligence Research (FAIR), Menlo Park, United Statesen
kaust.authorAlabdulmohsin, Ibrahimen
kaust.authorZhang, Xiangliangen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.