Robust Online Multi-Task Learning with Correlative and Personalized Structures

Handle URI:
http://hdl.handle.net/10754/625169
Title:
Robust Online Multi-Task Learning with Correlative and Personalized Structures
Authors:
Yang, Peng; Zhao, Peilin; Gao, Xin ( 0000-0002-7108-3574 )
Abstract:
Multi-Task Learning (MTL) can enhance a classifier's generalization performance by learning multiple related tasks simultaneously. Conventional MTL works under the offline setting and suffers from expensive training cost and poor scalability. To address such issues, online learning techniques have been applied to solve MTL problems. However, most existing algorithms of online MTL constrain task relatedness into a presumed structure via a single weight matrix, which is a strict restriction that does not always hold in practice. In this paper, we propose a robust online MTL framework that overcomes this restriction by decomposing the weight matrix into two components: the first one captures the low-rank common structure among tasks via a nuclear norm; the second one identifies the personalized patterns of outlier tasks via a group lasso. Theoretical analysis shows the proposed algorithm can achieve a sub-linear regret with respect to the best linear model in hindsight. However, the nuclear norm that simply adds all nonzero singular values together may not be a good low-rank approximation. To improve the results, we use a log-determinant function as a non-convex rank approximation. Experimental results on a number of real-world applications also verify the efficacy of our approaches.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Citation:
YANG P, Zhao P, Gao X (2017) Robust Online Multi-Task Learning with Correlative and Personalized Structures. IEEE Transactions on Knowledge and Data Engineering: 1–1. Available: http://dx.doi.org/10.1109/TKDE.2017.2703106.
Publisher:
Institute of Electrical and Electronics Engineers (IEEE)
Journal:
IEEE Transactions on Knowledge and Data Engineering
Issue Date:
29-Jun-2017
DOI:
10.1109/TKDE.2017.2703106
Type:
Article
ISSN:
1041-4347
Additional Links:
http://ieeexplore.ieee.org/document/7959634/
Appears in Collections:
Articles; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorYang, Pengen
dc.contributor.authorZhao, Peilinen
dc.contributor.authorGao, Xinen
dc.date.accessioned2017-07-06T09:43:06Z-
dc.date.available2017-07-06T09:43:06Z-
dc.date.issued2017-06-29en
dc.identifier.citationYANG P, Zhao P, Gao X (2017) Robust Online Multi-Task Learning with Correlative and Personalized Structures. IEEE Transactions on Knowledge and Data Engineering: 1–1. Available: http://dx.doi.org/10.1109/TKDE.2017.2703106.en
dc.identifier.issn1041-4347en
dc.identifier.doi10.1109/TKDE.2017.2703106en
dc.identifier.urihttp://hdl.handle.net/10754/625169-
dc.description.abstractMulti-Task Learning (MTL) can enhance a classifier's generalization performance by learning multiple related tasks simultaneously. Conventional MTL works under the offline setting and suffers from expensive training cost and poor scalability. To address such issues, online learning techniques have been applied to solve MTL problems. However, most existing algorithms of online MTL constrain task relatedness into a presumed structure via a single weight matrix, which is a strict restriction that does not always hold in practice. In this paper, we propose a robust online MTL framework that overcomes this restriction by decomposing the weight matrix into two components: the first one captures the low-rank common structure among tasks via a nuclear norm; the second one identifies the personalized patterns of outlier tasks via a group lasso. Theoretical analysis shows the proposed algorithm can achieve a sub-linear regret with respect to the best linear model in hindsight. However, the nuclear norm that simply adds all nonzero singular values together may not be a good low-rank approximation. To improve the results, we use a log-determinant function as a non-convex rank approximation. Experimental results on a number of real-world applications also verify the efficacy of our approaches.en
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)en
dc.relation.urlhttp://ieeexplore.ieee.org/document/7959634/en
dc.rights(c) 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.en
dc.subjectartificial intelligenceen
dc.subjectlearning systemsen
dc.subjectonline learningen
dc.subjectmultitask learningen
dc.subjectclassificationen
dc.titleRobust Online Multi-Task Learning with Correlative and Personalized Structuresen
dc.typeArticleen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.identifier.journalIEEE Transactions on Knowledge and Data Engineeringen
dc.eprint.versionPost-printen
dc.contributor.institutionData Analytics, Institute for Infocomm Research, 68705 Singapore, Singapore Singapore 138632en
dc.contributor.institutionAnt Finance, Alibaba, Hangzhou, Zhejiang Chinaen
kaust.authorYang, Pengen
kaust.authorGao, Xinen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.