Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification
Type
Conference PaperKAUST Department
Computational Bioscience Research Center (CBRC)Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Date
2016-01-15Online Publication Date
2016-01-15Print Publication Date
2015-10Permanent link to this record
http://hdl.handle.net/10754/609037
Metadata
Show full item recordAbstract
In this paper we study the problem of learning from multiple modal data for purpose of document classification. In this problem, each document is composed two different modals of data, i.e., An image and a text. Cross-modal factor analysis (CFA) has been proposed to project the two different modals of data to a shared data space, so that the classification of a image or a text can be performed directly in this space. A disadvantage of CFA is that it has ignored the supervision information. In this paper, we improve CFA by incorporating the supervision information to represent and classify both image and text modals of documents. We project both image and text data to a shared data space by factor analysis, and then train a class label predictor in the shared space to use the class label information. The factor analysis parameter and the predictor parameter are learned jointly by solving one single objective function. With this objective function, we minimize the distance between the projections of image and text of the same document, and the classification error of the projection measured by hinge loss function. The objective function is optimized by an alternate optimization strategy in an iterative algorithm. Experiments in two different multiple modal document data sets show the advantage of the proposed algorithm over other CFA methods.Citation
Wang, J., Zhou, Y., Duan, K., Wang, J. J.-Y., & Bensmail, H. (2015). Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification. 2015 IEEE International Conference on Systems, Man, and Cybernetics. doi:10.1109/smc.2015.329Sponsors
The research reported in this publication was supported by competitive research funding from King Abdullah University of Science and Technology (KAUST), Saudi Arabia.Conference/Event name
Systems, Man, and Cybernetics (SMC), 2015 IEEE International Conference onae974a485f413a2113503eed53cd6c53
10.1109/SMC.2015.329