Name:
VQA_Robustness_AAAI2019.pdf
Size:
1.148Mb
Format:
PDF
Description:
Accepted Manuscript
Type
Conference PaperKAUST Department
Computer Science ProgramComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Earth Science and Engineering
Earth Science and Engineering Program
Electrical Engineering Program
Physical Science and Engineering (PSE) Division
VCC Analytics Research Group
Date
2019-07-17Permanent link to this record
http://hdl.handle.net/10754/662272
Metadata
Show full item recordAbstract
Deep neural networks have been playing an essential role in many computer vision tasks including Visual Question Answering (VQA). Until recently, the study of their accuracy was the main focus of research but now there is a trend toward assessing the robustness of these models against adversarial attacks by evaluating their tolerance to varying noise levels. In VQA, adversarial attacks can target the image and/or the proposed main question and yet there is a lack of proper analysis of the later. In this work, we propose a flexible framework that focuses on the language part of VQA that uses semantically relevant questions, dubbed basic questions, acting as controllable noise to evaluate the robustness of VQA models. We hypothesize that the level of noise is negatively correlated to the similarity of a basic question to the main question. Hence, to apply noise on any given main question, we rank a pool of basic questions based on their similarity by casting this ranking task as a LASSO optimization problem. Then, we propose a novel robustness measure Rscore and two largescale basic question datasets (BQDs) in order to standardize robustness analysis for VQA models.Citation
Huang, J.-H., Dao, C. D., Alfadly, M., & Ghanem, B. (2019). A Novel Framework for Robustness Analysis of Visual QA Models. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 8449–8456. doi:10.1609/aaai.v33i01.33018449Sponsors
This work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research and used the resources of the Supercomputing Laboratory at KAUST in Thuwal, Saudi Arabia.Conference/Event name
AAAI Conference on Artificial IntelligencearXiv
1711.06232Additional Links
http://aaai.org/ojs/index.php/AAAI/article/view/4861ae974a485f413a2113503eed53cd6c53
10.1609/aaai.v33i01.33018449