Show simple item record

dc.contributor.authorHuang, Jia-Hong
dc.contributor.authorDao, Cuong Duc
dc.contributor.authorAlfadly, Modar
dc.contributor.authorGhanem, Bernard
dc.date.accessioned2020-03-23T13:14:42Z
dc.date.available2020-03-23T13:14:42Z
dc.date.issued2019-07-17
dc.identifier.citationHuang, J.-H., Dao, C. D., Alfadly, M., & Ghanem, B. (2019). A Novel Framework for Robustness Analysis of Visual QA Models. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 8449–8456. doi:10.1609/aaai.v33i01.33018449
dc.identifier.doi10.1609/aaai.v33i01.33018449
dc.identifier.urihttp://hdl.handle.net/10754/662272
dc.description.abstractDeep neural networks have been playing an essential role in many computer vision tasks including Visual Question Answering (VQA). Until recently, the study of their accuracy was the main focus of research but now there is a trend toward assessing the robustness of these models against adversarial attacks by evaluating their tolerance to varying noise levels. In VQA, adversarial attacks can target the image and/or the proposed main question and yet there is a lack of proper analysis of the later. In this work, we propose a flexible framework that focuses on the language part of VQA that uses semantically relevant questions, dubbed basic questions, acting as controllable noise to evaluate the robustness of VQA models. We hypothesize that the level of noise is negatively correlated to the similarity of a basic question to the main question. Hence, to apply noise on any given main question, we rank a pool of basic questions based on their similarity by casting this ranking task as a LASSO optimization problem. Then, we propose a novel robustness measure Rscore and two largescale basic question datasets (BQDs) in order to standardize robustness analysis for VQA models.
dc.description.sponsorshipThis work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research and used the resources of the Supercomputing Laboratory at KAUST in Thuwal, Saudi Arabia.
dc.publisherAssociation for the Advancement of Artificial Intelligence (AAAI)
dc.relation.urlhttp://aaai.org/ojs/index.php/AAAI/article/view/4861
dc.rightsArchived with thanks to Proceedings of the AAAI Conference on Artificial Intelligence
dc.titleA Novel Framework for Robustness Analysis of Visual QA Models
dc.typeConference Paper
dc.contributor.departmentComputer Science Program
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.contributor.departmentEarth Science and Engineering
dc.contributor.departmentEarth Science and Engineering Program
dc.contributor.departmentElectrical Engineering Program
dc.contributor.departmentPhysical Science and Engineering (PSE) Division
dc.contributor.departmentVCC Analytics Research Group
dc.identifier.journalProceedings of the AAAI Conference on Artificial Intelligence
dc.conference.nameAAAI Conference on Artificial Intelligence
dc.eprint.versionPost-print
dc.contributor.institutionNational Taiwan University
dc.identifier.arxivid1711.06232
kaust.personHuang, Jia-Hong
kaust.personDao, Cuong Duc
kaust.personAlfadly, Modar
kaust.personGhanem, Bernard
refterms.dateFOA2020-03-23T13:16:12Z
kaust.acknowledged.supportUnitOffice of Sponsored Research
kaust.acknowledged.supportUnitSupercomputing Laboratory at KAUST


Files in this item

Thumbnail
Name:
VQA_Robustness_AAAI2019.pdf
Size:
1.148Mb
Format:
PDF
Description:
Accepted Manuscript

This item appears in the following Collection(s)

Show simple item record