Show simple item record

dc.contributor.authorHuang, Jia-Hong
dc.contributor.authorAlfadly, Modar
dc.contributor.authorGhanem, Bernard
dc.contributor.authorWorring, Marcel
dc.date.accessioned2019-12-22T12:49:11Z
dc.date.available2019-12-22T12:49:11Z
dc.date.issued2019-11-30
dc.identifier.urihttp://hdl.handle.net/10754/660730
dc.description.abstractDeep neural networks have been playing an essential role in the task of Visual Question Answering (VQA). Until recently, their accuracy has been the main focus of research. Now there is a trend toward assessing the robustness of these models against adversarial attacks by evaluating the accuracy of these models under increasing levels of noisiness in the inputs of VQA models. In VQA, the attack can target the image and/or the proposed query question, dubbed main question, and yet there is a lack of proper analysis of this aspect of VQA. In this work, we propose a new method that uses semantically related questions, dubbed basic questions, acting as noise to evaluate the robustness of VQA models. We hypothesize that as the similarity of a basic question to the main question decreases, the level of noise increases. To generate a reasonable noise level for a given main question, we rank a pool of basic questions based on their similarity with this main question. We cast this ranking problem as a LASSO optimization problem. We also propose a novel robustness measure Rscore and two large-scale basic question datasets in order to standardize robustness analysis of VQA models. The experimental results demonstrate that the proposed evaluation method is able to effectively analyze the robustness of VQA models. To foster the VQA research, we will publish our proposed datasets.
dc.publisherarXiv
dc.relation.urlhttps://arxiv.org/pdf/1912.01452
dc.rightsArchived with thanks to arXiv
dc.titleAssessing the Robustness of Visual Question Answering
dc.typePreprint
dc.contributor.departmentEarth Science and Engineering Program
dc.contributor.departmentComputer Science Program
dc.contributor.departmentElectrical Engineering Program
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
dc.eprint.versionPre-print
dc.contributor.institutionUniversiteit van Amsterdam, Amsterdam, the Netherlands
dc.identifier.arxivid1912.01452
kaust.personHuang, Jia-Hong
kaust.personAlfadly, Modar
kaust.personGhanem, Bernard
refterms.dateFOA2019-12-22T12:50:04Z


Files in this item

Thumbnail
Name:
Preprintfile1.pdf
Size:
4.447Mb
Format:
PDF
Description:
Pre-print

This item appears in the following Collection(s)

Show simple item record