Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications

Handle URI:
http://hdl.handle.net/10754/627416
Title:
Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications
Authors:
Müller, Matthias ( 0000-0001-5249-8734 ) ; Casser, Vincent; Lahoud, Jean; Smith, Neil; Ghanem, Bernard ( 0000-0002-5534-587X )
Abstract:
We present a photo-realistic training and evaluation simulator (Sim4CV) (http://www.sim4cv.org) with extensive applications across various fields of computer vision. Built on top of the Unreal Engine, the simulator integrates full featured physics based cars, unmanned aerial vehicles (UAVs), and animated human actors in diverse urban and suburban 3D environments. We demonstrate the versatility of the simulator with two case studies: autonomous UAV-based tracking of moving objects and autonomous driving using supervised learning. The simulator fully integrates both several state-of-the-art tracking algorithms with a benchmark evaluation tool and a deep neural network architecture for training vehicles to drive autonomously. It generates synthetic photo-realistic datasets with automatic ground truth annotations to easily extend existing real-world datasets and provides extensive synthetic data variety through its ability to reconfigure synthetic worlds on the fly using an automatic world generation tool.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division; Electrical Engineering Program; Visual Computing Center (VCC)
Citation:
Müller M, Casser V, Lahoud J, Smith N, Ghanem B (2018) Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications. International Journal of Computer Vision. Available: http://dx.doi.org/10.1007/s11263-018-1073-7.
Publisher:
Springer Nature
Journal:
International Journal of Computer Vision
Issue Date:
24-Mar-2018
DOI:
10.1007/s11263-018-1073-7
Type:
Article
ISSN:
0920-5691; 1573-1405
Sponsors:
This work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research through the VCC funding.
Additional Links:
http://link.springer.com/article/10.1007/s11263-018-1073-7
Appears in Collections:
Articles; Electrical Engineering Program; Visual Computing Center (VCC); Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorMüller, Matthiasen
dc.contributor.authorCasser, Vincenten
dc.contributor.authorLahoud, Jeanen
dc.contributor.authorSmith, Neilen
dc.contributor.authorGhanem, Bernarden
dc.date.accessioned2018-04-08T07:45:41Z-
dc.date.available2018-04-08T07:45:41Z-
dc.date.issued2018-03-24en
dc.identifier.citationMüller M, Casser V, Lahoud J, Smith N, Ghanem B (2018) Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications. International Journal of Computer Vision. Available: http://dx.doi.org/10.1007/s11263-018-1073-7.en
dc.identifier.issn0920-5691en
dc.identifier.issn1573-1405en
dc.identifier.doi10.1007/s11263-018-1073-7en
dc.identifier.urihttp://hdl.handle.net/10754/627416-
dc.description.abstractWe present a photo-realistic training and evaluation simulator (Sim4CV) (http://www.sim4cv.org) with extensive applications across various fields of computer vision. Built on top of the Unreal Engine, the simulator integrates full featured physics based cars, unmanned aerial vehicles (UAVs), and animated human actors in diverse urban and suburban 3D environments. We demonstrate the versatility of the simulator with two case studies: autonomous UAV-based tracking of moving objects and autonomous driving using supervised learning. The simulator fully integrates both several state-of-the-art tracking algorithms with a benchmark evaluation tool and a deep neural network architecture for training vehicles to drive autonomously. It generates synthetic photo-realistic datasets with automatic ground truth annotations to easily extend existing real-world datasets and provides extensive synthetic data variety through its ability to reconfigure synthetic worlds on the fly using an automatic world generation tool.en
dc.description.sponsorshipThis work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research through the VCC funding.en
dc.publisherSpringer Natureen
dc.relation.urlhttp://link.springer.com/article/10.1007/s11263-018-1073-7en
dc.rightsThe final publication is available at Springer via http://dx.doi.org/10.1007/s11263-018-1073-7en
dc.subjectSimulatoren
dc.subjectUnreal Engine 4en
dc.subjectObject trackingen
dc.subjectAutonomous drivingen
dc.subjectDeep learningen
dc.subjectImitation learningen
dc.titleSim4CV: A Photo-Realistic Simulator for Computer Vision Applicationsen
dc.typeArticleen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.contributor.departmentElectrical Engineering Programen
dc.contributor.departmentVisual Computing Center (VCC)en
dc.identifier.journalInternational Journal of Computer Visionen
dc.eprint.versionPost-printen
kaust.authorMüller, Matthiasen
kaust.authorCasser, Vincenten
kaust.authorLahoud, Jeanen
kaust.authorSmith, Neilen
kaust.authorGhanem, Bernarden
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.