Network Monitoring as a Streaming Analytics Problem

Handle URI:
http://hdl.handle.net/10754/622573
Title:
Network Monitoring as a Streaming Analytics Problem
Authors:
Gupta, Arpit; Birkner, Rüdiger; Canini, Marco ( 0000-0002-5051-4283 ) ; Feamster, Nick; Mac-Stoker, Chris; Willinger, Walter
Abstract:
Programmable switches make it easier to perform flexible network monitoring queries at line rate, and scalable stream processors make it possible to fuse data streams to answer more sophisticated queries about the network in real-time. Unfortunately, processing such network monitoring queries at high traffic rates requires both the switches and the stream processors to filter the traffic iteratively and adaptively so as to extract only that traffic that is of interest to the query at hand. Others have network monitoring in the context of streaming; yet, previous work has not closed the loop in a way that allows network operators to perform streaming analytics for network monitoring applications at scale. To achieve this objective, Sonata allows operators to express a network monitoring query by considering each packet as a tuple and efficiently partitioning each query between the switches and the stream processor through iterative refinement. Sonata extracts only the traffic that pertains to each query, ensuring that the stream processor can scale traffic rates of several terabits per second. We show with a simple example query involving DNS reflection attacks and traffic traces from one of the world's largest IXPs that Sonata can capture 95% of all traffic pertaining to the query, while reducing the overall data rate by a factor of about 400 and the number of required counters by four orders of magnitude. Copyright 2016 ACM.
KAUST Department:
KAUST
Citation:
Gupta A, Birkner R, Canini M, Feamster N, Mac-Stoker C, et al. (2016) Network Monitoring as a Streaming Analytics Problem. Proceedings of the 15th ACM Workshop on Hot Topics in Networks - HotNets ’16. Available: http://dx.doi.org/10.1145/3005745.3005748.
Publisher:
Association for Computing Machinery (ACM)
Journal:
Proceedings of the 15th ACM Workshop on Hot Topics in Networks - HotNets '16
Conference/Event name:
15th ACM Workshop on Hot Topics in Networks, HotNets 2016
Issue Date:
2-Nov-2016
DOI:
10.1145/3005745.3005748
Type:
Conference Paper
Sponsors:
We thank our shepherd, Fadel Adib, the anonymous reviewers, Srinivas Narayana, Ankita Pawar, Rick Porter, Jennifer Rexford for for feedback and comments. This research was supported by National Science Foundation Awards CNS-1539920, and by European Union’s Horizon 2020 program under the ENDEAVOUR project (grant agree- ment 644960).
Additional Links:
http://dl.acm.org/citation.cfm?doid=3005745.3005748
Appears in Collections:
Conference Papers

Full metadata record

DC FieldValue Language
dc.contributor.authorGupta, Arpiten
dc.contributor.authorBirkner, Rüdigeren
dc.contributor.authorCanini, Marcoen
dc.contributor.authorFeamster, Nicken
dc.contributor.authorMac-Stoker, Chrisen
dc.contributor.authorWillinger, Walteren
dc.date.accessioned2017-01-02T09:55:31Z-
dc.date.available2017-01-02T09:55:31Z-
dc.date.issued2016-11-02en
dc.identifier.citationGupta A, Birkner R, Canini M, Feamster N, Mac-Stoker C, et al. (2016) Network Monitoring as a Streaming Analytics Problem. Proceedings of the 15th ACM Workshop on Hot Topics in Networks - HotNets ’16. Available: http://dx.doi.org/10.1145/3005745.3005748.en
dc.identifier.doi10.1145/3005745.3005748en
dc.identifier.urihttp://hdl.handle.net/10754/622573-
dc.description.abstractProgrammable switches make it easier to perform flexible network monitoring queries at line rate, and scalable stream processors make it possible to fuse data streams to answer more sophisticated queries about the network in real-time. Unfortunately, processing such network monitoring queries at high traffic rates requires both the switches and the stream processors to filter the traffic iteratively and adaptively so as to extract only that traffic that is of interest to the query at hand. Others have network monitoring in the context of streaming; yet, previous work has not closed the loop in a way that allows network operators to perform streaming analytics for network monitoring applications at scale. To achieve this objective, Sonata allows operators to express a network monitoring query by considering each packet as a tuple and efficiently partitioning each query between the switches and the stream processor through iterative refinement. Sonata extracts only the traffic that pertains to each query, ensuring that the stream processor can scale traffic rates of several terabits per second. We show with a simple example query involving DNS reflection attacks and traffic traces from one of the world's largest IXPs that Sonata can capture 95% of all traffic pertaining to the query, while reducing the overall data rate by a factor of about 400 and the number of required counters by four orders of magnitude. Copyright 2016 ACM.en
dc.description.sponsorshipWe thank our shepherd, Fadel Adib, the anonymous reviewers, Srinivas Narayana, Ankita Pawar, Rick Porter, Jennifer Rexford for for feedback and comments. This research was supported by National Science Foundation Awards CNS-1539920, and by European Union’s Horizon 2020 program under the ENDEAVOUR project (grant agree- ment 644960).en
dc.publisherAssociation for Computing Machinery (ACM)en
dc.relation.urlhttp://dl.acm.org/citation.cfm?doid=3005745.3005748en
dc.titleNetwork Monitoring as a Streaming Analytics Problemen
dc.typeConference Paperen
dc.contributor.departmentKAUSTen
dc.identifier.journalProceedings of the 15th ACM Workshop on Hot Topics in Networks - HotNets '16en
dc.conference.date2016-11-09 to 2016-11-10en
dc.conference.name15th ACM Workshop on Hot Topics in Networks, HotNets 2016en
dc.conference.locationAtlanta, GA, USAen
dc.contributor.institutionPrinceton University, United Statesen
dc.contributor.institutionETH Zürich, Switzerlanden
dc.contributor.institutionUniversité Catholique de Louvain, Belgiumen
dc.contributor.institutionNIKSUN, Inc., United Statesen
kaust.authorCanini, Marcoen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.