A New Study of Two Divergence Metrics for Change Detection in Data Streams

Handle URI:
http://hdl.handle.net/10754/555788
Title:
A New Study of Two Divergence Metrics for Change Detection in Data Streams
Authors:
Qahtan, Abdulhakim Ali Ali ( 0000-0001-8254-1764 ) ; Wang, Suojin; Carroll, Raymond; Zhang, Xiangliang ( 0000-0002-3574-5665 )
Abstract:
Streaming data are dynamic in nature with frequent changes. To detect such changes, most methods measure the difference between the data distributions in a current time window and a reference window. Divergence metrics and density estimation are required to measure the difference between the data distributions. Our study shows that the Kullback-Leibler (KL) divergence, the most popular metric for comparing distributions, fails to detect certain changes due to its asymmetric property and its dependence on the variance of the data. We thus consider two metrics for detecting changes in univariate data streams: a symmetric KL-divergence and a divergence metric measuring the intersection area of two distributions. The experimental results show that these two metrics lead to more accurate results in change detection than baseline methods such as Change Finder and using conventional KL-divergence.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Citation:
Qahtan, Abdulhakim, Suojin Wang, Raymond Carroll, and Xiangliang Zhang. "A New Study of Two Divergence Metrics for Change Detection in Data Streams."
Publisher:
IOS Press
Journal:
Frontiers in Artificial Intelligence and Applications
Conference/Event name:
21st European Conference on Artificial Intelligence, ECAI 2014
Issue Date:
Aug-2014
DOI:
10.3233/978-1-61499-419-0-1081
Type:
Conference Paper
ISSN:
0922-6389
Appears in Collections:
Conference Papers; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorQahtan, Abdulhakim Ali Alien
dc.contributor.authorWang, Suojinen
dc.contributor.authorCarroll, Raymonden
dc.contributor.authorZhang, Xiangliangen
dc.date.accessioned2015-05-26T07:48:21Zen
dc.date.available2015-05-26T07:48:21Zen
dc.date.issued2014-08en
dc.identifier.citationQahtan, Abdulhakim, Suojin Wang, Raymond Carroll, and Xiangliang Zhang. "A New Study of Two Divergence Metrics for Change Detection in Data Streams."en
dc.identifier.issn0922-6389en
dc.identifier.doi10.3233/978-1-61499-419-0-1081en
dc.identifier.urihttp://hdl.handle.net/10754/555788en
dc.description.abstractStreaming data are dynamic in nature with frequent changes. To detect such changes, most methods measure the difference between the data distributions in a current time window and a reference window. Divergence metrics and density estimation are required to measure the difference between the data distributions. Our study shows that the Kullback-Leibler (KL) divergence, the most popular metric for comparing distributions, fails to detect certain changes due to its asymmetric property and its dependence on the variance of the data. We thus consider two metrics for detecting changes in univariate data streams: a symmetric KL-divergence and a divergence metric measuring the intersection area of two distributions. The experimental results show that these two metrics lead to more accurate results in change detection than baseline methods such as Change Finder and using conventional KL-divergence.en
dc.publisherIOS Pressen
dc.rightsThis article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License.en
dc.titleA New Study of Two Divergence Metrics for Change Detection in Data Streamsen
dc.typeConference Paperen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.identifier.journalFrontiers in Artificial Intelligence and Applicationsen
dc.conference.date2014-08-18 to 2014-08-22en
dc.conference.name21st European Conference on Artificial Intelligence, ECAI 2014en
dc.conference.locationPrague, CZEen
dc.eprint.versionPublisher's Version/PDFen
dc.contributor.institutionDepartment of Statistics, Texas A&M University, College Station, TX 77843, USAen
kaust.authorZhang, Xiangliangen
kaust.authorQahtan, Abdulhakim Ali Alien
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.