Local correlation detection with linearity enhancement in streaming data

Handle URI:
http://hdl.handle.net/10754/564666
Title:
Local correlation detection with linearity enhancement in streaming data
Authors:
Xie, Qing ( 0000-0003-4530-588X ) ; Shang, Shuo; Yuan, Bo; Pang, Chaoyi; Zhang, Xiangliang ( 0000-0002-3574-5665 )
Abstract:
This paper addresses the challenges in detecting the potential correlation between numerical data streams, which facilitates the research of data stream mining and pattern discovery. We focus on local correlation with delay, which may occur in burst at different time in different streams, and last for a limited period. The uncertainty on the correlation occurrence and the time delay make it diff cult to monitor the correlation online. Furthermore, the conventional correlation measure lacks the ability of ref ecting visual linearity, which is more desirable in reality. This paper proposes effective methods to continuously detect the correlation between data streams. Our approach is based on the Discrete Fourier Transform to make rapid cross-correlation calculation with time delay allowed. In addition, we introduce a shape-based similarity measure into the framework, which ref nes the results by representative trend patterns to enhance the signif cance of linearity. The similarity of proposed linear representations can quickly estimate the correlation, and the window sliding strategy in segment level improves the eff ciency for online detection. The empirical study demonstrates the accuracy of our detection approach, as well as more than 30% improvement of eff ciency. Copyright 2013 ACM.
KAUST Department:
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division; Computer Science Program; Machine Intelligence & kNowledge Engineering Lab
Publisher:
Association for Computing Machinery (ACM)
Journal:
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management - CIKM '13
Conference/Event name:
22nd ACM International Conference on Information and Knowledge Management, CIKM 2013
Issue Date:
2013
DOI:
10.1145/2505515.2505746
Type:
Conference Paper
ISBN:
9781450322638
Appears in Collections:
Conference Papers; Computer Science Program; Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division

Full metadata record

DC FieldValue Language
dc.contributor.authorXie, Qingen
dc.contributor.authorShang, Shuoen
dc.contributor.authorYuan, Boen
dc.contributor.authorPang, Chaoyien
dc.contributor.authorZhang, Xiangliangen
dc.date.accessioned2015-08-04T07:11:33Zen
dc.date.available2015-08-04T07:11:33Zen
dc.date.issued2013en
dc.identifier.isbn9781450322638en
dc.identifier.doi10.1145/2505515.2505746en
dc.identifier.urihttp://hdl.handle.net/10754/564666en
dc.description.abstractThis paper addresses the challenges in detecting the potential correlation between numerical data streams, which facilitates the research of data stream mining and pattern discovery. We focus on local correlation with delay, which may occur in burst at different time in different streams, and last for a limited period. The uncertainty on the correlation occurrence and the time delay make it diff cult to monitor the correlation online. Furthermore, the conventional correlation measure lacks the ability of ref ecting visual linearity, which is more desirable in reality. This paper proposes effective methods to continuously detect the correlation between data streams. Our approach is based on the Discrete Fourier Transform to make rapid cross-correlation calculation with time delay allowed. In addition, we introduce a shape-based similarity measure into the framework, which ref nes the results by representative trend patterns to enhance the signif cance of linearity. The similarity of proposed linear representations can quickly estimate the correlation, and the window sliding strategy in segment level improves the eff ciency for online detection. The empirical study demonstrates the accuracy of our detection approach, as well as more than 30% improvement of eff ciency. Copyright 2013 ACM.en
dc.publisherAssociation for Computing Machinery (ACM)en
dc.subjectCorrelation detectionen
dc.subjectData streamen
dc.subjectLinear approximationen
dc.titleLocal correlation detection with linearity enhancement in streaming dataen
dc.typeConference Paperen
dc.contributor.departmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Divisionen
dc.contributor.departmentComputer Science Programen
dc.contributor.departmentMachine Intelligence & kNowledge Engineering Laben
dc.identifier.journalProceedings of the 22nd ACM international conference on Conference on information & knowledge management - CIKM '13en
dc.conference.date27 October 2013 through 1 November 2013en
dc.conference.name22nd ACM International Conference on Information and Knowledge Management, CIKM 2013en
dc.conference.locationSan Francisco, CAen
dc.contributor.institutionBeijing Key Laboratory of Petroleum Data Mining, Department of Software Engineering, China University of Petroleum, Beijing, Chinaen
dc.contributor.institutionDivision of Informatics, Graduate School at Shenzhen, Tsinghua University, Chinaen
dc.contributor.institutionAustralian E-Health Research Centre, CSIRO, Chinaen
kaust.authorXie, Qingen
kaust.authorZhang, Xiangliangen
All Items in KAUST are protected by copyright, with all rights reserved, unless otherwise indicated.