KAUST DepartmentComputer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division
Extreme Computing Research Center
Online Publication Date2018-05-13
Print Publication Date2018
Permanent link to this recordhttp://hdl.handle.net/10754/628865
MetadataShow full item record
AbstractThe temporal data is ubiquitous, and massive amount of temporal data is generated nowadays. Management of big temporal data is important yet challenging. Processing big temporal data using a distributed system is a desired choice. However, existing distributed systems/methods either cannot support native queries, or are disk-based solutions, which could not well satisfy the requirements of high throughput and low latency. To alleviate this issue, this paper proposes an In-memory based Two-level Index Solution in Spark (ITISS) for processing big temporal data. The framework of our system is easy to understand and implement, but without loss of efficiency. We conduct extensive experiments to verify the performance of our solution. Experimental results based on both real and synthetic datasets consistently demonstrate that our solution is efficient and competitive.
CitationYao B, Zhang W, Wang Z-J, Chen Z, Shang S, et al. (2018) Distributed In-Memory Analytics for Big Temporal Data. Lecture Notes in Computer Science: 549–565. Available: http://dx.doi.org/10.1007/978-3-319-91452-7_36.
SponsorsThis work was supported by the National Basic Research Program (973 Program, No. 2015CB352403), the NSFC (U1636210, 61729202, 91438121, 61672351, 61472453, U1401256, U1501252, U1611264, U1711261 and U1711262), the National Key Research and Development Program of China (2016YFB0700502), the Scientific Innovation Act of STCSM (15JC1402400), the Opening Projects of Guangdong Key Laboratory of Big Data Analysis and Processing (201808), Guangdong Province Key Laboratory of Popular High Performance Computers of Shenzhen University (SZU-GDPHPCL2017), and the Microsoft Research Asia.
Conference/Event name23rd International Conference on Database Systems for Advanced Applications, DASFAA 2018