Skip to main navigation Skip to search Skip to main content

DL-Store: A Distributed Hybrid OLTP and OLAP Data Processing Engine

  • Technical University of Munich
  • IBM

Research output: Contribution to Book/Report types › Contribution to conference proceedings › peer-review

3 Citations (Scopus)

Abstract

There has been a recent push in the database community towards supporting real-time analytical queries (OLAP) while sustaining a large volume of fine-grained updates (OLTP). Supporting these types of workloads require both an efficient data storage layer as well as a distributed architecture. In this demo, we address the latter point with our Distributed Lineage-based Data Store (DL-Store), which is a distributed data processing engine. DL-Store is built on top of L-Store, which is a lineage-based storage architecture designed to handle mixed OLTP and OLAP workloads, and provides scalability and elasticity by supporting multiple L-Store nodes. To maintain the desired consistency semantics, DL-Store employs a distributed transaction handler component which can horizontally scaled by provisioning additional transaction manager nodes. We leverage partitioning in the record space of the transactions to minimize communication across transaction managers while ensuring consistent execution. The demo shows our implementation of DL-Store over Apache Spark using a variety of use cases.

Original languageEnglish
Title of host publicationProceedings - 2016 IEEE 36th International Conference on Distributed Computing Systems, ICDCS 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages769-770
Number of pages2
ISBN (Electronic)9781509014828
DOIs
Publication statusPublished - 8 Aug 2016
Externally publishedYes
Event36th IEEE International Conference on Distributed Computing Systems, ICDCS 2016 - Nara, Japan
Duration: 27 Jun 2016 → 30 Jun 2016

Publication series

NameProceedings - International Conference on Distributed Computing Systems
Volume2016-August
ISSN (Print)1063-6927
ISSN (Electronic)2575-8411

Conference

Conference36th IEEE International Conference on Distributed Computing Systems, ICDCS 2016
Country/TerritoryJapan
CityNara
Period27/06/16 → 30/06/16

Keywords

  • OLAP-OLTP
  • Spark
  • Transaction handling

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'DL-Store: A Distributed Hybrid OLTP and OLAP Data Processing Engine'. These topics are generated from the title and abstract of the publication. Together, they form a unique fingerprint.

Cite this