Back to Top

■ Maximizing Determinism in Stream Processing Under Latency Constraints

Maximizing Determinism in Stream Processing Under Latency Constraints, Nikos Zacheilas, Vana Kalogeraki, Yiannis Nikolakopoulos, Vincenzo Gulisano, Marina Papatriantafilou, Philippas Tsigas, ACM DEBS, Barcelona, Spain, June 19 - 23, 2017 (received best paper award)

The problem of coping with the demands of determinism and meeting latency constraints is challenging in distributed data stream processing systems that have to process high volume data streams that arrive from different unsynchronized input sources. In order to deterministically process the streaming data, they need mechanisms that synchronize the order in which tuples are processed by the operators. On the other hand, achieving real-time response in such a system requires careful tradeoff between determinism and low latency performance. We build on a recently proposed approach to handle data exchange and synchronization in stream processing, namely ScaleGate, which comes with guarantees for determinism and an efficient lock-free implementation, enabling high scalability. Considering the challenge and trade-offs implied by real-time constraints, we propose a system which comprises (a) a novel data structure called Slack-ScaleGate (SSG), along with its algorithmic implementation; SSG enables us to guarantee the deterministic processing of tuples as long as they are able to meet their latency constraints, and (b) a method to dynamically tune the maximum amount of time that a tuple can wait in the SSG data-structure, relaxing the determinism guarantees when needed, in order to satisfy the latency constraints. Our detailed experimental evaluation using a traffic monitoring application deployed in the city of Dublin, illustrates the working and benefits of our approach.

Bibtex Entry.
  author    = {Nikos Zacheilas and
               Vana Kalogeraki and
               Yiannis Nikolakopoulos and
               Vincenzo Gulisano and
               Marina Papatriantafilou and
               Philippas Tsigas},
  title     = {Maximizing Determinism in Stream Processing Under Latency Constraints},
  booktitle = {Proceedings of the 11th {ACM} International Conference on Distributed
               and Event-based Systems, {DEBS} 2017, Barcelona, Spain, June 19-23,
  pages     = {112--123},
  year      = {2017},
  crossref  = {DBLP:conf/debs/2017},
  url       = {},
  doi       = {10.1145/3093742.3093921},
  timestamp = {Mon, 19 Jun 2017 09:32:11 +0200},
  biburl    = {},
  bibsource = {dblp computer science bibliography,}