EdgeDAG: a latency-aware initial operator placement strategy for edge-based distributed stream processing applications

Abstract

Over the past few years, the Internet of Things (IoT) has become popular. IoT devices are directly integrated into human lives. As a result, the data generated by IoT devices has increased. Processing applications can provide valuable information and it can reduce the amount of data passed to the cloud. The critical goal of these applications is to provide low latency results. Stream processing can facilitate the processing by offering a programming model for running an application in parallel. In the last few years, the number of IoT applications that rely on stream processing has increased. These applications process continuous streams of data with a low delay and provide valuable information. However, IoT devices are restricted in the case of bandwidth and computational resources. To meet the stringent latency requirements and the need for real-time results, the components of the stream processing pipeline can be deployed directly onto the edge layer to benefit from the resources and capabilities that the swarm of edge devices can provide. Edge network provides resources close to the user and helps to reduce the latency in time-sensitive applications. In this thesis, we propose a new optimization strategy for deploying the operators over the suitable resources at the edge of the network with the goal of minimizing latency while ensuring that the constraints of the devices and their network capabilities are respected. By implementing a new simulator, proposing a new type of optimization function and using a meta-heuristic algorithm called Grey Wolf Optimizer, we can provide a good mapping between the edge nodes and operators.

Date	31 Mar 2022
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Julien Gascon-Samson (Supervisor)