Energy-aware computational offloading using reinforcement learning techniques

Abstract

In the rapidly evolving landscape of 5G and the emerging 6G era, computational offloading is becoming a game-changer for mobile task execution within edge computing infrastructures. This paradigm shift involves the transfer of resource-intensive computational tasks to external servers nearby in the network, offering the potential for optimized efficiency. Yet, to ensure consistent Quality of Service (QoS) for the numerous users involved, meticulous planning of the offloading decisions should be made, which potentially involves inter-site task transferring to meet the diverse application requirements of mobile users. In this thesis, our focus extends to a multi-user Multi-Access Edge Computing (MEC) infrastructure with multi-site collaboration, where Mobile Devices (MDs) have the capability to offload their computational tasks to the available Edge Sites (ESs). Our goal is to minimize end-to-end delay experienced by these tasks and the energy consumption of the system. These two important measures collectively constitute the overall cost of the entire system and are fundamental for the user experience. The central challenge is to coordinate these goals, ensuring a seamless convergence of performance optimization, user satisfaction, and energy efficiency. To tackle this challenge, we introduce a sophisticated two-stage mechanism based on Reinforcement Learning (RL), a cutting-edge approach that enables us to iteratively refine the MDs’ decisions regarding task offloading to ESs, as well as the ESs’ decisions regarding the transfer of tasks between themselves. This iterative optimization process lies at the core of our approach, guiding the seamless coordination of computational tasks to achieve a careful balance between low delays and energy efficiency. The first stage is where individual MDs autonomously determine whether to offload their tasks to the attached ES or execute the tasks locally. This distributed task-offloading decision is done using an iterative RL-based mechanism called Stochastic Learning Automata (SLA). The next stage which ensures load balancing across the edge infrastructure is achieved using an offline-trained Deep Q-Network (DQN) employed at the end of each iteration in the first stage. These two stages are integrated into a multi-round cooperative computational offloading mechanism which iteratively optimizes the decisions made by both the MDs and the ESs, ultimately leading to the stable convergence of the optimization problem. Our experimental results using different numbers of MDs and ESs show that our framework decreases the delay of the tasks and the energy consumption of MDs. Compared to the solution proposed by prior work which doesn’t support load balancing at edge infrastructure our solution gives better results.

Date	8 Dec 2023
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Aris Leivadeas (Supervisor)