Reinforcement learning-driven adaptive spectrum utilization in cognitive B5G networks

Abstract

In recent years, the growth of wireless services and connected devices increased the demand for radio spectrum. Static spectrum allocation could not fully address this demand, because licensed bands may remain underused across time and location. Cognitive radio networks (CRNs) allowed secondary users (SUs) to access available spectrum while preserving primary user (PU) priority. In these networks, efficient spectrum use required reliable sensing, effective access decisions, infrastructure-side coordination, resilience, and trust management. Accordingly, in the research reported in the present thesis, we improved spectrum utilization by addressing cooperative spectrum sensing, SU-side spectrum access, and coordination between spectrum-consuming and spectrum-granting nodes. These tasks involved sequential decisions under uncertainty and coupled interactions across users and network tiers. Therefore, we used reinforcement learning to support adaptive sensing, access, and coordination decisions. First, we developed Partially Cooperative Multi-Agent Reinforcement Learning (PCMARL) for cooperative spectrum sensing in cognitive radio Internet-of-Things networks under spectrum sensing data falsification attacks. PCMARL allowed SUs to selectively cooperate and improved sensing reliability and energy efficiency without full network cooperation. Second, we extended SU-side optimization from sensing to joint sensing and access. The proposed IMPACT framework optimized sensing participation, cooperation, channel selection, and transmission power under imperfect channel state information. Since access control required both discrete decisions and continuous power control, we introduced a hybrid-action learning formulation for cooperative spectrum access. Third, we added the spectrum granting and coordination side of the network. The proposed Federated Hierarchical Reinforcement Learning (F-HRL) framework coordinated SUs, next generation Node Bs, uncrewed aerial vehicles, and high-altitude platform stations. This architecture supported grant allocation, PU priority, trust-aware scheduling, privacy-preserving federation, and service continuity during infrastructure disruption. The results revealed that improved spectrum utilization required SU-side sensing and access optimization together with granting side coordination, infrastructure resilience, federated aggregation, and trust-aware control. Overall, in this research, we developed an integrated learning-based framework for resilient spectrum management in future wireless networks.

Date	4 Jul 2026
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Georges Kaddoum (Supervisor)