A Distributed Q-Learning Approach for Adaptive Sleep Modes in 5G Networks

In 5G networks, specific requirements are defined on the periodicity of Synchronization Signaling (SS) bursts. This imposes a constraint on the maximum period a Base Station (BS) can be deactivated. On the other hand, BS densification is expected in 5G architecture. This will lead to an energy crunch if kept ignored. In this paper, we propose a distributed algorithm based on Reinforcement Learning (RL) that controls the states of the BSs while respecting the requirements of 5G. By considering different levels of Sleep Modes (SMs), the algorithm chooses how deep a BS can sleep according to the best switch-off SM level policy that maximizes the trade-off between energy savings and system delay. The latter is calculated based on the wake-up time required by the different SM levels. Results show that our algorithm performs better than the case of using only one type of SM. Furthermore, our simulations show a gain in energy savings up to 90% when the users are delay tolerant while respecting the periodicity of the SS bursts in 5G.