Semi-Markov modeling of DRX in 5G NR-U for energy efficiency: A deep reinforcement learning approach

Mahima Kumar, Anupam Gautam,
Semi-Markov modeling of DRX in 5G NR-U for energy efficiency: A deep reinforcement learning approach,
Physical Communication,
Volume 73,
2025,
102869,
ISSN 1874-4907,
https://doi.org/10.1016/j.phycom.2025.102869.
(https://www.sciencedirect.com/science/article/pii/S1874490725002721)
Abstract: The rapid expansion of 5th Generation (5G) networks and the growing demand for high data rates have increased the energy burden on a User Equipment (UE). Specifically, when operating in the unlicensed spectrum under 5G New Radio-Unlicensed (5G NR-U) networks, UE consumes more power compared to earlier networks due to increased channel sensing ability and more frequent signaling and transmission. To deal with energy efficiency in UEs, a power saving mechanism called Discontinuous Reception (DRX) was introduced. Traditional DRX schemes are based on fixed parameter values, which struggle to adapt, to dynamic traffic patterns, base station breakdowns, and unlicensed channel contention. Motivated by these limitations, we have proposed a novel framework for DRX mechanism. The proposed model captures the complex state transitions of the UE while explicitly accounting for base station breakdowns and repairs, as well as the unique requirements of unlicensed spectrum access. It notably incorporates Beam Search (BS) technique, which plays a critical role in enabling efficient and reliable communication in dynamic and interference-prone environments. To present an analytical comparison of power saving and delay, the proposed DRX mechanism is mapped with M/G/1 (single-server queue with Markovian arrivals, general service time distribution, and one server) Semi-Markov Process (SMP) model with NT (Number–Time) policy. In addition, the analytical results were validated by implementing Deep Reinforcement Learning (DRL) using a Deep Q-Network (DQN). The DQN agent is used to dynamically manage the UE within a simulated environment that incorporates realistic elements such as Poisson packet arrivals, exponential service times, an NT policy-driven sleep state governed by packet count and time thresholds, stochastic base station breakdowns and repairs, and a probabilistic BS procedure. Simulation results demonstrate significant improvements in energy efficiency, delay reduction, and responsiveness to network dynamics, illustrating the potential of DRL to enhance power saving mechanism in next-generation wireless systems.
Keywords: 5G New Radio-Unlicensed; Discontinuous Reception; Deep Reinforcement Learning; Semi-Markov Process; Energy Efficiency