Adaptive nonlinear integral-backstepping control for frequency stabilization in cyber-physical shipboard microgrids using double deep Q-learning

Mai The Vu, Duc Hung Pham, Van-Truong Nguyen, Quang Thang Do, Abdullah K. Alanazi, Tuan Hai Nguyen,
Adaptive nonlinear integral-backstepping control for frequency stabilization in cyber-physical shipboard microgrids using double deep Q-learning,
Engineering Applications of Artificial Intelligence,
Volume 160, Part B,
2025,
111943,
ISSN 0952-1976,
https://doi.org/10.1016/j.engappai.2025.111943.
(https://www.sciencedirect.com/science/article/pii/S0952197625019463)
Abstract: —The integration of diverse distributed energy resources is increasingly being enabled by shipboard microgrids (SbμGSs) as a promising solution to address the challenges posed by evolving maritime power systems toward decentralized and sustainable architectures. Open communication networks have been recently integrated with sustainable energy resources (SERs) for secure operation, regulation, and management of modern maritime power systems. However, deploying such emerging technologies introduces some faults and delays in communication channels which degrades the performance of marine power systems. This paper focuses on the development of a robust maritime controller-based nonlinear integral-backstepping (NI-BSC) for frequency stabilization of SbμGSs with high penetration of SERs, non-sensitive loads, and battery systems. Particularly, the double deep Q-learning (DDQL) is developed to adaptively design the NI-BSC and dynamically respond to the frequency stabilization challenges of SbμGS. By maximizing a long-term reinforcement signal, the neural networks (NNs) of DDQL (i.e., evaluation and target neural networks) are trained to reduce frequency fluctuations of SbμGS considering communication delays. The considered topology of SbμGS utilizes a separate maritime controller to regulate the solid-oxide fuel cell (SOFC) and thermostatic loads (heat pumps and freezer) while providing the possibility for sustainable units (e.g., wave energy and solar energy) to generate their maximum power. To conduct more realistic examinations, several typical scenarios of shipboard microgrids are carried out in a real-time setup. The real-time simulation outcomes reveal that the controller designed by DDQL outperforms the existing methodologies such as optimal controller, backstepping controller, and model-free sliding mode control, with an improvement in dynamic behaviors ranging from 54.57 % to 74.17 %.
Keywords: Shipboard microgrid systems; (SbμGS); Sustainable energy resources; (SER); Double deep Q-learning; (DDQL); Non-sensitive loads; Real-time testbed