Hierarchical collaborative optimization of generalized shared energy storage system connected to distribution system: A master-slave game based on deep reinforcement learning approach

Zhiguo Dong, Gongqiang Li, Fengxiang Xie, Xiaofan Ji, Hanqing Yang,
Hierarchical collaborative optimization of generalized shared energy storage system connected to distribution system: A master-slave game based on deep reinforcement learning approach,
Energy Reports,
Volume 14,
2025,
Pages 5979-5990,
ISSN 2352-4847,
https://doi.org/10.1016/j.egyr.2025.08.021.
(https://www.sciencedirect.com/science/article/pii/S2352484725004858)
Abstract: As an effective means of integrating energy storage resources, shared energy storage systems (SESS) participating in the distribution system optimization and scheduling can further achieve safe, stable, and efficient operation of the power system. In this paper, a hierarchical collaborative optimization of generalized SESS (GSESS) connected to distribution system based on master-slave game and deep reinforcement learning is proposed. Unlike conventional (CSESS) that only considers battery energy storage devices, GSESS constructed in this paper considers not only actual energy storage devices such as lithium battery, but also virtual energy storage that includes flexible loads and electric vehicles. Based on GSESS, dynamic integration of demand-side resources can be fully achieved. Then, a master-slave game model is established with distributed system operator (DSO) as the leader, energy aggregator (EA), load aggregator (LA) and GSESS as followers. To solve the established master-slave game model, a multi-agent deep deterministic policy gradient (MADDPG) algorithm with centralized training and distributed execution is employed. Finally, the optimal utility of energy storage resources and reasonable allocation of benefits is achieved. The effectiveness of the proposed method is verified through simulation analysis on the modified IEEE-38 testing system. Also, the comparison with CSESS and centralized optimization solver are conducted.
Keywords: Generalized shared energy storage; Master-slave game; Multi-agent deep deterministic policy gradient; Demand-side response