Demand response-based multi-agent deep reinforcement learning control framework for heating systems

Yaling Wu, Mingyang Cong, Qunshan Lu, Zhigang Zhou, Jing Liu, Dayi Yang,
Demand response-based multi-agent deep reinforcement learning control framework for heating systems,
Applied Thermal Engineering,
Volume 280, Part 1,
2025,
127957,
ISSN 1359-4311,
https://doi.org/10.1016/j.applthermaleng.2025.127957.
(https://www.sciencedirect.com/science/article/pii/S1359431125025499)
Abstract: This study addresses the challenge of demand response (DR) intelligent control in the terminal of district heating systems (DHS), caused by the multi-physical factors of thermal and hydraulic coupling, which leads to supply–demand imbalance. The study proposed an intelligent control framework based on DR, establishing a multi-zone thermal–hydraulic coupling model, where each zone corresponds to an agent in the multi-agent deep reinforcement learning (MADRL) framework. The multi-agent proximal policy optimization (MAPPO) algorithm is applied to optimize system performance. The results show that after applying the MAPPO algorithm, the root mean square error (RMSE) of the room temperature remains stable within ± 0.5 °C, with a maximum overshoot of 0.52 °C, significantly improving user comfort and stability. Compared to traditional Model Predictive Control (MPC) methods, MAPPO demonstrated clear advantages in temperature tracking and overshoot suppression, improving temperature control accuracy and significantly enhancing energy efficiency and system stability at the heating terminal. This method provides an innovative solution for the intelligent, energy-efficient, and high-performance operation of heating systems, surpassing the limitations of traditional control methods in complex and dynamic environments, with important practical application value.
Keywords: District heating system; Demand response; Multi-agent deep reinforcement learning; Generalizability