Adaptive bandwidth tuning via deep reinforcement learning for nonlinear temperature control of insulated wall

Yanming Zhong, Dong Li, Li Sun,
Adaptive bandwidth tuning via deep reinforcement learning for nonlinear temperature control of insulated wall,
Applied Thermal Engineering,
Volume 281, Part 2,
2025,
128532,
ISSN 1359-4311,
https://doi.org/10.1016/j.applthermaleng.2025.128532.
(https://www.sciencedirect.com/science/article/pii/S1359431125031242)
Abstract: Extreme orbital temperature fluctuations pose a persistent challenge for spacecraft thermal regulation. This study introduces a Deep Deterministic Policy Gradient-based Active Disturbance Rejection Control (DDPG-ADRC) scheme designed explicitly for the nonlinear dynamics of spacecraft insulation walls. The controller leverages a reinforcement learning framework that adaptively tunes ADRC parameters via a compound reward function, that integrates time-domain indicators (integral absolute error, overshoot) and frequency-domain margins to achieve gain and phase stability. This approach enables real-time bandwidth self-adaptation for distributed-parameter thermal processes with coupled radiation, conduction, and convection. Applied to a nonlinear thermal model, the DDPG-ADRC controller achieves a 17.8% reduction in IAE (160.35 vs. 195.17) and 8.3% faster settling time (411.75 min vs. 448.92 min) in response to setpoint shifts. Under compound disturbances (a 10% step in flow rate at 1000 min and a 1  K temperature step at 1333 min), the controller reduces recovery time by 26.3% (276.47 min vs. 375.62 min) and eliminates steady-state error. Robustness analysis with 100 Monte Carlo simulations under ± 20% model perturbations demonstrates a 48.0% (±3.168 vs. ± 6.091) lower IAE deviation and 11% (±9.758 vs. ± 10.960) less settling-time variance than traditional ADRC, along with an overall performance gain over PID control. This framework significantly extends the applicability of ADRC to nonlinear satellite thermal environments, offering an intelligent, adaptive, and computationally efficient solution for next-generation spacecraft thermal management.
Keywords: Satellite thermal regulation; ADRC bandwidth self-adaptation; DDPG optimization; Nonlinear disturbance estimation; Robust intelligent control