Dual-channel batch constrained deep Q-learning for sepsis treatment

Liu Huidong, Zhang Xiangfei, Yu Hang, Zhang Qingchen,
Dual-channel batch constrained deep Q-learning for sepsis treatment,
Engineering Applications of Artificial Intelligence,
Volume 159, Part B,
2025,
111545,
ISSN 0952-1976,
https://doi.org/10.1016/j.engappai.2025.111545.
(https://www.sciencedirect.com/science/article/pii/S0952197625015477)
Abstract: Sepsis is a highly complex and heterogeneous critical illness, requiring dynamic adjustments to treatment strategies based on individual patient characteristics. However, existing reinforcement learning (RL) methods for personalized treatment face several challenges, such as incomplete modeling of patient states, limited generalization capability of policies, and distributional shift issues in offline learning. To address these challenges, we propose a novel Dual-Channel Batch-Constrained Deep Q-Learning (DBCDQ) method to enable more precise personalized sepsis treatment. Specifically, we design a dual-channel mechanism that integrates the patient’s current physiological state with their historical treatment responses, enabling comprehensive modeling of dynamic patient characteristics and improving responsiveness to individualized treatment needs. Additionally, we introduce a batch-constrained mechanism into the policy network, which enforces consistency between the learned policy and the actual clinical data distribution. This mitigates distributional shift issues in offline reinforcement learning (ORL). We evaluate our approach on the sepsis data from the MIMIC-III dataset, and experimental results show that our method outperforms state-of-the-art methods and can reduce patient clinical mortality by 3.85%.
Keywords: Reinforcement learning; Personalized treatment policies; Sepsis treatment; Offline reinforcement learning