Deep sequential adaptive reinforcement learning for manufacturing process optimization

Shengbo Xu, Sai Ma, Qinkai Han, Hongtao Zhu, Fulei Chu,
Deep sequential adaptive reinforcement learning for manufacturing process optimization,
Advanced Engineering Informatics,
Volume 66,
2025,
103456,
ISSN 1474-0346,
https://doi.org/10.1016/j.aei.2025.103456.
(https://www.sciencedirect.com/science/article/pii/S1474034625003490)
Abstract: In modern manufacturing processes, the determination of manufacturing process parameters directly impacts the mechanical properties of the product. As manufacturing processes become more complex and diverse, there are still many challenges in determining process parameters for high-quality products. To improve manufacturing quality, a novel deep reinforcement learning method named sequential adaptive twin delayed deep deterministic policy gradient (SATD3) is proposed to learn manufacturing process knowledge and optimize process parameters efficiently. In the proposed method, prioritized experience replay mechanism is utilized to strengthen global optimization capability by prioritizing the learning of critical parameter combinations. To accurately evaluate the potential optimal parameter combinations, a novel sequential adaptive B-spline Critic network is designed to extract features from history sequential trajectory. Then the meta-attention module is utilized to enhance the decision-making ability of the Actor network for parameter combination optimization. Finally, the proposed method could determine the optimal process parameter combination. To verify the effectiveness and adaptability of the SATD3 method, four different processes are selected as research cases, involving subtractive, additive, and formative processes. The results indicate that the SATD3 method can achieve better performance for optimizing process parameters compared with other deep reinforcement learning algorithms and classical heuristic algorithms.
Keywords: Deep reinforcement learning; Sequential adaptive mechanism; Manufacturing process optimization