Optimization-based spectral end-to-end deep reinforcement learning for equity portfolio management

Pengrui Yu, Siya Liu, Chengneng Jin, Runsheng Gu, Xiaomin Gong,
Optimization-based spectral end-to-end deep reinforcement learning for equity portfolio management,
Pacific-Basin Finance Journal,
Volume 91,
2025,
102746,
ISSN 0927-538X,
https://doi.org/10.1016/j.pacfin.2025.102746.
(https://www.sciencedirect.com/science/article/pii/S0927538X25000836)
Abstract: We propose a novel approach to equity portfolio optimization that combines spectral analysis and classical equity portfolio optimization theory with deep reinforcement learning in an end-to-end framework. We introduce the End-to-end Frequency Online Deep Deterministic Policy Gradient (EFO-DDPG) algorithm, which leverages discrete Fourier transform to decompose asset return sequences into frequency components. Unlike traditional methods that treat high-frequency components as noise, EFO-DDPG learns to adjust the influence of different frequency components dynamically. Moreover, the algorithm embeds a mean–variance portfolio optimization problem within a deep learning network, enhancing interpretability compared to black-box approaches. The framework models the investment problem as a Partially Observable Markov Decision Process (POMDP), using a state processing block with transformer encoders to capture complex relationships in the market data. By integrating spectral analysis, portfolio optimization theory, and online deep reinforcement learning, EFO-DDPG aims to adapt to non-stationary financial markets and generate superior investment strategies.
Keywords: Spectral analysis; Deep reinforcement learning; Equity portfolio optimization; End-to-end