Robust Near-field Beam Tracking via Deep Q-network for THz Communications

This paper presents a robust near-field (NF) beam tracking algorithm for terahertz communications based on deep Q-network (DQN). Traditional NF beam tracking methods relying on mobility models are fatal in ultra-massive MIMO systems, where even the slightest error could result in beam tracking failures. Thus, the proposed algorithm aims to maintain a stable beamforming gain by tracking the mobile station through the analysis of received signals without requiring mobile dynamics. By utilizing DQN, the proposed algorithm strengthens its tracking capability from online experiences and updates the combining beam towards positions expected to maximize beamforming gain. Throughout simulations, we compare the proposed algorithm with the Bayesian filter-based NF beam tracking algorithm. The simulation results confirm the robustness of the proposed algorithm for NF beam tracking, especially for abrupt changes in mobile dynamics.