Abstract: Unmanned underwater vehicles (UUVs) are widely used in cooperative anti-submarine missions due to their high maneuverability and low detectability. However, the existing methods either incur high transmission overhead or face problems such as single-point failure and computational conflicts. Therefore, a hierarchical federated reinforcement learning technology for UUV cooperative anti-submarine warfare is proposed. A hierarchical federated learning framework is constructed to weaken the role of the central aggregator and improve system robustness and damage resistance. Furthermore, an adaptive cooperative training mechanism based on reinforcement learning is designed, and the optimal parameter selection scheme of each UUV is presented to resolve conflicts. Experimental results show that the proposed method improves both the performance and stability of cooperative anti-submarine missions.