Abstract: The integration of high-penetration renewable energy sources poses significant challenges to power system frequency stability. As mobile energy storage resources, electric vehicles (EVs) have great potential to participate in frequency regulation. To address the real-time control difficulty that massive EVs efficiently track secondary frequency regulation commands while satisfying user travel demands and battery safety constraints, this paper proposes a deep reinforcement learning (DRL)-based two-layer frequency regulation control strategy considering dynamic feasible-region constraints. Firstly, a dynamic aggregation model for the state-of-charge (SOC) feasible regions is established from individual EVs to the vehicle clusters, and a frequency regulation capacity optimization problem is formulated to quantify user behaviors and physical constraints as real-time safe regulation boundaries of the EV clusters. Secondly, a "centralized decision-making - hierarchical execution" framework is developed. At the central layer, the deep deterministic policy gradient (DDPG) algorithm is employed to learn the optimal power allocation among different functional zones within the continuous action space. At the functional zone layer, commands are decomposed to individual EVs based on SOC rules. Finally, an action projection layer is introduced to map the raw actions output by DDPG into the feasible region in real time, so as to ensure the safety and feasibility of the control process. Simulation results demonstrate that the proposed strategy achieves high-precision frequency regulation tracking while satisfying user travel demands and battery safety constraints, with a mean absolute error (MAE) of 0.084 MW and a root mean square error (RMSE) of 0.129 MW. Compared with the unconstrained reinforcement learning algorithm, the proposed method reduces the tracking error by 97.8% and improves the computational efficiency by 28 times in contrast to model predictive control (MPC). Meanwhile, the fairness of frequency regulation resources utilization among different regions is effectively ensured.