ParkingPPOEnv: A Curriculum-Based Reinforcement Learning Framework for Autonomous Parking

Abstract

Autonomous parking presents a unique challenge in robotics, requiring the synthesis of strategic decision-making and precise non-holonomic control. Traditional rule-based planners often lack the flexibility to handle dynamic environments, while end-to-end Reinforcement Learning (RL) approaches struggle with sample efficiency and convergence in dense obstacle scenarios.
This paper addresses these challenges through the development of ParkingPPOEnv, a high-fidelity Gymnasium environment for training autonomous agents. The system integrates a kinematic bicycle model, a 16-ray planar LIDAR perception system, and an Oriented Bounding Box (OBB) collision detection engine. To facilitate learning, we implemented a multi-phase curriculum strategy that progressively increases environmental complexity from 0 % to 90 % obstacle occupancy. This paper details the system architecture, the custom reward function design, and analyzes the agent’s performance. Initial results demonstrate successful maneuvering in static environments, though challenges remain in dense obstacle avoidance, highlighting the need for future hierarchical control integration.