Hybrid Prediction Integrated Planning for Autonomous Driving

Haochen Liu, Zhiyu Huang,Wenhui Huang,Haohan Yang, Xiaoyu Mo, Chen Lv

AutoMan Research Lab, Nanyang Technological University

Abstract

Autonomous driving systems require the ability to fully understand and predict the surrounding environment to make informed decisions in complex scenarios. Recent advancements in learning-based systems have highlighted the importance of integrating prediction and planning modules. However, this integration has brought forth three major challenges: inherent trade-offs by sole prediction, consistency between prediction patterns, and social coherence in prediction and planning. To address these challenges, we introduce a hybrid-predictions integrated planning (HPP) system, which possesses three novelly designed modules. First, we introduce marginal-conditioned occupancy prediction to align joint occupancy with agent-wise perceptions. Our proposed MS-OccFormer module achieves multi-stage alignment per occupancy forecasting with consistent awareness from agent-wise motion predictions. Second, we propose a game-theoretic motion predictor, GTFormer, to model the interactive future among individual agents with their joint predictive awareness. Third, hybrid prediction patterns are concurrently integrated with Ego Planner and optimized by prediction guidance. HPP achieves state-of-the-art performance on the nuScenes dataset, demonstrating superior accuracy and consistency for end-to-end paradigms in prediction and planning. Moreover, we test the long-term open-loop and closed-loop performance of HPP on the Waymo Open Motion Dataset and CARLA benchmark, surpassing other integrated prediction and planning pipelines with enhanced accuracy and compatibility.

Method Overview

Systematic overview of the proposed Hybrid predictions integrated planning (HPP) framework. DPP is established upon query-based ADS co-design optimizations of interactive planning with dual predictions integration (IPP and IOP), informed by BEV perceptions. With encoded perception scene context, DPP delivers predictions planning co-design in three-fold. Joint occupancy prediction are iteratively refined in MS-OccFormer, sharing mutual consistency over marginal motion prediction in GTFormer, which performs interactive reasoning between marginal prediction and planning. Reasoned outcomes and ego features are served to query dual predictions-aware planning in Ego Planner. Eventually, optimizations are scheduled to refine planning with dual predictions guidance.

Open-loop Planning (nuScenes)

The planner outputs a planned trajectory at 2Hz. All agents are roll-outed by log-replay per timestep in the dataset.

Closed-loop Planning (WOMD)

The planner outputs and optimizes a planned trajectory at 2Hz. The ego vehicle’s state are simulated accordingly at future time steps. The other agents are replayed from the log according to their observed states in the dataset.