ReCamDriving: LiDAR-Free Camera-Controlled Novel Trajectory Video Generation
By: Yaokun Li , Shuaixian Wang , Mantang Guo and more
Potential Business Impact:
Creates realistic videos of cars driving anywhere.
We propose ReCamDriving, a purely vision-based, camera-controlled novel-trajectory video generation framework. While repair-based methods fail to restore complex artifacts and LiDAR-based approaches rely on sparse and incomplete cues, ReCamDriving leverages dense and scene-complete 3DGS renderings for explicit geometric guidance, achieving precise camera-controllable generation. To mitigate overfitting to restoration behaviors when conditioned on 3DGS renderings, ReCamDriving adopts a two-stage training paradigm: the first stage uses camera poses for coarse control, while the second stage incorporates 3DGS renderings for fine-grained viewpoint and geometric guidance. Furthermore, we present a 3DGS-based cross-trajectory data curation strategy to eliminate the train-test gap in camera transformation patterns, enabling scalable multi-trajectory supervision from monocular videos. Based on this strategy, we construct the ParaDrive dataset, containing over 110K parallel-trajectory video pairs. Extensive experiments demonstrate that ReCamDriving achieves state-of-the-art camera controllability and structural consistency.
Similar Papers
Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry
CV and Pattern Recognition
Changes how a video camera moves in a movie.
Learning to Generate 4D LiDAR Sequences
CV and Pattern Recognition
Creates 3D car sensor data from words.
DriveLiDAR4D: Sequential and Controllable LiDAR Scene Generation for Autonomous Driving
CV and Pattern Recognition
Creates realistic driving scenes for self-driving cars.