ChangingGrounding: 3D Visual Grounding in Changing Scenes
By: Miao Hu , Zhiwei Huang , Tai Wang and more
Potential Business Impact:
Robots find things in changing rooms using memory.
Real-world robots localize objects from natural-language instructions while scenes around them keep changing. Yet most of the existing 3D visual grounding (3DVG) method still assumes a reconstructed and up-to-date point cloud, an assumption that forces costly re-scans and hinders deployment. We argue that 3DVG should be formulated as an active, memory-driven problem, and we introduce ChangingGrounding, the first benchmark that explicitly measures how well an agent can exploit past observations, explore only where needed, and still deliver precise 3D boxes in changing scenes. To set a strong reference point, we also propose Mem-ChangingGrounder, a zero-shot method for this task that marries cross-modal retrieval with lightweight multi-view fusion: it identifies the object type implied by the query, retrieves relevant memories to guide actions, then explores the target efficiently in the scene, falls back when previous operations are invalid, performs multi-view scanning of the target, and projects the fused evidence from multi-view scans to get accurate object bounding boxes. We evaluate different baselines on ChangingGrounding, and our Mem-ChangingGrounder achieves the highest localization accuracy while greatly reducing exploration cost. We hope this benchmark and method catalyze a shift toward practical, memory-centric 3DVG research for real-world applications. Project page: https://hm123450.github.io/CGB/ .
Similar Papers
Zero-Shot Visual Grounding in 3D Gaussians via View Retrieval
CV and Pattern Recognition
Find objects in 3D worlds with just words.
A Coarse-to-Fine Approach to Multi-Modality 3D Occupancy Grounding
CV and Pattern Recognition
Lets self-driving cars spot exact 3D object shapes
A Coarse-to-Fine Approach to Multi-Modality 3D Occupancy Grounding
CV and Pattern Recognition
Enables self-driving cars to spot objects precisely in 3D