Nebula: Enable City-Scale 3D Gaussian Splatting in Virtual Reality via Collaborative Rendering and Accelerated Stereo Rasterization
By: He Zhu , Zheng Liu , Xingyang Li and more
Potential Business Impact:
Makes virtual worlds load faster and look better.
3D Gaussian splatting (3DGS) has drawn significant attention in the architectural community recently. However, current architectural designs often overlook the 3DGS scalability, making them fragile for extremely large-scale 3DGS. Meanwhile, the VR bandwidth requirement makes it impossible to deliver high-fidelity and smooth VR content from the cloud. We present Nebula, a coherent acceleration framework for large-scale 3DGS collaborative rendering. Instead of streaming videos, Nebula streams intermediate results after the LoD search, reducing 1925% data communication between the cloud and the client. To further enhance the motion-to-photon experience, we introduce a temporal-aware LoD search in the cloud that tames the irregular memory access and reduces redundant data access by exploiting temporal coherence across frames. On the client side, we propose a novel stereo rasterization that enables two eyes to share most computations during the stereo rendering with bit-accurate quality. With minimal hardware augmentations, Nebula achieves 2.7$\times$ motion-to-photon speedup and reduces 1925% bandwidth over lossy video streaming.
Similar Papers
Voyager: Real-Time Splatting City-Scale 3D Gaussians on Your Phone
Graphics
Shows big 3D worlds on your phone fast.
Neo: Real-Time On-Device 3D Gaussian Splatting with Reuse-and-Update Sorting Acceleration
Hardware Architecture
Makes 3D virtual worlds look real on phones.
No Redundancy, No Stall: Lightweight Streaming 3D Gaussian Splatting for Real-time Rendering
Hardware Architecture
Makes 3D scenes render much faster.