Scalable Distributed Vector Search via Accuracy Preserving Index Construction
By: Yuming Xu , Qianxi Zhang , Qi Chen and more
Scaling Approximate Nearest Neighbor Search (ANNS) to billions of vectors requires distributed indexes that balance accuracy, latency, and throughput. Yet existing index designs struggle with this tradeoff. This paper presents SPIRE, a scalable vector index based on two design decisions. First, it identifies a balanced partition granularity that avoids read-cost explosion. Second, it introduces an accuracy-preserving recursive construction that builds a multi-level index with predictable search cost and stable accuracy. In experiments with up to 8 billion vectors across 46 nodes, SPIRE achieves high scalability and up to 9.64X higher throughput than state-of-the-art systems.
Similar Papers
Approximate Nearest Neighbor Search of Large Scale Vectors on Distributed Storage
Databases
Finds similar items in huge online lists faster.
DISTRIBUTEDANN: Efficient Scaling of a Single DISKANN Graph Across Thousands of Computers
Distributed, Parallel, and Cluster Computing
Finds information super fast in huge data.
HARMONY: A Scalable Distributed Vector Database for High-Throughput Approximate Nearest Neighbor Search
Databases
Finds similar items faster in huge data collections.