← All articles

AI & LLMs

Vector databases — search at a billion vectors

2026-07-16 · 6 min read

Once you've turned your data into embeddings, search becomes "find the vectors closest to this one." With a thousand vectors, you just compare against all of them — done. With a hundred million (every paragraph of your docs, every product, every image), comparing against all of them for every query is hopeless. That gap is the entire reason vector databases exist.

The problem: brute force doesn't scale

The obvious method — measure the distance from your query to every stored vector and keep the closest — is exact but linear. Double your data, double the work per query. At a hundred million high-dimensional vectors, a single search reads gigabytes and takes seconds. Multiply by thousands of users and it falls over.

Brute force versus an approximate-nearest-neighbour index Brute force compare to all N · slow ANN index query nearest hop through a few · fast
Brute force checks every vector. An ANN index navigates a graph, hopping neighbour-to-neighbour toward the answer in a handful of steps.

The trick: approximate, not exact

Vector databases give up one thing to win everything else: they return the approximately nearest neighbours, not the provably-nearest. That relaxation unlocks clever index structures — most famously graph-based ones like HNSW, where vectors are linked to their neighbours and a search just "walks" the graph, hopping closer and closer in a handful of steps. Search time goes from linear to roughly logarithmic, and you'd almost never notice the occasional near-miss.

The gist

Don't measure the distance to all hundred million vectors. Follow a map of who's-near-whom and hop to the neighbourhood of the answer in a few jumps. Slightly approximate, enormously faster.

What a vector database gives you

  • Fast approximate nearest-neighbour search at millions-to-billions of vectors — the core job.
  • Metadata filtering — "nearest vectors where category = docs and date > 2025" — combining semantic and structured search.
  • The usual database chores — inserts, updates, deletes, and persistence — so your index stays fresh as data changes.

It's the storage-and-retrieval engine under most RAG systems and semantic search. And it's worth remembering the honest trade underneath: a vector database is a bet that "close enough, right now" beats "perfect, eventually" — which, for search over meaning, is almost always the right bet.

Vector databaseEmbeddingsANNAI
← Back to the blog