By Ryze AI Team
Vector databases store embeddings and answer nearest-neighbor queries. They are genuinely useful, but they are also one of the most over-adopted components in modern AI stacks. Most applications do not need one, and adding one early creates operational burden that pays for nothing.
Under roughly fifty thousand vectors, an in-memory list with brute-force cosine similarity is fast and simple. A modern CPU computes cosine similarity over fifty thousand 384-dimensional vectors in a few milliseconds. Libraries such as FAISS, numpy, or even a plain Python list handle this comfortably. Start here, and only move on when measurements tell you to. The same applies to small browser or mobile apps, where a JSON file of vectors and a dot product in JavaScript is often enough.
You need a vector database when you have millions of vectors, frequent updates and deletions, metadata filtering combined with similarity search, multi-tenant isolation, or managed backups and replication. It also helps when you want approximate nearest neighbor search with tunable recall and latency, because brute force stops scaling linearly in a way that matters. If none of these apply, the database is overhead.
Managed services such as Pinecone, Weaviate Cloud, and Qdrant Cloud remove operational work at the cost of vendor lock-in and ongoing spend. Self-hosted options such as Qdrant, Milvus, and Weaviate give control but require you to run and upgrade them. Embedded options such as DuckDB with the vss extension or SQLite with sqlite-vss fit single-process applications and are underrated for small and medium workloads. Supabase pgvector is a strong middle ground if you already use Postgres.
The embedding model matters far more than the database. A good model in a simple store beats a bad model in a fancy one, because retrieval quality is dominated by how well the embeddings capture meaning. Evaluate recall on your own queries before investing in infrastructure. If recall is poor, switching databases will not help; switching or fine-tuning the embedding model will.
Start with an in-memory store behind an interface you control. When you outgrow it, swap the implementation for a vector database without changing the calling code. This keeps the early system simple while preserving the option to scale. Avoid coupling your application logic to a specific database's query language or metadata schema.
Resources: Qdrant documentation and Supabase pgvector.