LanceDB loaded 100k OpenAI embeddings 22 times faster than pgvector, while pgvector answered the same workload from eight simultaneous clients 1.8 times quicker. The gap in single-thread latency and storage efficiency also tilted in LanceDB’s favor, giving developers a data-driven way to pick a vector store.

Why a benchmark matters now

Vector search has moved from research labs to production services such as recommendation engines and retrieval-augmented generation (RAG). Most teams already run PostgreSQL, so the pgvector extension promises similarity search without new infrastructure. Dedicated stores like LanceDB, however, claim lower latency and cheaper storage. Teams must choose between “add it to what we have” and “run a purpose-built engine,” a decision that impacts cost and performance as datasets grow and request rates climb.

How the test was set up

Both systems indexed the same 100k vectors, each 1536 dimensions, generated by OpenAI’s embedding model. We measured ingestion speed, disk usage, single-thread query latency and throughput with eight concurrent clients.

Head-to-head results

  • Ingestion speed – LanceDB recorded a 22× advantage.
  • Disk footprint – LanceDB stored the vectors in roughly one-third the space pgvector used.
  • Single-thread latency – Queries ran about twice as fast on LanceDB.
  • Concurrency scaling – With eight parallel clients, pgvector delivered 1.8× higher throughput than LanceDB.

Architectural roots of the differences

LanceDB is an embedded library that runs inside the Python process that hosts the application. All operations stay in-process, so data never crosses a network boundary and the index updates with minimal overhead. This design shines for single-task workloads but hits a ceiling when multiple Python threads contend for the Global Interpreter Lock (GIL), which blocks true parallel execution of Python bytecode.

pgvector extends PostgreSQL on the server side. Each client connection launches a separate server process, sidestepping the GIL entirely. The PostgreSQL planner decides how to satisfy a similarity search, and the server can spin up many processes to serve concurrent requests. This isolation explains the better scaling under load.

Filtering and query planning quirks

Real-world RAG pipelines often combine vector similarity with traditional filters (e.g., WHERE user_id = 42). LanceDB applies a prefilter that behaves predictably across runs. pgvector relies on PostgreSQL’s query planner, which may choose a fast index scan or fall back to a slower exact scan depending on statistics. Running ANALYZE after bulk loading a pgvector table refreshes those statistics; without it, recall can drop to near zero, effectively breaking the search.

When each option makes sense

Pick pgvector if

  • Your stack already includes PostgreSQL and you want to avoid adding another service.
  • You expect many simultaneous users or API calls.
  • ACID guarantees and familiar DBA tools matter.

Pick LanceDB if

  • Your workflow is an ML pipeline that frequently ingests new embeddings.
  • You need the fastest write path and low latency for single-request agents (e.g., chat bots).
  • Disk cost is a concern and you can tolerate the single-thread performance ceiling.

Bottom line: If raw ingestion speed, minimal storage, and single-request latency matter most, LanceDB wins. If you must serve many users at once and rely on an existing PostgreSQL deployment, pgvector’s concurrency edge makes it the safer bet. Use the numbers from this benchmark to match the store to your product’s most critical metric.