Skip to content
All posts

pgvector or a dedicated vector database? How a startup should choose

Choosing a vector database for your first AI feature: when pgvector in the Postgres you already run is enough, and the signals that it is time for a dedicated vector store.

Shubham Somani 4 min read

The moment a startup adds semantic search or retrieval to its product, someone asks which vector database to use. It’s tempting to treat it as a big platform decision and start comparing dedicated vector stores.

For most first AI features, the better question is whether you need a new database at all. If you already run Postgres, the pgvector extension stores embeddings and runs similarity search next to the rest of your data. Here’s how we’d decide between pgvector and a dedicated vector database.

What a vector database actually does

Retrieval features turn text (or images) into embeddings: lists of numbers that capture meaning. To answer a question, you embed the question and find the stored embeddings closest to it. A vector database stores those embeddings and finds the nearest ones quickly, usually with an approximate index rather than comparing against every row.

Both pgvector and dedicated vector stores do this. The difference is in everything around it.

The case for starting with pgvector

  • One database, not two. Your documents, permissions and metadata already live in Postgres. Keeping embeddings there means no syncing between systems and no second thing to back up, monitor and secure.
  • Filtering with normal SQL. “Only this customer’s documents, published, in English” is a WHERE clause, joined against tables you already have. Getting permission filters right matters more than shaving milliseconds.
  • Transactions. When a document is updated or deleted, its embeddings change in the same transaction. Stale or orphaned vectors are a common source of wrong answers.
  • Skills you already have. Your team knows how to migrate, index, back up and debug Postgres.

We make the broader version of this argument in why we reach for Postgres first.

Signals you might need a dedicated vector store

None of these is automatic, but each is worth a real look:

  • Very large collections where the vector index no longer fits comfortably in memory on the database you can afford.
  • Vector search load that competes with your core app, so heavy retrieval traffic slows down ordinary queries.
  • Retrieval features you’d otherwise have to build yourself that a dedicated store offers and you’ve confirmed you need.
  • A team that can run it. A second datastore adds operations work. Be honest about who will own it.

Notice that most of these are problems of scale you’ll see coming. That’s why starting simple is usually safe: you can measure the problem before paying for the solution.

Before you switch, check the real bottleneck

When retrieval feels slow or wrong, the database is often not the cause.

  • Wrong answers usually come from chunking, missing filters or the lack of keyword search alongside vectors. Our post on why RAG stops working goes through the usual suspects.
  • Slow responses are often dominated by the model call, not the search. Measure each step before optimising one. See why your LLM feature is slow.
  • Missing indexes or an index that doesn’t match your distance measure can make pgvector look slower than it is.

Keep the switch cheap either way

  • Hide retrieval behind one function in your code, so the storage behind it can change without touching the rest of the app.
  • Store the source text and metadata in Postgres even if vectors move elsewhere, so you can re-embed or migrate later.
  • Keep a retrieval eval set, questions paired with the documents that should come back, so you can prove a migration didn’t make things worse. We explain how in test your LLM feature before users do.

The short version

  • Already on Postgres? Start with pgvector.
  • Filter on permissions and metadata in SQL, and update embeddings in the same transaction as the data.
  • Watch for scale and load signals before adding a second datastore.
  • Fix chunking, filters and model latency before blaming the database.
  • Keep retrieval behind one interface and measure it, so switching later is a project, not a rewrite.

If you’re choosing the stack for your first AI feature, see our fixed-scope offers or book a 30-minute call.