𝗪𝗵𝘆 𝗧𝘄𝗶𝗼 𝗖𝗵𝗼𝘀𝗲 𝗩𝗲𝗿𝘁𝗲𝘅 𝗔𝗜 𝗦𝗲𝗮𝗿𝗰𝗵 𝗼𝘃𝗲𝗿 𝗽𝗴𝘃𝗲𝗰𝘁𝗼𝗿
We built our first RAG system at Twio using pgvector. It was the fast choice. Our data lived in PostgreSQL. Adding embeddings there was easy.
As we scaled, our problem changed. We no longer asked how to store vectors. We asked how to understand thousands of messy broker documents, emails, and attachments.
Twio serves loan brokers. A single case contains: • Email threads • Payslips and bank statements • Loan forms and lender rules • Handwritten notes
The AI must answer questions like: • Which email mentioned the missing requirement? • Does this bank statement support the income? • Summarize all documents for this borrower.
If retrieval is weak, the answer is weak. If parsing is wrong, the model sees the wrong evidence. RAG is the memory of our product.
pgvector worked well for our first version because: • It required no new infrastructure. • It had low costs. • It allowed easy SQL debugging. • It was fast to ship.
But pgvector is just one part of a RAG pipeline. It left the rest to us: • Downloading attachments. • Extracting text from PDFs and scans via OCR. • Chunking documents and generating embeddings. • Designing metadata and retrieval queries. • Tuning indexes and ranking. • Monitoring database load.
A clean PDF is easy. A scanned bank statement is hard. An email with five attachments and tables is even harder. With pgvector, we had to fix every weakness in that pipeline.
The cost shifted from our cloud bill to our engineering time. Engineering time was our most limited resource.
Comparison: • Scanned documents: We build OCR with pgvector. Vertex handles most document processing. • Document questions: We design queries and ranking with pgvector. Vertex provides managed search. • Attachment bursts: Postgres carries the load with pgvector. Vertex keeps the load outside our main database. • Cost: pgvector has lower service costs. Vertex has lower engineering and maintenance costs.
pgvector is cheaper as a database extension. Vertex is cheaper as a product decision.
Vertex helps us in four ways: • Less infrastructure to manage. • Less document-processing logic to maintain. • Postgres stays focused on business transactions. • It scales as our document volume grows.
Vertex is not free. But building our own OCR, indexing, and ranking also has a cost. We pay that cost in engineer weeks.
Use pgvector if: • Your data volume is moderate. • Your documents are already clean text. • You need tight SQL filtering. • You want a fast, low-cost first version.
Our lesson is simple: Start with the tool that helps you learn fastest. Move to the tool that helps you operate best.
Source: https://dev.to/twio_ai/why-twio-chose-vertex-ai-search-over-pgvector-for-production-rag-51jm
Optional learning community: https://t.me/GyaanSetuAi