Multi-tenant RAG: isolating client data in AI agents

The problem When you build an AI agent for one client, RAG is straightforward: embed their docs, store them, query them. But the moment you take a second client, the simple version breaks. If you throw all documents into one vector store and rely on a client_id filter, you're one missed filter away from leaking one client's confidential data into another's answers.

What I built Melow is a multi-tenant AI agent platform. It's live in production running a dental Clinical Copilot that converts natural-language observations into structured SOAP notes and odontograms. Each client has their own documents, their own WhatsApp integration, and their own widget. The core requirement: absolute isolation between tenants.

The naive attempt and the failure I started with a single Qdrant collection with a client_id payload field. It felt elegant: one store, one embedding model, filtered queries. Then a practice reported a response that contained a sentence from another practice's documentation. I traced it to a query that had lost its client_id filter somewhere in the conversation flow. The vector similarity was high enough that the wrong text got pulled. That's a trust violation that ends business.

The fix: separate collections and hard enforcement I moved to a model where each tenant gets its own collection in Qdrant — literally named after the tenant ID. A middleware resolves the tenant from the request context before any database call, and it uses that tenant's collection directly. If the tenant can't be resolved, the call throws a 500 instead of defaulting to a shared pool. That means a query from tenant A physically cannot return vectors from tenant B. The collection name is the boundary — there's no filter to forget.

The tradeoff Per-tenant collections use a bit more storage, but each client's documentation is typically a few hundred pages, so the overhead is negligible. The real cost is operational: you have to manage collection lifecycle (create on signup, delete on churn) and keep the schema versioned. But isolation is worth it — a data leak would destroy the product.

Testing it I wrote 33 unit tests over the LLM service and the conversation engine. Every test that hits the vector store now asserts that the expected tenant collection was used. The test suite would have caught the missing-filter bug if it had been in place. I also bound Qdrant to localhost so the vector database is never exposed to the network. The whole thing runs on a single VPS behind Nginx and PM2.

The lesson Isolation isn't a filter you add — it's a boundary you enforce at the data access layer. Filters fail silently; separate collections fail loudly. If you're building multi-tenant AI agents, start with per-tenant storage from day one.

Building something like this? Tell me what you need and I'll scope it.