Cogniterm.ai
← Back to blog

The Case for Shared-Nothing Architecture in AI SaaS

July 25, 2026 • Expert RAG Engineering Team

Traditional SaaS platforms usually follow a multi-tenant pattern: one giant database table containing rows for thousands of customers, separated only by a tenant_id column. While this pattern worked well for web 2.0 CRUD apps, it introduces major flaws when applied to domain-specific RAG applications.

In Expert RAG, we rejected traditional multi-tenancy in favor of a shared-nothing vertical architecture.

The Flaws of Multi-Tenant RAG

1. Vector Search Leakage Risk

Vector search queries execute nearest-neighbor algorithms in high-dimensional vector space. Adding strict WHERE tenant_id = 'xxx' filters onto high-dimensional vector indices often degrades ANN (Approximate Nearest Neighbor) recall or causes performance bottlenecks. A single bug in a database row-level security policy could result in proprietary clinical or legal documents being cited across tenant boundaries.

2. Ingestion Pipeline Noisy Neighbors

Document ingestion is compute-intensive. Docling conversion and embedding generation require significant CPU and memory spikes. In a shared-tenant ingestion queue, one tenant uploading 10,000 PDF documents can stall ingestion jobs for all other operators on the platform.

3. All-or-Nothing Downtime

If a multi-tenant database undergoes maintenance or experiences a schema corruption, every single customer site goes offline simultaneously.

The Shared-Nothing Solution

In Expert RAG, every vertical site (med-expert, vet-expert, etc.) operates as a self-contained deployment unit:

  • Isolated Database: Dedicated PostgreSQL database and vector table per vertical.
  • Isolated Compute: Dedicated FastAPI web service and worker process.
  • Isolated Storage: Dedicated object storage bucket.
  • Isolated Config: Single vertical.toml defining domain branding, prompt persona, and model choices.
med-expert stack ──► [CF Pages] ──► [Railway Web/Worker] ──► [Supabase Med DB]
vet-expert stack ──► [CF Pages] ──► [Railway Web/Worker] ──► [Supabase Vet DB]

Benefits for Operators

  1. Uncompromised Data Boundaries: Cross-vertical data contamination is physically impossible at the network and storage layer.
  2. Predictable Performance: Heavy usage on one vertical has zero impact on latency or response times of another.
  3. Independent Lifecycles: Operators can upgrade, re-index, or migrate individual verticals independently without platform-wide maintenance windows.

Shared-nothing architecture allows operators to run high-trust expert sites with complete peace of mind.