Reference architecture

Enterprise RAG Pipeline

The indexing and query paths of a production retrieval system, with the entitlement checkpoint in the only place it can safely sit.

Most RAG diagrams show a single line: documents in, embeddings out, retrieve, generate. That version works until a real corpus with real permissions meets it.

This architecture separates the two paths that actually exist — an indexing path that runs continuously as content changes, and a query path that runs per request under a specific user's identity — and puts access control between the user and the retriever rather than after it.

Enterprise RAG reference architecture An indexing path runs from source systems through parsing, chunking, and embedding into a vector store, capturing access control metadata alongside. A separate query path takes a user query, resolves identity and entitlements at a checkpoint, retrieves only permitted content, reranks, generates, and returns a cited answer. INDEXING PATH — runs continuously Source systems DMS · EHR · ERP Parse & extract tables · scans Chunk semantic Embed Vector store your storage ACL metadata captured at index QUERY PATH — runs per request, per user User query authenticated Entitlement checkpoint resolved at query time Filtered retrieval hybrid search Rerank Generate Cited answer traceable filtering AFTER retrieval is too late — the material has already been aggregated
The entitlement checkpoint sits between the user and the retriever. Access control resolved at query time, against the same source of truth the document system uses.
Components

Component walkthrough

Parse & extract

The single largest determinant of final accuracy. Badly extracted tables and scanned pages cap the ceiling before retrieval is even involved. Format-aware processing, not a generic text splitter.

Chunk

Semantic boundaries beat fixed windows. Optimal size is corpus-specific and must be measured — it is not a constant you can copy from a tutorial.

ACL metadata

Permissions captured at index time and stored alongside each chunk, so the query path can filter without a second round trip to the source system.

Entitlement checkpoint

Resolves the requesting user's permissions and constrains the retrieval query itself. This placement is the whole point of the diagram.

Filtered retrieval

Hybrid lexical and semantic search over the permitted subset. Pure vector similarity underperforms on corpora with part numbers, citations, or heavy jargon.

Rerank

Frequently the cheapest meaningful accuracy gain once retrieval works at all.

Cited answer

Every assertion traceable to a source passage. In regulated settings this is the difference between adoption and abandonment.

Rationale

Decisions and trade-offs

DecisionWhat we do, and why
Filter at query time, not index timeOne index with per-chunk ACLs, filtered per request. Per-user indexes do not scale and go stale the moment permissions change.
Never filter after retrievalPost-filtering means the retriever already read across the boundary. If it also fed a reranker or a cache, the exposure has spread.
Hybrid over pure vectorAdds operational complexity, and pays for itself on any corpus containing identifiers, codes, or domain jargon.
Incremental re-indexingFull rebuilds do not survive a corpus that changes daily. Supersession handling matters more than raw ingest speed.
Citations are mandatoryCosts prompt tokens and engineering effort. Without it users cannot verify, and in regulated work unverifiable output is unusable.
Failure modes

Failure modes this addresses

Entitlement bypass

Retrieval returning content the user cannot open directly — through index-time permissioning, cache leakage, or aggregation across documents.

Confident answers from stale documents

A superseded policy answered as current. Supersession handling in the indexing path is what prevents it.

Indirect prompt injection

Retrieved content carries instructions. Treated as untrusted input, never as instruction, with tool access constrained downstream.

Silent accuracy decay

The corpus drifts, retrieval quality drops, nobody notices. Continuous evaluation against a maintained test set catches it.

Related

Want this sized for your environment?

The assessment produces a reference architecture scoped to your data, infrastructure, and regulatory position — not a generic diagram.

Request an AI Readiness Assessment Discuss Your AI Roadmap

Vendor-neutral. We hold no reseller agreements and take no vendor commissions.