Reference architecture

Enterprise AI Platform

The layer stack that lets many teams ship AI without each one re-solving architecture, security, and operations.

A platform is what replaces six teams building six stacks. Its value is not any single layer — it is that governance, identity, and observability become properties of the substrate rather than things each team is asked to remember.

The layers below are ordered by how expensive they are to change. Infrastructure decisions outlive application ones by years, which is why they are made first and deliberately.

Governance spans every layer rather than sitting on top. Guardrails and cost attribution live at the gateway, where an individual team cannot forget to apply them.

Components

Layer walkthrough

Infrastructure

GPU compute, Kubernetes or OpenShift, storage for models and indexes, and the network isolation your security position depends on. Hardest to change later.

Serving

Model serving with autoscaling, continuous batching, and quantization where it pays. Scale-to-zero matters more than peak throughput for most enterprise workloads.

Gateway

The control point. Authentication, model routing, rate limiting, guardrails, cost attribution, and audit logging — applied once, for everyone.

Data & retrieval

Ingestion, indexing, and entitlement-aware retrieval offered as a service. Each team building its own means each team writing its own permission bugs.

Application

Product team code, built on SDKs and templates. Teams should be writing product logic, not inference plumbing.

Governance

Inventory, risk classification, approval workflow, evaluation standards, and audit evidence. Spans every layer, which is why it works.

Rationale

Decisions and trade-offs

Decision	What we do, and why
Gateway as the control point	One place for policy, cost, and audit. The alternative is trusting six teams to each implement it identically, which does not happen.
Paved paths, not gates	Make the secure route the easy route. A platform that only blocks gets routed around, and shadow AI is worse than governed AI.
Retrieval as a service	Centralizing it concentrates the hardest correctness problem — entitlements — in one reviewed implementation.
Run on existing Kubernetes	AI becomes a workload on infrastructure you already operate, rather than a parallel stack with its own on-call rotation.
Cost attribution from day one	Retrofitting per-team attribution is painful. Without it, nobody owns spend and nobody can justify the platform.

Failure modes

What this prevents

Six stacks, six reviews

Each use case restarting the architecture and security argument from zero, taking as long as the first one did.

Unattributable cost

AI spend growing with nobody able to say which workflow drives it or whether it returns value.

Inconsistent guardrails

Safety and policy controls applied well by one team and forgotten by another.

Shadow AI

Teams routing around a platform that only says no, ending up with less governance than before it existed.