Enterprise AI Reference Architecture: The 8 Layers AWS, Azure and Google Share

Strip the product names out of the three hyperscaler blueprints and the same 8 layers are left standing. AWS, Microsoft Azure and Google Cloud each publish an enterprise AI reference architecture for agents, and they agree on far more than their marketing suggests. The disagreements are narrow, and each one sits exactly where that vendor wants your switching cost to live.

The short answer. An enterprise AI reference architecture from any of the three hyperscalers contains 8 layers. The first four are a single public front door with user identity, a managed agent runtime, a model access layer with throughput planning, and a grounding store. The other four are tools exposed through MCP, conversation state, guardrails on every model call, and tracing plus evaluation. Everything else is optional or vendor preference.

The three AI architecture documents being compared

None of the three providers publishes a single document called "the" AI reference architecture. Each publishes a cluster of pages, and they are written at different altitudes. Comparing them fairly means choosing the closest equivalent from each.

For Azure, the anchor is the Baseline Microsoft Foundry chat reference architecture, updated in August 2026. It is the most concrete of the three. It names every service, subnet and replica count, and it ships with a deployable reference implementation.

For Google, the anchors are two Architecture Center guides: Choose your agentic AI architecture components, last reviewed in April 2026, and the multi-agent reference architecture, reviewed in October 2026. These are decision guides rather than a single build. They describe choices and recommend a default at each one.

For AWS, the anchors are the Well-Architected Generative AI Lens, published in its current form on 19 November 2025, and the Prescriptive Guidance series on agentic AI, plus the Amazon Bedrock AgentCore developer guide. AWS is the least diagrammatic of the three. Its guidance reads as principles and a catalogue of services, not a wiring plan.

A concrete baseline exposes its assumptions. A principles document hides them. Microsoft's page tells you the agent's chat history goes into Cosmos DB, while AWS tells you to "maintain traceability" and leaves the store to you.

That asymmetry makes Azure look the most opinionated. In practice it is just the most honest about what it expects you to buy. Read the other two with the same suspicion and their preferences surface quickly.

One naming warning before you start

Google renamed its platform during 2026. The current Architecture Center pages refer to Gemini Enterprise Agent Platform and Agent Runtime, where older material and most tutorials say Vertex AI and Agent Engine. Microsoft went through the same churn, from Azure AI Foundry to Microsoft Foundry. If a diagram your team inherited uses the old names, the architecture underneath has usually not changed.

The 8 layers every enterprise AI reference architecture shares

Map each provider's components onto function rather than product name and the overlap is striking. The table below is the core of this post. A layer qualifies only if all three providers' guidance names a component that does the job.

Mapping is Zan Digital's own. It was built from the pages listed in the references and read in October 2026. Where a provider's guidance mentions a capability only in passing, the cell names the service the guidance points to.

Layers 1 and 2: one door in, one managed runtime

All three put a single controlled entry point in front of the agent and authenticate the end user there. Microsoft's baseline makes the Application Gateway the only internet-exposed resource, and every other service is reached over a private endpoint.

Behind that door, all three now offer a managed host built specifically for agents. This is the newest layer in every stack. Two years ago the default answer was "put it in a container", and the providers have since converged on a runtime that understands sessions, tool calls and agent versions.

Layer 3: model access is a capacity problem, not a model choice

The designs agree that the model call is a scarce resource you plan for. Microsoft recommends deploying both a provisioned and a standard model in the same data zone and configuring spillover between them. Google's multi-agent architecture names Provisioned Throughput as the fix for HTTP 429 errors.

I think this is the most underrated agreement in the set. Teams spend weeks choosing a model and an afternoon on quota. All three providers are telling you the afternoon is the part that breaks in production.

Layers 4 to 6: grounding, tools and state

Grounding data, tool access and conversation history are separate stores in all three designs. That separation is deliberate. Each has a different retention rule, a different access pattern and a different owner inside your organisation.

The tool layer is where the convergence is tightest. All three providers now treat the Model Context Protocol as the default way an agent reaches a tool. The protocol's wider adoption story is covered in the analysis of MCP as the agent interoperability standard.

Layers 7 and 8: guardrails and evaluation are no longer extras

Every design puts a safety filter on each model call. Microsoft states that each model in your deployment should bind a content-filter policy screening both prompts and completions. Google points to Model Armor and AWS to Bedrock Guardrails.

Evaluation is the layer most teams still skip. Microsoft's baseline says agents behave nondeterministically and asks you to run a test suite of realistic questions before adopting any new model version. That is the right instinct, and the mechanics are set out in the guide to building an LLM evaluation suite.

What all three AI architectures treat as optional

The optional list is as useful as the core, because it tells you what to defer. Every item below appears in at least one provider's guidance as an alternative, an upgrade path, or something to add only when a requirement forces it.

Google's design-pattern guide is explicit: "If you're early in your agent development, we recommend that you start with a single agent". It describes the swarm pattern as the most complex and costly to implement. Microsoft's production baseline is itself a single prompt agent, and lists multiple agents only as an alternative.

I agree with both, and more strongly than they put it. Most of the coordination problems teams attribute to their model are coordination problems they built themselves. The longer version of that argument is in the case that multi-agent systems are overrated.

Microsoft's baseline states plainly that it is not a multiregion solution, and that Foundry Agent Service does not provide built-in disaster recovery. That is an unusual admission in a reference design, and a useful one. It tells you the provider expects most production agent workloads to start in one region.

A2A appears in the Google and AWS material as the standard for agents calling other agents. It does not appear in Microsoft's single-agent baseline, because a single agent has nobody to call. Fine-tuning is similar. Google presents self-hosted open models mainly as an answer to fine-tuning and data residency needs, not as a default.

On private egress control, the designs genuinely differ. Microsoft's baseline requires Azure Firewall as an egress control point. Google's multi-agent guidance recommends a VPC Service Controls perimeter. The AWS pages reviewed for this post leave network design to the wider Well-Architected framework rather than the agent guidance.

Where Azure's AI architecture steers you

Microsoft's lock-in sits in layer 6, agent state. In the standard setup you supply your own Cosmos DB, Storage account and AI Search instance for agent state, chat history and files. Then, in Microsoft's words, Foundry Agent Service "manages them exclusively."

The baseline goes further. It tells you to "never interact directly with the system-managed data" and not to share those resources with other components. Each conversation's messages, tool calls and tool outputs land in a database you pay for and are told not to touch.

Conversation history is the asset that makes an agent better over time. It feeds evaluation sets, audit trails and fine-tuning data. A store you own on paper but cannot read directly is a store you will struggle to migrate.

The fix is not to avoid Foundry. It is to keep your own copy. Microsoft lists client-managed chat history as an alternative, and I would choose it for any workload expected to outlive the current platform generation.

The second steer: the gateway you have to build

The baseline notes that Foundry has no round-robin routing or circuit breaking across model deployments, and no built-in blue-green or canary rollout. It suggests building a gateway with API Management. That gateway becomes a Microsoft-specific routing layer sitting in front of every model call you make.

Where Google's AI architecture steers you

Google's lock-in sits in layers 3 and 5: the model and the framework. The components guide says "we recommend that you use ADK" and "we recommend that you use Gemini Pro". It concedes that ADK is "optimized for Gemini models and Google Cloud".

To its credit, Google also says ADK is built for compatibility with other frameworks and can be used with any model or deployment method. The steer is in the defaults, not in a hard restriction.

The runtime split that forces a choice

The constraint that matters is in the runtimes. Google's guide notes that Cloud Run and GKE cannot host Gemini, while its managed Agent Runtime is Python only and cannot host custom MCP servers. So the most portable runtime cannot run Google's flagship model locally, and the most integrated runtime limits your language.

For API management the guide recommends Apigee API hub, and for long-term memory it points to Memory Bank. Each is a reasonable product. Together they make a stack where every layer above the runtime has a Google default.

Where AWS's AI architecture steers you

AWS makes the loudest neutrality claim of the three. Its AgentCore guide states that AgentCore Runtime works with any foundation model in or outside of Amazon Bedrock, naming OpenAI, Gemini, Claude, Nova, Llama and Mistral models. It lists Google's own ADK among supported frameworks.

That openness at the runtime layer is real. The lock-in moves to layers 5 and 7, the tool gateway and the policy engine.

AgentCore Gateway converts your APIs, Lambda functions and existing services into MCP-compatible tools. AgentCore Policy then intercepts every tool call before execution, using rules written in natural language or in a policy language AWS calls Dogwood, described as compatible with Cedar.

Authorisation rules are the hardest artefact in an agent stack to port, because they encode what your business allows. Write them in a portable form first and compile them into the vendor's engine, not the other way round. The identity side of that problem is covered in the piece on agent identity and non-human access.

The framework comparison is not neutral

The AWS Prescriptive Guidance comparison table rates Strands Agents, the open-source SDK AWS first released, as "Strongest" on AWS integration. It rates AutoGen and CrewAI as "Weak" on the same row. The rating is accurate on its own terms. It also measures frameworks on a criterion only one of them was designed to win.

The vendor-neutral AI architecture skeleton

Take the 8 shared layers and keep the portable option at every point where a provider steers. The result is a skeleton you can build on any of the three clouds. It is not an anti-cloud design. It is a design that keeps the expensive decisions reversible.

Own 4 things: the model gateway, your tool servers, conversation state, and policy. Each one holds either your data or your rules, and each is where a provider placed its strongest default. Owning them costs you some managed convenience. It buys you the ability to move.

The model gateway matters most. A thin routing layer in front of every model call lets you add spillover, swap providers and enforce budgets without touching agent code. The cost of not having one is set out in the analysis of model switching costs.

Rent the front door, the runtime, the retrieval index and the tracing backend. A retrieval index can be rebuilt from source documents in hours. A runtime that hosts a container can be replaced by another one. These layers are commodities wearing product names.

Microsoft's baseline asks you to require and log approval for high-risk operations, and Google lists human-in-the-loop as a named pattern. Place that step in layer 7, beside tool policy, so approvals are enforced in one place. The design choices involved are covered in the human-in-the-loop architecture guide.

Where this argument is weakest

This comparison has four soft spots, and you should weigh them before redrawing your diagram.

First, the documents are not like for like. Microsoft published a deployable baseline. Google published decision guides. AWS published principles plus a service catalogue. Comparing a concrete build against a menu flatters the menu, because a menu never has to commit to a database. Some of Azure's apparent lock-in is simply Azure being more specific.

Second, the pages move faster than this post. Google reviewed its multi-agent guide in October 2026, two days before this post was written. The AgentCore developer guide carries no publication date and now lists services such as Payments and Registry that did not exist in earlier versions. Any cell in the tables above can be wrong within a quarter.

Third, portability has a real cost. The strongest counter-argument is that neutrality is a tax. Every owned layer is a service your team runs, patches and pages someone about at 3am. For a team of 5 shipping one agent, accepting the provider's defaults on all 8 layers may be the right trade, and I would not argue them out of it.

The test is how long the workload lives. A pilot that will be rebuilt in a year gains little from portability. A system that will hold three years of customer conversations gains a lot.

Fourth, the lock-in classification is judgement. The dark and light cells in the matrix are an editorial call made from the guidance text, not a measured score. A reasonable architect could mark Azure's guardrail layer as portable or Google's runtime layer as locked. The method is stated so you can disagree with it.

Frequently asked questions

Where to start this week

Take your current agent diagram and label every box with one of the 8 layer numbers. Any box without a number is either an optional component you added early or a gap. Both are worth knowing about before your next architecture review.

Then circle the 4 owned layers and ask one question of each: if we moved clouds next year, could we export this in a day? Wherever the answer is no, that is your lock-in, and it is now a decision you made on purpose rather than one the default made for you.

Related reading