What is metal to agents? Navigating the architecture of enterprise AI

Enterprise AI has reached a critical turning point as organizations move from isolated, chat-based experimentation to complex reasoning models and autonomous agents. While the productivity gains are real, underlying infrastructure costs are escalating rapidly, with token consumption predicted to increase 24x by 2030. Relying entirely on external clouds and proprietary APIs means your expenses scale directly with business growth, introducing major predictability and cost issues. 

When technology moves this fast, treating it as a series of disconnected software tools creates unnecessary technical debt. Bespoke scaffolding creates fragmented solutions that fail to scale. At Red Hat Summit 2026, Red Hat CTO Chris Wright introduced a unified framework to address this complexity called metal to agents. We look at this framework as an open, collaboratively-developed standard AI operating system (AI OS). Decades ago, we built a standard version of Linux across fragmented server environments to give customers architectural flexibility. Building a standard AI OS does the same thing for the era of intelligent applications. 

This integrated, open source stack spans from physical hardware and accelerators in the data center up to the autonomous AI agents delivering business value. It establishes a common platform for managing and optimizing AI inference workloads at scale, supporting a "train once, infer repeatedly" approach. Controlling the full stack on hybrid infrastructure helps your organization transition from being a token consumer to becoming your own token provider, helping you regain control over inference costs and platform governance. 

Defining the hybrid cloud foundation and software stack

The metal to agents framework organizes enterprise AI into 4 interconnected software layers built on a flexible hardware foundation, with "metal" (hardware) at the bottom and "agents" (agent services) at the top. This structure helps organizations avoid vendor lock-in and they're able to swap components as the market evolves. 

1. The hardware foundation: Hybrid cloud and accelerator choice

Before defining the software stack, an enterprise must establish its physical deployment strategy. Red Hat AI Enterprise is designed to run on any major accelerator architecture—including hardware from NVIDIA, AMD, and Intel—as well as custom silicon provided by cloud vendors within Amazon Web Services (AWS), Google Cloud, and Microsoft Azure. The platform includes expanded certification for hardware like NVIDIA Blackwell Ultra and AMD MI325X accelerators.

2. AI infrastructure: Creating a solid foundation

The lowest software layer belongs to IT operations, site reliability engineering (SRE), and platform engineering teams. Modern AI runs on Linux and scales using Kubernetes, making infrastructure stability critical before deploying any models.

Operating system reliability

Red Hat Enterprise Linux (RHEL) provides the stable, security-focused foundation required to protect the lifecycle of data science workloads.

To provide operational reliability, RHEL delivers 3 core capabilities:

Day 0 hardware support

Hardware innovation cycles frequently introduce new architectures, including accelerators from NVIDIA, AMD, and Intel. To prevent deployment delays, Red Hat co-engineers directly with hardware partners so organizations can take advantage of next-generation hardware to achieve immediate high-performance stability upon general availability.

Immutability via image mode

Red Hat Enterprise Linux AI (RHEL AI) standardizes scaling across hybrid clouds by using an image-based deployment model. Packaging the operating system kernel, hardware-optimized drivers, and core data science libraries into a single bootable container image reduces configuration drift. This approach provides a consistent and reliable environment from local developer laptops to production data centers.

Confidential computing from silicon to agent

Operational reliability requires data protection that works with autonomous AI workflows. RHEL integrates with hardware-level security frameworks, extending Trusted Execution Environments from the CPU to the GPU. This provides end-to-end cryptographic protection, isolating model weights and corporate data streams against unauthorized access.

Workload orchestration

Red Hat OpenShift functions as the unified Kubernetes platform for virtual machines (VMs), containers, and AI workloads. OpenShift native network isolation controls exactly what resources an AI tool can reach. Furthermore, using the Red Hat build of Kueue, the platform automatically shares and queues tasks for expensive accelerators across multiple teams. This dynamic allocation prevents graphics processing units (GPUs) from sitting idle between inference calls, helping maximize hardware use.

3. Inference services: Scaling token economics

Inference is the layer where performance, scalability, and operational costs intersect. Getting inference wrong leads to low accelerator efficiency, high latency, frustrated users, and unpredictable cloud bills.

To manage token economics at scale, the architecture separates inference serving from basic application logic. Red Hat uses vLLM, an open standard for inference serving where Red Hat is a primary commercial contributor. This standardized approach drives immense real-world savings, for example, Google Cloud recently deployed vLLM for a global digital media provider, achieving a 92% reduction in costs for trust and safety workloads.

Enterprise AI requests fluctuate and often exceed the capacity of a single GPU, so Red Hat engineered llm-d, an open source distributed inference framework, that intelligently routes and schedules requests across available hardware infrastructure. This coordination reduces latency and increases throughput, delivering 3 times more output tokens and 10 times faster time to first token (TTFT) compared to unoptimized setups. Today, this optimized inference architecture is helping enterprises achieve millisecond-scale processing for industrial edge automation, and allowing institutions like BNP Paribas to manage over 300 GPUs to deliver Models-as-a-Service (MaaS) at scale.

4. Model services: Centralizing management and data integration

Given the sheer volume of new open weight and proprietary LLMs appearing weekly, developers require choice without cobbling together their own stack or resorting to shadow AI. The model services layer enables enterprises to establish central governance over how models are served to internal development groups. 

When you pair this layer with Red Hat OpenShift AI, an integrated AI gateway gives platform administrators a single control plane to handle token quotas, credentialed access, request priorities, and precise token rate limiting from one centralized location. This means if a data science team needs to swap an older model for a more efficient option, they can make that change globally without touching individual application code bases. 

To eliminate deployment guesswork, the validated models by Red Hat AI program provides pre-optimized, production-ready compressed versions of major models—such as gemma, gpt-oss, Kimi, Llama, MiniMax, Mistral, Nemotron and Qwen—available directly through the Red Hat OpenShift AI model catalog. This layer also integrates essential data-to-model tools, including a Python repository featuring hardened versions of utilities like Docling and Training Hub, so teams can move from fragmented experimentation to repeatable production pipelines.

5. Agent services: Governance for autonomous workflows

At the top of the metal to agents stack sit AI agents. Unlike standard chat interfaces, agents are able to reason, plan multistep actions, call external tools, and collaborate with other agents to complete complex tasks. They operate through looping workflows, running continuous cycles of execution and feedback until a task is finished. 

However, because developers are adopting various independent frameworks like LangChain, OpenClaw, or CrewAI, organizations are facing rapid platform sprawl. This delta between an agent working on a developer laptop and one running at scale in a data center is known as the production gap. Unmanaged agent deployments create security and compliance risks, and can result in unexpected spikes in infrastructure usage.

The metal to agents framework addresses this challenge through Red Hat’s bring your own agents approach governed by AgentOps. This management layer standardizes agent deployments across the enterprise by delivering:

• Cryptographic identity: Every autonomous agent receives a verified, authorized identity using SPIFFE/SPIRE to enforce least-privilege access to sensitive enterprise assets.
• Lifecycle management: IT operations teams can version, update, and roll back agents exactly like traditional enterprise applications, managing them as standardized systems rather than siloed projects.
• Universal tool integration: Native support for the Model Context Protocol (MCP) gateway acts as a universal adapter, connecting agents to enterprise Software-as-a-Service (SaaS) APIs and corporate databases without requiring custom-code maintenance.
• Kernel-isolated sandboxes: To protect shared infrastructure from runaway or malicious code execution, the platform restricts agent execution to protected kernel-isolated boundaries.

• Observability: OpenShift AI provides full execution tracing and monitoring to track agent interactions and debug unpredictable behaviors, turning chaotic software sprawl into a more reliable Agents-as-a-Service (AaaS) model.

The strategic value of a metal to agents framework

Adopting a metal to agents framework is an architectural decision that directly impacts an organization's competitive advantage and bottom line.

Shifting from token consumer to token provider

Relying completely on external commercial APIs means your operational expenses scale directly with your business. Agents use looping workflows that consume significantly more tokens than standard AI models, so inference  costs can escalate quickly. By controlling the full stack on hybrid infrastructure, your organization becomes its own token provider—hosting optimized, self-hosted models that drastically lower the total cost of ownership (TCO).

Mitigating architectural sprawl

Without a unified platform, different business units will inevitably deploy siloed agent platforms driving up operational cost and fragmentation. A standardized metal to agents architecture provides the flexibility developers want while preserving the security protections, access controls, and tracking compliance organizations require.

Future-proofing against market volatility

The timeline for open source models to match proprietary models has shrunk significantly. By building your AI strategy on a modular, open foundation, you are creating infrastructure that can adapt to new technological developments without requiring a complete architectural overhaul.

Successful AI initiatives require looking at the entire environment holistically. Red Hat AI Enterprise provides the continuity needed to manage choice, stabilize costs, and develop protected autonomous workflows from the physical silicon up to the digital agent.

Learn more

• Explore the Red Hat AI Enterprise product overview
• Read the Forrester study: The Total Economic Impact™ Of Red Hat AI
• Watch the webinar: Metal to agents: Bridging the production gap for agentic AI