Demystifying agentic AI: How to build production-ready AIOps with open source models

In many cases, using agentic AI for incident response automation means sending infrastructure logs to frontier AI models. Every job failure log (complete with hostnames, IP addresses, and system topology) would leave infrastructure the moment it hit a large language model (LLM) endpoint, which would raise huge red flags for compliance teams in heavily regulated industries like financial services and healthcare.

As an alternative, these organizations could use open source models hosted on their own infrastructure, addressing data residency and compliance problems. But are open source AI models good enough to replace massive frontier models for structured operational tasks? We think they are. Pairing open source models with the right architectural patterns (agentic harnesses, focused skills, context isolation) gives them the ability to deliver frontier-quality analysis at a fraction of the cost.

Here we kick off part 1 of our 3-part series showing you exactly how to build and scale production-ready agentic AIOps using open source models. We break down why open source is ready for the challenge, how Red Hat OpenShift AI simplifies day-to-day operations, and how a skills-driven architecture cuts out the need for tedious model retraining.

The challenge: When operations scale faster than teams

Before we talk about solutions, here's the operational reality that drives organizations toward AI-augmented incident response.

The business problem

For our example use case we have a UK-based financial services firm operating hybrid infrastructure: 140 on-premise Red Hat Enterprise Linux (RHEL) virtual machines (VMs) and 3 Red Hat OpenShift clusters. They run 600+ Red Hat Ansible Automation Platform jobs per week for provisioning, patching, compliance scanning, and configuration drift correction.

When jobs fail, a ticket lands in their ticketing system and a notification fires in their ChatOps channel. Sounds manageable, except they're handling approximately 40 failure tickets per week. Each requires manual triage, investigation, and either remediation or escalation.

Time cost: 45-90 minutes of site reliability engineer (SRE) time per ticket, depending on complexity.

Scale challenge: Multiply that across the team, factor in overnight failures and on-call rotations, and you have a systemic operations problem. The business wants to handle 10x the current infrastructure without 10x the headcount.

Regulatory pressure: As a financial services firm, they operate under strict change management and incident response requirements. Every unplanned outage or configuration drift must be documented (root cause, remediation steps, responsible party) within defined service level agreement (SLA) windows. Missing those windows triggers regulatory penalties.

The regulator flagged 3 SLA breaches in Q1. Each traced back to a failed Ansible Automation Platform job that sat unactioned too long. The directive was clear, "We need to close the loop faster, and we need audit evidence that we're doing it."

Current pain points:

• Alert fatigue: 40 failure tickets per week, many repeating known patterns. On-call teams are overwhelmed, response times are slow.
• Knowledge fragmentation: Runbooks and domain knowledge are scattered across Confluence and individuals. Resolution quality is inconsistent.
• Audit overhead: Manual incident reports for compliance take 2-3 hours per major incident, often written after the fact from memory and chat logs.
• Escalation gap: Binary choice between "fix it yourself" and "wake someone up," with no graduated, proportional response mechanism.
• Scalability ceiling: Current team can't handle projected growth. Hiring more SREs isn't economically viable and doesn't solve knowledge fragmentation.

The opportunity: Agentic AI can triage, investigate, and act (with a full audit trail) while keeping humans appropriately in the loop for high-risk changes.

Projected outcomes if successful:

• Mean time to resolution (MTTR) reduction: From 45 minute average to target 12 minutes for known failure patterns.
• Audit efficiency: Auto-generated incident reports, compliance team reviews instead of authoring from scratch.
• Scalability: Handle 10x ticket volume without proportional headcount growth.
• Knowledge capture: Agent patterns distill runbook knowledge into a self-improving operational knowledge base.
• Cost control: Open source models on owned infrastructure versus per-token billing to external providers.

The technical challenge for SREs

From an SRE perspective, the problem is operational velocity versus operational safety.

Volume and variety: Failures span multiple domains. Ansible Playbook errors, Linux package management issues, OpenShift pod lifecycle problems, networking DNS/SSH failures. A single agent can't be expert in everything.

Context requirements: Good incident analysis requires institutional knowledge. When a package installation fails with "No package python3.14 available," the generic answer is "check your repository configuration." The organization-specific answer references:

• Your Satellite server topology (primary, disaster recovery, sync schedule)
• Your per-team content view model (which teams get which repos)
• Your escalation process (standard operating procedure (SOP) reference, fast-track Slack channel)
• Whether the package exists at all for your operating system (OS) version

Generic troubleshooting wastes time. Context-aware analysis gets to resolution faster.

Compliance constraints: In regulated environments, every action must be traceable. An AI agent that acts without logging is a compliance liability. One that logs everything (classification rationale, specialist analysis, confidence scores, reviewer verdicts) becomes a compliance asset.

Auditors from the Financial Conduct Authority review change management and incident response records regularly. Missing an SLA window doesn't just mean a late fix—it triggers regulatory penalties and mandatory remediation reports. Today, those reports are written manually (2-3 hours per major incident, often reconstructed after the fact).

Data sovereignty requirements: Every API call to a frontier model sends your data somewhere else:

{
  "job_id": "3847",
  "host": "rhel-node-01.meridian.internal",
  "error": "fatal: [rhel-node-01]: FAILED! => {\"msg\": \"No package python3.14 available\"}",
  "playbook": "/var/lib/awx/projects/ansible-plays/install-python.yml",
  "credential": "meridian-satellite-sync"
}

This payload exposes:

• Internal DNS topology (meridian.internal)
• Infrastructure naming conventions
• Credential identifiers (meridian-satellite-sync)
• System configuration details

In regulated environments under GDPR (Article 44-49 on international data transfers) and DORA (Digital Operational Resilience Act), this creates real data residency risk.

Operational requirements: The system must:

• Triage and classify failures accurately (route to the right specialist)
• Perform domain-specific root cause analysis (not generic troubleshooting)
• Generate audit-ready incident reports (compliance-friendly output)
• Support graduated autonomy (human-in-loop for high-risk, human-out-of-loop for routine)
• Scale to handle 10× volume without degradation
• Allow rapid knowledge updates (new SOPs, infrastructure changes) without model retraining

The data sovereignty problem

63% of large enterprises across EMEA cite sovereignty concerns as the greatest barrier to cloud adoption. For AI workloads processing operational data, the stakes are even higher.

Red Hat believes that open source is the foundation of sovereign AI. Open models running on Red Hat OpenShift AI give you control. You choose where inference runs, your data never leaves the platform, and you maintain full auditability of the AI supply chain.

This isn't a niche concern. Over two-thirds of EMEA enterprises have identified data sovereignty as a top IT priority for the next 18 months. For financial services firms, sending job output to external LLM providers creates regulatory exposure that compliance teams won't accept.

The capability gap is closing

Two years ago, choosing an open source model meant accepting a significant capability gap. A 7B parameter open model would hallucinate. A 70B model would struggle with complex reasoning. Frontier models were the only viable choice for production.

That's no longer true as there are much larger open source models (230B and beyond) available now.

The agentic revolution changed the equation

The rise of agentic AI (frameworks like LangChain's Deep Agents, Anthropic's Claude Agent SDK, and custom harnesses) transformed how we think about model capability. An agentic harness doesn't just send a prompt and hope, instead it:

• Manages context windows, keeping only relevant information in focus.
• Delegates to specialists, spawning focused subagents for specific domains.
• Provides structured skills, reference knowledge loaded as markdown.
• Parses and validates outputs, providing structured, actionable results.

This structured approach could help bridge the capability gap between open source and frontier models. A well-designed harness allows medium-sized open source models to perform very well for a given task at hand.

Example: Package management failure analysis

Here's a real comparison from our production AIOps pipeline. Same failure, 2 different models:

Frontier model (claude-sonnet-4-6):

Root Cause: DNF package manager reports "No package python3.14 available"
Recommended Action: 
1. Verify package name is correct
2. Check repository configuration
3. Consider using EPEL or enabling additional repos
Confidence: 85%

Open source model (qwen3-235b running on MaaS):

Root Cause: DNF package manager reports "No package python3.14 available" 
Analysis: Python 3.14 requires CRB (CodeReady Builder) and EPEL repositories.
Target host group indicates AI/ML team workload.
Recommended Action:
1. Submit Content View Request per SOP v2.3
2. Request CRB + EPEL enablement for aiml-workloads content view
3. Escalate via #platform-satellite fast-track (2-hour SLA)
Confidence: 92%

The open source model produced a better response by using an AI skill (provided as a markdown document encoding institutional knowledge about our Satellite infrastructure, content view model, and escalation paths). The model didn't need to learn this from training data. We gave it the answer as context.

Skills do the heavy lifting

Here's what that skill looked like:

---

name: analyze-package-management-failure

description: Package management failure analysis with institutional knowledge

---
# Analyze Package Management Failure
## Institutional Knowledge: Satellite Infrastructure
**Content View Model (per-team):**
- base-rhel9: All RHEL servers, core OS packages only
- ops-tooling: SRE team, monitoring, diagnostic tools
- aiml-workloads: AI/ML team, Python 3.11+, CUDA, data science libs (CRB + EPEL enabled)
**Escalation Path:**
- Standard: Content View Request SOP v2.3
- Fast-track: #platform-satellite channel (2-hour SLA for AI/ML workloads)

The model provided the reasoning engine. The skill provided institutional knowledge. Together, they delivered analysis that exceeded frontier quality because we encoded our specific operational context.

The cost equation

Frontier model pricing compounds quickly in an operations environment:

• Per-token billing: Every incident analysis involves multiple LLM calls (classification, specialist reasoning, review).
• Volume scaling: At 40+ failures per week, token costs become a significant line item.
• No cost ceiling: Usage spikes during incidents mean unpredictable bills.

Open source models running on your own infrastructure have a fixed compute cost:

Frontier Model Cost Model:
- $15 per million input tokens
- $75 per million output tokens
- Average incident: ~50K tokens (classification + analysis + review)
- 40 incidents/week × 52 weeks = 2,080 incidents/year
- Annual cost: ~$7,800 in LLM API fees alone
Open Source Model Cost Model:
- Fixed GPU infrastructure (Nvidia A100 or H100)
- Red Hat OpenShift AI subscription
- No per-token billing
- Predictable annual cost regardless of incident volume

When you're processing hundreds of incidents per week, the economics shift decisively toward open source.

Red Hat OpenShift AI Model-as-a-Service (MaaS)

Red Hat OpenShift AI provides Model-as-a-Service (MaaS), a portfolio of open source models running on OpenShift with vLLM inference acceleration across different GPUs.

Here are examples of the models available with MaaS:

In production, you don't use one model. You can mix models per agent:

• ops_manager (orchestrator): qwen3-235b (needs strong classification and delegation)
• sre_linux (specialist): gpt-oss-120b (focused domain, structured output)
• reviewer(validator): gpt-oss-20b (simple coherence check)

This layered approach optimizes cost while maintaining quality. Your triage agent doesn't need 235B parameters to check if a log contains "No package X available."

The architecture: Second generation agents

What makes open source models work for complex operational tasks is the harness, the framework that wraps and drives the LLM rather than letting it run unsupervised.

First versus second generation agents

First generation agents (classic ReAct pattern) struggle with multistep operations:

• Context limited to conversation history
• Performance degrades after ~50 tool calls ("dumb zone")
• No persistent state outside conversation
• Every task shares one context window

Second generation agents (Deep Agents, Claude Agent SDK) use four architectural pillars:

1. Explicit planning (to-do lists as persistent context anchors)
2. Persistent filesystem (virtual workspace for notes and structured artifacts)
3. Sub-agent spawning (delegating to specialists with clean contexts)
4. Context isolation (each specialist sees only what it needs)

Example: Incident response pipeline

Our AIOps pipeline uses the Deep Agents pattern:

ops_manager (long-running Deep Agent)
  ├── Receives AAP2 job failure webhook
  ├── Classifies failure domain (ansible/linux/openshift/networking)
  ├── Spawns specialist subagent (ephemeral)
  │   └── sre_linux (fresh context)
  │       ├── Loads skills: analyze-linux-failure, create-ticket, common
  │       ├── Analyzes incident data
  │       └── Returns structured TicketPayload
  ├── Spawns reviewer subagent (ephemeral)
  │   └── reviewer (validates ticket coherence)
  └── Submits ticket to Kira + notifies Rocket.Chat

Each sre_* subagent is ephemeral (created on demand, destroyed after use). This means:

• Context isolation: A package failure doesn't pollute the context of an SSH failure
• Focused expertise: Each agent loads only its domain-specific skills
• Clean state: Every analysis starts fresh, no accumulated history

A 235B parameter open source model performs well for the task at hand as it doesn't need to "remember" 50 previous incidents. It just needs to reason about the one in front of it, with exactly the right skills loaded.

Data sovereignty + cost + quality = production ready

3 factors make open source models production ready for agentic workloads:

1. Data sovereignty: Your logs stay on your infrastructure (no GDPR/DORA exposure)
2. Cost: Fixed infrastructure cost versus unpredictable per-token billing
3. Quality: Agentic harnesses plus skills deliver frontier-equivalent results

We've been running this in production for 6 months, processing 40+ infrastructure incidents per week. Our MTTR dropped from 45 minutes to 12 minutes for known failure patterns. Our compliance team is satisfied with the audit trail. Our infrastructure costs are predictable.

The open source model handles:

• Ansible Playbook errors and credential issues
• Linux package management and systemd failures
• OpenShift pod lifecycle and role based access control (RBAC) problems
• Networking DNS and connectivity issues

Same pipeline. Same skills. Same prompts. Different model. And it works.

Try it yourself

Want hands-on experience building an agentic AIOps pipeline with open source models? Try the complete workshop by reaching out to your Red Hat representative.

Resources

• Red Hat OpenShift AI
• LangChain Deep Agents
• Open Source as the Foundation of Sovereign AI
• Sovereignty Challenges for EMEA Enterprises

About this article: This post is based on a hands-on workshop created for Red Hat field demonstrations. The techniques described are running in production SRE environments processing real infrastructure incidents. Reach out to your Red Hat representative to try the full lab.