Why standardization is the key to agentic AI success: How a unified platform spurs innovation

You're tasked with bringing AI into your organization, but the journey from a proof-of-concept to a production-ready application is often a maze of fragmented tools and complex integrations, especially for agentic AI use cases. The real challenge isn't just the technology—it's building an AI strategy that is scalable, reliable, and manageable.

Standardization is a primary factor in meeting this challenge, primarily by reducing complexity and increasing efficiency. By combining Model Context Protocol (MCP) and Llama Stack on a platform like Red Hat OpenShift AI, you can create a unified and portable environment for your AI applications.

OpenShift AI can also support the integration of any other agentic frameworks and components you like, including tools like LangChain, LangGraph, and CrewAI, among others. Red Hat's approach can help simplify development, streamline operations, and give your enterprise the tools it needs to empower your multiple teams, like AI developers and data scientists to turn your AI vision into a reality within a trusted and consistent environment. 

Understanding the building blocks of an intelligent agent

To build resilient AI-enabled applications, you need a foundation that enables intelligent agents to access and use your enterprise's specific tools and data. This requires 2 key components: a standardized way to expose your services and a unified framework for interacting with them.

This is the essence of agentic AI, where an autonomous system can determine the best course of action and tools needed to achieve a goal. The combination of MCP and Llama Stack provides the framework for these agents to operate effectively.

MCP is an open protocol that standardizes how your AI systems integrate with external tools and data sources. It has been described as being a "USB-C port for AI applications." Just as USB-C allows you to plug in any compatible device—a monitor, a hard drive, or a keyboard—an MCP server exposes your organization's services in a standardized format. 

An AI agent can then plug in to this server to discover and use available tools, whether they are a database, a customer relationship management (CRM) system, or an internal API. This protocol provides a structured way for AI agents to connect with services, making it easy for them to get the information they need to carry out tasks.

Llama Stack is an open source framework that provides a unified, API-driven approach for building and deploying generative AI (gen AI) applications. It provides a consistent and portable way to abstract away the complexities of AI development and deployment and includes a suite of enterprise tools such as APIs for inference, retrieval-augmented generation (RAG), and AI agents. It also provides built-in components for safety, evaluation, and observability, helping organizations create production-ready AI applications that are both reliable and scalable.

An example of agentic AI in action

In enterprise IT, agentic AI can handle complex, multistep workflows that require dynamic decision-making. As an example, instead of a simple one-to-one request-and-response, a more sophisticated agent can act as an autonomous service desk manager, resolving issues with minimal human intervention.

Here is an example of what that kind of agent might do when a user submits a ticket for a "service outage":

1. Llama Stack receives the initial ticket. The system receives the user's report of a service outage and categorizes it as a high-priority incident.
2. The agent uses MCP and formulates a plan. The agent uses the MCP to identify and discover the relevant tools it has access to—such as a monitoring service, a log analysis tool, and a runbook automation system. With this context from the MCP, the agent formulates a multistep plan to investigate and resolve the issue.
3. The agent executes the plan using MCP to communicate with the discovered tools.
   • It first uses a monitoring tool to check the status of relevant services and dependencies.
   • It then calls a log analysis tool to review recent logs for a related application or a Red Hat OpenShift container, searching for errors.
   • Once a potential issue is identified (e.g., a pod is crashing), it uses a diagnostic tool to pinpoint the underlying problem, such as a misconfigured network policy or a full disk.
4. The agent takes corrective action. Based on the diagnosis, it uses MCP to call a pre-approved remediation script to restart the pod and clear the disk space. It then re-checks the service status to confirm the issue is resolved.
5. The agent communicates the outcome. It uses an internal tool to update the ticket status to "resolved," including a summary of the actions taken. It then drafts a message to the user, explaining the problem and confirming that the service is back online, all without a single human touchpoint.

By using the MCP to orchestrate these actions, the agent is able to operate as an autonomous and self-contained service desk, reducing the manual burden on IT teams giving them time to focus on more strategic initiatives.

A streamlined approach for development and production

The combination of Llama Stack and MCP significantly improves the developer experience. Instead of building for a specific, monolithic stack, your developers can use a standardized layer of abstraction through a single API. This enables a powerful "inner loop" development methodology, where they can build and test their AI applications on a local machine using a single SDK.

Because the APIs are consistent and the services are discovered through a standardized protocol, your developers can have confidence that their application will work exactly the same way when it is deployed to an enterprise-grade platform.

This portability is vital in the fast-moving AI environment. A developer can build their application to use one model today and, if a new model emerges that is better or more cost-effective, they can switch to it without having to rewrite their code. This reduces development time and effort, accelerates innovation, and provides a clear pathway for moving from development to production.

Gaining operational consistency for AI workloads

For your operations teams, managing a diverse array of AI models and services can be a major source of complexity. However, by running a standardized AI stack tailored for your organization on a platform like OpenShift AI, you gain a centralized and consistent environment.

OpenShift AI provides a comprehensive, trusted platform for deploying and serving LLMs and running the necessary databases for RAG applications. It also allows your teams to take advantage of their existing microservice architectures and connect to internal systems smoothly.

Your operations teams can also use their standard GitOps processes to deploy and manage these AI applications. This means the same workflows used for traditional cloud-native applications can be applied to AI workloads, simplifying management and maintenance at scale. It creates a simplified journey from development to production and provides operational consistency across your hybrid cloud environments.

By combining the power of open source technologies like Llama Stack and MCP—and by enabling the integration of other agentic AI frameworks such as LangChain, Haystack, LlamaIndex, and PydanticAI, among others—Red Hat AI provides a reliable foundation for building, deploying, and managing AI applications across your entire hybrid cloud.

Learn more

Ready to get started? Learn more about Red Hat AI through our product trials, consulting services, training and certification, and more.

• Red Hat AI product trials
• Red Hat AI consulting services
• Developing agentic AI with Red Hat AI
• Red Hat Learning Subscription
  • Red Hat Training and Certification AI courses