Generative AI in action

Gen AI is a digital transformation project, and for it to succeed, it needs an implementation strategy.

There are 2 paths your enterprise can take for its AI strategy: you can adopt a cloud-based AI service or build and host an AI platform yourself. These options require different levels of technical involvement and operational effort, and offer different levels of customization and control.

Cloud-based AI services

Cloud-based AI services are provided by a third-party vendor as a paid and managed solution, and offer access to frontier models via application programming interfaces (APIs). These services allow your organization to integrate AI models into your applications without hosting the model yourself. Some private commercial offerings also let you fine-tune the models they provide, or deploy models in a dedicated or more controlled environment.

Because this approach gives you ready-to-use AI solutions with minimal interaction with the model itself, it can be more straightforward and cost-effective for organizations that do not want to deal with the complexities of AI infrastructure management, have smaller operations teams, or are adopting AI at a lesser scale.

Self-hosted AI platforms

Building and hosting an AI platform yourself provides more choice and control over your models and environment. You can select the hardware, software, models, applications, and deployment location that best fit your organization’s requirements. For example, you can choose to host your models and applications in public clouds, private clouds, on-site datacenters, or edge locations. This approach also gives you more opportunities to customize your models and applications, more control over your data, and less dependence on third-party providers. However, it typically involves higher up-front investments and ongoing operational effort and maintenance costs than a cloud-based AI service. 

Model-as-a-Service

If you want to build a self-hosted platform but offer your users an experience similar to what they would get from a cloud-based AI service, you can implement the solution pattern for Model-as-a-Service (MaaS) as an internal cloud utility. By purchasing a dedicated AI platform, you can set up your IT team to act as an internal provider, offering developers an easy-to-use experience similar to public cloud offerings. This centralized approach allows developers to access curated models on demand without managing graphic processing units (GPUs) and other complex infrastructure. 

These platforms go beyond just serving models. They support the entire AI lifecycle, from initial development and fine-tuning to the deployment and long-term management of predictive, generative, and agentic AI. This strategy combines the use of a cloud-like consumption model with the deep control, security focus, and governance required to scale diverse AI initiatives across the entire enterprise.

To build and host an AI platform, you need:

• Access to foundation models for your use case. Examples include large language models (LLMs), code models, small language models (SLMs), open source models, and multimodal models.
• Access to hardware acceleration capabilities like GPUs.
• Access to an application platform with advanced AI tools and serving mechanisms.
• A governance solution for compliance and responsible AI use.

Whether you use a MaaS strategy or not, building a self-hosted AI platform gives you more control over your AI solutions. That makes it an obvious choice for organizations that operate in highly regulated industries, plan to use sensitive data and intellectual property (IP) within their AI solutions, or have larger operations teams that can handle the complexities of building, running, and maintaining AI infrastructure.

Table 1. Comparing approaches to AI strategies

Considerations for evaluating AI solutions

When you’re choosing an IT strategy for your organization, you need to keep transparency, efficiency, and relevance in mind. The way you think about these considerations will shift depending on your chosen path; for instance, a self-hosted strategy makes it possible for your internal teams to provide deep transparency, while with a cloud-based approach you will need to rigorously vet your vendors.

Ensure transparency in AI solutions

AI solutions should provide transparency, accountability, and explainability while ensuring data privacy, security, and regulatory compliance. That is critical for building trust, mitigating risks, and remaining competitive. Look for vendors that clearly disclose model architectures, training data, and performance metrics, and offer accountability mechanisms and explanations for AI-generated outputs. 

Optimize infrastructure and cost efficiency

Scalable, low-cost infrastructure solutions that support model optimization, distributed training, and efficient hardware configurations can help you minimize operational expenses, enhance performance, and adapt quickly to changing demands. 

While self-hosted platforms involve higher up-front infrastructure costs, you can mitigate these by using model optimization techniques like quantization and distillation. These allow you to reduce hardware dependency, lower infrastructure costs, and decrease your overall environmental impact.

Flexible and efficient inferencing

Red Hat AI optimizes model inference across hybrid cloud environments, allowing organizations to deploy their preferred models faster and more cost-effectively. Its runtime, vLLM, maximizes throughput and minimizes latency, while LLM Compressor uses advanced quantization techniques to improve inference speeds without sacrificing prediction accuracy. According to Forrester research, “Together, these components create a high-performance environment for serving models across a wide range of applications, and have helped interviewed enterprises double their GPU capacity.”³

Connecting models to data

The platform simplifies the integration of private enterprise data, providing standardized tools for preparing and ingesting information to ground AI outputs in specific business knowledge. Organizations can boost model performance and relevance through multiple customization techniques, including fine-tuning, retrieval-augmented generation (RAG), and prompt tuning. “This foundation ensures that models are aligned with enterprise scenarios and deliver tangible business outcomes. Interviewed enterprises have increased productivity with a 60% reduction in model training time,” according to Forrester research.³

Agentic AI innovation

Moving beyond simple chat interfaces, Red Hat AI accelerates the deployment and management of agentic AI on a scalable and flexible platform. It provides the infrastructure needed to manage and orchestrate autonomous agents that can reason through complex goals, such as onboarding a new client, by breaking them into executable tasks like updating databases or sending emails. This transition from chat to act is supported by managed workflows that maintain a human in the loop for high-stakes approvals and safety. 

According to Forrester, “Interviewed enterprises have been able to accelerate 400 AI projects by year 3 via 75% faster MLOps provisioning.”³

AI at scale across hybrid cloud environments

Red Hat AI allows organizations to manage and monitor the entire lifecycle of both predictive and generative AI models at scale, from single-server deployments to highly scaled-out distributed platforms. The platform ensures operational consistency and risk mitigation by providing a tested, supported AI platform powered by Red Hat OpenShift. Through centralized monitoring, teams can track bias, data drift, and anomaly detection to ensure models remain reliable, fair, and compliant with corporate governance standards. “Interviewed enterprises have seen up to 2% annual profit growth by year 3 from faster AI productization.”³

You can use Red Hat AI to build applications that support a range of AI use cases and address many business challenges. Thanks to Red Hat AI’s consistent user experience, stakeholders from AI developers to data scientists to IT operations teams can more simply develop and deploy AI solutions across hybrid cloud environments.

Common business use cases addressed by Red Hat AI include:

AI- and data-driven business operations

AI models can handle the massive amount and breadth of data that organizations collect to help them make more informed business decisions. Teams gain keener insights into the data that underlies the business, which can help them maximize revenue, optimize operations, and enhance customer experiences and employee productivity.

Recommendation engines

AI recommendation engines assess current situations against historical data to identify common factors and provide guidance. They can be used in many industries to deliver real-time suggestions for action.

Clalit Health Services recently established an advanced AI platform based on Red Hat AI to process historical medical data and train a LLM to identify at-risk patients who would benefit from preventive care and medication. The solution then provides recommendations on courses of action for patient treatment through a chatbot-like experience. Clalit is using the same platform to build learning processes and algorithms to identify new trends, patient and disease behavior patterns, and more.

Read the success story⁴