Red Hat at NVIDIA GTC 2025

Retrieval-augmented generation (RAG) helps align Gen AI apps to customer use cases, but large models can be expensive, and RAG has limitations. We’ll explore how Red Hat AI can provide a better approach with RHEL AI and OpenShift AI: • Fine-tune smaller language models with customer datasets for better accuracy and lower costs • Introduce LAB: Large-Scale Alignment for ChatBots, a novel approach for instruction alignment and fine-tuning • Achieve significant cost reduction with RAG + LAB-tuned small language models We'll demo LAB and discuss real customers who've benefited from this approach with Red Hat AI and OpenShift AI.

vLLM, now the de facto standard for open-source LLM serving, plays a pivotal role in driving the widespread adoption of open-weight LLMs. As enterprises and the broader community seek to reduce deployment costs and maximize inference performance, optimizing model execution time becomes essential. We'll explore how vLLM achieves significant performance gains, focusing on the quantized linear layers that deliver dramatically faster inference. We'll deep dive into the implementation and performance of vLLM's quantized linear layers, explaining how they enable both compute speedup and memory compression. We'll examine the computational regimes where different quantization strategies excel and relate that to different real-world LLM inference server workloads. Finally, we'll look at some of the advanced optimizations in vLLM’s mixed-input Machete kernels, implemented via NVIDIA CUTLASS, and discuss how these innovations pave the way for high-performance LLM serving.

In this session, we'll explore how Red Hat OpenShift AI empowers developers to create and deliver AI-enabled applications at scale across hybrid cloud environments. By offering native support for NVIDIA NIM, OpenShift AI unlocks seamless integration with state-of-the-art NVIDIA technologies, ensuring optimized performance. Learn how you can leverage NVIDIA NIM to deploy Generative AI (GenAI) applications on OpenShift AI for maximum efficiency and flexibility. This session will demonstrate the advantages of combining these platforms, including enhanced portability, enterprise-level security, and scalable deployment across both private and public clouds.

Retrieval-augmented generation (RAG) helps align Gen AI apps to customer use cases, but large models can be expensive, and RAG has limitations. We’ll explore how Red Hat AI can provide a better approach with RHEL AI and OpenShift AI: • Fine-tune smaller language models with customer datasets for better accuracy and lower costs • Introduce LAB: Large-Scale Alignment for ChatBots, a novel approach for instruction alignment and fine-tuning • Achieve significant cost reduction with RAG + LAB-tuned small language models We'll demo LAB and discuss real-world customers who have benefited from this approach with Red Hat AI and OpenShift AI.