Introduction to llm-d

Interactive demo•2 mins• Red Hat OpenShift

Published on May 20, 2026 by Christopher Nuland

Click through for a high-level tour of Red Hat’s LLM-D distributed inference demo, showcasing how KV cache–aware routing accelerates responses and improves efficiency.
3D graphic of OpenShift icon, a cloud, and a cursor aimed at a purple target
What you'll learn Next steps Related resources & demos Feedback

What you'll learn in this interactive demo

This experience highlights:

  • How LLM-D separates prefill and decode stages to scale each independently
  • Testing prompts and seeing cache hit rates in action
  • Comparing repeated and unique prompts to show the impact on latency and load distribution
  • Using the inference gateway for load testing and measuring session stickiness
  • Tracking per-pod cache hit rates, token throughput, and latency in Grafana

Next steps

Red Hat LLM-D helps scale distributed inference across environments, improving efficiency and reducing latency for generative AI applications.

Checklist
E-book
E-book
Blog post
Blog post
Interactive demo

Trusted Software Supply Chain

Learn how to secure your software supply chain end to end with Red Hat Trusted Software Supply Chain. This guide demonstrates Red Hat's approach to adding security guardrails across the full software delivery lifecycle on OpenShift.

2 mins

Red Hat OpenShift

 Interactive demo

Red Hat Trusted Application Pipeline for AI applications

Learn how to build and deploy AI-powered applications using Red Hat Trusted Application Pipeline. This guide demonstrates the full MLOps workflow from application setup and model serving to pipeline execution and live code iteration with OpenShift AI and DevSpaces.

5 mins

Red Hat OpenShift AI

We'd like to hear from you

We welcome your thoughts on our interactive demos at Red Hat. We're committed to delivering engaging, hands-on learning resources, and your input helps us continue to improve our offerings.

Share feedback
A woman wearing a Red Hat hoodie and headphones smiles while using a laptop

About the author of this page

Christopher Nuland headshot

Christopher Nuland

Technical Marketing Manager for AI Inference

Christopher Nuland is a Principal Technical Marketing Manager for AI at Red Hat and has been with the company for over six years. Before Red Hat, he focused on machine learning and big data analytics for companies in the finance and agriculture sectors. Once coming to Red Hat, he specialized in clou...