Red Hat OpenShift Lightspeed is now available as a technology preview

Red Hat OpenShift Lightspeed is now available as a technology preview. This is an important milestone along the path to a fully-supported OpenShift Lightspeed virtual assistant for Red Hat OpenShift. The tech preview release of OpenShift Lightspeed removes any dependency on third-party cloud-based large language model (LLM) providers with support for Red Hat OpenShift AI and Red Hat Enterprise Linux AI (RHEL AI).

Disconnected cluster support

Red Hat OpenShift has allowed for customers to run in completely disconnected environments for a long time. As new operator-powered features are added to the platform, they go through extensive testing work to ensure that they will work in these clusters that do not have any access to the internet. OpenShift Lightspeed can now be installed and used in these environments, which opens further possibilities for users of all skill-levels to leverage the assistant.

But a generative AI-powered virtual assistant that has no model provider to work with would not be very useful. That brings us to our second important feature.

Use a self-hosted model with Red Hat OpenShift AI vLLM and RHEL AI support

Red Hat OpenShift AI is Red Hat’s data science platform. While it has provided LLM inferencing and completion capabilities for some time, the 2.10 release introduced support for the vLLM model serving runtime.

Now, with the tech preview release of OpenShift Lightspeed, you can use OpenShift AI’s vLLM model serving runtime to connect the Red Hat OpenShift virtual assistant to a self-managed model of your choosing. Whether you wish to use Red Hat’s and IBM’s open source and indemnified Granite-family models, other publicly-available models, or models your organization has tuned and/or trained yourself, OpenShift AI can help get you there.

With the 1.2 release of RHEL AI, API access to the model inference server is now supported. This means that OpenShift Lightspeed can use the RHEL AI as the provider to generate responses with your choice of LLM.

Try OpenShift Lightspeed now

If you have a Red Hat OpenShift 4.15+ cluster, OpenShift Lightspeed is available in the Red Hat certified operator catalog now. If you have an existing subscription to Azure OpenAI, OpenAI, or WatsonX.ai, or if you have an OpenShift AI subscription, you can quickly and easily install OpenShift Lightspeed in your cluster and point it at the model provider of your choice. See the documentation for more information.