OpenShift AI overview

Artificial intelligence (AI), machine learning (ML), and deep learning (DL) are having a profound influence on application modernization efforts across diverse businesses and industries. The need to innovate and derive strategic value and new insights from data is fostering the expanding use of AI-enabled cloud-native applications and MLOps methodologies. At the same time, this brave new world can be complex, with implications for everyone—from developers to data scientists to operations staff. Operationalizing AI/ML is not trivial—often requiring months while generative AI (gen AI) innovation progresses daily. This mismatch can result in project failures that put your business at risk amid myriad challenges:

• Matching the pace of AI progress can be daunting, from keeping rapidly evolving tools and application services current and consistent to provisioning hardware resources like graphic processing units (GPUs), and scaling AI-enabled applications.
• Organizations must mitigate risks when investing in AI and still gain value, especially when using sensitive data to inform models and applications.
• Maintaining different platforms for application developers and data scientists can complicate collaboration and hinder the speed of development.
• AI-enabled application deployment must happen at scale and close to data generation points.

Built on Red Hat® OpenShift®, a leading hybrid cloud application platform, Red Hat® OpenShift® AI gives data scientists and developers a powerful AI/ML platform for building and deploying intelligent applications. Organizations can experiment with a choice of tools, collaborate, and accelerate time to market—all within 1 common platform. Red Hat OpenShift AI combines the self-service environment that data scientists and developers want with the confidence that enterprise IT demands.

Having a trusted foundation reduces friction throughout the lifecycle. Red Hat OpenShift AI offers a robust platform, a broad ecosystem of popular certified tools, and familiar workflows for deploying models into production. With these advantages, teams can collaborate with less friction and get AI-enabled applications into the market more efficiently, ultimately delivering greater value for the business.

Red Hat OpenShift AI (previously Red Hat OpenShift Data Science) is a flexible, scalable MLOps platform with tools to build, deploy, and manage AI-enabled applications. It is built with open source technologies and provides trusted, operationally consistent capabilities for teams to experiment, serve models, and deliver innovative apps. Red Hat OpenShift AI accelerates the delivery of intelligent applications, helping ML models move from early pilots into intelligent applications with greater speed—on a shared, consistent platform.

Red Hat OpenShift AI offers an integrated user interface (UI) experience with tooling for building, training, tuning, deploying, and monitoring predictive and generative AI models. You can deploy models to hybrid cloud environments, gaining the flexibility of running workloads wherever needed, with no commercial cloud lock-in. Red Hat OpenShift AI is based on the community Open Data Hub project and common open source projects such as Jupyter, Pytorch, and Kubeflow.

Simplify AI adoption

As an add-on to Red Hat OpenShift, Red Hat OpenShift AI provides a platform designed to increase AI adoption and enhance trust in AI initiatives. It combines the resources of open source communities with a robust AI ecosystem, increasing flexibility and freedom of choice by allowing users to select the right AI/ML technologies for an end-to-end experience. You can build your own predictive models or start with an external generative AI model. You can then fine-tune or enhance your model with retrieval-augmented generation (RAG) to inference with several model servers provided by the platform.

Data scientists can use their familiar tools and frameworks or access a growing technology partner ecosystem for deeper AI/ML expertise—without being burdened with a prescriptive toolchain. Rather than waiting for IT to provision necessary resources, they get on-demand infrastructure with 1 click rather than an IT ticket.

Improve operational consistency across teams

Red Hat OpenShift AI provides a consistent user experience that empowers data scientists, data engineers, application engineers, and DevOps teams to collaborate to deliver timely AI solutions effectively. It offers self-service access to collaborative workflows, acceleration through access to GPUs, and streamlined operations. Organizations can deliver AI solutions consistently at scale across hybrid cloud environments and at the edge.

Because Red Hat OpenShift AI is an add-on offering to Red Hat OpenShift, IT operations can provide more straightforward configurations to data scientists and application developers on a stable and proven platform that they can scale up or down with low effort. IT worries less about governance and security, with no need to chase down rogue cloud-platform accounts.

Gain hybrid cloud flexibility

Red Hat OpenShift AI provides the ability to train, deploy, and monitor AI/ML workloads in a cloud environment, in on-premise datacenters, or at the network edge, close to where data is generated or located. This flexibility allows AI strategies to evolve, moving operations to the cloud or to the edge as required by the business. Organizations can train and deploy models and AI-enabled applications wherever they need to meet relevant regulatory, security, and data requirements, including air-gapped and disconnected environments.

Figure 1 illustrates how the model operation lifecycle integrates Red Hat OpenShift AI as a common platform, expanding the capabilities of the leading application platform, Red Hat OpenShift. Organizations build on a proven hybrid cloud platform, offered as either a managed cloud service or self-managed, yielding tremendous flexibility. The cloud service version is available on Red Hat OpenShift Service on AWS and Red Hat OpenShift Dedicated (on AWS or Google Cloud Platform). The self-managed version can run anywhere Red Hat OpenShift runs, either on-premise or on any of the 3 major public cloud environments.

Our solutions also allow for expanding AI capabilities through collaboration with a rich ecosystem of dozens of AI software and software as a service (SaaS) partners. Red Hat OpenShift AI's flexibility and composability allow customers to assemble an end-to-end AI/ML platform that fits their specific needs.

For those just getting started with generative AI models, Red Hat OpenShift AI includes the components of Red Hat Enterprise Linux® AI—a foundational model platform to seamlessly develop, test, and run Granite family large language models (LLMs) to power enterprise applications. In addition to the Granite models provided with Red Hat OpenShift AI, the framework supports models from HuggingFace, Stability AI, and other model repositories.

Red Hat OpenShift AI is a foundational component in IBM watsonx.ai, providing fundamental AI tooling and services for generative AI workloads. Watsonx offers an enterprise studio for AI builders to deliver generative AI applications with low code and no code requirements, user-friendly workflows for model development, and access to a library of IBM foundation models and curated open source models. Red Hat OpenShift and Red Hat OpenShift AI are embedded technical prerequisites for watsonx software.

Core tools and capabilities provided with Red Hat OpenShift AI offer a solid foundation:

• Model building and fine tuning. Data scientists can conduct exploratory data science in a JupyterLab UI, offering out-of-the-box notebook images with common Python libraries and packages, including TensorFlow, PyTorch, and CUDA. In addition, organizations can provide their own custom notebook images, allowing them to create and collaborate on notebooks while organizing work in projects and workbenches.
• Model serving. Red Hat OpenShift AI provides a variety of frameworks for model serving routing to simplify the deployment of predictive machine learning or foundation models to production environments regardless of their compute resource requirements. The solution also allows customers to bring and use their preferred runtimes.
• Data science pipelines. Red Hat OpenShift AI also includes a data science pipelines component that lets you orchestrate data science tasks into pipelines and build pipelines using a graphical front end. Organizations can chain together processes like data preparation, build models, and serve models.
• Model monitoring. Red Hat OpenShift AI helps ops-oriented users monitor operations and performance metrics for model servers and deployed models. Data scientists can use out-of-the-box visualizations for performance and operations metrics or integrate data with other observability services.
• Distribute workloads. Distributed workloads allow teams to accelerate data processing along with model training, tuning, and serving. This capability supports prioritization and distribution of job execution along with optimal node utilization. Advanced GPU support helps handle the workload demands of foundation models.

Red Hat provides Jupyter Notebook images for Tensorflow and PyTorch as a part of the product, making it easier for teams to embrace these powerful technologies. The Jupyter spawner can deploy an organization's custom images to data science teams for consistency and flexibility, incorporating preferred libraries, tools, and languages. The service also includes the Git plug-in to JupyterLab, speeding integration with Git directly from the JupyterLab interface. Other common analytics packages provided as a part of the service simplify operation and make getting started more straightforward with the right tools for your project, including Pandas, scikit-learn, and NumPy. The project's UI allows data scientists to create their own workspaces to organize and share their notebook images and artifacts as projects and collaborate with other users.

The requirement to serve both predictive and gen AI models demands a flexible model-serving approach. Red Hat OpenShift AI supports multiple model-serving frameworks, and you can choose the provided multimodel and single-model inference servers or your own custom inference server. The model-serving UI is directly integrated into the Red Hat OpenShift AI dashboard as well as the projects workspace. Underlying cluster resources can scale up or down as your workload requires.

Red Hat provides site reliability engineering (SRE) support for the managed cloud service version, encompassing both the underlying OpenShift cloud services platform and the Red Hat OpenShift AI service. This support lets you focus on your business analytics rather than the underlying platform. Red Hat maintains high availability for the Red Hat OpenShift AI service, including the underlying Red Hat OpenShift managed cloud services environment. All updates, upgrades, and compatibility are managed as a part of the service, eliminating the need to track potentially complex compatibility matrices between analytics tools.

Red Hat OpenShift provides the services and software to let organizations successfully train and deploy their models and move them to production (Figure 2). In addition to Red Hat OpenShift AI, this process is integrated with Red Hat Application Foundations, which includes streams for Apache Kafka for real-time data and event streaming, 3scale for API management, and the Red Hat build of Apache Camel for data integration.

The Red Hat OpenShift AI dashboard provides a central place to discover and access all applications and documentation, easing adoption. Smart start tutorials offer best practice guidance for common components and integrated partner software and are available directly from the dashboard to help data scientists learn and get started in less time. The following sections describe the technology partner tools integrated with Red Hat OpenShift AI. Some tools will require an additional license from the technology partner. 

Starburst

Starburst accelerates analytics by making it fast and easy for your teams to capitalize on your data to improve how the business functions. Delivered as a self-managed product or a fully managed service, Starburst democratizes data access, bringing more comprehensive insights to data consumers. Starburst is built on open source Trino (formerly known as PrestoSQL), the premiere massively parallel processing (MPP) SQL engine. Built and operated by Trino experts and the creators of Presto, Starburst gives you the freedom to interrogate diverse data sets wherever they exist without having to move your data.

Starburst integrates with the scalable cloud storage and computing services Red Hat OpenShift provides, yielding a more stable, security-focused, efficient, and cost-effective way to query all your enterprise data. Benefits include:

• Automation. Starburst and Red Hat OpenShift operators provide auto-configuration, auto-tuning, and auto-management of clusters.
• High availability and graceful scaledown. The Red Hat OpenShift load balancer can keep services like the Trino coordinator in an always-on state.
• Elastic scalability. Red Hat OpenShift can automatically scale the Trino worker cluster based on query load.

 Anaconda Package Security Manager

Anaconda Package Security Manager provides curated access to an extensive set of notebook packages with popular Python libraries for use in Jupyter projects, with prebuilt Jupyter images available directly from the Red Hat OpenShift AI dashboard. Package Security Manager gives organizations access to the world’s most popular open source package distribution and management experience, optimized for commercial use, including:

• Open source innovation, with more than 9,500 Anaconda-curated data science and ML packages in Anaconda’s premium repository.
• Content trust features, such as conda signature verification, that help you keep vulnerabilities and unreliable software out of your data science and ML pipelines.
• Confidence with uptime service-level agreements (SLAs) and support you can depend on for production workflows.
• Policy enforcement that allows you to control access and manage vulnerabilities and licenses.
• An extended channel featuring more than 6,000 additional packages built from conda-forge recipes on Anaconda’s trusted infrastructure.

HPE Machine Learning Data Management Software

Organizations need data management solutions that facilitate everything from laptop experiments to critical enterprise deployments. HPE Machine Learning Data Management software (formerly known as Pachyderm) allows data science teams to build and scale containerized, data-driven ML pipelines with a guaranteed data lineage provided by automatic data versioning. Engineered to solve real-world data science problems, Pachyderm provides the data foundation that allows teams to automate and scale their ML lifecycle while guaranteeing reproducibility. With use cases that range from unstructured data to data warehouses, natural language processing, video and image ETL (extract, transform, and load), financial services, and life science, HPE Machine Learning Data Management Software provides:

• Automated data versioning that gives teams a high-performance way to keep track of all data changes.
• Data-driven containerized pipelines that speed up data processing while lowering compute costs.
• An immutable data lineage that provides a fixed record for all activities and assets in the ML lifecycle.
• A console that provides an intuitive visualization of your directed acyclic graph (DAG) and aids with debugging and reproducibility.
• Jupyter notebook support with a JupyterLab Mount Extension for a point-and-click interface to versioned data.
• Enterprise administration with robust tools for deploying and administering HPE Machine Learning Data Management Software at scale across different teams within the organization.

NVIDIA accelerates deployment of AI solutions

As AI/ML applications become increasingly critical to business success, organizations require platforms that can handle complex workloads, optimize hardware utilization, and provide scalability. Scalable data processing, data analytics, machine learning training, and inferencing all represent highly resource-intensive computational tasks. NVIDIA software makes it possible to accelerate all aspects of end-to-end data science by taking advantage of the parallel processing capabilities of GPUs.

NVIDIA NIM enhances the management and performance of NVIDIA GPUs within the Red Hat OpenShift environment, allowing AI applications to use the full potential of NVIDIA’s AI software and hardware. The integration of NVIDIA NIM and Red Hat OpenShift AI allows for better resource allocation, greater efficiency, and faster AI workload execution.

Intel OpenVINO toolkit

The Intel Distribution of the OpenVINO toolkit accelerates the development and deployment of high-performance DL inference applications on Intel platforms. The toolkit lets you adopt, optimize, and tune virtually any neural network model and run comprehensive AI inferencing using the OpenVINO ecosystem of development tools.

• Model. Software developers have the flexibility to use their own DL models. For time to market advantage, they can also use pretrained and preoptimized models available through Intel’s collaboration with Hugging Face for the OpenVINO toolkit.
• Optimize. The OpenVINO toolkit offers several ways to convert models for better convenience and performance, helping software developers achieve faster and more efficient AI model execution. Developers can skip model conversion and run inference directly from TensorFlow, TensorFlow Lite, ONNX, or PaddlePaddle formats. Conversion to OpenVINO IR provides optimal performance, reducing the time it takes for the 1st inference and saving storage space. The Neural Network Compression Framework can provide further improvements.
• Deploy. The OpenVINO Runtime Inference Engine is an application programming interface (API) designed to be integrated into your applications to accelerate the inference process. Its “write once, deploy anywhere” approach allows you to efficiently run inference tasks on various Intel hardware, including central processing units (CPUs), GPUs, and accelerators.

AI Tools from Intel®

AI Tools from Intel (formerly Intel AI Analytics Toolkit) give data scientists, AI developers, and researchers familiar Python tools and frameworks to accelerate end-to-end data science and analytics pipelines on Intel architectures. The components use Intel’s oneAPI libraries for low-level compute optimizations. This toolkit maximizes performance from preprocessing through ML and provides interoperability for efficient model development.

Using AI Tools from Intel, you can:

• Deliver high-performance DL training on Intel XPUs and integrate faster inferencing into your AI development workflow with Intel-optimized DL frameworks for TensorFlow and PyTorch, including pretrained models and low-precision tools.
• Achieve drop-in acceleration for data preprocessing and ML workflows with compute-intensive Python packages, Modin, scikit-learn, and XGBoost, optimized for Intel.
• Gain direct access to analytics and AI optimizations from Intel to ensure your software works together uninterrupted.

Elastic

The Elastic Search AI Platform (built on the ELK Stack) combines the precision of search and the intelligence of AI, letting users prototype and integrate with LLMs faster and engage gen AI to build scalable, cost-effective applications. The Elastic Search AI Platform allows users to build transformative retrieval augmented generation (RAG) applications, proactively resolve observability issues, and address complex security threats. Elasticsearch can be deployed where your applications are: on-premise, on your chosen cloud provider, or in air-gapped environments.

Elastic integrates with embedding models from the ecosystem including Red Hat OpenShift AI, Hugging Face, Cohere, OpenAI, and others via a single straightforward API call. This approach ensures clean code for managing hybrid inference for RAG workloads, with features that include:

• Chunking, connectors, and web crawlers for ingesting diverse datasets into your search layer.
• Semantic search with Elastic Learned Sparse EncodeR (ELSER), the built-in ML model, and the E5 embedding model, enabling multilingual vector search.
• Document and field-level security, implementing permissions and entitlements that map to your organization’s role-based access control.

With the Elastic Search AI Platform, you are part of a worldwide community of developers where inspiration and support are never far away. Find the Elastic community on Slack, our discussion forums, or social media.

With Red Hat OpenShift AI, organizations can experiment, collaborate, and ultimately accelerate their intelligent application journey. Data scientists gain the flexibility of using Red Hat OpenShift AI as a cloud-based add-on service managed by Red Hat or as a self-managed software offering, simplifying tasks no matter where they build their models. IT operations benefit from MLOps capabilities, allowing models to come into production more rapidly. Self-service for developers and data scientists advances innovation on an application platform already used and fully trusted by enterprise IT. Unlike competing approaches, data scientists can choose tooling with no restrictive toolchain, yielding new data insights without forcing arbitrary limitations.