Deploying an AI chatbot in Microsoft Azure Red Hat OpenShift fully integrated with Azure OpenAI

How can you integrate the power of Microsoft Azure OpenAI and Microsoft Azure Red Hat OpenShift in an easy and scalable way? How can you deploy a chatbot in Azure Red Hat OpenShift using Azure OpenAI as a backend for your integrations? How can you leverage Azure Red Hat OpenShift as a turnkey application platform to deploy and scale your chatbot?

Let's take a look.

Overview

In this blog post, the primary focus is on creating a chatbot application with seamless integration into Azure OpenAI. The goal is to develop, build and deploy a chatbot that serves as a user-friendly frontend, powered by Gradio, a Python library known for simplifying the creation and sharing of applications.

A crucial component enhancing this chatbot’s capabilities is LangChain, a versatile framework tailored for building applications driven by language models. LangChain empowers applications to establish contextual awareness by connecting language models to various sources of context, such as prompt instructions, few-shot examples and relevant content. This contextual understanding ensures the chatbot’s responses are grounded effectively, enhancing user interaction.

The unique aspect of this chatbot lies in its backend, where a robust GPT model is deployed on Azure OpenAI. This integration ensures a smooth user experience, leveraging the capabilities of OpenAI’s cutting-edge technology within Azure’s reliable environment.

This integration highlights the power of Azure Red Hat OpenShift, which serves as the platform for deploying this chatbot application. By harnessing the potential of Large Language Models (LLMs) like GPT, this article demonstrates the innovative possibilities that arise when advanced AI technology meets the secure infrastructure provided by Azure OpenAI and Azure Red Hat OpenShift.

Throughout the article, you'll find detailed steps for creating and deploying such an application, making it a comprehensive guide for developers and enthusiasts eager to explore how Azure OpenAI and Azure Red Hat OpenShift can work together.

We hope this article helps you harness the full potential of AI-driven applications, from development to deployment, in the Azure ecosystem.

Azure Red Hat OpenShift AI chatbot Azure OpenAI components

Azure OpenAI overview

Azure OpenAI Service offers convenient REST API access to OpenAI’s advanced language models, such as GPT-4, GPT-3.5-Turbo, and Embeddings series. The GPT-4 and GPT-3.5-Turbo models are now widely available. These models can be tailored for various tasks like content creation, summarization, semantic search, and translating natural language to code. Users can utilize the service via REST APIs, Python SDK, or the web-based interface in Azure OpenAI Studio.

Azure Red Hat OpenShift delivers on-demand, fully managed OpenShift clusters that are highly available, with joint monitoring and operation by Microsoft and Red Hat. At its core, it utilizes Kubernetes. OpenShift enhances Kubernetes by adding valuable features, transforming it into a turnkey container platform as a service (PaaS) that greatly enhances the experiences of both developers and operators.

Comparing Azure OpenAI vs OpenAI

Azure OpenAI Service provides customers access to sophisticated language AI models like OpenAI GPT-4, GPT-3, Codex, DALL-E, and Whisper, all within the secure and reliable environment of Azure. In collaboration with OpenAI, Azure OpenAI co-develops APIs, ensuring seamless compatibility and transition between models.

With Azure OpenAI, customers benefit from the robust security features of Microsoft Azure while utilizing identical models as OpenAI. Azure OpenAI offers private networking, availability in specific regions, and responsible AI content filtering, enhancing the overall user experience and ensuring responsible usage of AI technology.

Gradio

A highly effective approach for showcasing your machine learning model, API, or data science workflow to others involves developing an interactive application that enables users or peers to experiment with the demo directly through their web browsers.

Gradio, a Python library, offers a streamlined solution for constructing such demos and facilitating easy sharing. In many cases, achieving this requires only a concise snippet of code.

To take a look at other Gradio apps, check my blog post: Deploying and Testing Machine Learning Applications in Kubernetes with Gradio.

LangChain

LangChain stands as a versatile framework designed for developing applications driven by language models. Its core functionalities enable:

• Contextual awareness: LangChain facilitates the connection of language models to diverse sources of context, including prompt instructions, few-shot examples, and relevant content. This enables applications to respond in a manner grounded in the provided context, enhancing their overall effectiveness.
• Intelligent reasoning: By leveraging LangChain, applications gain the ability to rely on language models for reasoning. This involves determining appropriate responses and actions based on the provided context, enhancing the application’s decision-making process significantly.

The key value propositions offered by LangChain include:

• Modular components: LangChain provides abstract structures for interacting with language models, coupled with a variety of implementations for each structure. These components are modular and user-friendly, ensuring ease of use, whether integrated within the LangChain framework or utilized independently.
• Pre-configured chains: LangChain offers pre-built chains, which are structured combinations of components designed to accomplish specific high-level tasks. These ready-to-use chains simplify the application development process, allowing developers to focus on specific functionalities without the hassle of building complex architectures from scratch.

In essence, LangChain streamlines the development process, offering a flexible and efficient approach to creating context-aware and intelligent applications powered by language models.

Azure Red Hat OpenShift

The Microsoft Azure Red Hat OpenShift service enables the deployment of fully managed OpenShift clusters.

Azure Red Hat OpenShift is jointly engineered, operated, and supported by Red Hat and Microsoft to provide an integrated support experience. There are no virtual machines to operate, and no patching is required. Master, infrastructure, and application nodes are patched, updated, and monitored on your behalf by Red Hat and Microsoft. Your Azure Red Hat OpenShift clusters are deployed into your Azure subscription and are included on your Azure bill.

When you deploy Azure Red Hat on OpenShift 4, the entire cluster is contained within a virtual network. Within this virtual network, your master nodes and workers nodes each live in their own subnet. Each subnet uses an internal load balancer and a public load balancer.

Azure Red Hat OpenShift chatbot with Azure OpenAI: Demo analysis

Let’s shift the focus from theory to practical application and explore how this chatbot operates and interacts in real time. Once deployed within your Azure Red Hat OpenShift cluster, the chatbot application becomes accessible through a specific URL. This application, powered by Azure OpenAI as its backend, offers a user-friendly interface for interaction.

The images show the chatbot in action. The first screenshot poses a straightforward yet intriguing question: What is Azure Red Hat OpenShift?

The Gradio app, functioning as the frontend, utilizes LangChain libraries internally to connect with the deployed GPT 3.5 model in Azure OpenAI:

Specifically, we deployed the gpt-35-turbo model, utilizing the 0301 version, hosted in the Azure France Central region.

But how can you be certain that your chatbot is effectively utilizing the Azure OpenAI GPT Model as its backend?

To validate this, let's delve into the Azure Red Hat OpenShift Console and inspect the chatbot’s logs:

Upon careful observation, the logs reveal the chatbot’s process: It sends the human message using LangChain libraries to the Azure OpenAI URL. The GPT model in Azure OpenAI generates the response, which is then relayed back to the “human,” completing the chatbot’s interaction loop.

This seamless integration showcases the synergy between LangChain libraries, Gradio frontend, and Azure OpenAI backend, enabling a dynamic and interactive user experience.

In an upcoming article, I will delve deeply into the integration components of the demonstration and provide comprehensive insights into how you can deploy your very own version in your Azure Red Hat OpenShift cluster.

Stay tuned for an in-depth exploration of the deployment process and harness the power of this integration for your projects.

Learn more about Red Hat OpenShift cloud services