* [Topics](/en/topics "Topics") * [Artificial intelligence](/en/topics/ai "Artificial intelligence") * What is LLMOps? What is LLMOps? =============== Updated April 15, 2026•*5*-minute read Copy URL Jump to section --------------- OverviewLLMOps vs. MLOpsBenefitsUse casesStages of LLMOpsLLMOps platformWhy Red Hat What is LLMOps? --------------- Large language models ([LLMs](/en/topics/ai/what-are-large-language-models)) are [machine learning (ML)](/en/topics/ai/what-is-machine-learning) models that can understand and generate human language. LLMs like GPT-3, LLaMA, and Falcon are tools that learn from data to produce words and sentences. As these tools continue to evolve, organizations need best practices on the operation of these models. That’s where LLMOps comes in. Large Language Model Operations (LLMOps) are operational methods used to manage large language models. With LLMOps, the lifecycle of LLMs are managed and automated, from fine-tuning to maintenance, helping developers and teams deploy, monitor, and maintain LLMs. [Explore Red Hat AI](/en/products/ai "Red Hat AI") LLMOps vs. MLOps ---------------- If LLMs are a subset of ML models, then LLMOps is a large language model equivalent to [machine learning operations (MLOps)](/en/topics/ai/what-is-mlops). MLOps is a set of workflow practices aiming to streamline the process of deploying and maintaining ML models. MLOps seeks to establish a continuous evolution for integrating ML models into software development processes. Similarly, LLMOps seeks to continuously experiment, iterate, deploy and improve the LLM development and deployment lifecycle. [What is Model-as-a-Service?](/en/topics/ai/what-is-models-as-a-service) While LLMOps and MLOps have similarities, there are also differences. A few include: **Learning**: Traditional ML models are usually created or trained from scratch, but LLMs start from a [foundation model](/en/topics/ai/what-are-foundation-models) and are fine-tuned with data to improve task performance. **Tuning**: For LLMs, fine-tuning improves performance and increases accuracy, making the model more knowledgeable about a specific subject. Prompt tuning enables LLMs to perform better on specific tasks. Hyperparameter tuning is also a difference. In traditional ML, tuning focuses on improving accuracy. With LLMs, tuning is important for accuracy as well as reducing cost and the amount of power required for training. Both model types benefit from the tuning process, but with different emphases. Lastly, it's important to mention [retrieval-augmented generation (RAG)](/en/topics/ai/what-is-retrieval-augmented-generation), the process of using external knowledge to ensure accurate and specific facts are collected by the LLM to produce better responses. **Feedback**: Reinforcement learning from human feedback (RLHF) is an improvement in training LLMs. Feedback from humans is critical to a LLM’s performance. LLMs use feedback to evaluate for accuracy, whereas traditional ML models use specific metrics for accuracy. **Performance metrics**: ML models have  precise performance metrics, but LLMs have a different set of metrics, like bilingual evaluation understudy [(BLEU)](https://huggingface.co/spaces/evaluate-metric/bleu) and Recall-Oriented Understudy for Gisting Evaluation [(ROUGE)](https://huggingface.co/spaces/evaluate-metric/rouge) which require more complex evaluation. [Learn more about MLOps](/en/topics/ai/what-is-mlops "What is MLOps") Red Hat resources ----------------- [Keep reading](/en/resources "Keep reading") Benefits of LLMOps ------------------ With LLMOps becoming the best way to monitor and enhance the performance, there are three primary benefits to discuss: **Efficiency**: LLMOps allows teams to develop models faster, improve model quality, and quickly deploy. With a more streamlined approach to management, teams can collaborate better on a platform that promotes communication, development and deployment. Tools like [vLLM](/en/topics/ai/what-is-vllm), an open source infernece server that speeds up gen AI, can help use GPUs more efficiently. **Scalability**: LLMOps aids in scalability and management because more than 1 model can be managed and monitored for [continuous integration and continuous delivery/deployment (CI/CD)](/en/topics/devops/what-is-ci-cd). LLMOps also provides a more responsive user experience through improved data communication and response. **Risk reduction**: LLMOps promotes more transparency and establishes better compliance with organization and industry policies. LLMOps can improve [security and privacy](/en/topics/ai/what-is-ai-security) by protecting sensitive information and preventing exposure to risks. Learn how vLLM helped 3 well-known organizations scale faster. [Explore 3 real-world use cases](/en/topics/ai/how-vllm-accelerates-ai-inference-3-enterprise-use-cases) Use cases for LLMOps -------------------- There are a few use cases for LLMOps. **Continuous integration and delivery (CI/CD)**: CI/CD aims to streamline, accelerate, and automate the model development lifecycle.  It removes the need for human intervention needed to get new code resulting in reduced downtime and faster code releases. Tools like Tekton, which [Red Hat OpenShift Pipelines](https://www.redhat.com/en/blog/openshift-pipelines-and-openshift-gitops-are-now-generally-available) is based on, helps developer workflows by automating deployments across multiple platforms. [**Learn about operationalizing AI**](/en/products/ai/operationalize-ai) **Data collection, labeling, storage**: Data collection uses different sources to gather accurate information. Data labeling categorizes data, and [data storage](/en/topics/data-storage) collects and retains digital information that is attached to a network. **Model fine-tuning, inference, monitoring**: Model [fine-tuning](/en/blog/fine-tuning-and-serving-foundation-models) optimizes models to perform domain specific tasks. [AI inference](/en/topics/ai/what-is-ai-inference) is when an AI model provides an answer based on data. Model inference can manage production based on existing knowledge and perform the actions based on the inferred information. Model monitoring, including human feedback, collects and stores data about the model behavior to learn how models behave with real production data. [Why you should care about AI inference](/en/artificial-intelligence/inference) Stages of LLMOps ---------------- There are several stages or components of LLMOps and best practices for each: **Exploratory data analysis (EDA)**: The process of evaluating data to prepare for the machine learning lifecycle by creating data sets. * Data collection: The first step used to train the LLM collected from a different sources, like code archives and social media networks. * Data cleaning: Once collected, data needs to inspected to prepare for training, which includes removing errors, correcting inconsistencies, and removing duplicate data. * Data exploration: The next step is to explore the data to better understand its characteristics, including identifying outliers and finding patterns. **Data prep and prompt engineering**: The process of sharing accessible data across teams and developing prompts for LLMs. * Data preparation: The data used to train an LLM is prepared in ways, including synthesizing and concluding data that was collected. * Prompt engineering: The creation of prompts that are used for text that ensures LLMs generate the desired output. **Model fine-tuning**: The use of popular open source libraries like [Hugging Face Transformers](https://huggingface.co/docs/transformers/en/index) fine-tune and improve model performance. * Model training: After the data is prepared, the LLM is trained, or fine-tuned, by using a machine learning algorithm to learn the patterns in the data. * Model evaluation: Once trained, the LLM needs to be evaluated to see how well it performs, by using a set of data that was not used to train the LLM. * Model fine-tuning: If the LLM doesn’t perform well, it can be fine-tuned, which involves modifying the LLM’s parameters to improve its performance. **Model review and governance**: The process of discovering, sharing and collaborating across ML models with the help of open source MLOps platforms such as [Kubeflow](/en/topics/cloud-computing/what-is-kubeflow). * Model review: Once fine-tuned, the LLM needs to be reviewed to ensure that it is safe and reliable, which includes checking for bias and security risks. * Model governance: Model governance is the process of managing the LLM throughout its lifecycle, which includes tracking its performance, making changes to it as needed, and retiring it when it is no longer needed. **Model inference and serving**: The process of managing production details like how often a model is refreshed or request times. * Model serving: Once the LLM is reviewed and approved, it can be deployed into production, making it available through an application programming interface (API). When LLMs are large in size (think hundreds of billions of parameters) this often requires [distributed inference](/en/topics/ai/what-is-distributed-inference), the technique of splitting the model across multiple GPUs. Open-source projects like [llm-d](/en/topics/ai/what-is-llm-d) provide a managed, Kubernetes-native solution to orchestrate this distributed setup efficiently. * Model inference: The API can be queried by an application to generate text or answer questions. This can be done through a variety of ways, such as a [representational state transfer application programming interface (REST API)](/en/topics/api/what-is-a-rest-api) or a web application. **Model monitoring with human feedback**: The creation of model and data monitoring outlying or negative user behavior. * Model monitoring: Once deployed, the LLM needs to be monitored to ensure that it is performing as expected, which includes tracking its performance, identifying any problems, and making changes as needed. * Human feedback: This is used to improve the performance of the LLM, and it can be done by providing feedback on the text that the LLM generates, or by identifying any problems with the LLM’s performance. [What does AI look like at the enterprise?](https://www.redhat.com/en/topics/ai/what-is-enterprise-ai) What is an LLMOps platform? --------------------------- An LLMOps platform provides developers and teams with an environment that promotes collaboration through data analysis, experiment tracking, prompt engineering, and LLM management. It also provides managed model transitioning, deployment, and monitoring for LLMs. With better library management, the platform can help lower operational costs and reduce the need for highly skilled technical team members to complete tasks like data preprocessing, model monitoring, and deployment. [Learn how to choose a platform for AI and LLMOps](/en/topics/ai/what-is-an-ai-platform) Why Red Hat? ------------ [Red Hat® AI](/en/products/ai) is built for fast, flexible, and efficient inference through its [vLLM-powered](/en/topics/ai/what-is-vllm) server. It reliably connects models to your data to unify the customization and development of specialized agents on a single platform. Built on an open source foundation, our products give you full control of AI workflows from end-to-end at any scale. The Red Hat AI portfolio includes [Red Hat AI Enterprise](/en/products/ai/enterprise), a platform for deploying, managing, and scaling AI inference, agentic AI workflows, and AI-powered applications on any infrastructure. [Explore Red Hat AI](/en/products/ai) The official Red Hat blog ------------------------- Get the latest information about our ecosystem of customers, partners, and communities. [Keep reading](/en/blog "The official Red Hat blog") The adaptable enterprise: Why AI readiness is disruption readiness ------------------------------------------------------------------ This e-book, written by Michael Ferris, Red Hat COO and CSO, navigates the pace of change and technological disruption with AI that faces IT leaders today. [Get the resource](/en/resources/the-adaptable-enterprise-ai-ebook "The adaptable enterprise: Why AI readiness is disruption readiness") Keep reading ------------ ### What is AI in healthcare? Discover the benefits and challenges of AI in healthcare and how Red Hat is helping the industry. [Read the article](/en/topics/ai/what-is-ai-in-healthcare "article | What is AI in healthcare?") ### What is machine learning? Machine learning is the technique of training a computer to find patterns, make predictions, and learn from experience without being explicitly programmed. [Read the article](/en/topics/ai/what-is-machine-learning "article | What is machine learning?") ### What is deep learning? Deep learning is an artificial intelligence (AI) technique that teaches computers to process data using an algorithm inspired by the human brain. [Read the article](/en/topics/ai/what-is-deep-learning "article | What is deep learning?") Artificial intelligence resources --------------------------------- ### Related content * Blog post [Announcing Red Hat OpenShift Pipelines 1.21: Faster builds, smarter caching, and improved troubleshooting](/en/blog/announcing-red-hat-openshift-pipelines-121-faster-builds-smarter-caching-and-improved-troubleshooting) * Blog post [233% 3-year return on investment and 13 months to payback with Red Hat AI](/en/blog/233-3-year-return-investment-and-13-months-payback-red-hat-ai) * Blog post [Red Hat OpenShift sandboxed containers 1.12 and Red Hat build of Trustee 1.1 bring confidential computing to bare metal and AI workloads](/en/blog/red-hat-openshift-sandboxed-containers-112-and-red-hat-build-trustee-11-bring-confidential-computing-bare-metal-and-ai-workloads) * Blog post [AI for scientific research: Building the research platform that science needs with Red Hat AI](/en/blog/ai-scientific-research-building-research-platform-science-needs-red-hat-ai) ### Related articles * [What is explainable AI?](/en/topics/ai/what-explainable-ai) * [What is machine learning?](/en/topics/ai/what-is-machine-learning) * [Agentic AI vs. generative AI](/en/topics/ai/agentic-ai-vs-generative-ai) * [What are foundation models for AI?](/en/topics/ai/what-are-foundation-models) * [What is deep learning?](/en/topics/ai/what-is-deep-learning) * [What is MLOps?](/en/topics/ai/what-is-mlops) * [[node:rh-smart-meta-title]](/en/topics/ai/sovereign-ai) * [What is Model Context Protocol (MCP)?](/en/topics/ai/what-is-model-context-protocol-mcp) * [What is retrieval-augmented generation?](/en/topics/ai/what-is-retrieval-augmented-generation) * [What is distributed inference?](/en/topics/ai/what-is-distributed-inference) * [Understanding AI/ML use cases](/en/topics/ai/ai-ml-use-cases) * [What is agentic AI?](/en/topics/ai/what-is-agentic-ai) * [What is AI in the public sector?](/en/topics/ai/what-is-ai-in-the-public-sector) * [AI infrastructure explained](/en/topics/ai/ai-infrastructure-explained) * [Understanding AI in telecommunications with Red Hat](/en/topics/ai/understanding-ai-in-telecommunications) * [What is AI in healthcare?](/en/topics/ai/what-is-ai-in-healthcare) * [What is an AI platform?](/en/topics/ai/what-is-an-ai-platform) * [What is AI security?](/en/topics/ai/what-is-ai-security) * [What is enterprise AI?](/en/topics/ai/what-is-enterprise-ai) * [What is Model-as-a-Service?](/en/topics/ai/what-is-models-as-a-service) * [What is vLLM?](/en/topics/ai/what-is-vllm) * [What is generative AI?](/en/topics/ai/what-is-generative-ai) * [AIOps explained](/en/topics/ai/what-is-aiops) * [What is AI inference?](/en/topics/ai/what-is-ai-inference) * [What is llm-d?](/en/topics/ai/what-is-llm-d) * [RAG vs. fine-tuning](/en/topics/ai/rag-vs-fine-tuning) * [What are large language models?](/en/topics/ai/what-are-large-language-models) * [What is edge AI?](/en/topics/edge-computing/what-is-edge-ai) * [What are Granite models?](/en/topics/ai/what-are-granite-models) * [What are intelligent applications?](/en/topics/ai/what-are-intelligent-applications) * [SLMs vs LLMs: What are small language models?](/en/topics/ai/llm-vs-slm) * [Predictive AI vs generative AI](/en/topics/ai/predictive-ai-vs-generative-ai) * [AI in banking](/en/topics/ai/ai-in-banking) * [vLLM vs. Ollama: When to use each framework](/en/topics/ai/vllm-vs-ollama) * [How vLLM accelerates AI inference: 3 enterprise use cases](/en/topics/ai/how-vllm-accelerates-ai-inference-3-enterprise-use-cases) * [What is InstructLab?](/en/topics/ai/what-is-instructlab) * [What is parameter-efficient fine-tuning (PEFT)?](/en/topics/ai/what-is-peft) * [Why choose Red Hat Ansible Automation Platform as your AI foundation?](/en/topics/automation/automation-and-ai) * [LoRA vs. QLoRA](/en/topics/ai/lora-vs-qlora) * [Edge solutions for real-time decision making](/en/topics/edge-computing/edge-solutions-real-time-decision-making) * [What are predictive analytics](/en/topics/automation/how-predictive-analytics-improve-it-performance) * [How Kubernetes can help AI/ML](/en/topics/cloud-computing/how-kubernetes-can-help-ai) * [What is Kubeflow?](/en/topics/cloud-computing/what-is-kubeflow) * [Accelerate MLOps with Red Hat OpenShift](/en/technologies/cloud-computing/openshift/aiml) * [What is a data lake?](/en/topics/data-storage/what-is-a-data-lake) [More about this topic](/en/topics/ai "More about this topic")