* [Topics](/en/topics "Topics") * RAG vs. fine-tuning RAG vs. fine-tuning =================== Published May 12, 2026•*5*-minute read Copy URL Jump to section --------------- RAG vs. fine-tuningBuilding on top of large language modelsWhat is RAG and how does it work?What is fine-tuning?Considerations How Red Hat can help Retrieval-augmented generation (RAG) vs. fine-tuning ---------------------------------------------------- Both [RAG](/en/topics/ai/what-is-retrieval-augmented-generation) and fine-tuning aim to improve [large language models (LLMs)](/en/topics/ai/what-are-large-language-models). RAG does this without modifying the underlying LLM, while fine-tuning requires adjusting the weights and parameters of an LLM. Often, you can customize a model by using both fine-tuning and RAG architecture. [Explore Red Hat AI](/en/products/ai) Building on top of large language models ---------------------------------------- An LLM is a type of [artificial intelligence (AI)](/en/topics/ai) that uses [machine learning (ML)](/en/topics/ai/what-is-machine-learning) techniques to understand and produce human language. These ML models can generate, summarize, translate, rewrite, classify, categorize, and analyze text—and more. The most popular use for these models at an enterprise level is to create a question-answering system, like a chatbot. LLM [foundation models](/en/topics/ai/what-are-foundation-models) are trained with general knowledge to support a broad range of use cases. However, they likely aren’t equipped with domain-specific knowledge that’s unique to your organization. RAG and fine-tuning are 2 ways to adjust and inform the LLM with the data you want so it produces the output you want. For example, let’s say you’re building a chatbot to interact with customers. In this scenario, the chatbot is a representative of your company, so you’ll want it to act like a high-performing employee. You’ll want the chatbot to understand nuances about your company, like the products you sell and the policies you uphold. Just as you’d train an employee by giving them documents to study and scripts to follow, you train a chatbot by using RAG and fine-tuning to build upon the foundation of knowledge it arrives with. This video can't play due to privacy settings To change your settings, select the "Cookie Preferences" link in the footer and opt in to "Advertising Cookies or try disabling adblockers." Red Hat resources ----------------- [Keep reading](/en/resources "Keep reading") What is RAG and how does it work? --------------------------------- RAG supplements the data within an LLM by retrieving information from sources of your choosing, such as data repositories, collections of text, and pre-existing documentation. After retrieving the data, RAG architectures process it into an LLM’s context and generate an answer based on the blended sources. RAG is most useful for supplementing your model with information that’s regularly updated. By providing an LLM with a line of communication to your chosen external sources, the output will be more accurate. And because you can engineer RAG to cite its source, it’s easy to trace how an output is formulated, which creates more transparency and builds trust. Back to our example: If you were to build a chatbot that answers questions like, “What is your return policy?”, you could use a RAG architecture. You could connect your LLM to a document that details your company’s return policy and direct the chatbot to pull information from it. You could even instruct the chatbot to cite its source and provide a link for further reading. And if your return-policy document were to change, the RAG model would pull the most recent information and serve it to the user. [Learn more about RAG](/en/topics/ai/what-is-retrieval-augmented-generation) ### ### Use cases for RAG RAG can source and organize information in a way that makes it simple for people to interact with data. With a RAG architecture, models can fetch insights and provide an LLM with context from both on-premise and cloud-based data sources. This means external data, internal documents, and even social media feeds can be used to answer questions, provide context, and inform decision making. For example, you can create a RAG architecture that, when queried, provides specific answers regarding company policies, procedures, and documentation. This saves time that would otherwise be spent searching for and interpreting a document manually. [Learn how RAG is used in software engineering](/en/blog/redefining-development-retrieval-augmented-generation-rag-revolution-software-engineering) What is fine-tuning? -------------------- Think of fine-tuning as a way to communicate intent to the LLM so the model can tailor its output to fit your goals. Fine-tuning is the process of training a pretrained model further with a smaller, more targeted data set so it can more effectively perform domain-specific tasks. This additional training data is embedded into the model’s architecture. [LoRA and QLoRA](/en/topics/ai/lora-vs-qlora) are both [parameter-efficient fine-tuning (PEFT)](/en/topics/ai/what-is-peft) techniques that can help users optimize costs and compute resources. Let’s return to our chatbot example. Say you want your chatbot to interact with patients in a medical context. It’s important that the model understands medical terminology related to your work. Using fine-tuning techniques, you can ensure that when a patient asks the chatbot about “PT services,” it will understand that as “physical therapy services” and direct them to the right resources. ### Use cases for fine-tuning Fine-tuning is most useful for training your model to interpret the information it has access to. For instance, you can train a model to understand the nuances and terminologies of your specific industry, such as acronyms and organizational values. Fine-tuning is also useful for image-classification tasks. For example, if you’re working with magnetic resonance imaging (MRI), you can use fine-tuning to train your predictive AI model to identify abnormalities. [Explore predictive AI use cases](/en/products/ai/predictive-ai) Fine-tuning can also help your organization apply the right tone when communicating with others―especially in a customer-support context. It lets you train a chatbot to analyze the sentiment or emotion of the person it’s interacting with. Further, you can train your generative AI model to respond in a way that serves the user while upholding your organization’s values. [Explore generative AI use cases](/en/products/ai/generative-ai) Considerations for choosing RAG vs. fine-tuning ----------------------------------------------- Understanding the differences between RAG and fine-tuning can help you make strategic decisions about which AI resource to deploy to suit your needs. Here are some basic questions to consider: ### What’s your team’s skill set? Customizing a model with RAG requires coding and architectural skills. Compared to traditional fine-tuning methods, RAG provides a more accessible and straightforward way to get feedback, troubleshoot, and fix applications. Fine-tuning a model requires experience with natural language processing (NLP), [deep learning](/en/topics/ai/what-is-deep-learning), model configuration, data reprocessing, and evaluation. Overall, it can be more technical and time consuming. ### Is your data static or dynamic? Fine-tuning teaches a model to learn common patterns that don’t change over time. Because it’s based on static snapshots of training data sets, the model’s information can become outdated and require retraining. Conversely, RAG directs the LLM to retrieve specific, real-time information from your chosen sources. This means your model pulls the most up-to-date data to inform your application, promoting accurate and relevant output. ### What’s your budget? RAG is typically considered to be more cost efficient than fine-tuning. To implement a RAG architecture, you build pipeline systems to connect your data to your LLM. This approach saves on cost because it uses existing data to inform your LLM. This stands in contrast to the significant resources required by fine-tuning to perform specialized data labeling and the intensive computational power needed for repeated model training. While fine-tuning is historically considered the more expensive option, developments like [vLLM](/en/topics/ai/what-is-vllm)are helping to close the budget gap. vLLM is an [inference](/en/topics/ai/what-is-ai-inference) server and engine that improves the cost efficiency of serving fine-tuned models. [Learn more about vLLM](https://www.redhat.com/en/topics/ai/what-is-vllm) How Red Hat can help -------------------- [Red Hat® AI](/en/products/ai) is built for fast, flexible, and efficient inference through its [vLLM-powered](/en/topics/ai/what-is-vllm) server. It reliably connects models to your data to unify the customization and development of specialized agents on a single platform. Built on an open source foundation, our products give you full control of AI workflows from end-to-end at any scale. The Red Hat AI portfolio includes [Red Hat AI Inference](/en/products/ai/inference), an inference stack that provides the operational control to run any model on any accelerator across the hybrid cloud. Get fast, efficient, and cost-effective inference at scale. [Explore Red Hat AI](/en/products/ai) The official Red Hat blog ------------------------- Get the latest information about our ecosystem of customers, partners, and communities. [Keep reading](/en/blog "The official Red Hat blog") All Red Hat product trials -------------------------- Our no-cost product trials help you gain hands-on experience, prepare for a certification, or assess if a product is right for your organization. [Keep reading](/en/products/trials "All Red Hat product trials") Keep reading ------------ ### What is agentic AI? Agentic AI is a software system designed to interact with data and tools in a way that requires minimal human intervention. [Read the article](/en/topics/ai/what-is-agentic-ai "article | What is agentic AI?") ### What is generative AI? Generative AI is a kind of artificial intelligence technology that relies on deep learning models trained on large data sets to create new content. [Read the article](/en/topics/ai/what-is-generative-ai "article | what is generative ai?") ### What are large language models? A large language model (LLM) is a type of artificial intelligence that uses machine learning techniques to understand and generate human language. [Read the article](/en/topics/ai/what-are-large-language-models "article | What are large language models?") Artificial intelligence resources --------------------------------- ### Related content * Blog post [A decade of open innovation: Red Hat continues to scale the open hybrid cloud with Microsoft](/en/blog/decade-open-innovation-red-hat-continues-scale-open-hybrid-cloud-microsoft) * Blog post [Introducing AutoML and AutoRAG: Guided experience for AI engineers in Red Hat OpenShift AI](/en/blog/introducing-auto-ml-and-auto-rag-guided-experience-ai-engineers-red-hat-openshift-ai) * Blog post [Faster, cheaper, just as smart: Improving the economics of LLM inference with speculative decoding](/en/blog/solving-economics-llm-inference-speculative-decoding) * Blog post [Agentic AI demands a new infrastructure stack: AMD and Red Hat deliver](/en/blog/agentic-ai-demands-new-infrastructure-stack-amd-and-red-hat-deliver) ### Related articles * [What is agentic AI?](/en/topics/ai/what-is-agentic-ai) * [LoRA vs. QLoRA](/en/topics/ai/lora-vs-qlora) * [What are Granite models?](/en/topics/ai/what-are-granite-models) * [What are large language models?](/en/topics/ai/what-are-large-language-models) * [Predictive AI vs generative AI](/en/topics/ai/predictive-ai-vs-generative-ai) * [What is Mixture of Experts (MoE)?](/en/topics/ai/mixture-of-experts) * [What is generative AI?](/en/topics/ai/what-is-generative-ai) * [What are intelligent applications?](/en/topics/ai/what-are-intelligent-applications) * [How vLLM accelerates AI inference: 3 enterprise use cases](/en/topics/ai/how-vllm-accelerates-ai-inference-3-enterprise-use-cases) * [What is AI inference?](/en/topics/ai/what-is-ai-inference) * [What is vLLM?](/en/topics/ai/what-is-vllm) * [vLLM vs. Ollama: When to use each framework](/en/topics/ai/vllm-vs-ollama) * [SLMs vs LLMs: What are small language models?](/en/topics/ai/llm-vs-slm) * [What is AgentOps?](/en/topics/ai/agentops) * [What is parameter-efficient fine-tuning (PEFT)?](/en/topics/ai/what-is-peft) * [AIOps explained](/en/topics/ai/what-is-aiops) * [What is deep learning?](/en/topics/ai/what-is-deep-learning) * [What is MLOps?](/en/topics/ai/what-is-mlops) * [What is sovereign AI?](/en/topics/ai/sovereign-ai) * [What is Model Context Protocol (MCP)?](/en/topics/ai/what-is-model-context-protocol-mcp) * [What is distributed inference?](/en/topics/ai/what-is-distributed-inference) * [What is retrieval-augmented generation?](/en/topics/ai/what-is-retrieval-augmented-generation) * [Understanding AI/ML use cases](/en/topics/ai/ai-ml-use-cases) * [What is AI in the public sector?](/en/topics/ai/what-is-ai-in-the-public-sector) * [AI infrastructure explained](/en/topics/ai/ai-infrastructure-explained) * [What is AI in healthcare?](/en/topics/ai/what-is-ai-in-healthcare) * [What is explainable AI?](/en/topics/ai/what-explainable-ai) * [Understanding AI in telecommunications with Red Hat](/en/topics/ai/understanding-ai-in-telecommunications) * [What is machine learning?](/en/topics/ai/what-is-machine-learning) * [Agentic AI vs. generative AI](/en/topics/ai/agentic-ai-vs-generative-ai) * [What is LLMops](/en/topics/ai/llmops) * [What are foundation models for AI?](/en/topics/ai/what-are-foundation-models) * [What is AI security?](/en/topics/ai/what-is-ai-security) * [What is an AI platform?](/en/topics/ai/what-is-an-ai-platform) * [What is enterprise AI?](/en/topics/ai/what-is-enterprise-ai) * [What is Model-as-a-Service?](/en/topics/ai/what-is-models-as-a-service) * [What is llm-d?](/en/topics/ai/what-is-llm-d) * [What is edge AI?](/en/topics/edge-computing/what-is-edge-ai) * [AI in banking](/en/topics/ai/ai-in-banking) * [What is InstructLab?](/en/topics/ai/what-is-instructlab) * [Why choose Red Hat Ansible Automation Platform as your AI foundation?](/en/topics/automation/automation-and-ai) * [Edge solutions for real-time decision making](/en/topics/edge-computing/edge-solutions-real-time-decision-making) * [What are predictive analytics](/en/topics/automation/how-predictive-analytics-improve-it-performance) * [How Kubernetes can help AI/ML](/en/topics/cloud-computing/how-kubernetes-can-help-ai) * [What is Kubeflow?](/en/topics/cloud-computing/what-is-kubeflow) * [Accelerate MLOps with Red Hat OpenShift](/en/technologies/cloud-computing/openshift/aiml) * [What is a data lake?](/en/topics/data-storage/what-is-a-data-lake) [More about this topic](/en/topics/ai "More about this topic")