* [Topics](/en/topics "Topics") * [Artificial intelligence](/en/topics/ai "Artificial intelligence") * SLMs vs LLMs: What are small language models? SLMs vs LLMs: What are small language models? ============================================= Updated May 12, 2026•*6*-minute read Copy URL Jump to section --------------- What are small language models?Data setsResourcesInference at scaleBenefits Limitations Use casesHow Red Hat can help What are small language models (SLMs)? -------------------------------------- A small language model (SLM) is a smaller version of a [large language model (LLM)](/en/topics/ai/what-are-large-language-models) that has more specialized knowledge, is faster to customize, and more efficient to run. SLMs are trained to have domain-specific knowledge, unlike LLMs which have broad general knowledge. Due to their smaller size, SLMs require fewer computational resources for training and deployment, reducing infrastructure costs and enabling faster fine-tuning. The lightweight nature of SLMs makes them ideal for edge devices and mobile applications. SLMs vs LLMs ------------ SLMs and LLMs are both types of [artificial intelligence (AI)](https://www.redhat.com/en/topics/ai) systems that are trained to interpret human language, including programming languages. The key differences between LLMs and SLMs are usually the size of the data sets they’re trained on, the different processes used to train them on those data sets, and the cost/benefit of getting started for different use cases. As their names suggest, both LLMs and SLMs are trained on data sets consisting of language, which distinguishes them from models trained on images (e.g., [DALL·E)](https://openai.com/index/dall-e/) or videos (e.g., [Sora](https://openai.com/index/sora/)). A few examples of language-based data sets include webpage text, developer code, emails, and manuals. One of the most well-known applications of both SLMs and LLMs is [generative AI (gen AI)](/en/topics/ai/what-is-generative-ai), which can *generate*—hence the name—unscripted content responses to many different, unpredictable queries. LLMs in particular have become well known among the general public thanks to the GPT-4 [foundation model](/en/topics/ai/what-are-foundation-models) and ChatGPT, a conversational chatbot trained on massive data sets using trillions of parameters to respond to a wide range of human queries. Though gen AI is popular, there are also non-generative applications of LLMs and SLMs, like [predictive AI](/en/topics/ai/predictive-ai-vs-generative-ai). [Top considerations for building a production-ready AI/ML environment](/en/resources/building-production-ready-ai-environment-ebook) LLMs and SLMs are usually trained on different data sets -------------------------------------------------------- The scope of GPT-4/ChatGPT is an excellent example that demonstrates one common difference between LLMs and SLMs: the data sets they’re trained on. LLMs are usually intended to emulate human intelligence at a very broad level, and thus are trained on a wide range of large data sets. In the case of GPT-4/ChatGPT, that includes the entire public internet(!) up to a certain date. This is how ChatGPT has gained notoriety for interpreting and responding to such a wide range of queries from general users. However, this is also why it has sometimes gained attention for potentially incorrect responses, colloquially referred to as “[hallucinations](/en/blog/when-llms-day-dream-hallucinations-how-prevent-them)”—it lacks the fine-tuning and domain-specific training to accurately respond to every industry-specific or niche query. SLMs on the other hand are typically trained on smaller data sets tailored to specific industry domains (i.e. areas of expertise). For example, a healthcare provider could use an SLM-powered chatbot trained on medical data sets to inject domain-specific knowledge into a user’s non-expert query about their health, enriching the quality of the question and response. In this case, the SLM-powered chatbot doesn’t need to be trained on the entire internet—every blog post or fictional novel or poem ever written—because it’s irrelevant to the healthcare use case. In short, SLMs typically excel in specific domains, but struggle compared to LLMs when it comes to general knowledge and overall contextual understanding. [LoRA v QLoRA explained](/en/topics/ai/lora-vs-qlora) Red Hat resources ----------------- [Keep reading](/en/resources "Keep reading") LLMs and SLMs require different resources ----------------------------------------- Training any model for a business use case, whether LLM or SLM, is a resource-intensive process. However, training LLMs is especially resource intensive. In the case of GPT-4, a total of 25,000 NVIDIA A100 GPUs ran simultaneously and continuously for 90-100 days. Again, GPT-4 represents the largest end of the LLM spectrum. Other LLMs like [Granite](https://www.ibm.com/granite) didn’t require as many resources. Training an SLM still likely requires significant compute resources, but far fewer than an LLM requires. ### Resource requirements for training vs inference It’s also important to note the difference between model *training* and model *inference*. As discussed above, training is the first step in developing an AI model. Inference is the process a trained AI model follows to make predictions on new data. For example, when a user asks ChatGPT a question, that invokes ChatGPT to return a prediction to the user—that process of generating a prediction is an inference. Some pretrained LLMs, like the Granite family of models, can make inferences using the resources of a single high-power workstation (e.g., Granite models can fit on one V100-32GB GPU2), although many require multiple parallel processing units to generate data. Furthermore, the greater the number of concurrent users accessing an LLM, the slower the model runs inferences. SLMs on the other hand are usually designed to make inferences with the resources of a smartphone or other mobile device. AI inference at scale --------------------- There are many different factors that can impact the success of [inference](/en/topics/ai/what-is-ai-inference) at scale. Mainly, it depends on how efficiently and effectively your moving pieces are working together. Specifically, inference servers that can support larger AI models (like LLMs) and their more complex inference capabilities are essential to scaling AI workloads for the enterprise. [Why you should care about inference](/en/artificial-intelligence/inference) These AI tools use resources more efficiently to inference at scale faster: * [llm-d](/en/topics/ai/what-is-llm-d): LLM prompts can be complex and nonuniform. They typically require extensive computational resources and storage to process large amounts of data. llm-d, an open source AI framework, uses well-lit paths to help developers use techniques like distributed inference to support the increasing demands of sophisticated and larger resoning models like LLMs. * [Distributed inference](/en/topics/ai/what-is-distributed-inference): Distributed inference lets AI models process workloads more efficiently by dividing the labor of inference across a group of interconnected devices. Think of it as the software equivalent of the saying, “many hands make light work.” * [vLLM](/en/topics/ai/what-is-vllm): vLLM, which stands for virtual large language model, is a library of open source code maintained by the vLLM community. It helps large language models (LLMs) perform calculations more efficiently and at scale. It's helping organizations like [LinkedIn, Roblox, and Amazon](/en/topics/ai/how-vllm-accelerates-ai-inference-3-enterprise-use-cases) speed up their inference capabilities. [Find benefits and use cases of fast, efficient inference](/en/products/ai/fast-efficient-inference) Benefits of SLMs ---------------- There’s no answer to the question “which model is better?” Instead, it depends on your organization’s plans, resources, expertise, timetable, and other factors. It’s also important to decide whether your use case necessitates training a model from scratch or fine-tuning a pretrained model. Common considerations between LLMs and SLMs include: ### Cost In general, LLMs require far more resources to train, fine-tune, and run inferences. Importantly, training is a less frequent investment. Computing resources are only needed while a model is being trained, which is an intermittent and not continuous task. However, running inferences represents an ongoing cost, and the need can increase as the use of the model is scaled to more and more users. In most cases, this requires cloud computing resources at scale, a significant on-premise resource investment, or both. SLMs are frequently evaluated for low-latency use cases, like edge computing. That’s because they can often run with just the resources available on a single mobile device without needing a constant, strong connection to more significant resources. [From the Red Hat blog: Tips for making LLMs less expensive](/en/blog/making-llms-environmentally-and-budget-friendly) ### Expertise Many popular pre-trained LLMs―like Granite, Llama, and GPT-4― offer a more “plug-and-play” option for getting started with AI. These are often preferable for organizations looking to begin experimenting with AI since they don’t need to be designed and trained from scratch by data scientists. SLMs, on the other hand, typically require specialized expertise in both data science and industry knowledge domains to accurately fine-tune on niche data sets. ### Security One potential risk of LLMs is the exposure of sensitive data through [application programming interfaces (APIs)](/en/topics/api/what-are-application-programming-interfaces). Specifically, fine-tuning an LLM on your organization’s data requires careful attention to compliance and company policy. SLMs may present a lower risk of data leakage because they offer a greater degree of control. Limitations of SLMs ------------------- As businesses integrate SLMs into their workflows, it’s important to be aware of the limitations they present. ### Bias SLMs are trained on smaller data sets, meaning you can more easily mitigate the biases that will inevitably occur (compared to LLMs). However, as with language models of any size, training data can still introduce biases, such as the underrepresentation or misrepresentation of certain groups and ideas, or factual inaccuracies. Language models can also inherit biases related to dialect, geographical location, and grammar. Teams should pay extra attention to quality of training data in order to limit biased outputs. ### Narrow scope of knowledge SLMs have a smaller pool of information to pull from as they generate responses. This makes them excellent for specific tasks, but less suitable for tasks that require a wide scope of general knowledge. Teams might consider creating a collection of purpose-built SLMs to use alongside an LLM (or LLMs). This solution becomes especially interesting if teams are able to pair models with existing applications, creating an interconnected workflow of multiple language models working in tandem. SLM use cases ------------- The adaptability of SLMs makes them beneficial for a variety of use cases. ### Chatbots Use an SLM to train a chatbot on specialized materials. For example, a customer service chatbot might be trained with company-specific knowledge so it can answer questions and direct users to information. ### Agentic AI Integrate SLMs into an [agentic AI](/en/topics/ai/what-is-agentic-ai)workflow so they can complete tasks on behalf of a user. [Explore agentic AI use cases](/en/products/ai/agentic-ai) ### Generative AI SLMs can perform tasks such as generating new text, translating existing text, and summarizing copy. [Explore gen AI use cases](/en/products/ai/generative-ai) How Red Hat can help -------------------- [Red Hat® AI](/en/products/ai) is built for fast, flexible, and efficient inference through its [vLLM-powered](/en/topics/ai/what-is-vllm) server. It reliably connects models to your data to unify the customization and development of specialized agents on a single platform. Built on an open source foundation, our products give you full control of AI workflows from end-to-end at any scale. The Red Hat AI portfolio includes [Red Hat AI Inference](/en/products/ai/inference), an inference stack that provides the operational control to run any model on any accelerator across the hybrid cloud. Get fast, efficient, and cost-effective inference at scale. [Explore Red Hat AI](/en/products/ai) The official Red Hat blog ------------------------- Get the latest information about our ecosystem of customers, partners, and communities. [Keep reading](/en/blog "The official Red Hat blog") All Red Hat product trials -------------------------- Our no-cost product trials help you gain hands-on experience, prepare for a certification, or assess if a product is right for your organization. [Keep reading](/en/products/trials "All Red Hat product trials") Keep reading ------------ ### What is agentic AI? Agentic AI is a software system designed to interact with data and tools in a way that requires minimal human intervention. [Read the article](/en/topics/ai/what-is-agentic-ai "article | What is agentic AI?") ### What is generative AI? Generative AI is a kind of artificial intelligence technology that relies on deep learning models trained on large data sets to create new content. [Read the article](/en/topics/ai/what-is-generative-ai "article | what is generative ai?") ### What are large language models? A large language model (LLM) is a type of artificial intelligence that uses machine learning techniques to understand and generate human language. [Read the article](/en/topics/ai/what-are-large-language-models "article | What are large language models?") Artificial intelligence resources --------------------------------- ### Related content * Blog post [The future of AI demands a hybrid foundation](/en/blog/future-ai-demands-hybrid-foundation) * Blog post [AI in production at the industrial edge: A repeatable path with Red Hat and Intel](/en/blog/ai-production-industrial-edge-repeatable-path-red-hat-and-intel) * Blog post [Fragnesia and friends: When page cache vulnerabilities keep coming back](/en/blog/fragnesia-and-friends-when-page-cache-vulnerabilities-keep-coming-back) * E-book [Generative AI in action](/en/resources/generative-ai-in-action-ebook) ### Related articles * [What is agentic AI?](/en/topics/ai/what-is-agentic-ai) * [What is vLLM?](/en/topics/ai/what-is-vllm) * [vLLM vs. Ollama: When to use each framework](/en/topics/ai/vllm-vs-ollama) * [RAG vs. fine-tuning](/en/topics/ai/rag-vs-fine-tuning) * [LoRA vs. QLoRA](/en/topics/ai/lora-vs-qlora) * [What are Granite models?](/en/topics/ai/what-are-granite-models) * [What are large language models?](/en/topics/ai/what-are-large-language-models) * [Predictive AI vs generative AI](/en/topics/ai/predictive-ai-vs-generative-ai) * [What is Mixture of Experts (MoE)?](/en/topics/ai/mixture-of-experts) * [What is generative AI?](/en/topics/ai/what-is-generative-ai) * [What are intelligent applications?](/en/topics/ai/what-are-intelligent-applications) * [How vLLM accelerates AI inference: 3 enterprise use cases](/en/topics/ai/how-vllm-accelerates-ai-inference-3-enterprise-use-cases) * [What is AI inference?](/en/topics/ai/what-is-ai-inference) * [What is AgentOps?](/en/topics/ai/agentops) * [What is parameter-efficient fine-tuning (PEFT)?](/en/topics/ai/what-is-peft) * [AIOps explained](/en/topics/ai/what-is-aiops) * [Understanding AI in telecommunications with Red Hat](/en/topics/ai/understanding-ai-in-telecommunications) * [What is machine learning?](/en/topics/ai/what-is-machine-learning) * [Agentic AI vs. generative AI](/en/topics/ai/agentic-ai-vs-generative-ai) * [What is LLMops](/en/topics/ai/llmops) * [What are foundation models for AI?](/en/topics/ai/what-are-foundation-models) * [What is deep learning?](/en/topics/ai/what-is-deep-learning) * [What is MLOps?](/en/topics/ai/what-is-mlops) * [What is sovereign AI?](/en/topics/ai/sovereign-ai) * [What is Model Context Protocol (MCP)?](/en/topics/ai/what-is-model-context-protocol-mcp) * [What is distributed inference?](/en/topics/ai/what-is-distributed-inference) * [What is retrieval-augmented generation?](/en/topics/ai/what-is-retrieval-augmented-generation) * [Understanding AI/ML use cases](/en/topics/ai/ai-ml-use-cases) * [What is AI in the public sector?](/en/topics/ai/what-is-ai-in-the-public-sector) * [AI infrastructure explained](/en/topics/ai/ai-infrastructure-explained) * [What is AI in healthcare?](/en/topics/ai/what-is-ai-in-healthcare) * [What is explainable AI?](/en/topics/ai/what-explainable-ai) * [What is AI security?](/en/topics/ai/what-is-ai-security) * [What is an AI platform?](/en/topics/ai/what-is-an-ai-platform) * [What is enterprise AI?](/en/topics/ai/what-is-enterprise-ai) * [What is Model-as-a-Service?](/en/topics/ai/what-is-models-as-a-service) * [What is llm-d?](/en/topics/ai/what-is-llm-d) * [What is edge AI?](/en/topics/edge-computing/what-is-edge-ai) * [AI in banking](/en/topics/ai/ai-in-banking) * [What is InstructLab?](/en/topics/ai/what-is-instructlab) * [Why choose Red Hat Ansible Automation Platform as your AI foundation?](/en/topics/automation/automation-and-ai) * [Edge solutions for real-time decision making](/en/topics/edge-computing/edge-solutions-real-time-decision-making) * [What are predictive analytics](/en/topics/automation/how-predictive-analytics-improve-it-performance) * [How Kubernetes can help AI/ML](/en/topics/cloud-computing/how-kubernetes-can-help-ai) * [What is Kubeflow?](/en/topics/cloud-computing/what-is-kubeflow) * [Accelerate MLOps with Red Hat OpenShift](/en/technologies/cloud-computing/openshift/aiml) * [What is a data lake?](/en/topics/data-storage/what-is-a-data-lake) [More about this topic](/en/topics/ai "More about this topic")