Scalable and cost-effective fine-tuning for LLMs

For general purpose AI use cases, a large language model's (LLM) capability to understand patterns and relationships in a wide range of data is often enough. To gain a competitive advantage, however, an organization needs to leverage its own specific domain expertise—in other words, its secret sauce. The domain-specific nature of a business needs a customized LLM to fully exploit its taxonomy, skills and knowledge.

So the question becomes: To gain a competitive advantage, how can you adapt a general purpose LLM to a specific use case, knowledge domain, lingo, customer input, etc.? And how can you do it in a cost-effective way? Ideally, you want to start small, evolve quickly and continuously provide value to your business.

There are a number of ways to approach this, including prompt-tuning and retrieval augmented generation (RAG), but to overcome the limitations of these techniques, we need to also look at fine-tuning, the process of adjusting a pre-trained model on a specific dataset or task to improve its performance for a particular application.

There are some challenges related to fine-tuning and LLM refinement, however, including:

• Fine-tuning an LLM to understand a specific area of knowledge typically involves running expensive, resource-intensive training
• LLM refinements have typically required large amounts of carefully curated human-generated data, which can be time-consuming and expensive to get. This data can also carry security and privacy concerns
• Fine-tuning requires data scientists who are increasingly expensive and hard to find

How can Red Hat AI help?

Red Hat AI accelerates enterprise AI adoption with small, purpose-built models, efficient customization techniques and the flexibility to develop and deploy anywhere. The Red Hat AI portfolio is composed of Red Hat Enterprise Linux AI (RHEL AI), Red Hat OpenShift AI, accelerators and services giving customers a comprehensive set of capabilities.

RHEL AI is built to develop, test and run generative AI (gen AI) foundation models, and includes a version of InstructLab, a community-driven project that makes it easier for developers to experiment with IBM’s Granite LLMs.

About InstructLab

InstructLab (LAB) is based on IBM Research’s work on Large-scale Alignment for chatBots. The LAB method consists of three phases:

• Data curation: This is a collaborative approach aimed at subject matter experts who do not have data science training or experience. The LAB method allows these non-data scientists to contribute a curated taxonomy of domain-specific knowledge and skills
• Large scale synthetic data generation: A model is used to generate more examples based on the curated data. Using synthetic data like this is a valuable practice to influence a model and expand the domain knowledge available to a model. This data on its own is also a valuable asset and its automatic generation is economical, more secure and does not include any personally identifiable information (PII)
• Iterative large-scale tuning: Finally, the model is retrained based on the generated synthetic data. The LAB method includes two tuning sub-phases: knowledge tuning, followed by skill tuning

With the LAB method, the model keeps its original generative power and accuracy from its original LLM training while acquiring new skills.

Start small and scale as you go

Red Hat OpenShift AI provides a platform to run the LAB method in enterprise environments. OpenShift AI is an open source AI platform that helps manage the lifecycle of AI models and AI-enabled applications. It provides services to develop models and automate AI processes like feature pipelines, model training and tuning. It also includes out-of-the-box services for experiment tracking, model versioning and overall monitoring. OpenShift AI leverages and supports a number of popular open source projects. Specifically for the LAB method we use: 

• Data science pipelines (DSP): A service based on Kubeflow Pipelines for building and deploying portable and scalable AI workflows
• Kubeflow Training Operator (KFTO): An operator for fine-tuning and for scalable, distributed model training 

Red Hat has automated the LAB method using DSP and the KFTO to make the whole process more scalable, efficient and auditable. 

With DSP we can configure the LAB fine-tuning process using a directed acyclic graph (DAG) to provide a visual representation of the process. The different phases and the execution status are represented in a human-friendly way, understandable to all stakeholders. AI engineers can also monitor the progress of the LAB method process from the OpenShift AI dashboard and view the different outputs, including metrics and the tuned model itself. These are versioned and tracked automatically so AI engineers and data scientists can compare the changes in model performance as they iterate and modify parameters and resources. 

With the integration between KFTO and DSP, data scientists and machine learning operations (MLOps) teams can leverage the power of their existing OpenShift cluster in a cost-effective and scalable way. Depending on the desired investment, organizations can configure the resource quota and number of GPU-enabled OpenShift worker nodes to run the training phase. KFTO manages the scalability and efficient use of these resources on behalf of the user. OpenShift AI also helps subject matter experts, data scientists and AI engineers collaborate via a UI adapted to these different users.

Learn more about DSP Data science pipelines on OpenShift and start your journey towards scalable fine-tuning by reading How to fine-tune LLMs with Kubeflow Training Operator on the Red Hat Developer blog.