Introducing AutoML and AutoRAG: Guided experience for AI engineers in Red Hat OpenShift AI

As organizations move from AI experimentation toward scalable production, the complexity of manually building high-performing AI models and retrieval systems often creates a significant bottleneck. Red Hat OpenShift AI 3.4 helps address this with the introduction of AutoML and AutoRAG, now available as technology previews. These tools are designed to automate the labor-intensive aspects of the AI lifecycle, allowing teams to focus on business impact rather than manual configuration.

What is AutoML?

AutoML in Red Hat OpenShift AI 3.4 brings enterprise-grade automated machine learning to your fingertips. Whether you're a business analyst exploring data or a data scientist looking to accelerate your workflow, AutoML handles the heavy lifting of model development.

Think of AutoML as a co-pilot for your data science team. Rather than replacing data scientists, it handles the exhausting search for well-performing configurations, allowing practitioners to focus on problem framing, data quality, and business impact.

 How it works

AutoML consists of 2 powerful components working together:

The AutoML backend is our core engine, an AI pipeline built on Kubeflow Pipelines and the open source AutoGluon library. It automatically handles data processing, feature engineering, and model training. Engineers can trigger pipelines programmatically via the Pipelines API, but pipelines can also be created, configured, and monitored through the OpenShift AI dashboard.

The AutoML user interface (UI) embedded into the Openshift AI dashboard makes machine learning accessible to everyone. The workflow is simple: upload your CSV data or point to an S3 source, select your target column, choose your algorithm type (Binary Classification, Multiclass Classification, Regression, or Time Series), and launch your AutoML run. That's it.

 Within minutes, AutoML delivers:

•  A model leaderboard showing top-performing models optimized for your specific task
• Production-ready trained models available for deployment
• An interactive Jupyter notebook to load and experiment with the best predictor

What's on the horizon

We're just getting started with AutoML. In upcoming releases of OpenShift AI, we're planning further enhancements, including:

• Predictive models as agentic tools: Expose your trained models as callable tools within agentic systems, enabling AI agents to leverage your custom predictive models for decision-making and automation workflows.
• LLM-powered time series forecasting: Harness the power of large language models for more accurate and context-aware time series predictions.
• Expanded data format support and additional algorithm types to cover even more use cases.
• Explainability for ML algorithms.

What is AutoRAG?

AutoRAG provides automated optimization of RAG pipelines. RAG helps improve the accuracy of large language model (LLM) responses by grounding them in an organization's own data. Instead of relying only on training knowledge, RAG retrieves relevant information from an enterprise knowledge base at inference time. This improves responses by making them more factual, current, and domain-specific.

Building a performant RAG pipeline involves dozens of decisions regarding parsing, chunking, embedding models, retrieval strategies, and prompt construction. Each choice affects accuracy, latency, and cost. AutoRAG helps address this by automating the evaluation and selection of these components, benchmarking different configurations to find the optimal combination for specific data.

What can AutoRAG do?

AutoRAG treats the RAG pipeline as a modular optimization problem where each stage can be configured independently. 

Specifically, it automates:

• Document parsing and chunking: Evaluating multiple approaches to find the configuration that preserves the most useful signal for retrieval across different document types.
• Query expansion: Testing techniques like hypothetical document embeddings (HyDE) or multiquery expansion to see which delivers the best answer quality for a given corpus.
• Retrieval strategy selection: Benchmarking keyword-based (BM25), dense vector, and hybrid approaches to identify the best method for specific query types.
• Passage reranking: Evaluating reranker configurations to surface the most relevant content before it is passed to the LLM.
• End-to-end pipeline evaluation: Scoring configurations against a curated dataset using retrieval metrics (precision, recall) and generation metrics (faithfulness, relevancy).

Benefits of AutoRAG

AutoML and AutoRAG use cases

Predictive AI: Accelerating model development at scale

Enterprise data science teams routinely face a backlog of modelling requests—from churn prediction and fraud detection to demand forecasting and quality inspection. AutoML helps these teams deliver more models faster by automating the most time-consuming elements of the model development cycle.

A financial services organization, for example, might use AutoML to run parallel experiments across dozens of algorithm configurations on a credit risk dataset, automatically selecting the best-performing model and packaging it for deployment into a production scoring pipeline.

Generative AI: Optimizing RAG for domain-specific accuracy

Organizations deploying LLM-powered applications, internal knowledge assistants, customer support bots, or document Q&A tools face a common challenge: out-of-the-box RAG pipelines rarely perform optimally on first deployment. The optimal chunking strategy for dense technical documentation differs from that for short product descriptions. The best retrieval approach for a legal corpus differs from that for a customer FAQ.

AutoRAG enables teams to bring a scientific, automated approach to this problem. Rather than relying on expert intuition or trial-and-error, teams can define an evaluation dataset, run AutoRAG over a grid of pipeline configurations, and deploy the best-performing configuration with confidence.

Connecting predictive and generative AI

One of the more powerful emerging patterns is combining predictive AI outputs with gen AI capabilities. An AutoML-trained anomaly detection model might flag an anomaly, trigger a retrieval step over relevant operational runbooks, and surface a contextually relevant recommendation to an engineer—all in a single automated workflow. AutoML and AutoRAG each play a role in making these hybrid pipelines more accurate and reliable.

AutoML, AutoRAG, and the enterprise AI pipeline

Both AutoML and AutoRAG are best understood as optimization layers that sit within a broader enterprise AI platform. They are not standalone products. They require infrastructure for data ingestion, experiment tracking, model storage, serving, and monitoring.

For enterprises, this means that the value of AutoML and AutoRAG depends heavily on the platform on which they run. Key platform requirements include:

• Data access: Both approaches require access to clean, well-governed data—whether labelled training data for AutoML or a curated document corpus and evaluation dataset for AutoRAG.
• Experiment tracking: Systematic logging of every configuration, metric, and artifact produced during optimization is essential for reproducibility and governance.
• Model and pipeline registry: Once an optimal model or RAG pipeline has been identified, it needs to be stored, versioned, and promoted through a deployment workflow.
• Monitoring and drift detection: Both predictive models and RAG pipelines degrade over time as data distributions shift. Ongoing monitoring and re-evaluation are essential.

Limitations and considerations

While powerful, these tools are not substitutes for domain expertise:

• Data quality: AutoML cannot fix poorly labelled data, and AutoRAG cannot compensate for a low-quality document corpus.
• Evaluation design: Poorly constructed evaluation sets in AutoRAG lead to pipelines optimized for the wrong objectives.
• Explainability: Understanding why complex AutoML architectures perform well remains a challenge in regulated industries.
• RAG hygiene: AutoRAG accelerates refinement but does not replace foundational work, such as clean ingestion and well-formed prompts.

How Red Hat can help

Red Hat AI provides the platform foundation that makes both AutoML and AutoRAG practical at enterprise scale. Across the data, model, and application lifecycle, Red Hat AI delivers the infrastructure, tooling, and operational capabilities that organizations need to build, optimize, and sustain AI systems.

For AutoML: Red Hat AI provides data scientists with access to self-service model development environments, and automated workflows that simplify the entire machine learning lifecycle. With  integrated AI pipelines, model registries, and serving capabilities, AutoML-generated models move  reliably from experimentation to production—no complex infrastructure management required.

For AutoRAG: Red Hat AI simplifies the data ingestion and pre-processing required to build high-quality RAG evaluation datasets, and provides the inference infrastructure needed to benchmark RAG pipeline configurations at scale. Its model-serving capabilities, powered by vLLM, mean the optimized pipelines identified through AutoRAG evaluation can be deployed efficiently and cost-effectively.

Hybrid cloud: Red Hat AI's hybrid cloud architecture means AutoML and AutoRAG workloads can run wherever the data, compute, or regulatory requirements demand, whether in an on-premises data centre, a public cloud, or an air-gapped environment. MLOps and genAIOps capabilities provide the operational consistency to manage model and pipeline lifecycles over time, with the observability and governance controls that enterprise deployments require.

Want to go deeper?

Red Hat AI 3.4 is here, and AutoRAG and AutoML are both in technical preview. Discover how enterprises are moving from experimentation to production AI.

Learn more about Red Hat AI.