Supercharging local AI development with RHEL on NVIDIA DGX Spark

As enterprises move from the hype of generative AI to building governed, production-ready AI applications, several new challenges have emerged. Currently, many businesses route all of their prompts to massive, cloud-based LLMs, which can result in excessive costs, high latency, and unnecessary data exposure. 

To solve this problem, Red Hat in collaboration with NVIDIA is bringing enterprise-grade AI development directly to the developer’s desk. We are excited to announce the development preview of Red Hat Enterprise Linux 10 (RHEL 10) on NVIDIA DGX Spark. 

Powered by the NVIDIA GB10 Grace Blackwell architecture, NVIDIA DGX Spark boasts up to 1 Petaflop of performance, FP4 data format support, and 128GB of unified memory. By pairing this hardware with Red Hat’s trusted operating system and developer tooling, we are delivering a high performance AI developer workstation that enables the inner development loop and local agentic AI workloads at the edge. 

This combination allows developers to build, test, and evaluate complex agentic workflows locally with familiar tools like Red Hat Desktop prior to pushing those workloads into production.  

Why does this matter?

With RHEL on NVIDIA DGX Spark, we are tackling the primary roadblocks to enterprise AI adoption, including:

• Cost control and data sovereignty: By keeping sensitive traffic local and utilizing small language models (SLMs) for medium-to-low complexity tasks, businesses can reduce their cloud API spend while restricting their sensitive data to a local workstation. Additionally, the 128GB of unified memory on NVIDIA DGX Spark enables hosting large language models (LLMs) and can help further reduce cloud costs during model development. 
   
• Evaluation-driven development: Autonomous AI agents are non-deterministic, making standard unit testing insufficient. Running RHEL on NVIDIA DGX Spark provides the ultimate sandbox for rigorous testing. Developers can use embedded MLflow instances for “glass box” trajectory tracing, capturing tool calls and model queries more effectively, and performing LLM-as-a-Judge evaluations locally before pushing anything to production.
   
• Core-to-cluster consistency: By standardizing on RHEL as the underlying OS, developers can focus on building their applications without worrying about infrastructure friction. RHEL combines production stability with developer agility to help accelerate application development across hybrid cloud environments and industry leading chip architectures. Plus, transitioning from local workstation development to large scale deployments on Red Hat OpenShift becomes a more streamlined, predictable pipeline.

The future of AI is hybrid, and it starts at the developer’s desk. Reach out to your Red Hat representative to learn more about RHEL 10 in developer preview on NVIDIA DGX Spark and how it can help accelerate your AI journey.