Why AI infrastructure must be built in the open

In Part I and Part II of this series, we explored why efficient inference is becoming an economic necessity and why sovereign AI requires infrastructure independence. But who should build the infrastructure that powers AI?

AI infrastructure influences far more than performance. It determines how quickly organizations can adopt new hardware, integrate emerging models, and adapt as the AI capabilities continue to advance.

AI is moving too fast for one company

Every layer of AI technology is changing simultaneously. New foundation models are released constantly, accelerator vendors continue introducing new architectures, and organizations are deploying AI across public clouds, private clouds, edge environments, and on-premise infrastructure.

No single company can innovate across every part of that ecosystem—and it shouldn't have to. Every organization naturally prioritizes its own products, customers, and roadmap. But enterprises need infrastructure that evolves alongside the broader ecosystem, not just to keep up with a single vendor's priorities.

Open source AI is the path forward 

Open source creates a neutral ecosystem for collaboration and shared innovation.

Instead of solving the same orchestration, scheduling, observability, and interoperability challenges in isolation, organizations work in the open while continuing to create unique value. The result is software that is more resilient, more adaptable, and better prepared to evolve with AI demands.

This collaborative model benefits everyone. Hardware vendors gain broader platform support, cloud providers gain greater flexibility, software companies spend less time rebuilding foundational capabilities, and enterprises gain the freedom to adopt new technologies without being constrained by a single vendor's roadmap.

llm-d in action

llm-d demonstrates the power of open source AI. Developed by a powerhouse of open source contributors including CoreWeave, Google, IBM, NVIDIA, Red Hat, and others, llm-d is tackling one of the most important infrastructure challenges in enterprise AI: orchestrating inference efficiently across increasingly diverse environments while maintaining high performance levels.

Every improvement made by the community strengthens a shared foundation that others can adopt, extend, and improve. Rather than creating another proprietary silo, llm-d helps build an interoperable ecosystem where innovation compounds over time.

Community creates resilience

AI will continue to change, with new models, new hardware, and new deployment patterns reshaping how organizations build intelligent applications. No one can predict what the next breakthrough will be, which is why AI that is built via open source is AI built to last. 

Projects like llm-d demonstrate that the most resilient infrastructure isn't owned by a single company—it's strengthened by a community. By solving shared challenges together, the open source ecosystem creates software that gives organizations something increasingly valuable—the freedom to adapt.

Dive deeper 

Red Hat’s Brian Stevens and Robert Shaw discuss why organizations and enterprises are turning to open source ecosystems to solve shared AI challenges.

• Watch the full discussion
• Catch a quick highlight
• Read the blog post: What is llm-d and why do we need it?
• Learn more about llm-d 

Catch up on the series:

Part I: llm-d: Breaking the cost and capacity barriers

Part II: Scaling agentic AI: How llm-d enables infrastructure sovereignty