Technically Speaking

This video can't play due to privacy settings

To change your settings, select the "Cookie Preferences" link in the footer and opt in to "Advertising Cookies."

Defining sovereign AI with open source ft. Jered Floyd

2026 年 7 月 1 日 | Technically Speaking Team Artificial intelligence

What happens to your business operations when your AI workflows run on a proprietary black box? Red Hat CTO Chris Wright and Jered Floyd break down the 4 pillars of sovereign AI and challenge common misconceptions about data ownership and outline a clear path to platform autonomy.

Transcript

00:00 - Chris Wright
In the tech industry, we love a good catchall term. For years, it was digital transformation. But if you've been at any of the big tech conferences lately, you know the focus has shifted. Today, it's all about sovereignty. To some, it sounds like business jargon or just a policy check-box. But for the architect, sovereignty is about agency. So, joining me to cut through the noise is Jered Floyd from our field CTO team. Welcome to "Technically Speaking", where we explore how open source is shaping the future of technology. I'm your host, Chris Wright. Jered, great to have you here.

00:38 - Jered Floyd
Thank you.

00:39 - Chris Wright
So, imitation is the highest form of flattery. I stole from you this reframing of digital transformation into sovereignty. Tell me a little bit about why you frame it that way. What's important about that reframing, and more specifically, what's happened in the last 12, 18 months that's really brought this conversation to the foreground?

01:01 - Jered Floyd
Yeah, for sure. So, the reason I've taken that framing is because digital transformation, something we've talked about for years, it's gone through a peak and then we've moved on to newer topics, is a way of bringing together a whole bunch of technology shifts that have happened across computing. So, we talked about things like containerization, about microservices, about DevOps, all things that we just do today. But they really held together to drive a fundamental shift in how applications were built. And so, I think digital sovereignty looks the same way because it's about fundamental changes in how we manage data, how we manage autonomy around ownership of that data, operation of the software, development of the software, and then meeting regulatory rules and feeling assured about the integrity of our systems. Like, you saw some fines around things like laws like GDPR, but this is not so much about, "Oh, we're worried we're going to get fined." This is really about risks to just overall operation to the business. But also, there's a lot of understanding now of, "Oh, we had risks we didn't know before about our ability to operate, but also our ability to innovate." And so, by applying these technologies, these companies can move more quickly by bringing more know-how in-house, by bringing more operational assurance in-house and build new things faster. So, it's kind of a win-win. You get to mitigate some risk, but you also get to do more for your business.

02:31 - Chris Wright
Yeah, that's a nice combination of new capabilities and moving quickly with the rapid-changing demands while at the same time de-risking your business. Usually, it's trade offs, so this is an interesting way to take a position forward with agency, ownership, or control over your software stack and operations and all the things we'll get into, that also brings you capabilities and speed, which is an interesting shift. And I know there's a whole set of conversations around what exactly makes up sovereignty. Before we jump into that, I'll hear terms, sovereign cloud, sovereign AI. Are they the same? Is one a superset of the other? How do you approach the conversation between different labels with sovereignty?

03:24 - Jered Floyd
Yeah, so we have four pillars that we categorize sovereign operations into, or sovereign capabilities into. But for sovereign cloud versus sovereign AI, I mean, you can take any sort of technology space. Cloud, cloud is being able to run your capabilities outside of your data center. Sovereign means that you have levels of control over that. The same thing with AI. Are you able to run your AI technologies in your data center, or at least in an AI data center that is in the same region, the same country, under the same control domain? And so, the way we've been thinking about sovereignty overall is in four pillars. So, data sovereignty, operational sovereignty, technology sovereignty, and assurance sovereignty. If you look at data sovereignty, that has to do with, where is your data being stored and processed? Am I shipping it to another country which may have different laws where it may be subject to stronger or weaker laws? That's really important, especially because the data is the lifeblood of any modern business. So, having any one risk associated with that, huge challenge. Also, having that leak, having that be stolen, huge, huge, huge challenge. The second area, operational sovereignty, is who has control over your systems? If they're running in your own data center, you at least have physical control over those systems. You may have outsourced the operations to a third party. So, again, you have to be concerned about where that third party is, what access, what control they have to both the technology as well as, going back to the first one, the data. Then, I'd say technology sovereignty is, who has ownership over the technology that you're using? And this is an area where open source is absolutely critical. Because with technology, okay, so if I'm using a closed-source product, I have no ability to know where it's being built, make modifications to it, even audit that it's doing what I think it's supposed to be doing. With open source, almost all open source is global. The Linux kernel is developed in probably close to every country. I don't know if every continent. It'd be interesting if there are any Arctic or Antarctic software developers working on the Linux kernel.

05:41 - Chris Wright
The penguins, for sure.

05:43 - Jered Floyd
Yeah, definitely! So, with technology sovereignty, then you have the ability with open source to know that you have the ability to make changes at any time. You have the ability to continue operating even if your provider disappears. So, if you can't have a particular business provider anymore, you still have the software. And that's super critical to having that overall sovereignty, as well as assurance. And so, getting to assurance, assurance is, how do I know that I can meet these regulations? How do I know that the software's doing what it says? How do I know that the people who have access to my software actually have the access that they're claiming they have? And again, that's an area where you can do audits, where you can do audits against both the code itself, which is very authoritative, or your processes, slightly less authoritative, but at least you can test them. And then, overall assurance about, some of these regulations, for example, require that you actually test your ability to leave a service provider. I can't quite imagine, I only have about a dozen VMs in a major cloud provider, and I can't even imagine how long it would take for me to move those. Large financial institutions have to show and prove that they can do that right now.

07:04 - Chris Wright
Yeah, part of DORA has this. How quickly can you repatriate a workload from potentially failed infrastructure to another infrastructure? And it's essentially instantaneous in terms of what the expectations are. And so, certainly, automation becomes really important, but I think that layout of operational, technical, and these are aspects that I think don't always come to the foreground. Data, I think, is a pretty natural understanding, but there's operational data as well as the customer data. And that, I think, gets a little lost in this conversation. I'm glad you bring it to the foreground. I was speaking with a customer who was talking about a sovereign zone from a global hyperscaler. This sovereign zone required authentication into the physical access, into the colocation area with somebody in the US. Like, "Oh, I think there's some operational concerns, there," or running this infrastructure in a sovereign area, but taking the operational data into a different geography. These are the interesting corner cases that I don't think always surface. But I really like this view of open source as a tool in the toolbox of sovereignty. And of course, who doesn't love open source, right? We're here talking about open source all the time, but used in this context. And the interesting twist of the software license for open source comes from the project itself. And you have licensed agency over the software as you use open source software, quite independent from the business relationship you have from a provider. I think that's a really unique role that open source plays in this entire stack. I often think of, as we shift into the AI world where AI and data are talked about hand-in-glove with one another, and data being such a typical part of the sovereignty conversation. The AI stack, has this next-generation software stack feel, and today's generation is very much about Linux and applications. That next generation is about inferencing in production environments with PyTorch and vLLM, and maintaining that openness, the ability to see what's inside the auditing, understanding how it should work, and the licensing. I think these are really important aspects of sovereignty. How do those conversations resonate with users, with a bank that's looking at a regulatory question about resiliency, for example?

09:55 - Jered Floyd
I see stories all the time about businesses basically being held ransom for licensing renewals because a software license, traditional software licenses, you can use this for the next year. And then, if you're really unlucky, it just automatically shuts off and then you can't do anything at all, or you're just out of compliance with the license and possibly subject to huge fines. With open source licensing, you have the right to use the software with subscriptions. You might not get access to updates, but you still have the right to continue using the software and take over responsibility yourself, or find another provider that has more appropriate terms for your business. I think, from an AI perspective, that's also really interesting talking about data, because data is the core of every AI model. You don't use AI except for the purposes of processing your data, inferencing your data, trying to figure out trends in your data, and respond to requests. And so, all of the data sovereignty topics that we talked about are directly applicable to AI. And there, you're thinking about, again, where's my AI running, right? Is my AI co-located with my data? Am I sending my super critical data to a data center that happened to be able to get all of the accelerator chips that I needed or has the models that I want to use that aren't available to run on my own, in my own data center? Those are the things that you have to start thinking about when you're building your architecture. And for most of the critical business processes where you're applying generative AI, you don't need a chatbot. Instead, you start looking at smaller models. And some of these smaller models, smaller open-weight models, for example, you can run in your own data center. And some of these smaller models, you can even do the training yourself. So, there's a whole spectrum here of completely black-box LLM, which, a frontier model, which is super cool, but I don't know anything about how it was trained. I don't know if there are backdoors in its training. I don't know what data went into it, what biases went into it. Then, you have the smaller models that are open-weight models that often publish all the content that went into it, an inventory of the content that went into it, and those, I can run myself. And so, that gives me a little more sovereignty around the operations and the data, what's going in, what's going out. I know all those flows. And then, you start looking at the technology sovereignty side of this, and you look at being able to train small models yourself that you know absolutely everything. You know that there are no backdoors where it's going to send data off or insert a particular bias in a particular way. So, there's a whole spectrum of what fits your business need and what level of sovereignty, and then, what level of innovation you can apply to that that you might not have thought about before when you were just like, "Oh, it's all tokens. I just use the model that I think looks coolest and does the neatest things, and I'm just spending tokens." But now you get to think about, "Well, how's that gonna affect me next year? How's that going to affect me in five years? And where is that data going and where is that know-how going?" Where do you see the challenges in creating, building, and operating sovereign infrastructure? I mean, it sounds to me a lot of what you're talking about is not a lot different from what's possible today with taking open source projects, Linux, Kubernetes, KubeVirt, vLLM, building a private cloud infrastructure, deploying that within your own data center. Is there something that's deeply different, or is part of the value that we're leveraging all this well-understood technology?

Yeah, so a lot of this is the technology we've already been building, right? And open source drives the core of all of that. It's more about knowing where these things are happening, building processes to know, to follow where these things are happening. And then, the software that's needed to help automate that processing, making sure that this can be run in a reasonable way in your data center. I think you mentioned vLLM earlier, and vLLM is fundamentally about reducing your cost and being able to fit your inferencing into a box that's a box that your business can run. And so, the technology advances are really about, how do we empower companies to meet their sovereignty goals?

So, the operational view is, a lot of technology that we already understand, some automation to build, deliver the capabilities, but also, there's some kind of auditing required to understand, where is the data? Where are things running? Are there tools that we're building together in new open source communities, or are we also able to leverage the existing telemetry and observability that we have today already?

So, a lot of that is building on the observability that we have today. And then, we can edit this out, because I'm not sure if there is an actual tool that I can point to right now about how we're going to do that, although that's a good idea. I don't wanna say things like OpenSCAP, right? You know, that we have controls, control auditing, but it doesn't sound like it's the right answer, there. Like, it's the same things we're doing in zero trust. It's like you can ensure that you're auditing the control points that you have. And so, we have tools to assist with that that could fit in here, right? That we can have tools that help you inventory and audit your control points the same way you would for zero trust compliance, right? So, that's another thing that fits in between digital transformation before and digital sovereignty now, is zero trust is another one of those big-picture things that we talk about with IT infrastructure. And when we talk about what sort of level of knowledge and competency and clarity they have, that's a little past the hump, now. You don't hear about zero trust as much as you did a couple years ago. But it doesn't mean it's irrelevant. It means that the principles there about not having implicit trust, actually auditing every transaction, have been incorporated into control points and software. And we see that with things like service mesh, like Istio. So, those same sort of things will happen with sovereignty as well, is that, probably, Istio is a good spot to put some sovereignty capabilities that you can say, "Okay, well, I'm auditing that there's the authority to perform this transaction, to make this request." That that's a great point to insert checks on, "Oh, am I violating a sovereign boundary when I do this?" So, I don't know of any specific technology points that are working on this with that clarity today, but there are clear places where we can put sovereign checks in existing control infrastructure, and that's gonna be really important to application development in the future.

So, we can see this challenge of compliance globally. There's no such thing as one standard. So, maybe the question is more about, how do we shift away from the compliance checkbox and more to compliance as part of the platform feature set, that mindset shift? How do we manage that?

Yeah, absolutely, I mean, this is critical in every country. But the check boxes are in terms of regulations, but that's not the real benefit to a business. And so, when we're talking about the benefit to the business, then we're not talking about, "Well, I'm not gonna get fined." I mean, no one wants to get fined. That's a huge risk, but that's not the usual reason you do something. Instead, it's about, again, realizing the benefits of operating a sovereign environment. Having less dependency on an individual provider, depending on technologies that you can control the future of, having the flexibility to deploy them in new ways, rather than waiting for the next release from a particular provider or for them to roll out a service, or for them to deprecate a service and start charging you a whole lot more for a model that you've built your business around that's no longer the premier model that they wanna present. So, there's a lot of benefits there in terms of reducing the risk, but also increasing the agility and building out new applications, conventional applications, as well as AI-driven applications, by controlling your own infrastructure, by controlling the pace of the technology development that you want to have.

So, the mindset shift, to me, would include some of the automation that you've touched on. We've worked on things like policy as code in the context of something, a tool like Ansible. So, thinking in terms of extending automation through to those compliance checklists that just become part of how you run your business, which gives you the freedom and flexibility to then move quickly as the technology shifts. You're adopting new models, you're building new applications. Increasingly, those applications are AI-enabled. So, we talked about cloud, we talked about sovereign AI. This is the Year of the Agent, so we have to talk about agents and where agents fit into this picture. Funny enough, agency and autonomy get used in the context of agents, but has slightly different meaning, there. What do you see in terms of the spectrum of sovereignty for these different types of applications, the traditional or cloud-native applications, the next generation AI-enabled or agentic applications? Does everything need to run in your sovereign infrastructure? Do you have some hybrid? How are customers approaching that today?

Yeah, well, there's cost considerations on where you are running different types of applications. You may not have on-premises capabilities to run all of your applications. You may not be able to build that out quickly. You may not be able to get the accelerators that you need to be able to deliver those solutions. And so, it comes down to, how critical is this to your business and how quickly does it need to move, and what is my risk associated with it? So, probably outsourcing your HR software. Unless you build HR software, that's probably being provided as a service today. And as long as that service provider is meeting your requirements, that's great. The application that runs your core business, if you're a bank, your core banking software, you probably want that pretty tightly controlled. Regardless of your technology space, whatever makes your business differentiated, that's where you're going to focus, because that's both your highest risk as well as your biggest area for reward, if you're able to move more quickly.

Yeah, and I'm thinking in the context of agents, often, you're essentially encoding in agents internal standard operating procedures that are very much about the core of your business. So, that kind of concept holds over into the agent space. And when we think about AI in this context, you're a country or a company building sovereign AI infrastructure or not, is this something that you think of as a fundamental competitive differentiator? Is this something that's a must-have? How are companies and countries thinking about this?

Well, I think there's almost a meta level with applying sovereignty to agents. That there's a maturity level, here, which is like the first thing you need to be able to do is make sure that you, for the software that your humans are building, are meeting your sovereignty requirements. And once you do that, you can then also make sure that these agentic platforms that you're deploying, which are going and doing things that you haven't specifically necessarily designed them to do, it's even more important that you know what's happening, there. And so, building that maturity with your sovereign capabilities as you're deploying any sort of agentic workflow is critical, because you need to be able to audit what those systems are doing. Like, where is this agent sending my data? What data is it sending? Does it have permission to send that data? It's sending email to researchers of its own accord with my internal financial paperwork, looking for optimizations? Probably a bad idea. So, that's the biggest concern with deploying agentic workflows, is that you need to make sure that they meet all the same security, as well as sovereignty requirements, as your human-designed workflows. And that means having to be familiar with the technology, deploying them in inherently safe ways, in sandboxed ways. So, using platforms that allow you to actually control domains of what data can be accessed, even if that isn't in the application itself. Being able to use, again, technologies like Istio that can provide boxing around what's allowed to an application, auditing around what's allowed to an application. Because right now, we can't go to the spec and say, "Okay, well what did the product manager say that this agent should be doing?" Instead, it's doing a lot of new technology, making a lot of new technology or business process choices on its own. So, is this a competitive differentiator? Absolutely. Again, from being able to move quickly, but also being able to move quickly and auditably, audit-ably? So that you know what's going on, and making sure that you're not dependent on resources that you may not even know you were depending upon. And that can be resources, again, outside your region from a sovereignty perspective. It just could be businesses that you didn't know you were depending on from a sovereignty perspective. Because we've been talking about this in terms of geopolitics a lot, but it's not just about geopolitics. Again, getting back to where we started, it's about your agency and controlling what you do with your data and your systems. And so, all that is about knowledge, auditability, control points, really boring stuff we've been talking about for years. But going back to digital transformation, that was about technologies we already had. This is about technologies that we already have. We just have to mature and make them fit into the new requirements that are coming from the geopolitics, as well as from newer workflows, like agentic AI.

Very well articulated. I always enjoy talking with you, Jered. You've got such a great insight and a grasp of some of these nuances that it's important to tease them out. So, I really appreciate the time you've taken to help us understand what sovereignty means, debunk it from just a buzzword to some details, and show how it's even a competitive advantage and brings value to a business or a country. So, thank you so much.

Yeah, thanks, Chris, always great to talk.

Thanks to Jered for helping us understand that sovereignty isn't about isolation, it's about choice. Whether it's keeping your data under your control or making sure your AI models aren't locked into a single GPU vendor, the goal is resilience. As we've discussed, open source isn't just a way to build software. It's the only way to ensure your digital future remains in your own hands. Thanks for joining the conversation. I'm Chris Wright, and I can't wait to see what we explore next on "Technically Speaking".

About the show

What’s next for enterprise IT? No one has all the answers—But CTO Chris Wright knows the tech experts and industry leaders who are working on them.

Defining sovereign AI with open source ft. Jered Floyd

Transcript

Transcript

More about artificial intelligence

About the show