From healthcare to banking, there’s a lot of industries that require the utmost protection for sensitive data. But for those sectors, innovation is also essential. That’s why disconnected environments—or air-gap environments—can be seen across the IT landscape. But how do they work? How do technologists ensure the data is accurate and usable, and how can these environments foster innovation?

00:02 — Kim Huang
We all depend on interconnectivity to get things done. Our phones, our cars, wearable technology, and at our jobs, especially if they're technical jobs, it's amplified. It feels like everything has to be integrated. There's APIs, platforms that need to talk to systems that we never touch, but in some industries, there's a desire or more like a need to have everything locked down, isolated. That's because those industries may deal in sensitive information, data that a hacker would drool over, but these environments still need to do the same things that other environments do. They still need to store and manage that data, ensure its accuracy. They still need to provide space for developers to create and innovate. But how and what is it like to work in that type of environment?

01:00 — Angela Andrews
This is Compiler, an original podcast from Red Hat. I'm Angela Andrews.

01:06 — Kim Huang
And I'm Kim Huang.

01:08 — Angela Andrews
We go beyond the buzzwords and jargon and simplify tech topics.

01:12 — Kim Huang
Today's episode, we're connecting the dots on disconnected networks. Angela, in the spirit of being accurate, I spoke to someone who has expertise in disconnected networks. Yes. I spoke to Josie Pfeiffer, a Red Hat consultant. She's based in Europe. Josie focuses on infrastructure and security. She works with customers who use disconnected OpenShift Clusters.

01:46 — Josie Pfeiffer
A disconnected OpenShift Cluster is basically an environment that isn't directly connected to the internet, so you can also call it air-gapped.

01:56 — Kim Huang
Air-gapped, does that sound familiar, Angela?

01:59 — Angela Andrews
That is the word. That's the buzzword, air-gapped.

02:03 — Kim Huang
Yes. I'll admit, I didn't know a lot about air-gapped networks when I was going into doing this episode. So I did some research and I found a great article that was written actually by another Red Hatter, architect Stephen Wilson. I'll read from the article. An air-gapped network is one that is self-contained, but has no external or internet connectivity. Networks are often air-gapped as an extreme security method for government or corporate projects that require confidentiality or secrecy.

02:31 — Angela Andrews
In a nutshell, that is a perfect definition for what an air-gapped network is. The level of security and privacy that you need for an environment like this must be high. You cannot have some external threat coming into your network, so you have to cordon it off in such a way that everything works within, but nothing from the outside gets in unless it's absolutely purposeful and with very good reason, usually via sneakernet.

03:08 — Kim Huang
Sneakernet?

03:09 — Angela Andrews
Do you remember sneakernet?

03:11 — Kim Huang
No.

03:11 — Angela Andrews
Maybe. Wait. Let's go back. Okay. All right. Let's go all the way back. I know our listeners here, when they hear the term sneakernet, some of them are saying, "I remember sneakernet," and then we have some folks that are like, "What's sneakernet?" So back in the day when you wanted to take a piece of data or file from one machine to another, and maybe they weren't networked, maybe they were, but they weren't on the... Whatever. You would get up and your sneakers would walk across the floor and you'd have your little floppy disk in your hand and you'd go sticking in the other system. That's what sneakernet is.

03:53 — Kim Huang
Wow. Okay. So you're just hand walking it from one computer to another?

03:59 — Angela Andrews
Basically, and I did say floppies.

04:02 — Kim Huang
Yes, you did say floppies. Wow. Okay. All right, I get that. And that tracks with some of the things I was reading about air gap networks. Okay. All right. Sneakernet. Okay, I learned something today. I think I understand what a cluster is. A cluster is a group of nodes that are working together in an environment.

04:22 — Angela Andrews
That is correct.

04:23 — Kim Huang
And now we know disconnected network, or an air-gapped network, is a self-contained network that doesn't connect to the internet or any other outside network.

04:33 — Angela Andrews
That is correct.

04:34 — Kim Huang
So what's the difference between a disconnected network and a disconnected cluster?

04:39 — Angela Andrews
Well, this is me guessing here, but a cluster is disconnected if it lives in a disconnected network. So it can be clustered, meaning it's sharing network and storage, it's sharing resources together-

04:59 — Kim Huang
With other clusters that are in the same network.

05:02 — Angela Andrews
You can have a cluster in one network and a cluster in another network and never the twain shall meet.

05:08 — Kim Huang
No. And it's something that's only done when the utmost security is needed.

05:14 — Josie Pfeiffer
So it's not something you just do for fun. It's mostly just because these organizations are highly regulated. So financial services companies or insurance companies, any organization that handles sensitive data, also like healthcare for example.

05:32 — Kim Huang
But here's the thing, many of the same things that are in a regular more open network or a network that's connected to a cloud server, for example, also need to be in an air-gapped network. You still have the same things that need to be done. Does that make sense?

05:50 — Angela Andrews
The same bells and whistles are happening in disconnected networks that they are in regular networks that are connected to each other and the internet. That is correct. They still have to do the thing.

06:02 — Kim Huang
They still have to do the things. They still have to store data. They have to still do file sharing. That's usually done in TFTP or Trivial File Transfer Protocol. It's handling file transfers between a server and a client in a disconnected network. There's other components like web servers, DNS, et cetera. But what are the difficulties here? Josie says, for the OpenShift Cluster itself, there's the inability to get updates from an external source.

06:31 — Josie Pfeiffer
The cluster can't phone home, so to speak. You for example, can't get upgrades that you can just pull from our official Red Hat registries. The customer would then have to operate their own internal registry that the cluster can connect to. You want to usually have the latest and greatest in terms of security updates to make sure that you're not running a system that has known vulnerabilities in it and you want the latest features so your engineers are happy and there are two big things that you need for that. There's the platform upgrades that is just to run the basic OpenShift and there's the catalog, which is for operators. It's basically like the app store for OpenShift where you can install all kinds of add-ons that run on top of OpenShift and that add more functionality to the cluster.

07:25 — Angela Andrews
Okay. So in a disconnected environment, you still need to have updates and the latest and greatest operators and things of that nature. What an individual would do in a disconnected environment, they wouldn't say, "Well, we're disconnected. We are unable to perform these tasks." No, you're doing this your own version of sneaker net. Instead of to your next door neighbor, you're going to Red Hat and you're downloading this latest version or this image. And what you're going to do is you're going to check the hash. You're going to make sure that the contents that Red Hat has said, "This is what you're downloading," you're going to confirm that when you download it, you're going to make sure that that's the exact same hash that Red Hat has said. 

(08:15): 
If anything has changed, say there was a man in the middle, there was some nefarious behavior and you download said package and you ran the hash and the hash was different, you'd never use it. So there's this level of checking that you would have to do when you downloaded anything from somewhere else to bring it into a disconnected environment. You'd have to make sure that the integrity of said files would be what they said they were. The integrity was still intact before you brought it in to your disconnected network.

08:49 — Josie Pfeiffer
Maybe we can try an analogy. So let's say you did order something from Amazon and your package arrives at your doorstep, and then you might look at all the edges and you see if someone peeled off the sticky tape on it. And that is basically the integrity check. It's just seeing if someone tampered with the package and if it's still okay to deploy into the cluster.

09:15 — Kim Huang
Well, the need for that layer of security is clear to, well: you, me, Josie a lot of people listening. But just in case it wasn't, there's plenty of rules and compliance steps to ensure that things are kept locked up pretty tight.

09:30 — Josie Pfeiffer
For financial services companies, for example, there are regular audits where they need to get the rubber stamp that they're compliant with certain standards.

09:39 — Kim Huang
And that's a tricky part because different countries, different standards, different regulations to follow, and the punishment can be severe for companies that fail to meet the criteria.

09:50 — Josie Pfeiffer
So Switzerland, for example, also has a strong national regulatory body that imposes additional requirements on top. And the consequence of not meeting them can, for one, be financial. So there could be fines if you don't meet them. And it could also mean that if you're a bank, for example, that they revoke your license to operate as a bank.

10:15 — Angela Andrews
High stakes.

10:16 — Kim Huang
Yeah, very. But that's good. Well, if you're an avid online shopper like me, it is. It's good to know that these networks are in place, they're protecting your information. But how does one ensure the data handled within a disconnected network is accurate if it's not able to connect to another service to ensure that accuracy? People aren't static and neither is their information. A network doesn't just sit on the data, it needs to move it around, use it, process it, update it when it's needed. Josie breaks down the concept of mirroring, something that happens often in disconnected environments.

10:57 — Josie Pfeiffer
So mirroring is basically just taking something that exists in this public registry and putting it one-to-one into your internal disconnected registry. Then it will take the release images and operator images from our official Red Hat registries and we'll mirror them to an internal registry. The cluster will expect things to be at the official Red Hat registry location, but through that mapping, it can then find them in the disconnected registry.

11:30 — Kim Huang
All right. So this internal registry, what mirroring does is since it can't get information from outside the network, it's taking something that exists and putting it into an internal disconnected registry, and then the cluster will find and correctly identify data through mapping to that local registry. That's on premises.

11:52 — Angela Andrews
That sounds about right.

11:53 — Kim Huang
Bam. I got it.

11:54 — Angela Andrews
It tracks.

11:54 — Kim Huang
Yes. Okay. Well disconnected clusters, air-gapped networks. They're restrictive for very good reasons, but as things keep changing in tech, should that change how we think about them? Up next, we further examine the need for air-gapped networking and find out how shifts in available tech may affect our feelings around security versus innovation. Sit tight. We'll be back. (12:26): We left off with Josie explaining the functionality of disconnected OpenShift Cluster, but it made me think more broadly. It all seems safe and invisible to bad actors, but at some point, the data needs to be transferred into a public cloud or some other network in order for it to be used effectively. And there's still a layer of security there, something that many of us are familiar with: encryption.

12:53 — Josie Pfeiffer
Whenever you go on Amazon and you put in your card details for example, and it goes through some payment processing service in the background, that service will likely have to pass a PCI audit to be allowed to process your card data. And they have to show that they're handling your data in a secure way and that they have encryption in between in the network to make sure the card numbers aren't just legible for anyone.

13:21 — Kim Huang
But that's still a point of vulnerability. The data is moving from this super secure environment to a service. Won't attackers just congregate in places where the air gap stops and the services begin? It reminds me of Command Line Heroes. Season 8 we did the season on malware and a lot of the attacks that we covered in that season, they all took place at the point that's nearest the user themselves. And that's because with a lot of the experts we talked to for that season, they said that that's the point of vulnerability that a lot of bad actors identify. It's not necessarily these people have this very glamorized image of hacking where people are hacking into these super secure environments. But in all actuality, these people are--very smartly, if you're thinking about it--they're going for the weakest link in the chain.

14:14 — Angela Andrews
You couldn't have said it any better. It's literally the weakest link in the chain and it's usually the end user. The company that has their data moving in this air-gapped environment, they can't control what's happening on a client. So the best way to do these nefarious things is the low-hanging fruit. Go to the end user. They don't have what these huge environments have. They don't have the networking team and the security team and the DevOps team and the OpenShift team. They don't have all that. They have their little computers and who they bought them from gave them 90 days worth of antivirus and spywares. We're just trying to do our best out here. Definitely the weakest link.

15:05 — Kim Huang
Yeah. And there's the issue of convenience and ease of use. The data in these networks and clusters are used for businesses. Businesses that need to be able to move fast for customers that aren't as security focused. I know for me, I just want my shoes. I don't really think about PCI and... I mean I should.

15:26 — Angela Andrews
And you shouldn't have to.

15:27 — Kim Huang
Well, no, I mean as a technologist I feel like I should, but as a consumer, I just want my shoes. I'm not really thinking about the part that security and encryption and disconnected networks play in my online retail therapy.

15:40 — Angela Andrews
And you just said a lot right there, we should be thinking about it because we are putting our information, our financial information into some box on a browser, on some network. If we're home, we probably have a little bit more security. Heaven forbid we're sitting in a coffee shop on some free Wi-Fi without a VPN. Don't get me started there.

16:08 — Kim Huang
Looking to my left, looking to my right, trying to avoid your gaze.

16:12 — Angela Andrews
Exactly. We have a part to play. The URL, is the little lock on it? Is it encrypted? That's a small part. Do we have a VPN on? Are we better off using our phone as a hotspot as opposed to using some free on the back of a refrigerator somewhere? We should be more mindful of the part we play in securing our data.

16:43 — Kim Huang
I want to step even a little bit further back from the end user or the end consumer and talk about businesses and people who are providing services directly to users. If you're selling the shoes to me, if I'm buying the shoes from you, how much knowledge and how much consideration should you be giving to these types of situations?

17:03 — Angela Andrews
That's interesting because now there are a lot of companies who focus on just that. On the transaction, on moving money from one person's account to another, and those businesses are hired by maybe the smaller business to take some of the onus off of them. So are we writing the code for protecting people's data? No, we're going to enlist the help of someone else, but there are things, me as the small business owner, what can I do? Are the systems that are running my whatever, are they up to date? Are they on the latest versions of software? Are the workstations and kiosks in my business, are they up to date? So there's a lot of moving parts and there are a lot of people who need to be a part of the solution as opposed to being a part of the problem and saying, "Well, not my problem." No, we all have a part to play in that.

18:03 — Kim Huang
I like that. Not passing the buck, either to the customer or to the larger enterprise level tech company. I feel like it's a shared responsibility when you're talking about security.

18:13 — Angela Andrews
It is. So we want to do that retail therapy, we're going to make sure that the site that we go to and put in our credit card information is encrypted. We're not going to be like, this is such a deal and not do our due diligence and start typing in those 16 digits.

18:33 — Kim Huang
But Angela, my shoes though.

18:35 — Angela Andrews
That's where things go wrong.

18:36 — Kim Huang
Yeah. Okay. All right. I got to show some restraint. I have to put my own credit card information before these pumps that I really like. Okay, that's fine.

18:45 — Angela Andrews
Yeah, a little bit.

18:49 — Kim Huang
But I started thinking about the future and obviously everyone in tech is always looking ahead. Is there room for further advancement within an air-gapped environment? I imagine it to be, it's a restricted environment by default. What does it mean for technologists that work in these spaces? How can they be more innovative looking forward?

19:13 — Josie Pfeiffer
The client I'm with at the moment, actually is already experimenting with AI, and you can either create your own models based on the data in those disconnected systems and then use them in there. Or there are some really cool open source models that you can just put in there and maybe train them with data on top of that as well.

19:37 — Kim Huang
That's really interesting to me.

19:38 — Angela Andrews
Yes. So again, if you're introducing something that's outside of your disconnected environment into your disconnected environment, there has to be some analysis and checking in place. And as we're hearing about data and AI, we need to know where this data's coming from. So being able to trust the source. How are we vetting, how is that being vetted? And because this is such new technology, there's so much more growth that's going to be happening and we'll hear more and more about this as we go on.

20:16 — Kim Huang
Yeah. Josie thinks that that's a long way down the road. And to your point, Angela, this is all very new. We're not quite there yet.

20:27 — Josie Pfeiffer
I'm still a bit skeptical of how much can be automated directly with AI. If we think of a very classic use case that a user opens a service desk ticket because something isn't working. At the moment, I don't really see AI being able to work with a lack of information still. So if I just think about some service desk tickets, I generally get, it's usually lacking a lot of information and context. If I think I'm putting in a query to AI and I want AI to solve my problem for me efficiently without giving additional information, then I don't think that's very realistic at the moment.

21:11 — Kim Huang
So don't you worry, we're still going to need, at least in these disconnected networks we're talking about today, we're still going to need a lot of really great people to solve problems.

21:22 — Angela Andrews
That's not going away anytime soon. If anything, this is a burgeoning field of study. This is where people can start making their mark and understanding how do we bring this new technology into these types of environments safely into securely with reliable data. And I think we're at the tip of the iceberg. There are going to be pretty smart people out there trying to solve this exact problem, and it's just a matter of time to see how effectively that problem is solved in which use cases this can be used for. So stay tuned. We're not there yet, so everyone, stay put. We don't have to worry about the robots just yet but know that we're always improving and heading toward what she's mentioning.

22:16 — Kim Huang
Before we parted ways, I asked Josie what she likes about working in these super restrictive environments, despite the difficulties and inconveniences. She says, working around restrictions can actually help people to become creative problem solvers.

22:30 — Josie Pfeiffer
I think you have to be a nerd for it to be rewarding. You have to enjoy solving problems in interesting ways and using technology to find new ways to solve problems.

22:43 — Angela Andrews
Josie is just the type of technologist that I absolutely adore.

22:48 — Kim Huang
Really?

22:48 — Angela Andrews
They are creative, they're curious, they like to solve problems. And it's all about working smarter, not harder.

23:00 — Kim Huang
I didn't know a lot about disconnected networks or disconnected OpenShift Clusters before I started digging into the research. And there's the cyber security aspect. There's the aspect of environments for engineers and developers for innovation's sake, and how do we make sure that people are security-focused across not just a network, but also across an entire ecosystem, an entire experience for a customer or for an end customer to have their data be protected. There's so much to dig into in this episode, so the whole nature of it is like a black box. And the sneakernet and the very practical aspects of having these lockdown isolated systems, but then knowing that these systems exist to house data that's very sensitive. But there needs to be consideration for what happens to that data once it leaves that environment. I feel like there's a lot for technologists to still think about and discuss and still try to wrap their heads around, even though I'm sure that systems like these have been around for forever.

24:07 — Angela Andrews
I'm sure there are a lot of our listeners who aren't familiar with air-gapped environments, and this is their first foray into this topic. I'm hoping We have listeners that work in these air-gapped environments, and they have a lot of experiences that maybe even those who work in traditional environments, they can probably share it. So I'd like to hear from both sides. What are your thoughts on this episode? My folks working in air-gapped environments, what are some of the behaviors that people in traditional environments could take from air-gapped environments? Anything that you've listened to and learned and enjoyed, or maybe eyes were open to, we'd love to hear it. So hit us up on our socials @RedHat. Always use the hashtag #CompilerPodcast. Let's talk more about these disconnected environments. I'd like to hear what you have to say.

25:03 — Kim Huang
And if you are a part of the team that protects me and my data while I'm shopping for my shoes, I salute you.

25:16 — Angela Andrews
And that does it for this episode of Compiler.

25:19 — Kim Huang
Today's episode is produced by me, Kim Huang, with Johan Philippine and Caroline Creaghead. Thank you to our guest, Josie Pfeiffer, and a special shout out to Stephen Wilson. His article on disconnected networks helped shape this episode.

25:35 — Angela Andrews
Victoria Lawton loves a good integrity check.

25:39 — Kim Huang
Special thanks to Britt Duggan. Our audio engineer is Kristie Chan. Our theme song was composed by Mary Ancheta.

25:47 — Angela Andrews
Our audio team includes Brent Simoneaux, Leigh Day, Stephanie Wonderlick, Mike Esser, Nick Burns, Aaron Williamson, Karen King, Jared Oates, Rachel Ertel, Devin Pope, Mike Compton, Ocean Matthews, Paige Johnson, Alex Traboulsi, and the mystical Mira Cyril.

26:10 — Kim Huang
If you like today's episode, please don't keep it to yourself. Follow the show, rate the show, and leave us a review on your platform of choice. Share it with someone you know. It really helps us out.

26:21 — Angela Andrews
Thank you so much for listening. We'll see you next time.

26:24 — Kim Huang
Yeah. We'll see you on the sneaker net. Bye everyone.

26:28 — Angela Andrews
See you, everybody.