Scaling with Orchestrators

  |  Compiler Team   Cloud services Containers

Compiler • • Scaling with Orchestrators | Compiler

Scaling with Orchestrators | Compiler

About the episode

All the containers and pods you're spinning up need wrangling. An orchestrator is going to help you manage them: starting up new ones when the demand calls for it, and shutting them down when they're no longer needed. But those are only the most basic functions an orchestrator provides.

Paolo Santos, Assistant Vice President of Architecture and Quality Engineering at Globe Telecom, shares how they set up their orchestration platform—and how it's helped them take full advantage of cloud-native infrastructure.

Compiler team Red Hat original show

Subscribe

Subscribe here:

Listen on Apple Podcasts Listen on Spotify Subscribe via RSS Feed

Transcript

I'm kind of thinking of, you know, how all these companies have started doing like the April Fools type things. Oh yeah. So it's like a little gimmick for like one day and then they have to like change it back. And that kind of ephemeral type event I feel like is, is really indicative of this ability. Oh, that's a really fun example. Yeah. And the new pods have the new version and then the old, you know, and then we go back again, right? I don't have to repeat that whole sentence. You, you can't do that. Exactly. You can just roll it back when you're done because it's not as much an update as it is a temporary change. Yes, exactly. This is Compiler, an original podcast from Red Hat. I'm Emily Bock. And I'm Jennifer Scalf. On this show, we go beyond the buzzwords and jargon and simplify tech topics. This season, we're covering the fundamentals of IT infrastructure. On today's episode, we scale the heights of container orchestration. In our last episode, we covered the basics of containerization. One of the lessons of containers is that while breaking up applications into smaller microservices gives a lot of fine-grain control over resource allocation, it also means that there are a lot of moving parts to manage. Enter orchestration engines like Kubernetes. Jennifer, when did you first fall in love with orchestration platforms? Hmm. I'd say September 9th, 2014 as a guess. No, I know. I'm just kidding. That was the day that Kubernetes was released to the public. I don't actually remember that day specifically, but in the spring, the next spring, it's spring 20—yeah, 2015. I vividly remember sitting at a table with Dan Walsh when I was back in the financial services organization or part of our company, talking to a very large bank about security concerns that they had. , those have all been taken care of by now, thankfully. Perfect timing 10 years ago. Oh yeah. 1.0 is never perfect. Right? But it was really wild to be at that table. I was at the table when these conversations were being had and I'll never forget that. It was awesome. , and also the possibilities, right? So, you know, I, I've said a few times I'm on the day two side of things, so we kinda, we kinda get some of the negative, but to see the possibilities back then for what orchestration was gonna turn into, what Kubernetes was gonna do and did, it was amazing. So that's, that's when I fell. Yeah, for sure. It was a seismic shift. Like my previous company, I think it must have been around like 2016, 2017. I think that they started migrating to Kubernetes. Okay. And it was, it was a brave new world, for sure. Like I can't imagine being like in the room where it happens, where they're making those kinds of decisions. And I should tell, I just remember too, I went away on maternity leave and I came back and it was all what everybody talked about. So the beginning of 2016—oh, like what you said, the beginning of 2016 is all every, anybody's talking about. So I was like, "Oh, where did I go? Hey, whoa." Was I up on a mountain for—no, it was only, you know, three months. Oh, very fast moving. Yeah. No, I remember it was a huge deal when we did it and we thought we were so cutting edge, you know? Exactly. All up on the new trends. That was 10 years ago, Emily, that was 10 years ago. That's nuts. Definitely did change the ground out from under you while you're away. That's always fun. Uh huh. We also talked with Paolo Santos, Assistant Vice President of Architecture and Quality Engineering at Globe Telecom. At the time, Globe Telecom was in the midst of migrating to containers and Kubernetes and he shared the biggest reason why they were making such a big move. With Kubernetes or with OpenShift, it's all about autoscaling and being able to automatically downscale when there's no requests coming in. That has become one of the, I guess, biggest benefits that we had. We got a little bit into autoscaling in the last episode. I think we were working with a campground kind of metaphor, but I think in this episode we can go into some more detail. So what is autoscaling and how does Kubernetes make it happen? I agree. We need to go a little bit more in the weeds. So autoscaling itself—the concept is what we mentioned before, spinning up or spinning down pods and I need to take a moment to say the word pods a few more times because I don't think we've said it enough or maybe at all on the container episode and that's okay. You can use pods and containers interchangeably in certain situations. Most of the pods that we use in our, in the world I'm in now, in the vertical, for telecommunications, which I will call "telco" from now on, are one pod to one container, but you could have a pod with multiple containers depending on what your application needs. But usually what Paolo will be talking about and what I'm talking about is one container to one pod and he will probably in the future refer to pods so we should keep saying the word pods a few more times. Pod, pod, pod. Autoscaling means spinning them up and down. Depending on a lot of different triggers. We use the word triggers a lot. So it's not turned on by default when you install say OpenShift, be aware of that. Most people create what's called a horizontal pod autoscaler. Mm-hmm. So again, I'm saying pod a couple more times. You create that horizontal pod autoscaler for your specific workloads and those workloads roll up to what we just regularly refer to as a deployment of an application. So now we're getting into what we would think of as a traditional application. They're made up of a number of workloads. The pods have containers that enable those workloads. It just keeps going, rolling down that way, right? Yeah. My group is in the weeds of this all the time and so we don't really think of what is being autoscaled up and down in terms of the end application and I think Paolo will refer to that a little bit more. So that's good. We'll have that too. But I do want you guys all to know on the back end when you're going and looking this up that you want to look for horizontal pod autoscaler, you want to look at how those workloads specifically need to be configured and it might not be what you traditionally think of as a final application. Does that help? And I have no idea how to put that into our campsite analogy. No, I think, okay, I think I get it. So a pod will contain at least one container, but it can have more. Yes. And then autoscaling is determining, kind of, dynamically how many pods you need out at once. And then autoscaling is determining, kind of, dynamically how many pods you need out at once. Yes. So you can make there be more, you can make there be less depending on what you need. Yes. And I'm holding back a little bit. Yeah, to really simplify. Yeah. By default, it usually is how much CPU it needs, how much memory based off of the patterns, you know, of your application, of your container and there for the pod. There are also all kinds of custom triggers when you do this autoscaling. That, honestly, is where I've been really interested recently. I'm gonna try to hold back a little bit from that because most of the time folks are just using how much CPU, how much memory do these things, you know, need, and that's how they're deciding if they're gonna scale up and down. So that's as far into the weeds, probably, we should get right now, but that's what he's referring to. Yeah, yeah. And so horizontal scaling is how many pods Yes. Vertical scaling, if I can extrapolate, vertical scaling would be like how big the pods are. That would be like the CPU and resources and stuff. It's a little more complicated than that. Yeah. You're really not gonna run into vertical so much. Stick with the horizontal. Yeah. But the point is the scaling we're talking about here is like, number of pods. Yes. Not necessarily like how big they are directly. Most folks, day in and day out, are talking about how many. Exactly. Okay. So that all sounds great, especially in comparison to, you know, the manual management needed to scale up or scale down operations you have to do by hand without an orchestration engine that can help you with that scale. So Paolo recalled how much time Globe Telecom saved over the course of the elections in the Philippines last year. Moving into a Kubernetes architecture, that has given, I guess, a lot more comfort into the team, especially the operations team, because it has come to a point, for example, that we had to create a task force when there are big holidays. It's a Christmas holiday, you have to put it in the task force just to monitor the systems that you have, right? Because you wish and you pray that nothing goes down during such a big holiday, right? An election just came, we just had our elections a week ago, you know, and while we do have our legacy systems that we still had to monitor, there was more focus on those legacy systems, but for those that we have already migrated into the containerized architecture, everybody's starting to go, we can leave that behind because that's stable. Let's break that down a bit. A telecommunications company would need to scale up operations in the lead up to an election as campaigns kick into that high gear. So why would they trust an orchestration platform to be stable in the face of that kind of sustained high demand? It's the job. No, that's what it's for. That's exactly what it's for. No, joking aside, I'm trying again to go all the way up to what we think of as traditional programs, applications, and then back down into the weeds because they mentioned a few different examples of workloads. Mm-hmm. Right? He talked about the holidays, he talked about elections, and in the past—now I'm gonna go to my past, just for a bit. When we had bare metal, and even in virtualized environments, you really did have to monitor, it was a lot of monitoring, a lot more human monitoring until we did end up scripting quite a bit of it, to decide, "do we think we're ready? Do we have enough RAM? How about our database?" We haven't gone into databases and storage yet. "Do we have enough? Do we have enough?" It's sort of a best guess. A long time ago was a best guess, right? "Do we think we have enough?" Mm-hmm. And wow, when something got hot, when that one toy somebody wanted to buy at Christmastime so that one page kept loading, ooh. Yeah. I'm making jokes, but I'm trying to make it a little bit more tangible for folks, right? Like the Tickle Me Elmo situation. Yeah? Oh, wow. Right? It was at, it was chaos. I don't know what the equivalent is these days. I think folks have those surprise boxes or something. I always see these surprise boxes. I don't know. Like it's happening now with like Pokemon cards, I think, but for less— Again? Less fun reasons. Oh, oh no. It's more, so it's more like reselling them, but, which is kind of sad, but we'll find a happier— Yeah. Well, could you imagine if it was the 90s and now we've got, it's 30 years—oh, 30 years—how is this 30 years later? Suddenly people care about Pokemon cards again. Oh yeah. And so they're, they're hitting your website so hard. You didn't plan for this. So you're, you have what you have from, you know, the last 20 some years and you don't have enough storage, you don't have enough CPU, you don't have enough RAM and you're kind of stuck, right? If you're just bare metal, unless you can call somebody up and order, real quick, some more hardware. Yeah. But in this world, now that we have things like autoscaling, you break down each piece of that. Do they need, and I'm gonna go back to Paolo's example for a second—for voting, right? What do you need? You need communication between the locations, you need the website for folks to go to to know where their elections are being held and for the holidays, you would need a lot of supply chain related management—pods, containers, workloads. Well, and there's ebb and flow there too, you know. Yeah. Black Friday is gonna be stupid busy, but— Right. Thanksgiving, probably not. And if, if those, you know, whatever your toy of choice suddenly isn't cool again in a month, you've bought all this hardware. Mm-hmm. What are you gonna do with it? You don't need it for that. So— Yeah. Yeah, these are a lot of, I think it's a lot of tangible examples of why you would need to go scale up, scale down. Mm-hmm. I'm not gonna go too much more into the technical details, folks can rewind, right? Okay. We covered most of it, I think. Yeah. Like with so many things that we cover on this show, it's a little easier said than done. If it's so great, why isn't everyone running their infrastructure on containers and orchestration platforms? Paolo explained the challenges they faced with infrastructure overhauls. The adoption from, I guess, virtualization going into Kubernetes or containers, was not as fast, as I would say so, compared to the other countries. So when you do things on a virtualized architecture, there's a lot of operational toil that comes with it, when it comes to doing application upgrades or doing, you know, hypervisor upgrades, for example. And what happens is, and it still happens to this day, you know, because there are still legacy platforms or legacy applications that we have that are still running on virtual machines and in those kinds of applications that we have yet to modernize, you know, it takes operational toil because that means you have to get some cut over schedules from your change advisory board and do it at 9:00 PM till let's say, until 6:00 AM in the morning, you know, just because of the fact that we are a telecommunications company, but then it's still a balance that you'd have to do with the business, because data and telecommunications is not something that stops at 9:00 PM. You know? Especially with the Philippines being, having a large industry, for example, on BPOs or business process outsourcing, there's still a lot of data or network requirements that they would need during those off hours or which is going to be your US business hours. So that becomes a bit of, I guess, a challenge when it comes to virtualized infrastructure versus your containerized platform that is autoscaling, that you can do in-place upgrades, that you can do things better through infra-as-code, which means you'd be able to do things that other digital disruptors are able to do, like your Facebooks and your grabs and, you know, your Alipays that they can do changes on-the-fly, even during business hours. Ah, the dreaded update window, always a nightmare to figure out when, kind of, the best time for that can be. And I imagine, especially for a telecommunications company. So does switching to an orchestration platform really eliminate the need to schedule downtime for maintenance and upgrades? For the most part, yes. Especially for applications, there's this neat process called rolling updates where you can spin up new pods, which contain containers. While the old containers, pods, are still up and then taper down the old ones as the new ones come online. That makes sense. Yeah. And then those new ones, yeah, those new ones take over the work. So let me back up. Let's go in the weeds for just another second then. Literally, while these old pods are being replaced with the new ones, those containers, those pods, those pods contain containers with a new version. Mm-hmm. I wanna go a little bit more complicated, but I also don't. But basically you tear down the old pods and you spin up the new pods and the new pods have the new version of your containers. Yeah. And I keep kinda sounding like I've got that question mark at the end, because, well, I still think folks— There's always more nuance. Yeah. There is more nuance, but why I'm going a little like, yeah, you should probably go look up more of this is because those containers are not a singular application. And again, I feel a little bad for implying that on an earlier episode, but it's okay, guys. Come with us. Come with us on this journey of explaining extremely complicated orchestration platforms, because they are. I don't wanna lie to you all. These are complicated, but once they're set up, they work like magic. So it magically, you set these things in motion, they almost, they seemingly magically tear down the old pods and set up your new pods with your new versions and you have no downtime. Yeah. And for somebody that's worked in this industry for this long, the idea of no downtime is just mind boggling. Yeah. So we'll bring us back to earth for just a second. I have to. I can't. I have to pour a little bit of cold water on. Rain on the parade. A little bit, okay? It doesn't work like that for all the cluster management pods. There is a tiny bit of downtime, but it's so small and most of those cluster management pods are monitoring what the other pods are doing. So if they, the cluster management pods, go out for a minute and we haven't even really said the word operators, Emily, we'll say operators now. If they go down for, you know, a minute to be updated and then come back up again, most likely your users will not notice. Yeah. Because they're doing a lot of monitoring for changes and for stuff happening. So we'll, you know what, I'm gonna, I'm gonna not rain—I'm gonna put the umbrella up, it's not raining anymore, guys. It's okay. Yeah. But there is a little bit of downtime. I just don't wanna be completely, you know, oh, it's all rainbows and— Yeah. So it's, like, not zero downtime, but zero, like, impactful downtime. Yes. Theoretically, at least. I'm picturing kinda like, you know, when they repave highways, they'll shut down like one lane at a time to repave - Yes. So like the rest can still keep going. And then like they open kinda one at a time, except probably even cleaner than that and hopefully faster. Meanwhile, when we hung out last week and I had to go home, they shut down the whole thing for a minute. So I have a bad feeling in my—yeah, there's a little bit. It doesn't always work perfectly, but it was quick. It was only a couple minutes. Yeah. But I, I kinda see it like you were talking about like the, the cluster management containers and being like - Yeah. That would be the moment where they have to switch over to the next lane and there's two lanes closed at once instead of just one. Yeah. But still they didn't shut down the whole highway for what, six plus months to repave it all in one go. Right. Exactly. We did not have to do that. Yeah. Otherwise we wouldn't hang out that often, Emily. Come on. Exactly, exactly. And, you know, something about a rolling pun, I don't know, there's, there's something there. It's definitely another one in there. Exactly, exactly. So I think not only that we talked a little bit about, you know, getting through that kind of an update with the zero downtime theoretically, at least. But also if something goes wrong, undoing that is not as much of a huge undertaking. Well, with the old architecture that you have, that means if your change window is six hours and it fails, sometimes, at worst, it also takes six hours to roll it back, you know, versus, you know, I guess that's the thing that you'd have to demonize and explain to the business that with, even if you're not going to go deep into the technology with the business, helping them to laymanize it to them that, "hey, if I do this upgrade, even if it fails, I can roll it back and just run a rollback pipeline and that's it." Rollbacks. Now we're talking rolling updates, rollbacks. We'll say roll as much as we say pod. So do they work in kind of the same way as those rolling updates? For the applications, yes. Mm-hmm. And when I say applications, I mean the pods and then the containers, but I'll keep going back up and down that and saying "um" in between each of them. So for your, for your pods, for your containers inside your pods and here, here we go, here we go. You ready? When you modify your deployment and it rolls out the new version. Mm-hmm. So it rolls out... Okay, wait, wait, wait. So old pods with the new version get torn down and the new pods have the old version you rolled back up. Gotcha. So you're, you're kind of updating in reverse. Yes. And I wanna say that one more time, so I'm gonna pause for a second. No, I'm glad you, like, paused beforehand because I really needed it to figure that out. So I, I'm literally just repeating what you're saying here, because it took me a minute. Right. So now you're in a disaster. "Oh no, something went wrong, must go back. Okay, now we've got broken pods. We tear those down, we spin up new pods that have the old version instead and now we're back on, you know, 1.0." Exactly. Yeah. Okay. I followed. But I do have to say, I have to, again, take the rain cloud out for just a moment—that's for your applications, your containers. Mm-hmm. You know, your pods that contain those containers. For the cluster itself, which is where most of my current work is, it's not actually with the application, you can't as easily roll back, like the cluster itself, the version of OpenShift, okay? Mm-hmm. So I just need everybody to know that, but it's okay if you reach out to our support folks, that's why we're there. There are ways we can help you with that if you're talking about the cluster itself— Yeah. Not the applications. I just want to be really, really, really clear, because I don't know if we're talking to folks right now, the folks that are listening that are going to be managing applications, user applications, or if they're going to be managing the cluster, itself. So just want to differentiate that. It sounds like Paolo's talking about the applications though, which is good. Yeah, I think that's probably the most common use case, but I do like that we got to look out for the persona that is, you know, orchestrating the whole thing. After my heart. Exactly, exactly. Not everything can be quite so easy, but that does sound like it allows for a lot more flexibility. And Paolo explains how that flexibility allows them to quickly react to fleeting opportunities. With trends comes being able to adapt to the needs of the customers, whether that be creating, you know, a two week promotion, for example, for data just because a new movie comes up, you know, there are promo, data promos that you'd want to do for that and for you to be able to do that in the old ways, you couldn't just wait for, let me do that at 9:00 PM because by that time that trend might not, might be gone by that point in time. But with containerization, you know, being able to introduce new SKUs, for example, on-the-fly, doing things through DevOps, you know, you're able to do things, you know, even during business hours, you know, and, and that's one thing that we would like to journey and want to keep on doing in Globe. Yeah. So like deploying something in a single day, maybe even for a single day is a really fast deployment time, especially compared to, you know, pre-Kubernetes world. Oh, absolutely. And we had said earlier about the toy example. Mm-hmm. A hot toy comes out, you've gotta be able to be ready for it and then say a month or two later, after the holidays, nobody cares about that toy anymore. You don't need all the infrastructure you had to. Yeah. Exactly. To hold that toy up. And so, but also he's talking about some, another, some jargon. We say that lovingly, but we should probably mention, now, which is your CI/CD pipeline, CI/CD pipeline, which stands for continuous integration and continuous delivery pipeline. Mm-hmm. And how that's managed and then I'm gonna throw in my favorite again that we haven't gone into too much, but maybe later—the operators. So we have a GitOps operator that actually monitors, mentioned briefly earlier, it monitors your Git repos, see if there's a change and pushes that out. And it's lightning, but this really also depends on a lot of the policies. Sounds like he has policies set up to enable their DevOps folks to push things out very quickly and that's great. We have the technology, they have the policies. Yeah. It's kind of like the antithesis to the traditional waterfall kind of a method where you don't have to build the whole thing and then push the whole thing out at once. You can do little tiny, teeny bits and it goes out all the time and with those kinds of rolling updates, no one ever really notices anything went down at all. Mm-hmm. Like I'm, kind of thinking of, you know, how all these companies have started doing like the April Fools type things. Oh, yeah. So it's like a little gimmick for like one day and then they have to like change it back and that kind of ephemeral type event, I feel like, is really indicative of this ability. Oh, that's a really fun example. Yeah. And the new pods have the new version and then the old, you know, and then we go back again, right? I don't have to repeat that whole sentence. You, you can do that. Exactly. You can just roll it back when you're done because it's not as much an update as it is a temporary change. Yes, exactly. So we should say CI/CD one more time because real hot. CI/CD. So, there you go. Okay. It's so hot right now, for sure. It's all up, up with agile, all those kinds of things. Oh, yeah. The whole idea is get stuff out fast and frequently so you can deliver value faster and more often. Exactly. And you don't have to wait for like two or three years after you've built something to see if anyone still wants it. Which is kind of terrifying in this day and age. It really is. We talked about scaling and deployment times. So next up, Paolo is going to describe how Globe Telecom made it all happen. Naturally, all this change took time and they didn't roll everything out at once. It was a phased approach, you know? I guess this is where the whole balance between proving a new technology to the business rather than doing it in a big bang, sometimes doing it. So what we did in reality was do it in a canary approach, which means from one application workload, for example, migrating it to the OpenShift platform, what we did is swing over 10% of the traffic first, you know, get it to quote unquote simmer from, I think a maximum of two weeks and then swinging it to, you know, and then you start building that confidence with the business, the next time that you're going to ask, why don't you increase it to five a day, right? And then, and then trying to build that was, I guess that's one of the biggest successes that we have, in terms of doing that. So what's the advantage of a slower rollout instead of doing it all as a big bang, quote unquote? Well, instead, in case something actually goes bang, right? Exactly. That's kind of what I was thinking. I kind of froze for a second because the times that I helped roll out a new webmail a million years ago and we just switched that right over. Now we could, it was a very small group of users and, but there were things we did not see coming. Oh, man, I would have loved to have had the canary kind of rollout and I'm fairly sure for those wondering it's because of the canary in the coal mine. So you move a few of your users over, in whatever mechanism—there's a million different ways, back in the day it was like a load balancer you'd used to move that. I know we're talking about—well, it's still a similar concept if folks know. Yeah. A little bit. Yeah. And so you move a little bit of your users over, see how they do, move a little bit more, see how they do, see how the application does, see how the infrastructure does and then a little bit more, a little bit more and it, it makes things so much saner and you can switch back really quick as we've, you know, gone into great detail about earlier. Yeah. Yeah. The canary approach. Kind of like having like a built-in beta group, almost. Yes. So you're, you're not risking all your eggs in one basket. You can risk a few at a time and then roll back easily if things don't go so well. There are four more metaphors that I just thought of. We'll just move on. And this kind of rollout changes your infrastructure, but it also changes how your teams operate too. Like in our examples from earlier, those quick turnaround times mean team processes are going to need some shifting too. You know, trying to introduce a new technology to a company, it comes with processes and trying to mix the old process with the new technology, there will always be challenges to that. There's also a lot of proving that you'd have to do, "Oh, you don't do it that way anymore, you'd have to do it this way." There's a simpler way of doing that. While there's education that happens and operations teams appreciate that, but it's something new to them, as well. "Oh, so you don't have to do with this anymore," you know, that kind of thing. And, and, and I guess that's the biggest challenge that we had, but it's also—we saw that as a half glass full or an opportunity, as well because that keeps everybody learning, as well. That kept that whole five to 10 year process that the operations team, for example, has been using to get updated now because we're introducing new technologies. One of the things that I keep telling people is that, you know, you cannot buy and introduce an iPhone to your company and think that the process that you have with Nokia is going to fit on using an iPhone. Makes so much sense that, you know, the telecommunications company uses phones for their metaphor, but I think it really fits and it makes me think of, you know, the previous company I was at where we migrated to Kubernetes, before we migrated, we literally had a team that would have to stay after hours on Tuesdays to deploy the latest version of everything we were working on. Post-Kubernetes, we did not have to do that anymore. And I imagine that's only one example among many. Oh, wow. That is a great example. And I was thinking—and my brain was just going deeper into the telecommunications. There was a time when, and this is going way back, but you couldn't move certain phones from one network to another. The hardware wouldn't allow you to do that. And I'm debating how deep I wanted to go into that. It doesn't happen so much anymore, but, and I'm, maybe that's what he was referring back to is when, back in the day, he's probably more talking about the operating systems on the Nokia phones versus the iPhone these days. We would have to manually monitor when some new application or web mail or whatever it is was coming out. When we had bare metal or even sometimes in a virtualized environment, you can't, sometimes, guess how much you would need of memory, RAM, hard drive space, et cetera. And there was a bit of a scramble. And now this was quite a while ago, some people haven't scrambled. We have load balancers, we can move stuff around. But it's a different world now, being able to very quickly spin up those pods and their operators. He was saying operators as in people, operations teams, but the operators in the cluster management also. At least we're not talking about the phone operators, also. We are not, oh gosh, now we're going back a hundred some years. I love it, Emily, let's go. Back to the past. Oh, Back to the Future. They were pulling those cables out, popping those cables back in. In my head, now that's what I'm gonna see when I'm thinking of the pods. It's just gonna be cable operators, yeah. It's not, not that kind of, I guess. Kind of. I like it. All right. We gotta have the fancy glasses if you're doing that too though. Of course. Oh my gosh, stop. All right. All right. And so now, I think that we've got all of that working, the infrastructure and now this team structure and processes, it's time to wow the powers that be who might not understand what's now possible. You have to go through a change advisory board, justify the change that you're doing and then they're going to ask how much of a downtime window that you're going to do and trying to change that and say, "Nope, I'm not going to ask for a downtime window." That's kind, kind of new to them. And then proposing that I don't want to do it in the evening. I want my engineers to be able to do it in the morning in the business hours. So that, in itself, is a shift to them, like, "what are you talking about?" But you've gotta be creative in terms of doing analogies to them, you know, even to the point of saying things like, you know, "can you look at your iPhone right now and go into your app store?" Then you notice that Facebook and your Spotifys and your other digital disruptors, for example, they're able to push updates to you, you know, every two weeks and sometimes you just swipe down and everything changes into your UI. Yes, we too can live the dream like the hyperscalers push out those updates like hotcakes. But, philosophical question, is there such a thing as too many updates? Oh, I would definitely say yes. Everybody calm down just a little bit. No. But joking aside, like this is the dream that's hard to articulate. Now I'm, I'm in, I'm in this all day, every day. We don't have to work at 2:00 AM on a Sunday anymore. And now there, I've felt over the last maybe four years a sense of, but I went through that. I had such a hard time when I was a new system admin. "But I suffered." "I suffered. Yeah. Why don't you?" And it is, there is a little, I don't know, like a little guilt to it, almost. I don't know how to describe it from some of us who have been doing it for a very long time. But the newer folks, this is just the expectation. If you have operators, devops, sysadmins who are, you know, in the next generation, we'll just say they are expecting this, also. So thinking about on the flip side, they're not gonna be working at 2:00 AM their time on a Sunday. Yeah. Because we don't have to live that life anymore. So if you don't have that mentality in your organization already— It's hard to imagine a time where you would've had to. Yeah. And it's hard to imagine getting rid of that completely. There's still this twinge in the back of people's minds that, "oh, but we have to have people ready for, we have to be prepared for a 3:00 AM update." No, you don't anymore. You really don't. I promise you guys, you really don't. You can set up your infrastructure and your orchestration such that you can have rolling updates too many times, sure. Mm-hmm. Yes, as much as you want. I promise. Like I see it every single day at massive telecommunications and very small, you know, mom and pop shops. You don't have to work 24 hours a day anymore. You really don't. Come with us. Yeah. It's like you don't have to wait for your free call minutes after 9:00 PM anymore either. Like, you know, That was a thing. Speaking of telecommunications, like it's so weird in a world so removed from having to do that to think about what life was like when it was there. Exactly. That is a great example. We, we tend to focus on the negative, but there are so many positives— Mm-hmm. And today. You can make a phone call. You don't have to wait until 9:00 PM to make a cheaper phone call. You don't have to set up an upgrade maintenance window on Friday night at 8:00 when you think like less people will be doing something. You don't have to do that anymore. You really don't, I promise. And if you, if you don't have that yet in your organization, reach out to your infrastructure folks and let's figure out what, what it takes. 100%. I think we covered a lot here. So as is tradition now, I think we could go back to a little bit of recap. So we talked about autoscaling and the definition of pods. So a pod can have a container or more than one container and you can autoscale, controlling how many pods you have, which is useful for things like trends in the industry, where you need a brief period with lots of pods and then you want them to go away, after the fact. What else did we talk about? We talked about— Upgrades. Upgrades and rollbacks, as well, both of which are helped immensely by both containers and orchestrators that help you manage them. Oh, and the GitOps. We talked about how you do have to have policies in place for all those updates, because you don't want to hit too many. Exactly. And a little bit about how your infrastructure can also be mirrored in your actual, like, team structure and the processes that they have to follow, as well. And in moving to new and exciting things, such as containers—that can change the processes that you have to live by. And make everybody's lives a little easier. Yeah. So to sum up, orchestration, it turbocharges containerized infrastructure and helps you keep track of everything. Just make sure you're properly prepared for the rollout. Exactly. And to all of our listeners, please remember to hit us up on social media at Red Hat and use the hashtag #compilerpodcast. And that does it for this episode of Compiler. This episode was written by Johan Philippine. Thank you to our guest, Paolo Santos. Compiler is produced by the team at Red Hat with technical support from Molly Brock. If you like today's episode, follow and review our show on your platform of choice. Until next time.

More about containers

Allowing applications to run consistently, and at scale as needs change. Containers make this possible. 

Blog post

Introducing using OpenShift Serverless for event-driven applications

2020 年 11 月 17 日
Blog post

Red Hat introduces Advanced Cluster Management for Kubernetes

2020 年 4 月 28 日

About the show

Compiler

Do you want to stay on top of tech, but find you’re short on time? Compiler presents perspectives, topics, and insights from the industry—free from jargon and judgment. We want to discover where technology is headed beyond the headlines, and create a place for new IT professionals to learn, grow, and thrive. If you are enjoying the show, let us know, and use #CompilerPodcast to share our episodes.