Datacenters And Sustainability

We closely examine different aspects of technology— data storage, AI, software development, and  hardware capabilities. But one part of that equation often goes ignored: The energy grid. It’s an unfortunate truth, but the datacenters we depend on to address everyday challenges can themselves be a drain on our global resources. How do technologists reconcile the work they do with the rising environmental impacts? And how do we work together to find a solution?

00:02 — Kim Huang
I went on vacation recently. The place I stayed in was remote, and the internet was spotty, non-existent in most areas. I took full advantage of it because these days it feels like we're always on and we're always connected. But that connectivity comes with a price. And I'm not just talking about utility bills. Telecommunications, digital media of all kinds, emergency services, on and on; they depend on things like IT infrastructure, cloud computing, data storage and retrieval, and all those things depend on datacenters. Those datacenters require a lot of energy to keep the world's information up and running and accessible when we need it. But here's the conflict: As technologists, how uncomfortable is it to be working in an industry that could be a source of great global energy consumption? How do we solve that problem? How do we address how much energy waste goes in to keeping datacenters running?

01:12 — Brent Simoneaux
This is Compiler, an original podcast from Red Hat.

01:16 — Angela Andrews
We are your hosts.

01:16 — Brent Simoneaux
I'm Brent Simoneaux.

01:19 — Angela Andrews
And I'm Angela Andrews.

01:20 — Brent Simoneaux
We go beyond the buzzwords and jargon and simplify tech topics.

01:25 — Angela Andrews
Today, we are exploring one person's work to address energy consumption in the enterprise.

01:33 — Brent Simoneaux
Producer Kim Huang is ready to get us started.

01:37 — Kim Huang
So I wanted to speak with someone who came from a background addressing topics like this, like climate change, environmental conservation. So I spoke with Cara Delia.

01:50 — Cara Delia
My name is Cara Delia. I'm a principal community architect at Red Hat, and I look after the verticals of financial services and the topic of climate sustainability.

02:03 — Kim Huang
So Cara comes to Red Hat with an interesting background. She studied environmental science, but when her program got cut during the school, she went into environmental policy. She started working in Washington DC, but after a while she realized she wanted to do something different.

02:18 — Cara Delia
I knew I had a wide depth of experience and skills and wanted to use it in a different way. And the opportunity within the Red Hat OSPO (the open source program office, that sits within the office of the CTO), there was an opportunity that came for this role. And I was like, this is really just kismet.

02:39 — Kim Huang
The role she's talking about, her current one involves upstream open source communities and consortiums.

02:46 — Cara Delia
And so I was working with one community, OS Climate, and that was focused around sustainable finance. And it was through this that I also started seeing across our organization internally and then also the market just, I mean, I knew how important sustainability was personally, but I could see organizations and people within those organizations that I was talking to of this is something really important and we can use our resources individually for a greater good. And so that's what... it came full circle in that I started in wanting to change the world and focused on climate.

03:28 — Kim Huang
Sustainable finance. What does that even mean? Cara explains.

03:34 — Cara Delia
With sustainability, you think of ESG, there's climate, there's so many different descriptors for it, but sustainable finance is specifically making sure that the assets that are a part of corporations' portfolios are that they're lending money for the greater good, for social justice, for greener options.

04:02 — Brent Simoneaux
So Kim, when Cara says ESG, what does she mean?

04:07 — Kim Huang
ESG stands for Environmental, Social and Governance. And it first came to prominence in reporting from the United Nations. It's a framework that organizations can use in their own corporate strategy to deliver a value for all scenario as it pertains to areas like pollution or environmental conservation. But it's not just for the environment. It also addresses working conditions for employees, privacy practices for customers. So companies using the ESG framework depend on real time data. Projects like OS Climate create a way for that data to be centralized on a common platform so that data scientists can build a model to show a clear picture of, for example, a company's energy consumption, their carbon footprint and other things.

04:58 — Angela Andrews
This is a topic that I am not familiar with at all: sustainable finance? It sounds like a really weird made up term when I think finance, I think money, but I would love for my finances to be sustainable.

05:14 — Kim Huang
I think we all do.

05:17 — Angela Andrews
So you have my attention, but when you broke it down to what it really means, being good citizens and lending money for social justice and green projects, that is huge. When we're talking about mass corporations, I always thought that that's where the change should happen, at the corporate level. Because when sustainability happens, it should really trickle down to the rest of us and not the other way around. Let the corporations make the sacrifices and show how it should be done. We can see things happening and then we can trickle down and say, "This is how sustainability should work." So I know this is a new term for me, sustainable finance, but I really like what it stands for.

06:01 — Brent Simoneaux
Well, I think this framework of ESG, it's something that I hear a lot about in corporate strategy. This is something that a lot of companies are talking about. What I didn't realize and what I'm learning from Cara is that a lot of the strategy relies on data, right?

06:21 — Kim Huang
Yes.

06:21 — Brent Simoneaux
Which is a technical problem. And who better to address this problem than someone like Cara, right?

06:30 — Kim Huang
Right. So you're probably wondering though, this is a lot of talk about sustainable finance assets, risk models, what does that have to do with datacenters? Along with all the work she's doing with OS Climate, Cara is also a technical lead within CNCF, the Cloud Native Computing Foundation. She represents Red Hat on the CNCF'S technical advisory group. It's kind of a cross section of tech companies. Some of them are small and some of them are the biggest names in the industry.

07:01 — Cara Delia
We are working together to have combined efforts around carbon tracking, creating best practices for carbon tracking especially around datacenters and anything around cloud native technology. We actually do have a project that has been sent to the CNCF, Project Kepler, that is intended to help with some cloud native technology and datacenters as well.

07:31 — Kim Huang
It sounds like there are a lot of parties involved in trying to figure out this problem and trying to figure out data center's impact on sustainability. But what exactly is Project Kepler and what does it do? We'll tackle that next. (07:53): So Project Kepler. Kepler stands for Kubernetes-based Efficient Power Level Exporter. What is it?

08:02 — Brent Simoneaux
That's a mouthful.

08:03 — Kim Huang
It is. It is.

08:06 — Brent Simoneaux
Well, what is this, Kim?

08:06 — Kim Huang
Yeah. I don't know.

08:06 — Brent Simoneaux
Okay.

08:10 — Kim Huang
I really don't. I think I know. Let's see if I have a handle on it. But let's get Cara in real quick to give us some background.

08:19 — Brent Simoneaux
Okay. Okay.

08:22 — Cara Delia
Kepler comes into this story from an IT optimization standpoint. So it's optimizing energy intense processes, and really making supply chains more transparent. It was founded by Red Hat and within our emerging technologies group, which is actually where I sit, and it was created in collaboration with IBM Research. It was intended to capture power usage metrics from Kubernetes clusters to see where their efficiencies to be more effective. And it uses eBPF, which is Extended Berkeley Packet Filter in the Linux kernel to be able to use machine learning models to estimate power consumption by the workload in a way for it to be able to be exported as a set of Prometheus metrics for tracking carbon footprint.

09:17 — Brent Simoneaux
All right. Angela, can you help us decipher some of this?

09:22 — Angela Andrews
Brent.

09:25 — Brent Simoneaux
Is that a yes?

09:26 — Angela Andrews
Okay. Let me try my hand at this.

09:28 — Brent Simoneaux
Okay.

09:30 — Angela Andrews
I may be way off, but what this project is about is tapping into energy consumption in Kubernetes. So there has to be this tool and it's seeing well, how much energy, how much power is being consumed by all these different processes inside the cluster, different containers, different services, and it's trying to figure out the metrics. Where is all this consumption coming from in the cluster and with this tool, eBPF? And yes, I've heard of it and I've never used it, but especially in this context, if this is a tool you use to figure out how much power is being consumed with certain workloads, and then you're able to use say, Prometheus, which is great to visualize your metrics, then you get an understanding of how some of your hungriest workloads may behave. And then you can figure out, "Well, how do I offset some of these very consumptive workloads?" So it sounds as if we're trying to figure out a way to look at technology a little bit smarter and how it's consuming energy and maybe having a better way to track it? Question mark?

10:38 — Brent Simoneaux
And I think optimize, right?

10:40 — Angela Andrews
Yeah, exactly.

10:41 — Brent Simoneaux
So she said optimizing for energy intense processes.

10:46 — Kim Huang
Yes. And the optimization part will come in a little bit later, but that was pretty spot on Angela. These metrics are then... these Prometheus metrics are then used for scheduling, scaling, reporting, and visualization, which provides sysadmins with the information on the carbon footprint of their workloads. So performance metrics, tracking performance metrics, platforms are doing so. That's nothing new. But you have to keep in mind, a lot of these sources of data are disparate. They can often be walled behind proprietary software. They're not housed in the same place, but Kepler can change all that.

11:28 — Cara Delia
That transparency where around the metrics that some other metrics providers, you definitely, because they're not open source, you're not able to see all of the sources or all of the inputs into those metrics. And so Kepler continually adjusts and fine tunes through pre-trained models using node data from power estimating agents that are running on servers. And so those metrics can be combined with power usage to calculate the carbon footprint of that workload. So if you're looking at the workload, you can see where the opportunities are, whether or not you power down a certain workload or you power up one, they can based on what are the outputs of the carbon intensity.

12:21 — Kim Huang
You called it Brent,

12:22 — Angela Andrews
Spot on. Spot on.

12:23 — Kim Huang
Optimization.

12:24 — Brent Simoneaux
Optimization, right?

12:25 — Kim Huang
Yes.

12:26 — Brent Simoneaux
Yeah.

12:26 — Kim Huang
So let's take a second and think about when you think of a data center, what is it? We're talking about obviously a lot of racks, a lot of servers, a lot of racks. We're talking about AC running at 40 degrees Fahrenheit 24 hours a day, seven days a week, 365 days a year. That's kind of how I imagine it.

12:46 — Brent Simoneaux
Yeah.

12:46 — Kim Huang
Am I off?

12:47 — Angela Andrews
You are spot on. Everything is always running all the time inside of a data center. Every server, every piece of equipment, everything at the top of the rack requires energy and it's running constantly. And it's doing so because you don't want your circuit board to overheat. You don't want your equipment to fry. So that's a lot of air conditioning. And if you think about it, when you're home, you cannot imagine running your air conditioner all year round. Correct?

13:14 — Kim Huang
Mm-hmm.

13:16 — Brent Simoneaux
What's so wild about this is that the way we talk about datacenters and especially the cloud, it's such an abstract concept. It's so...

13:27 — Angela Andrews
But is it though?

13:29 — Brent Simoneaux
No, it's not. I think-

13:31 — Angela Andrews
Okay. Just think about this for a minute. The datacenters that we used to house in our buildings, in our offices, they've just moved into bigger buildings. So it's like everyone's data center just up and moved somewhere, and all of that consumption is now happening in one place.

13:46 — Brent Simoneaux
Yeah. What's really interesting about all this is the way that we typically talk about datacenters, we talk about the cloud, it's very ephemeral. It feels very abstract, but they're very physical objects that need a very particular environment. And from what I hear you saying, Kim, they also use a lot of energy.

14:13 — Kim Huang
Yes. Some kind of numbers we have, which obviously are dated at this point if you're thinking about how many datacenters there are in the world and how this technology is proliferating, it's increasing 10 to 30% over a year because you're getting more and more and more datacenters, more and more workloads. It's just a very kind of exponential type of growth. According to the International Energy Agency, datacenters account for about 1.5% of all global energy consumption. And if you go to just the United States where I reside, it's 3% of electric power.

14:54 — Brent Simoneaux
Let's pause here for a second, because those are some kind of wild numbers.

15:01 — Kim Huang
They don't seem like a lot though, right? 3% is like, "All right." You know what I mean?

15:06 — Brent Simoneaux
But if you think about the total electric power in the United States, 3% of, I don't even know what that is, but I'm guessing 3% of that is a lot of energy.

15:20 — Kim Huang
I would probably say it's enough to power a couple of cities, maybe mid-size cities. That may not seem like a lot though. 3% compared to other things that we use every day if you're talking about consumer electronics, it doesn't seem like a lot. But with the needs increasing over time, like I said, 10 to 30% year on year, ignoring it isn't exactly a good plan. Cara says that Kepler can integrate with Kubernetes to not only display workloads and their energy consumption, but again, it can optimize their performance.

15:58 — Cara Delia
The Kepler metrics can be employed by a Kubernetes scheduler to place the upcoming workload on the compute node, and it can improve performance per watt. And again, it's through auto scaling algorithms. And so one of the nice parts too is that it can be integrated with continuous integration and deliveries so CI/CD pipelines to optimize the efficiency of energy consumption, and they can actually be placed into some sort of dashboard that really presents what the power consumption is at different levels and include containers, pods, name spaces, or different compute nodes in that cluster.

16:47 — Angela Andrews
That is pretty interesting because I'm trying to figure out when the scheduler decides, "Oh, where am I going to put this workload, right," is it looking at the most optimal hardware in which to place these workloads? I mean, are we talking about servers that have a more greener, for lack of a better word, workload that can hold those workloads? I'm really trying to understand what does the scheduler, it says, "Okay, I have this workload and I need to scale it, and I need to make sure that it is definitely on its most efficient systems," and I'm wondering if it has a lot to do with how the hardware behaves. And that's a good question. So I know that it has a lot to do with how the energy... they're tracking how energy is consumed, but the hardware has to play a part. That's all I'm saying.

17:43 — Kim Huang
Yes absolutely.

17:44 — Angela Andrews
Like where does the hardware come in and how do we decide the most efficient hardware to place said workloads on?

17:53 — Kim Huang
My response to that would be that would depend on the infrastructure and the support teams at the company that chooses to adopt Kepler, right? They would be kind of the, I guess, owners of the hardware question that you're posing, Angela. But it's very interesting, and honestly, I didn't really think about that before, but you do have a point because not all hardware is created equal. There may be a question of which server is more optimal and which one runs, I guess greener than the others. (18:27): So a company, let's say a healthcare company is trying to get a sense of their energy usage. Maybe they think the spikes in their energy consumption are related to when doctors are accessing health data or when they are, for example, a very busy hospital is admitting a high volume of patients on certain days of the week. The company or the organization that runs the hospital could use a project like Kepler to build out their own comprehensive dashboard to observe these patterns of energy consumption and verify that information instead of having it just be, I don't know, a guess. And they could even use the machine learning features within Kepler to estimate future usage and then modify whatever internal infrastructure that they need to modify.

19:11 — Brent Simoneaux
Huh.

19:11 — Angela Andrews
Interesting.

19:12 — Kim Huang
Does that sound right?

19:12 — Angela Andrews
Oh, for sure. It sounds like we're moving in the right direction, but it's still all boils down to the hardware, right?

19:20 — Kim Huang
Yeah.

19:21 — Angela Andrews
That's always going to be the final arbiter as to how efficient these workloads are running. If to be able to support a higher volume of patients, you're going to be scaling up usually, and that means more pods, that means more containers, that means more people accessing that endpoint, and that is an energy consumption increase. How is your hardware going to handle this? And being able to see those peaks and valleys, maybe you can plan better. Maybe you can do... when you do your hardware procurement, you can kind of move in that direction and see how those workloads kind of ebb and flow. And then when you bring in more energy efficient hardware, you can see those same peaks and valleys, but maybe they're not peaking like they used to because you're taking into consideration how Kepler helps you see the efficiency and then Prometheus where you see those metrics and you're looking at that dashboard and you can react and interact with those usage spikes.

20:27 — Kim Huang
Right.

20:27 — Angela Andrews
Does that sound... I feel you on that though. That was a great story or use case to make it make sense.

20:36 — Kim Huang
I guess a good spinoff from that would be the difference between an on-prem situation or an off-prem situation. If you're dealing with a data center that A, for example, I'll use my hospital example again. They don't necessarily control that data center, and they may not control the hardware that's being used. Maybe there's something on a macro level where parties that control the data center can be incentivized to use greener servers or to use servers that can be optimized to run more efficiently.

21:11 — Angela Andrews
It's definitely moving in the right direction. If we are trying to be more energy conscious, and we're trying to be more green as a community, as a bunch of technologists, the technology community, and we're building projects like Kepler and we're graphing them, and we have sustainability finance where we're trying to make sure that we are doing the right thing with our resources, talking about the greater good, then we should be behooved to make sure that, what can we do to be better? Do we have to buy different hardware? Do we maybe move from one cloud to a more energy efficient cloud if that's a thing? I don't know. That may become a thing if you think about it, because if you have taken up the mantle as an organization that cares about sustainability, you may be looking for a place to run your workloads that believes the same thing, and they have things in place to make that work.

22:10 — Kim Huang
Wow.

22:11 — Brent Simoneaux
Angela, you just said, what can we do? And as technologists, what I'm hearing from Cara is that there's quite a lot that we can do.

22:23 — Angela Andrews
Indeed. It's always in our hands. It starts from up top, but then when we see that wow, changes are being made, well, we can start making those changes too. Maybe it can mean how efficiently our code is running. How are we looking at what our code is doing? Is it the most efficient way to run a particular process? Are there more efficient ways that take up less resources?

22:48 — Kim Huang
Yes.

22:48 — Angela Andrews
Do we tweak our programs to make them more energy efficient? And I don't even know if that's a thing. I'm just going to assume it is, because everything's possible nowadays, but I think there's something we can do, and we're talking about being more sustainable.

23:04 — Kim Huang
Yes. And then I would think in a open source community kind of collaborative spirit, I feel like there's a space for technologists to kind of reach beyond their teams and even beyond their organizations to advocate for these cross sections of solutions for hardware and software. I think that there's a lot of room and maybe even a lot of appetite for that kind of collaboration, that kind of collaborative effort to, I mean, offset, or at the very least, reverse what's happening with energy consumption for datacenters.

23:36 — Brent Simoneaux
And this is sustainability and climate change and things like that. It's one of those big problems that no single person or no single company can solve on their own. It takes a lot of companies and a lot of people coming together to solve a common challenge.

23:55 — Kim Huang
Indeed.

23:56 — Brent Simoneaux
If we're all working separately in our own little silos or whatever, our little companies or rooms or whatever, we're all less effective than we are working together.

24:08 — Kim Huang
I think that's exactly what Cara is also getting at. I asked her as someone who found themselves working in this space in a very unconventional way, how she feels about the work that she's doing now. And she says, kind of what you're saying, Brent, that it's just more evidence that participating in an open source community and collaborating with different companies and different organizations can push everyone towards a common goal.

24:37 — Cara Delia
Whether they consider themselves a technical or non-technical person, anyone can take part in open source and truly feel like open source collaboration is what's going to help us to achieve climate related goals so much faster by using common set of core practices and technologies that are accessible by all.

25:02 — Brent Simoneaux
I want to come back to something you said at the top of the episode, Kim. You were describing this, I guess, internal conflict that you had about something that you value, which is sustainability and maybe how your profession or the industry that you're in is somehow contributing to that.

25:31 — Kim Huang
Yeah, that's interesting. And yeah, there is a level, and I'm sure I'm not the first person to invent this feeling, but I really care about, well, the planet because I live on it. We all kind of do and we all have to live here together. But I've never honestly thought about sustainability in this light. Technologists have worked together to address so many different challenges on a global scale, but it makes sense that the lens would also be directed inward towards the proliferation of datacenters and towards the impact they have on the power grid. I hope that projects like Project Kepler can influence how we think about cloud native technology as it evolves and it changes. And I'm really glad to see open source communities being a part of that and they're really excited and galvanized to make change.

26:24 — Brent Simoneaux
Yeah.

26:24 — Angela Andrews
Agreed. This episode was such a clinic in understanding a lot of new terminology and technology and to really put it in perspective that we play a part in protecting our planet. And it doesn't seem like, well, where does our part come in? But collaboration and working in open source and working together to solve these types of problems. That's how it all... that's how it works. That's how the good stuff comes. And we've seen it time and time again. Open source communities are where good things happen when folks come together. (27:03): So I am very encouraged by projects like Kepler and others that are probably on the horizon that are addressing climate change as well. And I can't wait to see what's next because we have to live here. So let's keep it around for a little while longer, shall we? Let's do what we can. (27:26): So what do you think about what you just heard? Sustainable finance, ESG framework, emerging technology, climate change, and it's all happening within the datacenters and the power consumption. There was so much information in this episode. We want to hear what you thought. I know you have thoughts on it because I do. Use the hashtag #compilerpodcast. We would love to hear what you thought about this episode. Can't wait to hear what you got to say. (27:52): And that does it for this episode of Compiler.

27:59 — Brent Simoneaux
Today's episode was produced by Kim Huang and Caroline Creaghead.

28:04 — Angela Andrews
A big thank you to our guest, Cara Delia.

28:07 — Brent Simoneaux
Victoria Lawton empowers and sustains us every single day.

28:13 — Angela Andrews
Our audio engineer is Robyn Edgar. Special thanks to Shawn Cole. Our theme song was composed by Mary Ancheta.

28:21 — Brent Simoneaux
Our audio team includes Leigh Day, Stephanie Wonderlick, Mike Esser, Nick Burns, Aaron Williamson, Karen King, Jared Oats, Rachel Ertel, Devin Pope, Matias Foundez, Mike Compton, Ocean Matthews, and Alex Traboulsi.

28:36 — Angela Andrews
If you like today's episode, please follow the show, rate the show, leave a review, share it with someone you know. It really helps the show, and we like to hear about it.

28:45 — Brent Simoneaux
All right, everyone. We'll see you next time.

28:48 — Angela Andrews
See ya.

28:48 — Brent Simoneaux
All right.