What Can Video Games Teach Us About Edge Computing?

From button-mashing brawlers to massive battle royales, there’s a lot of fun to be had in online gaming. That is, if players’ connections are reliable. If you’ve ever dealt with input delay, or wrestled with lag, you know how important a quality connection is in online gaming. But with edge computing on the horizon, what impact might there be on how we play?

The internet, as we’ve explored in past seasons of Command Line Heroes, is a patchwork of international agreements and varying infrastructure. But there’s something coming to change the ways we connect. In this episode of Compiler, we explore what edge computing could mean for people who enjoy video games, and what this form of entertainment could teach us about the technology.

00:06 - Angela Andrews
Intro. Let's do it.

00:14 - Brent Simoneaux
All right, Kim, what do you have for us today?

00:17 - Kim Huang
I wanted to talk today about edge computing. Angela, have you heard anything about edge computing?

00:22 - Angela Andrews 
Oh, yes, actually I have. I didn't really know what it was until about last year when I came to Red Hat and now it's a topic of conversation, so I hear a lot about edge computing.

00:37 - Brent Simoneaux
This is one of those topics that I hear a lot of people talking about, but if I'm being honest with myself, I don't know that I could explain it.

00:45 - Kim Huang
Cool. Well, I'm hoping to explain a little bit more about it then. And today, I want to use kind of an unconventional lens to take a look at edge computing.

00:56 - Angela Andrews
And what's that?

00:57 - Kim Huang
Video games.

00:58 - Brent Simoneaux
All right, let's do it.

01:00 - Angela Andrews
Fantastic.

01:09 - Angela Andrews
This is Compiler.

01:11 - Brent Simoneaux
An original podcast from Red Hat.

01:14 - Angela Andrews
We're your hosts.

01:15 - Brent Simoneaux
I'm Brent Simoneaux.

01:16 - Angela Andrews
And I'm Angela Andrews.

01:18 - Brent Simoneaux
We're here to break down questions from the tech industry—big, small, and, sometimes, strange.

01:25 - Angela Andrews
Today's question: What can video games teach us about edge computing?

01:36 - Brent Simoneaux
Producer Kim Huang is here to help us out.

01:40 - Kim Huang
Okay. Before I get into video games, and I promise this is going to pay off, I need to know something.

01:47 - Angela Andrews 
Sure. What do you need?

01:48 - Kim Huang
How many pizza places do you have near you?

01:51 - Angela Andrews
Too many. There's one on every corner, almost.

01:54 - Kim Huang
How about you, Brent?

01:55 - Brent Simoneaux
Yeah. I live in a city downtown and there are one, two, three... There are four pizza shops within a few blocks distance from me.

02:06 - Kim Huang
Okay. So the reason I'm asking is because I spoke with Griffin Ashe.

02:12 - Griffin Ashe
My name is Griffin Ash. I'm a lead architect for Telco and edge computing here at Red Hat.

02:18 - Kim Huang
And he had a really interesting way of explaining exactly how edge computing works.

02:25 - Griffin Ashe
Conceptually, or at a very high level, where we like to dissect or start as an entry point for edge computing is it's a movement of compute resources, closer to the devices and people that produce and consume data.

02:42 - Kim Huang
He usually gives an example of there being only two pizza spots in the entire country.

02:48 - Griffin Ashe
If there were one or two large pizza shops on the East Coast and the West Coast only, the experience of me ordering, getting delivered, and then consuming a pizza probably wouldn't be that good.

03:05 - Brent Simoneaux
Let's play this out, then. I'm hungry. I want to order a pizza. I call the East Coast pizza shop. What happens?

03:14 - Kim Huang
If you lived close to the pizza shop, it would probably be a very normal experience. But for mostly everyone else, I guess they would take your order. You would tell them what toppings you wanted. You would tell them if you wanted any sides or any drinks. They would package it all up, give it to a delivery person, and then that delivery person would take it to a plane or a train or a boat, I guess, depending on where you lived.

03:42 - Angela Andrews
Yum. Boat pizza.

03:44 - Kim Huang
And it would take hours, maybe even days, to get to you. And by the time that you got your pizza, I imagine it wouldn't be a very fresh, very hot meal.

03:57 - Griffin Ashe
Edge computing does for data what the pizza shop down the street from my house does from my ability to consume pizza.

04:06 - Brent Simoneaux
So in the pizza metaphor, I end up with cold pizza, even though I'm hungry. What does that look like in the real world though, outside of this metaphor?

04:17 - Angela Andrews
So in the real world, edge computing moves data processing closer to the people that need it. And it's using devices that are already a part of the infrastructure. So users can get a much better experience because you don't have to send data back to a central site for processing, which can impact the performance of the application.

04:37 - Brent Simoneaux 
Got it.

04:41 - Kim Huang
So we have the core concept of what edge computing is. Let's transition a bit and talk about video games.

04:52 - Jared Sprague
All right. My name is Jared Sprague.

04:54 - Kim Huang
So I spoke with Jared Sprague.

04:56 - Jared Sprague
I'm a principal software engineer at Red Hat.

04:59 - Kim Huang
And he really knows his craft, and he's also a member of the gaming COP.

05:04 - Angela Andrews
And what is the gaming COP?

05:06 - Kim Huang 
It's the gaming Community of Practice. Think of it as not a formal team, but a group of people who are really enthusiastic about open source technology in gaming.

05:17 - Jared Sprague
Over the past few years, I've been lucky enough to have the opportunity to work on several game-related development projects at Red Hat, mostly around events.

05:29 - Kim Huang
Jared and I talked a little bit about how edge can help the experience in video games. And he talked about LAN parties for a little bit.

05:40 - Angela Andrews
He went to LAN parties?

05:42 - Kim Huang

Yes.

05:43 - Angela Andrews
I've heard of those.

05:44 - Brent Simoneaux
What is a LAN party?

05:46 - Kim Huang
Brent, a LAN party is when a bunch of people, usually friends, say about six or seven of them, will get together and run a game on a local server. So they'll all get together in the same space, under the same roof, and play a game together, run locally. Does that make sense?

06:04 - Brent Simoneaux
Got it. So a group of people in the same space, and all of their computers are connected together.

06:12 - Kim Huang
Exactly.

06:14 - Jared Sprague
And that is the pinnacle of edge computing and gaming, in my opinion, because the two big killers in multiplayer gaming are latency and server load. And when you're in a LAN party, you've solved those two things, because you have almost zero latency. And then you have six to ten people on a single server, so there's practically zero server load.

06:40 - Brent Simoneaux
Okay. Let's define those really quickly. What is latency?

06:43 - Kim Huang
Angela, do you want to take this one?

06:45 - Angela Andrews
Sure. So latency happens when you have to wait for data to come your way. So you're waiting and it can take milliseconds to actual seconds for something to come. That is what latency is.

06:59 - Brent Simoneaux
And what about server load?

07:01 - Angela Andrews 
Server load is when the capacity of the server has been met and exceeded, and it doesn't respond as it should because of said server load. It actually is bogged down. So that's what server load is. And LAN parties solve both of those problems.

07:23 - Brent Simoneaux
So these are really big problems in gaming. What does that look like in the real world? How does latency and server load issues show up when you're playing a game?

07:31 - Kim Huang 
Well, it shows up in a lot of ways. It could show up in the game client trying to constantly correct. So when you're playing a game, there's two sources of information, there's information coming from the game itself. So think of the level that you're on, if you're playing a platformer or the kind of characters that are in the game. And then there's information coming from the player. Inputs they're putting into whatever they're using to play a game, whether it's an actual controller or it's a keyboard. Those are two sources of information. And the game client is constantly trying to anticipate and correct based on those two data points. 

So sometimes when you'll play a game and you load it and you try to walk, and sometimes if you walk, it'll look like your character's not moving correctly, or your characters standing in one spot and then boom, it's standing in another spot, that's because the game client can't figure out where your character is on the server.

08:33 - Brent Simoneaux
Oh. So I'm curious now, how edge computing could solve this.

08:39 - Kim Huang
It basically makes it so that you have a virtual kind of LAN party. It makes it so that the load, that would usually be on one server, would be moved closer to the player themselves. So that instead of having inputs having to travel so far, they don't have to travel as far.

09:03 - Angela Andrews
So Kim, what else do you have for us?

09:06 - Kim Huang
Well, in the case of edge computing, it's something that can potentially solve the two biggest problems that video game developers are constantly trying to address. So you have people who are playing against other people who are all over the world or across the continent, and their experiences can vary depending on whatever their internet connectivity is. How, then, can we make those experiences better? I spoke with Derek Reese.

09:37 - Derek Reese
Hello, I'm Derek Reese.

09:39 - Kim Huang
He's a principal engineer and another member of the gaming COP. Basically, he consults with other people, other Red Hatters, on industry processes and game systems architecture. He knows how games are made, from start to finish, and he knows how to put together the code that runs the game itself, like tools and servers.

10:03 - Brent Simoneaux
All right. So what did Derek have to tell you, Kim?

10:06 - Kim Huang
Okay. So he used a really popular game as an example.

10:11 - Derek Reese
There's a game that I'm sure you may have heard of, that's Grand Theft Auto.

10:16 - Brent Simoneaux 
I have heard of it.

10:17 - Kim Huang
Okay, great. It's a really big game, AAA, very popular title. They have an online component of Grand Theft Auto and it's very, very, very popular. It's a huge revenue source for the company.

10:31 - Derek Reese 
And a lot of the direction where the engineers spend their time is about what has the most impact on players. And so in that case, it's like, "Keep the cash shop open. Keep micro-transactions working. Make sure the servers are running. Make sure that everybody can log in and they have their player experience the way they want it to. Make sure you're on top of cheaters and hackers and that sort of thing."

10:57 - Brent Simoneaux
And what does this have to do with edge computing?

10:57 - Kim Huang
I'll explain that in just a second. If developers are so busy with upkeep and maintenance on gaming experiences, they don't have too much time to do anything else. It turns out, there was this bug...

11:12 - Derek Reese
...where on certain systems, and especially on PCs with lower end, but multithreaded processors, the game would take five to six minutes to load. And this is a game that, in other modes, when you're not doing multiplayer, only takes about a minute, maximum, on these same systems. And so a lot of players are very frustrated by this. What was happening is during loading of the multiplayer arena, all of the items that you can purchase in the game, it would be stored in a JSON file. And the code to load the JSON file essentially was so inefficient and got hung up on that file, that it would take five minutes on a single core to work through it.

11:55 - Kim Huang

It was something that was a relatively easy fix, but because the engineers were so busy with everything else, they just couldn't get around to it. Finally, it was actually one of the players. And they got so fed up, they took out their debugging tools, figured out what was happening, and fixed the game. So the thing about edge computing, what makes this related, is edge computing moves some of the processing closer to the player. In theory, that could free up development teams to focus on bugs and issues, like this one. Not having to spend so much time maintaining the online experience, they could use that time to focus on other things.

12:32 - Brent Simoneaux
Oh, okay. So that makes a lot of sense. So what else did Derek tell you about, Kim?

12:38 - Kim Huang
Well, a lot. So I want to go through, really quickly, Brent, and kind of break down exactly what happens when you press a button on your controller.

12:47 - Brent Simoneaux
Okay.

12:49 - Kim Huang

Okay. So the first thing you do, press a button to move your character forward. It goes through the game, locally, it gets processed, and then it gets put into something called a packet.

13:03 - Derek Reese
And that packet goes out through the home users' internet connection that gets routed to the ISP. The ISP's going to work through its... If there's existing peering agreements, then yay, gets things to where they need to go a lot faster. But in most cases, you're going to get kind of bounced around, based on what the ISP thinks is the most efficient path to the destination server, which is usually hosted at the game company or in a warehouse, or increasingly these days on something like AWS.

13:28 - Kim Huang
So whatever your request is, in this case, it would be moving your character forward, that has to travel all the way back from the server to your processor.

13:39 - Brent Simoneaux
So this is kind of the equivalent of, say, the East Coast pizza shop, right?

13:45 - Kim Huang
Yes. So imagine the pizza delivery guy trying to put into his GPS, the fastest way to get to your house. And maybe the fastest way is a highway with tolls. Maybe it's a highway with no tolls. Maybe it's an older state road with a lot of curves. Maybe it's a mountain back road. It could be a number of different ways to get there, but whatever way that that person takes, it's going to impact the amount of time that you have to wait for your pizza to get there.

14:15 - Brent Simoneaux
So it has to go a great distance and that can take some time. What is the alternative?

14:21 - Kim Huang
Well...

14:21 - Derek Reese
When we're talking about edge, we're kind of circumventing all that. So if you're on your mobile phone, instead of a mobile phone communicating wirelessly to the closest tower and then from the tower to an ISP and from that ISP to the game company after it gets routed all over the place. Instead, your signal makes it to the tower and then there's a box that's on that tower or at the next building over. So you're no longer getting routed all the way to the ISP, but before you even get there, your destination is available and some computational items can just be calculated and sent right back to you with essentially a micro server. In this case, microservices that are responsible for things that don't need to go back to the main server or can be asynchronously connected to the server over time. So maybe when things are less busy, that data can be synced back upstream.

15:13 - Kim Huang
These edge accounts could work for a real-time strategy game, like StarCraft, where the AI runs on a player's computer and it's a pretty intensive load. So if the load is intensive and the player's computer is processing everything locally, you're going to run to kind of an upward limit as to what that AI can do. But if you offload that to a server in some type of edge infrastructure, all of a sudden you can play StarCraft against a very sophisticated AI.

15:47 - Brent Simoneaux
Oh, wow. So it ends up providing this really interesting and robust gameplay as well for end users, right?

15:56 - Kim Huang
It definitely broadens the... well, pardon the pun, the playing field, when it comes to gaming experiences.

16:04 - Brent Simoneaux
Wow. That's so cool. 

I want to return us to our original question: What can video games teach us about edge computing? And I'm wondering what you think, Kim?

16:24 - Kim Huang
I think that video games are a really good potential use case because they already have these kind of challenges built into what they do. So they already have issues with processing information with trying to balance internet connectivity between two different points, or sometimes multiple points, on a map or in the world. So they're already a very good, rich use case for edge to really make a big impact on the way that they do business and the way that they make games. So there's a lot of implications there. That's kind of why I wanted to talk about edge computing in that lens.

17:06 - Brent Simoneaux
This one of those terms that I hear a lot of people talking about. What other industries could this potentially revolutionize?

17:16 - Angela Andrews
Well, I see a lot of innovation happening with edge computing in healthcare. I see it happening in education. I see it happening in transportation. I mean, I don't think there is an industry out there that couldn't benefit from being able to put data closer to the people who need to use it. I see edge becoming more and more prolific in a lot of these industries just for that reason. There's tons of use cases across all of these verticals, where edge could really be able to move data out of their data centers closer to the people that need it. 

And that does it for this episode of Compiler.

18:10 - Brent Simoneaux
Today's episode was produced by Kim Huang. Victoria Lawton makes sure this show has a pizza place on every single corner, so it's just like a LAN party.

18:23 - Angela Andrews
Our audio engineer is Kristie Chan. Special thanks to Shawn Cole. Our theme song was composed by Mary Ancheta.

18:30 - Brent Simoneaux
Big, big thanks to our guests, Griffin Ashe, Jared Sprague, and Derek Reece. Also a special thank you to Red Hats' gaming COP. They were a big help putting together today's show and you can check out their collection of community built games at arcade.redhat.com.

18:50 - Angela Andrews
Our audio team includes Leigh Day, Laura Barnes, Claire Alison, Nick Burns, Aaron Williamson, Karen King, Boo Boo Howse, Rachel Ertel, Mike Compton, Ocean Matthews, and Laura Walters.

19:06 - Brent Simoneaux
If you liked today's show, please follow us for future episodes. We would love to see you here next time.

19:14 - Angela Andrews 
Love, love, love. Thank you so much for joining us. Bye, y'all.

19:17 - Brent Simoneaux
Thanks, everybody.