Are Big Mistakes That Big Of A Deal? Part 2

Mistakes are part of growth. If we’re lucky, we’re in an environment where they’re not punished harshly. That lets us fix the problem, learn to do better, and move on—but also to tell the story once the sting has passed.

Last episode, we heard three stories of people blundering into trouble and coming out the other side a little bit wiser. This episode adds three more stories of mistakes being made—but the culprit isn’t always as clear.

00:02 — Johan Philippine
Brent, Angela, last episode, we heard some stories from people who made mistakes by messing around with tech they didn't understand. I've got some more stories for you.

00:15 — Angela Andrews
Of course, you do. There are more stories to be told.

00:19 — Johan Philippine
There's going to be stories to be told for a very long time to come.

00:23 — Brent Simoneaux
So part two, huh?

00:24 — Johan Philippine
Yep, part two of "Are big mistakes that big of a deal?"

00:29 — Brent Simoneaux
Here we go.

00:32 — Johan Philippine
In today's episode, we're going to hear some more stories of big mistakes, but this time, the people committing them, they mostly knew what they were doing.

00:42 — Angela Andrews
Oh.

00:42 — Brent Simoneaux
Okay.

00:43 — Johan Philippine
We're going to hear three more stories of mistakes that are about crossed wires and miscommunications more than they are about forging ahead in the dark.

00:54 — Angela Andrews
Okay.

00:55 — Johan Philippine
So with that in mind, let's get to it.

00:57 — Angela Andrews
Let's do this.

01:01 — Brent Simoneaux
This is Compiler, an original podcast from Red Hat.

01:05 — Angela Andrews
We're your hosts.

01:07 — Brent Simoneaux
I'm Brent Simoneaux.

01:08 — Angela Andrews
And I'm Angela Andrews.

01:10 — Brent Simoneaux
We're here to break down questions from the tech industry, big, small, and sometimes strange.

01:17 — Angela Andrews
Each episode we go out in search of answers from Red Hatters and people they're connected to.

01:23 — Brent Simoneaux
Today's question, we ask once more: Are big mistakes that big of a deal?

01:32 — Angela Andrews
Producer Johan Philippine is here to help us out.

01:36 — Johan Philippine
All right, so act four. I call this one: It's not my fault. Now, we teased this story at the end of the last episode. The speaker was Chris Kelley. He is a senior software engineer here at Red Hat and at the time of the story he had just made the switch from being a researcher to a developer.

01:58 — Brent Simoneaux
What does that mean, exactly?

01:59 — Johan Philippine
Well, he earned his PhD and he went into doing some research, and at some point he decided to move from doing research to actually building things and building software that people would use.

02:14 — Angela Andrews
That is so cool.

02:16 — Johan Philippine
And the company that he was working with helped him make that transition. And when he made that transition, he was able to move from a little office where he worked with one other person, to a bigger open floor plan office, which is what a lot of developers are a little bit more accustomed to, or at least were in the before times. This transition also included an assigned mentor to help him out if he ran into any issues.

02:40 — Johan Philippine
Now for his first assignment as a researcher turned developer, he was asked to write a filter that would prevent illegal characters from being written into a string for database entries.

02:53 — Chris Kelley
I started off by making a test case for myself, so I wrote a test case where it would strip out all of the illegal file name characters, and the test string was my name and a bunch of garbled illegal characters. And once the test passed, it would strip out all the illegal characters and just leave my name, which I thought was pretty handy. You'd be looking for strings in the long list of logs, you want a string you're familiar with to pick out easily. Your own name is probably the most familiar string you could possibly get.

03:23 — Johan Philippine
Now, Chris was well trained, he knew how to tackle the task and he tested his solution quite thoroughly on a dummy version of the production database.

03:33 — Brent Simoneaux
Okay.

03:34 — Johan Philippine
That way if he messed up the filter, there'd be no consequences to the actual database of customer data. Makes sense so far, right?

03:41 — Brent Simoneaux
Mm-hmm.

03:41 — Angela Andrews
So far so good.

03:43 — Johan Philippine
He was able to finish this task pretty quickly.

03:45 — Chris Kelley
A couple of hours of poking around and I was able to make this filter work. I was pretty happy. It's like, yeah, great. I've got on the scoreboard. So I was really happy went and myself a cup of coffee. Excellent. I go move on to the next task.

03:58 — Johan Philippine
It worked.

03:59 — Brent Simoneaux
I have a feeling ...

04:02 — Angela Andrews
Plot twist.

04:02 — Brent Simoneaux
Yeah. I have a feeling there's a plot twist coming.

04:04 — Angela Andrews
Feeling a little too confident right now,

04:07 — Johan Philippine
Feeling a little too happy, right? Yeah. Well, it worked, but it worked a little bit too well.

04:12 — Chris Kelley
I only realized that something had gone horribly wrong when I got a call from the database admin, about an hour later. He wasn't happy. So in the dummy database, what was supposed to have happened was, the illegal characters was supposed to have been stripped out. And what actually happened was instead of the entry, the test entry that I was supposed to have been modifying, being modified, it had modified every entry in the database. There was 750,000 of them and they were all identical. So it’d overwritten 750,000 fields in the database.

04:49 — Brent Simoneaux
750,000 fields.

04:54 — Johan Philippine
Yep. And pop quiz to see if you were paying attention. What did he overwrite the data with?

05:01 — Angela Andrews
I'm going to say Chris.

05:03 — Brent Simoneaux
His name.

05:03 — Johan Philippine
It was his name. Yeah. 750,000 times to the customer database.

05:10 — Brent Simoneaux
It's a lot of Chris.

05:11 — Angela Andrews
Poor Chris.

05:13 — Chris Kelley
By the time I'd finished, they were all populated with my own name, which was how the database admin knew that it was me that was responsible.

05:22 — Brent Simoneaux
Wouldn't it be funny if he was like, I don't know.

05:25 — Angela Andrews
Right? If F ups could just be that clear as to who did what.

05:31 — Johan Philippine
Could you imagine being the database administrator and just like maybe a message comes up, that something a little weird is going on, and you look in your database and you see the new guy's name.

05:42 — Angela Andrews
Chris, Chris, Chris, Chris, Chris,

05:44 — Johan Philippine
Chris, Chris, Chris, Chris, Chris, Chris.

05:45 — Brent Simoneaux
I feel like if I was Chris, I would just like assume that there was another Chris working in the office and just be like, it was Chris. The other Chris.

05:55 — Johan Philippine
Yeah. I think it may have been his full name. Chris Kelley.

05:58 — Angela Andrews
Man.

05:58 — Johan Philippine
Just to be extra thorough with the test script.

06:04 — Johan Philippine
So how could this happen? Right? Chris was running his test in the sandbox database, not the one used in production. He was kind of flabbergasted and on the receiving end of some choice words.

06:15 — Johan Philippine
So he went to find his mentor to figure out what he had done. And they checked out the code. It all looked really straightforward. They get the database admin to come join them. Now, calm down after being "extraordinarily angry" to try and figure it all out. They comb through the code and at one point Chris's mentor realizes what has happened and kind of puts his face in his hand and just like, Ugh. Oh no.

06:41 — Chris Kelley
So it turned out at some point in the past that the sandbox environments had been configured to point at the production database rather than the sandbox, because they were either decommissioning it or moving to another version or something along those lines. I don't remember the exact details, but it had been temporarily set to point at production, which was fine in the sense that they'd taken steps and make sure that it was read-only. However, comma,

07:07 — Angela Andrews
Oh, oh yeah. However,

07:13 — Chris Kelley
I'd accidentally taken advantage of a SQL injection vulnerability in the code. So my garbled string had just so happened to include the close quote and comment characters next to each other. So where my query was supposed to go for my test field in the database, do this, it instead had gone, do this, but to everything rather than do this to my field.

07:42 — Angela Andrews
Sheesh poor Chris.

07:44 — Johan Philippine
Yeah. I mean he knew what he was doing and he was trying to do it-

07:46 — Brent Simoneaux
Yeah.

07:47 — Angela Andrews
He did.

07:47 — Johan Philippine
In the way that he was taught and something still went wrong. So he thought he was working with a sandbox environment, but that had been set to point straight to the production environment temporarily. Not really his fault.

07:58 — Angela Andrews
Not really his fault at all.

08:00 — Johan Philippine
Yeah. And they'd taken precautions to avoid just this issue by setting it to read-only.

08:05 — Brent Simoneaux
Yeah.

08:05 — Angela Andrews
Yeah.

08:06 — Johan Philippine
But a SQL injection vulnerability overrode that read only setting and Chris's name ended up all over the customer database.

08:14 — Angela Andrews
Wow.

08:14 — Johan Philippine
That's just bad luck.

08:17 — Angela Andrews
Okay. So how do we un-ring this bell for Chris here?

08:22 — Johan Philippine
Well, now that they knew what the issue was, they went about fixing it.

08:26 — Chris Kelley
That was the first take home lesson was, you get to the bottom of these sorts of things much quicker if you collaborate with everyone, rather than immediately defaulting to finger pointing and blame analysis. And generally trying to prevent yourself from being fired.

08:41 — Chris Kelley
If everyone just sort of clubs together, you get to the bottom of things quicker. So we were able to remediate this pretty easily that the production database gets backed up regularly. We restored it to the most recent backup. Everything was there. No problem. If we'd spent loads of time arguing about it and not getting anywhere, we might have missed that rollback or something like that,

09:02 — Johan Philippine
Backups, they will, they will save you.

09:05 — Angela Andrews
They're going to save the world. Like having a good backup is how we survive in this world. Otherwise, we are doomed.

09:15 — Brent Simoneaux
Yeah. That's one thing that I'm sort of picking up from part one, you know, is that backups are like very important.

09:23 — Angela Andrews
Yes.

09:24 — Brent Simoneaux
Two also what he was saying about working together really quickly instead of finger pointing.

09:30 — Johan Philippine
Yeah.

09:31 — Angela Andrews
That's a time saver. That really is, when you can communicate and trust the folks that you work with, you get things done much quicker. That is so important. And it shouldn't be overlooked in this scenario.

09:45 — Johan Philippine
Yeah. Not at all. Well, we're going to move on to the next story. It's also from someone who was new to their career in tech and through a moment of inattention ended up doing more than he expected.

10:01 — Brent Simoneaux
Okay.

10:02 — Angela Andrews
Dun, dun, dun.

10:06 — Johan Philippine
This is act five. I call it: Spacing out. So Xander Soldaat, he is a Cloud Success Architect here at Red Hat. But at the time of the story, he was about three to four years into his career as a Linux administrator. And he was working on clearing out some servers so they could redeploy them. Now he didn't want to completely erase everything on them so that they could save time. So they wouldn't have to reinstall everything. They just wanted to erase some of the applications and then remove just enough to get them ready to redeploy.

10:39 — Angela Andrews
Okay.

10:40 — Xander Soldaat
I wanted to hurry up a little bit and get it done. So I was a little bit too fast on the old enter button. And before I knew it, I caused myself several hours of extra work. It was a simple RM minus RF. And then the folder was slash opt, but I put a space where it shouldn't be and it ended up being RM minus RF slash space opt

11:10 — Brent Simoneaux
Wait, What does RM minus RF do?

11:14 — Angela Andrews
Remove, so remove, it's a pretty safe command usually.

11:21 — Brent Simoneaux
Yeah.

11:21 — Angela Andrews
But when you go ahead and add the R, which means recursively through all your sub folders and then the F means force like, okay, don't ask me again. It literally is a death knell. Because he had the space between the forward slash and the word opt, it looked and said, oh, so you want me to remove everything recursively under slash, I got you covered. So wait, okay. When you do this and we've all done it, when you do this, does everything start moving in really slow motion? Like what happens here?

12:03 — Johan Philippine
Well, generally this command moves pretty quickly, right? When you're doing just like one folder, it acts super fast. Right? Angela?

12:10 — Angela Andrews
Yes.

12:11 — Johan Philippine
Now Xander was expecting this to be over really quickly, and have the terminal come up again and be able to type in another command pretty soon. But when he saw that the server was hanging for a little bit, he figured out something was wrong.

12:22 — Angela Andrews
It was thinking.

12:23 — Brent Simoneaux
Something's wrong here.

12:25 — Johan Philippine
Yeah. It was doing the thing. Just, it was doing it more than what he thought it would.

12:30 — Brent Simoneaux
Yeah.

12:30 — Angela Andrews
Oh gosh.

12:33 — Xander Soldaat
I hit enter. And it took maybe about second because, I thought, "Hey, this is taking a bit longer than it should." And because the opt folder wasn't terribly big, so it shouldn't have taken very long. And that's when I saw the space. And I was like, oh no. And just like when you're in a movie, you see the actor and having that moment of realization and everything around them shrinks. And it's just their face in the middle of the scene. Well, that's kind of what it felt like at that moment. I felt like a bit of a panic attack.

13:18 — Brent Simoneaux
Oh, no.

13:19 — Angela Andrews
No, he sure did. He knew exactly what he did. And that is a horrible feeling.

13:25 — Johan Philippine
Yeah.

13:26 — Angela Andrews
What's a good analogy for doing an RM dash RF on root. It's almost like what, sitting on a branch and cutting the branch while you're sitting on it. It's not good.

13:38 — Johan Philippine
That or you're giving someone a haircut and then take the clippers and you meant to do it at one setting, but you end up shaving everything off in like a big stripe in the middle of the head.

13:47 — Angela Andrews
Yeah.

13:48 — Brent Simoneaux
Oh,

13:49 — Johan Philippine
You know where you took off way more than you expected to.

13:51 — Angela Andrews
He took off way more than he expected to.

13:53 — Johan Philippine
Exactly. Yeah. He trimmed that server just a little bit too much.

13:58 — Angela Andrews
Poor guy. He had the best intentions.

14:02 — Johan Philippine
Oh. And he did what you were telling Joanna Delaporte to do last episode too, which was hitting control C as soon as possible. But by then the damage had been done.

14:09 — Angela Andrews
Damage was done. Oh man.

14:11 — Johan Philippine
It was time to face the music.

14:15 — Brent Simoneaux
So what happened?

14:17 — Xander Soldaat
Well, the next step was, cursing a little bit to myself, and seeing if anyone was watching or had seen what I had done. That's kind of like an instinct, I guess. But basically I had to go to the team lead and say, look, I made a mistake. Server is basically unusable right now. Do we have any backups? And, and if so, can we restore one.

14:44 — Xander Soldaat
And turned out they did have a backup and we restored it. It wasn't a really a massive major operation to do so, but it was rather frightening because it was basically, it was my first real mistake at work.

15:00 — Angela Andrews
It took him three to four years to make a real mistake. Woo. He must be good.

15:05 — Johan Philippine
Yeah. Right. Impressive.

15:06 — Brent Simoneaux
That's pretty good. Yeah.

15:07 — Johan Philippine
Yeah. That's a pretty good track record. Right?

15:09 — Angela Andrews
He is pretty good.

15:11 — Johan Philippine
But yeah, here we are again, backups. They're the greatest fallback.

15:15 — Angela Andrews
Backups, backups, backups, I guess they knew somehow that, that was a good backup. Maybe they had tested this restore procedure once or twice before, and yeah, they got their data back, which is amazing. That's really why we back up all the things, is so we can recover in the event of this or any other mistake or catastrophe.

15:38 — Brent Simoneaux
I think my favorite thing about this story is him realizing what had happened. And then looking around to see if anyone has noticed.

15:48 — Angela Andrews
Did y'all see what I just did? It's like you getting caught, like you did something wrong, and it's like, oh my God, did anyone see what I just did? That's a funny part of this story. Yeah. Poor thing. I feel bad for these folks. I feel so bad for them.

16:06 — Johan Philippine
Well, he learned an important lesson here that a lot of us could really use, taking your time is great. And double checking your work, even before you do something you've already done dozens of times, is always a good idea, because that overconfidence can really come back to bite you.

16:25 — Johan Philippine
All right. I've got one last story for you. This one is all about timing and miscommunication. This is act six, and I call it pedal off the metal. It's the 1980s, Christine Caulfield is in her early twenties. She would later become a Principal Software Engineer here at Red Hat. But when we meet her for this story, she's about a year into her first real job.

16:56 — Johan Philippine
She's working at a commercial apprenticeship with British Aerospace, working out of a hangar actually. She was in charge of the offices' VAX systems.

17:06 — Angela Andrews
Ooh, that's a word you don't hear very often anymore.

17:09 — Johan Philippine
Right? These are some old systems that you don't really see anymore, but they're quite popular back then. One of them, one of the larger systems included an early version of email for their office and also the design software that engineers would use to design and create airplane parts.

17:28 — Brent Simoneaux
Mm.

17:28 — Angela Andrews
Okay.

17:29 — Brent Simoneaux
So Christine is working at this aerospace company. They're building planes. She's in this airplane hangar.

17:37 — Johan Philippine
That's right.

17:37 — Brent Simoneaux
Okay.

17:38 — Johan Philippine
She was doing her apprenticeship and she was in charge of the VAX systems, right? So she didn't necessarily know all that much about airplanes and aircraft manufacturing, but-

17:45 — Angela Andrews
She was a systems administrator.

17:47 — Johan Philippine
Exactly.

17:48 — Angela Andrews
She was administering the system. Okay, cool.

17:51 — Johan Philippine
They also had another system, another VAX system, separate from the design one that we were just talking about. It's also a VAX system, but it controlled the Computer Numerically Controlled machines that actually made the aircraft parts on the shop floor. And she abbreviates this as CNC machines.

18:10 — Christine Caulfield
When you write a program for a CNC machine, as far as I understand it, you'll have a cutter and that you've programmed it where that goes on the piece of metal. So you start with a big block of metal, maybe a rectangular block of metal. And then you write a program that tells the machine how to cut bits out of that metal to make into a part, a bit like sort of sculpture, but done by a robot if you like.

18:35 — Angela Andrews
Or a cricut machine.

18:36 — Johan Philippine
Yeah.

18:36 — Angela Andrews
For all my crafters out there.

18:40 — Johan Philippine
Just to extend her metaphor a little bit here. Think like Michaelangelo sculpting David out of marble, right?

18:46 — Brent Simoneaux
Yeah.

18:46 — Johan Philippine
He's taking that block of marble and cutting away everything that wasn't the David, except in this case, instead of Michaelangelo, it's a robot, and instead of a marble human, it's aluminum or other metal aircraft parts.

19:00 — Brent Simoneaux
I'm also guessing these are pretty big too. Right?

19:04 — Johan Philippine
It depends, some of these parts, they're pretty small.

19:07 — Brent Simoneaux
Yeah.

19:08 — Johan Philippine
And they would be cut pretty quickly within a few minutes.

19:10 — Brent Simoneaux
Gotcha.

19:11 — Johan Philippine
While others, these were these big blocks where it would take like a whole day to get them done.

19:16 — Brent Simoneaux
Okay.

19:16 — Johan Philippine
On the day of her story, Christine is about to finish up work and head home.

19:24 — Christine Caulfield
So I was in there on my own. It was about four o'clock in the afternoon, I think. And I got a phone call from one of the CAD/CAM programmers at their department, that said, can you reboot these VAXs?

19:36 — Christine Caulfield
And I said, yeah, of course, because although they stick people in the CAD/CAM department were a sort of programmers. They didn't have privileged access to the computers. That was our domain. And I'd always been told that these people knew what they were doing to some extent. So when someone rang me and said to reboot machine, you think, yeah, it's a common thing with computers. You turn it off and on again, if something goes wrong. So I thought that's fine. So I logged into the computer and rebooted it as I was asked. I'm the sort of person who says, if you ask me to do something, I will just go and do it. Because why wait and do it later on when you can get it done and then it's done.

20:14 — Johan Philippine
All right, job's done, five o'clock rolls around. And she heads home for the day. Right? She did what she was asked. And that was it.

20:23 — Johan Philippine
Now I mentioned that Christine was working out of a hanger. Their office was actually above the main hanger and was a pretty open space except for the manager's office. This office was right by the entrance to the open floor and had a "panopticon" view of the goldfish bowl office, so that he could keep an eye on his domain.

20:44 — Brent Simoneaux
All right. Foucault.

20:45 — Johan Philippine
They meant two things. You had to walk by the office to get in. And it was very visible, impossible to miss.

20:53 — Brent Simoneaux
Yeah.

20:54 — Johan Philippine
The next morning, Christine comes back to the office to work.

21:00 — Christine Caulfield
I came in the morning and I had to walk past the manager's office. And there was this large piece of metal in his office. I thought, oh that's new. I went down to my desk and got on with my job.

21:12 — Johan Philippine
Now this was a big piece of metal, right? It was an airline rib, about five feet high and three feet wide.

21:18 — Brent Simoneaux
Oh wow.

21:18 — Johan Philippine
With lots of lugs sticking out of it. It was also aircraft grade aluminum. So it was very shiny. And it's not really something that you would see in the manager's office.

21:29 — Angela Andrews
Unless he's a collector. I mean otherwise, why would it be in his office?

21:33 — Johan Philippine
Well, we're about to find out.

21:36 — Christine Caulfield
My supervisor called me over, a bit later on in the morning, and sort of pointed it out and said, "Did you reboot a VAX last night?" I said, "Yeah." He said, "This happened," he's pointed to this metal and said "It was busy machining this Airbus rib when you shut it down and it just played havoc with the CNC machine."

21:57 — Brent Simoneaux
Oh no.

21:58 — Christine Caulfield
"And we scrapped about 40 grands worth of material."

22:01 — Angela Andrews
Whoa.

22:04 — Christine Caulfield
There was a large gash running down, sort of says almost diagonally, like when someone slashes a painting in a museum, maybe, you know, that sort of thing.

22:12 — Christine Caulfield
It didn't go all the way through. But it was just a sort of big hole. It was not very regular at all. In just in the wrong place.

22:20 — Johan Philippine
So it turns out that the CAD/CAM programmer believes he had asked Christine to reboot it later, but that's not what she heard. When she rebooted it, it was earlier than the programmer actually intended, and that corrupted the robot sculptor's instructions in the middle of a job. And then rather than continuing to produce the part as instructed or even stopping the sculpting in the middle of the job, it kept on cutting in an unintended line.

22:51 — Angela Andrews
Wow.

22:52 — Johan Philippine
So this large expensive piece of metal, which she mentioned was worth £40,000 in the 1980s.

23:00 — Brent Simoneaux
Yeah.

23:00 — Johan Philippine
Which is now more than 150,000 US Dollars.

23:04 — Brent Simoneaux
Oh man.

23:05 — Johan Philippine
After accounting for inflation and all that.

23:07 — Brent Simoneaux
Yeah.

23:08 — Johan Philippine
It's got this huge three inch gash going across it, making it completely useless. Also remember that the machine shop was in another building. So to do this little piece of theater, the boss had to transport this big piece of metal, quite a ways. And then up a floor to display it to the entire office.

23:27 — Angela Andrews
Point made sir, point made.

23:30 — Johan Philippine
She didn't lose her job or anything, but it did change how she responded to requests from then on.

23:36 — Christine Caulfield
I think the main takeaway from that and something I use almost every day in my job now is to find out what people really want.

23:45 — Christine Caulfield
When they ask you to do something, what are they actually after? Because sometimes what they think they want is not what they actually want. If you don't know what you're doing, as I didn't at the time, then ask people what they want because they will tell you and you will learn something.

24:01 — Christine Caulfield
If you do know what you're doing, as I mostly do now, still ask people what they want, because they will learn something and you will also learn something too, because it might be something you didn't know either, but either way, it's good to actually get into conversations with people about what is actually required. Because once you start assuming things, and taking things at face value, bad things happen as we find out.

24:26 — Brent Simoneaux
Yeah. It is always a good idea to repeat back what you have heard.

24:32 — Angela Andrews
Yeah.

24:33 — Brent Simoneaux
Or what you have understood because so much gets lost in translation.

24:39 — Angela Andrews
So what you're saying is, you want me to reboot this server?

24:43 — Brent Simoneaux
Yeah, exactly.

24:45 — Angela Andrews
Now?

24:45 — Brent Simoneaux
Now.

24:47 — Angela Andrews
Question mark now.

24:49 — Brent Simoneaux
Yeah.

24:49 — Angela Andrews
That's great advice though. It really is.

24:52 — Brent Simoneaux
Yeah.

24:53 — Johan Philippine
I feel like that's something computers should ask us before we do anything.

24:57 — Angela Andrews
Sometimes computers do ask us if that's what we want. Like when we're shutting something down and we haven't saved, sometimes they do ask, but this is not that.

25:09 — Johan Philippine
Yeah. And even when they do check with us, we still make mistakes. So it's not foolproof by any means.

25:16 — Brent Simoneaux
Yeah. I mean, because there are humans involved. I'm thinking about these six stories that you've brought to us Johan, across both of these episodes.

25:30 — Brent Simoneaux
So our question is like, are big mistakes that big of a deal? And I think, when you make a mistake, it is a big deal. It looms emotionally large for you.

25:47 — Brent Simoneaux
Like you have this embodied reaction and then I think it probably looms in your mind for a long time. But also what I'm hearing you say Johan is that it's also not a big deal, right?

26:00 — Johan Philippine
Yeah. Because everyone makes them.

26:02 — Angela Andrews
Exactly.

26:03 — Johan Philippine
And most of the time, they move on, and their career continues, right? It's not going to be something that necessarily ends your career, ends your job and just has everything crashed down around you. Right. Even though it may feel that way in the moment, in the long term, especially in the tech industry, there's that capacity to learn from it and to move on.

26:23 — Angela Andrews
That's the key.

26:24 — Brent Simoneaux
But the only way that happens though, is if the people around you don't make it into a big deal, right?

26:30 — Johan Philippine
Yes,

26:31 — Angela Andrews
Yes.

26:33 — Johan Philippine
Yeah, exactly.

26:34 — Angela Andrews
So your environment matters as well.

26:36 — Johan Philippine
Yeah. Yeah. It's great when you're able to find a team where you're not expecting them to point fingers and assign blame, you're more working together to fix things when problems arise.

26:47 — Angela Andrews
You're a much freer contributor, when you're in a healthy environment, that knows that yes, mistakes happen, but we'll work together and we'll figure it out. And we'll all learn from them. So mistakes happen. But what you learn at the end of them is always an amazing lesson. And I think we've heard that from our guests.

27:08 — Johan Philippine
Yeah. And then a few years down the line, when it doesn't sting as much, it's a great opportunity for a story to tell.

27:14 — Angela Andrews
And you can be on the podcast.

27:16 — Johan Philippine
Exactly.

27:17 — Brent Simoneaux
Part three.

27:18 — Angela Andrews
Coming at you.

27:23 — Angela Andrews
So listeners, we know that you have your F ups somewhere in your history. If you think real hard, you can look at this episode and say, you know what? I've made a mistake, but I've also learned from it.

27:36 — Angela Andrews
And we want to hear your F ups, please share them with us. You can tweet us @RedHat on Twitter and you can use the #CompilerPodcast.

27:46 — Angela Andrews
So let us know. We're really interested in hearing about some of the mistakes you may have made, and also how you learned from them.

27:53 — Angela Andrews
And that does it for the second F ups episode of Compiler.

28:01 — Brent Simoneaux
Today's episode was produced by Johan Philippine and Caroline Craighead, Victoria Lawton is perfect. She's never made a mistake in her life.

28:13 — Angela Andrews
Our audio engineer is Kristie Chan. Special thanks to Sean Cole.

28:19 — Angela Andrews
Our theme song was composed by Mary Ancheta.

28:21 — Brent Simoneaux
A big thank you to our guests, Xander Soldaat, Chris Kelley and Christine Caulfield.

28:28 — Angela Andrews
Our audio team includes Leigh Day, Laura Barnes, Stephanie Wonderlick, Mike Esser, Claire Allison, Nick Burns, Aaron Williamson, Karen King, Boo Boo Howse, Rachel Ertel, Mike Compton, Ocean Matthews and Laura Walters.

28:46 — Brent Simoneaux
If you like today's episode, go ahead and follow the show. You can rate us, you can leave us a review. And even more importantly, you can share it with someone that you know, it really does help us out.

28:58 — Angela Andrews
It sure does. Thank you so much for listening. We'll see you next time.

29:02 — Brent Simoneaux
All right.

29:03 — Angela Andrews
Bye.