How Bad Is Betting Wrong On The Future?

Technologists are often asked to make decisions based on future industry advancements—or basically, things that haven’t happened yet. It’s already difficult to choose the right path for a project without the pressure to be clairvoyant. But everyone wants to feel like they are leading the pack on the next big thing. 

What do we need to know to make a good prediction for where technology is headed? Alternatively, what do we need to know to avoid the wrong choice? We speak to experts in the DevOps space about betting wrong on the future, how development projects go awry, and what teams can do to get things back on track.

00:02 - Brent Simoneaux 

Okay. So Kim, you sent us this link. I'm opening it right now. This is a Wikipedia page that says... At the very top, it says "list of failed and over budget custom software projects." What is this?

00:23 - Kim Huang 

This is a list of some big swings on the part of so many different government organizations, large software companies. And if you look at the problems and the reasons why they failed, you'll see a lot of the same themes pop up over and over again.

00:44 - Angela Andrews 

The usual suspects: I see scope creep, cost overrun, low user adoption, just delays, delays, delays. Nothing on this list surprises me.

00:56 - Brent Simoneaux 

Oh, this is kind of fascinating.

00:58 - Angela Andrews 

Yeah. Look at the money. The money!

01:02 - Brent Simoneaux 

That's what's striking me about this list is there is a column that says expected cost.

01:07 - Angela Andrews 

Yes.

01:07 - Brent Simoneaux 

And we're talking three to six billion US dollars-

01:12 - Angela Andrews 

Mind blown.

01:14 - Brent Simoneaux 

And then there's another column right next to it that says outcome. And it's just canceled, scrapped.

01:21 - Kim Huang 

It doesn't look good. No.

01:22 - Brent Simoneaux 

Canceled, scrapped, abandoned, discontinued, canceled, canceled, scrapped, scrapped, canceled. I shouldn't laugh because it's so painful but-

01:33 - Kim Huang 

It makes me wonder; technology's an environment where things tend to go wrong more than they go right. And when you're trying to build something so ambitious and so large in its scale and its scope, how do you avoid making the wrong bet on what's going to happen down the road?

01:58 - Brent Simoneaux 

This is Compiler, an original podcast from Red Hat.

02:02 - Angela Andrews 

We're your hosts.

02:04 - Brent Simoneaux 

I'm Brent Simoneaux.

02:05 - Angela Andrews 

And I'm Angela Andrews.

02:07 - Brent Simoneaux 

We're here to break down questions from the tech industry, big, small, and well, sometimes strange.

02:14 - Angela Andrews 

Each episode, we share stories from the people behind the code.

02:19 - Brent Simoneaux 

Today's question, how bad is betting wrong on the future?

02:23 - Angela Andrews 

Producer Kim Huang is here with our story.

02:30 - Kim Huang 

So that list I talked about earlier, it was referenced in a book. The book is called DevOps Culture and Practice with OpenShift. A lot of the text of the book talks about how to get things off the ground. And DevOps is obviously... that's what it's for is to get teams from zero to cruising altitude as soon as possible. So to that point, I wanted to talk to some of the people who are behind the book. One of those people is Noel O'Connor.

03:04 - Noel O'Connor 

I'm a senior principal architect working in the EMEA Solutions Practice. What I do at Red Hat, I do a lot of things. I'm primarily in the consulting area. Basically helping customers around the world, basically adopt–perhaps–technology and approaches.

03:22 - Kim Huang 

I asked Noel, who's based in Ireland, about leaders making the wrong bet on the future. Planning for what's there, what's present and not really what there could be down the road. He talks a lot about what he calls marketecture.

03:37 - Angela Andrews 

I'm interested in finding out what marketecture is.

03:40 - Kim Huang 

Marketing architecture. It comes off as a... not a slang, but like a dirty word, the way that he said it, because he talked about how sometimes you have people who talk about technology and project this air of, there's a product that they're making and the product is going to do this amazing thing, but it actually doesn't work.

04:08 - Noel O'Connor 

Really good marketing has a fairly significant impact. And we've seen that down through stuff that's gone on in the last number of years on technology projects that failed due to "how hard can it be?" And it can actually be very, very hard. I mean, a lot of good marketing makes things seem really simple and really flexible. And its job is to position technology in the right place and the right light. Whereas actually a lot of times they don't want to show how this could potentially go wrong or things like the cases where it doesn't fit.

04:43 - Brent Simoneaux 

Well, let's talk about the future a little bit here.

04:45 - Kim Huang 

Sure.

04:48 - Brent Simoneaux 

Right. Because this is something that we have to make decisions here and now, and a lot of the decisions that we have to make about the products that we're building or the technologies that we're using, we have to be able to look out into the future and make a really good guess about what the future is going to be. And this is really difficult.

05:13 - Angela Andrews 

Yes. I know it is because there are companies who… you think about it, they have this product that tends to become ubiquitous and they do just one thing, I don't want to say wrong, but going away from where people think technology should go. You're heading in the right direction, but you make one false move and you've definitely fallen off a cliff. So it's nice to be able to foreshadow, but there's really no way to know because we all want to be innovative, we all want to be on the cusp of the next big thing. We don't know what that is. So someone's going to take a bet on something and they're going to lose that bet. And that has to happen I think, in order for technology to move in the right direction. I know it doesn't sound like it makes sense, but someone has to lose in this scenario because every good idea can't be the right idea. So that's just my opinion about it. There has to be some winners and losers when we're talking about the future.

06:31 - Brent Simoneaux 

It's not just about the products that we're building either. It's about the technologies that we're using.

06:36 - Angela Andrews 

Yes. Very much so.

06:38 - Brent Simoneaux 

To build and deploy those projects as well, because those are really important. And I'll say often they can be really expensive as well. Right?

06:48 - Kim Huang 

Yes.

06:49 - Angela Andrews 

Yes. 100%.

06:50 - Brent Simoneaux 

Yeah. But those are other decisions. It's not what you're building, it's how you're building it as well.

06:58 - Kim Huang 

Absolutely. And what teams have to come across and work with each other, not necessarily from a development standpoint, but just on a logistical standpoint. They have to work together to allocate budget for hires. If you need to get somebody on board that has a specific set of skills, or if you need to make a team bigger, or if you have to buy a certain type of testing environment in order to test your app. All of that needs to be factored in. And those decisions don't necessarily come from the people who are actually writing code. Right?

07:34 - Brent Simoneaux 

Yeah.

07:35 - Kim Huang 

That's the rub. For me, I feel like guessing about the future is fine, as long as you admit to yourself and to everyone around you that you're guessing. As long as you're honest about that.

07:48 - Kim Huang 

Noel had something great to say about "the horizon that you can see" and how important it is to plan for the horizon that you can see within your line of sight, while still being innovative.

08:02 - Noel O'Connor 

If somebody knows what the future's going to bring, I'd like them to tell me how I'm going to win at casinos. Because it's like a... Event horizon sounds like the wrong word, but basically you can only see so far. I mean, we're in the realms of containers now and clouds. And basically if you'd said 15 years ago, was this going to happen? Nobody would've known. And also Red Hat, we constantly pop our heads above the parapet, take a look around and see... Because we work so much in open source, it's basically, there's a lot of community stuff going on and you can see trends that are building and stuff that's falling away.

08:40 - Kim Huang 

So Noel started out talking about how situations where you have "marketecture" people trying to have that crystal ball kind of mentality. That's a situation where it's really ripe for people to make the wrong bets in technology. I wanted to talk about the real practices and methodologies that we as technologists could employ to prevent teams from going down a bad path. And for that, I spoke to Tim Beattie, also in Ireland. He's a former Red Hatter and he wrote the book on DevOps, along with Noel.

09:22 - Tim Beattie 

And I'm the co-founder of Stellafai, which is a solutions company that focuses heavily on visualizing and measuring business outcomes and sharing that for others to learn from.

09:34 - Kim Huang 

Here's the thing. When we think of DevOps as Red Hatters, we think of practices and collaboration and innovation and trying to iterate on discovery very quickly. But when other people think of DevOps, it has come to mean something else in the industry. It might not come as a surprise-

09:56 - Brent Simoneaux 

Angela laughs.

09:57 - Kim Huang 

Oh, I know. She's already anticipating what I'm going to say. It may not come as a surprise, but sometimes DevOps is looked upon as something you hire another person to come into your company and do, instead of what teams and organizations do as far as changing their culture. I wanted to know how that happened.

10:21 - Angela Andrews 

It became a buzzword and we were hiring for the buzzword and no one... They thought it was a product, or it was... What is it, the phrase that we used? Marketecture. It was one of those things that is like, "I'm going to buy me some DevOps and we're going to get some DevOps in here." And it's like, "No, that's not how it works. That's not how any of this works!" No, you really have to change your culture internally. It will not work or make sense unless you do that. So yes, it's more of a mindset and a methodology and it's not something you can buy online.

10:59 - Kim Huang 

I'm trying not to laugh. This is a very serious subject. I'm trying to be serious. I'm sorry. But I keep thinking of like, "I'm going to come in here and I'm going to DevOps everything and DevOps that chair and DevOps this desk."

11:16 - Angela Andrews 

You're getting DevOps, and you're getting some of that DevOps. Seriously. It's funny now, but it was all the rage.

11:24 - Tim Beattie 

For a long time "lean" was a very in demand buzzword and then agile replaced that and then DevOps replaced that.

11:34 - Angela Andrews 

What's next?

11:35 - Kim Huang 

I don't know, make up a word, I guess.

11:37 - Angela Andrews 

I mean, I already know the next hot word. It's called zero trust, but we're going to be buying some of that zero trust real soon.

11:47 - Kim Huang 

Oh, you get a zero trust and I get a zero trust.

11:50 - Angela Andrews 

We're all getting some zero trust.

11:52 - Kim Huang 

Yes. I was curious, I wanted to posit an example for Tim to work on. In Tim and Noel's relationship, they identify each other as Tim being the practices guy and Noel being the tech guy. I wanted to–I know this sounds bad… I wanted to test Tim to see what his response would be if I were a hypothetical tech founder.

12:23 - Brent Simoneaux 

Okay.

12:24 - Kim Huang 

So I asked him, imagine a tech founder who's trying to build an app. They have the concept, maybe. They may even have done some practices like impact mapping to figure out business goals and deliverables, but they're experiencing a lot of scope creep and a lot of increasing costs.

12:44 - Brent Simoneaux 

Those things that we were seeing on that list that we looked at at the top of the episode.

12:49 - Kim Huang 

Yeah. For real.

12:50 - Angela Andrews 

What you don't want to see inside of your project.

12:53 - Brent Simoneaux 

Yeah.

12:53 - Kim Huang 

Exactly. So I asked Tim, what are some things that would be very useful for a person in that situation to think about?

13:01 - Brent Simoneaux 

Yeah.

13:03 - Tim Beattie 

Where we encourage learning, where we are prepared to adapt and throw things away and try completely different directions to run lots of experiments, sometimes very cheap experiments. And there's a load of practices we talk about in our book around building the foundation of culture to help put that in. There's something that takes time and also you have to continuously nurture and improve, because the moment, and there's some big examples in the news where there hasn't been safety and that's where companies end up in all sorts of problems.

13:37 - Tim Beattie 

My go-to practices are around in designing of experiments, figuring out what are the hypotheses that we want to test and how can I go and prototype some of my ideas and test them in as lean and cheap a way as possible. And that doesn't necessarily mean building the tech. It means figuring out what we think might improve or provide the real gains and seeing if I can go and learn from some real end users about that. And that maybe means executing things manually or even putting a very low fidelity prototype in front of some users and getting their feedback.

14:20 - Brent Simoneaux 

Bring this to life for us Kim, a little bit. So he's talking about doing these, what he is calling cheap experiments. Bring that to life for us.

14:30 - Kim Huang 

It's like earlier when I was talking about buying a testing environment for your applications. And I'm definitely not the expert in this, but I think what he's trying to get at is that a lot of founders get into a situation where they have, not a plan, but a list of desirable things that they want to do, and a list of things that they need in order to do them. And they go from there. So you want to do something as cheap as possible, and as fast as possible. You don't want to blow your budget on something that only gets you from point A to not point B. You want to get to the next step, to development and eventual release as soon and as fast as possible. So some of the examples here, he talks about prototyping and testing. It's something that's really important because you also need to know what your user wants. Just trying to find out the deliverables and the functional requirements you have listed, you have to find out what users want and match those up to what you want to deliver.

15:33 - Angela Andrews 

Exactly. So that point really hits home because to find out what users want, you have to talk to them.

15:41 - Brent Simoneaux 

Yeah.

15:42 - Angela Andrews 

You really do.

15:43 - Brent Simoneaux 

It sounds so simple, right?

15:45 - Kim Huang 

Yes.

15:46 - Angela Andrews 

Yeah. But I'm sure if you looked at that list, some folks weren't hitting the mark because they weren't doing that early and often.

15:54 - Brent Simoneaux 

Yeah.

15:56 - Kim Huang 

Early and often. And that is something that I think a lot of the practices within DevOps culture, really stress. Don't just do it one time and then say, "Welp." Dust yourself off, dust your hands off and go, "That's it. And I don't need to go back and find anymore information." No, you have to constantly be looking and discovering things.

16:19 - Brent Simoneaux 

And as we're thinking about the future, we talked about how it's really impossible to really predict the future or know exactly the future. But you can know some things about what's happening now. In terms of your users and in terms of different things. And you can know those things by doing some of these cheap experiments. You don't have to guess at some of this stuff. You can know some of this stuff within a reasonable doubt.

16:50 - Angela Andrews 

Right.

16:52 - Kim Huang 

Right. I think this is, Brent, very interesting and a really good segue from what you were saying.

16:57 - Brent Simoneaux 

Yeah.

16:58 - Kim Huang 

Which brings me to the next point that Tim and Noel left me with. From an IT perspective, planning is dead.

17:07 - Tim Beattie 

One thing that its made me think of, if I think back to the planning I used to do 15 years ago, 20 years ago, it wasn't planning at all, it was guesses. We were making wild guesses on everything. Moving away from linear thinking where it's a big long path from A to B, it's actually, you're going round in loops. You're going round in circles. And what you end up doing could be incredibly different to what you started out… what you thought at first. Because you've learned so much along the way.

17:37 - Kim Huang 

What his point is, is that planning is dead because in order to really think about the future, you have to accept that things go wrong just as often as they go right.

17:49 - Angela Andrews 

That's true. I mean, you learn in those spots where... You learn from your mistakes, as they say. When you find out that something goes wrong: "Well, okay. All right. That's one way not to do something. Let's figure this out again." So, that's constant. And you think of what you've amassed along this process. So I get what he means when he says planning is dead.

18:14 - Brent Simoneaux 

But he's not saying that all planning is dead though. Right?

18:18 - Kim Huang 

Of course not. I think that they're saying down the road, that planning is going to be not as important, not as imperative as it has been in the past, just because there's so many things changing all the time happening in sync with what development teams are doing. It makes your plans irrelevant within two or three years, especially if you're working on projects as big as the ones we talk about in the beginning of the episode. Some of those projects went on for so long that there were developments in the field that made the project obsolete.

18:54 - Brent Simoneaux 

Is he saying that long term planning is dead and following that rigidly? Is he just saying we need to move our sight line up? It's more about short term planning in a way that has a lot of nimbleness. Is that it?

19:13 - Kim Huang 

Maybe.

19:14 - Angela Andrews 

That sounds...

19:15 - Kim Huang 

Yeah.

19:16 - Angela Andrews 

Yeah. To me, it sounds like that's what he's saying, because technology 30 years ago; it didn't iterate as quickly. So you could really plan for the next 10, 15 years. Technology didn't move that fast. But how can you do that now? It's almost impossible. So your sight line has to be so much shorter and you have to be okay with that, but be ready to iterate when the time comes if something changes. You said it perfectly, that resilience that we need now, because things pivot so much more quickly than they used to.

19:56 - Brent Simoneaux 

So these two have written an entire book about DevOps. What does DevOps give us? What are some of those practices that help us be more nimble?

20:08 - Angela Andrews 

Oh, definitely. Can I take this one?

20:10 - Kim Huang 

Yes, please.

20:11 - Angela Andrews 

I have a couple of ideas about this.

20:12 - Brent Simoneaux 

Go.

20:12 - Angela Andrews 

So the whole purpose of DevOps is everything is in code. So you're not wedded to anything. As long as you have your repos where your code exists and your pipeline, you can really... I want to say you can be resilient as long as your code is there and it can be deployed somewhere. What the cloud brings us, it brings us that elasticity. It brings us that flexibility. I think this is where you are going with this, but correct me if I'm wrong. But what DevOps buys you is flexibility. It buys you repeatability. That's the whole thing. To be able to repeat your processes without putting your hands on things. To me, that's what it's buying folks right now. It buys them a lot of agility.

21:07 - Kim Huang 

Yes. It's not the long jump, it's hopscotch.

21:11 - Angela Andrews 

There you go.

21:11 - Kim Huang 

You're getting to the point where a lot of people… the long jump, it sounds very appealing. But in actuality, you leave yourself vulnerable and open to a lot of challenges. In the approach with DevOps, you have this continuous feedback loop with your teams, as far as what you're doing and why you're doing it. Then you get to test that early and often. You get feedback and go right back into a discovery phase. And now you have this never ending loop of iteration. It makes development cycles easier and more efficient. Here's Tim again.

21:54 - Tim Beattie 

I mean, I think it allows us to try things out. It allows us to experiment at relative ease, compared to maybe 10, 20 years ago where it would've taken a big investment and a big leap of faith to try something new out. All of the technology and the practices and the mindset around DevOps takes us down the path of, we should embrace the idea of experimentation. We run some experiment and it didn't work out because we learned from it. And we've done that relatively cheaply because we're able to build things and deploy them, and put them in the hands of users and learn from them much, much faster, at a much, much lower effort and lower resource cost than what we used to have to do.

22:39 - Tim Beattie 

So when we talk about the future, does DevOps allow us to future proof? No, because the future is still an unknown. But what it does allow us to do, it allows us to just step lots of different directions into the future. And with the paths that we do then take down, we can apply with a bit more confidence, because we're actually getting some empirical outcomes as we go along.

23:07 - Brent Simoneaux 

Oh, that is so interesting. So he's saying instead of making one big bet on something, maybe you can make a lot of smaller bets.

23:19 - Kim Huang 

Yes.

23:20 - Brent Simoneaux 

And then follow where those bets lead you.

23:22 - Kim Huang 

Mm-hmm (affirmative). That's what discovery is.

23:26 - Brent Simoneaux 

Huh.

23:26 - Angela Andrews 

No longer do we have to go for that long game, because we're being more nimble. What was the word you used? The smaller bets.

23:36 - Kim Huang 

Mm-hmm (affirmative).

23:37 - Angela Andrews 

Yeah. Why not? Follow them where they take you as opposed to going out there on a limb.

23:43 - Kim Huang 

That's right. Those small bets, just doing a little at a time can allow people to discover new things that they probably never thought of in the first place. And most of all, practices like those included in DevOps reduce the likelihood of you ending up on a Wikipedia list.

24:04 - Angela Andrews 

That's a list you don't want to be on.

24:06 - Kim Huang 

No.

24:11 - Brent Simoneaux 

All right. So Kim, Angela, let's come back to our question. If you remember it was how bad is betting wrong on the future? From the Wikipedia list, it sounds really bad. But what you've brought us today Kim is a set of practices and a way of thinking about the future to help us be more future resilient. So I'm hoping we could be very practical right now. What are just a couple of the things that we can change about how we think about the future or a couple of things that we can do to make us more resilient?

24:55 - Kim Huang 

So the first hurdle is to change the way we think about the future, and accept the fact that while we are building things and we are in our, by their very nature, these silos. We're in a competitor state with other organizations and other teams and other creators trying to do the same things that we're doing, trying to serve the same customers that we're trying to serve. It seems like designing things and developing things, inherently you're going to run the risk of someone coming up with some kind of alternative that makes whatever you're building obsolete or makes it undesirable in comparison. There's always a chance of that happening.

25:39 - Kim Huang 

I think it's important to understand that there are practices in place to get from zero to a cruising altitude, in a metaphorical way, very quickly. There are practices that you can employ to find out what the people that are going to use your app, what do they want? The people that are going to use whatever product you're trying to build, what do they really need? What is it that's going to make them choose your product amongst a host of other ones that are being offered to them? To me, that's part of trying to break down the future and trying to get away from this crystal ball mentality and going towards a, what I call hopscotch mentality, where you're getting to the next small kind of milestone, yes, at a time. Exactly.

26:31 - Brent Simoneaux 

But also what I love about that metaphor is: hopscotch is fun.

26:35 - Kim Huang 

Yes, it is.

26:37 - Angela Andrews 

Yes, it is. It makes it so much less stressful when you think about moving toward the future. It makes it a lot less stressful. Just a square at a time.

26:46 - Brent Simoneaux 

Yeah. And it takes a lot of the weight off of the decisions that you're making.

26:50 - Angela Andrews

Definitely. Can you imagine having to have the forethought of what something needs to be for the next 10 years, 15 years?

26:59 - Kim Huang 

A lot of pressure.

27:00 - Angela Andrews 

A lot of pressure. How about we just do one square at a time and see where it takes us.

27:05 - Angela Andrews 

And that does it for this episode of Compiler.

27:13 - Brent Simoneaux 

Today's episode was produced by Kim Huang and Caroline Creaghead. Victoria Lawton always brings us back to the future.

27:23 - Angela Andrews 

Our audio engineer is Elisabeth Hart. Special thanks to Shawn Cole. Our theme song was composed by Mary Ancheta.

27:32 - Brent Simoneaux 

A big thank you to our guests Noel O'Connor and Tim Beattie.

27:37 - Angela Andrews 

Our audio team includes Leigh Day, Stephanie Wonderlick, Mike Esser, Laura Barnes, Claire Allison, Nick Burns, Aaron Williamson, Karen King, Boo Boo Howse, Rachel Ertel, Mike Compton, Ocean Matthews, and Laura Walters.

27:55 - Brent Simoneaux 

If you liked today's episode, please follow us. Rate the show. You can also leave us a review. It really does help out the show.

28:04 - Angela Andrews 

It sure does. And we appreciate it. We'll see you soon.

28:07 - Brent Simoneaux 

All right. See you next time.