The Return on Investment of Threat Modeling

The Security Table team dialogues about the importance of data and metrics in understanding and communicating risk. After Matt defines ROI, Izar emphasizes that while data is crucial, it doesn't always come in numerical form. Instead, risk can be expressed in various ways, such as trends, and doesn't necessarily need to be quantified in traditional terms. Chris stresses that executives need tangible metrics and data to make informed decisions, especially when communicating with legal teams and other stakeholders.

They then talk about visibility and understanding the attack surface. Izar explains that the attack surface represents an organization's exposure to potential threats. The goal is to provide a comprehensive picture of the organization's vulnerabilities and the measures taken to address them. Instead of inundating executives with technical reports, Izar suggests telling a story that conveys the essence of the risks and the steps taken to mitigate them. Chris, however, emphasizes the importance of concrete data and the challenges executives can face in understanding technical nuances.

Lastly, the dialogue touches upon the real-world implications of threat modeling and its ROI. Matt Coles highlights the potential legal and business repercussions if things go awry. The discussion underscores the evolutionary nature of threat modeling, with Izar noting that while one might start with limited expertise, continuous learning and adaptation lead to improvement over time. The overarching theme is the balance between technical details and business-oriented communication, ensuring that executives understand the value and impact of threat modeling initiatives.

Links referenced:

• US Executive Order 14028 on cybersecurity - https://www.federalregister.gov/documents/2021/05/17/2021-10460/improving-the-nations-cybersecurity
• CISA, Secure by Design, Secure by Default - https://www.cisa.gov/securebydesign
• Secure Software Development Framework (SSDF) from NIST - https://csrc.nist.gov/Projects/ssdf

FOLLOW OUR SOCIAL MEDIA:

➜Twitter: @SecTablePodcast
➜LinkedIn: The Security Table Podcast
➜YouTube: The Security Table YouTube Channel

Thanks for Listening!

Transcript

Chris Romeo 0:09

Hey folks. Welcome to another episode of the Security Table. This is Chris Romeo, joined by my good friends, Izar Tarandach and Jerry Garcia. No, no, no. Sorry. Sorry. I forgot I was a little, little. A little confused for a second, but it's actually Matt Cole, not Jerry,

Izar Tarandach 0:28

just because he looks a bit, a bit hazy.

Chris Romeo 0:32

Just look, maybe he could sing a couple of, uh, a couple of bars for us, maybe?

Matt Coles 0:37

Absolutely not.

Chris Romeo 0:38

Absolutely not. Thought we might get a, you know, truckers or, you know, lemme see. Something Magnolia Sweet Magno. I don't remember.

Matt Coles 0:48

I have no idea what what you're talking about.

Chris Romeo 0:50

Grateful Dead songs, but, so yes.

Matt Coles 0:55

College was a haze because of, uh, great Grateful dead song.

Chris Romeo 0:59

There you go. You're not, you're not old enough for that to line up, unfortunately.

Matt Coles 1:04

Tell that to the rest of my floor in college.

Chris Romeo 1:07

So we're gonna continue our. Jaunt into the world of threat modeling. So in our previous episode, we had a great conversation with Jim Manco about where he sits in the world of threat modeling, and we heard about how he, uh, where, where his ideas were kind of coming from. I thought that was pretty eye-opening. His view of consultants and how they abused threat modeling in the early years of AppSec, I thought was, uh, was a good perspective. It was one that I hadn't. Necessarily seen firsthand being inside a product company and, and not having consultants that were being, that were doing the threat modeling for us. I thought that was a really interesting viewpoint, but we want to talk about the return on investment for threat modeling. And so I thought we'd start by, let's define return on investment just for those people that maybe, um, aren't as deep into the business side of. Running a company or being a part of a big company where return on investment's a big deal. Matt, why don't you kick that one off for us? What is return on investment?'cause I know Izar likes to just come in and say same or basically flip around what you said completely. So, Matt, return on investment. What is it?

Matt Coles 2:18

Um, so actually, uh, I'm gonna defer back to you because I think you've got the, the, the broader business sense here. Uh, so I. I mean, I know generally what ROI is, right? It's a common term we use throughout the security world and, and obviously, uh, if you have any of the big name certifications, ROI must have come up in those. But in a, in a bare sense, it's, it's a cost benefit system, right? It's, it's a system of how much money do you spend versus what do you get in return. There's more to that. So I wanna throw it back to you actually,'cause I think you have a good sense of this. Obviously, you're very successful in, uh, in running a business. So you've done ROI and you've educated others on how to do ROI for security. So let me throw it back to you. Help us understand what ROI is is here in context.

Chris Romeo 3:08

Yeah. I think your, your definition is right on as far as it is, what's the value that's being generated for the dollars for the resources could be human resources as well that are being expended towards a given cause. And so when I think ROI, I think there's kind of startup ROI world, and then there's big corporate ROI world. And in the big corporate ROI. Often executives are looking for some type of metric of what am I getting for the investment that I'm making of dollars in people, security team. Executives don't care about how many alerts are generated, how many SAST findings there were, how many, uh, vulnerable third party packages we have. Executives don't think like that. Executives run businesses and return on investment is as simple as saying, you're asking me Izar to spend$500,000 and expend 500 people each a week of their time. What am I getting? What are you providing me as the result of me making this investment? And so it's really a business case conversation. It's, it's, are you providing me enough value as an executive where I can say, okay, I'll invest that money because I see what I'm gonna get out of it. That's really return on investment in, in the startup world, it's, it's a lot smaller'cause there's not as many people talking about it, but it is still important. Like it, when I was running Security Journey for example, we would be often have a decision in front of us and it would have a dollar cost and we'd have to, we'd have to wrestle with as a small group of executives to say, does that make sense? Should we move forward with this or do we hold off and wait another quarter? Or, you know, before we invest these dollars. So that's kind of my take on return on investment. Hopefully it, it helped Matt to, to provide some context from where I'm coming from.

Matt Coles 4:59

Oh, absolutely. Um, so if I could just, Put some thoughts in here and then is our, I mean, we're good. We'll just unleash the, unleash the Kraken Izar uh, so, uh, you know, just, you just keep drinking your monster over there and get, get pumped up for this. Uh, so if we think about cost and cost and benefit, right? How much we spend, obviously, you know, it's, it's people and time and te and tools and technology to not only do threat modeling, but also to manage the results. Right. So how many people do you need? Do you, do you need bring in consultants? Are you doing training ahead of time or does everyone know what they're doing? And then can you actually execute? But I would suggest at least the initial benefit of threat modeling is actually in terms of revenue protection and, and preventing rework because the outcomes of threat modeling are, uh, issues that you find that may influence the design that need to be fixed. You fix those upfront, that improves your design, but really what you're doing is preventing vulnerabilities and, and other things that will appear later in the development cycle and things that will get released to customers resulting in effectively revenue protection as your benefit.

Chris Romeo 6:20

Can I put my, my executive hat on for a second? Absolutely, please do. And we can have a little, we can have a little, uh, mock kind of approach to how this might actually go down in a business. Because when I hear what, basically what you just said, my immediate question to you then is, how are you gonna prove that to me? How are you gonna prove that there's less rework? What's your metric gonna be that's gonna drive this to allow me to say, okay, now Matt told me this and this and that, and things are gonna be better if we make this investment. How am I gonna measure you? What, what number are you gonna provide for me that lets me measure whether you were telling me the truth when you set this business case up for?

Matt Coles 6:58

That's a great question. I don't have necessarily have a good answer for you. Izar?

Izar Tarandach 7:06

Well, in the past five minutes, I have seen the past 20 years of my life run in front of my, my eyes. Right?

Matt Coles 7:14

Are we that old? my God. Yeah.

Izar Tarandach 7:17

Amazing. Least. So yeah, that time passes when you're having fun uhhuh. But, uh, you know, the, the, the thing is the the, when I was a young threat modeling padawan and somebody asked me, why should I do this thing? The question itself was unthinkable to me. Well, what do you mean what, what do you get if you do this? Don't you realize that if you don't do this, the whole world is going going to come crumbling down and the space time continuum is going to fold on itself and there's going to be one big black hole where your company was? Turns out that I was wrong and that there are people who will gladly go about their lives without doing that. And, uh, that conversation with Jim, for me, it was enlightening on, on, on a lot of different fields, but his insistence on the wasteful side of threat modeling showed me a, a couple of things. First, that unfortunately Jim needs to get better informed about the new ways of doing threat modeling that have flourished in the past few years. So where we, we, we have addressed a lot of that waste. But not only that, not not only the waste we have, we have addressed in general. The effort and what to take out of the, of the, uh, the process. And, and I think that today it's easy to say it, it's actually not easy. It's, it's clear to say that threat modeling has evolved to a point where we are not only identifying and mitigating and accepting risk through design weaknesses, we are going way beyond in terms of understanding what are the systems that we are building, which are forever getting more and more and more complicated. We, we, we are giving, uh, uh, transparency over these things that we are putting out there and that are responsible for so much of, sorry, what civilization is doing right? And we have extended into privacy. We have extended into so many different fields that deal with this whole experience that we live today. That the question to me, has to be turned around and say, how much are you going to lose if you don't? Especially with us coming to a point where people are starting to talk about, again, that word liability. When you, when you write, uh, a software and you put it out and the, the, the flag that the three of us have been bending about of what's reasonable security, is there anything more reasonable in terms of security than taking time to sit back and think what could possibly go wrong? If you have to to discuss, is this something that I want to do or not? I would say that you have a significantly more deep problem than what's the ROI on this thing.

Chris Romeo 10:16

Yeah, but remember I'm an executive, okay? And if you tell me that I need to somehow extract the value myself out of the beauty of threat modeling, I'm not gonna do that. Because I don't think, I'm not a security person. I'm a business person. I care about dollars and cents and bottom line, I have an, I have a number I gotta hit this quarter to make Wall Street happy. Yeah. So let me, let me add, that's what I care about. So that's, so that's why you gotta tell, you can't, you can't flip it back on me. I'm not gonna do it. I'm just, no way...

Matt Coles 10:46

can I just add

Chris Romeo 10:46

that's happening.

Matt Coles 10:47

I just have one go.

Izar Tarandach 10:49

Just before we go there, I, I have an unflinching admiration for people who are able to get a pot of water, throw all the data of their business inside, let it simmer for a while, and come out of something that they call a threat modeling a threat model. That's, that's great, especially if, uh, tomato paste on it. But, uh, uh, the, the, the, the point is that not everything translates down to dollars. At some point we have to move away from those numbers and say it's, it's, it's, it's a quality thing. It's not a quantity thing.

Chris Romeo 11:24

Everything boils down to dollars in business, the executive level. No, no. I'm still got my executive hat on.

Izar Tarandach 11:31

Yes.

Matt Coles 11:31

So let, let, let me introduce it.

Chris Romeo 11:34

Let Matt, let Matt get in here.

Matt Coles 11:36

I'm gonna, so I'm gonna, I wanna add something to the benefit column here and, and I think this will maybe talk a little bit closer to what you, with your executive hat on is thinking. So revenue protection is a little bit more, is a little amorphous. We can talk about things like reduced support calls. Better or improved customer customer engagement, improved customer acceptance. But Izar, you brought up something very particular. So we've seen the past couple years with the, uh, US uh, executive order around cybersecurity and the CISA, secure by Design, secure by default, uh, and the, the SSDF, uh, you know, secure software development framework from NIST and, and, and the collection of agencies across the world that are adopting those similar practices. If we look at threat modeling as a, an aspect of due diligence, and now I'm gonna bring in the legal aspect here, due diligence, the thing that you do to ensure that your software is free of, of easily discoverable, discoverable vulnerabilities, and so that you can meet regulatory compliance and legal expectations and obligations. Threat modeling becomes an aspect of due diligence. Due diligence drives regulatory compliance. Regulatory compliance means that you have revenue protection, and now you have a dollar and cents discussion. Does that, does that meet your smell test there, Mr. Executive?

Chris Romeo 12:58

I mean, I think you, I think you're, I think you're taking me on a, a pathway. I'm, I'm following you on the pathway, but I still need more. It's still too nebulous to say revenue protection and, um, due diligence and all of these things. They mean stuff, but there's no way I can measure it right? Now, where, where you were going with reduced support calls, reduced support costs. Now this is a metric I can look at. Because I'm not, we're not gonna roll threat modeling out in one week. We're not gonna say, next week is threat modeling week. Get ready everybody. Everybody's doing nothing else. We're not doing anything. We're not building any new features in our company. All we're doing is threat modeling, right? You're gonna roll this out over a period of time, 2, 3, 4 quarters. If we're talking large enterprise now, what if we pilot that as an executive? I'm gonna say, okay, Matt, I love this idea. I like where you're going here. Lower support costs and, and, and, and less rework. Let's do a pilot for a quarter with a particular business unit and collect your data and then come back to me and show me that you in fact have lower support costs and you have less rework and you, you know, all those things you can measure because I'm not gonna write you a check for$5 million on day one. I'll write you a check for, for$500,000 on day one. If I believe in the idea and I think you got something, but I can't roll it out enterprise wide without a, without a proof of concept that shows me your data backs up what you were telling me and Izar's about to fly out his chair.

Izar Tarandach 14:28

No, but, but, but, but Chris, think about it. Okay? What you're telling me is you, Mr. Executive, are willing to write a check for DAST because DAST gives you numbers at the end. Threat modeling doesn't.

Chris Romeo 14:48

Can I take my hat off for a second and just yell into the microphone? No, I mean, I'm, I'm gonna.

Matt Coles 14:53

You could have chosen of all, any other letter, but you had to choose DAST.

Izar Tarandach 14:56

No, no, I, I, I went there.

Chris Romeo 14:59

But listen, as an executive, I don't know what DAST is. If I'm a COO, for example, I probably don't know what DAST is. I don't care what DAST is. Rights. I care about security, findings. I care about, um, improvements that you're making. I care about the metrics that I can go to the board with and say, when the board looks at me and says, Hey, COO is, is cybersecurity getting better? The same, staying the same, or is it getting worse? I need to be able to make the, I need to be able to say, oh, we're getting better and here's why,

Izar Tarandach 15:27

Chris, that, that's the thing again. And, and, uh, I'm, I'm guessing that I'm going to throw like the unpopular opinion of today. Those metrics, the, the metrics that people tend to use today, and, and I I, I got surprised by, by what you said back in Jim's, uh, uh, episode, they don't really mean anything. The number of vulnerabilities that you're going to, the number of weaknesses that you're going to identify at the end of threat modeling, uh, session don't mean anything.

Chris Romeo 15:55

Mm-hmm.

Izar Tarandach 15:56

Because there are so many different factors impacting that.

Chris Romeo 16:02

Okay. So let's take...

Izar Tarandach 16:03

What's meaningful is what's the coverage of the, the system that you are threat modeling. What's meaningful is what are the threats that you are evaluating? What's meaningful is...

Chris Romeo 16:11

What's the mitigations?

Izar Tarandach 16:13

The mitigations, if any. What are the developers doing with what they're learning? Are they learning something? Are you accepting risk? And why are you accepting that risk? Those are all intangibles that you can explain. You can tell a story of risk to an executive.

Chris Romeo 16:30

Executives definitely speak risk, 100 percent.

Izar Tarandach 16:32

But you are not, but that risk is not going to come in the in, in the, in the language of numbers and three callers.

Chris Romeo 16:40

Oh, it has to.'cause I don't speak anything else as an executive. You can't, but you can't explain to me why something is a high critical finding, because I'm not gonna understand the technical... Some I'm, I'm being, I'm being unfair here, I'm being stereotypical. Let me say, most executives are not gonna follow you on a journey of why something is such a big problem that you found and mitigated.

Izar Tarandach 17:00

Look, I, I can use a laser printer to print the most high-def report ever. Or I can go in front of the board with a set of crayons and explain to them. In the language that they need to understand, not that they want or that they can't, that they need to understand because they're dealing with a thousand other very, very complicated and important factors.

Chris Romeo 17:21

Mm-hmm.

Izar Tarandach 17:22

I can use my three crayons to say this is where we are today in terms of risk. Inot everything needs to come in terms of numbers.

Chris Romeo 17:29

Yeah. But that's how I, to how, how, how do you, how do I generate a report? I have to, I have to, you know, there's this thing called Sarbanes Oxley, which I hate the fact that I know what this is, but I have to, I have to, as an executive. I have to sign a document and send it to the United States government that says that you, how you explain the risk to me is something that we actually are, we're... I'm, I'm putting my freedom and my livelihood and all of my money on the line. So I'm not gonna let you come in there and draw me a crayon picture.'Cause I'm gonna say Izar, are you gonna write me a check if I, if, if they come to put me in jail and take all my money away and sue me for lying on this because of your, crayon picture?

Izar Tarandach 18:12

But now we are talking two different approaches. We, we have the risk and governance people doing the amazing work that they do. That's, Lord knows if I understand that, that they can put those things in terms that SOCs understands and SOCs receives and, and accepts, and me as an, uh, uh, I won't even say as an AppSec person, but as, as a security person to come and say, listen, this is where our security posture is today. This is where it was, yeah. A month ago, and these are the things that I'm going to do in this month so that we...

Chris Romeo 18:41

mm-hmm.

Izar Tarandach 18:41

We're better next month and this is how I translate risk. Okay.

Chris Romeo 18:45

Now we've morphed the conversation though. We went from Matt making an investment, conver having an investment conversation with me about rolling out threat modeling and moving it out into an organization where I'm gonna write a check from my budget to a risk and compliance.

Izar Tarandach 19:00

No, no, no. We we're still at the same place. We're still at the same place.

Chris Romeo 19:03

There's two different things,'cause if Matt wants me to invest, that's a different conversation than what am I gonna sign on the Sarbanes Oxley report.

Izar Tarandach 19:10

Look, we're in this, I, I think that we are still in the same place because right now what we are working on is on the pitch. That we're going to come to the board and say, I need time and I need money to do threat modeling.

Chris Romeo 19:21

Okay.

Izar Tarandach 19:21

'Cause I'll be able to express that risk in these ways that are not the Sarbanes Oxley numbers, but that are going to give you an understanding of what's the risk and the residual risk and the things that we are doing to lower that risk across the organization. Just because we are going to take the time to sit back and think what could go wrong.

Chris Romeo 19:40

Okay.

Izar Tarandach 19:40

Right.

Chris Romeo 19:41

Alright, I'll, I'll play along. What, what are you gonna give me?

Izar Tarandach 19:45

I'm gonna give look at visibility.

Chris Romeo 19:47

That I can understand. No, visibility is a, is a descriptor. What, what am I, what, what am I gonna have visibility of? What's the subject that I'm going to gain?

Izar Tarandach 19:58

Your actual attack surface and what you're doing about it.

Chris Romeo 20:01

Uh, what is an attack surface?

Izar Tarandach 20:04

How exposed you are.

Chris Romeo 20:08

Exposed to what?

Izar Tarandach 20:09

Everything.

Chris Romeo 20:09

I'm playing executive here. Everything?

Izar Tarandach 20:11

Anybody who comes and tries to take a bite out of you, we are going to put together threat modeling, threat intelligence, threat everything.

Chris Romeo 20:18

So you're gonna send me a pile of threat models and threat intelligence reports?

Izar Tarandach 20:23

No. I'm going to tell you a story.

Chris Romeo 20:25

Okay, so what are you gonna, what's the tell story? What, what's gonna be the... I'm kind of walking into a corner here. What's gonna be the, what's gonna be the backbone of that story that you tell me?

Izar Tarandach 20:33

Backbone of that story?

Chris Romeo 20:34

It's gonna be anecdotes.'cause I can't, I can't go to court with anecdotes. No. Our lawyer, our legal team will not support anecdotes in court.

Izar Tarandach 20:40

It's, it's observation, observational. It's, okay, we know that this is the things that we are defending.

Chris Romeo 20:46

Okay.

Izar Tarandach 20:46

We know that these are the things that are trying to attack it because of A, B, C, D, E.

Chris Romeo 20:51

How will I know...,

Izar Tarandach 20:52

These are the reasons why they would.

Chris Romeo 20:54

How will I measure what the most important things are so that I can explain it to our legal counsel?

Izar Tarandach 21:04

So

Chris Romeo 21:07

I know the answer to my question, but I just want you to say it.

Izar Tarandach 21:09

No, no. The, the, the dance, the dance here goes from risk...ification to, to, to, uh, to prioritization, right? And over time I came to understand it to me at least personally. Those are two different things.

Chris Romeo 21:25

Yeah.

Izar Tarandach 21:25

One thing is to say how much risk you run there and the other say how much you're going to prioritize whatever fix needs to come first.

Chris Romeo 21:30

But my point is, you're gonna have to give me data. You're gonna have to gimme metrics.

Izar Tarandach 21:34

But my point is that that data doesn't come in numbers always. That data can be expressed in different ways.

Chris Romeo 21:40

Okay, I see where you're going.

Izar Tarandach 21:41

It still tells us the same story.

Chris Romeo 21:42

You can gimme yes, you can gimme a red, yellow, green. That's a, that's a fine thing that happens in, in these conversations all the time.

Izar Tarandach 21:48

But not only that, I can tell you where we were last month. So I can give you a trend.

Chris Romeo 21:52

Yeah, I want trend, but I really want trends that are, as an executive, I don't wanna know. Red, yellow, green, right? I want some more data because I am technically savvy. I do understand how things work and I want to see a trend line. I wanna see if how we're getting better. So I want you to gimme a score for on a per product or application basis for last quarter and this quarter. And I wanna look at those numbers and I wanna see those numbers trending up.'cause if I see those numbers trending down, we've got problems.

Izar Tarandach 22:22

So it it, it's the difference between intel... giving somebody an intelligence analysis and giving them the raw intelligence. Okay. You, you're leaving them to do their own analysis. If I give you a bunch of indicators, numbers that you decided, because me as a, as a security professional with experience, I may have decided as, as I have done that, many of these numbers actually don't say anything. They're just numbers. Okay. You can go and, and, and build your story. And perhaps your story is different from mine.

Chris Romeo 22:51

Mm-hmm.

Izar Tarandach 22:52

Because mine comes with an interpretation. Comes with an analysis.

Chris Romeo 22:56

Mm-hmm.

Izar Tarandach 22:56

Coming, comes with an understanding of what's happening out there in the business.

Chris Romeo 22:59

And over time, I'm gonna come to trust your analysis more. The first time you deliver it to me, I'm not gonna trust your analysis very much.'cause once again, I'm the one whose butt's on the line.

Izar Tarandach 23:10

Yep.

Chris Romeo 23:10

If, if, if what you told me is not correct, they're coming for me. They're not coming for you. I might try to come for you after that, but for me, because I'm the one who wrote the signature on the line of the reports that went to the federal government and got filed with the stock exchange and all of those things, right? And so over time, I'm gonna come to trust you more as a, like, if you're my CISO for example, you're gonna, I'm gonna, you're, I'm gonna start building tru... My level trust level's gonna go up over time too. I'm gonna get to the point where I'm like, whatever Izar tells me is gold because I trust him. And I've had, I've looked at some of the data enough to know, How he's drawing his conclusions. Executives are smarter than I'm giving them credit for here. Right? Like they can look at the raw data and they didn't get to be an executive because all they can do is summarize and...

Izar Tarandach 23:57

Look when you go to a new doctor, okay, you, you have a choice. You can decide to implicitly trust them because they are a doctor, or you can say he better prove himself to be first or herself or themselves. Mm-hmm. Okay. It's the same thing. You, you go to any kind of expert, you either implicitly accept the authority or you say, this person has to prove themselves to me.

Chris Romeo 24:21

I mean, everybody... there, there's always one doctor who graduated at the bottom of their class.

Izar Tarandach 24:26

Yep.

Chris Romeo 24:27

Don't forget that when

Izar Tarandach 24:27

it doesn't mean, doesn't mean that he's, doesn't mean that he is, uh, uh, uh, less of a doctor because there are billions of people who didn't go to to that class at all. Somebody has to be the last.

Matt Coles 24:38

So let's just be careful here. We're not talking apples to apples comparison, right? A doctor is like a consultant versus an employee. Like the board has an engineering team that hired somebody to be a, an expert here, right? They're not asking a third party. They're asking,

Izar Tarandach 24:57

no, let, let's go with another one. You have a lawyer on retainer, you're paying the retainer, but the first time that you're using them, you, you have to make a, a qualitative decision, are you going to trust them as is or are they going to have to prove themselves?

Chris Romeo 25:10

I mean, every time I work, even as a small business owner, every time I, I, I, I don't just implicitly accept what my lawyer says. I think about it for a second and go, okay, yeah, okay. I can follow that logic. I don't just say, because you're a lawyer, I'm gonna do exactly what you told me to do,

Izar Tarandach 25:27

but you're not going to him and saying, give me a list of the precedent so that I can go case over case and decide if your line of, uh, reasoning is the right one or not.

Chris Romeo 25:36

That is true as well.

Izar Tarandach 25:38

You, you, you, you do an informed decision. There's a difference between an informed decision and what's the name of the thing? Uh, back, uh, backseat, uh, driving. Uh,

Chris Romeo 25:48

Yeah.

Izar Tarandach 25:49

There's a difference between those two things.

Chris Romeo 25:50

I mean, so first of all, if I'm a big company executive, I have the, there's, we have our own legal team, and those lawyers are technically on the hook just like I am to some degree, right? They're carrying some liability based on the things that they're telling me. So it's, it's not quite as, as easy as it's an outside counsel and, and there.

Matt Coles 26:13

But likewise, so is your, so is your CISO or your VP of engineering who are communicating...

Chris Romeo 26:19

mm-hmm.

Matt Coles 26:20

...around, let's bring it back to threat modeling, right? If I'm doing threat modeling for cybersecurity and, and or want to do that and delivering information as a CISO to the board. The CISO is in the same boat as the board if something, if things go south.

Chris Romeo 26:35

Yeah.

Matt Coles 26:35

Right.

Chris Romeo 26:35

I mean, we saw with Uber, right? Right. We saw the CISO get brought up on charges. Now that was a little, I'm not gonna comment.

Matt Coles 26:43

That was, that was extreme.

Chris Romeo 26:44

Read the news stories. There was a little more moving parts to that as to who said what and who did what and whatnot. Right.

Izar Tarandach 26:51

Right. But people constantly claim, I did it to the best of my abilities. You can't expect more than that from me.

Matt Coles 26:56

That's the due diligence part, right?

Izar Tarandach 26:57

I did. I did it as well as I could. What I did was reasonable. Right? Yeah. Now, to bring that back to the ROI of threat modeling, if we consider that this begins at the, just because there, there should be, there must be an hierarchy. I'm going to call it like that, the lowest levels of the rung. Okay. Then it floats up and it floats up and it floats up and it brings that, that picture of your attack surface, the risk you under the, the residual risk, all that good stuff over time. That picture is bound to not only if you do everything right, to not only get more clear and more visible, which is not always the case, and we know that very well. But it's going to improve as well, because as we have said, X number of times, threat modeling is evolutionary. People start sucking at it. I sucked at it. I like to think that I got better at it over time. Right? Yeah.

Matt Coles 27:54

Yeah.

Izar Tarandach 27:54

So the important thing here is that the, the return of investment here is, is again, And analog to the, the, the, the training saying like, what happens if we train them and they leave? What happens if we don't train them and they stay?

Chris Romeo 28:11

Mm-hmm.

Izar Tarandach 28:12

So what happens if we threat model and we figure everything out? What happens if we don't threat model and somebody else is going to tell us what we forgot?

Matt Coles 28:20

Somebody else will figure it out.

Chris Romeo 28:22

Somebody else will threat model for us.

Izar Tarandach 28:23

Yeah.

Chris Romeo 28:24

Alright, so we don't, we only have a few minutes left. Lemme take my executive hat off.

Matt Coles 28:28

Oh, I actually had one other thing for the executive.

Chris Romeo 28:30

Oh, oh, hold on, I'll put my executive hat. Hold on. I can just pick it up.

Izar Tarandach 28:32

He, he wants a raise. He wants a raise.

Chris Romeo 28:34

Alright, Matt, I'm back as executive.

Matt Coles 28:35

So benefit and, and the last benefit, and I'm gonna just drop it out there, quick comments if you want. It's not revenue protection, but it's definitely revenue generating at some point, threat modeling along with other security activities in the lifecycle will be a barrier to sales. Right? We already see this with the, with the CISA attestation for the federal government, if you're selling to the federal government or your critical clinical infrastructure, you have to develop the attestation form, which means you've done some amount of security. Now, the threat modeling is actually is as we know, not part of that directly, but at some point that likely is, is likely to to be introduced.

Chris Romeo 29:19

Mm-hmm.

Matt Coles 29:19

In which case not doing it isn't just revenue protection, meaning reducing my risk. But now it is directly enabling sales to occur because I now have met the criteria for procurement.

Chris Romeo 29:32

I mean,

Izar Tarandach 29:33

and my last line, my last line. Of all the activities in the SDLC threat modeling is the one that improves all the others. I, I, I said that many times, and I'll say it again. You can use, use it as a hanger to put all the other activities, hang it on, on, on it, and they will be better. If you have a good threat model, your security testing is going to be better.

Chris Romeo 29:58

Mm-hmm.

Izar Tarandach 29:59

If you have good threat modeling, your, your secure implementation is going to be better.

Matt Coles 30:03

And your vulnerability response will be better. Your, RCA exercise will be better.

Chris Romeo 30:07

Yeah.

Izar Tarandach 30:08

As a return of investment, you are multiplying the efficiency of all the other things that you do, including best.

Chris Romeo 30:15

You gotta, you gotta prove that to me though, like, that's my point though, is you need, you gotta gimme data. You can't just tell me it's improving it. Like you can't come to a meeting with an executive and say, well, threat modeling is improving all these things. And then just stop. Because they're gonna say, okay, how, let's, how is it improving? Gimme some data. Let me, let me see how you meas..., how you drew that conclusion.

Izar Tarandach 30:36

Let's get two teams and threat model on one and not threat model on the other. And use the artifacts of the threat model the right way on one and not do it on the other.

Chris Romeo 30:45

Yeah.

Izar Tarandach 30:45

And then let's compare the overall happiness of the developers.

Chris Romeo 30:48

I mean, I think, I mean, listen, I'm an executive. I don't care about the happiness of developers. I care about how much...

Izar Tarandach 30:53

As much as it hurts, I hear you.

Chris Romeo 30:56

I know I'm being, this is the raw version. This isn't me as the, you know....This is what, no, when, when, I wouldn't really say that.

Izar Tarandach 31:01

But when I say the happiness of the employees, and, and this is my closing statement. I go back to what I have thought to myself is the right way of going about the return of investment of threat modeling. It's asking the people who are involved in the process, would you do it again? But that's, that, that's way, that's way under the level of the executive hat that you are wearing.

Chris Romeo 31:23

I mean? That's,

Izar Tarandach 31:24

that's the people who are actually doing the thing.

Chris Romeo 31:26

If you collected that data for me though, and you showed me, Hey, with our pilot group, we did a, we had a business unit, we had everybody threat model for a quarter. And the funny thing is, with an NPS style survey, we, we averaged 8.975. Meaning I don't, people were promoters of this. Or you could just say, well, you could do a binary, well, you do it again. Um,

Izar Tarandach 31:49

I don't survey,

Chris Romeo 31:50

but the, but, but if you did though, you could then show me. Now, isn't it interesting that almost a hundred per or 90% of people that did this threat modeling process said they would do it again? Because they see the value. Now we're talking about, now we've got data. Now I can go, well, Matt, maybe with your little rollup, maybe we should do three business units. Let's roll this thing up to three business units now. Yeah. And and that's data that would stand behind. Yeah. See how, and, and I can, and then when the board looks at me and says, Mr. COO, why did you raise, why did your, you need to increase your budget by this amount? Well, because we did a pilot with this thing. We had really good results. We have some data to back it up. We think it's really gonna do the things Matt was talking about here. It's gonna introduce opportunities for new revenue. It's gonna protect old revenue. Um, we made this investment. Here's what we saw from the data. So now we're gonna invest in three. We're gonna roll this out to three business units. You know what? The board may come back and say, you know what? Why don't you do that with five? Let's increase the budget a little bit because we like the, we like the trend line of this. It's improving our cybersecurity story. So there, there's where data enables you to kind of make, make things work. And since we're all in agreement, that'll be the end of the security table for this week. Thanks Izar. Thanks Matt. That was a great, uh, dialogue. I will take my executive hat off. Set it on the table over here so I can back, go back to being normal. Uh, just kidding executives out there. I'm not. I'm not, I'm just, I'm just poking fun and, and, uh, I was being somewhat stereotypical in the average, uh, executive. I understand lots of executives are different levels of technical knowledge and everything else. And so, um, don't take offense. It wasn't intended. I'm just, we were trying to, we were trying to reflect a, a conversation on, on how things would actually be thought...

Izar Tarandach 33:31

Spoken like a true executive.

Chris Romeo 33:36

All right. Thanks everybody. Thanks for joining this episode.