Capture the Flag or NOT?

There is an overemphasis on Capture The Flag in the security world. Instead, the industry should focus more on the 'builder' perspective to develop robust systems rather than the 'breaker' mindset typically associated with penetration testing and CTF competitions. In addition, we must shift the industry's reward and recognition structures to incentivize building secure-by-design systems.

A CTF is a type of cybersecurity competition where participants solve security-related challenges to find flags representing vulnerabilities or secrets within a system. A CTF and bug bounty are similar, as both test cybersecurity skills but have different goals and outcomes.

Red teaming is not just about penetration testing but also about testing the operations of the people who manage defenses. 

Finally, the discussion ends with pondering the question of "winning" in cybersecurity and agreeing that providing a system free of defects and ensuring security assurance should be the ultimate goal.

FOLLOW OUR SOCIAL MEDIA:

➜ X: @SecTablePodcast
➜ LinkedIn: The Security Table Podcast
➜ YouTube: The Security Table YouTube Channel

Thanks for Listening!

Transcript

Copy Transcript Copied

Chris Romeo 0:10

Hey folks. Welcome to another episode of the Security Table, we sometimes debate the philosophy behind the matrix. Was it the red pill? Was it the blue pill? But sometimes we talk about application security and other related things too. And so I feel like we're, we're still on this, this thread. Oh, who are we? Who are we? Who is anybody really? So I'm Chris Romeo, joined by my friends Matt Coles and Izar Tarandash. today, gonna continue pulling on that thread of what we affectionately referred to as reasonable application security. I think there's a, a connection with what we want to talk today, talk about today, but I think it's, it's also much bigger. And so the premise in front of us was really an image that, that I saw somebody tweet and it was in regards to capture the flag. And so this individual who I should have been more prepared and ready to quote, give attribution to

Matt Coles 1:05

What else is new? I mean, come on,

Chris Romeo 1:07

So what else? We we're known at the security table for being very unprepared. Um, he, he, he basically just had this premise of capture. The flag is overrated. It's overhyped, it doesn't generate Doesn't help you generate CBEs. I thought it was a pretty interesting construct and it's gonna, it's gonna allow me to stand on a soapbox a little bit later and talk about what I think about red teaming and pen testing and the focus we give in the industry.

Izar Tarandach 1:38

Ooh,

Chris Romeo 1:39

let's, let's unpack the CTF question first. So, so you guys te tell me I'm not a CTF'er. Okay. I've never played a CTF, so I feel like I'm only qualified to share my opinions of it, not necessarily describe what it is. So, I mean, so let, let's work up. What is it? What is a CTF?

Izar Tarandach 1:57

Matt might you you go; you're good with the definitions.

Chris Romeo 2:02

Do you have a

Matt Coles 2:03

Great. great. the,

Chris Romeo 2:04

with CTF written on it and a definition?

Matt Coles 2:08

well, I, I, I, I do, I do have a flag with his on it. Uh,

Izar Tarandach 2:14

Consider it captured.

Matt Coles 2:16

So, so, so, yeah. Great. I feel I'm, I'm Mr. Dictionary over here. Great. Um, so CTF, right? Capture the flag. Uh, Competition for people to get together and, um, break stuff. Uh, well, not so much break stuff as find stuff, I guess is the important part, right? So we're talking about a structured competition, uh, with a defined goal in mind or set of goals, maybe some milestones along the way. Uh, and, and people, uh, get together and. And ultimately it's a competition, right? So, so there's, there's one objective, you know, a piece of data or, or a control of a system or whatever. Uh, and obviously we're talking, uh, we're talking se security here. There's other capture of flags. You know, if you're, if you're a gamer, you might be going for that literally a flag, uh, on the screen or you're doing paintball or something. Very similar concept. Uh, hopefully folks have that pretty clear in their mind. You're looking for something, you're trying to get it before the other team does, uh, capture it. Hold it. Destroy it. Do whatever you need to do with it. And, and, winter, winter, winter, chicken dinner,

Chris Romeo 3:25

I love a good chicken dinner. So Eza, what else? Like what else?

Izar Tarandach 3:28

So that that's.

Chris Romeo 3:29

provide?

Izar Tarandach 3:29

Yeah, so, so that's, that's some of the fun. Of course there, there's a whole bunch of, uh, cts that happen either like individually, you go and you, you have a specific set of problems that progress usually in, uh, difficulty. for each one of them, you get to that string. That is the flag. You apply to the interface, you get the next problem. And, um, We have seen some of those also applied in companies as an educational tool or in hackathons, internal hackathons. They, they're very, uh, popular for that, that kind of thing. And stepping away from the definition and, and going into the is this good for, is I think why we're here I to me, okay. First of all, uh, the competition part, I think that, uh, a lot of people that are in this professional are interested in the, the things that we deal with. Uh, are competitive by nature. So are good ways of, uh, pitting what I know against what you know, and coming out of something that's, uh, quantifiable out of it. Uh, if I have more flags than you, if I have more points than you, then uh, I win and you lose. And, uh, I'll enjoy your dinner and dinner, but, uh, what I like to question is the educational point, point of view, right? Cause. What, what would be to, in my, my head, what, what's the closest thing that, uh, you could parallel uh, uh, CTF? To me, it would be bounty, right? But the difference between both of them is that the CTF, by the definition can be exploited, and there is something at the end of the rainbow. So it's like the CTF is a, is a sure thing. The bug bounty is you go in and you apply the things that perhaps you learned even in the CTF and happen, may not happen. You don't know you're going blind. can it. Own experience. Do I know what I'm doing here? And that, that

Matt Coles 5:29

So or would you. Sorry, would you, so, uh, what you're calling others that CTF has a, has is a structure, is a structured activity that has definitive goals and milestones that are achievable. Right? That's, that's what you're ultimately saying here is a CTF. A CTF is a structure for deli with content that is known, that tests for skills as opposed to bug bo, which is content that is unknown. Testing for the, like what flags do we have? Not how do you get to the flags we know of?

Izar Tarandach 6:02

Actually, I am more simplistic than that. I'm saying going to the CTF. You gotta, you gotta guaranteed dopamine hit if you do A, B, C, D and you go into the bug bug, uh, bug bounty and you don't know what's gonna come out of it. Perhaps you find something and perhaps you won't. Right. So I, I, I, I do see the parallel between both of them. I don't think that they, working on one is, is a good ramp up to the other.

Matt Coles 6:36

Would you, would you characterize, would you say though that a bug bounty is a, is first and foremost the test of a system, whereas a CTF is a test of skills?

Izar Tarandach 6:48

Oh, that's a beautiful way of putting it. So you you mean that the, the target of the exercise is different.

Matt Coles 6:57

The outcome is different, right? Yes, you need skills to do a bug bounty, but you don't know what the target is. You don't know what that flag is when you do a bug bounty. So you have to have skill. You're testing skills, but you're really looking for the, the goal of running a bug bounty is not to know whether an attacker is good, but to know whether they find, find the flags, right. Find something, and, and again, those flags are. Unknown at the time. Whereas the CTF, all the flags and all the milestones on all the path through the system are known to the organizers. At the very least, you're testing the skills and, and potentially the collaboration and, and, um, and inner working, you know, interpersonal, uh, and, and technical operation of a, of an individual or a team to meet, to, to get to those objectives and milestones and, and the flag ultimately to the flags.

Izar Tarandach 7:45

Okay,

Matt Coles 7:45

So the goal of the, the goal of the exercise is different.

Izar Tarandach 7:48

so analogy time, it sounds to me like you're saying that the CTF is like a shooting range you have like very controlled situations, very controlled limits, and you are basically. Checking out if you know how to a weapon, how, how to shoot in a target,

Matt Coles 8:08

Mm-hmm.

Izar Tarandach 8:08

the bug body is closer to being out there and doing those things that people who do things with weapons do. You have all kinds of, uh, unknowns and, uh, the, the boundaries are not known. The, the, the terrain is not known. Who's shooting back at you is not known. So all that, and then of course, it extrapolates to bad people doing bad, bad things, which is actually the, the word of attackers attacking our applications. So three, three here of indirection. Yeah.

Chris Romeo 8:36

Let's give it a, let's take it a step further though, and talk about outcomes of each one.

Matt Coles 8:41

Mm-hmm.

Chris Romeo 8:41

about, when you're putting bug bounty against CTF, the, the outcome that's most, I think most people are aiming towards is enhancing their knowledge and the CTF, they to break stuff better so that they can apply it to breaking stuff in wherever else they break stuff. The bug bounty, the outcomes dollars in your pocket. So similar path, but. Bug Bounty. So I'm thinking about Matt Bug Bounty from the perspective of the person who's doing it, not the, organization who's

Matt Coles 9:12

there's, there's two. There's two aspects there that to consider. I think you're, yeah,

Chris Romeo 9:15

real CTF, like there's no real organizational value. I guess there could be organizational CTF, um, of, of enhancing the knowledge of the people that work in your company.

Matt Coles 9:25

well, I would add.

Chris Romeo 9:26

individual first.

Matt Coles 9:28

Oh, well, I, I, I would actually, uh, just you, I think you covered the bug bounty and the CTF from the goal of the participants pretty well, right? A test of skills. It's a test of skill. It is a test of skills, and there's money associated probably in both ends, right? There's, there's either fame and fortune from, and participating in winning a CTF and you gain, gain skills or you show off your skills, right? You either get bragging rights or you get a payout or you get a promotion, you get whatever. same thing for the bug bounty, right? Although that's more of a financial impact, or maybe it's swag or maybe it's a leaderboard or whatever the case may be. So for the participant, you get, you get tangible benefits from either of those activities and very similar tangible benefits. The bug bounty from a organizer's standpoint, the bug bounty is I get somebody else to do testing for me that I then re, uh, compensate them for. And so I learned better about my system and I learned then, uh, what, um, Uh, you know, I, I, I, I can improve that. A CTF, though, I think. is really from a, from an organizer's standpoint, if it's a corporate organization organizing it with their employees, they're testing their employees abilities. Maybe it's a team building exercise. So they get, you know, some interpersonal connection there. They get to collaborate, they get to test out processes. We can talk later about red team versus Blue team, cuz there's another area to go into. Uh, Or if it's a organization that does it like the us um, as a US CTF. Uh, I, I'll have to pull it cuz it was, that was at Defcon last year run by a good friend of mine. Uh, or, or, or, uh, at least mcd by a good friend of mine, uh, uh, who, who, um, this was, you know, bring teams together. They do CTF, um, they learn about new attack techniques, right? So it's a chance to show off new industry trends or learn about new attack behavior or, um, or, you know, show off, show off in a different way, right? So there are tangible benefits across the board. Not all of them are financial

Chris Romeo 11:32

So the original of Twitter post tweet got me to this, I just said Twitter post, I'm so embarrassed, uh, that got us to this conversation. Was an image with the header at the

Izar Tarandach 11:45

Facebook posted

Chris Romeo 11:47

right. Um, I put it on the Facebook. Um,

Matt Coles 11:51

You didn't ticket or talk it

Chris Romeo 11:53

I didn't, I don't understand that. I don't, I don't even want to go there. Matt, you, you almost, you almost open a can of worms suggesting that I would be on TikTok now. I'm not

Matt Coles 12:01

I,

Chris Romeo 12:01

of worms. That's,

Matt Coles 12:02

disclaimer, I'm not on TikTok either, so

Chris Romeo 12:05

that's a whole other, that's a whole other discussion, debate as far as where that data goes. But the

Izar Tarandach 12:09

The security table dance.

Chris Romeo 12:11

That this tweeter did was stop doing CTF security was not meant for competitions years of CTFs yet. No real world CBEs found. You want to a hack something? Don't use CF ti time use this. And then it's an analog modem with the, with the, so I don't know if it's a, if, if this is just purely a trolling that I just fell for, or, but it does bring this issue up like, are we too focused on CTF in the world of security and we're going outside AppSec. Now we're talking about security in general. Do we focus too much on CTF?

Izar Tarandach 12:48

Okay. Oh,

Matt Coles 12:50

Go Go ahead. Ard. You first.

Izar Tarandach 12:52

Last time that I participated in CTF was like, what, two years ago? And for one task, like the, the one that the points were like absurdly bigger than everything else. And it ended up like was one binary that had something like eight different layers of, of. Garbling and obscurity on top of it, and ended up going binary in one language, encapsulating another one, and so on and so forth until the flag, and know what it, it has absolutely nothing to do with security. It was more about reverse engineering and, and figuring out some very basic crypto, but I didn't care because it, it kept happening and the dopamine kept hitting and I kept solving a problem. and I could not give less of a care about am I having a security experience, am I learning more? Because it felt good.

Chris Romeo 13:51

Hmm.

Izar Tarandach 13:51

And I think, and my point is that, uh, we, we, we already have to fight so much when, when we are teaching, when we are pushing, when we are, uh, building these programs, we, we have to fight so much for people to get interested in it and for people to give you their time and, and their attention. and I think that that dopamine hit is, is a positive. If I can get somebody at least minimally interested in some aspect of security using that mechanism, I'll freaking use it.

Chris Romeo 14:20

but how practical, like you just described something that's never gonna happen in a modern web application architecture

Izar Tarandach 14:26

Oh, no, no, no, no. That, that, that was like that, that was just illustrate effect that I got so engrossed in into it. Right.

Chris Romeo 14:32

but

Matt Coles 14:33

And,

Chris Romeo 14:33

something that doesn't, that has no real world applicability though? Or is there some real world case case for this architecture

Matt Coles 14:41

You're think you're think you're thinking about it wrong. I think we're thinking about it wrong, and maybe that's, maybe that's, we're putting a lot, we talked about it. We, we just talked about it actually. CTF versus bug grati. Right. The goal of the exercise is different. We're not in a CTF. We're not looking for new vulnerabilities. We're not looking at necessarily inventing new attack techniques, although that probably happens in a CTF probably pretty frequently. But that's not the, maybe that's not the outcome that we should be asking ourselves about, right? If the, if the, if the statement is CTFs don't produce any vulnerabilities, therefore they are bad. Meaning their waste of time because they don't produce vulnerabilities. I would suggest that their as, that the, the goal, the expected outcome is the wrong outcome.

Chris Romeo 15:28

Hmm.

Matt Coles 15:29

it's not a waste of time if it's a team building exercise. It's not a waste of time. If it's a opportunity to get people interested in the field of cybersecurity, it's not a waste of time if it gets people thinking about, you know, new, new attack techniques or new memory defenses or new coding constructs or other things, you know, that are non-tangible benefits that come out of or are, you know, not, not directly related to the CTF benefits. Uh, and therefore it's not a waste of time. If you're looking to find defects, well, that's a bug bounty,

Izar Tarandach 16:01

Mm-hmm.

Matt Coles 16:01

right? If you want to do cvs, c cvs, that's a bug bounty. We don't know what the target is. When you construct a CTF, you plant a flag. You know, that's a, that's a target, that's a piece of data. And you say, oh, that data is protected by these things. And well, in order to get through that, somebody's gonna have to know how to do a memory injection and, and, and a heap, heap, heap overflow, and, uh, you know, route to, you know, return to lipps and all this other stuff. Well, They may find a way around those, those attacks and do something completely different, but that's not the outcome you're looking for. The outcome is basics or a particular path or a particular goal in.

Chris Romeo 16:40

I'm somebody who could care less about CVEs. And it's not just because I don't have any, let the record show. I just, I don't, I mean, I don't, I don't measure my worth in the security industry based on the number of mire, cve, e entries that exist in the database. But

Matt Coles 16:56

but we are not pen testers.

Chris Romeo 16:58

but I think when I was about ready to go. I'm gonna go there in that direction though, right? Like, I don't consider myself a breaker. Using the classic owas terminology, builder breaker, defender.

Matt Coles 17:10

Mm-hmm.

Chris Romeo 17:10

consider myself a breaker or really, I mean, we're really a defender. I, I consider myself a builder and as a builder I think about how can we spend our time better so that we can build better stuff that doesn't have the problems. Like if we can build something that doesn't have any of the issues, tech, they'll have nothing to break. There'll be nothing to find if we, if we, and, and so this is, I'm, I'm kind of, I'm about ready to unleash my, um, soapbox moment here because, and I've written about this before. I've talked about it publicly, and it's an unpopular opinion. Get ready to put up the unpopular opinion flag. I don't care. In our industry, we focus too much on breaking, we focus too much on red teaming, too much on CTFs. Um, these guys are all writing the resignation letters to the security table right now, and so it's okay,

Izar Tarandach 18:01

Uh,

Chris Romeo 18:02

but we spend, we spend too much time. Like, and, and all you gotta do is go walk up. So find a, a kid who's, who's a kid, I shouldn't say that. A student, a young, a, a younger person who's in university, getting ready to go into cybersecurity and ask them, what's the number one thing you wanna do? And what do they all say? I

Matt Coles 18:21

Break stuff.

Chris Romeo 18:22

secure Java code. eliminate cross. No, they all say they wanna break things and so as an

Izar Tarandach 18:29

and move fest.

Chris Romeo 18:31

and move fast. That's fine. Move fast. That's me, That's my soapbox moment. Thank you. That is an

Matt Coles 18:35

is.

Chris Romeo 18:36

excellent graphic. CBEs. Who cares? I feel like I'm like, I'm the get off my lawn guy when it comes to, you know, that's, that's my new persona I'm taking on here. But, We spend too much What is this? Scary? What is it? Good Four? No, I mean, we spend too much time focused on it. Like, if we could take half of the effort we spend breaking and invested in building, wouldn't we be able to eliminate all the stuff? We're a lot of the stuff that we're breaking. Like imagine if a kid came out of a CS program on cyber and said, I wanna write more secure code. Ah, feel like it was a a, a great moment. I, I, I tell I'm on a rant. I'm in the, I'm

Matt Coles 19:12

They would, they would get laughed at out of that classroom so fast.

Chris Romeo 19:16

But

Matt Coles 19:17

But you know what,

Chris Romeo 19:18

what's wrong with Dennis Miller. I'm like, I don't want to go on a rant or anything, but I'm already there like,

Matt Coles 19:22

Yeah. so,

Chris Romeo 19:24

this rant. I

Matt Coles 19:24

so lemme let me throw the argument, let me throw the argument that I, that I think we always hear is well, In order to be a good defender, you have to understand what the attackers, you have to think like a hacker, right? You have to, you have to understand.

Izar Tarandach 19:36

no, Mad Mat. Mad Mat.

Chris Romeo 19:40

Exactly. Bad, bad. Matt is now being muted by the host of this podcast right now cuz I have the mute button and I'm muting him because Adam's sh Adam Showtek just called and said, did someone say, think like an attacker? Take them out. them off. The podcast. Eza is leaving. For people listening via audio.

Izar Tarandach 19:58

I, I'm, I'm, uh,

Chris Romeo 19:59

the set.

Izar Tarandach 20:00

no, I, I, I'm going to put Myst hat for a second.

Chris Romeo 20:03

Okay,

Matt Coles 20:04

There you go.

Izar Tarandach 20:06

did you just say think like an attacker?

Matt Coles 20:12

Wait, wait. So

Chris Romeo 20:13

like a hacker too. I heard that too.

Matt Coles 20:16

I'll, I'll just have to put money into the Jar Square jar for that

Chris Romeo 20:20

Now you're gonna

Izar Tarandach 20:21

for speed. Okay, continue Matt.

Matt Coles 20:27

Did you, did you fall? You fell off your soapbox apparently though,

Chris Romeo 20:31

Well, I'm still screaming at The fact that you said think like an attack. So you're where you, where you begin. I'm gonna re, I'll rehash just to prove I'm a, I'm an active listener that I did hear what you said before I started screaming

Matt Coles 20:45

did you really? Let's, let's see. This, let's, let's make sure you

Chris Romeo 20:48

said is the, the, the classic argument is that to be a good builder slash defender, you have to understand how to think like a beep. I couldn't say it, I can't bring myself to say

Matt Coles 21:01

And, and, and, and please note, please note for viewers at home. I, I don't feel that way. I'm just re re replying with what the counterargument is. Blah

Izar Tarandach 21:11

People see

Chris Romeo 21:13

Yes. He's

Matt Coles 21:14

People want good people on both sides. Say

Chris Romeo 21:16

need the pundit view. Definitely. So, but yeah. Let's, so let me, let's unpack that a little bit though. Like, is that true? I. Do, do actually, like, where do we land on that? Do we, do you believe like I, I mean, I think you

Matt Coles 21:28

of

Chris Romeo 21:29

you have to have, you do have to have a solid foundation to be a good builder on how things work.

Matt Coles 21:36

Yep.

Izar Tarandach 21:36

Yeah.

Chris Romeo 21:36

have to understand like if, if you're a developer, say you're an AppSec engineer and you tell me during the interview, Hey, I've never actually shaken it. I've never actually exploited cross-site scripting versus SQL injection, but I've studied them a lot like that. Is that like you do have to have that foundational knowledge? What I'm saying is like at the industry level, we're so hyper focused on breaking everything it, it's a detriment to. Focusing on more secure by

Izar Tarandach 22:05

Look, uh, I, I. Oh yeah, yeah, exactly. Yeah, exactly.

Chris Romeo 22:12

cws. unpack

Matt Coles 22:13

we have a source of, we have a, we have a source of, of, of ingra. We have a source of commonly known security knowledge that we can leverage. right? I don't need to know the technique that an attacker uses to exploit a se to exploit, uh, cross-site scripting vulnerability. If I understand how a cross-site scripting vulnerability gets introduced in code or is exposed in a network service, I can defend against it or I can build a avoid it in building. There's a lot of skills involved with how to do testing. Now, I will say I used to be a QA tester, uh, early on in my career, used to do security QA testing. So having that structured mind of being able to think about a system un identify attack points and put together a test plan is I think, important to, to know how that's going to happen. But being a pen tester to be the attacker. I cannot put myself in those shoes, right? I, I'm, that's not my mo right? And I don't think that goes for the, the two of you well.

Izar Tarandach 23:20

look, we, we, we have to be, we have to be honest about something. Uh, when was the last time that you saw a movie that shows that amazing point in time when somebody sits in front of a black screen and starts clicking, clicking, clicking, clicking, clicking, and says, I just wrote the library, the defense against success. It doesn't happen.

Chris Romeo 23:44

I'm gonna write

Izar Tarandach 23:45

What, what, what, what? What happens?

Chris Romeo 23:47

gonna watch it. Nobody.

Matt Coles 23:48

Champ, G P T Champion, G Champion G P T a script for you and you can

Izar Tarandach 23:52

what

Matt Coles 23:52

act it out.

Izar Tarandach 23:53

scanning the power plant. What happens is, uh, the guy that's played tour doing, I don't know what, with uh, uh, some, some, anyway, my point is pen testing. sexy. Defending is not kids coming up the ladder. Many of them, they are going for the fame, for building their nicknames and stuff like that. Nobody build their nick, sorry. Most people don't build their their name by building stuff for defending stuff. They do it by breaking stuff. And we have some great, great, great examples of people who started breaking and came to the other side of the table and started defending it and done it wonderfully and moved the the state of the art forward. Right. even if we look at the black hat, it's sort of. New Wish that Black hat has a defender track, right? I don't know how how many years, but it's not recent.

Chris Romeo 24:48

Yeah, true.

Izar Tarandach 24:49

It started as, as, as a full on black hat thing. And I think that of course, that there's a reason for that. Fantastic. breaking things. It's sexy defending. It's not, we, we are the gigs of the gigs

Chris Romeo 25:06

Yeah, that's that's true. And, and let the record show, I did just have chat. G p t write a short movie script write where a developer writes a secure library to save the day. So it's in, we're now accepting investors if you wanna be a part of this project. Um, I'm gonna, I'm thinking maybe Canu, Canna Smith

Izar Tarandach 25:24

I don't know. I, I don't know. I see a new career for me here. Like, look at this profile,

Matt Coles 25:31

need, you need to get, you need to get gal Gado or somebody you need. Have some, get a little diversity in there. You know,

Chris Romeo 25:36

I mean it's, you know, this is, this is, this is a, uh, this is my next project. I'm gonna work on this. This is, it's gonna be a short movie and no one's gonna ever watch it, but that's okay. So,

Matt Coles 25:45

you'll be surprised. You'll be surprised they'll be probably the 800 views by the time you.

Izar Tarandach 25:49

no, back, back, back to our thing. Okay, so we agree that fantastic sexier than, than defending. Now what do we do with that?

Matt Coles 25:58

Well, more important and actually a reason why pen testing is more, is more sexy than, than defending what happens when you pen test. There's bashing on the keys. There's stuff happening on screen, right? There's something to show and there's something to hear, and there's the, the sound coming from, from the people who are doing it. On the flip side, you have the defenders who are running around like crazy, you know, crazy people with their head cuts, cuts off, cut off. Oh my God, this guy is falling. They breached our network. What are we gonna do? Or step one of the policy says, do x. Step two says, and, and there's not much to see. Right? uh, you know, technology demos are really hard when there's nothing. Um, sorry, some sort of bug is flying around. Um, technology demos are hard when there's nothing to show. Right. A couple lines on screen running

Chris Romeo 26:51

Yeah.

Izar Tarandach 26:52

Wait, wait, wait. You sound like someone who has never sat in front of the logs coming from a firewall and going, yes, yes. I see your.

Chris Romeo 27:02

That was like a Gollum thing going my

Matt Coles 27:05

you were, if you were, if you were Uper a honey Pott, would you sit there and go, oh, he didn't do, oh, he did that. Oh, he, he's gonna go there. He's gonna go there.

Izar Tarandach 27:14

You, you

Matt Coles 27:14

But would you watch that? What would that?

Chris Romeo 27:17

I came up in, I came up in the world of computers at it that was actually happening. Live on we're like, hold, what did he just do? Wow. He or she really just took off, took down our entire system here. That was pretty cool. You know, that was the defender. But I mean, Eza to your point, like what do we do? And I mean, I think at the end of the day, unfortunately there's not a lot we can do because is the motivation, it's the extrinsic motivation that drives people. To want to break stuff and to get the fame and the glory that comes along with doing it. I think as an industry, we should be investing more on the defender of the builder side, and that's what, you know, you guys are the same as me. We've, we've, we've dedicated our lives in our careers over the last fif 10 or 15 years to doing this. Do I think we're gonna move the needle? A lot. No, but I think if we can do something to influence the people inside of big companies that are, that are, that are looking at the bottom line and then return on investment for what they're doing in security, I think over time people and people are migrating more towards secure building over the last 10 years than pen

Izar Tarandach 28:29

Okay. Off the top of my head, Chris, what if your next project, instead of the movie was we reversed the model of the CTF and now rather than going and getting the flag, you have to protect the flag. We get engines to do automatic attacks and you get to see the attack happening, and you get to stop that thing.

Chris Romeo 28:50

that does that. People, people have Yeah. There's

Izar Tarandach 28:53

I haven't

Matt Coles 28:53

Yeah.

Chris Romeo 28:54

There's a company that's doing blue team where I think

Izar Tarandach 28:58

oh, and I missed the

Chris Romeo 28:59

Yeah,

Matt Coles 29:00

and remember N E C C D C.

Izar Tarandach 29:02

Yeah, but that's live like that. You have to people attacking you. But it's a, it's a red team, blue team and they focus on the blue team is the, the, they don't, they don't give points for the red team break breaking the blue team spirit. They they give points for the blue team. Having a system le is live and running at the end right?

Chris Romeo 29:25

and it's, it much just to be very clear with the terms, right? Those environments are. breaker versus defender, not so much breaker versus builder or or breaker versus builder and defender. When I think red versus blue, it's, it's the defender that's representing, that's, that's protecting the system more at the network level and the service level versus writing better and secure stuff from, from the start.

Matt Coles 29:53

Now can we, can we just take a quick, uh, sidetrack here? We talked, we started with CTF versus Bug bounty. We didn't talk about red team exercises.

Izar Tarandach 30:02

Good

Matt Coles 30:03

there is a very, very similar thing, right? Red team, red teaming, well, I'll use my definition whether it's, uh, you'll, you guys can tell me if I have it right here. A red team is a test of, of an operation, a test of an operational system where you're testing both its defenses as well as the operations of the people who are managing it. The defenders function,

Chris Romeo 30:24

Agree.

Matt Coles 30:25

right? It is not testing.

Izar Tarandach 30:27

mm-hmm.

Matt Coles 30:28

not print testing per se, and, and, and people often confuse them. I think that they call pen testing and red teaming or bug bounty and red teaming to be the same thing, but they're not. Their goals are, are their, at their techniques are the same or, or very similar, but the goals are different.

Chris Romeo 30:45

Yeah, I think in the industry there's, there's a lot of people have, have used those as synonyms now, cuz you know, if you're doing a pen test with a physical component, technically you're red teaming, right? Cuz you're, you're going beyond the technology, you're testing the processes and the people at the security station, at the gate, at the, the door.

Matt Coles 31:05

May maybe if you're, if you're, if you're test well, if you're, yeah, I guess if you're, if you're doing physical penetration testing and you're gonna break in, you're trying to break in without being detected, I guess technically that is red teaming. Although your goal is to find the vulnerability, not necessarily to make sure that you don't get caught.

Chris Romeo 31:22

Yeah.

Matt Coles 31:23

Right.

Chris Romeo 31:24

I mean, some red teaming engagements, pen testing engagements. The goal is not to get caught, is to try to get in, violate the physical security constraints, violate some logical security controls, walk away with the computer out of the data center if you can do it and sell

Matt Coles 31:42

Right. Make best friends with the, make best friends with the security guard while you're at it, you know

Chris Romeo 31:46

an attack of, of the, the policies, like you said, your definition was, it was beyond the, the technical controls. It was attacking the policy and the people behind the of the system. is a lot of time what's happening there. So, I don't know. I mean, where do we, where do we go with this? Like, should we start a movement? Should we start a, a new, a new manifesto? You know,

Matt Coles 32:09

Well, how would you, so let's start with how would you. How would you incentivize this? So I guess if you look at bug bounty and you look at a CTF, right? You can incentivize behavior. Behavior is incentivized because it's sexy. There's stuff to see, there's stuff to hear. Um, there's, there's financial payout, there's a leader board, whatever. How would you incentivize, not the defender, but the builder side, how do you incentivize that, let's say from a competition standpoint? how? How would you even approach that?

Chris Romeo 32:42

I mean, off the cuff, I think outside the constraints of a single. Competition. That's really what security champions exist for inside of companies is you, your security champions program is to try to provide a reward and recognition structure for people doing the right thing a builder perspective. So I don't know, do we have the security champion Olympics

Izar Tarandach 33:07

No, but it isn't, but it isn't the, the, the reward is in the doing itself. I mean once, once, okay. So Security City, we can look at it and say it's a hobby thing. It's, it's a, you are interested in it thing. A pen test. It's a professional thing. It's you paying somebody to go and poke at your stuff.

Chris Romeo 33:25

Mm-hmm.

Izar Tarandach 33:27

the incentives there are are pretty different now from the builder point, point of view, incentive is the thing itself. If the pen test doesn't find anything, if the bug bounty doesn't find anything, I won. And I win by doing the right things.

Matt Coles 33:43

Mm-hmm.

Izar Tarandach 33:43

we are going back to the discussion that we always have to, how do we incentivize developers to do a bit more of what we want them to do and a bit less of what they are already doing. So I think that we, we are extrapolating now to, to a whole, system of incentives that feeds off each other.

Matt Coles 34:02

Okay. also have to think that this is a long, this is a long game. Discussion. Right, right. You to start with No, no. Yes. But I mean, from a, from a system development standpoint and a and a developer's standpoint, they have to be com. They, in order to be, um, in order to get the reward, they have to delay gratification. I'm gonna build this thing, and then at some point later, I'll know if I have won. And by the way, do you win at release? Do you win at that first pen test and they don't find anything? Do you win 10 years from now when your, your system, um, escapes unscathed with no CBEs,

Chris Romeo 34:40

But we're trying to get bigger. We're trying to get bigger

Izar Tarandach 34:43

You don't win.

Chris Romeo 34:44

if you want to, if you

Matt Coles 34:45

but you do,

Chris Romeo 34:45

look

Matt Coles 34:46

you just said.

Chris Romeo 34:47

look at how the pen test look. Look at how pen testing and breaking is so top of mind in our industry. It's a thing everybody wants to do. Like that's so much bigger than we're talking about tactical, incentivizing developers on the ground. And maybe that's the foundational layer that we're missing. But I wanna get to the point where breaking and building are at the same level from an industry perspective.

Izar Tarandach 35:11

But it won't. It won't, it won't. Because when you pen test something, of, uh, delayed gratification, if you don't have any findings and you got a good enough set of pen testers, and that's a whole different discussion, how you choose your pen tester. Then from a certain point of view, you say, at this point in time, apparently I don't have any glaring issues. Right. On the other hand, if you're defending they didn't find anything, You haven't quite proven anything to yourself. You have just proven that a group of people have not found anything at this time, so you, you are trying different set of assumptions.

Matt Coles 36:00

Well, the problem is you're pro, you're trying to prove it negative, right? You're trying to, you're trying to prove you're free of defects,

Chris Romeo 36:07

And

Matt Coles 36:07

which is

Chris Romeo 36:08

security assurance, right? That's where I grew up in security, was going through the assurance process and trying to prove. And it got into the levels that I didn't even understand of formal methods and things. I never that worked. I can pretend like I know what it, what it actually is, but,

Matt Coles 36:27

A lot of math. Yeah. It's just

Chris Romeo 36:29

but at the end of the day, it doesn't work for an operational production system because you takes you six months to prove the version you released six months ago is secure. And by that time we've generated, you know, 200 releases a day. And so it just, it doesn't, it doesn't add up.

Matt Coles 36:45

I, I, I Good. Yeah.

Izar Tarandach 36:46

go there. I'll go there. You, you mentioned formal methods. Absent of formal methods, the only thing that you can actually say is my version is secure. To the best of my knowledge right now, that anything. In 15 minutes, in, in 15 years.

Chris Romeo 36:59

True.

Izar Tarandach 36:59

So to quote, the best movie ever made poster here in the corner, only way to win the game is to not play it.

Chris Romeo 37:06

Yeah.

Matt Coles 37:07

Well, so. We can't not play it here, let me let, let me, let me, let me flip this a little bit and maybe, you know, let's think a little bit out of the box. Throw the security aspect out the window for a moment. What makes developers want to be developers? Because developers are builders, not security people, they're builders.

Izar Tarandach 37:33

Yeah.

Matt Coles 37:34

makes them want to be engineers? What makes them want to be developers and programmers solving problems? And you invent and you incentivize it by giving them problems to solve.

Izar Tarandach 37:44

Ah, solving the problems that they want to solve. Not all problems.

Chris Romeo 37:49

he's bringing him, he's bringing us full circle. I see where he is going. can see two steps ahead. I can see where he is going. He's bringing us back around to the CTF and saying, CTF is solving problems. We just need it to be something that's tailored towards developers. so that they can, they can solve their kinds of problems versus

Izar Tarandach 38:08

But, but then Chris, goes back to the experience that you have with the, the training stuff and all that, right? When we started gamifying all that stuff and giving people snippets of code for them to go and, and do the right thing because we just. Show them not it. We, we know how that, that it works better than not having it, but that it didn't really get to people's imagination.

Chris Romeo 38:29

Yeah. It industry. The,

Matt Coles 38:31

there's a, well, there's a missing.

Chris Romeo 38:33

changed.

Matt Coles 38:33

a missing, sorry. There's a missing link there. Right? When you do that, you're just sort of, you're just, it's just the skills test, like within the scope of that training exercise for the developer. It's not the real world application, right? So how do you take that snippet, or how do you take that, that exercise, not the snippet itself, but the exercise and bring it back to the defense of the thing the developer just built, because that's where you'll get the full value from it.

Chris Romeo 39:00

GitHub needs to add locks. and every time somebody does something somebody can nominate them for a lock when you're doing your code review and then, the security and, uh, GitHub, that is pat hub, pen pat and petting trademark. Um, not really, but like, but I mean, imagine that type of a external of validation like GitHub Stars is a great validation for like, when you look at an open source project, if it has. Two GitHub stars, you're like, eh, maybe I'll find a different prep package to use. This doesn't look great. Imagine on your GitHub profile though, if you could have that where other people had recognized a, a positive security pattern and gamified it for you.

Izar Tarandach 39:41

collision, collusion,

Matt Coles 39:47

There's a, maybe there's an AI opportunity there, but, uh,

Chris Romeo 39:49

modeling guy that comes back and starts pointing out the flaws

Izar Tarandach 39:53

What? What do we do here? We have, we have three of those here.

Chris Romeo 39:56

I know. That's the challenge. That's the challenge. I thought I had a good idea, but All right, we're outta time on the security table for today. Um,

Matt Coles 40:03

We are making

Chris Romeo 40:04

like we solved. I think we, we made some progress in the right direction.

Izar Tarandach 40:08

Wait, we were trying to solve anything

Chris Romeo 40:09

Oh. I'm always trying to solve for

Izar Tarandach 40:11

Nobody told me.

Chris Romeo 40:12

Oh, I should have told you that upfront. But, uh, I think,

Matt Coles 40:15

you didn't capture the flag.

Chris Romeo 40:16

It was a good, it was this good and spirited discussion. I mean, I think that's the highest easer and my blood pressure has been during an episode a result of Matt's, uh, describing, think like a beep. Um, I can't even say it. I can't even bring myself to say it, but we'll continue our spirited debate because Matt tried it at home and Za is representing now a Star Wars bomb wearing So let the record show he

Matt Coles 40:47

Wait, wait,

Chris Romeo 40:48

Darth

Matt Coles 40:48

that Star Wars or Space Balls?

Chris Romeo 40:51

Oh, that's true. It'd be bigger. Nah, it'd be bigger space. He

Izar Tarandach 40:55

Prepare for low degrees speed. So everybody have a, a ludicrous Memorial day and uh, enjoy your weekend

Chris Romeo 41:03

Yeah, enjoy your weekend. Have a great time. Thanks

Izar Tarandach 41:06

I'll see you guys on the next one.

Chris Romeo 41:07

Yeah, thanks for listening to us on the security table. Thanks.