The Security Table
The Security Table is four cybersecurity industry veterans from diverse backgrounds discussing how to build secure software and all the issues that arise!
The Security Table
Capture the Flag or NOT?
There is an overemphasis on Capture The Flag in the security world. Instead, the industry should focus more on the 'builder' perspective to develop robust systems rather than the 'breaker' mindset typically associated with penetration testing and CTF competitions. In addition, we must shift the industry's reward and recognition structures to incentivize building secure-by-design systems.
A CTF is a type of cybersecurity competition where participants solve security-related challenges to find flags representing vulnerabilities or secrets within a system. A CTF and bug bounty are similar, as both test cybersecurity skills but have different goals and outcomes.
Red teaming is not just about penetration testing but also about testing the operations of the people who manage defenses.
Finally, the discussion ends with pondering the question of "winning" in cybersecurity and agreeing that providing a system free of defects and ensuring security assurance should be the ultimate goal.
FOLLOW OUR SOCIAL MEDIA:
➜Twitter: @SecTablePodcast
➜LinkedIn: The Security Table Podcast
➜YouTube: The Security Table YouTube Channel
Thanks for Listening!
Hey folks. Welcome to another episode of the Security Table, we sometimes debate the philosophy behind the matrix. Was it the red pill? Was it the blue pill? But sometimes we talk about application security and other related things too. And so I feel like we're, we're still on this, this thread. Oh, who are we? Who are we? Who is anybody really? So I'm Chris Romeo, joined by my friends Matt Coles and Izar Tarandash. today, gonna continue pulling on that thread of what we affectionately referred to as reasonable application security. I think there's a, a connection with what we want to talk today, talk about today, but I think it's, it's also much bigger. And so the premise in front of us was really an image that, that I saw somebody tweet and it was in regards to capture the flag. And so this individual who I should have been more prepared and ready to quote, give attribution to
Matt Coles:What else is new? I mean, come on,
Chris Romeo:So what else? We we're known at the security table for being very unprepared. Um, he, he, he basically just had this premise of capture. The flag is overrated. It's overhyped, it doesn't generate Doesn't help you generate CBEs. I thought it was a pretty interesting construct and it's gonna, it's gonna allow me to stand on a soapbox a little bit later and talk about what I think about red teaming and pen testing and the focus we give in the industry.
Izar Tarandach:Ooh,
Chris Romeo:let's, let's unpack the CTF question first. So, so you guys te tell me I'm not a CTF'er. Okay. I've never played a CTF, so I feel like I'm only qualified to share my opinions of it, not necessarily describe what it is. So, I mean, so let, let's work up. What is it? What is a CTF?
Izar Tarandach:Matt might you you go; you're good with the definitions.
Chris Romeo:Do you have a
Matt Coles:Great. great. the,
Chris Romeo:with CTF written on it and a definition?
Matt Coles:well, I, I, I, I do, I do have a flag with his on it. Uh,
Izar Tarandach:Consider it captured.
Matt Coles:So, so, so, yeah. Great. I feel I'm, I'm Mr. Dictionary over here. Great. Um, so CTF, right? Capture the flag. Uh, Competition for people to get together and, um, break stuff. Uh, well, not so much break stuff as find stuff, I guess is the important part, right? So we're talking about a structured competition, uh, with a defined goal in mind or set of goals, maybe some milestones along the way. Uh, and, and people, uh, get together and. And ultimately it's a competition, right? So, so there's, there's one objective, you know, a piece of data or, or a control of a system or whatever. Uh, and obviously we're talking, uh, we're talking se security here. There's other capture of flags. You know, if you're, if you're a gamer, you might be going for that literally a flag, uh, on the screen or you're doing paintball or something. Very similar concept. Uh, hopefully folks have that pretty clear in their mind. You're looking for something, you're trying to get it before the other team does, uh, capture it. Hold it. Destroy it. Do whatever you need to do with it. And, and, winter, winter, winter, chicken dinner,
Chris Romeo:I love a good chicken dinner. So Eza, what else? Like what else?
Izar Tarandach:So that that's.
Chris Romeo:provide?
Izar Tarandach:Yeah, so, so that's, that's some of the fun. Of course there, there's a whole bunch of, uh, cts that happen either like individually, you go and you, you have a specific set of problems that progress usually in, uh, difficulty. for each one of them, you get to that string. That is the flag. You apply to the interface, you get the next problem. And, um, We have seen some of those also applied in companies as an educational tool or in hackathons, internal hackathons. They, they're very, uh, popular for that, that kind of thing. And stepping away from the definition and, and going into the is this good for, is I think why we're here I to me, okay. First of all, uh, the competition part, I think that, uh, a lot of people that are in this professional are interested in the, the things that we deal with. Uh, are competitive by nature. So are good ways of, uh, pitting what I know against what you know, and coming out of something that's, uh, quantifiable out of it. Uh, if I have more flags than you, if I have more points than you, then uh, I win and you lose. And, uh, I'll enjoy your dinner and dinner, but, uh, what I like to question is the educational point, point of view, right? Cause. What, what would be to, in my, my head, what, what's the closest thing that, uh, you could parallel uh, uh, CTF? To me, it would be bounty, right? But the difference between both of them is that the CTF, by the definition can be exploited, and there is something at the end of the rainbow. So it's like the CTF is a, is a sure thing. The bug bounty is you go in and you apply the things that perhaps you learned even in the CTF and happen, may not happen. You don't know you're going blind. can it. Own experience. Do I know what I'm doing here? And that, that
Matt Coles:So or would you. Sorry, would you, so, uh, what you're calling others that CTF has a, has is a structure, is a structured activity that has definitive goals and milestones that are achievable. Right? That's, that's what you're ultimately saying here is a CTF. A CTF is a structure for deli with content that is known, that tests for skills as opposed to bug bo, which is content that is unknown. Testing for the, like what flags do we have? Not how do you get to the flags we know of?
Izar Tarandach:Actually, I am more simplistic than that. I'm saying going to the CTF. You gotta, you gotta guaranteed dopamine hit if you do A, B, C, D and you go into the bug bug, uh, bug bounty and you don't know what's gonna come out of it. Perhaps you find something and perhaps you won't. Right. So I, I, I, I do see the parallel between both of them. I don't think that they, working on one is, is a good ramp up to the other.
Matt Coles:Would you, would you characterize, would you say though that a bug bounty is a, is first and foremost the test of a system, whereas a CTF is a test of skills?
Izar Tarandach:Oh, that's a beautiful way of putting it. So you you mean that the, the target of the exercise is different.
Matt Coles:The outcome is different, right? Yes, you need skills to do a bug bounty, but you don't know what the target is. You don't know what that flag is when you do a bug bounty. So you have to have skill. You're testing skills, but you're really looking for the, the goal of running a bug bounty is not to know whether an attacker is good, but to know whether they find, find the flags, right. Find something, and, and again, those flags are. Unknown at the time. Whereas the CTF, all the flags and all the milestones on all the path through the system are known to the organizers. At the very least, you're testing the skills and, and potentially the collaboration and, and, um, and inner working, you know, interpersonal, uh, and, and technical operation of a, of an individual or a team to meet, to, to get to those objectives and milestones and, and the flag ultimately to the flags.
Izar Tarandach:Okay,
Matt Coles:So the goal of the, the goal of the exercise is different.
Izar Tarandach:so analogy time, it sounds to me like you're saying that the CTF is like a shooting range you have like very controlled situations, very controlled limits, and you are basically. Checking out if you know how to a weapon, how, how to shoot in a target,
Matt Coles:Mm-hmm.
Izar Tarandach:the bug body is closer to being out there and doing those things that people who do things with weapons do. You have all kinds of, uh, unknowns and, uh, the, the boundaries are not known. The, the, the terrain is not known. Who's shooting back at you is not known. So all that, and then of course, it extrapolates to bad people doing bad, bad things, which is actually the, the word of attackers attacking our applications. So three, three here of indirection. Yeah.
Chris Romeo:Let's give it a, let's take it a step further though, and talk about outcomes of each one.
Matt Coles:Mm-hmm.
Chris Romeo:about, when you're putting bug bounty against CTF, the, the outcome that's most, I think most people are aiming towards is enhancing their knowledge and the CTF, they to break stuff better so that they can apply it to breaking stuff in wherever else they break stuff. The bug bounty, the outcomes dollars in your pocket. So similar path, but. Bug Bounty. So I'm thinking about Matt Bug Bounty from the perspective of the person who's doing it, not the, organization who's
Matt Coles:there's, there's two. There's two aspects there that to consider. I think you're, yeah,
Chris Romeo:real CTF, like there's no real organizational value. I guess there could be organizational CTF, um, of, of enhancing the knowledge of the people that work in your company.
Matt Coles:well, I would add.
Chris Romeo:individual first.
Matt Coles:Oh, well, I, I, I would actually, uh, just you, I think you covered the bug bounty and the CTF from the goal of the participants pretty well, right? A test of skills. It's a test of skill. It is a test of skills, and there's money associated probably in both ends, right? There's, there's either fame and fortune from, and participating in winning a CTF and you gain, gain skills or you show off your skills, right? You either get bragging rights or you get a payout or you get a promotion, you get whatever. same thing for the bug bounty, right? Although that's more of a financial impact, or maybe it's swag or maybe it's a leaderboard or whatever the case may be. So for the participant, you get, you get tangible benefits from either of those activities and very similar tangible benefits. The bug bounty from a organizer's standpoint, the bug bounty is I get somebody else to do testing for me that I then re, uh, compensate them for. And so I learned better about my system and I learned then, uh, what, um, Uh, you know, I, I, I, I can improve that. A CTF, though, I think. is really from a, from an organizer's standpoint, if it's a corporate organization organizing it with their employees, they're testing their employees abilities. Maybe it's a team building exercise. So they get, you know, some interpersonal connection there. They get to collaborate, they get to test out processes. We can talk later about red team versus Blue team, cuz there's another area to go into. Uh, Or if it's a organization that does it like the us um, as a US CTF. Uh, I, I'll have to pull it cuz it was, that was at Defcon last year run by a good friend of mine. Uh, or, or, or, uh, at least mcd by a good friend of mine, uh, uh, who, who, um, this was, you know, bring teams together. They do CTF, um, they learn about new attack techniques, right? So it's a chance to show off new industry trends or learn about new attack behavior or, um, or, you know, show off, show off in a different way, right? So there are tangible benefits across the board. Not all of them are financial
Chris Romeo:So the original of Twitter post tweet got me to this, I just said Twitter post, I'm so embarrassed, uh, that got us to this conversation. Was an image with the header at the
Izar Tarandach:Facebook posted
Chris Romeo:right. Um, I put it on the Facebook. Um,
Matt Coles:You didn't ticket or talk it
Chris Romeo:I didn't, I don't understand that. I don't, I don't even want to go there. Matt, you, you almost, you almost open a can of worms suggesting that I would be on TikTok now. I'm not
Matt Coles:I,
Chris Romeo:of worms. That's,
Matt Coles:disclaimer, I'm not on TikTok either, so
Chris Romeo:that's a whole other, that's a whole other discussion, debate as far as where that data goes. But the
Izar Tarandach:The security table dance.
Chris Romeo:That this tweeter did was stop doing CTF security was not meant for competitions years of CTFs yet. No real world CBEs found. You want to a hack something? Don't use CF ti time use this. And then it's an analog modem with the, with the, so I don't know if it's a, if, if this is just purely a trolling that I just fell for, or, but it does bring this issue up like, are we too focused on CTF in the world of security and we're going outside AppSec. Now we're talking about security in general. Do we focus too much on CTF?
Izar Tarandach:Okay. Oh,
Matt Coles:Go Go ahead. Ard. You first.
Izar Tarandach:Last time that I participated in CTF was like, what, two years ago? And for one task, like the, the one that the points were like absurdly bigger than everything else. And it ended up like was one binary that had something like eight different layers of, of. Garbling and obscurity on top of it, and ended up going binary in one language, encapsulating another one, and so on and so forth until the flag, and know what it, it has absolutely nothing to do with security. It was more about reverse engineering and, and figuring out some very basic crypto, but I didn't care because it, it kept happening and the dopamine kept hitting and I kept solving a problem. and I could not give less of a care about am I having a security experience, am I learning more? Because it felt good.
Chris Romeo:Hmm.
Izar Tarandach:And I think, and my point is that, uh, we, we, we already have to fight so much when, when we are teaching, when we are pushing, when we are, uh, building these programs, we, we have to fight so much for people to get interested in it and for people to give you their time and, and their attention. and I think that that dopamine hit is, is a positive. If I can get somebody at least minimally interested in some aspect of security using that mechanism, I'll freaking use it.
Chris Romeo:but how practical, like you just described something that's never gonna happen in a modern web application architecture
Izar Tarandach:Oh, no, no, no, no. That, that, that was like that, that was just illustrate effect that I got so engrossed in into it. Right.
Chris Romeo:but
Matt Coles:And,
Chris Romeo:something that doesn't, that has no real world applicability though? Or is there some real world case case for this architecture
Matt Coles:You're think you're think you're thinking about it wrong. I think we're thinking about it wrong, and maybe that's, maybe that's, we're putting a lot, we talked about it. We, we just talked about it actually. CTF versus bug grati. Right. The goal of the exercise is different. We're not in a CTF. We're not looking for new vulnerabilities. We're not looking at necessarily inventing new attack techniques, although that probably happens in a CTF probably pretty frequently. But that's not the, maybe that's not the outcome that we should be asking ourselves about, right? If the, if the, if the statement is CTFs don't produce any vulnerabilities, therefore they are bad. Meaning their waste of time because they don't produce vulnerabilities. I would suggest that their as, that the, the goal, the expected outcome is the wrong outcome.
Chris Romeo:Hmm.
Matt Coles:it's not a waste of time if it's a team building exercise. It's not a waste of time. If it's a opportunity to get people interested in the field of cybersecurity, it's not a waste of time if it gets people thinking about, you know, new, new attack techniques or new memory defenses or new coding constructs or other things, you know, that are non-tangible benefits that come out of or are, you know, not, not directly related to the CTF benefits. Uh, and therefore it's not a waste of time. If you're looking to find defects, well, that's a bug bounty,
Izar Tarandach:Mm-hmm.
Matt Coles:right? If you want to do cvs, c cvs, that's a bug bounty. We don't know what the target is. When you construct a CTF, you plant a flag. You know, that's a, that's a target, that's a piece of data. And you say, oh, that data is protected by these things. And well, in order to get through that, somebody's gonna have to know how to do a memory injection and, and, and a heap, heap, heap overflow, and, uh, you know, route to, you know, return to lipps and all this other stuff. Well, They may find a way around those, those attacks and do something completely different, but that's not the outcome you're looking for. The outcome is basics or a particular path or a particular goal in.
Chris Romeo:I'm somebody who could care less about CVEs. And it's not just because I don't have any, let the record show. I just, I don't, I mean, I don't, I don't measure my worth in the security industry based on the number of mire, cve, e entries that exist in the database. But
Matt Coles:but we are not pen testers.
Chris Romeo:but I think when I was about ready to go. I'm gonna go there in that direction though, right? Like, I don't consider myself a breaker. Using the classic owas terminology, builder breaker, defender.
Matt Coles:Mm-hmm.
Chris Romeo:consider myself a breaker or really, I mean, we're really a defender. I, I consider myself a builder and as a builder I think about how can we spend our time better so that we can build better stuff that doesn't have the problems. Like if we can build something that doesn't have any of the issues, tech, they'll have nothing to break. There'll be nothing to find if we, if we, and, and so this is, I'm, I'm kind of, I'm about ready to unleash my, um, soapbox moment here because, and I've written about this before. I've talked about it publicly, and it's an unpopular opinion. Get ready to put up the unpopular opinion flag. I don't care. In our industry, we focus too much on breaking, we focus too much on red teaming, too much on CTFs. Um, these guys are all writing the resignation letters to the security table right now, and so it's okay,
Izar Tarandach:Uh,
Chris Romeo:but we spend, we spend too much time. Like, and, and all you gotta do is go walk up. So find a, a kid who's, who's a kid, I shouldn't say that. A student, a young, a, a younger person who's in university, getting ready to go into cybersecurity and ask them, what's the number one thing you wanna do? And what do they all say? I
Matt Coles:Break stuff.
Chris Romeo:secure Java code. eliminate cross. No, they all say they wanna break things and so as an
Izar Tarandach:and move fest.
Chris Romeo:and move fast. That's fine. Move fast. That's me, That's my soapbox moment. Thank you. That is an
Matt Coles:is.
Chris Romeo:excellent graphic. CBEs. Who cares? I feel like I'm like, I'm the get off my lawn guy when it comes to, you know, that's, that's my new persona I'm taking on here. But, We spend too much What is this? Scary? What is it? Good Four? No, I mean, we spend too much time focused on it. Like, if we could take half of the effort we spend breaking and invested in building, wouldn't we be able to eliminate all the stuff? We're a lot of the stuff that we're breaking. Like imagine if a kid came out of a CS program on cyber and said, I wanna write more secure code. Ah, feel like it was a a, a great moment. I, I, I tell I'm on a rant. I'm in the, I'm
Matt Coles:They would, they would get laughed at out of that classroom so fast.
Chris Romeo:But
Matt Coles:But you know what,
Chris Romeo:what's wrong with Dennis Miller. I'm like, I don't want to go on a rant or anything, but I'm already there like,
Matt Coles:Yeah. so,
Chris Romeo:this rant. I
Matt Coles:so lemme let me throw the argument, let me throw the argument that I, that I think we always hear is well, In order to be a good defender, you have to understand what the attackers, you have to think like a hacker, right? You have to, you have to understand.
Izar Tarandach:no, Mad Mat. Mad Mat.
Chris Romeo:Exactly. Bad, bad. Matt is now being muted by the host of this podcast right now cuz I have the mute button and I'm muting him because Adam's sh Adam Showtek just called and said, did someone say, think like an attacker? Take them out. them off. The podcast. Eza is leaving. For people listening via audio.
Izar Tarandach:I, I'm, I'm, uh,
Chris Romeo:the set.
Izar Tarandach:no, I, I, I'm going to put Myst hat for a second.
Chris Romeo:Okay,
Matt Coles:There you go.
Izar Tarandach:did you just say think like an attacker?
Matt Coles:Wait, wait. So
Chris Romeo:like a hacker too. I heard that too.
Matt Coles:I'll, I'll just have to put money into the Jar Square jar for that
Chris Romeo:Now you're gonna
Izar Tarandach:for speed. Okay, continue Matt.
Matt Coles:Did you, did you fall? You fell off your soapbox apparently though,
Chris Romeo:Well, I'm still screaming at The fact that you said think like an attack. So you're where you, where you begin. I'm gonna re, I'll rehash just to prove I'm a, I'm an active listener that I did hear what you said before I started screaming
Matt Coles:did you really? Let's, let's see. This, let's, let's make sure you
Chris Romeo:said is the, the, the classic argument is that to be a good builder slash defender, you have to understand how to think like a beep. I couldn't say it, I can't bring myself to say
Matt Coles:And, and, and, and please note, please note for viewers at home. I, I don't feel that way. I'm just re re replying with what the counterargument is. Blah
Izar Tarandach:People see
Chris Romeo:Yes. He's
Matt Coles:People want good people on both sides. Say
Chris Romeo:need the pundit view. Definitely. So, but yeah. Let's, so let me, let's unpack that a little bit though. Like, is that true? I. Do, do actually, like, where do we land on that? Do we, do you believe like I, I mean, I think you
Matt Coles:of
Chris Romeo:you have to have, you do have to have a solid foundation to be a good builder on how things work.
Matt Coles:Yep.
Izar Tarandach:Yeah.
Chris Romeo:have to understand like if, if you're a developer, say you're an AppSec engineer and you tell me during the interview, Hey, I've never actually shaken it. I've never actually exploited cross-site scripting versus SQL injection, but I've studied them a lot like that. Is that like you do have to have that foundational knowledge? What I'm saying is like at the industry level, we're so hyper focused on breaking everything it, it's a detriment to. Focusing on more secure by
Izar Tarandach:Look, uh, I, I. Oh yeah, yeah, exactly. Yeah, exactly.
Chris Romeo:cws. unpack
Matt Coles:we have a source of, we have a, we have a source of, of, of ingra. We have a source of commonly known security knowledge that we can leverage. right? I don't need to know the technique that an attacker uses to exploit a se to exploit, uh, cross-site scripting vulnerability. If I understand how a cross-site scripting vulnerability gets introduced in code or is exposed in a network service, I can defend against it or I can build a avoid it in building. There's a lot of skills involved with how to do testing. Now, I will say I used to be a QA tester, uh, early on in my career, used to do security QA testing. So having that structured mind of being able to think about a system un identify attack points and put together a test plan is I think, important to, to know how that's going to happen. But being a pen tester to be the attacker. I cannot put myself in those shoes, right? I, I'm, that's not my mo right? And I don't think that goes for the, the two of you well.
Izar Tarandach:look, we, we, we have to be, we have to be honest about something. Uh, when was the last time that you saw a movie that shows that amazing point in time when somebody sits in front of a black screen and starts clicking, clicking, clicking, clicking, clicking, and says, I just wrote the library, the defense against success. It doesn't happen.
Chris Romeo:I'm gonna write
Izar Tarandach:What, what, what, what? What happens?
Chris Romeo:gonna watch it. Nobody.
Matt Coles:Champ, G P T Champion, G Champion G P T a script for you and you can
Izar Tarandach:what
Matt Coles:act it out.
Izar Tarandach:scanning the power plant. What happens is, uh, the guy that's played tour doing, I don't know what, with uh, uh, some, some, anyway, my point is pen testing. sexy. Defending is not kids coming up the ladder. Many of them, they are going for the fame, for building their nicknames and stuff like that. Nobody build their nick, sorry. Most people don't build their their name by building stuff for defending stuff. They do it by breaking stuff. And we have some great, great, great examples of people who started breaking and came to the other side of the table and started defending it and done it wonderfully and moved the the state of the art forward. Right. even if we look at the black hat, it's sort of. New Wish that Black hat has a defender track, right? I don't know how how many years, but it's not recent.
Chris Romeo:Yeah, true.
Izar Tarandach:It started as, as, as a full on black hat thing. And I think that of course, that there's a reason for that. Fantastic. breaking things. It's sexy defending. It's not, we, we are the gigs of the gigs
Chris Romeo:Yeah, that's that's true. And, and let the record show, I did just have chat. G p t write a short movie script write where a developer writes a secure library to save the day. So it's in, we're now accepting investors if you wanna be a part of this project. Um, I'm gonna, I'm thinking maybe Canu, Canna Smith
Izar Tarandach:I don't know. I, I don't know. I see a new career for me here. Like, look at this profile,
Matt Coles:need, you need to get, you need to get gal Gado or somebody you need. Have some, get a little diversity in there. You know,
Chris Romeo:I mean it's, you know, this is, this is, this is a, uh, this is my next project. I'm gonna work on this. This is, it's gonna be a short movie and no one's gonna ever watch it, but that's okay. So,
Matt Coles:you'll be surprised. You'll be surprised they'll be probably the 800 views by the time you.
Izar Tarandach:no, back, back, back to our thing. Okay, so we agree that fantastic sexier than, than defending. Now what do we do with that?
Matt Coles:Well, more important and actually a reason why pen testing is more, is more sexy than, than defending what happens when you pen test. There's bashing on the keys. There's stuff happening on screen, right? There's something to show and there's something to hear, and there's the, the sound coming from, from the people who are doing it. On the flip side, you have the defenders who are running around like crazy, you know, crazy people with their head cuts, cuts off, cut off. Oh my God, this guy is falling. They breached our network. What are we gonna do? Or step one of the policy says, do x. Step two says, and, and there's not much to see. Right? uh, you know, technology demos are really hard when there's nothing. Um, sorry, some sort of bug is flying around. Um, technology demos are hard when there's nothing to show. Right. A couple lines on screen running
Chris Romeo:Yeah.
Izar Tarandach:Wait, wait, wait. You sound like someone who has never sat in front of the logs coming from a firewall and going, yes, yes. I see your.
Chris Romeo:That was like a Gollum thing going my
Matt Coles:you were, if you were, if you were Uper a honey Pott, would you sit there and go, oh, he didn't do, oh, he did that. Oh, he, he's gonna go there. He's gonna go there.
Izar Tarandach:You, you
Matt Coles:But would you watch that? What would that?
Chris Romeo:I came up in, I came up in the world of computers at it that was actually happening. Live on we're like, hold, what did he just do? Wow. He or she really just took off, took down our entire system here. That was pretty cool. You know, that was the defender. But I mean, Eza to your point, like what do we do? And I mean, I think at the end of the day, unfortunately there's not a lot we can do because is the motivation, it's the extrinsic motivation that drives people. To want to break stuff and to get the fame and the glory that comes along with doing it. I think as an industry, we should be investing more on the defender of the builder side, and that's what, you know, you guys are the same as me. We've, we've, we've dedicated our lives in our careers over the last fif 10 or 15 years to doing this. Do I think we're gonna move the needle? A lot. No, but I think if we can do something to influence the people inside of big companies that are, that are, that are looking at the bottom line and then return on investment for what they're doing in security, I think over time people and people are migrating more towards secure building over the last 10 years than pen
Izar Tarandach:Okay. Off the top of my head, Chris, what if your next project, instead of the movie was we reversed the model of the CTF and now rather than going and getting the flag, you have to protect the flag. We get engines to do automatic attacks and you get to see the attack happening, and you get to stop that thing.
Chris Romeo:that does that. People, people have Yeah. There's
Izar Tarandach:I haven't
Matt Coles:Yeah.
Chris Romeo:There's a company that's doing blue team where I think
Izar Tarandach:oh, and I missed the
Chris Romeo:Yeah,
Matt Coles:and remember N E C C D C.
Izar Tarandach:Yeah, but that's live like that. You have to people attacking you. But it's a, it's a red team, blue team and they focus on the blue team is the, the, they don't, they don't give points for the red team break breaking the blue team spirit. They they give points for the blue team. Having a system le is live and running at the end right?
Chris Romeo:and it's, it much just to be very clear with the terms, right? Those environments are. breaker versus defender, not so much breaker versus builder or or breaker versus builder and defender. When I think red versus blue, it's, it's the defender that's representing, that's, that's protecting the system more at the network level and the service level versus writing better and secure stuff from, from the start.
Matt Coles:Now can we, can we just take a quick, uh, sidetrack here? We talked, we started with CTF versus Bug bounty. We didn't talk about red team exercises.
Izar Tarandach:Good
Matt Coles:there is a very, very similar thing, right? Red team, red teaming, well, I'll use my definition whether it's, uh, you'll, you guys can tell me if I have it right here. A red team is a test of, of an operation, a test of an operational system where you're testing both its defenses as well as the operations of the people who are managing it. The defenders function,
Chris Romeo:Agree.
Matt Coles:right? It is not testing.
Izar Tarandach:mm-hmm.
Matt Coles:not print testing per se, and, and, and people often confuse them. I think that they call pen testing and red teaming or bug bounty and red teaming to be the same thing, but they're not. Their goals are, are their, at their techniques are the same or, or very similar, but the goals are different.
Chris Romeo:Yeah, I think in the industry there's, there's a lot of people have, have used those as synonyms now, cuz you know, if you're doing a pen test with a physical component, technically you're red teaming, right? Cuz you're, you're going beyond the technology, you're testing the processes and the people at the security station, at the gate, at the, the door.
Matt Coles:May maybe if you're, if you're, if you're test well, if you're, yeah, I guess if you're, if you're doing physical penetration testing and you're gonna break in, you're trying to break in without being detected, I guess technically that is red teaming. Although your goal is to find the vulnerability, not necessarily to make sure that you don't get caught.
Chris Romeo:Yeah.
Matt Coles:Right.
Chris Romeo:I mean, some red teaming engagements, pen testing engagements. The goal is not to get caught, is to try to get in, violate the physical security constraints, violate some logical security controls, walk away with the computer out of the data center if you can do it and sell
Matt Coles:Right. Make best friends with the, make best friends with the security guard while you're at it, you know
Chris Romeo:an attack of, of the, the policies, like you said, your definition was, it was beyond the, the technical controls. It was attacking the policy and the people behind the of the system. is a lot of time what's happening there. So, I don't know. I mean, where do we, where do we go with this? Like, should we start a movement? Should we start a, a new, a new manifesto? You know,
Matt Coles:Well, how would you, so let's start with how would you. How would you incentivize this? So I guess if you look at bug bounty and you look at a CTF, right? You can incentivize behavior. Behavior is incentivized because it's sexy. There's stuff to see, there's stuff to hear. Um, there's, there's financial payout, there's a leader board, whatever. How would you incentivize, not the defender, but the builder side, how do you incentivize that, let's say from a competition standpoint? how? How would you even approach that?
Chris Romeo:I mean, off the cuff, I think outside the constraints of a single. Competition. That's really what security champions exist for inside of companies is you, your security champions program is to try to provide a reward and recognition structure for people doing the right thing a builder perspective. So I don't know, do we have the security champion Olympics
Izar Tarandach:No, but it isn't, but it isn't the, the, the reward is in the doing itself. I mean once, once, okay. So Security City, we can look at it and say it's a hobby thing. It's, it's a, you are interested in it thing. A pen test. It's a professional thing. It's you paying somebody to go and poke at your stuff.
Chris Romeo:Mm-hmm.
Izar Tarandach:the incentives there are are pretty different now from the builder point, point of view, incentive is the thing itself. If the pen test doesn't find anything, if the bug bounty doesn't find anything, I won. And I win by doing the right things.
Matt Coles:Mm-hmm.
Izar Tarandach:we are going back to the discussion that we always have to, how do we incentivize developers to do a bit more of what we want them to do and a bit less of what they are already doing. So I think that we, we are extrapolating now to, to a whole, system of incentives that feeds off each other.
Matt Coles:Okay. also have to think that this is a long, this is a long game. Discussion. Right, right. You to start with No, no. Yes. But I mean, from a, from a system development standpoint and a and a developer's standpoint, they have to be com. They, in order to be, um, in order to get the reward, they have to delay gratification. I'm gonna build this thing, and then at some point later, I'll know if I have won. And by the way, do you win at release? Do you win at that first pen test and they don't find anything? Do you win 10 years from now when your, your system, um, escapes unscathed with no CBEs,
Chris Romeo:But we're trying to get bigger. We're trying to get bigger
Izar Tarandach:You don't win.
Chris Romeo:if you want to, if you
Matt Coles:but you do,
Chris Romeo:look
Matt Coles:you just said.
Chris Romeo:look at how the pen test look. Look at how pen testing and breaking is so top of mind in our industry. It's a thing everybody wants to do. Like that's so much bigger than we're talking about tactical, incentivizing developers on the ground. And maybe that's the foundational layer that we're missing. But I wanna get to the point where breaking and building are at the same level from an industry perspective.
Izar Tarandach:But it won't. It won't, it won't. Because when you pen test something, of, uh, delayed gratification, if you don't have any findings and you got a good enough set of pen testers, and that's a whole different discussion, how you choose your pen tester. Then from a certain point of view, you say, at this point in time, apparently I don't have any glaring issues. Right. On the other hand, if you're defending they didn't find anything, You haven't quite proven anything to yourself. You have just proven that a group of people have not found anything at this time, so you, you are trying different set of assumptions.
Matt Coles:Well, the problem is you're pro, you're trying to prove it negative, right? You're trying to, you're trying to prove you're free of defects,
Chris Romeo:And
Matt Coles:which is
Chris Romeo:security assurance, right? That's where I grew up in security, was going through the assurance process and trying to prove. And it got into the levels that I didn't even understand of formal methods and things. I never that worked. I can pretend like I know what it, what it actually is, but,
Matt Coles:A lot of math. Yeah. It's just
Chris Romeo:but at the end of the day, it doesn't work for an operational production system because you takes you six months to prove the version you released six months ago is secure. And by that time we've generated, you know, 200 releases a day. And so it just, it doesn't, it doesn't add up.
Matt Coles:I, I, I Good. Yeah.
Izar Tarandach:go there. I'll go there. You, you mentioned formal methods. Absent of formal methods, the only thing that you can actually say is my version is secure. To the best of my knowledge right now, that anything. In 15 minutes, in, in 15 years.
Chris Romeo:True.
Izar Tarandach:So to quote, the best movie ever made poster here in the corner, only way to win the game is to not play it.
Chris Romeo:Yeah.
Matt Coles:Well, so. We can't not play it here, let me let, let me, let me, let me flip this a little bit and maybe, you know, let's think a little bit out of the box. Throw the security aspect out the window for a moment. What makes developers want to be developers? Because developers are builders, not security people, they're builders.
Izar Tarandach:Yeah.
Matt Coles:makes them want to be engineers? What makes them want to be developers and programmers solving problems? And you invent and you incentivize it by giving them problems to solve.
Izar Tarandach:Ah, solving the problems that they want to solve. Not all problems.
Chris Romeo:he's bringing him, he's bringing us full circle. I see where he is going. can see two steps ahead. I can see where he is going. He's bringing us back around to the CTF and saying, CTF is solving problems. We just need it to be something that's tailored towards developers. so that they can, they can solve their kinds of problems versus
Izar Tarandach:But, but then Chris, goes back to the experience that you have with the, the training stuff and all that, right? When we started gamifying all that stuff and giving people snippets of code for them to go and, and do the right thing because we just. Show them not it. We, we know how that, that it works better than not having it, but that it didn't really get to people's imagination.
Chris Romeo:Yeah. It industry. The,
Matt Coles:there's a, well, there's a missing.
Chris Romeo:changed.
Matt Coles:a missing, sorry. There's a missing link there. Right? When you do that, you're just sort of, you're just, it's just the skills test, like within the scope of that training exercise for the developer. It's not the real world application, right? So how do you take that snippet, or how do you take that, that exercise, not the snippet itself, but the exercise and bring it back to the defense of the thing the developer just built, because that's where you'll get the full value from it.
Chris Romeo:GitHub needs to add locks. and every time somebody does something somebody can nominate them for a lock when you're doing your code review and then, the security and, uh, GitHub, that is pat hub, pen pat and petting trademark. Um, not really, but like, but I mean, imagine that type of a external of validation like GitHub Stars is a great validation for like, when you look at an open source project, if it has. Two GitHub stars, you're like, eh, maybe I'll find a different prep package to use. This doesn't look great. Imagine on your GitHub profile though, if you could have that where other people had recognized a, a positive security pattern and gamified it for you.
Izar Tarandach:collision, collusion,
Matt Coles:There's a, maybe there's an AI opportunity there, but, uh,
Chris Romeo:modeling guy that comes back and starts pointing out the flaws
Izar Tarandach:What? What do we do here? We have, we have three of those here.
Chris Romeo:I know. That's the challenge. That's the challenge. I thought I had a good idea, but All right, we're outta time on the security table for today. Um,
Matt Coles:We are making
Chris Romeo:like we solved. I think we, we made some progress in the right direction.
Izar Tarandach:Wait, we were trying to solve anything
Chris Romeo:Oh. I'm always trying to solve for
Izar Tarandach:Nobody told me.
Chris Romeo:Oh, I should have told you that upfront. But, uh, I think,
Matt Coles:you didn't capture the flag.
Chris Romeo:It was a good, it was this good and spirited discussion. I mean, I think that's the highest easer and my blood pressure has been during an episode a result of Matt's, uh, describing, think like a beep. Um, I can't even say it. I can't even bring myself to say it, but we'll continue our spirited debate because Matt tried it at home and Za is representing now a Star Wars bomb wearing So let the record show he
Matt Coles:Wait, wait,
Chris Romeo:Darth
Matt Coles:that Star Wars or Space Balls?
Chris Romeo:Oh, that's true. It'd be bigger. Nah, it'd be bigger space. He
Izar Tarandach:Prepare for low degrees speed. So everybody have a, a ludicrous Memorial day and uh, enjoy your weekend
Chris Romeo:Yeah, enjoy your weekend. Have a great time. Thanks
Izar Tarandach:I'll see you guys on the next one.
Chris Romeo:Yeah, thanks for listening to us on the security table. Thanks.
