Security Champions as the Answer to Engineering Hating Security

What happens when engineers transform into security champions? Is this beneficial, and what are the implications of this transformation? Izar reveals his transition from a naysayer to a supporter of security champions, and Chris and Matt seek to understand his current position. They explore the position of Security Champion and discuss the components of a good security champion program.

Matt defines security champions as developers with influence who can be a bridge between security and engineering. They receive advanced training and bring resources to their team to lead them to effective threat modeling. While security champion programs may have potential pitfalls, such as overloading team members, good security champion programs should benefit the individual and the business. Chris emphasizes the importance of providing opportunities for growth, learning, and networking to make the program appealing to potential champions.

With the potential issue of champions leaving an organization, they highlight the need for companies to keep up with salary expectations as champions grow in their roles. They also touch on the challenge of preventing security champions from being disliked by their team once they transition from being developers.

There are several resources for those interested in building a Champions program, including Dustin Lehr's Security Champion Success Guide and Chris Romeo's Security Champion Framework available on GitHub.

The episode concludes with a call for listener feedback and input, emphasizing the hosts' desire for an interactive and engaging conversation with their audience.

FOLLOW OUR SOCIAL MEDIA:

➜Twitter: @SecTablePodcast
➜LinkedIn: The Security Table Podcast
➜YouTube: The Security Table YouTube Channel

Thanks for Listening!

Transcript

Security Champions as the Answer to Engineering Hating Security

[00:00:00] Chris Romeo: Hey folks. Welcome to another episode of the Security Table. This is Chris Romeo, joined by Izar Tarandach and Matt Coles. And we're gonna talk about Well, we're kind of gonna make a connection to something that we talked about last time, but in our pre-podcast discussion, Izar shared it in such a crisp way. I was like, I'm gonna, I'm gonna hand the reins over to him and let him tee up this particular conversation for you.

[00:00:38] Izar Tarandach: So in a previous, I think one before the last episode, we spoke about, uh, do engineers hate security people. But what happens when we turn engineers into security people through that, uh, magic formula that, uh,

[00:01:06] Matt Coles: Oh, great. We're gonna get de-monetized. Right? 

[00:01:08] Izar Tarandach: No, it's like less than 30 seconds. But yeah. But uh, what happens if we turn engineers into security champions? Is it a worthy endeavor? Does it give what we think that it does? And disclaimer, I am known as the naysayer of the security champions. But, uh, through the good offices of people like our Chris and Dustin Lehr, I, uh, I sort of changed my mind a bit, so let's see if we can change my mind a bit more. What do you guys think?

[00:01:38] Chris Romeo: a, that's a big move too. I mean, it reminds me of a very quick story that we've already told once, and again. When Izar and I were… and Jim Manco gets up and he says, "Hey, I think threat Modeling's actually a good idea." And the, the camera pans to Izar and I, we fallen out of our chairs. We are laying on the floor. He looks at me and he wait, he, he literally did this. We didn't fall out our were he looked at me and he wait. What did he just say?

[00:02:04] Izar Tarandach: No. Did he, did he say what I thought that he said?

[00:02:07] Chris Romeo: I'm like, I think he said, I think he just said threat modeling's a good thing to do. And Jim had been very much a naysayer of threat modeling for a number of years, and so that was a powerful move. So, um, let's, let's dive into security champions though. And, and Matt, why don't tee up for us this idea of security champions. Let's just assume people listening, maybe they don't what we're talking about when we say security champions.

So give us a working definition if you would.

[00:02:31] Matt Coles: Right. So security champions are, uh, folks who we've, uh, we tap as security, as secure, uh, teams. Security champions are those that we tap for additional responsibilities and specifically to be a bridge between a security team and an engineering team. so these are folks who come, who, who should come from an engineering team who either have security experience or have desire to learn about security and act as a liaison a, a boots on the ground, so to speak, uh, between with the security team. of providing direction, uh, the security champions taking action in their local environment with their, with their, their development group and with their, you know, their product managers and others, to implement the security program, whatever that is. So I think that in a nutshell is what, what we generally consider as for security champions. and you know, a whole gamut of the people who, who end up becoming security champions, right? They get, ideally they come from engineering team, right? They, they're technical folks who have not only technical capability, but also influence. You know, influence among their engineering engineering org so that they can, for instance, you know, uh, educate people on the value of threat modeling and, and execute on that as an activity without necessarily having to pull in the security team.

Although you could, they can, and, and as a liaison, they, they should be able to bring in resources. but we generally train them. They get advanced training, they get, uh, you know, first crack at tools or, or you know, new, new techniques or whatnot, um, so that they can, they can take action within their environment doing all the local coordination, right?

That's how, I think that's generally what we consider across the board here. Uh, security champions.

[00:04:28] Chris Romeo: I think that's a good definition of security champions and, and I'll, I'll expound a little bit more on the, why do we need security champions? Because I totally agree, Matt, with what you're saying about kind of what, what they are, what their job description or kind of focus is. I think it's important for folks to understand why do we need security champions because. And, and the fact that I always go back to, there's one thing that I look at in the BSIMM report every year, when it comes out. It's only one thing. The rest of it I just throw off to the side. I'm always curious to see what's the ratio between the number of developers to the number of what they call software security group members.

Now, I don't, I hate that term software security group, like I don't wanna be an SSG, but it's always an interesting metric because then I always have to remember. But wait. These are the companies that take security and AppSec and software security so seriously that they'll spend a lot of money to be able to do to, to get, be assessed by a commercial assessor who's coming in and going through a formal review and them generating a report and a score and everything for them. And so BSIMM 12, it was 135:1 developers to software security group members, that's for the people that really take it seriously. So it makes me wonder, is it 200 to one? Is it 500 to one in different companies? It probably is in, in some of the bigger companies that exist there. And so for me, champions exist as a bridge between my security team and my engineering team because I'm never gonna be able to resource. I don't care what anybody tells me. And there are people that argue with me about this to say, yes, you can resource a team big enough to put Security team members and distribute them in engineering. I've never seen a company where there was enough room in the budget for me to hire a hundred people just to be my connecting connective tissue into engineering for AppSec.

[00:06:22] Izar Tarandach: Especially in nowadays.

[00:06:23] Matt Coles: never.

[00:06:24] Chris Romeo: It's just, 

yeah, it's So, 

[00:06:26] Matt Coles: Never. Well, and and in fact, actually, Chris, just to add to that, in our space, well, I, I'll say mine and is our right word. Well, pre, previously, I guess is our view, uh, with product security, there aren't that many there. There probably aren't that many people in a particular geographical location, now we're doing remote. Even still, there just aren't enough of us. Right. We have a cybersecurity skills, you know, a workforce shortage in general, a hundred people for a security team. Outside of just that role is a challenge. So, yeah, it's a, there's a scalability problem there.

[00:07:01] Chris Romeo: So I'd to hear Izar, your journey from being a naysayer to

[00:07:07] Izar Tarandach: A slightly less naysayer.

[00:07:08] Chris Romeo: ...starting to think that could maybe work because to, unpack, I wanna unpack the, the challenges that you had with security champions and that's a, that's a great way to explore this in more depth, because for a long time you were the one saying, this doesn't work because of X, Y, and z.

[00:07:26] Izar Tarandach: Yep. So much like Matt's definition, I find that whenever we start talking about security champions, somebody very quick jumps up and says, the uses the expression force multiplier. I. So there aren't enough security people out there. Software is hitting the world. We need more people into more software. Hey, let's get people who are already in the software, turn them into sort of security people, give them training, give them belts, and these people are going to act as force multipliers of the, uh, of the security people who are embedded into these teams.

So first of all, my, my initial reaction is, okay, so we are asking people who hate us to turn into us.

[00:08:28] Matt Coles: Oh, it gets worse than that. Let's talk about that in a bit, but go ahead,

[00:08:31] Izar Tarandach: I need to give them a damn good incentive to do that. The incentives that I have seen out there, frankly I have not found a single one that . If I were an engineer would cause me to do that. Couple of.

[00:08:51] Matt Coles: can I, actually, can I, can I just jump in here? 'cause I, before I forget the thought, it isn't there, there's, do you, do you find in your experience as you're in your journey here, uh, that it isn't just that you're asking engineers who probably hate the security team initially become Security because their role changes if they become a, a security champion, but often they're "volun-told."

[00:09:21] Izar Tarandach: Yes, that's, that's, that's another point that goes when the incentive is not good enough and somebody jumps very quick, somebody is told, now it's you. many times that happens. I don't have numbers, but from observation "anec-data," as they say, it happens to the least influential . member of the team. Exactly the person who does not have the voice to push the initiatives that we are asking for.

[00:09:53] Chris Romeo: Yeah,

[00:09:53] Matt Coles: right?

[00:09:53] Izar Tarandach: On top of that, wait, wait, wait, wait, wait. There's more

[00:09:57] Matt Coles: That point 

[00:09:58] Izar Tarandach: On top of that, when we do find somebody who is influential enough, that person usually comes already with a big workload, and now we are putting them in a tug of war where we say, I want more of your time for security.

You still have to do all the stuff that you were doing before. I'm not giving you any kind of incentive, reward or otherwise, and by the way, you are going to be accountable for this.

[00:10:21] Chris Romeo: Mm-hmm.

[00:10:23] Izar Tarandach: I've seen people burn out and leave for less.

[00:10:26] Chris Romeo: Yeah. And let me, let me give you as, as someone who's been a big champion proponent over the years, I ran Cisco's security advocate program for about five or six years, turned it into a global conference series where we had internal, AppSec product security conferences happening in the US and San Jose with 1,000 people there and in Bangalore, India, Shanghai in, uh, London.

Like all these things as somebody that's, that's kind of been down this road and, and seen what it took and tried some things that didn't work and tried some things that did work. The biggest takeaway that I came, that I came up with is you have to make the Champions program about the champions. You have to think about it, what's in it for them? So many champions programs that exist out there try to be force multipliers, but they think about it in the terms of the business. Okay, Matt, you're gonna start a security champions program. And so what you, you tell me as your leader, what's in it for me as the business? What am I getting out of this?And they build their programs according to that structure. What happens is you end up with this program, that, to your point, there is nothing in it for the developer. The engineer other than, uh, my arm got twisted, I gotta go be a champion. 'cause my boss told me I have to be, and this sucks and whatnot because we didn't flip our mindset around and say, okay. Let's build a champions program. is gonna be attractive to developers, where they're gonna go? I want to be in that, and I can tell you this, after doing this for five years at Cisco, I took the, I took a dying Champions program from 20 people to about 500 when I left. And the way I did it is I created something that people wanted to be a part of. They, I went from, in the early days walking around the campus in San, uh, in around San Jose in various Cisco buildings, knocking on cube walls saying, Izar, will you be a security advocate? Um, it's gonna be really cool, getting a small group of people to do it to before right before I left Cisco, people were, we, you had to, you had to go through a series of requirements to get in.

[00:12:29] Izar Tarandach: What, what?

[00:12:30] Chris Romeo: You couldn’t just call me and join the group because we had matured to the point where people saw it as, I wanna be one of those security advocates. I wanna be associated with that brand. Those are people that are making stuff happen here inside the company. They're getting, uh, enhanced training and opportunities. I got to watch a number of people that were champions become security professionals. They left the world of engineering to become security professionals.

[00:12:52] Izar Tarandach:  Oh, that, that's, that's my, that's my next point. But first of all, why is it that I, I hear vibes of, do you want to be in Glee club? But

[00:13:02] Matt Coles: Not Fight Club?

[00:13:03] Chris Romeo: I can’t sing. I can’t sing at all. So not for me. 

[00:13:05] Izar Tarandach: but let, let, let, let. Oh, we don't talk about it. We don't talk about that one . But let, let's talk for a second about that. Uh, uh, they moved to security people. So that, that's the thing that, that's the one thing that I found that for some people at the end of the, the road, there was some attraction in there.

People who would come to me and say, listen, I'm really interested in, in starting a career in security, but I have no clue where to start. And then my next, my next question would be, would you consider start as a security champion? That makes absolute sense, but consider their journey At the end of it, they are looking already at an escape point.

So by definition, when you get people to the level that they can actually be really, really, really useful. I'm not discarding the, the, their, their usefulness before that, but really, really, really useful. When they can really be a force multiplier, they'll either join the security team or they'll pop out and go be a security person somewhere, somewhere else.

[00:14:08] Chris Romeo: This was a percentage, right? I had a percentage, a small percentage of people that went all the way through the program through a master's degree that we set up at San Jose State that was tailored for Cisco for the security advocates team. I had a handful of people that went through that and became and, and transitioned to security. I would say that the way that I approach it for the average developer is today when I'm recruiting a security champion, the way I, I sell it to them because it is a marketing and sales move. Let's just be honest.

[00:14:36] Izar Tarandach: It is. It is.

[00:14:39] Chris Romeo: You're a developer now, you're a senior software engineer.

You're good at writing code. If you layer the application security knowledge and skills that you'll get as a champion with that, you're in the top 5%, maybe the top 1% of your industry. 'cause you're a developer who understands AppSec. You're not an AppSec person who's a developer. And when somebody gets to that level and they can achieve that level of knowledge, I mean, I just tell 'em. a, you're becoming a, you're entering unicorn territory right now as a developer that's got that foundation, that doesn't wanna be a security person, but understands the principles that we all and talk about all the time, and try to get people to adopt and prescribe and adapt.

[00:15:17] Izar Tarandach: But, but, but Chris unicorn here is, is a good. Good label because we are talking about this myth create, uh, creature that a lot of people don't believe exists, and most of them are not looking for.

[00:15:33] Chris Romeo: So

[00:15:33] Izar Tarandach: spoke about this in the, in the past.

[00:15:35] Chris Romeo: what I'm describing though is, so throw away unicorn. I threw that in that, that, that's that. term, but think about it. Let me

[00:15:42] Matt Coles: They come special.

[00:15:43] Chris Romeo: a dev, if you have a senior software engineer, Who has spent a few years learning about AppSec, but still just wants to write the best code on Earth and build cool products, they at the top of their industry as a software engineer, a software engineer you, so you don't think a software engineer with AppSec knowledge and skills is better than a software engineer who knows, doesn't care about or know about security.

[00:16:05] Izar Tarandach: If you measure being at the top of their, of their industry by how wanted they are, by potential employers. We have already, uh, uh, agreed in past episodes that employers are not asking people, do you know, AppSec? And they don't consider that an added value when bringing engineers in. So it might make them personally a, a better engineer because as we agreed in, per in previous episodes, security code, a secure code is quality code.

So yeah, they're going, they're going to be to be writing better code. Is that going to give them a leg up when they go and interview? Questionable.

[00:16:43] Chris Romeo: I mean, I think it only gives them, it's not a negative.

[00:16:46] Izar Tarandach: It's not a negative. Definitely.

[00:16:47] Chris Romeo: not 

gonna take 

[00:16:48] Matt Coles: Negative hit. 

[00:16:48] Izar Tarandach: Totally.

[00:16:49] Chris Romeo: I would say a progressive, security first company. think we're gonna see more of this. This is gonna become a bigger deal over,

[00:16:57] Izar Tarandach: That's what I hope for.

[00:16:58] Chris Romeo: all these regulations and all these documents that are coming out that are gonna do nothing but drive security into the forefront of everybody's mindset.

And so,

[00:17:07] Izar Tarandach: completely that that's what I would love for,

[00:17:09] Chris Romeo: I think this prepares these to and once again, I'm, this is my sales pitch to them. This is what I think they're gonna get out of it is

why should I be a security champion? Well, we're building a program that'll help you to move and grow and be, and, and improve your career over a period of time.

[00:17:26] Izar Tarandach: but look.

[00:17:27] Chris Romeo: a real benefit.

[00:17:28] Izar Tarandach: It, it's, it's a bit equivalent to somebody asking, why should I have a, uh, an MBA? It's something that's years in the future, you're going to be trained for this. It's going to take time. You, you'll be there at the end.

[00:17:40] Chris Romeo: Yeah.

[00:17:42] Izar Tarandach: On the way. They're going to have to pay their dues.

Is that carrot sweet enough to dangle in front of them to justify ? The fact that on those years they're going to have that tug of war between their developer work and now the security work. They're going to have to, let's be honest, go through the whole horror cases that we as security people go and that we spoke about.

Getting time, getting resources, convincing people casually, people asking people, imploring people, doing all that good stuff.

[00:18:20] Chris Romeo: Mm-hmm.

[00:18:21] Matt Coles: By the way, that's what we expect of them. That's the, that's the tasks that we push to them and an organization that has implemented security, champion program is backing that up with resources with some amount of resources, whether that's direct support, handholding, training, tooling, Right? Ideally it's not a throw it over the wall, although I imagine that happens sometimes.

[00:18:49] Chris Romeo: address, to Izar, to your direct point, I would say to use a term you used earlier, I have anec-data to say that yes, it does. It's not a, it's, it's not, I don't have industry-wide data. I haven't been able to apply this model to 10 different companies yet and see if I can get the same result. I did it with one that was a, that's a big company.

That tends to be a challenging case study because

[00:19:11] Izar Tarandach: Yes.

[00:19:11] Chris Romeo: it's engineering first. Engineering, at least in the days when I was there at Cisco, engineering ruled the place, like you didn't tell engineering what to do. You could, you could suggest sometimes they would laugh at you. Sometimes they would say, that's a good idea, but they had that type of power and so, so, but yeah, it's anec-data at this point.

It's not a, uh, it's not a, a, a model that I can say is can't prove that that is the case everywhere. But I think over time with what attention we see with champions programs, I think we're gonna have more and more anec-data. As people are getting into this and now they're starting to talk about what's happening in champions programs, think we're gonna see if there's, if there's some reality here or if, if I, I got lucky,

[00:19:54] Izar Tarandach: So,

[00:19:54] Chris Romeo: I did it.

[00:19:54] Matt Coles: And I just, and I just wanna highlight, you know, we've been talking about champions primarily from an engineering stand, from an engineering standpoint, from a developer standpoint, I do wanna highlight that. Strictly, we've been talking about software, but I do wanna just highlight for folks who may be interested in, in learning more about what security champions are, either within your own, own organization or, or just in, in industry in general. It also works for hardware, it also works for vulnerability and incident

[00:20:23] Izar Tarandach: Yep.

[00:20:23] Matt Coles: Vulnerability. I'll say vulnerability response. I'm not sure about incident response, but, but certainly would work for those, those areas. So not just development, software development, but other aspects of, of system development, hardware, software, and then both pre and post-release.

[00:20:40] Izar Tarandach: So, where am I in in that journey right now? Where, where did you guys get me to over the, the last few years? I think that I have abandoned the idea of having a bunch of minis and I have adopted the idea of having a bunch of minions, so to use the word that Matt used earlier, the liaisons. My thing now is security liaisons for that force multiplying thing I want in every engineering team, somebody who is a point of contact.

With the security team in the sense that I'm reducing the workload for the security team to have one point of contact where they can go with their questions, where, where they can go with their requests. What's going to happen internally in the engineering team. It's their thing. I'm, I'm transforming that, that team into a black box.

I have one single interface. I go to the person and I say, this is what I need to happen. Can you help me make it happen? If it's a question, they'll know way better than me, who's best equipped to answer if it's a request, they know way better than me where the team is, and if the team can stand by the request or not, by not having any, uh, preordained expectation of them rising their own security knowledge, security minded and security abilities.

I am not putting an undue load on top of them, but at the same time, I am prodding them and I, and I am giving them enough material to understand why I'm asking the questions that I'm asking, to understand why the things that I'm asking for are better than not having them. Right. So in separate from them.

I find those people that want to be mentored, that want to be taught, that want to, to be challenged, to go and learn more security because perhaps those are the ones that, the carrot of having that thing at the end of the couple of years tunnel is interesting.

[00:22:57] Chris Romeo: So I guess let me, let me ask a couple of clarifying questions, just 'cause I'm curious. I haven't, I haven't heard you lay this out like in, in such a way. So I'm, I'm really excited to hear kind of your, how, how, how your thought process has moved along, but also how you see this working in, in, in a, um, let's leave the, the, let's leave the small startup behind and let's say, let's just pick a, a, let's say we have a thousand engineers inside of a company. How does scalability work from a liaison perspective? Because it sounds like you still have the security team doing more than perhaps I would prescribe

[00:23:34] Izar Tarandach: Yep.

[00:23:34] Chris Romeo: … for what I think of as a champion. So for a, if I, if I have a thousand engineers. In your mind, how big is the security team and what is the, what's the role and scope of the security team working with the liaisons?

[00:23:48] Izar Tarandach: So here, here it is. It's not as much the number of engineers, it's the number of engineering teams or projects or units of work or whatever it is. My point is that security team, one of the, and again, anec-data, but I think that there is a body of research to support this. I think that one of the biggest reasons for burnout in security people is the constant context switching.

We have to serve all these different projects at the same time. We have to . Take that stack that has, what's the project? What are they building, what could go wrong? Who are the contact people? Who's responsible for what, who's against, who's for how much time? They, they give me, is the product manager somebody who I can sell security or not?

All, all those, those variables that define my interface with that project, I have two, three times a day, four times a day change that, that stack in between the projects that, that I'm serving.

[00:24:48] Matt Coles: And to add to that, by the way, do I have the skills either as an individual or within the team for working with the technologies involved?

[00:24:56] Izar Tarandach: Yeah.

[00:24:57] Matt Coles: Because those are not necessarily fixed in any given point in time, or there may be bleeding edge.

[00:25:02] Izar Tarandach: And to add to that, where's this team located? I'm, I'm in the US, they're in Europe, they're in China. The difference in time between us, I have to get everybody now in a meeting. How do I get one meeting with ten people? Right? The moment that I have that gofer in the team, all that gets removed and I can think about security.

I don't have to think about all the trappings around. Having that line of communication open and well maintained, that moves to them all that. Who is the person in the team who's responsible for the why? Who's responsible for the backend? Uh, and has that changed this week? All those internals of the team, they, they become opaque to me.

Now I have a question about the UI. I have a question about the backend that person's going to, to route that to the, the right answer to, to the right person, person who can answer and bring that to me. So it's not as much as I am giving technical, I, I am able to cover more technical issues as a security person,

I am able to do more because I am doing less in other things that are less related to my job as a security person. So

[00:26:17] Matt Coles: So can I challenge something here? So we started this with the definition of threat of a security champion to be an influential member, uh, tech, a technical influential member of the organization who can help drive security from within.

[00:26:34] Izar Tarandach: yep.

[00:26:35] Matt Coles: But I, I think I heard the way that you just described how you utilize such an individual, a program manager or a scrum master or a, or a, a team, you know, a team lead. Could all, any of them could do that, potentially even an admin could do some of those things when, like scheduling and whatnot. So do you really need technical, influential member of the team?

[00:27:00] Izar Tarandach: That's, that's not a challenge. That's a really, really, really great observation. The point now is, remember, I've removed all my expectations of security learning from that person. Now, the people who are coming to me to learn security, to be mentored are people who really, really, really want it. And if I have them in as many teams as I can, that's awesome.

[00:27:23] Chris Romeo: So I’m still, I’m still, I, I, I appreciate the con additional context you provided in that, but you didn't answer my ultimate question, was how many people though, because from a scalability perspective, we have to, we have to test this model to say, is it reasonable?

[00:27:39] Izar Tarandach: But that's the thing.

[00:27:40] Chris Romeo: So, but how many people on the, like if you're, so I get your security team members are now focused, they don't have to task switch all the time, but how many for a thousand?

If I have a thou Okay. I'll give you a, I'll, I'll, I'll extrapolate my, let's say we have, we believe in eight people per development team and we have a thousand engineers. So we have roughly what, um, how many teams? A hundred. A hundred and Say we have a hundred

Make it easy. How many, how many security, how many people do you need on your AppSec team to liaison with a hundred teams of eight people inside the

[00:28:11] Izar Tarandach: so again, I, I don't have data. I wasn't able to put these ideas 100% to work anywhere. I'm looking forward to doing it, but right now I don't have any data to back this up. The only thing that I can offer is as someone who sat in that place in a security team, I remember that way more than I wanted of my time was spent doing the that, that gofering, rather than pointing at threats and mitigations and risks and whatnots. So if you have one person for every 10 product teams out there, that person is getting burned out right now. And by doing this, you manage to reduce the load on them by a factor of three. All of a sudden you have one person for 30. 30, uh, uh, teams.

[00:29:15] Matt Coles: but, but I mean Izar, you could solve this problem, I think, also through a judicious use of program program managers, I'll say program managers as a role type, but that isn't necessarily that within the security team.

[00:29:31] Izar Tarandach: Yep.

[00:29:32] Matt Coles: other words, if the security resource, expert is having trouble scheduling with a team. You have a program manager in the security team who can do that liaison You don't need the engineering person. you, they'll, they'll reach out to the or within their system. So it can work from both either end, but you're not really solving the problem, are you? Right, because it's the, the other part of security champions, I think is to make businesses, the engineering teams, the business more self-sufficient. Right. So today you're talking primarily about how do you facilitate security to do work with the engineering teams and facilitating conversations and, and, and getting, getting a foot in the door and not, how do you elevate the role of security so that they're not pressuring engineering all the time and engineering can bake stuff in on their own or, uh, uh, adapt, you know, guidelines or, or standards to the work that they need to do. Organically within the team.

[00:30:35] Chris Romeo: I mean, guardrails and paved roads we've talked about already, like that, that plays into this too.

[00:30:40] Izar Tarandach: That plays into, into this a, a huge, huge, huge way. Okay. Now, Matt, to your point of the, the, the manager, true. You can, and I think that's even when we were working . Every day together, we, we had the opportunity of seeing many very good program managers come in and out of the PSO.

[00:31:04] Chris Romeo: Hmm.

[00:31:04] Matt Coles: Yep.

[00:31:05] Izar Tarandach: It's not an easy job. It's not everyone who can do it well, and it's not everyone who wants to do it that way.

Not many program managers that I know. Dream of spending most of their times looking at a Rolodex and constantly updating it because developers come and go very quick. Teams change very fast. Projects change very fast. So my point is, when we, when we promote threat model, we say that we have to collaborate with the team because they know the product way better than we ever will.

So I'm translating that into this and saying people in the team know the dynamics of the team much better than we ever will,

[00:31:51] Matt Coles: Mm-hmm.

[00:31:51] Izar Tarandach: right? So I, I think that by separating these two, uh, um, roads, the one that serves the business with the security liaison and the one that, let's be honest as Chris put so well serves the people because they improve, they become better engineers by learning more security and eventually, who knows, becoming security or anything like that. I, I think that I, again, I, I can't speak from experience because I haven't tried this yet, but my gut feeling is that there is value in there. It lets us focus on, on both sides of the conversation much more.

[00:32:25] Chris Romeo: I think there's value in both, both sides. I, I, I think the security liaisons and the security champions as you kinda what you've described versus what I've described, will fit different companies in different ways. There are some engineering cultures that security liaison is the only way it's gonna work.

[00:32:41] Izar Tarandach: Yes, 

[00:32:42] Chris Romeo: But there’s also some where you, we can have a more, more proactive approach to everybody inside the organization. And I wanted to share just a little, little, some more anec-data, because it does kind of play into your liaison idea here though. So as the Cisco evolved, a role was created called a security account manager, a SAM.

[00:33:08] Izar Tarandach: Mm.

[00:33:09] Chris Romeo: And each technology group in the beginning was assigned a SAM, and they became almost the reverse liaison to what you're describing. But I think this is complimentary to idea that you're kind of working on. I. They were responsible for connecting with the executive levels of a particular slice of Cisco.

'cause Cisco was technology groups, business units. They were responsible for connecting with them, but also for being a resource for liaison purposes, up and down the stack across all the different teams. And so that was a, an evolutionary progression that we made beyond our security advocates team because we, we, we started to realize, There's more to this than just people, teaching people, and getting them fired up about security. There was also a liaison side that was required now that we ended up doing it differently. And I didn't design that. I'm not, I'm not, I'm just reflecting back on what I saw happen in front of me. But it was very much a, a, it was a liaison-like relationship that came out of that Sam, um,

[00:34:17] Izar Tarandach: Look at, at the end of the day, I, I think that . The problem, the main problem that we're trying to solve here is efficiency, right? We're trying to make the security process efficient. Now. We've been trying for a long time to do it with security champions. The problem is that even though it's a good idea, and eventually successful with programs like what you described, it takes a long time to get the return of investment until that champion can become somebody that you can say, that guy force multiplies me.

He's a mini me. It's gonna take a lot of time and pain. On the other hand, something like a security liaison starts giving you return of investment on day one because the, the, the efficiency now, the, the, the thing that you, the friction that you're trying to reduce sits in the security team and now you're giving them more time and, and a, a, a bit of a calmer environment to work on.

[00:35:17] Matt Coles: Why can't it be? Why? Why can't that be a dual role though? Why can't the security person, the security liaison, be the champion? So in

[00:35:25] Izar Tarandach: Totally.

[00:35:26] Matt Coles: start with basics, and then, and then they, as they grow in capability, the way, I, I, I, I appreciate what you're saying about mini, the mini me. The question is, did they become equivalent you? Because that's where you reach ma, maximum efficiency, right?

[00:35:45] Izar Tarandach: the moment that you turn to them and say, we have an open position on security team, are you interested in joining us?

[00:35:51] Matt Coles: they say, no, I wanna stay in development.

[00:35:54] Izar Tarandach: I have never heard that,

[00:35:56] Matt Coles: I know you haven't, that would be peak efficiency, right?

[00:35:59] Izar Tarandach: that that would be, that would be. Yes.

[00:36:02] Matt Coles: Right. And they become, they become basically security within the

[00:36:06] Izar Tarandach: Yep. Yep. I, I, I think that both of us remember a certain, uh, uh, uh, what, what was he? He was an engineering manager that worked with us more, more with reviewing a certain product, and over time they started developing security testing by themselves. They started asking questions. They started answering questions.

Until one day they said, I'm leaving this place and I'm becoming a security professional. And we told them, good luck on your way and, uh, here join our slack. That's, that, that, that's a great, that's a great ending. But it's one of those, what happens if we train them and they leave? Or what happens if we don't and they stay?

[00:36:48] Chris Romeo: Yep.

[00:36:49] Matt Coles: Well, I, I, I guess that's,

[00:36:50] Izar Tarandach: Who, who is it that said that? Just to give credit. Thank you. Yeah.

[00:36:54] Matt Coles: there, there is, right? Well, there's that, but also, I mean that you, you want to train them if they le if they leave right. That's on the business for failing to, to meet their needs. Right. But you've trained them. You shouldn't, you shouldn't train them to stay.

[00:37:11] Izar Tarandach: Ha, shouldn't you? Because if you say you should lead, you should train them to leave or not train them to stay.

[00:37:22] Matt Coles: guess, I guess that's not,

[00:37:24] Izar Tarandach: No, no, what, what you're telling me is that there is already value on the fact that you're training them, which I do not dispute. I think that people will learn, will keep learning, and as they learn, they'll be able to apply that.

[00:37:37] Chris Romeo: Yeah.

[00:37:37] Izar Tarandach: But finally you got that polished, almost security engineered person and next thing you know, you are drinking a beer with them because they're leaving the the business. Wouldn't you feel a bit cheated

[00:37:49] Chris Romeo: I mean,

[00:37:50] Matt Coles: maybe.

[00:37:51] Chris Romeo: it comes down to your, how you approach the world though. Right. And I would, I think, you know, the three of us, we've all, we've, we're always looking for ways to, to grow people, to mentor people, to build them up. I don't take it as an offense when somebody, leaves or transitions

[00:38:06] Izar Tarandach: an offense.

[00:38:07] Chris Romeo: becoming a security person.

Like, it's, it's, I'm happy for them and, and it's, it means I did my job as I, if I'm a senior member of the team, be looking to invest into people, grow them, mentor them, and hopefully they get, they, hopefully they go to bigger things in the company I ever do.

[00:38:22] Izar Tarandach: But, but Chris, that that's, that's because we are, although there is a stereotypical security person, and we spoke about that in the past, we try to be good people. So we connect to people on the people level. And when somebody comes to us and says, I want to grow into this, I consider going into a security, uh, uh, function, growth. We want to support that growth. We want those people to, to be the best that they can. But at the same time, we have to be honest and say we are in business. And the business gave us a mandate to train people for security so that those people could serve as security people in the business, not in the next business. So we are doing a great job as person, but not as business people.

[00:39:02] Chris Romeo: The way I always describe it, I, I, I have a bit of a jokey way of, of describing it to people. I'll, I'll, I'll be talking to a developer and I'll say, once again, selling them on the benefit of becoming a champion. Like, Hey, you're gonna learn all these new things. I think it's gonna make you a better developer at the end of the day.

And, and I expect you're gonna work here for a hundred years, but just in case you don't, and they always do same thing. I, I, it always gets a chuckle, but it, it allows me to communicate a subtle point that people move around in jobs like I can't advocate, like, I hope you leave. But it's a reality. It's, it's if you grow your skill, a lot of times in, in big companies, and we're going off on a whole other tangent, we gotta wrap this thing up. But if you start as a junior member of staff, you can, you never get promoted and salary bumps equivalent to as if you join the company as a senior engineer.

[00:39:54] Izar Tarandach: Mm-hmm. 

[00:39:55] Chris Romeo: They just never, I haven't seen a company yet that does it. And so you have to leave to take that

[00:40:01] Izar Tarandach: Yep.

[00:40:02] Chris Romeo: to get to an equal level based on your experience. And its terrible that corporate America can't find a way to keep up with people because you do end up in investing in these people as champions, as liaisons. You're pouring into them, you're mentoring them, and then like you said, you're enjoying a beer at their going away party, but it's really not their fault or your fault. It's the organization's fault for not having a way to keep them at a salary level. That's equivalent to how they've grown. so we've just identified one of the biggest problems in HR, and so welcome to the HR podcast. We're shifting gears now. We're, we're a whole new podcast about HR.

[00:40:38] Matt Coles: So I wanna, I wanna raise two points, but I'm in the interest of time. I'm gonna only raise one and we'll, and we'll table the, the cycle one for, for maybe a later, later episode, bringing it back to what we were talking about last week. How do you, how do you, how do you not get engineering to hate security? Here's the question I wanna pose to both of you. If you identify a member of an engineering team, whether they're a junior engineer or a senior engineer and, and a, you know, influential within the group they learn and they take on that responsibility, they graduate, we'll say, graduate from a champion a, to a security leader within the organization. Because they were a developer and now they've become security. How do we keep them from being hated by their own team?

[00:41:31] Izar Tarandach: Wow. Okay. Okay. Okay. Okay. So, so, yeah. Yeah, yeah, yeah, yeah. Yeah. So, so, so, real story. Real story. Okay. I had this guy defined as a security champion, every sprint, he would wait until the end, raise his hand and say, okay, can we talk about security? Now that lasted for a month. After a month it was okay. Are we at the level now that, uh, Johnny Peloni will just, uh, say, can we talk about security now? And I saw that team turn into like, you're not one of us anymore.

[00:42:08] Chris Romeo: Hmm.

[00:42:09] Izar Tarandach: You are now the security guy here. I wrote the line of code. Can you come and read it and make sure that it's secure? So yeah, it's a thing. It's a thing. We may be alienating people when we elevate them to the, to that, uh, sorry, not elevate, but when we put them into that role.

[00:42:28] Matt Coles: We categorize them. 

[00:42:29] Izar Tarandach: Yeah. And, and, and, and it's, it's, it's, it's a true thing. We, we are, what's the term, otherizing them because we are making them all of a sudden developer plus.

[00:42:38] Chris Romeo: Yeah, that's true. That's true. That is something to think about. Let's, uh, wrap up our conversation, our first conversation we've ever had about champions. I'm sure we'll have some more in the future. Um, I wanna plug a couple things we mentioned Dustin, I think if we haven’t, Dustin Lehr. Uh, he's written Security Champion Success Guide. Check that out. You can just Google for it. It's, uh, it's a whole site. He's put a bunch of his thoughts into it, into what it takes to build a Champions program. Um, I've also released a Security Champion Framework.

[00:43:05] Izar Tarandach: Mm-hmm.

[00:43:06] Chris Romeo: Izar already commented on it. Uh, but that's out there on GitHub, Creative Commons License. So you can go tear it up, throw it away, add to it if you want, you wanna do. But it's a, uh, another couple of resources out there as you're thinking about Champions programs. And once again, thanks for joining us on the Security Table and the HR podcast, which could be coming soon.

[00:43:26] Izar Tarandach: And people, it's getting cold and silent around here, so if you have something to say, please let us know. We have, uh, so many places where you can go and write your opinions and your ideas and, uh, question us and tell us that we have no idea what we’re talking about, and we would love to have your input as well. So please go there and do that.

[00:43:42] Chris Romeo: And that's a great way to leave it right there. Thanks folks.