The Security Table
The Security Table is four cybersecurity industry veterans from diverse backgrounds discussing how to build secure software and all the issues that arise!
The Security Table
CVSS 4.0 Unleashed with Patrick Garrity
Patrick Garrity joins the Security Table to unpack CVSS 4.0, its impact on your program, and whether or not it will change the game, the rules of how the game is played, or maybe the entire game.
FOLLOW OUR SOCIAL MEDIA:
➜Twitter: @SecTablePodcast
➜LinkedIn: The Security Table Podcast
➜YouTube: The Security Table YouTube Channel
Thanks for Listening!
Hey folks, welcome to a live edition of The Security Table. This is Chris Romeo, and I am joined by my normal cast of characters, and we have a guest, but Matt Coles is here, and the man who only needs a first name on virtual platforms, he doesn't actually need a last name, he's like 8
Matt Coles:He usually goes by IT.
Chris Romeo:He usually does answer by IT or any other number of names. But, uh, yeah, so we're super excited to have Patrick Geraghty with us today. Patrick, our audience probably doesn't, some of them probably know you, but, uh, give, give just a brief background about who you are and kind of what you do.
Patrick Garrity:Yeah, so I'm hanging out. This is my office. Turned it into a skate shop. I've been in cybersecurity for the better part of the last decade. Started my career transitioning out of a managed service provider. So lots of hands on experience in IT. And then, um, went and helped build Duo Security on the sales engineering side, go to market side, moved to Europe. And then, um, we got acquired by Cisco. Uh, spent some time helping, uh, Census, Spinelli University of Michigan, helping do discovery, and then, um, worked on a company that, uh, called Blumira that does detection and response in the, in the SIEM space for small and medium sized business, and then, more recently, about two years ago, I got into the vulnerability management space, uh, by joining, uh, Nucleus Security, and, um, combination of help with, uh, you know, anything and everything, but more recently, over the last year, spent a lot of time in the security research side of things. Thank Um, which, you know, led, led me to, uh, some of the work on doing data visualizations with VulnData.
Chris Romeo:And that's what we were admiring before we, uh, clicked the button to go live here. Your, uh, that, that, uh, multi, multi colored picture that's kind of sitting over your shoulder right above there. Um, what is, so, so very quickly, just tell us what that is.
Patrick Garrity:Yeah, so this is the, uh, CISA, Cybersecurity Infrastructure and Security Agencies Known Exploited Vulnerabilities list. So, at BlackHat, it was right around a thousand vulnerabilities, and, um, I just want to drive awareness that, like, everything is vulnerable. Um, and so... I spent some time doing tree maps and then did some overlays to create a data viz that kind of shows, like, regardless of what company you are, the products in your environment, um, are vulnerable. And so this kind of shows Microsoft and Oracle and Cisco and Apache and you name it, pretty much every company has vulnerable products. And I think that's a really important narrative, especially when we're talking to like executives or stakeholders that don't understand security. Um, when we're talking about vulnerability management, right? So, uh, that, that was a big part of, uh, you know, why I intended to do the work. Um, and it, it kind of took off. So this is cool.
Chris Romeo:Very cool. Yeah. So topic we wanted to explore today is CVSS. Specifically the 4. 0, the new version of CVSS, but I think it'd be beneficial to lay a foundation first before we dive into, like, what are the differences? What is 4. 0 doing for us? Let's not assume that everybody who's watching, listening, knows what CVSS is or how it applies to their world. So, Matt, why don't you set that up for us since I know you've, uh, done a little bit of thinking about CVSS. Yeah, I've,
Matt Coles:uh, I've been involved with CVSS for a while, uh, as a user as well as a member of the SIG. So CVSS, the Common Vulnerability Scoring System, is a production of FIRST, the Forum for Incident Response Teams. You know, so incident response and security teams. So first. org. Uh, the SIG, um, put together, uh, now a total of four versions of, well, more than that, but, uh, certainly four that are in popular loop popular. Popular use. Uh, fun morning this morning. So, uh, CVSS 2. 1, which many are probably familiar with if they've seen anything in the National Vulnerability Database, uh, from earlier years, and then, uh, 3. 0, 3. 1, which is the most recent in In release version, in use version of CVSS, and then more recently 4. 0 that launched in, in beginning of November of this year. So basically the Common Vulnerability Scoring System, for those who aren't familiar, is a method for determining the likelihood of exploitation and the, uh, relative impact of exploitation. Looking at severity, not risk. So very important. If you take any given issue, any given vulnerability and you try to, and we're talking specifically vulnerabilities here, not weaknesses, but vulnerabilities, looking at what's its severity relative to other vulnerabilities. You can use the CVSS calculator to generate a score from basically 0. 0 to 10. 0. where obviously 10 is critical, zero is, is not, uh, and, and then everything in between low, medium, high, critical on the scale, uh, and it's, uh, now the de facto standard or, or the current standard that's in use for the National Vulnerability Database. Many of the security scanners, vulnerability scanners will, will reflect CVSS scores. Um, you know, SAST and other tools as well. Uh, and, and so it's a generally recognized, uh, method for, for determining severity of, of vulnerabilities.
Chris Romeo:Okay. So how good is it, I guess, is where I want to start. Let's, let's, let's just not spend any more time in the kind of the, the warmup of the conversation. I
Patrick Garrity:think Matt is biased. Can I take this? Yeah, no,
Chris Romeo:Patrick. I want to hear your, I want to get your take first.
Patrick Garrity:All right. Yeah, and you're gonna find, like, um, most of what I'm critical about is not actually, like, CVSS, it's the aspect of how it's been adopted. Yes. And... Not biased at all here. Yeah, and, and to be honest, like, the end users are highly impacted by this because the reality is, is like, it shouldn't be them on, on them to fix this problem. Um... And so I think that's, we'll talk about what's so important with v4, but the biggest problem and challenge is that, uh, CVSS predominantly the base score is used, um, in almost every vulnerability management tool, and every time you see a publication saying there's a criticality of 9. 8 or 10, um, or 9. 9, right? Um, that's not taking into consideration Uh, any additional intelligence as it relates to threat. Um, and CVSS has the capability to be enriched. However, um, you know, NVD, the score there, is the base score, and that populates almost every vulnerability management tool in the world. Um, so, when we look at CVSS, everyone has standardized on Uh, the base score, which, you know, I think 60, 56 percent of things are critical or high with that. So, um, I don't know, that's, that's my fundamental criticism is the fact that, like, uh, there, it's very promising. Um, but I think, you know, and we'll talk about V4. I think there's some great things there, but unfortunately, very, very few vulnerability management programs want to use it, enrich it, um, with context of asset and, and threat. So fundamentally that, that results in a pretty negative experience, uh, when you use CVSS for volume prioritization, right? So, so can we
Matt Coles:just, uh, make sure that users are familiar, uh, you know, folks who are listening are familiar with. So when, when Patrick was talking about base score, so CVSS has a, has a collection of metrics. So in 3. 1, there's a collection of metrics, um, uh, base, which are the, the core of likelihood and exploit, likelihood, uh, or, ability of an attacker to exploit and impact, and then there are extended metrics around environmental and temporal which can be used to modify the score and get some, you know, use that additional intelligence basically or from a, from a consumer standpoint to, uh, to tailor the result based on their usage and their Um, you know, their environment. That's how it's intended to be used. It isn't always used that way, as Patrick highlighted. Very,
Patrick Garrity:very rarely! I think it's used that way.
Izar Tarandach:Because it's hard!
Matt Coles:Well, there is, there is, there is a challenge there, right? So in v4, we do try to fix that. Before we jump into v4, I'll, you know, so... There's definitely, yeah, there's definitely some, some challenges there and the SIG took in that input and tried to do a better job with, with V4 and hopefully we got a baby step farther along.
Chris Romeo:So somebody, somebody give me an example, like how would CVSS be used? Before we go to V4, let's use V3, right? As the, because that's been the de facto standard for years, a couple of years, I think, if I remember correctly.
Matt Coles:2019 3. 1 came in, in 2019. So 3. 0 was a little bit before
Chris Romeo:that. Okay, so 5 years basically it's been in play here. So, uh, Patrick, give me a use case here though. Like, I want to make sure I'm understanding how CVSS is supposed to play in my... How am I supposed to use it?
Patrick Garrity:Yeah, so, well, I think that's an interesting question. So, first, I think there's different ways in which you can use it, and that I've seen people use it. So, most commonly, most widely adopted scoring system is, is adopting the base score. Um, And frankly, like, PCI DSS, um, requires you to fix all things, I think, four or higher, which is basically fixing almost all your bonds anyway. So we all laugh at that. Um, but, um, in addition to the base score, right, and saying we're going to fix Criticals and highs, everything seven or above. That's, that's mostly how people have operated or used it, or maybe prioritize things based on what scores are the highest. Um, but the reality is, is like, that's very difficult for most organizations to patch 60 percent of their vulnerabilities, right? Um, I think most organizations end up patching somewhere around like 10, 10, 15, 20 percent in a good case scenario. And then there's other approaches. So enrichment of the temporal and environmental metrics, which then You know, adjust your scoring accordingly, and we'll prioritize things that, um, have threat or, or environmental context that increase your risks. Um, I see very few people adopting those things today, and I think, like, we'll talk about 4. 0. I think there's some great shifts, and I think part of it's even a marketing problem, um, and terminology problem, um, product terminology problem, um, with some of it. Um, And then the other way I've seen is people actually using the metrics themselves. So, for instance, like, um, you have attack vector, or you have, um, uh, I'm spacing on the metrics, I should pull them up as I'm talking. Um, but, you know, you can make informed decisions to say, Oh, wow, this is exploitable, uh, via network. Like, it, it, you know, you can remotely RCE this, for example, with, I think, two attributes, you can figure closely that out. So, you know, Yahoo, for example, and I, this is public information, Chris Madden uses decision trees with stakeholder specific vulnerability categorization to incorporate threat. So something like KEV in a decision and then further use the CVSS metrics themselves to say, Oh, the attack vector is network. Um, Oh, no privileges are required. Like that's a really high, um, higher probability. So it makes sense to decision on that. Um, so those are, those are kind of like three different ways I've seen, uh, in the wild, um, CVSS being adopted, uh, as it relates to vulnerability management programs. And that's on the, um, sorry. Yeah, that, that's more on the internal vulnerability management teams. I think there's another side of this, which is like vulnerability disclosure, bug bounty, slash like software development. Um, which I don't spend as much time on, right? Um, that, that Matt or Izar might be more familiar with as far as some of the ways there, uh, and how it's used.
Matt Coles:uh, you know, so what? One thing I do want to highlight, and this is sort of why NVD, we see NVD using only the base scores. Today in 3. 1, the base score is intended to be used by vendors. People who, who have vulnerabilities and they, and they get the, you know, they get a CVE for it, for instance. The base score is intended to be used from, from a vendor standpoint. The temporal and environmental metrics are intended to be used by consumers of vulnerabilities and how that vulnerability will impact them, right? So for instance, if they have a, if there's a library, then upstream, you know, provider provides a library and that library has a vulnerability, you get a base score. Then when you consume it, When you consume it, you're then, when you, when you consume it, you're going to then modify its characteristics based on how you, how you make use of that library, or that component, or whatever the case may be. Yeah,
Chris Romeo:yeah, I want to interact with, uh, so we have the ability to look at comments that people are putting as we're having this conversation, and so I want to put up Dan, uh, Kuykendall's, uh, comment here, because I want to have the panel kind of react to this, like, it seems like this, that, like, what Dan's suggesting is kind of in line with what, uh, Patrick, you were saying, um, I think you made this little, this kind of similar statement, but Patrick, what's your reaction here? Do you think Dan's, is Dan right on here and his assessment of, of how CVSS has been classically approached?
Patrick Garrity:Um, yeah, so I, I think a few things, right, which is, I think there needs to be more vendor responsibility of adopting CVSS enrichment. Um, so the, like, truthfully, um, part of the challenge is that it has. Uh, difficult for vendors to get customers to provide, choose the metrics. So, like, I don't know that the customers should choose. I think, like, it should be auto enriched. Um, maybe the environmental metrics, there's some choice in that. But, like, that's where we need to get to, to get broad adoption of CVSS beyond base score, is, like, the work has to be done for the consumer. And I know there's some work that they're going to have to put in, but, like, None of the vuln management tools out there do this for them today that I'm aware of, that I've ever seen. Like, all of them just incorporate a base score. There might be, like, a rare exception to that, but, like, the most popular tools, um, you don't really have the luxury of modifying even the CVSS scores. Um, so that, that, that's a real thing that needs to be overcome in order for people to, to really get value out of CVSS with their vuln management programs. And it is something, too, like, even, even on our side, like, We believe that v4 makes it much easier for the product companies that are delivering VM tools to do that, uh, which is pretty exciting.
Matt Coles:So in this case, Patrick, you're talking about using, uh, you're not talking about changing the way that CVSS works, you're talking about the implementation by tool vendors that should come with context.
Patrick Garrity:Yes.
Matt Coles:And we've talked about this, I think, on the security table in the past about the problem with many of these tools, SAST, bad word, DAST tools, as Chris is known for, uh, that, and S, and S, and potentially, you know, or, you know, static code analysis at this point, uh, context is always a problem. And so do the tools know what it's, what they're looking at? Can it, would it be able to provide the right information or at least give options to, I know that some tools will ask, like, what type of system are you building? And, and, uh, yay. Yes, another, another DAST fan over here. Um, so, um, yeah, so it's, it's interesting to, to see how that turns out. It's nice to see that, you know, it's nice to know from a SIG standpoint, I think, that CVSS has the capabilities. I'll leave it at that.
Patrick Garrity:And there are like, there are some things I think too, like there is more OSINT, like there wasn't OSINT data regarding vulnerability threat intelligence, very accessible until more recently. Um, so I think the other things is it would be nice to even see, like, maybe upstream, like, NVD enrich CVSS scoring with a threat intelligence that's available, like, SysEqEv, Non Exploited Vulnerabilities List, and, um, EPSS would be another example. Um, so I think there's some interesting things that could happen, um, that would make it easier, because in some ways, like, if, if CVSS did that work, actually, before it hit the vendors, That would even make it easier for it to get broader adoption. So generally speaking, like, I love to be a voice, even myself, of trying to get vendors, um, not only us, to adopt open standards like CVSS, CVSS BT Score, um, BTE, uh, which we'll get into when we talk about 4, EPSS, CAV. Um, like all these things are great for, um, product companies to be implementing. Uh, and putting in their products, and we're seeing a lot more adoption of that, which is great.
Matt Coles:Well, and actually, can, oh, sorry, can I just, uh,
Chris Romeo:Yeah, please respond.
Izar Tarandach:Can you hear me now first?
Chris Romeo:Yes, Izar's back!
Izar Tarandach:Yes, yes, yes!
Matt Coles:So, uh, this is probably a good segue into v4, because of some of the capabilities that we're adding in v4 that vendors can take advantage of, or vendors can use, that consumers can then leverage, uh, in their, in their use. Uh, so, to introduce it a bit. So CVSSV4 takes and expands the the number, the metrics have expanded. We've done some changes from 3. 1 to address. Uh, confusion, the way that things were, were being set in, in 3. 1 that often resulted in misscoring or confusions about scores, uh, especially when it came to things like how scope is handled, uh, and we can talk at length about that if you want, um, but, but the really, the key part I wanted to highlight because, you know, what Patrick was mentioning was supplemental metrics. So, while supplemental metrics don't affect the score, it's additional data points that vendors can provide to their consumers so that they can, um, interpret the results appropriately and make important decisions from it. So, it was important to highlight that just because of the topic. Um, we can dive into that in more detail, uh, as we go.
Chris Romeo:Yeah, I'd love to get, um, I'd love to get Patrick's take on CVSS4, kind of, from that perspective, uh, as far as what you've seen as you've kind of studied it and analyzed it.
Patrick Garrity:Yeah, so I think, I think like proceed with caution, I tell people, and the reason, the first reason why I say proceed with caution, isn't because, um, of any reason other than like, it just came out, um, It's really not being used or leveraged, it's not in the NVD, um, so saying like suddenly we're going to do 4, CSS version 4, doesn't mean much, um, I think right now. And so there's a long tail of time, like almost like with any product coming to market, where like all these things need to be true to where it's going to be usable at least for, uh, vulnerability management or product security teams, right? Um, at the same time, like, there's no rescoring of previous vulnerabilities, so if you standardize on version 4, you still have to leverage and use previous versions for older vulnerabilities. Um, so, you know, I think this is just the dynamic of, like, the reality of, of, um, how CVSS has worked, um, in consideration. That being said, like, I think it's really promising if we're, like, going, okay, from now on, moving forward, like, maybe we consider leveraging and using CVSS version 4. Um, but once again, like we need vendors to get on board with enrichment. Auto enrichment, I think, is huge. I think talking to some intelligence firms, the concept of enriching and creating a BT score is possible without having any asset information. So I think there's some really, uh, interesting and promising things that could be done. Um, oh, and I see that an advisory. So, um, you know, that that's the first thing from my perspective. I do think that the change, um, in the scoring methodology is going to provide better distribution. If you're only using a base score based on my work of doing calculations, it looks like the scores will generally be as high or higher. Um, but, um, that being said, they'll be better distributed. Um, uh, which is a, a big bonus as well. So, yeah, and that statement could be inaccurate. I did my work in, like, June. Um, but my guess is, is, like, generally using the base score is probably going to feel the same, um, uh, as it did before. Um, so you really, if you're going to adopt CVSS version 4, want to look to how you can, uh, you know, get to a threat and environmental side. And the last thing I'll just say in, in version 4 is, I think the marketing side, like, in some ways, what CVSS did in putting nomenclatures together, um, it's not as if the nomenclatures didn't really exist before, but this is an example of where marketing can really help. Um, these nomenclatures break out the base score, which is called CVSS, uh, dash B. Then you have the base and threat metric score, which is CVSS base, uh, plus threat metric. And then, um, you have BTE and BE. So, BTE would be your base score plus the threat enrichment plus the environmental metrics. And that would be the, like, the ultimate goal is pretty much a universal risk score. Um, uh, I might have said that wrong. Matt's like, yeah, maybe, maybe a little bit, but, but I think you're getting into...
Matt Coles:Less risk, more severity, right?
Patrick Garrity:Yeah, but if you're getting into asset context with the environmental conditions, then I think, like, it, it does get much closer to risk. Um, and so I just think, like, that, that's some of the promising aspects of, I think, CVSS version 4. Um, depending on which vendors, and I think, like, Nucleus is an example of the company I work for. Where since we're like an aggregator and looking at all this Um, we have the opportunity to do that, whereas, like, that's a lot harder to do on a scan tool, for instance, right? Um, so I'm, I don't know, we're pretty, like, I'm pretty bullish, and Nucleus is pretty bullish as far as, like, the, the possibilities of, um, CVSS version 4, uh, and how it could be used and adopted, uh, within Vuln Management and product security programs.
Chris Romeo:Izar, you've been waiting so long to say something.
Izar Tarandach:Yeah, can I get one in there? Oh! So... I don't know if you guys discussed this already when I was off, but my thing with CVSS and going very much into what Patrick said about marketing is that there was sort of a miscommunication somewhere in the way that people understand it and the way that people use it, and as you alluded before, people started adopting it as a show of risk. Very interesting. And, uh, uh, CVSS very quickly for me became what I internally call the panic index. Oh, it's a 10, it's a 10, let's run around with our heads on fire, and yeah, it's a 10. And, uh, uh, as you guys know way better than I did at this point, it's not always the idea, right? But one big thing that was always there for the users, for serious users of CVSS, in my opinion, is that, yeah, you get a nice index out of it, but the index itself is not the whole story. The index has to be used together with the vector so that you can tell the story behind the index. And people very quick became very adept at interpreting the index, but not quite so at the vector. And, uh... What I'm afraid of now is that CVSS gets derived into C-V-S-S-B-B-T, BTEE, TEBB, whatever we mix in there and people are going to have an even worse time at understanding that. So touching back into what you mentioned about marketing, now that the standard is out, what is the SIG planning to do in order to make this thing. Easier for people to consume. And perhaps, fix the mistakes of the past?
Chris Romeo:Nobody wants that one.
Patrick Garrity:Well, you asked about the SIG, and Matt's the only one on the SIG!
Izar Tarandach:NERVE!
Matt Coles:Alright, so, let me try to unpack that, uh, a little bit, but uh,
Izar Tarandach:But I was so explicit!
Matt Coles:You were. So, I think, first off, you know, putting better... So, one of the things that we did, we tried to do in the SIG was to, um, really focus on documentation, so the specification, the user's guide, there's an FAQ, there's examples to help people with scoring, and also given that the, um,
Patrick Garrity:Hey Matt, on the, on the training note, right? Yeah, yeah. That you're talking about, like, I think last night or the night before I went to first. org and I went through the, the, the V4 training.
Matt Coles:Oh good, how was it?
Patrick Garrity:Yeah, I just want to emphasize, like, that's available to everyone.
Matt Coles:Great.
Patrick Garrity:It was good, it took like half an hour to an hour. It's a good overview for people that are in vulnerability management or want to become more. familiar with CVSS. So, um, yeah, kudos on that.
Matt Coles:Thank you. That was really important. Uh, obviously training, training is hard. Documentation is necessary. One of the things I think you highlighted earlier was, you know, that people are unfamiliar with certain aspects of how CVSS is intended to be used or, or could be used. Uh, there's, I imagine a number of folks probably don't know that there are, there's guidelines for vulnerability chaining. There's guidelines for, uh, different types of components. component usage and how, how that will affect your CPSS scores, uh, and there's different, uh, and there's guidance or information in the user's guide and the examples, um, for, uh, some of the, we tried, we tried to get ahead of some of the new questions around things like, uh, so what, so a couple of the big changes that happened in V4 from 3. 1 was we changed how scope works, right? So scope, originally was it either has or has not changed based on, uh, you have a vulnerable thing and then you have some, some impacted things, uh, and, and scope change occurs when you, when the vulnerability occurs in one component but affects a different component. But what wasn't clear was what does scope change really mean from those, from those metrics? And so in v4, we split those out so now you can have some, an impact against the vulnerable system and or. The subsequent system, the impacted systems. And so you get better clarity. And so we took a lot of effort to make sure documentation was clear on those points. Uh, likewise, many of the other aspects around how usage, uh, you know, usage of the, of the calculator and, and as to Izar's point about what the, what the index or severity label means versus the score. And so you get better information, better data out of the calculator, uh, or, or when you generate the score, you get better data that's, that's encoded in the vector string that can allow you to make better informed decisions. Uh, and, and that was a really big focus area for us. I also see there's something in the chat. Somebody was asking about, well, what about things like medical devices and whatnot? And so we, um, one of the nice things about V4 is, um, that it has, has added metrics or metric values around functional safety. So safety of people, not just of systems. Uh, and so this was an attempt to start to introduce, um, the, the human aspect to, to, to CVSS. So previously it was primarily around data and function, confidentiality, integrity, and availability of data, but now it can also include, uh, whether or not a, uh, a vulnerability can impact, uh, you know, either directly or, or indirectly impact uh, safety in a, in a system. So if, if that system is being used as a medical device or, you know, favorite, favorite example, nuclear power plant control, uh, then, then those are, those are kind of important to understand that this vulnerability, you know, may not be serious, but it may have a safety impact, and therefore you want to take it, take attention to it. So, yeah. Can I ask
Chris Romeo:a, can I ask a question about the complexity of this thing?
Matt Coles:Thank you.
Patrick Garrity:Oh, that's, that's my grape. Is this
Chris Romeo:is CVSS 4. 0 calculator and the entire algorithm that's been created. Is this for everybody first, or is this just for people who are in the trenches deep enough where they can understand all these different things. Because I'm looking at the calculator and I've got attack factor, complexity, requirements, privilege required, user, I mean, I've got a lot of things that are...
Patrick Garrity:And your eyes are just glassed over, right?
Chris Romeo:But it's kind of, but I don't understand.
Izar Tarandach:But wait, you're asking about the calculator or you're asking about the algorithm behind the calculator?
Chris Romeo:Well, I mean, the calculator implements the algorithm, I hope. Yes. I hope it's something different. Wait, yes, but? What does that mean?
Matt Coles:Yeah, what does that mean, Izar?
Izar Tarandach:Okay, so, on 3. 1 and before, you had this nice formula, you changed the levers, and the levers serve as flags for multipliers of weights, that at the end give you the nice number. But now when you look behind the thing, you have, what was it called, uh, the... Equivalence nuts. Equivalency nuts. The equivalency sets that point out to strings that have to be built dynamically, and then each, each, uh, group of strings points to a bucket, and that bucket represents something else, and... It wasn't easy to go through all that stuff, I have to tell you.
Matt Coles:But you don't have to.
Patrick Garrity:So, so I think... Yeah, I think there's a few things to unravel here, like... 99 percent of the world will never understand how CVSS works, and they shouldn't have to, right? Right. Um, but there's a fundamental disconnect between CVSS being a protocol that does this thing, right? Like, does everyone understand how... Emailworks and SMTP and all that stuff. No, but they use it every day. We do. Yeah, you all do. But yeah, but outside, you know, outside the circle. So I think there's this reality that it's like, well, the challenge, right, is the event that most, there's only a few. Very few sophisticated, very mature organizations that can adopt CVSS, um, at a, at a level of scale. You have to have sophistication, right? Um, and so, I'm not saying everyone in the world is dumb, but it's just the reality that it's like, Is this the most important thing for them to understand in the job role? Um, probably, probably not for most, um, if you're in vuln in management, like, you probably should know some of it, but the reality is, is I meet a lot of vulnerability management and product security teams that know absolutely nothing about vulnerability, um, uh, scoring, right? Um, and so that's where I think, like, I'm incredibly biased towards, um, there's a disconnect, and I think part of the marketing needs to be to vendors, um, in, in interoperability, where they're adopting these standards. And they're doing the work for the consumer. Like, they're enriching threat intelligence of the CVSS score. So they don't have to know if they're using CVSS B or BT or BTE. But the reality is, is like, good luck trying to get Tenable or Qualys to do that, or any other, you know, I'm not trying to criticize, but like, you have hundreds of vendors out there, um, that you have to go advocate to actually make this a possibility. Um, and so, I think that's like the biggest challenge that CVSS v4 is gonna have. Is the right adoption within the tools that are out there in these existing customer environments because like they're going to keep on using the CVSS score that whatever the tool in their environment is spitting out. Um, and that's the, you know, that's the harsh reality.
Chris Romeo:So I want to come, I want to come back to Matt here and ask this, this complication question because I didn't get his answer on that, but like, is, is this too complicated or is it okay? Because it's not for everybody.
Matt Coles:Well, so. I want to, I want to tackle the is this for everybody. CVSS should be used, in my opinion, my personal and my professional opinion, that CVSS should be used by anybody who's, who's scoring, who needs to know, wants to do an apples to apples comparison at the very least, you know, or wants a consistent way of calculating the severity of an issue. Now, do they need to understand the math behind it? To Patrick's point, I think the answer is no, right? People rely on technologies all the time, but they don't necessarily understand how it all works underneath. And that's okay, because the calculator abstracts out four of them. It's important to know what the metrics are, what they mean. So when we say attack vector, what does attack vector mean? What does it mean to be network versus adjacent? Right, that's important to know, but that's important to know if you're doing, if you're creating vulnerabilities or working on vulnerabilities, that's important to know because that's, that's how you would then specify mitigations or how you would look at risk or, or other things that, that should be part of your, of a job function there. And so, um, is this for everyone? It's, I think it's for everyone who has a usage for it. And whether or not it's using a easy to understand formula, or, and by the way, the old CVSS formula I don't think was, was trivial to understand either. Uh, the use of, of equivalency sets doesn't make it worse. Uh, it is, it is a bit more math y. Uh, there's, there's more, some more theory to it, but, uh, but it gives more, Um, expressive, uh, results, right, which is the important part. Um, so, I think in that regard, I would say it is for everyone who has a need to use it.
Chris Romeo:Izar?
Izar Tarandach:So here's the thing. First of all, clarification. I agree. 4. 0 will bring us better results. With that said, I have to play the devil's advocate and say... Are we creating a trust me bro effect here, where I say, this is the CVSS result, and behind the curtain, how it got, how it got gotten to, it's not really a problem, just trust us, we went through the math, and the math is good. And at the same time we advocate for people to know as much as they can about the things that they run and the things that they own, so are we giving mixed messages in here? And what if somebody, a vendor, comes in and says, you know what, I question this CVSS result that I got. And nobody can, a small number of people can actually go and say, Okay, let's look at the math and see how you think that this thing broke.
Matt Coles:Yeah, so I will, I'm going to give you my perspective here, which isn't necessarily the common perspective. Well, I'll give you my perspective. I don't know what the rest of the SIG would say, but, uh, You know, the, the, the CVSS SIG is made up of, uh, what, 50 different, 50, 50 plus individuals from, from organizations that range from, from individual people who are, you know, who do this for, who do vulnerability management or vulnerability discovery for living all the way up to small, medium, and large organizations and governmental and non governmental agencies. Some people have a heavy background in math and computer science. Others have, you know, history and another back, you know, and program management and program development experience where they're doing their vulnerability managers. So a whole range of experiences and understandings and perspectives. And this was, uh, you know, implemented, tested internally. Beta tested with, from, you know, with outside people who provided comments. We got comments from the industry about the scoring method, about the approach, about the calculator and all of its details and all the documentation, uh, and over the course of the past X number of years. And so this has been well vetted, I would say, right? So it's important to highlight that this is a community effort, obviously, of initially of members who, people who are members of FIRST, which are people who have a vested interest in understanding how vulnerabilities are scored and managed and acted upon. And then. And then beta tested with the community at large, the people who will be consuming the results of the calculator and all the documentation, etc. And so over that period, we tried to address any of the comments that came in. So, is it perfect? Unlikely. Is it supportable? I believe it to be. Uh, and, um, and then time will tell if it becomes trustworthy.
Patrick Garrity:Yeah, I think, uh, you know, one, one thing Izar, too, on that note, like, I research threat intelligence sources, EPSS, exploit prediction scoring systems, CVSS, I will tell you, like, I do think that people ideate a little bit too much on perfection. Um, and, and it's really like when you have 240, 000 vulnerabilities, it's really easy to pull out all the ones that are anomalies that are misscored or like should be higher, or like, I can tell you every scoring system I've looked at. Um, and like, hey, why, why does, why does every threat intelligence source have completely different vulnerabilities on it? Um, open source and non open source, and I think that's a, that's, that's a bit of the reality of like the, the complexity and the state that, that we're in right now, uh, too. Um, and so I try and caution people, like, every time someone shows up to the EPSS SIG, they point out that, like, there's 10 KEV vulnerabilities that have a very low score. Um, and then it's explaining, like, well, yeah, this is a machine learning model, and this is how it works, and, like, oh, by the way, like, um, Just because something was exploited 10 years ago doesn't mean that it's going to be exploited in the next 30 days. Um, so a little bit different there, like, because we are talking about ambiguity with machine learning, um, and it's not logic, you know, uh, driven necessarily, but, um, Yeah, I think that's one of the, one of the biggest challenges we have in vulnerability management is like, everyone is just like, where's the silver bullet? Where's the silver bullet? Where's the silver bullet? Uh, and, and reality is, is like, there's a lot of, lot more work to be done. Um, and people need to acknowledge like, where to prioritize. Um, their efforts, and starting with a CVSS base score is probably one of the worst things you can do. Um, uh, for a vulnerability management program.
Matt Coles:As a consumer.
Patrick Garrity:Yes, as a consumer, sorry.
Matt Coles:As a vendor, that's where you start.
Patrick Garrity:Yeah, it's critical.
Izar Tarandach:So, thanks to both of you for the answers, but just to add a bit more fire on the pile, uh, Patrick, you went to a PSS. And in there, there isn't even a semblance of trying to make the algorithm known. It's already known, hey, it's closed, no, you can't look behind the curtain, and you know what, just accept the number, work off of it. And, uh, I don't know, something inside just tickles me, that we are trying to get all this clarity and all this understanding, and everybody to look at the same thing. It's not even a silver bullet. It's to look at the same thing the same way so that we can go after the important things before everything else. And at the same time we are modeling the way that we get to that angle. So, while I agree with everything that both of you said, And yes, this is getting better than it was a year, six months, three months ago. And it's probably going to get better as we go forward. Something still doesn't fall right with this whole believe us thing.
Patrick Garrity:Trust
Izar Tarandach:I would like to have some more clarity.
Patrick Garrity:like vulnerability management has failed at this for decades, so that's why nobody has trust.
Izar Tarandach:Yeah, so now we are failing in a different way. We're failing in a way of, uh, remember when to, to, uh, run a computer program, you had to go to people that wore white coats to closed labs, and you had to just ask them to please talk to the machine. The feeling now is that, again, we are putting an intent... An interpretation layer between the things we want to know and the people who are telling us what we need to know. And, uh, I think that with the whole everything is open approach, I probably got infected by that virus and something tickles me wrong. But I definitely see what you two are saying. So I'm not saying that it's inherently bad, I'm just saying that it's funny in this time and age, I guess. Well, so what, what,
Matt Coles:thank you for your perspective. What do you need to see? So if we're looking at the future of, you know, say CVSS, right? So we're looking at 5. 0 in the future, right? What do you need to see out of it? What do you think is missing?
Izar Tarandach:Okay, let me come clear here. No, let me come clear. I have a degree in math. The only reason they gave it to me is if I don't use it ever. So, I'm coming from that point, okay? The thing that I liked about CVSS 3. 1 was that I could look at that line, at that, at that equation, set of equations, look at the weights, and start understanding immediately what was it that, why one thing is considered to be more important than another in building that final panic index, right? And on CVSS4, it's very, to me at least, very difficult. It's not as clear as I think it should be. I think that the whole thing that I'm saying is, it would be great, and perhaps the reason I just don't know, it would be great if there was some documentation to the documentation that would explain better. how those sets of things were gotten to and what they actually represent. And I understand that it was a collection of data from a lot of different sources and somebody bucketized those things and from there came the, uh, the sets. But I think that it'd be great if somebody took the time to take us by the hand and lead us through the whole process, just so that adoption can be a bit easier because people understand a bit better the things that we didn't understand about 3. 1 and before. You see what I mean? It goes to that marketing piece that Patrick brought up. If we want this, and I want this, to be adopted, it has to be clear. And right now it's not to me.
Matt Coles:Fair enough.
Patrick Garrity:I think, yeah, I do think to some extent I agree with you, but I think time will tell as we see how things are scored to, um, that was one of the things I recommended that I was a little bit disappointed in, and maybe, maybe I interpreted this wrong, but I'm like, why don't we go and rescore the last 20, 000 vulnerabilities, right? And see how this model works. Um, and so I, I, like, I'm not part of the SIG. I'm speaking on my, my, my behalf, but I'm like, Logically, if I'm building a product, I want to see the outcome, and I do think there is a little bit of this, like, yeah, just trust us, like, we believe it's the best math, um, and I think there, it's, it's documented and clear, but also I think we're moving past it. Thank you. A point of like logic in a lot of these algorithms or machine learning where there is no logic. Um, well, I don't know, maybe the wrong word is no logic, but, um, Like you can't, you can't just say like, here's why the machine learning model told us why this vulnerability is the highest rate.
Izar Tarandach:There's no explanation to the logic that perhaps exists.
Patrick Garrity:Yeah. And so like when we're looking at artificial intelligence and machine learning and things like that, it like totally throws things out the door. What I can tell you from my research is what I did and what I suggest other people do is they should spend time intimately understanding what what they're going to use to measure in their vulnerability management or product security programs. So like if you're going to use CVSS, you should probably spend a lot of time becoming familiar with it and how it works. So that you know what impact that's going to have on your organization. Like, then, for me, I went to EPSS, and I was like, let me learn everything about this, and like, why are these low scores, and I'm looking at threat intelligence, and there's known exploitation. Well, you, you know, it took me a while to start looking at these different parts of the pie, and realizing, like, each one of them has its own unique value, and the more you understand and use, probably the better your vulnerability management program is going to be. But like, that's a hard thing right now because none of these things are correlated. Um, the, the data sets are out there and all over the place. Um, and so I think, you know, to some extent we're, we're, we're talking about like Most OSINT, um, you know, vulnerability intelligence sources just came around in the last, I don't know, two, three, four years. Kev, Kev, two years this month, right? And, um, EPSS, maybe three. Uh, so I, I think we're, like, really, uh, early on. Um, and I, I do think, like, you know, there's some interesting, like, uh, like creating scoring systems by different vendors and trying to obfuscate all this. Um. I think you have like Qualys true risk score, um, Tenable has DPR, but the challenge is, is then we're, we're moving to proprietary standards and those don't scale across an enterprise across all the different tools. Um, so yeah, I think, I think some of it's maturity, right?
Matt Coles:Yeah, so Patrick, just to your point about rescoring, so we did, I mean, we, we certainly did within the SIG, and then I, and then I think during the beta process, there was additional rescoring of select, uh, select score, select things, um, from the, the members of the SIG, um, or from, uh, you know, that, that were publicly available vulnerabilities. The hard part, I think, that we, you know, that needs to be recognized here is many of the old vulnerabilities that are in NBD don't have sufficient information to know All of the details like you're not a vendor at that point. You're a consumer, not a vendor. And so, uh, you know, you can't necessarily know all of the intricate aspects to be able to create the base score at the very least. Right? Um, so if there's, if so, for instance, if the system, if the score says scope change, yes. If you're looking at a 3. 1 score originally in NVD, it's, it's challenging to know whether the CINA impact was for the vulnerable system or the impacted system or some combination thereof, right, so that you can then create a proper V4 score with a vulnerable impact and a subsequent impact. And so those, those sort of challenges, uh, are, are, and I see you're smiling. I know you probably were.
Patrick Garrity:Well, I did, uh, what I did in my research is I created ranges, right? So I had a high and a low range. So like the attributes that didn't map one to one, I like literally determined like, well, this is the lowest score it could be and the highest score it could be. And then I, I baseline that across all attack, uh, all vector strings for 3.1 and 3.0. Um, and yeah, basically it showed it's like, you know, I have the, the visual somewhere, but like generally those plots were higher. Um, uh, but they were better distributed. So like not, not 10 percent of the vulnerabilities are scored as a 9. 8. Um,
Chris Romeo:Wait, that's bad? All right. So I want to, I want to interact. I know we've got, we've had a great audience that's been following along, uh, both on LinkedIn and YouTube with us. And so I want to, I want to interact with some of the. Comments and questions that we've seen come through here. We've interacted with a few of them just because they were in the middle of the conversation, but let's take a look at this one from, uh, Buddy Bergman, who's asking a specific question in regards to using the vendor's enriched vulnerability severity, specifically calling out PCI's patching requirements for critical or high. Um, Anybody have? Looks like Patrick has an opinion on this one. Patrick, what advice would you give Buddy here, though, in regards to this issue?
Patrick Garrity:I'm not a lawyer. Uh, yeah, I mean, like, like, here's the thing, right? It's like, the way it's written, from my understanding, is you basically have to use the base score in patch 4 or higher for your PCI environments. Um, what I thought would be promising, right, is maybe compliance standards. could incorporate to say, hey, you can use the BT score, and that would then be sufficient. Um, so I think there's some promising hope, like... Yeah, good luck lobbying PCI to change their standards, and oh, oh, by the way, all the other new standards, and I do think that standards are becoming more important, like just this month we saw NYDFS, um, also, uh, acknowledge vulnerability management and doing it based on Um, risk prioritization. Um, so maybe they would qualify BT or BTE score as a consideration, uh, for that requirement. And that's what is exciting to me about Open Standards and CVSS is like, yeah, if we can have compliance requirements that say, hey, We'll let you use the BTE score and do 7 or higher? Wow, that's gonna make everyone's life a lot easier. Um, so, you know, that would probably be my, you know, tying to PCI and, and all the other regulations coming down the, the, the, um, pipe. Like, they're too vague. Like, do it based on risk scoring. Like, do it based on Kev. At least Kev is, like, Kev is a start, right? But, um, I think there's just so much more than that from my perspective, and if we incorporate the nomenclatures into compliance, it could help out a ton.
Chris Romeo:Here's another one, uh, from Itamar. Uh, what are some open source initiatives vendors can partake in to help with enrichment? So, back on that enrichment kind of topic here, um, Matt, is there anything that comes to mind from your perspective? I know you got a good, you have a wide view of... What's happening in the space.
Matt Coles:Well, so...
Chris Romeo:I don't know what that... The dog was just recommending something. I couldn't translate it though. Izar is normally our translator. But go ahead.
Matt Coles:So, um, so the, uh, um, remember the arrangement that Patrick was talking about was really having tools when they spit out scores to provide additional information or make use of that additional information to refine the scores that come out. Or the vectors that come out. And so open source initiatives, uh, I guess. Open source initiative vendors, meaning open source maintainers, um, can help with enrichment by when they build tools, allow users to provide or allow for the ability to detect conditions that can be used for context, right? So if you're, you know, maintaining OpenVAS or, uh, or, you know, a similar open source tool like that, um, you know, it was a vulnerability scanner. Maybe have a collection, have the ability for a user to provide either at the CLI or as a configuration file information about the system that's being scanned so that you get better results, right? If you're using SEMGREP or, or, um. Uh, or, uh, you know, um, I don't know, FlawFinder or whatever, um, that, that that can also likewise provide context so you get better, better scoring information, right? The hardest, the hard part with, uh, the hard part with the TE portion, the, the, the, um, you know, non base or beyond base metrics is it comes down to How you're using a component or how you're using a system or an application, uh, and, and that would allow you to modify the base score, right? So the base score will calculate. Is the, is the vulnerability going to be impacted over a network and does it require authentication and, or do you have to trick a user into doing something, uh, for user interaction, uh, and does it affect confidentiality, integrity, and availability in one or more systems? The extended metrics, the threat environmental metrics, uh, provide things like, well, do I care about confidentiality in my environment? A tool doesn't necessarily know that right off the bat. But you have, but what a human does and, and, or there's something that can be provided to provide that information, um, and that can give better score results. And that's how you should use the calculator as a consumer. And so whether you're, I guess the call to action here is whether you're an open source, uh, maintainer of a tool or even a commercial vendor of a tool, it'd be nice to have this information so you can get more realistic scores. And, and, and to Izar's point. Not everything needs to be a 5. 0 bright red color or, uh, or, you know, sky is falling fix now because, um, when you can get better and more accurate outcomes.
Chris Romeo:So there's no room for Chicken Little in the world of CVSS.
Matt Coles:The CVSS will let you do that. If you want, uh, I would argue your, your vulnerability management program there shouldn't be room for that.
Izar Tarandach:Hey, it's actually the other way around. Sorry, Patrick. I think that there's too much space for chicken little in the CPSS world, because if you misuse it, that's all you get. Sorry, Patrick.
Patrick Garrity:Yeah. I was just going to mention the question on the, uh, you know, open source stuff and contributions. I w I will say if you have threat intelligence or, um, exploitation data, I work on the EPSS SIG, like. You know, we incorporate that in order to build a machine learning model, so if you have any of that type of information, we use like actively in nuclei, Metasploit, I think F5 contributes. GrayNoise, Fortinet, Cisco, a bunch of other vendors, um, so I don't know, I think that kind of hits on the question that was asked earlier, um, regarding, you know, making contributions, um, and, and I will say like the, you know, EPSSIG for good or bad is pretty, uh, loose in regards to like, there's a lot of people in it. Um, uh, so it's a good place to hang out, um, in regards to, like, coordinating with different people, uh, as it relates to vulnerability management standards, so you can check that out too.
Chris Romeo:We're almost out of time, but I want to react to this kind of last comment that I just saw pop in here, um, and just see do we agree with the statement that's been made here. CVSS is shared responsibility. Vendor provides a base score. Consumer adjusts with threat and environmental based on their specific use case. It's not solely one or the other. Do we agree with this statement as a, as a way to kind of wrap up our CVSS conversation?
Izar Tarandach:In a perfect world.
Matt Coles:Generally, I agree. And then I, in our documentation, the documentation for CVSS supports this.
Patrick Garrity:I think that, um, I agree. I think there's a middle person, which is the tooling and the tooling is actually the core of the problem of CVSS today, um, where the tooling is not taking any responsibility nor building the capabilities to support, uh, CVSS functionality. So like that, that's where I think the shared responsibility is failing, um, uh, today that we're seeing, hopefully we see that change with, uh, version four.
Chris Romeo:All right. Well, uh, we're out of time here for today's security table. Thanks to everybody who joined in and contributed, commented, opinion, gave us opinions, ask questions. Thanks Patrick, for joining us as a guest here as always, Matt and Izar. Thanks for being around the security table. We will catch everybody next week.
Patrick Garrity:Thanks so
Izar Tarandach:much. Thanks Patrick.
