WEBVTT 0:00:03.760000 --> 0:00:09.300000 Managing alert noise, false positives and escalation criteria. 0:00:09.300000 --> 0:00:12.880000 So in the previous video, you got an understanding of the triage process 0:00:12.880000 --> 0:00:16.540000 and how a log becomes an instant. 0:00:16.540000 --> 0:00:20.780000 Now that is communicated to you as an instant responder as well as the 0:00:20.780000 --> 0:00:29.460000 information that is typically contained in the, for example, a case or 0:00:29.460000 --> 0:00:33.160000 a ticket that is then escalated to you as the instant responder. 0:00:33.160000 --> 0:00:37.060000 We're now going to touch on alert noise, something that I mentioned in 0:00:37.060000 --> 0:00:42.680000 the previous video as well as more specifically in the context of alert 0:00:42.680000 --> 0:00:47.920000 noise, we're taking a look at false positives and all of that with regards 0:00:47.920000 --> 0:00:48.800000 to alert accuracy. 0:00:48.800000 --> 0:00:53.700000 But more importantly, the escalation criteria, which I sort of want to 0:00:53.700000 --> 0:00:57.740000 reiterate because we've gone through it, I believe in the first course 0:00:57.740000 --> 0:00:59.380000 within this learning path. 0:00:59.380000 --> 0:01:03.040000 So it's probably a good idea to revisit it because it is quite important 0:01:03.040000 --> 0:01:08.280000 and it's something you're going to be dealing with a lot in your meetings, 0:01:08.280000 --> 0:01:14.660000 in your weekly standups, let's say with your, with the SOC or whatever. 0:01:14.660000 --> 0:01:18.300000 But to get things started, what is alert noise? 0:01:18.300000 --> 0:01:21.640000 You may already be, you know, have a vague idea of what this is, but it's 0:01:21.640000 --> 0:01:24.040000 always good to formalize the definition. 0:01:24.040000 --> 0:01:30.960000 So alert noise is the flood of security alerts that are either a irrelevant, 0:01:30.960000 --> 0:01:37.440000 b, duplicate, c, low value, or just plainly false positives in a SOCS 0:01:37.440000 --> 0:01:39.400000 monitoring queue, right? 0:01:39.400000 --> 0:01:44.800000 And alert noise arises when detection rules are too broad and this is 0:01:44.800000 --> 0:01:46.880000 very, very important, right? 0:01:46.880000 --> 0:01:52.000000 So the too broad threat intel feeds match benign traffic or the same event 0:01:52.000000 --> 0:01:53.940000 reaches the same multiple times. 0:01:53.940000 --> 0:01:57.280000 This is something that's very common and it's going to affect you. 0:01:57.280000 --> 0:02:03.360000 Trust me. And the result of this is, you know, often a queue that is packed 0:02:03.360000 --> 0:02:08.960000 with chatter or noise, if you will, that then obscures genuine threats, 0:02:08.960000 --> 0:02:12.460000 which is something you don't want, right? 0:02:12.460000 --> 0:02:17.300000 In addition to obscuring genuine threats, it drains the analyst's attention 0:02:17.300000 --> 0:02:23.640000 and inflates metrics, key metrics in with regards to triage like the mean 0:02:23.640000 --> 0:02:27.280000 time to triage or MTTT. 0:02:27.280000 --> 0:02:31.000000 There's also mean time to respond. 0:02:31.000000 --> 0:02:34.120000 But we will not get into that right now. 0:02:34.120000 --> 0:02:38.860000 In any case, effective noise reduction through rule tuning, deduplication 0:02:38.860000 --> 0:02:43.700000 or suppression filters, as well as business context enrichment is therefore 0:02:43.700000 --> 0:02:49.520000 critical to keep analysts focused on alerts that truly warrant investigation. 0:02:49.520000 --> 0:02:53.700000 So again, this is something that's very, very relevant to instant responders. 0:02:53.700000 --> 0:02:57.700000 So when we're talking about alert accuracy or the accuracy of alerts, 0:02:57.700000 --> 0:03:00.960000 what are these key terms that I just mentioned that, you know, are used 0:03:00.960000 --> 0:03:07.360000 to sort of classify or measure or contextualize alerts in terms of the 0:03:07.360000 --> 0:03:10.140000 accuracy? Well, we obviously have the false positive. 0:03:10.140000 --> 0:03:16.140000 That's, you know, usually frequently referred to via its abbreviate or 0:03:16.140000 --> 0:03:18.700000 in its abbreviated form, which is FP. 0:03:18.700000 --> 0:03:22.840000 So this is an alert for activity that is benign or expected. 0:03:22.840000 --> 0:03:25.220000 You then have true positives. 0:03:25.220000 --> 0:03:31.460000 So TP, these are alerts for genuine malicious activity or policy violating 0:03:31.460000 --> 0:03:37.420000 activity. And then you have your true negatives, fairly simple to understand, 0:03:37.420000 --> 0:03:41.920000 TN. So in this case, there's no alert because nothing bad occurred. 0:03:41.920000 --> 0:03:44.180000 And then you have your false negative FN. 0:03:44.180000 --> 0:03:48.700000 So this is when something malicious has happened or an incident has occurred 0:03:48.700000 --> 0:03:53.620000 or a breach has occurred, but no alert was fired. 0:03:53.620000 --> 0:03:55.820000 So it's a silent failure. 0:03:55.820000 --> 0:03:57.840000 There's no other way of putting it. 0:03:57.840000 --> 0:04:02.720000 And then another term that is quite important, but not really related 0:04:02.720000 --> 0:04:06.780000 to alert accuracy, but I think it's important that we sort of explain 0:04:06.780000 --> 0:04:09.920000 right now is the escalation criteria. 0:04:09.920000 --> 0:04:17.140000 So what is escalation criteria in terms of its, you know, defined meaning? 0:04:17.140000 --> 0:04:23.400000 Well, escalation criteria can be defined as predefined conditions that 0:04:23.400000 --> 0:04:28.820000 must be met before tier one promotes an alert to a tier two incident case. 0:04:28.820000 --> 0:04:34.140000 For example, critical severity or high severity on a crown jewel asset 0:04:34.140000 --> 0:04:36.380000 or an important asset. 0:04:36.380000 --> 0:04:38.500000 Okay. So I want you to keep this in mind. 0:04:38.500000 --> 0:04:44.120000 And let's dive into, you know, the alert accuracy terminology. 0:04:44.120000 --> 0:04:51.340000 But now sort of enhancing or enriching as it were, what they mean, what 0:04:51.340000 --> 0:04:53.760000 they look like and why it matters. 0:04:53.760000 --> 0:04:58.380000 So starting off with true positives, TP, what does this mean? 0:04:58.380000 --> 0:05:03.300000 We've already defined it, but in this case, the detection system or seem, 0:05:03.300000 --> 0:05:07.420000 let's say, raises an alert and the activity is malicious. 0:05:07.420000 --> 0:05:11.300000 Very good. So a simple example is a seam rule is triggered by a partial 0:05:11.300000 --> 0:05:14.080000 download string command. 0:05:14.080000 --> 0:05:17.380000 And the workstation is indeed infected with malware. 0:05:17.380000 --> 0:05:21.500000 Why this matters is because it confirms that the rule works as intended 0:05:21.500000 --> 0:05:25.040000 and consequently the responder must act. 0:05:25.040000 --> 0:05:30.400000 In the case of false positives, the system raises an alert, but the activity 0:05:30.400000 --> 0:05:31.640000 is benign. Okay. 0:05:31.640000 --> 0:05:36.680000 So the same rule fires when an admin legitimately uses PowerShell to download 0:05:36.680000 --> 0:05:38.320000 a corporate script. 0:05:38.320000 --> 0:05:41.740000 And in this case, I'm referring to the partial download string command. 0:05:41.740000 --> 0:05:45.940000 So this points to what I'd mentioned in the first slide where the detection 0:05:45.940000 --> 0:05:51.900000 rules may be too broad or not specific enough and therefore sort of catch 0:05:51.900000 --> 0:05:57.980000 additional events that are not really alerts or don't really represent 0:05:57.980000 --> 0:05:58.900000 malicious activity. 0:05:58.900000 --> 0:06:03.580000 Like, for example, as I stated here, like an admin legitimately using 0:06:03.580000 --> 0:06:10.440000 the very same PowerShell command that I highlighted with the true positive 0:06:10.440000 --> 0:06:14.200000 example to download a corporate script. 0:06:14.200000 --> 0:06:19.400000 So why this matters is because this wastes the analyst time and there 0:06:19.400000 --> 0:06:23.480000 are too many false positives and too many false positives create alert 0:06:23.480000 --> 0:06:25.940000 fatigue quote unquote. 0:06:25.940000 --> 0:06:27.900000 And then you have your true negative TN. 0:06:27.900000 --> 0:06:32.460000 So in this case, the system does not alert and nothing malicious happens. 0:06:32.460000 --> 0:06:35.660000 So that's sort of baseline operations where it's a good day. 0:06:35.660000 --> 0:06:39.840000 So the example here is that, you know, overnight backups run on servers, 0:06:39.840000 --> 0:06:42.320000 no security rule fires and that's fine. 0:06:42.320000 --> 0:06:43.860000 It's expected traffic. 0:06:43.860000 --> 0:06:48.880000 So all is well. And why this matters is because it indicates normal operations 0:06:48.880000 --> 0:06:50.440000 are correctly ignored. 0:06:50.440000 --> 0:06:52.500000 That's sort of the key thing here. 0:06:52.500000 --> 0:06:54.660000 Then you have your false negative. 0:06:54.660000 --> 0:06:59.640000 So this is sort of the no, no, or this is where things start becoming 0:06:59.640000 --> 0:07:05.600000 really serious because the system or the seem does not alert even though 0:07:05.600000 --> 0:07:07.960000 malicious activity occurs. 0:07:07.960000 --> 0:07:11.640000 So I'll give an example of when you're likely to encounter this. 0:07:11.640000 --> 0:07:17.940000 So an attacker uses a novel lull bin, which is an abbreviated abbreviation 0:07:17.940000 --> 0:07:22.860000 for living off the land binary that the existing rules that the detection 0:07:22.860000 --> 0:07:27.620000 engineer wrote or that you wrote do not address or do not cover. 0:07:27.620000 --> 0:07:33.700000 And as a result of that, no alert is generated while data is exfiltrated 0:07:33.700000 --> 0:07:37.640000 or the attacker is free to do what they are doing or wanted to do. 0:07:37.640000 --> 0:07:41.940000 And the reason this matters is because this is the most dangerous case 0:07:41.940000 --> 0:07:47.100000 or scenario. A real threat slips through undetected and as a result, rules 0:07:47.100000 --> 0:07:48.940000 must be updated. 0:07:48.940000 --> 0:07:53.540000 And this is where we'll get into things like indicators of compromise. 0:07:53.540000 --> 0:07:58.820000 We'll talk about indicators of attack when we get into the endpoint analysis 0:07:58.820000 --> 0:08:03.100000 section or host analysis section of this course because that's when it 0:08:03.100000 --> 0:08:06.100000 becomes fairly relevant. 0:08:06.100000 --> 0:08:12.140000 But this is where you start to see the importance of indicators of compromise 0:08:12.140000 --> 0:08:13.800000 and so on and so forth. 0:08:13.800000 --> 0:08:15.880000 We'll get to that in the next video. 0:08:15.880000 --> 0:08:19.220000 So that now brings us to the escalation criteria, which I introduced a 0:08:19.220000 --> 0:08:22.060000 few slides back. 0:08:22.060000 --> 0:08:26.540000 But we've already gone through escalation and what that looks like in 0:08:26.540000 --> 0:08:27.660000 the process here. 0:08:27.660000 --> 0:08:32.500000 But what I really wanted to focus on in this video was the escalation 0:08:32.500000 --> 0:08:36.780000 criteria. More specifically, or if I were to define that, when does an 0:08:36.780000 --> 0:08:41.580000 alert become an incident, or when does the SOC tier one analyst escalate 0:08:41.580000 --> 0:08:47.280000 or communicate this to the instant responder? 0:08:47.280000 --> 0:08:49.500000 Right, so there's three columns. 0:08:49.500000 --> 0:08:53.620000 You have column one, which is the criterion category. 0:08:53.620000 --> 0:08:57.900000 And then the typical questions asked with regards to that criterion category. 0:08:57.900000 --> 0:09:02.700000 And then the practical checks as well as examples of what that looks like. 0:09:02.700000 --> 0:09:07.760000 So when we're talking about severity or risks score, what are the questions 0:09:07.760000 --> 0:09:09.560000 that are typically asked here? 0:09:09.560000 --> 0:09:10.980000 Well, it's fairly obvious. 0:09:10.980000 --> 0:09:14.980000 Does the alert carry a critical or high score after three hours? 0:09:14.980000 --> 0:09:19.720000 So these are the questions that the tier one analyst would be asking themselves 0:09:19.720000 --> 0:09:24.840000 before they reach out to you or they create a case, right? 0:09:24.840000 --> 0:09:28.200000 And when I say you, I mean the incident responder. 0:09:28.200000 --> 0:09:32.280000 And the practical checks and examples here would be, you know, for example, 0:09:32.280000 --> 0:09:36.640000 a CVSS score that is greater than or equals to nine, which is, you know, 0:09:36.640000 --> 0:09:42.660000 bad news, exploited on an internet facing or public facing whole strike. 0:09:42.660000 --> 0:09:47.000000 Another example would be a seam rule confidence of again, greater than 0:09:47.000000 --> 0:09:48.560000 or equal to 90%. 0:09:48.560000 --> 0:09:51.880000 So that's one criterion category. 0:09:51.880000 --> 0:09:54.000000 You then have asset criticality. 0:09:54.000000 --> 0:09:58.880000 So the typical question asked here by the tier one analyst as part of 0:09:58.880000 --> 0:10:05.420000 whether or not is the target a crown jewel or, you know, a high value 0:10:05.420000 --> 0:10:11.460000 system, business critical system, or does the alert involve a privileged 0:10:11.460000 --> 0:10:16.580000 account? Examples, practical checks and examples would be, you know, a 0:10:16.580000 --> 0:10:17.380000 domain controller. 0:10:17.380000 --> 0:10:21.760000 That's something that's, you know, serious payment card environment. 0:10:21.760000 --> 0:10:26.860000 Has there been activity malicious activity with regards or specific to, 0:10:26.860000 --> 0:10:32.280000 you know, the payment card environment or is the alert related in any 0:10:32.280000 --> 0:10:37.300000 way, shape or form to the executive or an executive's mailbox? 0:10:37.300000 --> 0:10:40.300000 So this is, these are the questions that the tier one analyst will be 0:10:40.300000 --> 0:10:42.020000 asking themselves. 0:10:42.020000 --> 0:10:45.660000 And it's not really a matter of yes or no, it's very important that they 0:10:45.660000 --> 0:10:53.040000 do because, again, this will affect the quality of the, the incidents 0:10:53.040000 --> 0:10:56.280000 or the cases that are escalated to you. 0:10:56.280000 --> 0:11:03.180000 And you want to be investigating or doing your job in the, in the form 0:11:03.180000 --> 0:11:04.700000 of analyzing real incidents. 0:11:04.700000 --> 0:11:09.940000 So hopefully understanding the process a little bit better. 0:11:09.940000 --> 0:11:14.700000 Another criterion category would be the kill chain stage where the sock 0:11:14.700000 --> 0:11:22.420000 tier one analyst, you know, sort of analyzes what, what part of the kill 0:11:22.420000 --> 0:11:25.820000 chain stage an alert is relevant or specific to. 0:11:25.820000 --> 0:11:29.800000 So the typical questions asked would be, has the activity moved beyond 0:11:29.800000 --> 0:11:36.000000 recon, recon reconnaissance or initial access into privilege escalation, 0:11:36.000000 --> 0:11:39.940000 C2 lateral movement to exfiltration, and a practical check or example 0:11:39.940000 --> 0:11:43.860000 would be a beacon on port 443 to a strange ASN. 0:11:43.860000 --> 0:11:48.380000 So that's something that the tier one analyst is really not tasked to 0:11:48.380000 --> 0:11:58.100000 be dealing with or is to, you know, be doing exactly what's been listed 0:11:58.100000 --> 0:12:04.260000 here. So if there's evidence of, let's say, C2, you know, C2 beacon, then 0:12:04.260000 --> 0:12:07.220000 they definitely need to escalate to you. 0:12:07.220000 --> 0:12:11.160000 Okay. And then you have to go in then perform your own triage and validation 0:12:11.160000 --> 0:12:15.820000 and analysis. And then once you're verified that, okay, this is bad. 0:12:15.820000 --> 0:12:20.900000 There is a, you know, we are seeing beaconing here, either through network 0:12:20.900000 --> 0:12:24.940000 analysis or in point analysis, once you've understood the full scope of 0:12:24.940000 --> 0:12:28.740000 what you're dealing with, you then begin containment eradications, one 0:12:28.740000 --> 0:12:32.000000 and so forth. But hopefully you're starting to see the whole lifecycle 0:12:32.000000 --> 0:12:37.380000 come into, or you're starting to see the picture form or become, you know, 0:12:37.380000 --> 0:12:41.880000 clearer. And then you have the regulatory impact, you know, as a criterion 0:12:41.880000 --> 0:12:45.700000 category where the typical question asked by the tier one analyst is could 0:12:45.700000 --> 0:12:50.220000 the event trigger breach notification laws. 0:12:50.220000 --> 0:12:55.060000 So fairly self explanatory, a practical check, an example would be GDPR 0:12:55.060000 --> 0:13:02.760000 article 33, you know, personal data exposure, or PCI DSS scope, and, you 0:13:02.760000 --> 0:13:06.480000 know, hyper, for example, so on and so forth. 0:13:06.480000 --> 0:13:12.740000 You then have the rule confidence and false positive history. 0:13:12.740000 --> 0:13:18.380000 So the typical question asked here is how often has this rule produced 0:13:18.380000 --> 0:13:19.660000 false positives? 0:13:19.660000 --> 0:13:23.660000 That's a very, very important question for the SOC tier one analyst to 0:13:23.660000 --> 0:13:24.700000 be asking themselves. 0:13:24.700000 --> 0:13:28.340000 So if they see something that they've been seeing for a while, and they've 0:13:28.340000 --> 0:13:34.580000 previously classified it as false positive, you know, that's something 0:13:34.580000 --> 0:13:40.320000 they should factor into the escalation criteria or to the escalation decision 0:13:40.320000 --> 0:13:44.060000 making process. So an example or practical check, an example here would 0:13:44.060000 --> 0:13:47.040000 be, you know, for example, the partial downloader that I've referenced 0:13:47.040000 --> 0:13:52.340000 multiple times. So the partial downloader rule with less than 1% false 0:13:52.340000 --> 0:13:57.500000 positive, you know, you set it to auto escalate on the first hit, you 0:13:57.500000 --> 0:14:03.720000 know, and then the other criterion category would be containment feasibility. 0:14:03.720000 --> 0:14:09.240000 This is where the tier one analyst is really playing with fire as it were. 0:14:09.240000 --> 0:14:19.160000 And they tier one analyst fully remediate this particular incident as 0:14:19.160000 --> 0:14:24.920000 it were within, you know, the constraints of the playbook, right? 0:14:24.920000 --> 0:14:29.900000 And the practical check or example here is, you know, for example, if 0:14:29.900000 --> 0:14:34.720000 yes, for example, think of, you know, blocking a hash or isolating a host 0:14:34.720000 --> 0:14:39.380000 and the risk is low, then they'll not escalate it to you, which is either 0:14:39.380000 --> 0:14:41.880000 going to be a good thing or a bad thing. 0:14:41.880000 --> 0:14:46.600000 Good in the sense that it's, you know, something that they can do have 0:14:46.600000 --> 0:14:54.020000 the ability to do and are within their remit in terms of, you know, roles 0:14:54.020000 --> 0:15:00.620000 and responsibilities to do, bad in the sense that they mark it up or they're 0:15:00.620000 --> 0:15:04.480000 not able to do it, especially eradication. 0:15:04.480000 --> 0:15:15.400000 And they don't escalate and, you know, a common example would be a persistence 0:15:15.400000 --> 0:15:19.100000 mechanism and the attacker is able to maintain their presence within the 0:15:19.100000 --> 0:15:24.900000 environment. So you can see how important the triage process is now. 0:15:24.900000 --> 0:15:30.840000 And then finally, the criterion category we have here is business hours 0:15:30.840000 --> 0:15:34.400000 and the SLAs or service level agreement. 0:15:34.400000 --> 0:15:39.680000 So the typical questions ask, this is very common, very, very common. 0:15:39.680000 --> 0:15:44.080000 If you've worked in technical support, you already know about SLAs or 0:15:44.080000 --> 0:15:48.100000 customer support, even as a SOC and tier one analyst. 0:15:48.100000 --> 0:15:51.960000 So the question would be, does the policy demand a human review for critical 0:15:51.960000 --> 0:15:53.920000 alerts 24 seven? 0:15:53.920000 --> 0:15:57.660000 The practical check and examples in this case would be, you know, for 0:15:57.660000 --> 0:16:02.940000 example, after hours critical automatically escalates to on call IR lead 0:16:02.940000 --> 0:16:09.320000 or whoever is responsible for receiving escalation, as it were. 0:16:09.320000 --> 0:16:12.700000 So that's the escalation criteria. 0:16:12.700000 --> 0:16:17.100000 More simply, you know, when does an alert, this is the criteria that determines 0:16:17.100000 --> 0:16:22.880000 when an alert becomes an incident or when an alert is escalated, right? 0:16:22.880000 --> 0:16:25.420000 With that being said, that's going to be it for this video. 0:16:25.420000 --> 0:16:27.540000 And I will be seeing you in the next video.