Detection Engineering with Google Cloud

View Show Notes and Transcript

Detection rules aren’t just for fun—they’re critical for securing cloud environments. But are you using them the right way? In this episode, Ashish Rajan sits down with David French, Staff Adoption Engineer for Security at Google Cloud, to break down how organizations can scale Detection as Code across AWS, Azure, and Google Cloud.

  • Why prevention isn’t enough—and how detection fills the gap
  • The biggest mistakes in detection rules that could blow up your SOC
  • How to scale detections across hundreds (or thousands) of cloud accounts
  • The ROI of Detection as Code—why security leaders should care
  • Common low-hanging fruit detections every cloud security team should implement

David has spent over a decade working in detection engineering, threat hunting, and building SIEM & EDR products. He shares real-world insights on how companies can improve their detection strategies and avoid costly security missteps.
Questions asked:
00:00 Introduction
03:06 What is Detection as Code?
03:41 What was before Detection as Code?
05:36 Business ROI for doing Detection as Code?
07:49 Building Security Operations in Google Cloud
12:41 Threat Detection for different type of workload
14:54 What is Google SecOps?
20:36 Different kinds of Detection people can create
24:46 Scaling Detection across many Google Cloud accounts
28:47 The role of Data Pipeline in Detection
31:44 Detections people can start with
34:14 Stages of maturity for detection
36:43 Skillsets for Detection Engineering  
39:32 The Fun Section

David French: [00:00:00] Any leadership or, blue team or defensive practitioners who are dealing with their detective security controls they know that, we're not just writing detection rules for fun. We need to treat them as being just as important as our preventative controls. So as I mentioned earlier, if you've got people logging into security tools and are able to create, update, or delete your detective security controls like on a whim or without any testing or approval. That's something that detection as code, I think, addresses.

Ashish Rajan: Detection as code is not just possible in AWS, Azure, but all three of them these days. And today, I had a chance to speak to David French, who is a Staff Adoption Engineer for Security at Google Cloud.

And we spoke about if you're starting Google Cloud today, or if you come from an AWS and Azure background, you went through an M& A and now you're multi cloud because of Google Cloud. This is the episode for you because in this episode we spoke about how do you even start doing detection in Google Cloud?

What are some of the logging requirements you need to maintain? What are some of the foundational pieces you need to have in your team when you start building detection? And how do you scale that across a [00:01:00] large amount of Google Cloud accounts? Now, in this conversation, we also spoke about the skillset and the challenges of doing so.

What is a good detection indicator type, the behavior type. And if you know someone who is working towards building a detection as code capability, perhaps they're starting today because they recently found out they have Google Cloud in their environment as well, and really large footprint of it. This is definitely the episode for them.

And if you have been listening or watching a Cloud Security Podcast episode for a while, this is probably your fourth or fifth time. I would really appreciate if you can take a moment on whichever audio or video platform that you're listening or watching this on, whether it's YouTube or LinkedIn or Apple or Spotify.

Definitely take a second to hit the subscribe and follow button because it means a lot to have your support with us and helping us get known with more people. Thank you so much for supporting and I hope you really enjoy this episode of detection as code as much as I enjoyed talking to David and I'll see you in the next episode.

Peace. Hello and welcome to another episode of Cloud Security Podcast. Today I have David French with me. Hey man, thanks for coming on the show.

David French: Yeah. Thanks for having me.

Ashish Rajan: First of all, I know we have the topic of [00:02:00] detection as code, but before we get into any of that if you can share a bit about yourself your professional background, so people get an idea about what's the journey you've traveled and where you are these days?

David French: Yeah, sure. I've been in security now for about 10 years. I'm currently at Google Cloud. I'm a staff security engineer by trade specialized in detection engineering. I started off as a stock analyst. I moved on to detection engineering, threat hunting for financial institutions and large software development companies.

I've also worked on the vendor side as a security researcher and engineer, building out SIEM and EDR products and developing detection content for those platforms.

Ashish Rajan: Yep.

David French: Yeah. Currently at Google Cloud. This is a role where I'm advocating for defensive security practitioners who use Google SecOps.

And yeah, that's currently where I'm at. Love to share knowledge, log in, speaking at conferences, sharing code, all that kind of stuff. More recently I've been sharing kind of best practices and example code and tutorials for people to get started with implementing detection as [00:03:00] code.

Ashish Rajan: So what is detection as code for people who may have not been introduced to that concept yet?

David French: Yeah, detection as code this is something that's been appearing in conversations more frequently the last four to five years, essentially detection as code is applying software development practices to the creation and management of detection content. And when I say detection content, I'm talking about rules that identify certain behaviors or signatures.

Yeah, so really with that, you're borrowing. DevOps practices that have been around for a while. You're using things like GitHub, software development platforms, CICD tools like GitHub actions and GitLab CICD, all that good stuff.

Ashish Rajan: So I guess maybe to paint a bit more idea for people as well.

What was it before? I guess obviously what happens in a non detection as code world? Obviously specifically if we can talk about Google Cloud as well, but in the kind of talks you give and the people you talk to as well, what's the pre [00:04:00] detection as code world and which led us to this path?

David French: Yeah. So the traditional approach is let's say you work in a SOC for a large financial institution. Just as an example you have tools like a SIEM, an EDR, firewall, email gateway all of these tools and yeah, this traditional approach is people logging into those tools manually right in the UI.

Ashish Rajan: Yep

David French: And they're making use of detection content from vendors that's already there. They might be writing their own custom detections.

So they're really operating in the UI and making changes, creating rules, dealing with the whole life cycle of a rule in all these various tools. So with that approach, and by the way, that's still fine for many companies, if you're a smaller company with a smaller team but potential shortcomings or pitfalls of that are changes are being made by individuals without input, testing, review, approval from anyone else on the team.

And [00:05:00] in my experience, what I've seen is people make mistakes, right? You can blow up your SOC's queue with force positives or page people out of bed, if you make a mistake, tuning a rule. Or worse you misbehavior, right? Whether it's an attacker coming at you and you find out after the fact that someone tuned a role and it led to false negatives or I've experienced the red team coming up the blue team, right?

And then you missed it off and whole team feels bad. Leadership are asking what's happened. It's terrible for morale. So yeah, those are some of the shortcomings with that traditional approach. But like I say, still works for some teams, right? If you're a smaller team, smaller company.

Ashish Rajan: So is this more of a, and because you're talking specifically about SOC in Google Cloud environments, but what is to what you said that so many different parts people have to look at, especially the SOC team, it's just not your Google Cloud, you're looking at probably some kind of email gateway, whether it's G Suite or something else.

You also have other components, other applications, other softwares. You also mentioned earlier that people may be using GitHub [00:06:00] Enterprise, or some kind of CICD pipeline. Just to give some idea for people, is that more a, from a security operation practice perspective, what is the incentive for, say, any director or VP of security operation listening to this, who may be for in a small organization or maybe in a mid to large or large enterprise kind of organization, what is the business ROI to even validate? Why do we need to invest time into detection as code? Because sounds to hear what you said, maybe it's a good thing, but what's the business ROI with it?

David French: Yeah, I think any leadership or, blue team or defensive practitioners who are dealing with their detective security controls they know that, we're not just writing detection rules for fun. We need to treat them as being just as important as our preventative controls.

Yeah. As I mentioned earlier, if you've got people logging into your security tools and are able to create, update or delete your detective security controls, like on a whim or without any testing or approval, that's something that [00:07:00] detection as code, I think addresses, right? I think the first step thinking about implementing it or the value of it, right? Is that you've got all these tools that we mentioned before, SIEM, EDR, firewall, and you've got existing detection content out there.

Ashish Rajan: Yeah.

David French: The first step really is like taking inventory of what you have, pulling it into a single location. So you've got a single source of truth and then deciding on, okay, what's my schema, what's my change control process around.

Am I going to test for regressions when anyone is suggesting changes, and then who needs to approve it before those changes get deployed out to my tools. So it's really just, I think, a sign of maturity, moving towards that kind of method where you're just taking your detection content more seriously.

And giving it that care and feeding and those guardrails around it to control changes that happen to it.

Ashish Rajan: I think you touched on something really interesting over there as well. In terms of most of the security products people work with, they're all about threat detection. And we're [00:08:00] already doing detection kind of world anyways.

And my primary assumption has always been that, hey, prevention is probably the best part. Like prevention is better than cure, as they say. But prevention is really hard to do and it takes a longer time. In the meanwhile, detection is the one secret weapon you can keep using to atleast, every time a bad behavior or something which should not happen occurs, you have some kind of detection for it.

And I, it almost feels, the reason I asked for the business ROI thing is because a lot of times when people start working on it, it almost feels like asking how long is this piece of thread a question. Whereas that, hey, what am I and obviously the answer may be different for different people, but I'm also thinking about people who are building a security operations team in the cloud today, maybe specifically Google Cloud as well.

What are some of the foundational pieces they would have to consider? And by the way, this doesn't just for small company, but it could be people who already have AWS, Azure now have Google Cloud coming in. I don't know what kind of detection or SOC security operation I should be doing for Google Cloud.

So maybe [00:09:00] to lay the groundwork for from maybe some native services, perhaps Google Cloud has or what can people consider when they are building a security operation capability for the Google Cloud business?

David French: Yeah, I think so Google SecOps comes to mind which is our security or security operations platform driven by Google Threat Intelligence.

But before we dive into that, we can talk about some of the logging that's available in Google Cloud, right? Yeah. So just the fundamentals, understanding what Logs are available to you and what events are logged when certain things happen in your environment is a good foundation so starting with Google Cloud audit logs think of these as the who, what, when, where of things happening in your environment, right?

Creating an audit trail. So subtypes of that, you have admin activity logs. They're always enabled, can't turn them off. Things like a service account key being created in your environment or two step [00:10:00] verification policies being changed, storage bucket policies being changed, that kind of thing.

So just important things, important or notable things happening in your environment. And then you have data access audit logs. So think of this as API calls being logged when access is happening to storage buckets or database queries are being run, that kind of thing. Yep. That volume is obviously going to be quite high if you turn everything on.

And if you ship it to whatever SIEM you use, you're going to, you're going to run up. A large bill, if they bill you by volume so there are filters you can figure right to, to decide what gets sent to your third party system, whether it's your SIEM or Google SecOps.

Ashish Rajan: Yep.

David French: And yeah, other important ones to call out really, I would say you've got cloud DNS logs.

Who's calling out to what any command and control or crypto mining kind of call outs happening, that kind of thing. DLP events, Cloud Armour gives you visibility into web app attacks that might be happening. And then of course, if you [00:11:00] use Google Workspace, all that stuff, people accessing your data, making changes admin activity. Any suspicious email activity coming through. So Google SecOps being our security operations platform, right? What we have in there are things called curated detections.

And they're developed by Google Threat Intelligence. And think of those as packs of rules, right? So if you use Google Cloud or Google Workspace, or Okta, AWS think of these like main platforms, right? You find in a modern enterprise. We have rule packs for those to detect suspicious or outright malicious behavior, right?

And those are developed by, yeah Google threat intelligence. And what SecOps does as well, it ingests intelligence feeds from various sources as well, right? One might be data from VirusTotal, right, on, on files or IP addresses or domains. You might have another one that detects activity from tour IP addresses or IP addresses associated with VPN or [00:12:00] anonymization services, all that stuff.

So what we do with those curated detections or those packs of rules. We tell you which logs you need to onboard into SecOps to feed those rules, events in the detection engine. Essentially you're being strategic about what data you want to filter and ship to your SIEM, right?

Because A cost, you need to be mindful of, and B if you're looking for a needle in a haystack and you're just onboarding everything, just background noise of systems just existing and doing their thing, you're just making that haystack even bigger, right? You're making it harder for your analysts or detection engineers to find that signal in the noise, really. So yeah I'd say start there.

Ashish Rajan: Because as you said that and coming from like a one of those AWS Azure kind of background. I'm sure there are people who are in that space hearing this conversation going.

Okay, I know it's certain things in Azure and AWS kind of thing. For example, the way someone would do security in a AWS or Azure would have an [00:13:00] identity to begin with. That hey is it so there's a concept of local AWS users called IAM users. Essentially, the idea is that it's not just the Active Directory Enterprise users that I have, but I also happen to have local users.

You mentioned service accounts earlier. It's a great example of it. Non human users on all of that. So that was your cloud audit logs that kind of covers that and you can put some detection for it if you want, and then you're going to go a step layer into your network security for lack of a better word in terms of detection where you have the access for storage devices.

Probably some web facing services as well. But a lot of people may also have virtual machines, projects that are they're running policies they want to have. Policies, could be prevention policies, but are there for the different kinds of workload, like most days, most of these workloads, these days are Kubernetes or I don't know anyone's doing virtual machines anymore, but most people seem to go down the path of Kubernetes these days.

But if people had both, what are some of the things [00:14:00] we can look out for there from a threat detection that they could use from a logging perspective to set the right foundation and also to start maybe addressing some of the tactical things quickly that they can look at?

David French: Yeah, I think probably some thinking about like compute instances and just typical suspicious behavior that might stand out in your environment.

So we've got detection content for things like reverse shells, right? And compute instances. Or if you're doing infrastructure as code at your company, is someone logging into the console and making changes suspicious, right? Or should changes only be made by certain accounts? And if someone's logging in and making changes, whether its an admin whose bypassing processes or an attacker. You can look into that. And then yeah there's other low hanging fruit, like audit logging, getting disabled, people changing.

Ashish Rajan: But is this like a playbook in SecOps or would you say SecOps is the I'm thinking [00:15:00] more, there's a similar one in AWS called Amazon GuardDuty, which is like a threat detection, threat intelligence service, which looks at say the top five services within AWS to know, hey, who's doing crypto mining, who's been doing SSH brute force, who is potentially being attacked or someone's like a malicious DNS, the kind of things you called out.

So is SecOps like a managed service from Google Cloud for threat information that people can use to build detection on in their SIEM? Or does it come with a pack like the ones in Azure Sentinel, where you can actually make your own playbooks? What's the, if you double click on the whole Google SecOps thing, so people understand that a bit more, because a lot of people prefer using native services from the cloud provider because you get a lot more for a bit of a bang for your buck. There's a lot more integration in there. So how would you describe Google SecOps from that perspective?

David French: Yeah. So Google SecOps runs on Google Cloud, right? Yep.

And think of it as, as [00:16:00] doing more than a traditional SIEM, right?

You can certainly ship logs to it. It ingests them, it parses them into a normal normalized schema so you can easily search across data sources. You've got those curated detections that I mentioned that run over those logs and detect not just cloud but endpoint behavior as well and all that, right?

Ashish Rajan: Oh, like G Suite and everything.

David French: So yeah, there are Google workspace packs, but I'm also talking attacker tradecraft on Windows endpoints and all that, right? So it's not just cloud. And yeah, like you mentioned there's a rules engine running in there and you can go into that and write your own custom detections.

Ashish Rajan: Yep.

David French: Because each detection content is not a one size fits all approach.

I've worked at a vendor before where I'm writing detections to run on thousands of customers, and there's just some tuning and baselining that needs to go into that. And yeah, so SecOps to expand on that it's also got SOAR capabilities.

It also does enrichment of your events [00:17:00] automatically. So you could ingest the metadata from all of your Google Cloud resources into SecOps. So when an alert fires or an event's logged it's enrichable, that information. So I have to go in my console and start clicking around so usernames metadata about departments or Compute instances buckets all that stuff.

So all this stuff that we would be clicking around trying to connect the dots It does that for you so you can spend more time worrying about detection response and investigation, essentially, instead of the quality of your data. In addition to that, we've got this thing called Security Command Center Enterprise also runs on Google Cloud.

And you can think of that as a multi cloud solution that helps you detect and respond to threats, right? So it works on, it works with Google Cloud, AWS, Azure.

Ashish Rajan: Yep.

David French: So when you have think of that as adding a CNAPP to SecOps.

Ashish Rajan: Oh, right. Okay. Wow.

David French: Yeah. So with that, you have all of those detection kind of content packs I mentioned [00:18:00] earlier.

Yep. That run on those platforms but it also gives you the ability to configure policies and rules in your environment. So the ones out of the box will be mapped to things like NIST controls, right? So if compute instances shouldn't have a public IP address assigned, it can flag for that. If you've got things like overly permissioned or overly privileged service accounts it will just flag these things, right?

When your environment starts to drift and it can either fix them automatically. Or it can just alert you with a risk score, right? And it, another cool thing it does, it can it's got this thing called virtual red teaming. And what it will do whichever of those kind of big cloud service providers you use, right?

It will go in and it will grab the metadata from everything in that environment. And then what it does, it graphs that out and then it runs these red team playbooks against it. To see where your weaknesses are.

Ashish Rajan: Oh, wow. Okay. Yeah.

David French: So things like I mentioned overly permissioned service [00:19:00] accounts.

If you've got poorly managed SSH keys where if an attacker exploited an instance and got in it will highlight where they can move laterally to further infiltrate your environment.

Ashish Rajan: Yep.

David French: Yeah. So things like that. Yeah. And that's how multi cloud kind of CNAPP solution.

Ashish Rajan: Interesting. So based on a MITRE framework for cloud root cause for cloud incidents they found that one of the top three was more around, hey, a lot of compromised keys or credentials and people talk about, hey, the threat intelligence feed that Google SecOps has, does that kind of include that information as well?

So it looks at the fact that, hey, by the way, I see compute instances using a compromised SSH was just found GitHub or whatever. It flags it. Does that kind of thing would be like one of the low hanging fruits that people can look at as well?

David French: Yeah, that, yeah, I don't think SecOps does that out of the box.

I can come back to that, but security command center enterprise definitely.

Ashish Rajan: Okay. Okay. I was talking the wrong product, but so the security [00:20:00] command centre does do that as well.

David French: It does. Yeah. Yeah. And if you use a service like GitGuardian or TruffleHog was filming, maybe Semgrep does that as well. I think, you can certainly ingest those alerts into SecOps. And then you could have that either presented to an analyst in a case, or you could link that to a playbook where you could go out and remediate, right? You could rotate the secret or disable an account or whatever your response section might be. So yeah, there are other ways to integrate that information.

If you're already paying for a service that does that kind of Keeping an eye on the web to see if anything is leaked or gets out.

Ashish Rajan: Fair. And to what you mentioned, the different kinds of threats as well, people can have their own custom detections that they want to create. I guess when we were talking about this earlier, you were talking about the different kinds of detection, like the whole the two kinds you mentioned, like what are the kind of detections people can expect that they can work with?

Or they should consider as they start building on that, I want to say it has like a detection engineering capability. Maybe [00:21:00] Google Cloud becomes that first cloud in their organization where they start building detection engineering capability. Because of capabilities like Google SecOps and security operations center.

How would you describe the kind of different kinds of detection people can create and if there's an intelligent way of doing it as well?

David French: Yeah, so I think understanding if I'm a detection engineer starting a new company, right? And I'm tasked with figuring out what detection content we currently have, what logs we have, what our vendors supply us with, and like, how I can build upon that to Increase my coverage, right?

Because every organization is different. So yeah, I understand your organization and the vertical you operate in and the threats to your organization. So what attack groups are known to come at your organization? What are their typical tactics, techniques and procedures, that kind of thing. Typically, you will want to make use of the intelligence sources you already have. So if you're paying for Google threat intelligence or any other [00:22:00] vendor that is providing you with feeds of either indicators or their own rules, you want to certainly deploy that or correlate indicators over your logs.

Looking for things like activity happening in your environment coming from VPN or anonymization services that your company hasn't approved for use, right? If I use I don't know, Palo Alto's global protect as my VPN client and everyone at the company uses that someone coming from Nord VPN I definitely want to take a closer look at that, right?

Ashish Rajan: Yep.

David French: So taking care of those kind of detections, like what's happening in the wild. What is my intelligence telling me capitalizing on that and exhausting all of that first. And then in terms of looking for, I'm talking about Google Cloud again, right? Defending my Google Cloud environment. Typically if you have a detection and response engineer or a SOC analyst, who's looking at logs and writing detections, they might not be an expert cloud security engineer.

So one thing I've [00:23:00] had success with at other companies is make friends with your engineers and admins, right? Get them on a call and workshop. Hey what's your day job? What are the five things that should absolutely not happen? They'll say, oh, okay. We don't allow service account key creation, or maybe we do from a couple of accounts.

So that would be suspicious because we use workload identity federation instead, or we use Terraform to manage all of our, our environment or infrastructure. So if anyone's logging into the console and making changes that would be suspicious. So Yeah, making friends with your admins and at a large enterprise.

You might not be the person identifying data sources and deciding what gets on boarded to your SIEM or whatever platform you use right for monitoring and detection so making friends with admins you can get into the console with them and look at the logs like what i'll give you an example, GitHub enterprise has got audit logs and If you turn those on and ship them [00:24:00] to your SIEM out of the box, they don't log the source IP address in the end.

Oh, so you could have a detection that looks for a personal access token being created by a GitHub user and they've cloned a hundred repos. You go, okay what's this guy doing? Like, where's he where are they coming from? And you don't have the IP address. So that's a prime example of don't just turn a data source on and ship everything to your SIEM without kind of understand the data source and exploring the settings and, carrying out some actions or simulating some attack behavior and seeing what date you get. Yeah, because by the time an alert fires and you go and look at the alert, you're, you're disappointed, right? Because your data is incomplete or missing.

Ashish Rajan: Yeah, fair. And I guess

David French: I won't tell you how, I won't tell you how I found out that one.

Ashish Rajan: Yeah. I was going to say, I didn't realize it doesn't have the source IP. It's makes it a bit more You have to turn it on. Considering, it's probably like the place where your source code is as well.

And you want to be aware of who's using it and where. So maybe they can check [00:25:00] out one of your talks about it. Cause I think you've laid out something that's really interesting, the whole idea behind whether you're building a detection based on an indicator or whether it's a behavior based one how does one, once you start building it, how do you scale it out into a, I've obviously we're talking about a very small example of one Google Cloud account, but it's I don't think it's common anymore to just have one Google Cloud account these days. I don't know which company probably has hundreds going on at any given point in time. How does one scale detection say, for example, to what you said, you may hear this conversation or watch this conversation and make one detection out of this because, hey, I spoke to I don't know, Jennifer in the development team and she gave me this information.

Now I know that, Oh, I need to look for some kind of secrets across my entire Google. I'm just making an example here into my GitHub enterprise. Now, if it was a Google Cloud specific detection, how do you scale that from one account where you've tested it?

You're like, yep, David, I'm happy with this to scale it out [00:26:00] to hundreds of Google Cloud accounts, or maybe thousands.

David French: Yeah, this is prime application for detection as code. Going back to that. Yeah, let's imagine someone in your organization is giving you some intelligence or an idea on a new detection.

They want to put in place. My process, if I jump into my CM or whatever I have and start writing detection logic. Without having the underlying events when that behavior is carried out in my environment, you're essentially shooting in the dark there. So what I like to do is in a lab environment or a dev environment, right?

I'll pretend I'm the adversary and carry out those actions or simulate that behavior and this will generate the events in my logs that I want my rule to match on, right? By that point, I'll decide if I want to just have a single event rule, right? Think of that more like a signature, right?

Something happened. One, one event or detect on a sequence of behavior or a cluster of behavior happening correlated by a user account or a host or an [00:27:00] IP address, right?

Someone is enumerating my billing settings in a Google Cloud project or organization, right? And then they make various API calls to enumerate billing settings. Maybe they move on to spin up a bunch of compute engine instances. Maybe they. they've got hold of an account that can assign other permissions to itself.

So they start doing that. So whether these events happen in a sequence or a cluster of behavior, or if you've implemented some kind of risk scoring I generally start there, right? So you've got your initial version of your detection logic. If you write a rule, you own it. So I wouldn't just throw that over the fence and start alerting my SOC or detection and response engineers, especially if there's a paging them out of bed, right?

You're just going to annoy people and it wastes time. Yeah, you write it, you own it in that sense. So I would probably monitor that detection for a week or two, depending on how much of, if this is an emergent threat versus attacker tradecraft that's been out there for a while and something I just want coverage for my organization.

So I would eat my own dog food, right? I'm getting alerts from that for maybe [00:28:00] one project or two projects in Google Cloud. I'm tuning it based on what's normal for my organization any anomalies that might happen. And then, yeah, essentially going back to detection as code. If I've got that in say my code base in GitHub and I've defined which, which log types or which, what logs from which Google Cloud project this detection is running on I can easily change that when I'm ready to scale it and define what projects or orgs I want that rule to run on.

Yeah, and then going from there really, so you, you're not really done at that point when you have a detection, you're constantly responding to alerts from it, triaging alerts, tuning it. And eventually, technologies come and go, you might have, if you never do any testing or review, you might have a rule that's running and not being fed any logs or is trying to detect the behavior for an application your company doesn't even use.

So it's this life cycle, right? You never really done.

Ashish Rajan: And I think at some point in our conversation, you also mentioned data pipeline as well. Where does that kind of fit into all of this?

David French: Yeah. So [00:29:00] again, yeah people were talking a lot more about, in terms of detection engineering, people were talking a lot more about data pipeline data quality, or some people were using the term data fabric.

It means you're not just writing, you're not just shipping logs to a centralized location, right? Like a SIEM and running detection rules over that. You're not done at that point. People are caring a lot more about data quality these days. So when you think about that data pipeline, something happens, an event is logged somewhere along the line that event gets paused.

And you're writing detection rules for it. As I said before, environments drift, right? So data quality issues happen, right? They might be latency between the event getting logged and it getting to your detection engine. Vendors this is probably the most annoying one vendors change their logging schemas, right?

They change their logging schemas on us. It's just a fact of life and field names [00:30:00] change or values in a field that your rules matching on changes and your detection breaks silently, and then you miss behavior or it just sits there for months, right? And no one notices. Yeah, when we think about a data pipeline it's just caring more about the quality of the data that you rely upon.

And if you're a detection engineer, I almost guarantee it. No one cares about the quality of that data as much as you do, right? We spoke about a large enterprise where you've got admins looking off the different clouds and different SaaS applications, that kind of thing. They might turn on the login for you and ship it to, to where your detection engine is but I don't care about the quality as much as you. Implementing what I like to call health checks or probers. I have, I don't know, I care about my Google Cloud audit logs. I can just have a health check that reaches out to my Google Cloud environment, carries out this kind of test action whether it's just a simple API call from a certain account and I can have a rule running that's monitoring for that behavior. It fires an alert. It gets closed by my source. And no one ever has to look at it. [00:31:00] But if that doesn't happen, right? Every hour or every six hours or every day, however long, whatever cadence I'm doing it. My team gets an alert to say, hey, we're no longer getting these logs or they're not being parsed properly or there's latency.

So really, I think just being more proactive about the care and feeding of your data. That's where the data pipeline kind of stuff comes in.

Ashish Rajan: So being a bit more intentional about, building your own pipeline so you can feed more detection in the future as well.

David French: Yeah, absolutely.

Yeah. Like I said, you don't want to ship the kitchen sink from a cloud environment to your CMU. You're going to run up a bill and increase the amount of noise that someone's going to have to wade through.

But yeah, just being more deliberate about what you filter, what you sanitize, what you transform, what you enrich.

That will falls under data pipeline management in my opinion. Yeah.

Ashish Rajan: And I guess are there any top three, maybe not difficult, but maybe easy detections that come to mind that people can actually test out themselves and I don't know, could be anything, could be a storage device supported the [00:32:00] internet or whatever comes to mind for people to wet their feet with, Hey, sounds like a great idea.

I probably should start doing it. Are there any common detection scenarios that come to mind that people can dive into as a starting point without, it could be quite overwhelming going to another developer and finding out that there are these 25 things that they hate about the application that you have a great detection for.

Are there some easy detections people can try out themselves to wet their feet a bit and then start building from it?

David French: Yeah. Thinking about long lived credentials, service accounts is a good example. So if you have, if you allow service account keys in your environment just exploring or baselining or understanding, like what does service account key use or creation look like in my environment? Who's creating them?

What's my process to, to create them. Looking at the kind of key rotation interval as well, right? So if you've got one that's only rotated once a year, like dive into that and look into that. It's not always about [00:33:00] detection. You can also explore and build your case to implement better prevention, right? Or control. So if we do a lot of service account keys, can we rotate them every 30 days? If we other workloads, get up actions. GitLab, CICD, CircleCI, like insert vendor name here. What does service account key usage look like in that environment? If someone gets hold of that key, are they going to have access to my environment to carry out actions for six months?

Or can we move to something that's more secure, aligns with best practices like workload identity federation, right? I can go back to my detection as code pipeline, right? I use. Workload identity, federation, and GitHub actions. When that calls out to my Google Cloud environment, that token is only valid for three minutes which is more than enough to manage my detection content in my security tools.

Yeah, just, I don't know, maybe want to get started some homework baseline, what does that look like in my environment, in my Google Cloud audit logs.

Ashish Rajan: Yeah, and to your point, the logs that are available [00:34:00] in Google Cloud anyways so they can perform the action themselves and see what events get triggered to what you were saying earlier as well to really know what they would be building detection on top of,

David French: yeah, absolutely. Yeah, and like I said, every organization is different. So just baselining, understanding what's normal and then going from there, really.

Ashish Rajan: So what would be for, maybe for the security operation senior people or the leaders in there who probably would like to have some maturity to their detection.

They maybe already have common detections already there. What would be the different maturity stages for a, I don't know, like a crawl, walk, run phase for building detection as code capability?

David French: Yeah, I think take an inventory of what you have, right? What detection content is currently in my tools. What does my vendor provide out of the box?

What am I making use of there? And then figuring out, do I have a single source of truth for my detection content? Whether you're pulling everything back to a code [00:35:00] base like in GitHub Understanding what that coverage looks like, right? What are my data sources that I care about?

What do I have today? Mapping rules to MITRE ATT&CK is just something that we've been doing for years, and it's a valuable exercise, unless you're playing bingo and just saying I've got one detection per cell and high fiving yeah, firmly against that yeah yeah yeah, or I won't go there.

We've got 100 percent coverage, right? Anyway yeah, so do I have that single source of truth for my detection content? Do I understand my current coverage where my gaps are, where I want to go there? Do I have change control over my detection content or can someone just log into my tools and disable it or modify it?

Ashish Rajan: Yeah, actually that's a good point. Yeah, that's a good one.

David French: And then just measuring things like, this emerging threat comes out, like, how quickly can I create and deploy a detection? How quickly can I handle detection tuning when someone reports a false positive or misbehavior? Things like how quickly can I roll out changes to [00:36:00] detections?

What's my mean time to detect? What's yeah, go, yeah. When I think of mean time to detect performing some analysis and reporting around when the event occurred, when my detection fired, and when someone picked it up. If something happens in my Google Cloud environment, for instance, and I don't get an alert for three hours later, depending on what that was that the attacker could have carried out a lot of actions.

Ashish Rajan: Yeah.

David French: Yeah, so some fruitful thought there, I think, or recommendations. I'd also recommend checking out Kyle Bailey's detection engineering maturity matrix. He's mapped everything out by people process technology in steps like or stages defined, managed, optimized. So if you run a detection engineering program or just getting started, I think that's a great resource as well.

Ashish Rajan: Actually talking about running a team for detection, what kind of skill sets do you think people need these days for, we're talking about building detections, data pipelines, sounds a lot more than what people traditionally believe a SOC analyst [00:37:00] is these years though. What are some of the skills that people at that level need now?

David French: Yeah, absolutely. Yeah. Starting with SOC analysts, right? If you've got that investigative or defenders mindset where you might not be able to write code, you might not be able, you might not be familiar with software engineering practices. But if you're able to explore logs and do log analysis or triage and alert, and figure out if it's a true or false positive when the behaviors in our environments I think more complex now than just Oh, Windows endpoint got popped and I have to worry about the main admin we're talking about cloud environments where you might pick up an alert and that asset doesn't even exist in your cloud environment anymore, right?

It's just something that spun up and spun down, but there are logs there. Yeah, if you can pick apart alerts and analyze logs and come to conclusions. You're still definitely an asset to your security team. And we need those people, like the defender's mindset is hard to teach.

Ashish Rajan: Yep.

David French: And then, [00:38:00] yeah, building upon that so detection engineering, I think people have been writing rules and signatures for decades, right?

But I think detection engineering is a discipline where you care about data, the quality of your data. You care about picking apart attacker tactics, techniques, and procedures, how to simulate them. What logs do I need? Like, how do I code and stitch API's together and what's the art of the possible in, in terms of not just writing a query in a SIEM query, like a SIEM search box.

But how can I stitch together like various data sources, do enrichment, alert on anomalies. Really diving deeper than just writing a signature, that matches on an indicator. Yeah, those detection engineers or Oh, data geeks, right? People who care about the quality of data. And yeah, I think in what we do as detection engineers people who also have basic and intermediate knowledge of software engineering, DevOps practices, right?

So being able to write [00:39:00] Python code to stitch together APIs, basic knowledge of how CICD pipelines work version control and GitHub, that kind of thing. Yeah, so I think some skills that would make up a solid team. And I think practitioners with kind of unique experiences and skill sets will always make the best detections.

So you might have someone who's an amazing network engineer. Someone who knows Windows Endpoint and Attacker Tradecraft there. Someone who knows Google Cloud, AWS, so you can't know it all, right? So it's just, people have just got different things they can bring to the table.

Ashish Rajan: Be more collaborative, I think that's pretty much where we're going with this as well.

David French: Yeah, absolutely, yeah.

Ashish Rajan: Awesome, that's most of the technical questions I had. I've got three fun questions for you as well, man. Okay. Just about you, so you'll be we'll be steaming through them really quickly as well. First one being what do you spend most time on when you're not trying to do detection as code and solve that world of detection?

What's what do you spend most time on?

David French: Outside of work?

Ashish Rajan: It could be anything. Yeah.

David French: Yeah. [00:40:00] Yeah let's do outside of work. So I'm a Brit living in Colorado. I've been here for five years now. If you like the outdoors, this state is just a playground, right? I think I've got an addiction to buying outdoor gear now.

Like I've filled up my garage with everything you think of. So yeah when I'm not working I'm disconnecting from technology. I'm fishing, hiking kayaking.

Ashish Rajan: That's awesome. Yeah. Oh, that sounds like a fun activity as well. Second question.

What is your favorite cuisine or restaurant that you can share with us?

David French: Favorite cuisine? It changes. At the moment it's Indian food because I can't get good Indian food where I live in Colorado.

Ashish Rajan: Being a Brit, I can totally understand that. UK definitely has great curries, but I don't know man.

Colorado has the outdoors, but not the curries,

David French: yeah, the food where I live isn't great, but luckily my wife is a good cook. Oh, nice. I picked up an Indian cookbook, so I've been trying to hand at dal and lentils and curries and all that. Yeah, I miss that. Whenever I go back to England, I I have my [00:41:00] fill.

Ashish Rajan: Perfect. But but dude, thank you so much for answering those as well. And where can people find you and connect with you online to talk more about detection as code and how Google SecOps can help them perhaps better their detection in their organization.

David French: Yeah I think I'm most active on LinkedIn these days.

Ashish Rajan: Yep.

David French: I was going to say for better or worse.

Yeah yeah, I think I'm most active on LinkedIn these days. You can find me doing stuff on GitHub. I publish on Medium sometimes. I think I'm most active on our Google Cloud security community at the moment with my job. So yeah, if you've got any questions. You can reach out LinkedIn or yeah, Google Cloud security community.

I'll I'm around there. Yeah. So where you can find this.

Ashish Rajan: I'll I'll put those in there as well, but dude, thank you so much for coming in. And yeah, I really enjoyed the conversation. So I'm looking forward to having more of these as people try and make more detection happen in their Google Cloud environment, but thanks so much for coming in.

David French: Yeah. Thanks for having me. It's great. Thank you.

Ashish Rajan: Thanks, everyone. Thank you so much for [00:42:00] listening and watching this episode of Cloud Security Podcast. If you've been enjoying content like this, you can find more episodes like these on www. cloudsecuritypodcast. tv. We are also publishing these episodes on social media as well, so you can definitely find these episodes there.

Oh, by the way, just in case there was interest in learning about AI cybersecurity, we also have a sister podcast called AI Cybersecurity Podcast, which may be of interest as well. I'll leave the links in the description for you to check them out. And also for our weekly newsletter, where we do an in depth analysis of different topics within cloud security ranging from identity, endpoint, all the way up to what is a CNAPP or whatever the new acronym is that comes out tomorow. Thank you so much for supporting, listening and watching. i will see you next time.