Ashish Rajan: [00:00:00] For people who don’t know Magno who’s Magno, how’d you get to where you are today, man.

Magno Logan: Sure. Okay. So, so yeah, my background is on development, web application development. That’s where I got my college degree from. But I was always interested in security and during. my college, I was selected for a scholarship to study in the U S for one year.

I was one of the few Brazilian selected at that time was 2008, 2009. And so there, I got the opportunity to study some security courses and web application security, computer forensics, and all that stuff. And that really got me really interested into security. So that’s when I decided to shift my career to application of security because it was a field that was.

Just starting there at the time, at least in Brazil when where I’m from. Yeah, so I got back to Brazil. I finished my degree there and I started doing a post-graduate degree in information security [00:01:00] and started from there, learning about dos the last stop, their application security vulnerabilities and all that stuff. ,

Ashish Rajan: That’s pretty awesome. So wait I love the fact that as far as the developer and started taking on, I guess, more cybersecurity qualifications. So where do you work now? And what’s your current day job like?

Magno Logan: Currently I live in Canada, I work for Trend Micro Canada, and I do cloud and container security research.

So I’m part of a team internally it’s called Team Nebula. . And so it’s a small team inside, inside trend micro that really focuses on finding vulnerabilities and defining honeypots and studying attacks that’s related to cloud environments, serverless dev ops tools and also container and Kubernetes, Docker and Kubernetes.

Ashish Rajan: Interesting and probably the best person to bring in, to talk about Kubernetes security, at least the starting conversations . You do container and cloud security. So I can probably ask you this What is Cloud security for you?

[00:02:00] Magno Logan: Yeah, that’s a great question. I think, it’s a still , a concept that’s very cloudy to me. No pun intended there, but, what is the cloud? So it’s like, just someone else’s server, it’s just another data center.

So I think it’s a little bit more than that, but it’s using those. Services in a Self service way. So you can just, . You can go to, to any cloud provider today and just spin up a new instance, deploy a server, it has a lot of APIs that you can do automation with. Cloud security is protecting those environments that are being created around with the cloud service providers, but are also around this methodology off of API first and self-service and elasticity.

Ashish Rajan: I love the. answer you gave, because I feel like everyone answers in a different way. Cause to your point. Some people think are, Hey, if someone else’s server and some people just go like, no, no, no, no. It’s like this data center thing or whatever. So what about Kubernetes then for [00:03:00] some people who I’d be listening to this episode for the first time and although this is a Kubernetes month and we’ve been talking about Kubernetes needs for some time already.

What’s your definition of Kubernetes and what does Kubernetes security mean for you?

Magno Logan: As Docker was created and on launch in 2013, as a way to use containers to deploy your applications and have them kind of isolated and separated in, in each container, people we start.

Deploying and using that in a large scale. ? So like hundreds and thousands of containers and it was hard to manage. . So it’s like having to, to make sure that , you’re controlling and managing all those, those containers in your environment and make sure that they’re healthy, that they have the proper resources for CPU and memory and all that stuff.

And that if something goes down , that you need to know so that you can maybe spin up another container or try to fix it. . So I think that Kubernetes comes to solve this [00:04:00] problem here. And I know that there are other solutions out there, but. Kubernetes , is a way to, to manage,

It’s is a container orchestration solution to manage all those containers in a way , that’s kind of it does a lot of the, managed stuff for you, so you don’t need to handle, like, if you apply Kubernetes in your applications properly on a Kubernetes cluster, it can handle the rollbacks, the updates.

If a container goes down and can create another one from you, it’s easy to scale the replica and also scale down the rapid does if you need. So it’s a solution to manage all those containers that you need to manage right now.

Ashish Rajan: I love the answer because. It’s simplified it to what it needs to be, but also it kind of talks about why people cared about it so much as well.

So what’s the role of cloud native and all this, and why people keep using if it is container orchestration, why call it cloud native?

Magno Logan: So, Kubernetes is part of of this [00:05:00] ecosystem. That’s called cloud native so cloud native means a group of applications that were born to run on the cloud.

.And so as we discuss it, what is cloud and cloud security earlier? So cloud native means applications. There are declarative that they are API first that they usually run on containers, that they are easy to scale. So all that, characteristics are related to cloud native solutions and Kubernetes is only one product , that’s part of this cloud native ecosystem. So if, you know, if you heard about the cloud native computing foundation, which is a foundation, that’s a part of the Linux foundation, there are many other projects that are considered and that are part of the CNCF, so Kubernetes is just one of them.

And also Kubernetes itself uses other cloud native tools, , for it to work. So you have core DNS, you have that etcd and many others,

Ashish Rajan: also all of those components that people talk about, [00:06:00] and I would love to get into some of these components as well, etcd and kube proxy, and all that. Some of those components have been derived from CNCF open source project?

Magno Logan: Yes, exactly. Yeah. So, so at etcd Core DNS, S they’re all part of the CNCF ecosystem, and they’re considered also considered cloud native tools and cloud native projects.

Ashish Rajan: Interesting. So how would this be different say when people deploy Kubernetes in cloud versus on premise, or on their own self hosted one.

Magno Logan: So the problem is the complexity, ? So I personally think that Kubernetes is really complex. It took me a little bit of time to really understand the ins and outs and that , the bits inside how Kubernetes work, it’s not easy has it has learning curve. That’s, very complex. So I think it really depends on first, if you have the resources and the people to, to handle [00:07:00] that. . So for example, I need I just need to run a cluster. I just need to run some, some applications. And I don’t, I don’t want to know about the. I don’t want to manage that cluster.

So definitely go with the managed solutions that are provided and they’re offered by the cloud cloud providers. So you have EKS, you have AKS, you have GKE and many others, because that’s easier, you don’t need to manage some stuff, . And the cloud provider takes care for you .

On the unmanaged, or on premises, you have to handle everything. So it’s much harder and you need to have dedicated people , to be your cluster admins that know what and understand . What’s going on there. . So I think that’s, that’s the big difference. There is some, also some compliance concerns,

for example, if there are some regulations that don’t allow you to use this, this managed services, . You’re probably going to have to deploy your own cluster and manage that. So for compliance for government solutions, you might need to, to [00:08:00] deploy your own cluster as well.

Ashish Rajan: Interesting. And I love the comments coming in as well.

Something for quickly acknowledge some of those as well. So when he said, hello, Charles, hello, Jay loves the fact that you’ve off to a great start with IPA and cloud bonds already. And, and I hope I’m

using , I’m using it’s ice skating last.

Yeah. I was going to say that that is a very,

Magno Logan: very Canadian.

Ashish Rajan: Yeah,

There you go. We got one of those souvenirs and I love the comment from Heitor so Kubernetes is so important that it is the foundation or the architecture of 5g and future 60 mobile technologies. Security in the areas relevant.

Wow. Did you know what’s about the 5g and 6g,

Magno Logan: so yeah, they’re using Kubernetes for that as well. And there’s also another topic that Kubernetes is, is very it’s highly adopted right now. It’s Kubernetes on edge, [00:09:00] like edge devices, ? So your mobile phone, , your IOT devices, they all can be a node or part of a cluster.

So that’s this very interesting, even I seen people run Kubernetes on, Rasberry pies as well. So you have a cluster of a bunch of Rasberry pies and you can run Kubernetes as well.

Ashish Rajan: Oh, thanks. Well, thanks for introducing me to that concept as well. Thanks, man. I really appreciate the comment as well.

So now, since you kind of differentiate between the cloud structure, as well as self hosted one let’s get into the grid of I’m a security person, and I’ve told people that if you want to use Kubernetes security, this is the first point to start.

So what are some of the common, obvious entry points or that adversaries use as tactics to exploit kubernetes.

Magno Logan: Yeah. So the most common ways that a factor is getting into Kubernetes clusters nowadays, that’s what we’ve seen in and what we noticed from our research into our, [00:10:00] honeypots Kubernetes

so we analyze those weekly and there’s two main entry points, one is, I think the most common one is the API server, so if you have the API server exposed to the internet, it doesn’t have the proper configurations, or if it’s misconfigured, it allows people to deploy pods into your cluster.

without authentication or with a weak authenti cation, so that’s the main point of entry. Attackers are using either your Kubernetes API server if that’s exposed or your Docker Daemon API, if that exposed as well. So you have to be very careful, even if you’re just using Docker and not Kubernetes to not expose those API endpoints to the internet.

. So that’s the main thing. The second entry point of a Kubernetes cluster. It’s the application, so it’s, you’re running a web application that can be vulnerable to many different tasks, and so if there is a vulnerability on your application, that’s running inside the Kubernetes cluster, [00:11:00] and I can run commands inside that cluster, inside that pod of where the applications running.

I have remote command execution, so I can run commands inside the pod. I can get information from the cluster. I can try to break out of the pod and access the node. Right and get the cluster admin and all that kind of stuff. . So it’s still here, even though you’re using Kubernetes, you’re using containers, application security is still a strong topic, you need to be aware of. It doesn’t matter if your foundation is secure, your containers are secure. Your Kubernetes cluster is secure, but then your application is vulnerable. Your code is, is, is shit forgive me for using that word. But if you code it, if you don’t know how to, program and use secure coding approaches on your application, then there’s nothing that Kuberneteses can protect you from that.

Ashish Rajan: API server, totally get it. Docker, Damon as well, application as well. So does this vary between say having a cloud layer [00:12:00] on top of this, does that make it a different, like some SSRF in there as well. Would that kind of impact this?.

Magno Logan: Yeah. , so if you’re using Kubernetes in the cloud, , and I compromise a pod inside your cluster, I can reach the API metadata from the pod itself. I don’t need to break out of the pod to access the node , to get your API keys. So your access keys are secret keys.

I can do that , from the pod itself and depending on the permissions after those keys, it’s game over. So there are situations where attackers compromise your cluster , and they either deploy new pods, new containers to run crypto miners, usually Monero, or they installed the Malware, the Monero crypto miner inside your running pods already,

or they break out of your cluster. And they deploy new instances on your cloud environments. Usually be your instance. There are very expensive, and they start money mining cryptocurrencies there as well.

Ashish Rajan: Wait. [00:13:00] So just for the context of people who may not know what breaking out a container means, what is breaking out a container?

Magno Logan: Sure. Breaking, off with the containers is a form of privilege. Escalation inside your cluster, breaking out the container. The container is kind of an isolated process, that’s protected by the container isolation there. So breaking out of the continuum, accessing the hosts, a content, the host files, the host information.

So , that’s kind of a breaking out of the container. And one of the most common ways of breaking out of a pod in a Kubernetes cluster is deploying. A privileged pod that has access to your whole node. So Kubernetes has the master node or the control plane, which is the main node that controls all the other nodes that we call worker nodes.

So those nodes are the ones that run your applications. So , if I compromise one of that, I break out of that and kind of accessing the instance itself. It could be a VM or a better meadow. So I’m actually seeing that.

[00:14:00] Ashish Rajan: Interesting. So, all right, I’ve got, I’ve gone through this and I love the team that you’re actually going in the attack duration when it can come back and do some defense later.

So from a attacker’s perspective, okay. I’ve identified either it’s my application security or my Kubernetes API, or my Docker Daemon API, or it could be access to like a metadata , of the whole service. And I’ve managed to do some form of privilege escalation with escaping the containers.

Sounds like This to your point about the complexity. There’s like so many layers to this, are there like tools out there that I can, like, I don’t know. That tells me when it’s an API publicly available or am I just sitting on the internet trying to report on the, a URL or how do I go about doing some recon on this?

Magno Logan: Sure, sure. Yeah. So if you’re depends on your target, if you’re targeting a managed services, like from a, from a cloud provider, usually they have specific URL names for, for [00:15:00] those clusters. And so you can, you can do some recon on that specific URL format and look for either valid clusters or being there default API ports off the cost of the API server. Like, okay. If it’s unmanaged, usually sports seeks for, for free, if it’s a managed cluster, it’s usually unemployed for, free by default. So. You can do that. You can also use tools that are your are available to you like Shodan or Censys,

that scan the internet already. And you can look for specific indicators that this is a Kubernetes cluster. So looking for that specific board, looking for indication that this is, this is a cluster of Kuberneteses, so there that the API server, when it replies to it usually replies in a JSON format and it has a specific format for the responses.

So you can, you can check that as well.

Ashish Rajan: Interesting. And I think I want a follow up question from Gerald here. Are there specific tools to scan for Kubernetes and its vulnerabilities as well?

[00:16:00] Magno Logan: Sure. Yeah, there are many tools out there. I don’t think that there is like one single tool that you can do everything.

I know that Aqua, which is one company that does a lot of container and Kubernetes security work and research. They have a tool called kube-bench. So it’s a tool to assess your Kubernetes cluster, according to the CIS benchmarks. So it’s more like a defense, you need to look to see if you’re compliant with the CIS benchmark and there’s also kube-hunter,

so to kube-hunter, then it scans your cluster or target cluster that you use. You should probably have authorization to scan. You should have that. And it’s going to tell you, like, , what is compromised? Like what are the vulnerabilities write checks? I think it’s 30-37 checks of potential vulnerabilities, misconfiguration information, disclosure, everything, that in a cluster , that shouldn’t be exposed.

Those are the two main tools. I think there are many others smaller tools that he can [00:17:00] use, but still. Kubernetes , is API server, so it’s the main part of the control plane is an API server, so you can do either curl requests, you can use some web security testing tools also to assess your, your Kubernetes cluster as well.

Ashish Rajan: Awesome. And thanks for sharing those tools as well, but I definitely feel there is some more complexity that goes into trying to understand even just a threat landscape for this as well, to your point. This is just one cluster that we focused on when we spoke about API. And the container is there is this problem larger. If you have a massive cluster, like in enterprise where there’s not just one cluster, there’s like multiple clusters. What’s the low hanging fruits over there for trying to get in, like, what else is there, like a thing that affects multiple clusters?

Magno Logan: Sure. Yeah. So, Kubernetes, it’s not a tool on itself, it relies on other tools as [00:18:00] well. As I mentioned, it has the etcd, which is the data storage database for Kubernetes. It has the core DNS, which is the default tool for networking and communication inside your cluster. So all those tools you need to be aware of their configurations.

No, if they’re not exposed as well. For example,etcd, it also has an API. Right by default, it’s not exposed to the internet, but , if you’re inside the network, so one other thing that attackers are doing is they’re compromising the network or , the cloud environment. And then they’re looking internally for Kubernetes clusters,

so they’re already inside. So even if your API servers are not exposed, they can do a reconnaissance of the whole network and find API servers that are open on that network, because you think that’s protected, you’ll think, Oh yeah, no, nobody inside my network is going to attack my cluster.

But you need to be aware, not just off the insider threat. But also a attackers compromising [00:19:00] your networking and then scanning , for your Kubernetes cluster. Is there.

Ashish Rajan: So to your point etcd being that, I guess, for lack of better words, a key value store, which has all the cluster information, you can just be like looking out for that API in particular as well.

Magno Logan: Exactly. Yeah. If that exposure that’s exposed, if I can communicate with the etcd. So the way that Kubernetes works we say it’s a declarative state. So I tell Kubernetes what I want and, but I don’t care how it’s going to do it as long as it does it. So I either, yeah.

Requests or YAML files, I tell Kubernetes, Oh, I want it to cluster like that. I want this pods, I want these web application. I want this number of nodes. And I send that to the API server. The API server then sends that to etcd and etcd stores that information there. And then there are some components like the scheduler and the controller manager that checks etcd [00:20:00] to see.

Okay. , is what’s in etcd right now is what’s being reflected on my cluster. Oh, no, it’s different. They don’t match. It’s basically dif different verification, if difficult comparison, Oh, they don’t match. Okay. So we need them to match. So now I need to create the part, I need to create the web application so that what’s running in my, on my cluster.

It’s exactly what what’s on etcd. So that’s the problem, if I have access to etcd, I can create the parts. I can create the containers and everything. And then Kubernetes is going to take care of, for that for, for me or for the attacker. So that’s why , it’s a very sensitive piece in, in the Kubernetes cluster,

Ashish Rajan: actually. You’re because I can tell, like, I’m just trying to think if you actually had access to the etcd API, you can point it to launch an application, which is. A Docker container or a container that you host in Docker hub. You can point you to that if they allow for it, download it.

And because you will be in the pod, you can access all the [00:21:00] applications. Anyways,

Magno Logan: yeah, exactly. So, yeah, that’s, that’s a good, that’s a good thing that you mentioned network policies, so by default, Kubernetes doesn’t implement network policies,

it’s a flat network inside the cluster. Any pod can talk to any other pod and that’s a big problem, because if, my malicious pod can talk to the Kubernetes API server, which also running as a pod or the etcd, which sometimes is another pod, that’s a big problem. That’s a big,

Ashish Rajan: yeah. I love this man.

So you know, have you seen the attack meter thing for Microsoft, for Kubernetes? What are your thoughts on that? Like, does that cover, like, I was fine. Maybe why they introduce what that is and then give people don’t know what that is and does, but what are your thoughts around that?

If it’s actually valid and does a good job,

Magno Logan: There’s two matrix that there’s the, the Microsoft Kubernetes threat matrix that was created in April last [00:22:00] year. And it was released by Microsoft. The team, I think, is the security team at Azure that released that. And then there is now the MITRE they’ll fish out MITRE attack for containers.

So that that, that other matrix, that new matrix that was created by MITRE is, is a product of the other Microsoft matrix. Microsoft is, is the one is one of the sponsors after this new project. So last year, MITRE has reached out to the community in December last year, asking for a real world scenarios and data around , what the attackers are doing in those containerize and, and clustered environments.

So they reach out to the community. We saw their blog posts and our team was one of the first to reach out to them to tell them, okay, here, here’s the research that we’ve been doing on Docker and Kubernetes for over two years now, at least at least on Docker and Kubernetes since last year. And, and here’s what we’ve seen.

Here’s, here’s the [00:23:00] data from our honeypots. Well, , what the attackers are doing and what they’re trying to explain that the commands that they run, the scripts that they use. So we provided all that information to MITRE along other organizations as well. And that was what created the MITRE attack for containers.

So we’ve been working closely with MITRE since December last year to release that for the community. And so we came up with seven seven techniques from, from all those tactics, techniques and procedures. We came up with seven techniques five of them, they already existed on the miter enterprise framework, but they didn’t have the focus on containers itself.

So they added that focus. And there are other two other techniques that were brand new. They didn’t exist on the MITRE framework yet. So, yeah, the MITRE framework, , we worked closely with them and it was a, a great work great efforts since last year. And yeah, it was one of the projects that I was [00:24:00] really involved with with MITRE and I helped them , to publish that.

Ashish Rajan: So that’s pretty

awesome. And I think what’s valid calling are, why do you feel? Cause a lot of people may look at it and go, Whoa, I don’t do pen testing. I don’t do bug bounty. I, why do I even care about the ATTACK MITRE or whatever it comes out of it? What do you say to those people?

Magno Logan: I think the MITRE attack framework is a great framework.

That’s widely adopted among the security community right now. And it can be used for both teams, the red teams. For pen testing for explaining vulnerabilities, but also for the blue teams. For understanding that the attacks, the common attacks on their environments, what threat actors are using, which techniques are they using to, to attack either the container environment, the cloud environment.

. And he can do also by industry sector. . Okay. What are the most common threat actors that attack financial sector, the government and all that stuff. So I think it’s a very complete and complex matrix, but it can be used both ways. You can [00:25:00] understand the techniques that they use and like just replicate them to see if they work on your fantastic engagement.

Or change them a little bit as well. But you can also use that. Okay. So. Why did the MITRE, what’s very center from MITRE is that they only care about real world scenarios right there. They don’t care about theory, theoretical things or things that never happened. It never, nobody exploited in the wild, at least in a honeypot, for example.

So that’s what I think what gives value to, to the matrix is okay, it’s, you know, what the attackers are using, so you can look for the most, use it techniques and focus on protecting your environment against those as well.

Ashish Rajan: I love it. And I think to your point, I probably had another reason if you don’t mind.

I think what I was going to say was as a security architect, When I was a security officer at ages ago, whenever that was every time I would try and tell people, Hey, we should put, say, I’m gonna [00:26:00] use the Kubernetes example. Cause one of the projects that I was doing with people who were like, what’s the problem with having a API of Kubernetes on the internet.

This is I’m talking like three, four years ago. And I remember this very quick conversation. And the whole point was because every micro-services says people want to make it on, on the internet. Or do you find with that? I understand obviously you have the right controls behind it, but. They kind of use the same analogy as a microservice onto our capabilities, API that, Hey Kubernetes API should be on the internet.

Like, ah, I’m not really sure if it should be on the internet. But one thing that helped me at that point was knowing the Tesla thing that happened where , there was crypto mining done on go Tesla Kubernetes API or whatever the explosion on the internet. Using that example, I was able to convince people like we should, it is, it does sound like a bad idea to have the API.

So I feel like. If a person from blue team is listening to this and they sometimes struggle with convincing other people for, cause the question that you get from the [00:27:00] other side is like, why does it matter what are people are going to do we want to be on the internet, but more to your point either a recon method or like a tactic that people that adversary can use.

It goes a long way in convincing that is if that’s what you’re trying to do, but we focus our attacks for about 30 minutes now. And I want to switch to defense as well. You mentioned CIS benchmark. What thoughts on the CIS benchmark and isn’t it, is that an effective tool as a starting point?

Magno Logan: Yeah this, the CIS benchmarks, there are many other benchmarks against platforms and applications and solutions out there, and so they created one for Kubernetes the CIS Kubernetes benchmark. And if I, if I’m not mistaken, I think the two main people involved in the creation of the CIS benchmark were Liz Rice and Rory McCoon, , which are great professionals.

And they’re very well known around the cloud native and Kubernetes security ecosystem. [00:28:00] So it’s a big document, it’s, it’s a large PDF. And I went through that last year and it’s very complex, but it’s because. , it touched on every kind of every single point from your cluster. I want a best practice.

It tells you like, okay, how it should be. Okay. It should be this, this set of permissions, this set of user it shouldn’t be exposed to the internet. And it tells you also how to check if that’s the case on your cluster, it tells you the exact commands that you need to run. And it also tells you, okay.

If it’s not that case on your cluster, here’s how to fix it. So there’s for each item on the CIS benchmark. They have. Okay. They explain why it should be like that. They explain for you how to check it and then they explain for you how to fix it. , so that’s why it’s, it’s a long document and it has a lot of information there, but yeah, I don’t think anybody would go through manually through the benchmark and [00:29:00] checking the cost there one by one.

So I think that the two that I mentioned earlier, the cube bank two, does that for you in an automated way. So you run kube-bench against your cluster, and it’s going to tell you for each item from the CIS benchmarks, which ones are you compliant , which ones you’re not compliant. So, so I think that’s, that’s really interesting to do.

Ashish Rajan: Oh, interesting. So to your point, it sounds like it requires a certain skill set as well. But so if I’ve got it taking on the opportunity, I want to go CIS, CIS, the only thing that people can use to our kube-bench, you mentioned earlier as well. So apart from CIS and kube-bench, are there obvious patterns when people are trying to implement Kubernetes, that they can be looking out for?

Like, I think you’ve already mentioned, make sure at least try and not have Kubernetes API on the internet because it’s more susceptible because a lot of the blue teams are tasked with, Hey, this is my Kubernetes. built him a security architect or [00:30:00] Ms. Security architect. Tell me what I need to do over here. So what are some of the obvious, low hanging fruits in that scenario?

Magno Logan: So the CIS benchmarks would be, I think the, the most common one, and I think it’s widely adopted already and the good thing about the CIS benchmark. is that, that it has a benchmark for, for like a standard Kubernetes, if you’re doing unmanaged cluster or on premises, but it also has a benchmark , for EKS on AWS.

It has a separate benchmark for AKS on Microsoft or GKE, I think, but it should have at least two are free cloud providers there that, that they have that their managed services for Kubernetes. And you have a specific benchmark for that. All so that’s the main thing. Besides that I know that , the Kubernetes SIG security team, that’s a group inside Kubernetes that discusses has weekly meetings around the Kubernetes security.

And [00:31:00] I’m one of members of that group. They’re working on a hardening. Of Kubernetes document. So that’s, I think not publicly available, it’s still on a draft mode, but they should be releasing a Kubernetes hardening guide very soon. So it’s created by the people that work on Kubernetes.

I mean, Kubernetes security every day. I, so I think that’s really interesting. For other compliances, I would say go to your cloud provider, so if you’re using a managed services, I know that AWS, Google, and Azure, they, they probably have a best practice document for using those services.

So, so make sure that you take a look at that because they are a little bit different than, than your your regular Kubernetes deployment. So you need to understand those, those differences as well.

Awesome. And I think Tulio has a question which is kind of linked to similar as well. I think I’ve kind of answered it, but did you want to add anything to this question about what’s the framework compliance cover. Kubernetes [00:32:00] security?

How I don’t think there’s any, any compliance framework. Like if you’re, if you’re talking about PCI or SOCs or HIIPA I don’t think any, any of those compliance documents already cover Kubernetes environment, it might cover container environments. But I’m not sure.

It’s been a while since I dealt with, with those, those compliance documents. Luckily so, so yeah. I would say start with the CIS benchmarks and if using a managed service check their, their best practice, I think that’s the best recommendation. Who, who else besides their own cloud provider to give best practices and recommendations on their own service.

Ashish Rajan: Yeah. And

I think maybe another way to ask this question is have you seen implementations done for say. Every organization would have some form of compliance requirement usually. So do you have one PCI that they’ve done a PCR deployment for application? [00:33:00] They heard you and I talk about Cuban and he’s like, Oh my God, it’s the best thing ever.

And they go and deploy the whole, the same thing in Kubernetes. Is it possible to apply those compliance frameworks on like a Kubernetes cluster? Have you seen that done, like apply like a PCI or make something PCI certified if it’s on Kubernetes cluster and I know I’m putting a, an odd ball at you, but Trulia sky’s question kind of.

Struck that, like, I wonder if people are running compliance, driven environments on quote unquote compliance, driven environments on capabilities or incubatees.

Magno Logan: So yeah, , that’s a great question. I think that using Kubernetes, it might abstract you, some of the requirements that are, are in your compliance document.

So you don’t need to implement those. But then you’re also, you’re also going to rely on your, your, your auditor or, or the person that’s checking for your compliance if they understand Kubernetes as well. Because they’re going to, they’re going to go [00:34:00] line by line on that requirement and you’ve got to check, Oh, where’s the IDS, where’s the firewall, where’s the IDS.

And you don’t have that. Exactly. , so. It it’s, it’s, it’s challenging. I don’t think anyone has at least publicly done that, but, but yeah, I would be interested to know more about

Ashish Rajan: it. Yeah. Oh yeah. Well, I’ll let you I hope I’m pronouncing his name his or her name. And I hope if you have an example you can share with us, but I will be exceedingly considering an example where someone has built something which requires compliance standards to be filled out in X humidity’s cluster.

I would be out because that itself has come convincing someone about Kuberentes, even though it’s being used in 5g and 6g, that to me would be really interesting at that point. So I think we spoke about the some of the frameworks around it. And we’ve, haven’t gone into the supply chain aspect of it as well yet, .

Coming back to the question that I was asking about where it’s like multiple clusters [00:35:00] and organization. Think about it like a, I dunno, a really large organization. They won’t just have one humidity’s custody. They’ll have multiple Kubernetes cluster floating around everywhere. Usually going through our DevOps pipeline or a CI/CD pipeline.

Like, what are your thoughts on, I guess scalability perspective definitely makes sense. But from an attacker perspective, how do you look at like a CI/CD pipeline in this form? What are you looking for in there?

Magno Logan: Okay. So yeah, supply chain is a big topic right now. It’s a hot topic in the community because of the SolarWinds attack, and vulnerability, there’s also a kind of a supply chain if I could say that on the Linux kernel, where they submit malicious commits to the kernel as well and lately with the big news that related to restart of the Colonial pipeline in the US.

So there’s also all kinds of things that we can talk about it here. But I think the main point was , when Docker hub was released, [00:36:00] People were using that as their main source of container images and container registry. They weren’t doing it like, okay. A private registry or anything.

They were relying on Docker hub to check for security, the same way that you’re relying on your app store or Google play , on the apps that you download on your phone. So you rely on them. So the same thing that happened in the beginning, attackers were we’re deploying. Docker containers to the Docker hub and downloading that and making like similar names to famous images.

So you would download that and that has had a vulnerability. So , has almost, I think almost it’s almost gone now, because you have, you have to as container image scanning, and you also have your privacy registers that you should be using for you to avoid that, those kinds of situations.

So right now attackers are using regular images , from Docker hub. , they download a Alpine image, Ubuntu image, and then they install their [00:37:00] malicious tools after the, they are inside your, your, your container environment, your, your, your cluster, so now it comes to the point of.

Okay. What are you allowing inside your cluster? So now you have the container image, scanning tool for protecting against images, getting to your cluster. But now if it’s a good image and like, it has some, some malicious to that that’s installed there. How do you check for that?

So now you have on Kubernetes, something called admission controllers, and as the name says, they control the admission of containers on your cluster. They check the check. If your container was, is counted properly, if it doesn’t have any vulnerabilities. So you define that policy.

And, and one, one famous admission controller here. That’s being deprecated run right now, but that’s a good scheme. It’s the pod security policy, it’s a bunch of, of, of security settings that you can tell your cluster to only allow a pod to be deployed on your cluster if they [00:38:00] apply, if they follow those policies

in those rules. And, and now pod security policies is being deprecated and they’re working on a new version of a pod security policy on there. I don’t think they have a proper name yet. So that’s being done also by the Kubernetes SIG security team. But there are other options out there.

There are some tools like OPA, which is a open policy agent that has a specific language policy language. And they call policy as code by using that Rego language that you can define some policies , to allow or block anything on your cluster as well. And there’s another tool called Kyverno , which means it’s also Greek set, same as Kubernetes.

And it’s like a governance of your Kubernetes cluster itself. And it doesn’t use a specific language. It just uses the ammo files like, like any Kubernetes cluster. So those are the tools that I think you should check out if you want to protect what goes inside your cluster.

[00:39:00] And later, After you deploy your pods, you need to care about the runtime, you need to care about , if your cluster is compromised and something was modified later, right after your pod is deployed, that the admission controller one protects you against that it’s already deployed.

It is already writing, so now you need to care about the runtime and anything that’s running on your pods, usually you should have only one process run your in your container. Sometimes that’s not the case. , so you need to have a solution that that does kind of monitors your, your, your pods, and also protects against any changes.

And somebody’s installing some packages installing SSH or something like that. So I would say that Falco, which is also a cloud native tool and open source too, would be the solution to protect against runtime issues on your cluster.

Ashish Rajan: I love the answer. And I’m going to ask you to scale it because great answer for it.

One cluster, but as [00:40:00] a security person in a large company, I’m dealing with like multiple clusters man. And like, so I can not have need to have multiple versions of Falco or like, how would you look at scaling that security thing?

Magno Logan: Sure, . So I think for scaling, you either need to have multiple versions or you can try checking out the, manage a solution. So you have OPA, but OPA is supported by this company called Stier. And they have they have their paid services, which is kind of a manage OPA where you can have the cloud OPA where you have everything on their console, and you can scale that in different clusters and, and monitor that as well and apply the policies to different things.

You can do that yourself, but it’s going to require more work, because it’s open source, if you want to do it for free, then, then you’re going to have to automate that yourself. I think the same thing applies to Falco. I think that that the company behind Falco that donated Falco to the [00:41:00] CNCF is called Sysdig.

And they also have their managed services and solutions to, to scale that out. So it depends on your choice if you have the, the right people and resources to do that yourself. Go for it. If you just want to pay someone to do that for you. And yeah. It’s like, okay, it’s going to cost you.

So, so that’s your choice.

Ashish Rajan: So bring this a full circle, if you were given the opportunity to deploy our Kubernetes cluster with the right security in it, how would you do it? And I guess, like, what would you consider as a mature solution for a Kubernetes deployment?

Magno Logan: The first question that I would ask you, or the person asking me to deploy the cluster is what’s going to run inside your cluster,

is it’s just a website, is it the sensitive information? Is it financial information? Like what’s inside there, what’s the value of that information to you? Because that’s going to determine that’s the first thing that’s going to determine. How much are you [00:42:00] wandering fast on it, right on the security of it.

Besides that, I think a few considerations , are in order here. So let’s say either using manage our own managed, that’s the second choice that you’re going to have to make. If you don’t, we’ll have a lot of resources and you don’t have a lot of time, you, need to, to deploy that fast,

I would say the managed solutions is, is better. And, and why is it better? Not just from a speed perspective, but also from a security perspective is the, because you don’t need to worry about the control plane, the, the control plane, you don’t have access to it. It has a security default, and it has security protection.

That’s handled by your cloud provider. You, of course, you want to make sure that your cloud provider is doing the right choices. But you don’t need to manage that. You don’t need to worry about it. And they even have a cluster of control plane. So you’re not relying on one, a VM or one machine to run your control plane.

And if that goes down, everything stops working. [00:43:00] So that’s , the, the second choice, I think, unmanaged and managed as well. And yeah, I, I think one other thing that people tend to forget a lot and, and that’s very crucial inside the Kubernetes cluster is the what we call RBAC or role-based access control.

It’s kind of hard to understand in the beginning, but if you, if you take a look at it RBAC is the main access control used by many different web applications, so you have, you have , the regular user role, you have the manager role, you have the admin role,

it it’s basically that it’s just that Kubernetes doesn’t have the users, it has the rows and the row bindings that you attached to the users that you want to access your cluster. So getting RBAC, I think it’s very important to reduce the blast radius and, and kind of reduce the attack surface of , any compromise on your cluster as well.

Ashish Rajan: It’s pretty awesome. And I love the detail that you’re going into. So I hope someone takes [00:44:00] away the important pieces that you have kind of dropped in at some little gems everywhere. So. It’s just all well and good, but I may have people who have probably, this is the first time they listened to Kubernetes and he’s like, Oh my God, this is a technology that is going to be used for 5g and 6g.

I need to learn about this. What do you recommend for people to start with learning?

Magno Logan: That’s why I created this github list that Kubernetes also Kubernetes security list. It’s basically a GitHub list with a bunch of links and resources and books and talks on how first, how to start on Kubernetes and the Kubernetes basics.

But then we go deep into Kubernetes security. And so I show presentations from previous conferences. I have listed some books that you can download for free. You just need to provide like an email or something. , and some trainings as well. Like you have the Kube-goat from, from Madhu.

That was here a few sessions ago. So you can play [00:45:00] around on your own and deploy a cluster even, or you’re on your own machine and play around with that. so I think that’s I would say I’m a bit biased, but, that’s the main resource and, and I try to keep that as up-to-date as possible.

And there’s a lot of, of contributors already. I have over as over 10 people that submitted new links and, and contributed to this list. So I think it’s pretty good right now.

Ashish Rajan: Awesome. And I’ll definitely make sure I’ll have that link on the show notes as well for people too, but usually it is easiest face someone’s impatient.

They can find Magna Logan on GitHub and the following. One of his popular repositories in there as well. It’s pretty awesome. This was , pretty awesome, man. I think and we definitely had one person who’s shouted out Amod, and , he definitely was. So he’s definitely grateful that we really share that link.

So hopefully that helps you, man. So last section, and this is our fun section. I know we’ve talked about attack defense. We spoke about what a good maturity looks like, but I do want to [00:46:00] know and give an opportunity for people to know you as a person as well. And a fun section is just three questions.

And I know you’ve kind of heard this before as you’re like of our show as well. So first one, where do you spend most time on when you’re not working on Kubernetes technology and all of this?

Magno Logan: Okay. Yeah. I actually didn’t talk about the responses for me, so yeah, that’s, that’s, that’s a section that I, that I look forward the molas and that sometimes I’m very curious to know more about the, the guests and the hosts there, but yeah.

Let me see maybe

drinking a Sasha say

yeah, no, no, I’m not a heavy drinker. I drink like. Once or twice a week depends on the size of the cup. So that’s why

Ashish Rajan: fair enough. You’ve done. Well, man, you’ve done well. I’m still going through this.

Magno Logan: Yeah. Yeah. So, so here in Canada now that’s summertime almost [00:47:00] summertime pretty much.

So I like to go outdoors. I like to walk my dog to the park and I like to, I love doing some Brazilian barbecue as well, so yeah,

Ashish Rajan: yeah, yeah. Oh actually, you know what? This is a good question. What do you feel is the difference between a regular barbecue and a Brazilian barbecue? Like Y I, I just cannot explain it to my wife or anyone, but I’m hoping you guys,

Magno Logan: I think the first thing that’s different is the way that they cut the meat.

So, so it’s, it’s a bit different, so even if you cut the. The meat it’s different. It’s even hard for us to find the proper cut meat cuts here in Canada. It’s there’s some names and some difference there. Another thing is usually with, with the Brazilian barbecue, it’s usually like, okay, it’s a real barbecue,

it’s real meat. It’s not burgers it, not sausages or hot dogs. The barbecue is it’s usually chicken or beef or pork. And it’s [00:48:00] like the seasoning of the meat as well. Like sometimes we, we red meat, we usually just put some, some salt on it and that’s it. We don’t, we don’t add any, any other seasoning or like barbecue sauce, anything like that.

So that’s pretty standard and yeah, I think, I think in the South of Brazil, that’s, that’s like what they do every, every weekend. So, so I think that’s, that’s very famous there

Ashish Rajan: as well. What’s a favorite cut for barbecue.

Magno Logan: The meat cuts for barbecue, I think would be the, in Portuguese we say right, but in English, it’s the, I forgot the name.

It’s top yeah, I forgot the name. So there’s that? That’s all

Ashish Rajan: I’m sure if you bring in Google ,

Magno Logan: some steak cap or something like that. I forgot the name. When I see it on the, on the, on the supermarket, then I get it. But I forgot the name.

Ashish Rajan: Well, by the way, Gerald loves your groth, figures.

And how many is this question? Like? This [00:49:00] is how many, yeah,

Magno Logan: that’s a good question. I think I have two big ones. There’s four smaller ones, actually six smaller ones and a medium one there. And I also have a Lego one that’s out there.

Ashish Rajan: Wow. There you go. You have your fun, another comment, just personal with them.

If I’m

Magno Logan: a big fan of the Mandalorian.

Ashish Rajan: Yeah. There you go. Next question. What is something that you’re proud of, but is not on your social media?

Magno Logan: Okay. Sure, sure. I think something that I worked on for over 15 years, I think more than security was, was martial arts. I did martial arts since 2003 and two and two, actually 2019, 2018.

And I’m a fifth degree black belt in a martial art called ninjutsu or it’s a Japanese martial art based on what the samurais and the Ninja drains [00:50:00] before. So I think that’s really awesome.

Ashish Rajan: Wow. So wait, do you do whole sort fighting and everything? Yeah. Wow. So wait, it’s, what’s sward the main weapon for summarize, or was there something else that was well in there for

Magno Logan: summarize?

Yeah, they, they usually had two Swartz, like a

dollar one. But for, for new jobs, they, they, they use any, any kind of weapon, that you just didn’t care about. The weapon itself, they, they only care about doing the job completed their mission. So they could use either a sword or, or like a steak or, or the shooting cans.

That the Ninja stars and, and all that stuff. So it’s, it’s fun to play around with those. I haven’t been doing lately because since I moved here to this city that I’m on in Canada, they don’t have a gym and right after I moved to the pandemic started. So I’m between there. But uh, yeah, I’ve done.

I’ve practiced in Brazil. I’ve practiced in Argentina. I went to Japan to get [00:51:00] my fifth degree black belt as well in 2014. And I I’ve also practicing the U S and Europe

Ashish Rajan: wait. So you’re actually saying that has been as kids. I used to watch all the Ninja among us and anime. And like, there is no way someone teaches this in real life.

So they do teach in real life.

Magno Logan: Exactly. Do you know, I don’t know if you know their, the series a Ninja Uriah. I haven’t heard of it. Yeah, actually Riah. So the, the father of Gerais in that series, which is a famous Japanese series that I used to watch when I was a kid that the father or the matron of the Ninja dryer is actually the master of new shoots.

So he was, he was kind of interpreting himself in that role there. So that’s really nice. I can send you the link. They have the videos on YouTube. I can send you.

Ashish Rajan: I would definitely look at it. Look forward to that, man. All and last question, what is your favorite cuisine or restaurant that you can share?

Magno Logan: Okay. So yeah, besides the Brazilian [00:52:00] barbecue I really liked burritos. So, so yeah, exactly. So there are some, some good Mexican places around here and it’s, it’s similar to, to the Brazilian food as well. So you have rice, you have beans, you have salad, then you have meat. So it’s, everything’s combined together in one thing.

So for me, it’s very practical and it’s very tasty. So I like that.

Ashish Rajan: Oh my God. That definitely would make me hungry as well. And especially in Brazilian barbecue, the more you mentioned it and someone actually has the. Oh, they have it every day. Oh my God. Julio, like having it every day. Oh my God.

Yeah. They bought Magno and Matt would Magno and I are jealous of you, man. Like definitely made us envious of what you have. Yeah, exactly. That’s the, for real, that is totally true. That is pretty awesome. But sweet. So thanks so much for all this all the time as well. And it seems that you [00:53:00] definitely have a crowd favorite as well here.

Where can people reach you and feel free to say it in Portuguese as well? For the, for the Portuguese audience as tuned

Magno Logan: in. Sure. Sure. Yeah. Yeah. I’m pretty much on, on most social networks as Magna Logan altogether. So he did get TBE, LinkedIn Twitter. It saw Magna Logan. So you can find me there yeah, and feel free to, to reach me.

And if you have any questions, anything around application security, container security, or call security, I’ll be happy to talk about. Yeah.

Ashish Rajan: Awesome. All now, thanks so much for coming on, man. I think I learned a lot and show the audience landlord as well. And I can’t wait to have you back on again for anyone else who’s listening in.

And they, you, if you enjoyed the conversation, we have conversations like this or cloud security every week and every weekend we go live over here. So if you’re someone who finds this valuable information, you subscribe and you would probably see someone like Magna, come on again, and maybe some other [00:54:00] people as well, who can come and share and shed some light on what.

Cloud security could be, but thanks so much for everyone joining and we’ll see you next weekend. Thanks Magno. I’ll see you soon.

Magno Logan: Thanks for having me. Bye.