How is eBPF impacting Kubernetes Network Security? In this episode, recorded LIVE at Kubecon EU Paris 2024, Liz took us through the technical nuances of eBPF and its role in enabling dynamic, efficient network policies that go beyond traditional security measures. She also discusses Tetragon, the new subproject under Cilium, designed to enhance runtime security with deeper forensic capabilities. A great conversation for anyone involved in Kubernetes workload management, offering a peek into the future of cloud-native technologies and the evolving landscape of network security.
Questions asked:
00:00 Introduction
01:46 A bit about Liz Rice
02:11 What is eBPF and Cilium?
03:24 SE Linux vs eBPF
04:11 Business use case for Cilium
06:37 Cilium vs Cloud Managed Services
08:51 Why was there a need for Tetragon?
11:20 Business use case for Tetragon
11:32 Projects related to Multi-Cluster Deployment
12:45 Where can you learn more about eBPF and Tetragon
13:50 Hot Topics from Kubecon EU 2024
15:07 The Fun Section
15:35 How has Kubecon changed over the years?
Liz Rice: [00:00:00] If you're the application developer, you shouldn't have to care, is it running on a VM or Kubernetes? It shouldn't really be your concern because you're writing code that, does business logic. That's the kind of goal of Cillium is to get all of that networking super efficiently with, network policies across wherever you're running your workloads.
I'm told this is the biggest Kubecon ever.
Ashish Rajan: KubeCon E2024.
Liz Rice: Yeah.
Ashish Rajan: Cilium, the open source project, got graduated last year. This year, they came out with a sub project for Cilium called Tetragon. Now, if you haven't been hearing about this, we are at KubeCon Paris, and we were talking to Liz Rice from Isovalent.
She's a Cilium contributor, she's been involved with CNCF for a long time. And we were talking about the business use cases for where people are using Cillium that graduated last year and why people have been loving it so much. Network security on Kubernetes and especially even if you use Azure, AWS or GCP, they're all using Cillium in some way or form for network security.
So if you are working in the Kubernetes workload space, you probably know about Cillium, but you may or may not [00:01:00] have heard of Tetragon, which is a sub project for runtime security. In this conversation with Liz, we spoke about how the community has evolved in terms of the eBPF community, the Cillium community, where you can start learning about. eBPF and Cilium as well. If you're starting today, we also spoke about Tetragon, the sub project for Cilium, which has GA and you can use a runtime. what are some of use cases for that as well? And what's in the future in store for Cilium projects as well. All that and a lot more in this episode with Liz Rice from Isovalent.
And if you're here for the second or third time listening or watching this on YouTube, LinkedIn, definitely give us a like and subscribe. If you're on Apple, iTunes or Spotify, give us a review rating. It's always appreciated. Takes only a few seconds and if you find valuable, we'd This episode, please do share it with someone who's working on the eBPF space or Cilium space.
And I hope you enjoyed this episode. I will see you next one. Welcome to Cloud Security Podcast. We're having a conversation with Liz Rice. Maybe to start off with, if you can get a bit of introduction about yourself and where you are and what you're up to.
Liz Rice: Yeah, my name is Liz Rice. I am Chief Open Source Officer at Isovalent, which is the company that originally created the Cilium project that a lot of people know for like cloud [00:02:00] native networking and security.
And yeah, we're here at KubeCon, I'm pretty involved with the CNCF as well and with OpenUK yeah.
Ashish Rajan: Something else you're quite involved with is eBPF as well.
Liz Rice: Yeah, definitely. And
Ashish Rajan: Maybe, Cillium, the project got graduated last year. And maybe it would be good to give people a primer on what is eBPF, what is Cillium?
Liz Rice: Cilium is built on top of this technology called eBPF. And eBPF allows us to run custom programs inside the kernel. And the kernel, just to make sure everybody's on the same page the kernel is the bit of the operating system that interfaces with hardware. So if you're writing application code, you don't really think about it, but when you write a line that says, read something out of a file or send a network message, your application code actually has to ask the kernel to do that on your behalf.
So whenever you're doing anything, like accessing a file or sending a network message or allocating memory or also things like process coordination because the kernels also coordinated different processes all of those things [00:03:00] the kernels involved in and then with eBPF we can insert custom programs that either observe or even change the way the kernels behaving So we can see all these different events We can manipulate things in the networking stack so we can be really efficient about networking, and we can do interesting things regarding security, since this is the, Cloud Native Security Podcast,
Ashish Rajan: we should
Liz Rice: probably concentrate on that one.
Ashish Rajan: Is it like SE Linux by any chance?
Liz Rice: Yeah, so SE Linux has a very kind of hard coded interface in the kernel, and there are only so many things you can do. with that Linux. It actually uses an API within the kernel called the Linux Security Module Interface. And you can hook eBPF programs to that API.
And then you can have much more, custom, bespoke policies. So rather than, always alerting on or preventing a particular, type of file access, you can make contextual decisions. You can write your own code, or you can use, It's a tool that somebody has written [00:04:00] that uses sort of dynamic policy.
So we can be much more creative. We can be much more dynamic if we're using eBPF rather than just having a sort of hard coded response to a particular policy.
Ashish Rajan: It's an interesting element because Cilium being an open source project that graduated last year and eBPF being there for people to use and it's part of the CNCF community as well.
Do you find that between graduating as a project in CNCF, Cilium, to today, do you find that network security conversation has evolved a bit more, or are you still finding that, the reason I asked for the primer as well is because as much as we'd all like to assume that eBPF is like a very popular thing, it is popular in a big community, but are you finding there's a lot more people involved in it now, and is maybe if you can share a business use case for people at the aha moment for, Oh, this is where it makes sense.
So what's that business use case for Cilium that kind of made it shine and become like the graduated project that it is today with a lot of customers?
Liz Rice: Cillium, because it's using eBPF, it's an incredibly efficient networking [00:05:00] plugin for Kubernetes. And it provides better performance.
There's performance graphs that you can look at. They'll show eBPF. Making your container networking more efficient, but we can also use it for network policies and again being very efficient about these granular decisions you might want to make about which workloads are allowed to talk to each other.
We can use Cilium to connect not just workloads within a cluster but also multi cluster multi cloud. I did a talk yesterday actually about how we use this cluster mesh feature in Cilium to very seamlessly connect your workloads. So you can have Let's say a front end service in one cluster that can talk to a back end service that might have pods in the same cluster, but there might also be pods in another cluster, which you might want to use perhaps for like high availability reasons.
You could use it as part of a graceful handover situation. There's all sorts of reasons why you might want to. Or another [00:06:00] really good use case that we see for cluster mesh is having your Stateful services in one cluster that you use for those stateful things, because that requires quite a lot more setup and maintenance, let's say, and your stateless services can be in whatever cluster they want, but when they need to talk to those stateful services, databases and what have you, they can do that. reach it from wherever. That would be an example of the kind of things that people are using. There's tons of different ways that you can use Cilium to connect your workloads. And our vision really here is that you shouldn't have to care where your workload is running, Cilium will take care of the connectivity for you.
Ashish Rajan: Maybe coming through the bias of the whole cloud security landscape where Amazon or Google or Azure has a lot of that for you already. Has there been like a Kubernetes service before that could have been used for it? Why was there a need for Cilium? Because where I'm going with this is that there's obviously gaps that are, this is more good for what can be done right now for between clusters.
I want to get to the runtime part as well. So that's where I'm [00:07:00] like going from the fact that Amazon Azure, Google Cloud whatever cloud that you put in there, is it different in those managements compared or Cilium the same doesn't matter?
Liz Rice: You can run in all of those clouds, and in fact, all the major providers are using Cilium in some shape or form in there.
Yeah. networking solutions. So for example, Google's data plane V2 is based on Cilium. Microsoft have, I might get the terminology not quite right, but it's Azure CNI powered by Cilium. I apologize to everybody in Microsoft if I've mangled the name for that. But yeah, it's a Kubernetes CNI powered by Cilium.
And AWS picked Cilium as the default CNI plugin for their EKS Anywhere solution and you can also substitute so for example in AWS the defaults, if you're running in their EKS in the cloud their default networking solution is called VPC. Yeah, but you can use that in chaining mode with Cilium.
So that Cilium [00:08:00] is still handling the kind of network connection, it can do the network policy enforcement, but you're still using AWS's CNI for allocating the IP addresses and it's hooked into the AWS VPC, but Cilium is also there doing things like network policy.
Ashish Rajan: So do you see examples of customers of the open source project Cilium use both the cloud available feature tagging that along with Cilium and that becomes that network security layer for the Kubernetes deployments they have.
Liz Rice: Absolutely, because you can do more policies. You can do things like not just layer three and layer four, but also layer seven network policies with Cilium.
You can do things like FQDN policies. So you can say my workload is allowed to speak to, example. com, but it's not allowed to speak to cryptocurrency miner. com. So
Ashish Rajan: that's a good way to ignore , yeah fair. I want to talk about Tetragon as well, which is the next project that is come out of the space as well.
Why was there a need for [00:09:00] Tetragon?
Liz Rice: So I think that the backstory of this is really Isovalent and some of our enterprise customers telling us that this network policy is great. We can see these and prevent these connections to cryptocurrencyminer. com. But wouldn't it be cool if we could get more forensic information about why that connection, how that came about in the first place?
How did we get the compromise that, you Put this executable that's decided to open this dodgy network connection because with eBPF we can hook into all manner of different places in the kernel Yeah this became the kind of seed of the project that is now Cilium tetragons as a sub project within the Cilium family, but you can use it independently which is why you know, we have it as a sub project and you can use it to or you can have policies to look at things like file access.
What kind of sensitive files do you want to know about if they're observed and potentially block what kind of network [00:10:00] connectivity, but not just, like I say, you're not just getting the information about the network connection, but also the process the node, the Kubernetes pod, the executable name, the user ID, all the kind of contextual information that you want as a.
If you're trying to do forensic on a security incident, you've got all this data about all the things that led up to that sensitive file access, or that privilege escalation, or that dodgy network connection. Yeah, it's super powerful. And we can do enforcement as well.
Ashish Rajan: Oh, wow. Would you compare it to having Threat detection capability and maybe I don't know, IDS IPS is the right word, because a lot of that kind of goes into that territory of knowing the origins of what went wrong and what triggered in the first place.
So it's like that the space kind of it is in.
Liz Rice: Yeah, exactly. But with all this kind of contextual information that makes it easier to use, you don't have to use it in Kubernetes. You can use it in all manner of environments. But particularly in a cloud native environment, you've got this kind of dynamically changing [00:11:00] world of pods and nodes and services and having that information. It's like a next generation IDS.
Ashish Rajan: It is GA now, right? People can use it. Yeah,
Liz Rice: we went 1. 0 with Tetragon I think that was just about around the time of the Chicago graduation period,
Ashish Rajan: so last year then. Yeah. Okay, and have you come across some business use cases already from customers?
Liz Rice: Oh, absolutely. Like I say, the idea from it was, from conversations with some of our enterprise customers. So yeah, I would say, sometimes you talk to someone and their eyes light up when they understand the possibilities of what you're doing. What do you do with this?
Ashish Rajan: I think the last time we were talking about multi cluster deployments and other challenges with it as well, in terms of having Cilium, Tetragon are you finding that the challenges in the multi cluster space has, obviously there's a lot broader challenges as well. I will probably put that caveat on top in terms of the challenges that are currently being resolved.
I feel there seems to be some positive signs, thanks to Cilium and Tetragon. The business use cases that you mentioned are pretty clear as well. In terms of just the multi cluster environment in general as well, are there [00:12:00] other areas you guys are looking at as well, which are possible next sub project as well in terms of?
Liz Rice: I guess the natural extension of thinking about multi cluster is also being able to connect to non Kubernetes workloads, so legacy workloads. And that, again, is something that we are tackling in Cilium. Your ability to seamlessly connect to those legacy workloads without having to worry about.
Whether it's in kubernetes, if you're the application developer, you shouldn't have to care where you know, is it running on a VM or kubernetes? It shouldn't really be your concern because you're writing code that you know does business logic Yeah, that's the kind of goal of Cilium is to get all of that networking super efficiently with network policies across wherever you're running your workloads.
Ashish Rajan: And where can people learn about eBPF and Tetragon? Is there like public sites for it as well? Because what's a good starting point? Because I don't even know where do you direct people and what's a good project to start learning it off with?
Liz Rice: I guess the first thing I'll say is that eBPF is essentially a kernel technology.
So [00:13:00] you can go, I mean I've written a book and the, done a few things online about Getting started with eBPF, at some point it does become kernel programming. For the folks for whom that is their passion, it's fascinating. But for the majority of people, they probably won't actually write eBPF code themselves, and they will use it through tools like Cilium and like Tetragon and like the other, there's several other CNCF projects that are using eBPF as a platform.
I would definitely say Cilium. io is a great place to start learning about Cilium. Also, if you want to get hands on, Isovalent has a number of different labs that you can learn about Cilium and different features within Cilium. There's also a getting started with eBPF lab there as well.
So isovalent. com slash labs.
Ashish Rajan: Yeah, perfect. And what are some of the hot topics that I am walking away with? We're on day two of KubeCon now. Has there been any topics that have stood out for you in terms of that's interesting? I think, or maybe how to change from when you came to KubeCon thinking this is what most [00:14:00] topics would be around and two days in do you find they have changed?
Liz Rice: The first thing I'd say is not really a topic, but just that the buzz is, it seems like the old days, this is the largest KubeCon ever, and it really feels like we've got back to the energy of the kind of pre pandemic days, which is fabulous. And then topic wise, you can't avoid AI as a topic here.
Yes, I think I was concerned, to be honest, that we would You know, we're just talking too much about AI and that perhaps this is, it's pretty much in a hype cycle right now. In reality, I think there's some sensible things being talked about around things like how does Kubernetes, scale and work with things like, the GPUs that you need and the different kind of, workloads that people need to do AI.
Yeah, so those are genuinely relevant conversations that people are having here which I think is good. It's not necessarily my kind of expertise, but it's It's, good to keep an eye on how these conversations are progressing.
Ashish Rajan: There might be Cilium [00:15:00] there somewhere.
Liz Rice: Yeah exactly.
And if you're going to do a scalable network layer underpinning your AI training, it probably is going to be based on Cilium.
Ashish Rajan: No, that's just good topics to talk about it. But I guess those are some of the tech questions I had, I have three fun questions as well for you. Okay. So the first question is, what do you spend most time on Cilium challenges?
Liz Rice: You mean work wise or not work wise?
Ashish Rajan: You can pick either.
Liz Rice: At the moment, I feel like I want to just be sleeping.
Ashish Rajan: Yeah. It's a one hour jet lag. Come on, Liz. I know. It's just been, it's
Liz Rice: been a long week.
Ashish Rajan: Yeah.
Liz Rice: Yeah, no. Normally I would say I spend quite a lot of time cycling and just generally trying to stay fit and music as well.
Ashish Rajan: What's the number of KubeCon that you've attended so far?
Liz Rice: It's quite a lot. I think the first one I went to was probably 2016 or maybe 2017. So yeah, it's certainly been a large number of years.
Ashish Rajan: And you were attending both North America and Europe?
Liz Rice: Yeah, and I've also been to one China.
Ashish Rajan: Oh wow.
Liz Rice: It's however many that is, how many years that is, it's been quite a few.
Ashish Rajan: Times two because you have North America and Europe [00:16:00] in there as well. And how has it changed between the years, and feel free to pick a geography as well, but how has it changed the community, but how are you seeing a change between North America, Europe, because you get to see both perspectives as well.
Liz Rice: Yeah, in terms of the kind of community that's here, one change that's an inevitable consequence of the way Cloud Native is being adopted and is becoming more mature, is that in the early days it was predominantly maintainers, people who were building the software we're increasingly seeing end users which in many ways is absolutely fabulous because it means the things that we're building are now, they're being used They are genuinely You know useful things.
Yeah, and I think if you are a maker of things, it's great to see your thing being used
Ashish Rajan: Yeah,
Liz Rice: I was also talking to somebody earlier today. We were talking about the new end user Technology advisory board TAB. Anyway, now we have so many end users who have experience, running [00:17:00] cloud native in production at scale, doing, proper work with it.
And they've become the experts. So the developers and the maintainers of the projects are, deep experts in a particular niche. And now we have the end users who have the expertise in how do you operate this. Yeah. These systems. Yeah. And they're sharing knowledge amongst each other around, what are their best practices and what are their learnings.
We've seen, for years we've had end users coming and doing talks about their experiences, but I think it's increasingly, they have the broadest expertise.
Ashish Rajan: Yeah. Yeah. Wow. And in terms of just the KubeCon itself as a, between North America and the community, I guess you find that there's a difference between how North America evolved versus how Europe has evolved.
Liz Rice: I'm told this is the Biggest KubeCon ever.
Ashish Rajan: KubeCon EU 2024.
Liz Rice: Yeah. Bigger than any North American one. Which is a surprise, because pre pandemic, the U. S. ones were always bigger than, or the North American ones were always bigger than [00:18:00] the European ones. So I think that says something about how the European community has grown and evolved and really you know, taken Cloud Native to heart.
It's amazing. I think there's also something about the economic cycle where in Europe you have things like labor protection laws that mean people are a little bit, there's less radical hiring and firing than you see in the US things are just kept a little bit more on an even keel here, and I think that maybe when, In North America, you see a sense of optimism.
Sometimes it's crazy optimism. And then, when things aren't going so well, it's, doom and gloom. And here we tend to stay on a bit more of an even keel, I think. Yeah. So you can see that in the atmosphere. There's so many people out there that are Show floor is really buzzing.
Yeah, I think. Yeah, of
Ashish Rajan: course.
Liz Rice: But it's not like crazy hype and nobody's going, nobody is getting too carried away. Yeah.
Ashish Rajan: Yeah. Fair enough. They understand what they're using it for. They understand the business use cases as well, but it's not being like no one beating their chest over it, yeah. Yeah. Awesome. And to my final fun question for you, [00:19:00] what's your favorite restaurant or cuisine you can share with us?
Liz Rice: Since we are in Paris and since I have just had one, I am going to say A macaron is a very good thing.
Ashish Rajan: Oh, that's a rub on Shilpi as well for what do I get as guest for on the podcast?
But thank you for sharing all that. And I appreciate your Parisian input as well. I'm going to try some macarons after this, but really appreciate, thank you for coming on the show.
And I'll put the show notes. of your social information as well as people can reach out to you as well.
Thank you for listening or watching this episode of Cloud Security Podcast. We have been running for the past five years, so I'm sure we haven't covered everything cloud security yet. And if there's a particular cloud security topic that we can cover for you in an interview format on Cloud Security Podcast, or make a training video on tutorials on Cloud Security Bootcamp, definitely reach out to us on info at cloudsecuritypodcast. tv. By the way, if you're interested in AI and cybersecurity, as many cybersecurity leaders are, you might be interested in our sister AI Cybersecurity Podcast, which I run with former CSO of Robinhood, Caleb Sima, where we talk about everything AI and [00:20:00] cybersecurity. How can organizations deal with cybersecurity on AI systems, AI platforms, whatever AI has to bring next as an evolution of ChatGPT, and everything else continues.
If you have any other suggestions, definitely drop them on info at CloudSecurityPodcast. tv. I'll drop that in the description and the show notes as well so you can reach out to us easily. Otherwise, I will see you in the next episode. Peace.