Zvonko Kaiser: [00:00:00] If I have confidential data, let's say x ray images of my patients and I want to do inferencing on that. But I want to have this data unencrypted on a CSP running if I'm bursting out because my on prem resources exhausted. When you have confidential data, you want to protect it. People are used to run random bash scripts or models from the internet, right?

Confidential compute will not save you from everything, but it can help you on specific occasions.

Ashish Rajan: Confidential computing, confidential containers is something that you probably will start hearing more often now that CNCF, NVIDIA, and a lot of other people have started talking about it as well.

Fortunately, we had Zvonko from NVIDIA who spoke about confidential Containers, Confidential Computing, what are the threat models for it, why do people care, which industry specifically can benefit from having these, and also we spoke about lift and shift that's happening in the AI LLM space, and where can you use Confidential Compute, Confidential Containers to have at least a level of trust in what you're putting into your service, whether it's inside your CSP or whether it's basically [00:01:00] in a bare metal container or Kubernetes.

All that a lot more in this conversation and we got into a bit more weed than we normally do. So I hope you enjoy this conversation. If you, especially if you're looking at confidential containers, that is something which is very important for people to probably deal with financial data, health data, and a lot more things that you may consider sensitive data.

So just to get that level of trust for within attestation. That's where confidential computing can definitely help I can keep going on. This was a really interesting, exciting episode. If you are here for second, third time, I would really appreciate if you drop us a review rating on iTunes, Spotify.

If that's where you're listening to us. If you're watching this, definitely subscribe to the YouTube and support us. Because liking these videos and liking these episodes are one of the reasons why we know that if you enjoy content like this, you should create more of this and I'll let you enjoy this episode.

As always, I will see you in the next one.

We interrupt this episode for a message from this episode's sponsor, SentinelOne. As cybersecurity professionals, we constantly seek ways to enhance our threat detection and response capabilities. SentinelOne's Purple AI uses advanced AI and natural language processing to [00:02:00] streamline threat investigations and provide actionable insights.

It's designed to help your security operations team work smarter and faster. If you are interested in leveraging AI to boost your sec ops, Purple AI is worth exploring. Check out more details at SentinelOne. com slash purple. Now back to the episode.

Ashish Rajan: Welcome to the show. And welcome to another episode of Cloud Security Podcast. Zvonko, if you could share a bit about yourself, where you are at the moment, how'd you got to that role as well, that'd be great to set some context for the scene.

Zvonko Kaiser: Currently I'm working at NVIDIA. I'm leading the Kata and confidential containers effort at NVIDIA. Before NVIDIA, I was at Red Hat and we were building all the nice stuff about GPUs to support in OpenShift to make the GPUs a first class citizen in Kubernetes Oh, wow, so we've done that right GPUs are everywhere And now we are concentrating on confidential containers and confidential computing and we want to make the GPU now again a first class citizen in the confidential computing space for any people that can easily use GPUs in confidential computing or in confidential containers

Ashish Rajan: when you say confidential computing The first thing comes to mind [00:03:00] is that encryption at rest?

Encryption transit, isn't that

Zvonko Kaiser: all that is to encryption? No, there's more right? So we solve data at rest Yes, we have encrypted database, data in transit, TLS and all the other encryption standards but what we currently do not have is how do we protect data in use if you are accessing database entries?

You're decrypting the information and running it on a server, but the server is completely open, right? No description any tenant on the server can look up in the memory what you're doing, right? So the idea of confidential computing is to protect data in use meaning the memory is encrypted your register state is scrubbed so that the hypervisor cannot deduce what you're doing, like you have a stack pointer, you have frame pointer, stuff like that, so that the hypervisor has no access to the VM inside of the secure enclave where you're running it, because it's completely encrypted, and there are all the other things that are concerned about confidential computing, for example, how you route interrupts, how you interact with the VM, right?

It's all protected. Even on one system you could have multiple [00:04:00] VMs from multiple tenants running on the same system without them knowing what the other one is doing. Because for each VM, the CPU will assign a specific encryption key. Yeah. So each VM has their own encryption key. And if one breaks out, he will be on the host.

He can, yeah do stuff to the other VMs, but he cannot access any confidential data running in the VM.

Ashish Rajan: You mentioned confidential containers as well. Yeah. So what's the difference between confidential compute, confidential container, and now that we understand there's data and encryption of data in use as well.

Yeah. So what are the use cases for each one of these for compute as well as container in confidential computing, confidential container?

Zvonko Kaiser: There are people who are still running like legacy VM applications. They do not need containers. They don't need kubernetes because it's a static set. Maybe just one database and one application or whatever else.

So it's a very static environment. But for people who need kubernetes support, who want to run containers, right? This is where confidential containers comes into place. And essentially how we're [00:05:00] running confidential containers. is with a project that's called Kata, which is Kata containers. And maybe you heard about Qbert, which is a VM running in a pod.

With Kata, it's the complete opposite. It's a pod running in a VM. Oh, okay. Yeah. And it's completely transparent to the kubelet. Meaning your kube, oh, I'm going to start a flamewar, right? So kubectl or kubectl, however you name it, right? You can interact with the pod completely transparently.

You don't even know. That's your pod is running in a VM. So Kata is CNI, , CRI compatible. So all the storage networking interfaces or runtime interfaces are implemented in Kata, which means all the things that you can configure in Kubernetes will apply to the Kata VM and the Kata container as well.

Ashish Rajan: Obviously this is NVIDIA, but most people these days are using managed communities, containers. So this is available for them to use at the moment? Yes, it is. They can use containers as well? Yeah, they can. Are there specific industries that normally focus on this? Because [00:06:00] I almost feel to what you called out, I'm sure it's a threat model for most people out there that, hey, what is the level of trust that I can put on my CSP or whatever?

Yeah. So I would imagine most people, or at least in my mind, I feel there's a lot of industry that would be interested in doing confidential computing. Do you find there's a certain sector of that focuses more on this versus the other?

Zvonko Kaiser: I wouldn't say a specific sector. I would generalize it to if you have confidential data.

Yeah. If you have research data that you don't want to share. If you have multi tenancy you want to share more of your data. hardware that you have. The minute you have confidential data and want to protect it, this is where confidential compute comes into play. Before it was like the threat model for Kata or for VMs was like, we do not trust the user who is coming in on our infrastructure.

We don't trust the workload, the code. So we want to protect us from those workloads. But for confidential computing, it's the complete opposite, right? We, as a workload owner, We do not trust the infrastructure. If I have confidential data, let's say, x ray images of my patients, and I want to do inferencing on that.

[00:07:00] Would I want to have this data publicly , unencrypted on a CSP running if I'm bursting out because my on prem resource is exhausted? When you have confidential data, you want to protect it.

Ashish Rajan: Would you say then, in the case of threat modeling this, from a confidential computing, confidential container perspective.

What are some of the common threats that you see and people should consider? Because I imagine like a lot of audience that we have maybe some of them are hearing for the first time this, what is Confidential Computing, Confidential Containers. A lot of people normally approach protecting applications or compute with the Threat Model.

I'm curious in your perspective, what do you see as common Threat Models or Threat Actors or Threat Paths people should consider when they're looking at the whole Confidential Computing, Confidential Container space?

Zvonko Kaiser: It depends really on what you are trying to protect you against, right? So if we are taking traditional containers, it's a modern way of packaging stuff.

It's just a process with some namespaces around it, right? So if you're worried about container escapes, which can take over the whole node, and in Kubernetes, you could even relabel yourself as a [00:08:00] control plane and take over the whole cluster, right? So if you're worried about container breakouts, then you should think about isolating it with a Kata container right? So if you're worried about a untrusted code that could even escape your VM, you need to look how to protect the other VMs that are running on the infrastructure. That's where confidential compute comes into play. And many people are not afraid, but are worried that the infrastructure or the host extract data or the confidential data from their workload. And everybody has a watch, you have fitness data, you have health data, you have, this is all assets that you can sell to parties that can leverage it for financial things, or it could be used for any other malicious stuff, right? To get more information about you to, I don't know, do some sim swapping on you and get your bitcoins or stuff like that, right? So any information that they can get from you needs to be protected.

Ashish Rajan: Yeah, use case like that makes sense from a threat modelling perspective, but we spoke about secure [00:09:00] enclave as well.

What is Secure Enclave for people like me who don't know about it?

Zvonko Kaiser: So it's an isolated environment where you're protected through software from other parts of the system. Yeah. With Secure Enclave, you have hardware support of that. Meaning the CPU, and there's a secure processor on most of the CPUs, that is taking care of isolating resources of your node into a region where no one has access to it. Not even other VMs and other devices on the system. Let's say a network card or a GPU or stuff like that. So you need to do special things to allow, for example, the GPU to access data inside of the secure enclave. And one, one specific topic about confidential computing is attestation.

Meaning you want to prove the workload owner that your system, your hardware and software are in a trustworthy state. Yeah. With specific settings that you. What do you expect to be and [00:10:00] specific versions that you are expecting that the firmware of the CPU may have or the software that you have

Ashish Rajan: so we spoke about attestation.

Yeah. What is the value of attestation for a business considering attestation for confidential computing? What's the value?

Zvonko Kaiser: Secure Boot, right? What does Secure Boot do is to prove to you that you're running the kernel the firmware and let's say the unit RD that you're expecting it should run, right?

Yeah, it's hashed. It's measured Yep And then in the TPM you're extending the PCR values and then at the end you get one hash value that you can compare to expected value, right? You're running the right kernel So yeah attacker could exchange your firmware could exchange your kernel Your complete host would be running with a malicious kernel without you knowing So you need to prove to the workload owner or to you to the owner that you're running the versions that you're expecting.

So you're starting at the host, right? Secure boot, measured boot. Then you have the VM. You want to know that in the VM, you're running the right kernel, the right firmware, the right guest image. And if you have devices like GPUs, you want to [00:11:00] also make sure that. The GPU is measured that you have the right firmware version, the right driver version.

And then on top of it, you want to check if those things are revoked or not. The certificates that are signing those stuff, are they revoked or not? Because there could be a CVE, there could be some bug. The process of attestation is giving you the security, the trustworthiness of your software and hardware that you're running the right versions.

And the right pieces inside of on the host and of course inside of your secure enclave

Ashish Rajan: I'm thinking of this more from a mindset that if I have secure enclave if I have confidential computing The trust level is definitely a lot more higher for the workload that i'm running.

Zvonko Kaiser: Yes

Ashish Rajan: And I can be a lot more confident that I am using the right hashes So like right container images and everything.

Is there a because your talk was more around lift and shift strategy for AI as well. What's the lift and shift strategy for AI? Because I think now we set the context for, okay, we have spoken about secure enclave, we've spoken about confidential compute, confidential containers. So with all that context, I think, what is the whole, [00:12:00] Like using confidential computing for lift and shift strategy for AI, which is also your talk is about here at KubeCon Paris.

What are you seeing in the industry as a lift and shift thing that you're referring to there?

Zvonko Kaiser: So if we are looking at former trust execution environments, there was the need to recompile workloads. There was the need to link to specific libraries. There was some need to do code modifications to your workload.

And NVIDIA's premise for enabling confidential compute, be it VM based or container based, was no code modifications needed to any AI CUDA workload that you're running, meaning you can take your CUDA workload today and run it on a confidential environment without any code modification. And the same is true for confidential containers.

We built the architecture inside of Kata in the same way that we build it on bare metal, meaning no code changes to the containers needed. You can run your CUDA containers one-to-one on a confidential environment, be it on bare metal, be it with Kata or with conventional containers. It does not [00:13:00] matter.

It's just a matter of changing the runtime, how you are going, wanna run the, your CUDA container air or AI ML pipeline, whatever else.

Ashish Rajan: Because I was gonna say that probably is the biggest thing we most people talk about 'cause you just to change the way and like it's the whole X 36 64 bit that people should talk about.

Is that kind of like the used to be challenge where. If you go for the 32 bit versus 64 bit, is that the same thing with containers as well, where people would be afraid of doing a lot more confidential computing because they're like, oh, it's not just the hardware changing, now I've changed the entire way, the program, the code, so that's not needed anymore, is that what you mean?

Zvonko Kaiser: No, that's not needed anymore. If you're running Kubernetes, the only thing that you need to do is change the runtime class in your PodYAML, and that's it. That's it? That's it. Wow. Okay. And it's the beauty of it, right? Yeah. Wow. Just change the runtime class and you're running on a traditional container runtime on Kata runtime or a confidential containers runtime.

So just to give an away, so confidential containers is just an umbrella term for kata containers plus some components. That are needed for attestation and a secret key release. Okay,

Ashish Rajan: You [00:14:00] mentioned GPU earlier as well. Like earlier, you wanted GPU to be first class citizen. You want GPU to be first class citizen for the confidential container and confidential computing space as well.

What's the role of GPU in all that? I know NVIDIA, GPU, I get that. What advantage does GPU give to adding confidential computing?

Zvonko Kaiser: Yeah, so the obvious reason is computing power, right? If you're looking at OpenAI, how big the farm they have, how big Azure is right now with their offerings, right? So you need vast amount of compute power to train your models, right?

This is the obvious reason. And even inferencing, right? You need machines that can handle all the inferencing requests that are coming in into your cluster, right? I don't know how many people are using today ChatGPT, but I would say it wouldn't be possible with just CPUs, right?

Ashish Rajan: Absolutely, but if it was a non LLM scenario, where you're not building an LLM model, you can still use confidential computing, confidential containers without having to use GPUs.

Zvonko Kaiser: Yes. Yes. Okay. Yeah. This is just in Plug A play. So again, it's just again, a runtime class. So [00:15:00] you can say, okay, I wanna run this confidential containers without CPU. You're using the Kata CPU Runtime class. Yeah. And if you want a GPU, you should just say, okay, I want use the Kata GPU Runtime class.

And this will pull in all the needed artifacts for the VM and set up the system and also set up your PCX Trust apology inside of the vm. Yeah. Depending on the use case. So really the end goal of what we want to do with Kata is support sophisticated use cases like GPU direct, RDMA, and GDS, if people want to do accelerated IO to some storage.

But again, it's just a change of the runtime class, if you don't run a CPU or the GPU. Wow. Essentially, do not care.

Ashish Rajan: That's I'm sure that's music to a lot of people listening to this as well. It also, I do want to set the context for it. What happens with shared responsibility when CSPs come out with shared responsibility?

People just assume that, hey, everything is taken care of. I don't worry about this. I sense a similar direction for the world in a way when they hear confidential computing, confidential containers. Is that still, is, would that be true? Like we [00:16:00] don't need to do any more security?

Zvonko Kaiser: No, that's not true at all.

People are used to run random best scripts or models from the internet, right? You shouldn't do that, right? You should check what you're running and confidential compute will not save your confidential data from running a random script from the Internet. If it's a remote shell, talking to the malicious guy and just copying data over the Internet.

So you need to protect your enclave, right? Be it through APIs. With validation, sanitization, and stuff like that. You heard about prompt injection, right? Where people are doing crazy stuff to expose data, confidential data from the model. There are some ways on, to sanitize, validate, stuff like that.

Because, if you have a specialized use case, You know how the requests are going to look like and what you expect that comes out, right? Yeah. Yeah. Yeah, so confidential compute will not save you from everything, but it can help you on specific occasions, like limit resources of a web interface where you're talking to.

If someone can break out about the request, all of the web interface, [00:17:00] he still has no access to the model that's running your vector database that's running, right? So those assets are still protected. The attacker can shut down the VM, but still you're not having access to any confidential data.

Ashish Rajan: Is there a project that people can get involved?

I found it an interesting project to be involved in as well. Is there any project that is either part of the CNCF or not part of the CNCF that you're working on where people who are watching and listening to this can get involved with? And maybe even help make this better or bring their perspective to it.

Is there a project that's running for this as well?

Zvonko Kaiser: Yeah. So we have the Kata project, which is an open infra project, and we have confidential containers, which is the CNCF project, both on GitHub. Yeah. We're welcoming all new contributors to work on everybody who is interested in bring your use case.

We are happy to add features and implement new features that are needed for any use case that we have. We are broadly coming from different companies. We have Intel, we have AMD, we have ARM, we have Alibaba, we have NVIDIA, we have IBM. All major architectures that supporting secure enclaves, we are supporting it.

We are [00:18:00] working with closely with all CSPs to enable confidential containers. So just for interest Azure has a confidential containers offering. Based on Kata with all the things that you're doing. Everybody's welcome if you love upstream work working with different people with GPUs, you're welcome to join and work with us.

Ashish Rajan: I've got three questions for you, so non technical, so it should be just to get to know you a bit more. So first one being, what do you spend most of your time on when you're not doing work on Kata containers and Confidential computing and Confidential containers?

Zvonko Kaiser: Oh my let me think. I have a son who is very demanding.

He's learning currently basketball. We are doing archery together. Oh wow. Yeah. Soccer, all the sports. He likes to draw, right? So we are drawing a lot of together. He is also a builder. So I have him, I don't know how this is called, but every month you get like engineering kits Okay, and you can build stuff like oh wow.

Yeah, and then he's all on it and he's also a big lego enthusiast So i'm giving him my old hardware like jetson devices and stuff like that and he is working with them So [00:19:00] recently he made a robot who can play super mario on his switch Oh, so yeah rudimentary, but still it works without an AI just if then else, you know the basic stuff Yeah, but it's really nice.

It's lego techniques, right? You know spending a lot of time with him. I have a dog going for a walk bicycling jogging running Yeah

Ashish Rajan: Just outdoorsy. Yep. Awesome. And what is something that you're proud of that is not on your social media?

Zvonko Kaiser: Let me put it this way. Most of the people like. They are born in a village, stay in a village, and die in a village, right?

They haven't seen anything. And I have lived in most of the states in Germany, in the beautiful states. I was born in Stuttgart, right? So where Mercedes comes from. I lived near Munich. Now I'm living north of Berlin. I lived already two years in Canada. Wow. All right. So he has seen a lot and I think we are proud of it, that he's open to different cultures to different people, to different nationalities without any prejudices, right?

Yeah. So on a lot. And he also gets into [00:20:00] contact for me into the tech space. For our generation, it's already overwhelming. I don't know how it's going to be for the next generation with all the stuff coming up, especially if you watch the GDC note, which Jensen showcased robotics with AI and stuff like that.

Yeah, it's going to be crazy. Yeah, next generation.

Ashish Rajan: We still find a lot of times we still keep trying to keep on. You almost have to be always switched on to keep up with what's going on as well. So I can't even imagine for next generation. So final question. What is your favorite cuisine or restaurant that you can share with us?

Zvonko Kaiser: I would say my favorite cuisine is Italian. I like mostly the red wines from Italy, but I'm open also to other red wines, but Mostly Italian cuisine, red wines, yeah.

Ashish Rajan: Oh, nice, okay, awesome. Where can people find out more about you or connect with you to know more about the space and the work you're doing?

Zvonko Kaiser: One way is LinkedIn. Find me on LinkedIn for those people who are not developers and don't know what Slack is. For [00:21:00] developers who know what Slack is, you can find me on the Kubernetes. On the kubernetes 6 slack space on the kata slack space on the confidentiality container slack space. One of those you can find me with my handle and yeah, this would be the best way.

Ashish Rajan: Awesome. I'll put those in. Thank you so much for coming on the show. I really appreciate this. Thank you very much. Thank you so much. Thank you for listening or watching this episode of cloud security podcast. We have been running for the past five years. So I'm sure we haven't covered everything cloud security yet.

And if there's a particular cloud security topic that we can cover for you in an interview format on cloud security podcast, or make a training video on tutorials on cloud security bootcamp, definitely reach out to us on info at cloudsecuritypodcast. tv. By the way, if you're interested in AI and cybersecurity, as many cybersecurity leaders are, you might be interested in our sister AI Cybersecurity Podcast, which I run with former CSO of Robinhood, Caleb Sima, where we talk about everything AI and cybersecurity.

How can organizations deal with cybersecurity on AI systems, AI platforms, whatever AI has to bring next as an evolution of ChatGPT and everything [00:22:00] else continues. If you have any other suggestions, definitely drop them on info at cloud security podcast. tv. I'll drop them in the description and the show notes as well.

So you can reach out to us easily. Otherwise, I will see you in the next episode. Peace.

‍