Bar-el Tayouri: [00:00:00] With traditional application security, you have a SCA, you have SAST, you should put it in the pipeline. You know how to scan it. And I guess you, most of the companies already have their workflows on which type of alerts are more important to which are less important. Here, what you can do, you can find all the components, you can detect all the risk of the components.

Then you can run red teaming on the model and on the application. And then somehow you take all the findings and decide to create policies.

Ashish Rajan: If you're securing AI applications in the cloud, you're most likely using AWS services like Bedrock or Azure services for OpenAI and other services that potentially are third party.

But what is the security of an AI application? In this episode, I have Bar-el from Mend.io and we spoke about what is technically supposed to be security for AI? Is it the security of the cloud? What are the different layers of applications that AI applications, as people call it, what is it? And what are some of the components that you need to think about?

For example, [00:01:00] is it simply a DevSecOps pipeline? Pretty much what we've been doing all this while. All that and a lot more in this conversation about how do you secure AI based applications in the cloud. And if you are a leader who's thinking of building a security strategy for AI applications while you look after cloud security as well, this is the episode for you.

We spoke about all of this and a lot more. Also, if you know someone who is trying to work towards building a security strategy for AI in the cloud environments, definitely share this episode with them because they'll definitely find it valuable just to understand the different layers, no pun intended, are there in an AI application and what kind of security should you be expecting, whether it's shadow AI and a lot more.

So we go through all the buzzwords and explain what they are. So you know and understand where the security gaps can be. As always, if you have been listening or watching a Cloud Security Podcast episode for some time and you've been finding them valuable, I would really appreciate if you could give us a subscribe or a follow on the audio or video platform that you prefer to listen or watch this on.

If you're watching this on YouTube or LinkedIn, definitely give us a subscribe, follow there. Or in case you're [00:02:00] listening on Apple or Spotify, definitely give a subscribe or follow there as well. It really means a lot, all the support that you've shown us. So thank you so much for doing that. And thank you for taking that one second to subscribe to the podcast on your favorite audio video platform.

Now enjoy this episode and I'll talk to you soon. Peace. Hello, welcome to another episode of Cloud Security Podcast. Today I have Bar-el. How are you, man? Thank you for coming in.

Bar-el Tayouri: My pleasure. It's so exciting to be here.

Ashish Rajan: Maybe to kick it off, could you share a bit about yourself so folks have a better idea about yourself and what you've done in the past professionally?

Bar-el Tayouri: Sure. It's pretty short past. I started my journey as a kid hacking into transportation cards. Then I do then I went into the army for five years to doing security. So I have a lot of experience with security research and networking, cryptography, medical stuff.

Ashish Rajan: Oh, nice.

Bar-el Tayouri: And then I joined as a first engineer into augmented reality startup to see how it's like it in the other side. I have also a, like lead [00:03:00] data science. I have a playlist of music, of generated music before it was cool to do GenAI. Also and then I co-founded a Atom Security, which is a prioritization company for cloud native alerts which acquired by Mend, which is the company I'm working for ,now as a head of Mend AI, which is our new AI security product.

Ashish Rajan: Interesting. And considering you worked on AI before AI was cool and you were talking about all of this. I think something else I found in some of the conversations you've written is around AIBOM now, I think people know what SBOM is. What is an AIBOM in the context of applications being developed with AI models?

Bar-el Tayouri: So the thing is, why SBOM is so interesting? And it's because except from the regulation maintaining some inventory of what you have, it's give you so many insights that you weren't aware about. And the same concept of SBOM which is basically a list of packages their dependencies and their [00:04:00] dependencies in your software.

Same concept apply to AI. I would think about it as AI components. And where you have a lot of AI components in your organization, in your software and then you want to list them to maintain some inventory of them. And so AIBOM, it's basically a list of AI components.

Ashish Rajan: Oh, okay.

Cause you know how and because you focus on that appsec space quite a bit, could you give me some examples of what kind of components can I, because I guess there is this whole world of unknowns for when a lot of people talk about AI models, application using AI models being built by AI models.

There's so many variations to it. So when you say AIBOMs and the components being used within an AI enabled application, what would be some example components? So people have some idea?

Bar-el Tayouri: Yes. And I agree. Super confusing, because especially the term AI security, unlike cloud security or SaaS security, when you're securing SaaS.

With AI security, it's sometimes like it [00:05:00] can be securing AI or using AI for security.

So I'm speaking about securing AI systems. Which means when you develop your AI system, like AI driven software, it means in the heart of the system, there is some some model or many models and many layers that use these models like in some way, so I'll give you an example using a open source model, It's an AI component and using a remote model through inference provider is also in a component.

So inference provider is a component, even though it's remote and using a data set in order to train to train some model is a component. And, but now that's the complicated part because we still, unlike networking or operating system, when it's very easy to see some diagram of layers. For example, [00:06:00] in operating system, like everyone many people familiar with kernel and user mode, right?

So you have some sense of layers hardware, user mode.

Ashish Rajan: Yeah.

Bar-el Tayouri: And networking, you have this OSI model of all the TCP IP stack.

Ashish Rajan: Yep.

Bar-el Tayouri: Of layers.

Ashish Rajan: Yep.

Bar-el Tayouri: So with AI, we still don't have it. But it's pretty clear that in the AI components, we can think about it as layers because we have the model.

And then we can, that's the base model. We can fine tune it.

Ashish Rajan: Yep.

Bar-el Tayouri: And we can and then we can use a RAG above it. So we're taking data and we use some vector embedding over it. So we using the model with it. So that's a layer above it. And and then we have agent. That is like usually using some data source like RAG, but it's can run code, read more data sources, send emails maybe.

So that's a layer above it. And each layer have many layers inside [00:07:00] it.

Ashish Rajan: Interesting.

Bar-el Tayouri: So it's many components.

Ashish Rajan: I guess you add a few more too. I love the example you used for the, the seven layer OSI model that a lot of people know in networking that, presentation layer all the way down to whatever the bottom layer used to be.

I can't remember, but in the case of an AI context what we're talking about when we talk about AIBOM or the bill of material for what is in an AI enabled application it's the application itself, to what you're referring to, then the, all the components that we have loved and known about applications for a while, where it's your, for the libraries that are being used. It's also your, the safety of the code, if it's being hosted on a server, the safety of the server, infrastructure behind it. And then there's also the the AI model it's talking to, which you're fine tuned. You have a RAG layer on top, and on top of that, you have basically your application.

I think in my mind, if I were to picture this as a diagram, so it's people, easy for people who are probably listening and not watching the YouTube video, there's a component, which is the application [00:08:00] itself. Then there is the AI component itself, which is the layers that you're referring to, where there's a model, fine tuned RAG, and then you keep building on top of it with the agents and stuff.

And then there is the output, and whether the output comes back to the application or goes somewhere else. Would that be a, a simplified version of the components that you're referring to?

Bar-el Tayouri: It's an interesting way to think about it. I would, I always thought about it as like all these layers are inside the application.

Ashish Rajan: Ah, okay.

Bar-el Tayouri: Because you have application, so you have like in traditional application, you have database.

Ashish Rajan: Yep.

Bar-el Tayouri: And of course, in with Cloud Native, you have infrastructure, you have many containers.

Ashish Rajan: Yep, yep.

Bar-el Tayouri: And then inside it. You have a new type of a lot of new type of layers of AI.

Ashish Rajan: Interesting. And is that in the server or in the application as a microservice?

Bar-el Tayouri: Each microservice can have a lot of layers. . But there should be some shared resources because if you're [00:09:00] using. It's possible. And so it's maybe both.

Ashish Rajan: Yep. Yep.

And I guess I think that kind of definitely helps me understand. And thank you for sharing that as well.

Because I understand, because this is a confusion that a lot of people in the organization I guess not an organization, but generally speaking have, where the way I describe it is how the general public thinks it works out. And maybe some for some applications where you can't put it in, it's that's the case.

But I love how you said it's integrated inside your application. And that's to your point about the bill of material now is that what my AI model is what my agents running are, and maybe to add another layer to it. Do people just go straight to this or are there like stages that organization go through in adopting AI?

Bar-el Tayouri: So it's interesting for my experience like I speak with our customers on a, on a weekly basis and I see some, one of the strong patterns I usually see is that everyone, virtually everyone now building [00:10:00] AI systems and it's never started. It's not that you decided to build AI system and then it's in production.

Usually you have this stage of experimentation where you see developers just playing with the technologies and usually with a kind of a simple use case. Let's say only using LLM, maybe even one open source LLM, and the complicated ones will do some RAG. But then after a few iterations on the experimentation, quick it's getting into production.

And then in the last stage, you'll see kind of optimization. Although we are engineers and usually as engineers, we have this tendency to optimize things that not exist, but it's still so many times people start with the optimizations. But it seems like this is the main stages like experimentation, production, optimization.

Ashish Rajan: Oh, interesting. [00:11:00] And I think is this where the cost also becomes like, how people talk about, oh I don't know. Open AI is 200 or a month or whatever. Like the cost obviously for an organization, optimization could mean different things in a context of cloud. Is there a stage where they use your cloud based application?

The Bedrocks of the world and OpenAI of the world where. To your point in the adoption, when people start using AI for the first time, are they straight away going for one of these popular models? And that's where the experimentation starts.

Bar-el Tayouri: Exactly. It always starts with the simplest use case. So it will usually be OpenAI or Bedrock. And then we have this iteration, experimentation, production. But then we'll start another, and then in the optimization that's the stage where I see usually usage of open source of Hugging Face, maybe embedding [00:12:00] models of Hugging Face.

If you're building RAG, then maybe usually I see that people will keep the OpenAI for generations, but the embedding may be like it may be more easier. It's maybe easier just to switch on the embedding models from remote inference providers to open source.

Ashish Rajan: Oh, and so when you say embedding, what do you mean?

What's an example of that? Is it just going, integrating into everything else, or?

Bar-el Tayouri: So you mean example of how it, like, how people do this migration, let's say?

Ashish Rajan: Yeah.

Bar-el Tayouri: Ah, yes today, if you're using OpenAI, you'll see the list of models. You'll see generation models and embedding models.

Ashish Rajan: Yep.

Bar-el Tayouri: And it's basically in HuggingFace, you have many very good embedding models from HuggingFace and so you can just, download this model from HuggingFace and start building a new vector database based on this model.

Ashish Rajan: Oh, so [00:13:00] actually, maybe we should probably explain this to people as well, because I don't know how many people know what embedding models are as well.

And what's the difference between an embedding model and a regular OpenAI model because there's so many models. So what's the difference between an embedding model and a regular OpenAI model that most people know about or Anthropic model people know about?

Bar-el Tayouri: Yeah, it's a great question.

And actually it's a good point.

So everyone speaks about RAG, right? It's became the strongest buzzword.

I'm not sure if more than AI agents or maybe less but RAG is a kind of a type of agent. Yes, and for a good reason. I see it also with our customers. After the first iteration, let's call it a simple use case, people will start building RAG, AI agents will go on the most more advanced use cases in experimentation than in production.

So what's RAG? Let's attack it from this place. RAG is the ability to add more context to a model without fine tuning it without doing a [00:14:00] pre training, which is great, but how it works. So you have basically two models unless it's a generative RAG or like more advanced use cases, but you have two models. The first model is generation.

And the second model is embedding and generation is model is text to text. You get text and you, it generate more text and the embedding is taking text and create vector out of it. A bunch of numbers.

Ashish Rajan: Yep.

Bar-el Tayouri: And why it's interesting because if you will use two similar text the embedding the, it's the embedding model will create two embeddings, two points in the space that's going to be close to each other. And it sounds like not interesting, but actually when you're using a vector database, which means taking, let's say, a whole document, split it and put all the embeddings, all the points, all [00:15:00] the data points on the, on a database, which means that when you have a new text, let's say a query, you can find only the relevant places in the document.

That probably related to this query.

Ashish Rajan: Oh, okay. Also, it's to make it more efficient.

Bar-el Tayouri: Yes. It's a, because we can't take the whole document, it depends on the, the, how, what's the size of the database. We can't take the whole database and put it into the LLM every time, right? We'll ask a query, we'll put the whole database.

Put it with the query. So what we want is to find only the candidates that will relevant to this query, to the question.

Ashish Rajan: Yeah.

Bar-el Tayouri: And we'll then put it in the question to the LLM, to the generator model.

Ashish Rajan: Yep. Yep. Interesting. Actually, that makes so much more sense as well. And you've done a great job explaining it because I think I had some understanding of it, but this kind of clarifies it even more as to why, and this is probably the reason why now [00:16:00] we're able to do it so efficiently as well, instead of trying to load, to your point, entire database into memory for, find and this is where most of the compute was going in the beginning I imagine now to bring this back onto a security context. And now that we have laid the foundation for what is AIBOM, how people using AI in terms of starting with experimentation, going all the way to production in terms of, a lot of leaders who listen to this conversation are going to be thinking about, okay, there sounds to be a lot of components moving around here. What do you see your customers talk about in terms of security strategy? I guess what would be, maybe let's start with components. What are some of the good components to have in an AI security strategy for leaders listening in?

Bar-el Tayouri: So maybe let's take the example of RAG because we've just spoke about it. So in RAG, we have, for example, two components of the embedding model. Let's say it's open source and the generator model, let's say, let's assume it's an inference provider. So on the component level we have [00:17:00] compliance risk and security risks compliance, like sounds not interesting, but no one's want to find out in the day before production that your model has the wrong license is GPL or something.

And it's super common actually, because it's we're still yes, I heard a few stories of it. It's crazy. Everyone got this awareness of open source packages that you need to check the license, but no one's know about about models and data sets, but they also got license.

Ashish Rajan: Oh, okay. That's new. I did not know that either.

Okay. That's a good one. Okay.

Bar-el Tayouri: And now with DeepSeek, it's a bit more in the awareness because you know about DeepSeek privacy policy, right?

Ashish Rajan: Yeah.

Bar-el Tayouri: Basically can record the data and do it and store it in China.

Ashish Rajan: Yeah.

Bar-el Tayouri: Many countries also like forbids the application from the app store.

Anyway, so that's example for components that has compliance risks, right? We have model license and privacy policy for inference providers. On the safety side, on the security side the model can be malicious. For example, all the actual model can has some bias. That's a major [00:18:00] thing. Because if you're using a, if you're using the model internally, it's maybe not interesting, but if it's an assistant of insurance company and the model has a, has bias, maybe it will open a, will be exposed for a suits.

Ashish Rajan: Yeah. Legal problems could be happening.

Bar-el Tayouri: So that's on the component level and all on the application level. It's like when you contextualize all the components into one application, let's say it's a assistant based on RAG and that you can have RAG data leak. You can have for example, or poisoning, if you updating the data source based on based on queries and data user sense, someone can poison your data source.

And now the internet is a full of indirect prompt injection. People try to put in many places prompt injections. The hope that some of the LLMs will take it into the will use it.

So we see it all the time, and that's a [00:19:00] huge risk.

Ashish Rajan: Would you say that from a, maybe to put another layer to this as well for people who are and I guess to your point a lot of people from that security operations team and other stuff would love to definitely double down on the whole threat as well.

There's another component of, from cybersecurity leader perspective, who would be thinking about, hey, we spoke about infrastructure to your point, if we spoke about different layers that AI could have as well. In terms of people who are building this, they're obviously hearing these terms like LLM firewalls.

They're also thinking about now, oh, there's different layers to this application where the application itself has layers of AI. I guess people, where is the divide in obviously I want to clarify for people what components are for an LLM firewall and what components are to the application stuff that you and I are talking about where the AI may already be integrated into the application.

Could you clarify that? So at least people know oh, okay so when people talk about LLM firewall. It's that component of infrastructure making an API call or something. We'd love [00:20:00] to hear from you as well in terms of what's an easy way for leaders who are listening in to differentiate the two components for when they hear a word of LLM firewall versus application that's using LLM.

How should they approach looking at security for them? At a super high level. You don't have to go into deep examples. At a super high level.

Bar-el Tayouri: Yes, I get it. Let's say So let's make it clearer this landscape of of maybe, so inside AI security we got application security for AI driven applications and security for AI driven applications, basically.

Now, if you double click that, you'll have types. You'll have shift left and shift right, and it's a bit more complicated in AI because with AI you're using your infrastructure is cloud from the first second, especially if you're fine tuning models that are very heavy. But let's say in order to simplify it as a beginning, you have what you have on runtime.

[00:21:00] After deployment and what you have pre deployment, so after deployment you, there's a, you will probably want to put some firewall and maybe also to scan your database for prompt injections if you're using some RAG that is a continuously updated and on the pre deployment, you want to secure your all the components find if you have a malicious model before production, right?

You want to find everything before production and try to find all the vulnerabilities on the application level before production. You don't want only when the attacks is successful to catch it. So we want to do it before. And that's the same concept as traditional application security. Just with AI, it's a bit more complicated.

And because you're using the cloud since the beginning, so you have some type of runtime. And also with AI, unlike application, unlike traditional software,

Ashish Rajan: Yeah,

Bar-el Tayouri: when you can scan the code and [00:22:00] understand the code and then try to predict the vulnerabilities and we call it in the industry, SAST tools and like static analysis application testing it's not exactly the words, but you understand so with AI models, for example, you can't understand the numbers.

And then predict what's going to be the vulnerabilities.

Ashish Rajan: Actually, yeah, because it goes back to the vector database you were talking about, how the embedding works.

Bar-el Tayouri: It's the same, but you just can't understand. It's really hard to make sense out of all the data you have. And that's why you want both to scan the code to understand how it uses the models, let's call the layers of the agents or the agents layers and the RAG layers and also to scan the model.

But you'll do it usually dynamically, like you'll try to run the model and test it with a malicious conversation adversarial testing of prompts. So [00:23:00] that's the main differences. You need to use more on to rely more on dynamic testing for models and applications.

Ashish Rajan: Yeah.

Bar-el Tayouri: But you also need to do the static for code because it's related and connect all the dots.

Ashish Rajan: Okay, because to what you were saying in the beginning, because as now people are building AI based applications, going back to the layers that you spoke about, you still have the code. It's not that the code has gone away.

You still have the code that still has to go through your SCA for dependency checks, still have to do static code analysis, still probably have a pipeline as well. Actually, talking about pipelines is what would be some of the hurdles? I know people talk about DevSecOps pipeline and have that as a security integrated into a pipeline when an application is being deployed.

What does that look like for this? And what are some of the common challenges people get to see when they're trying to do this with AI based applications?

Bar-el Tayouri: So I think on one way, if you're a security conscience company, not someone who developed LLM without any attack surface.

Ashish Rajan: Yeah.

Bar-el Tayouri: It's pretty clear what you [00:24:00] should do, but it's also pretty clear there's no one answer. Because with traditional application security, you have SCA, you have SAST, you should put it in the pipeline, you know how to scan it. And I guess you, most of the companies already have their workflows on on which type of alerts are more important and which are less important. Here, what you can do, you can find all the components. You can detect all the risk of the components. Then you can run red teaming on the model and on the application. And then somehow you take all the findings and decide to create policies.

And so let's say you do it every time a developer is doing PR. What do you do with all the recommended results? How do you like decide which one are important for you and which one are less important for you? So that's hard. And that's like the main challenges, but on the other side It's better from the experimentation stage and to start getting used to work with [00:25:00] security tools in the CICD, security tools, because then it's becoming way easier and there are less questions when it's in production.

For example, once you're already using DeepSeek, it's really hard to stop it before production or after it's in production. Our job as a security industry. It's to allow innovation and not to block it.

Ashish Rajan: Yep.

Bar-el Tayouri: And so that's why it's so important to, from my experience, to put everything like to make it as automated as you can from as early as you can.

Ashish Rajan: And I guess would that be to your point? The general disconnect people have between development and security when they talk about DevSecOps. Is that still there in AI or has AI solved that problem for us?

Bar-el Tayouri: I think it's just a more challenging because you still have security. You still have application security, but now you have a new persona.

Usually you have it, which is AI security. So instead of having these continuously challenge between [00:26:00] developers and security, you have AI security, application security, developers, and of course, legal department. Because of compliance. If you're living in Italy

Ashish Rajan: Yeah.

Bar-el Tayouri: And using DeepSeek. It may be a problem.

Or Ireland or now Australia.

Ashish Rajan: Yeah.

Bar-el Tayouri: Yeah. So it's just more complicated. Yep. And but I think there is a way for everyone to work together. I assume that the right way is that the R and D and product will be really will be like a mindful on the behavior of the application.

Because if you're building an assistant, it's behavior is actually part of your product. Unlike traditional application security, where it's something like on the side, it's not affecting the behavior of the application. Here, it's really affected. If you have bias, or if every user can attack it with prompt injection.

Ashish Rajan: Yeah.

Bar-el Tayouri: So I would expect now, not only the developers, both the developers and the product managers. [00:27:00] And the product leaders to to be aware of it and run all the tests and red teaming as soon as possible. And even the data scientists when they train the model, or even when you choose the model to work with before you build the application, you can also run all these tests to make sure it's not malicious to make sure it's not bias and not prone to prompt injections.

And then you can make it more secure by design. So all the stakeholders should should care about it. And then and then you're good.

Ashish Rajan: Actually, that's an interesting point, because, to what you said because it's dynamic, does that make it challenging from a, because I guess the way I'm thinking about it as you're sharing this based on what you've heard from your customers from an AI perspective, as someone who's a security leader or someone who's trying to basically secure AI applications there's obviously a component of the different layers and understanding of what that is and having like the license one was a brilliant example because I don't know how [00:28:00] many people are thinking about license at the moment or have thought about licenses as they moved AI applications into production. What's interesting in that was, most of the times when you think of a statistical analysis tool or a SCA or whatever, there's usually like a, hey, there's a vulnerability out.

Now DeepSeek had some vulnerabilities out. But in terms of CVE or a risk framework, for lack of a better word, is there a known risk framework that people can apply to something like this? Because you almost, in this land of what you said, it's so dynamic. And as you're talking about this in a, in this interview, I'm sure there's another model being announced somewhere that's being used by another developer in production potentially right now.

Is there some examples of risk framework or maybe I have solid risk framework that you have seen that works perhaps? Because these days people think more from a, hey, I have core detection to what you said in my pipeline. Now I have detection for what's the license for my AI model?

What are some of the known vulnerabilities is automatically where my [00:29:00] head is going. But then my, oh, what risk framework am I applying to this? There's no one's giving me a CVE for a model that's not being used by many people because I want to be cool. And I had this new unique LLM model that I'm using.

Have you got some thoughts on the whole risk framework part as well? So people can plan for mitigation after.

Bar-el Tayouri: So there is no, NIST is working on something.

Ashish Rajan: Okay.

Bar-el Tayouri: We have also OWASP top 10, the version two of it, OWASP top 10 for LLM applications. That's the long version of it. Which is a great framework for high level for threats.

It's really good. It's, it tells you the threats, for example, from injection and vector embedding weaknesses, which is let's say, RAG poisoning or a RAG leak.

Ashish Rajan: Yeah.

Bar-el Tayouri: System from leakage, like very important threats. It's not very specific type of let's say like CW is. When it's a specific type of vulnerability or CVEs When it's a known vulnerability for a specific component.

Ashish Rajan: Yeah.

Bar-el Tayouri: And it's really problematic to have also CVEs because if you think [00:30:00] about it

Ashish Rajan: Yeah.

Bar-el Tayouri: And models unlike packages, open source packages,

Ashish Rajan: yeah.

Bar-el Tayouri: Not always have very good versioning. For example, let's say GPT-4.

Ashish Rajan: Yeah.

Bar-el Tayouri: Even if you take the GPT -4 preview date.

They still change it and they can change it. They're behind the scenes.

Ashish Rajan: Yep.

Bar-el Tayouri: So there's no, if there's a vulnerability, they may fix it. Or not fix it. And it's really hard to track vulnerabilities and assign CVEs to them.

Ashish Rajan: Yeah.

Bar-el Tayouri: So by design, it's different. But it doesn't mean you can't mitigate the risk.

Okay. It's just not straightforward as upgrade the package.

Ashish Rajan: And I guess to your point every organization may have to just take it on a case by case basis for based on what data is going in into that particular application, which is using AI and whether they want to maybe use that as a starting framework to put the risk on and then use the OS top 10 LLM model [00:31:00] for finding out, hey, which one of these applies to us and then use that as a next step to mitigate or not to mitigate.

Bar-el Tayouri: Exactly. You can use, for example, when you do threat modeling, you can understand that your application is prone to system prompt leakage. You don't want and currently the system prompt is going to be embarrassed if it's going to leaked. So as a prerequisite to building the application, think how you can make the right prompt system prompt.

But I think there is a kind of a way to think about mitigation.

That instead of thinking only about remediation.

Ashish Rajan: Yeah, like

Bar-el Tayouri: Aswell fixing a specific vulnerability and it's better to have this hardening mindset, especially with AI. When you look on code or a model, it's pretty clear how you can reduce the attack surface without fixing a specific vulnerability, but reducing that the attack surface from attacker perspective, it's going to be way hard, way harder to [00:32:00] exploit the vulnerability. For example, if your interface, if you take the output of the LLM and then format it into like more into an output with more constraints. It's going to make that attacker's life way harder.

And that's without fixing any vulnerability yet. So that's major. Also, if you put guardrails before and after you put many checks. That's going to be major even before you fix a specific vulnerability.

Ashish Rajan: Yeah, I think you've hit it on the nail, man, though, right there. I think maybe to zoom out, because you explain this really well in terms of hey, what's a runtime threat versus a deployment time threat a thing as well.

Have you got some advice on for people who are listening to this, probably already have AI applications in production at this point in time?

Bar-el Tayouri: For sure, and if they don't know, it's probably shadow AI, if I get, you think you don't have it just not aware of it from my experience. It's always like that.

Ashish Rajan: Yeah, so for people who probably have shadow AI and trying to figure this out, for the people who think there is [00:33:00] experimentation going on in AI and not in production, it's called shadow AI. And for people who probably have been in the journey for some time, but haven't had the chance to do a security review or audit of it.

And then third one being, they've listened to this conversation so far, they know where to start. So for the first two use cases, what do you recommend? Because I feel like to your point, I love the attack surface example. I love the guardrail example, but a lot of people like, Oh, the cat's already out of the bag.

It's already in production. I don't know what I'm doing over there. Versus the people who are. I don't even know if I have shadow AI if I were to obviously, there are very different use cases, each one of them, how would you recommend people approach aI security today, considering that it's changing ongoingly and there's new models coming out.

Is there a simple framework that comes to mind for you from you on, hey, this is what I'm seeing customers do. And I think this is something, or maybe even your personal opinion on this as well. How should these people approach doing security of AI applications?

Bar-el Tayouri: [00:34:00] Yeah, actually, no matter what first step, remove the shadow AI. In the beginning, I thought that I met like more sophisticated customers that has some awareness of applications. And we'll run the checks, run the POC. We started working with the customers. There's always shadow AI and the ratio between what you are aware and what you're not aware is a factor of 10.

That's from my experience. It's so surprising. But it's true. I know it's it's unbelievable but it's true. And that's the first step. Because if you're this sophisticated customer who's aware of AI in production and more like that you probably already done some things like you don't have DeepSeek in production for high chance.

But in the shadow AI, like where you're not aware of, you have DeepSeek in production. So the shadow AI is your weakest point. And that's the first step. Remove the shadow AI.

Ashish Rajan: Okay.

Bar-el Tayouri: No matter if you're sophisticated or not.

Ashish Rajan: Right.

Bar-el Tayouri: Second step [00:35:00] is let's say look in the component layer and find all the risks in the component layer.

And the main reason is that if you have if you have the wrong license, for example, of model or data set,

Ashish Rajan: Yeah.

Bar-el Tayouri: Major bias. Or a malicious model. You want to catch it as soon as possible and stop it. Hopefully before it's in production, but also in the development environment, it's pretty bad.

Ashish Rajan: Yeah.

Bar-el Tayouri: It's like having a trojan. And that's bad. It's happening. It's actually happening and the only way to do that is to do the shadow AI. That's why we always start from the shadow AI because you can think, I don't know, I can trace on my organization, but actually someone's probably is already in the, at least in the experimentation stage of using Hugging Face.

Then you were exposed to malicious models. That's the second step.

Ashish Rajan: Okay.

Bar-el Tayouri: Third step is running a red teaming and like getting all the components into a contextual risk behavior, a behavioral risk. And so you [00:36:00] run red teaming, this dynamic testing, about prompt injection bias.

If you're even more sophisticated, which is better, even in the threat modeling run the red teaming on the models, not on the application. So you can choose the models that secure by design. We still don't have this terminology of secure by design and zero trust and things like that with AI security.

But it's like the layers. It's really clear that something there it's just not finalized yet all the terminology.

Ashish Rajan: Yeah.

Bar-el Tayouri: And then the last step, once you know all the risks, components, behavior, prompt injections, you need to create policies to enforce all of it. So every, it really depends on the organization.

If you're global, if you have customers in Europe, and if you care about safety, maybe safety is less relevant for you and all the security.

Ashish Rajan: I love this. And what about, in your mind, you're obviously talking to a lot of people about. Doing security for AI applications. What do you see as a [00:37:00] future?

And obviously I put that with a finger in the air at the moment, because as I was joking earlier, they might actually be an AI model that gets announced as we are probably publishing this episode.

Bar-el Tayouri: 100%

Ashish Rajan: Yeah, so if we're keeping all of that in mind, what does the future of AI security look like in terms of a lot of people who will listen to this conversation or watch this on our YouTube channel, what is something that they can do to prepare for, whatever the dynamic nature of this AI model world is?

Bar-el Tayouri: So it's pretty clear that it all starts from developers and product. And you want to to make everyone involved, to make all the stakeholders part of it. To put mitigations to LLM as a job, to harden the system prompt to make everything you can to harden and to put a to get visibility, that's like the things that will probably will stay in the future to do red teaming kind of dynamic testing, shadow AI, hardening, and the thing [00:38:00] that's going to change is the way we use AI.

And because as we use more and more in agentive frameworks and even like a, when we orchestrate multi agents there, we have new type of problems. That the current tools just don't keep up and can't cover. And then really the new solutions. But the beautiful thing is that because you always start with the experimentation and you have shadow AI, the shadow AI part should catch this use case.

So you should know that you have in the experimentation stage, usage of, let's say, orchestration of multiple agents. Agent swarms. So at least you can prepare before it's in production, and that's a beautiful thing about pre deployment AI security.

Ashish Rajan: Actually, it's a good point because, usually what happens when people develop applications, people don't really care about what's happening in development or tests.

But to what you said, this [00:39:00] is probably one time where you should probably flip this on its head and go, actually, I need to be mindful of what is actually being experimented on. So to your point about AI agents swarms or any DeepSeek, or if you don't want to work with DeepSeek that is, any of these that kind of come out from that perspective in the experimentation stage, before you even start putting actual data in there, which you care about.

Bar-el Tayouri: Exactly. We don't know the future. We can just try to prepare it when it's a but it's like short term going to happen.

Ashish Rajan: Yeah.

Bar-el Tayouri: Yes. And I super agree with you. It wasn't intuitive to me. But when I started working with customers and see that on a daily basis, it was it became clear how important it is.

Ashish Rajan: Yeah. And another probably final one is an and it's funny, I think I had a conversation on LinkedIn this morning about because OWASP recently released like a AI agents guide. And I was just like we don't even know what agents look like right now, but we have a guide for it. But anyway, keeping that opinion aside, there is this, [00:40:00] and the commentary was interesting for people who shared what they thought of on my LinkedIn about it in terms of the balance between innovation and security, because it's like the same old playbook again, where now development engineering is trying to develop fast so they can innovate faster and security is being called that, hey you're blocking us by trying to find shadow LLM or whatever, is there a, I don't know what the right answer or there's a perfect answer to this, but have you found a solution?

Anywhere people are able to find a balance between innovation in AI and while still doing security? Is that even a possibility that can happen?

Bar-el Tayouri: Look, from my perspective, innovation should be like should always be top priority. I can't handle anyone putting a hard stop on my work. And as a company, especially in in this AI revolution, You will just die.

You will just be behind if you won't put innovation as a priority. Our [00:41:00] job as security is to find a way, not to block, and I think if you look on the history, the good security vendors The good security companies and the good security categories were the ones and the ones that lasted, but the ones that enabled innovation and not blocked it.

So I'm sure we're going to do it with AI security.

Ashish Rajan: Yeah, that's a great way, man. I think that's all the technical questions I had. I've got three fun questions for you as well, by the way. First one being, what do you spend your most time on when you're not trying to I guess trying to figure out how to secure AI app, AI based applications.

Bar-el Tayouri: Wow. So I love to, I love to playing piano.

Ashish Rajan: Oh, like a professional piano or just I

Bar-el Tayouri: wish professional piano, but but that's why I also generated the music. It was a piano music because I didn't have, I didn't go to the gift of being, uh, professional. So at least I will do it professionally.[00:42:00]

Ashish Rajan: Fair. Fair. Okay, that's a good way, man. Second one, what is something that you're proud of that is not on your social media?

Bar-el Tayouri: It's pretty clear. I have a lovely, two lovely kids and like now it's a newborn. She's two months old. And I'm super proud of both of them. It's such a pleasure.

Ashish Rajan: That's awesome, man. We should show this interview to them one day when they're a bit more older to understand this. That'll be pretty good. I'm sure they'll both love it.

Bar-el Tayouri: They will watch it. It's not going to be 3D. They'll do it in, they'll have augmented reality glasses. Yeah.

It's going to be boring for them..

Ashish Rajan: What do you call the yeah, they're actually, immersive experience is what they're having. Yes. What you and I are seeing right now. And the third and final question, what is, what's your favorite cuisine or restaurant that you can share with us?

Bar-el Tayouri: Wow. I love I love Ethiopian food.

Ashish Rajan: Oh, like injera and everything.

Bar-el Tayouri: Yes. Like injera. It's so good. The only problem is I don't find people in the office to order it with me, but when I [00:43:00] do. It's a great party.

Ashish Rajan: Wait, so do you enjoy the Doro Wat and everything as well?

Bar-el Tayouri: I'm not giving the names. I know only we always order injera, which is like the bread, and then all the nice things around it.

Ashish Rajan: Little condiments around it. Do you actually go for the chicken or the goat dish? I think it's called, it's a curry, I think it's called Doro Wat, but, I'll definitely check it out.

Bar-el Tayouri: I order the vegetarian. I usually order a vegetarian.

Ashish Rajan: Okay. Fair. Okay, so sorry. That's why Doro Wat is a non veg one, so I don't know if there's a vegetarian version to it, but if anyone in your office if you want to get them on board and if they probably are non-vegetarian you probably should order that for them.

I think that's what converted me for Ethiopian food. I did not know it's like a wedding food. Anyway, I'm going to, I, if I talk about food, I'm going to talk for the next four hours. But dude this was really I think for me, I got to learn quite a bit. Where can people find you on the internet and what are you guys doing in Mend and what can you get help people with?

And where can they find more and connect with you on.

Bar-el Tayouri: Yeah. Amazing. Like we write so much on on my Twitter account, [00:44:00] Linkedin, so much content and blogs on the Mend website we cover, for example all the OWASP Top 10. Every two weeks we cover another threat. So we have all these new thesis about how AI vulnerabilities are going to look like and actually shaping.

And I write about it on my LinkedIn, Twitter.

Ashish Rajan: I'll share the link as well.. And for people who don't know about Mend, what does Mend do?

Bar-el Tayouri: Application security. Securing all the applications, all pre deployment. And our new focus is for AI driven applications.

Ashish Rajan: Awesome. And I'll put the link for the Mend website in the description as well. But dude, thank you so much for spending time with us. And I look forward to having more conversations as you get to know more about security for AI applications. Because it's so dynamic. Next time we talk, we'll probably be having an immersive conversation.

Bar-el Tayouri: Yes. My pleasure.

Ashish Rajan: Thank you. So thank you so much, Bar-el. Thank you so much for listening and watching this episode of Cloud Security Podcast. If you've been enjoying content like this, you can find more episodes like these on www. cloudsecuritypodcast. [00:45:00] tv. We are also publishing these episodes on social media as well.

So you can definitely find these episodes there. Oh, by the way, just in case there was interest in learning about AI cybersecurity, we also have a sister podcast called AI Cybersecurity Podcast, which may be of interest as well. I'll leave the links in description for you to check them out, and also for our weekly newsletter where we do in-depth analysis of different topics within cloud security, ranging from identity endpoint all the way up to what is the CNAPP or whatever, a new acronym that comes out tomorrow.

Thank you so much for supporting, listening and watching. I'll see you next time.

‍