Building Applications with Zero Trust Principles

View Show Notes and Transcript

Episode Description

What We Discuss with Maximilian Burkhardt:

  • 00:00 A bit about Max Burkhardt
  • 07:01 What is Zero Trust?
  • 12:52 Building Blocks of Zero Trust
  • 14:32 Zero Trust + Continuous Authentication
  • 20:49 Guide for Zero Trust Principles
  • 24:26 Zero Trust and Devops Pipeline
  • 30:25 Implementing Zero Trust Principles
  • 36:40 Skill Sets required to implement Zero Trust
  • 40:51 Defending applications built on Zero Trust Principles
  • 46:54 The Fun Section
  • And much more…

THANKS, Maximilian Burkhardt!

If you enjoyed this session with Maximilian Burkhardt, let him know by clicking on the link below and sending him a quick shout out at Twitter:

Click here to Thank Max at Twitter!

Click here to let Ashish know about your number one takeaway from this episode!

And if you want us to answer your questions on one of our upcoming weekly Feedback Friday episodes, drop us a line at ashish@kaizenteq.com.

Resources from This Episode:

Ashish Rajan: [00:00:00] hey

Max Burkhardt: absolutely. It’s great to be here.

Ashish Rajan: Oh man. I think it’s really interesting to kind of bring people who have known me from before, or at least

But the beard, but I appreciate you coming on board and thank you

Max Burkhardt: for coming to see the new visual rework

Ashish Rajan: I’ll show you. That’s a good way to put it. I’m going to tell my wife about this. Like it’s have you honey, have you noticed any visual?

Max Burkhardt: 21? Yeah, this is my

Ashish Rajan: 2021. So max, for people who probably don’t know you, and I’m curious if you can tell us a bit about yourself and your journey into Cyber security.

Yeah,

Max Burkhardt: absolutely. So my name is max. I’m a security engineer working in San Francisco. I sort of grew up in the bay area, California area and have worked here or sort of my whole life, Security for me started as a passion that was very early somewhere around, middle or high school. I started learning about this as I was kind of getting into tech.

And fortunately I had a few opportunities that allowed me to really dive into that more deeply, mostly actually through working in a sort of IT roles in various schools along the way. So I, I did some IT support at my high [00:01:00] school. I did some IT support at my university. And was able to kind of always have an eye towards like, oh, what’s the secure way to do this?

Or how do I learn about how to compromise this server that I just set up and started, , basically learning things from there. I , kind of transitioned that into starting a cybersecurity competition team at UC Berkeley, which is I, which was a really fun time. We can talk about that if you want.

But basically it was just always trying to learn more about this. I think the sort of puzzle aspects of information security are really, really good. Just that sort of satisfaction of understanding how to exploit works or how to defend the system. It’s something that is super thrilling to me.

And so then I was able to fortunately carry that through, into a career first in pen testing, and then now in various application defense.

Ashish Rajan: Alright. Oh, so you were in like on the pen testing consulting side and then you move forward to the company side.

Max Burkhardt: Yeah. Yeah. So, yeah, basically when I graduated university.

Well, there were kind of two things going on. One is that I knew that consulting would give you a really good variety of. Yeah. , the company I worked for, I was doing projects that were two to four weeks in length. And each [00:02:00] time you’d hop between those projects, you could have a radically different tech stack or target, or like, , kind of goal in mind near assessment.

And so that was really great. And I knew that would be good for learning is that, one week I would be doing OWASP to web application security stuff , and the next week it might be trying to like write a fuzzer for some sort of binary protocol. So that was neat. But then the other. Aspect, which I’m not going to lie about is that, , I wanted to definitely officially be a hacker.

And I think that if I kind of went into the offensive side of things for a little bit, I knew that I would get , the experience and space , to learn this. And so that was kind of what led me there. But over the last , six years or so I have moved into the defensive side because honestly , it’s a little harder and I think also a bit more rewarding.

Ashish Rajan: That’s awesome, man. I think we can definitely do it with a lot more skilled people like yourself on the defense of that as well. So thanks for coming on the other side jumping from the dark. But now on the topic, the zero trust principles, it’s a term which has a different definition for a lot of people.

And I’m curious, what does zero trust mean for you? Maybe let’s just go with that.

Max Burkhardt: Yeah. First off, like I know that this term has sort of [00:03:00] become there’s a lot of hype around it it’s become a buzz word. I don’t really know when that happened. It seems like it was kind of recently. I know that like NIST one of the United States, like a standards agency has some official terms for this. But just sort of personally, the way that I have thought about it. Well, the easiest, quickest thing, which is not entirely cracked is no VPN, right? That would be , my fastest way to say, what is zero trust is get rid of the VPN.

But more accurately. I think the way that I see zero trust is it’s a philosophy of trying to make it, so that individual components of a system, whether they be web applications or servers or something like that are sort of given more context with which to make authentication decision. And so when a user shows up at the door of a web application or an SSH server, instead of just having maybe like a single username and password or a single SHQ sort of authenticate that user, that system gets a little more context.

Like where’s this user coming from? How have I seen this IP address before? Is this a device that’s managed by the company? , I’m owned by has there been some sort of multi-factor authentication, [00:04:00] there’s all these additional signals that we’re now making accessible to the underlying backend applications that lets them be much smarter about whether or not to allow access to something.

And so this really lets us raise the bar in terms of defense because you it’s much harder to. Fake all of those requirements or slip past all those defenses. If you have to do it for every single system you’re trying to connect to. Right. Well, we’ve known it’s been not a great architecture for a long time, but this concept of sort of a hard shell in the soft center is pretty pervasive in a lot of networks.

And I think zero trust is really just sort of a rebranding of the decades, long effort that’s been going on in InfoSec to try and make it a hard shell and a harder.

Ashish Rajan: Yup. Yup. I can definitely say that. We just re we just put a different lipstick on it, I guess, for lack of a better word. So maybe let’s just put it this way.

I guess we would have a lot of people who are thinking about from an application perspective, but they would not have thought about zero trust in an application. So maybe a good place to start would be what are some of the building blocks for say to your point no VPN kind of access to an [00:05:00] application.

Yeah.

Max Burkhardt: Yeah. So, this is a point where I think that yeah, application security and infrastructure security really blend together. And I mean, this is a cloud security podcast. And so I think if you talk to a lot of cloud security practitioners they’ll agree that, , yes, it’s infrastructure security, but there’s a lot of application stuff that’s going on as well.

Right? Like I primarily work with AWS at my day job at. And, , I’m technically working on infrastructure, but all I’m doing is making rest API calls on systems that are making other risks if API calls with each other. Right. So it feels a lot like appsec in a lot of ways. And so basically what are the building blocks?

So you need some sort of interface that can sort of surround your application and a, give you confidence to put it on the internet. And. Give you some authentication hooks that will allow you to integrate with like a broader authentication scheme. That first point about like giving you confidence to put it on the internet, right?

Like you are probably not going to go put your plain no JS like default web server directly exposed to the Wilder net. , you could try, it might be fine, but that’s probably not an [00:06:00] architecture I would pursue. So you’ll want something where you have confidence in like, oh, I can patch this. Or my cloud security provider can patch it.

Actually. That’s the one that we, we generally recommend is like, let Amazon handle the patching of possible. You want something that’s going to be able to receive your requests and then , sort of from other actions, but then authentication is really this key part of the zero trust system is right.

There’s zero trust at the beginning of the conversation. But theoretically, when a user establishes the session, like they will have proven that they are trustworthy via some sort of authentication. And so here’s where you can hook into a lot of modern authentication capabilities that give you the sort of stronger insight about who is connecting under what circumstances.

So a good example of which is, , what we used in the blog post, I wrote that, I think so, sorry, I started this whole conversation is. We have a variety of applications that we want to sort of behave in the zero trust manner. They are on the internet and via AWS application load balancers. , that’s a, a HTP front-end to the internet that [00:07:00] Amazon patches and keeps secure for us and we can configure it to be do what we want.

And then that is able to provide authentication hooks to our single sign on provider. And so that’s another kind of key point about all of this. A lot of zero trust. As I see it is about sort of distributing the load of authentication. It’s about making it so that an individual applications can make those good security decisions.

But at the same time as a security team, you want to be able to have some sort of single point of visibility and that sort of single sign on can become really powerful.

Ashish Rajan: Okay. Right. So you’re saying building blocks would be primarily authentication based or are there, like I said, so obviously there’s a component that you mentioned with.

They use the products which are provided by HR service provider. So they don’t have a bachelor like maintain anything. Then the other component is authentication. Is are there other pieces that a hundred or nothing to your point cause I feel like application probably has a lot more or, but that kind of covers most of the

Max Burkhardt: perspective.

Yes. Moving on. And like it’s, it’s kind of hard to draw exactly where the lines these building blocks are, but I think another key thing after you’ve gotten so, okay, so you have some sort of system that can authenticate it user. That system needs to be able [00:08:00] to present information securely and reliably to the application about who this user is.

It is tempting to try and write these sort of rappers around apps that do all the authentication than just like pass the user’s traffic into the application. Say like here we promise it’s secure, but really at that point, you sort of like, reimplemented a VPN. And so I think this is where you really get into the application security side of things is if you have a system that can authenticate a user.

Maybe create some sort of cryptographically signed token saying who they are, what they should be allowed to do, or what groups they’re in, then your application can then do authorization, logic, or access control logic on that information that I was able to get from sort of your zero trust layer, and , access control is a really hard problem.

The amount of complexity that you quickly end up in, as soon as you try and build one of these things is very significant. But. A core piece of that is making sure that your authentication layer can effectively and efficiently say, like, we’re sure this is this user. And here are some, here’s some additional context about them so that you can make access control decisions.

Ashish Rajan: Oh, so would you say Zero Trust is kind of [00:09:00] is also continuous authentication as well then? I mean, but continuous integration, which is seamless to the end user, they don’t realize that, Hey, I’m being asked for MFA every time offended request, but it’s more, it’s consistently validating, Hey, this is.

Logging in,

Max Burkhardt: Hey, this is the Ashish . I think that’s what you should be shooting for. Right. And there’s a lot of fun attacks to think about how you might be able to get around that. So, , one of the things that I certainly think about is like, , as a pen tester, it’s a very profitable exercise to compromise someone’s machine, take their cookies and then browse the web.

Is that right? It’s a great 2FA bypass or somebody may have signed in with 2FA at one point, but then they get a session cookie. Like authenticates the session for the next eight hours or whatever. And if an attacker can get access to that, then , , they’re off to the races. And so it’s interesting to think about how you can use some of these sort of more modern authentication mechanisms to try and avoid that sort of thing.

So a good example might be if you are doing some sort of mutual TLS as well. A regular web session or you’re like periodically sending the user to like like a sort of invisible redirect MTLS [00:10:00] page in order to kind of go feel that off, that might allow you to detect a stolen cookie or.

Defeat that attack because even if an attacker has access to a cookie, they may not necessarily have access to like assert that would, I would identify them as that device. So continuous authentication is an interesting one because , performance matters. Right. And any security team that tries to go say, like for every request you must like put in username and password MFA and use the certain stuff like that.

Like , you’re probably going to be assigned to a new project or something, so that’s not going to work, but I think there’s ways that you can fit in some of those things. By focusing on the user experience about authentication the first place. You can make sure that like re authentication is pretty easy on like, so a huge one for us is that web authen is such a nicer mode of multifactor authentication than the old, like text you a code or code generator once was right.

Like we have, we’re fortunate to have a employee base. That’s mostly. And the fact is that people can just go to their web application and then touch their fingerprint to touch ID. And it’ll do a web authen multi-factor and it’s like a really beautiful and [00:11:00] smooth user experience. And so people don’t complain about multi-factor nearly as much as they used to.

And they had to go like dig out their phone and like figure out what code to put in and stuff like that. Because I always

Ashish Rajan: find that this MFA, you should be a thing because it makes you a separate device. But now nowadays you have YubiKeys. You can just put that, like that into your Mac book or windows, laptop, whatever.

But that kind of helps you bypass as long as that’s connected in. You don’t need 2FA on everything as well. I’m sure it’d be, we’re going to use that and reduce the annoyance.

Max Burkhardt: Yeah. So that’s, that’s a good point about like the multiple devices things. I think that that has been a confusing factor.

, throughout the rollout of MFA, I, I think the way that I look at it. , dividing like this device must be physically separate from this one is maybe not the most useful mechanism for thinking about that attack. It’s more like what levels of access might give you access to someone’s machine or their session or the password, but not necessarily the device.

And so with a lot of the web authenticators, the thing that you’re kind of checking for is physical presence. And. There are many attacks where an attacker gains with her digital presence on someone’s machine, but not any sort of physical one. And so that tends to be a pretty [00:12:00] powerful differentiator there.

Yeah,

Ashish Rajan: that’s interesting because they’re almost like you went from an authentication perspective and thinking about zero trust. We have , the trust that we have on the device itself that, Hey, this is Ashish logging in from a company laptop . And then there’s the other layer off of that is this is actually a Ashish because he has a 2FA or YubiKey or whatever.

And then the layer after that, another layer is, Hey, can he access facebook.com for like, I’m just going to take or whatever they call meta com now or whatever they call now. So if you can go to the metaverse, if he has the ascending close, there’s almost like three layers of authentication, just like, so to your point about the access control thing, it does get quite complicated when the number of applications or the number of metaverses.

I don’t know, 10, 20 15, cause depending on SAAS applications stuff as well. So what are your thoughts on like those kinds of things? Does that get complicated or is that just

Max Burkhardt: simplify? It, it can, I think that, an important design decision is you’re thinking about building this sort of system is like, what sort of things do we want to handle centrally and what sort of things do we want to farm out to the applications themselves?

So for instance, , and our F figuration, we can [00:13:00] sort of say centrally, like here’s the conditions in which we want to require multi-factor authentication. Right? And so we don’t need every application developer, who’s working on an internal app to do this or that, to think about what the multi-factor requirements are like, that information is available to them.

If they want to do something based on it. But. Hmm, centrally. We can confirm that everyone who’s accessing something used a a multi-factor key in order to get access to that sort of thing. And so , that’s the place where we’re doing some sort of central control to give us that sort of guarantee, but still that information is available to the backends.

If they, if they wish for it, there are other cases where you might want to delegate more. So for instance, let’s say I’m trying to think of a good example. You could potentially do something where you have an application that wants to show some information about. Wants to allow login from mobile devices and laptops, for instance, but might want to not display all of the information on mobile.

So maybe there’s certain information that’s too sensitive for mobile devices, because those are more easily lost or stuff like that. And so that would be an interesting case where it’s like, well, the central system will let login [00:14:00] happen under any circumstances, but it needs to give some sort of signal to the backend of like, this was a device.

That is guaranteed to be a laptop. And like, we know that this is a company machine and therefore it’s okay to show them more information, but then, or other cases like it might not be. And so in that case, really, you’re kind of delegating to the backend system because as a sort of a unique use case for them.

Ashish Rajan: Interesting, actually and it’s kind of a, my next question is kind of what we talked about over here. It’s also just trying to quickly go through the comments. Hey, good day to all Iowa. And the question from Vineet is, is there a standard or guidance practice which devs can follow with zero trust principles?

Max Burkhardt: Yeah, so I mentioned there’s a NIST technical document about zero trust that I found to be a pretty well-written one. It is a little dry because it’s a standards technical document, but, I think it kind of gives a good. View of a lot of these things without a lot of vendor hype, which I think is kind of the problem with a lot of trying to learn about zero trust is that you Google it.

And then you find a lot of vendors who are selling you various zero trust things. , it’s kind of being used more in a buzzword context. So I think that’s a good place to start. [00:15:00] But also I think there’s a lot of conversations like this one happening in , the internet and YouTube and podcasts and stuff.

And so. Hearing people’s takes on and try trying to think about that philosophy of like, how do we distribute this authentication context and how do we make it so that individual apps are empowered to make good, authentic decisions is really where you want.

Ashish Rajan: Yeah. Awesome. And hopefully that answers the question Vineet , because I’ve always been curious about the misinformation.

That’s there with zero trust as well. They know, I think you and I have just touched on the fact that had his device and that is the, user itself 2FA and all that. But you almost kind of go well, is that enough? Because technically zero trust, like how you’re trusting the authentication device. But it’s, it’s kinda like the space. What do you trust? Like you eventually have to trust

Max Burkhardt: something. That’s right. And that is where you have to think about. Sort of the possibilities for attacks and what your threat model is. Right? So like something that I’ve been, I mentioned a few times is this like concept of device trust, the way that a lot of vendors I’ve seen implant device stresses, you have some sort of like agent that’s running on the machine, that’s supposed to verify certain [00:16:00] things and that.

We’ll provide certificates to that machine that are like signed through some sort of central authority on some schedule. And then if you can present one of those certificates, you’re considered to be a trusted machine. And so the theory there is that if you don’t, if you. Get one of those certificates, then you must have this agent running and therefore the agent must think that your machine is compliant.

And we all know that’s not entirely true, right? Like, so it’s can be stolen. You can access things and key chain access. If you can convince the user to like type in their password at one point. And like this there’s all sorts of tricks that are red teamer or actual attacker might be able to use to get one of those certificates.

But at the same time, that is a. Level that gives you something, right? Like if somebody steals a phone that never was device trusted, like even if they can still sessions off that they’re never going to be able to log into an application that like required that certificate author. So you have to think about for each of these controls, like there’s going to be some sort of case in which an attacker can subvert it.

Right? Like, Somebody’s YubiKeys can be stolen. Like sometimes an actor does have physical access. And so it’s important to think about those things, but it’s also important to remember that most of the time, the things that are getting [00:17:00] people compromised are phising and password reuse across multiple cloud providers.

Right? And so there are a few techniques that are going to be really effective against those things that appear to be the major drivers and breaches.

Ashish Rajan: Yeah, and I think in a way it’s good that you’ve got pentester background as well, because you can kind of talk about the attacking side of things as well.

How someone misuse it. And I’ve got another question here from also, we don’t answer the questions for any, thank you. I’ve got a question from Ayo as well. How does zero trust principles fit into a devops pipeline?

Max Burkhardt: That’s a good question. I would say that , a lot of this. Concept of zero trust being about like letting applications make their own security decisions actually fits very well into the dev sec ops philosophy of make the people who are writing the code also responsible for deploying and securing that.

Couldn’t right. So if you are thinking about like, sort of your ideal dev ops engineer, They build the application. They want to make it resilient. So it doesn’t go down. So they don’t get paid at night. They want to make it secure. So they don’t end up in like security incident review because their data got leaked and stuff like that.

And zero trust can [00:18:00] give that ideal engineer sort of the information they need in order to make their product really secure. Right. Like in, in a sort of earlier world where you think about, like, we just have a bunch of applications that are running on this central network, right? Like. An application developer has very little context to say like, oh, I can make sure that no attackers connect to my service because all they know is like, who has an internal IP address, right.

That’s not a very solid basis to make us authentication decision on. So I think it is philosophically aligned in terms of technical things like maybe. I don’t think there’s necessarily a strong link there. The tools that you would use to build a zero trust thing fit into a DevOps scifi, but also can work with other sort of development methodologies.

But I think that the idea behind both of them is very similar. It’s like push that sort of information down to the edge. So that individual components can be resilient and secure. Just

Ashish Rajan: an interesting point as well. And I’ll be keen to know , your thoughts as well on this. Cause it’s such a trust of the keyword being trusted as an organization, you may choose not to completely [00:19:00] trust your DevOps pipeline, or you may choose to go as long as the pipeline I trusted, like, , I wonder if there’s that because to your point about the philosophical.

As an organization, we could just trust that fact that yeah, they know what they’re doing. Max sounds like a smart person,Ayo sounds like a smart person. When they combine and make a devops pipeline with zero trust, I’m going to trust it. I think that’s how probably a lot of people would take

Max Burkhardt: it.

That’s a good point actually in like attacking CIS systems is, has been one of my favorite things. And it’s something I think a lot about because fundamentally. I think this doesn’t always get enough time. Maybe it is more now, like I’ve seen some cool blog posts about it, but , a CIS system is literally a remote code execution as a service, right?

Like its whole job is to take this blob of data. Some of which is new and changed and none of it has been reviewed yet. And then to run the code inside that blob of data and then give you some results. And so. Fundamentally the challenge of securing that is, is pretty tough. And so I think it’s an area that’s going to see a lot of cool development in the next few years as like vendors start building more security features into their CIA systems or teams start thinking more about like, okay, what is [00:20:00] our chain of trust here?

Exactly. But yeah, that’s a tricky one because , you want to, you want to make sure that there are a number of security controls before somebody can push something to your CI but. Once that happens, it can be hard to defend. And especially if you’ve maintained an open source project, like we’ve seen a lot of open source projects that have a big problem with this because they have need, or even a harder problem.

They want to accept code from anyone. They’ve got a system that runs code as a service, and then also they need some way to like securely. Final artifacts. It’s a tough one.

Ashish Rajan: Yeah. And I think , it reminds me of a thing. Cause I think I’ve started having a conversation with someone about the logs being generated by OCI system system and open source tool.

Again, not none of the secrets were being I guess masked and you almost need a log. If you can see the passwords or secrets going through. I could clear the AWS session token going, I’m going, this is not like, so technically I don’t even have to access my CICD, this system. I just need to have access to my logs and I can basically recreate the thing.

I can see the person’s credential there. And sometimes PII. I mean, yeah, there was so much depth to it, but yeah, as an organization, some [00:21:00] people may choose, , I don’t really worry about this. Sort even my net drug don’t reach it. Yeah. Show whatever,

Max Burkhardt: man. Yeah. Yeah. I mean, I think it would be cool to be able to get these systems to the point where it’s like the CI system has so few permissions that it’s fine.

If everything is totally in the clear, because that’s the place where you want developers to be able to be really free and how they do stuff. . Right. I do want that to be the high velocity point where they can run whatever tests and not get in the way of like bogged down securities. I think the really challenging part is then, okay.

Once you have. Like run your tests. How do you then run some more verified tests and do a build that, , is like off of good reviewed code or things that have gone through your stack analysis, whatever like that. So , I think that will be something that will be cool to see. As people build more paradise.

Ashish Rajan: Yeah, cause it’s it’s so commonly, almost like to your point, no one questions, the, I guess the legitimacy of how you’re really protecting the CICD pipeline, because I remember for a long time, and this is probably the last point. I would start talking about this after this, but Jenkins didn’t even have like a login page.

It was for so long, it took so long to build a log in, but the new thing. It’s the CICD . So to your point, which has access keys to that’s probably our [00:22:00] admin access keys as well, internet environment. It allows me to change the code. So, all right, I’m going to stop my rant there, but very different parts. I am talking about.

devops pipelines and applications. Maybe it’s a good segway into the project that you did and presented on. I’d love for my audience to kind of hear the, I guess the thinking behind it and how you kind of build a zero trust principles in the application that you. .

Max Burkhardt: Yeah. So specifically with this project and I won’t go into the full depth because there’s a blog post and a talk, you can watch it if you want to.

But basically we had a number of internal web applications at Figma and in my experience, all companies have some of these. Right. And then we knew that as our company was growing, they were just gonna be. And we wanted to make it so that it wasn’t going to be a huge lift for engineers who want to build an internal application to do something, whether or not it’s deploying the site or viewing some internal documentation or like , interacting with a development server or something like that.

We wanted to make sure that, engineers were empowered to easily spin these things up with security properties that we were comforting. And the previous model that [00:23:00] had been used was involved like these clients certs that were required to access these apps and like that worked pretty well.

Like that’s, it’s kind of hard to , it’s hard to fish a TLS cert, but it was really a pain on the operation side. Cause like distributing the cert was hard and like browsers and I’m always good at handling clients or it’s, it was kind of a mess. And so the idea was how can we use these modern features of cloud providers, like AVS and our all right.

So provider Okta and. Everything, so that we can kind of present this interface to engineers saying like, , use this Terraform config to set up access to your app, and then you will immediately get a, , login with our SSO and multi-factor authentication and device trust and like a bunch of other little things on top of that.

And so, yeah. The foundations of the solution are built with AWS LDS on it connects through Cognito to use Samuel with Okta. That was a little bit of a an interesting. Like AWS, quirk basically like ALBs do OITC authentication. Okta actually does support direct ODC, but we ended up [00:24:00] using Cognito as a sort of middle service there, which has been kind of helpful because it means that we can , kind of plug in arbitrary off backends to that.

And so if we want to have an application, that’s like employees, plus some other people, you can kind of add them in at the Cognito layer, which is like, kind of any trick. So, so that was kind of how we sat that authentic flow. And then we were able to utilize some of Okta’s features to build really strong security policies that we feel it gives our apps a ton of fish resistance and a really good visibility into what people are logging into and accessing and that sort of thing.

The other area that We were able to utilize here as like, , the ALBs confronted sort of centralized logging. That was very helpful. And then, sort of cherry on top that we were able to add, which I was pretty happy with is that, , we, we built the system for acts and internals, these internal web apps.

But it wasn’t long before someone said like, well, What if I build an API that we, that I want to be internal and it wouldn’t be accessed through a browser it’s like through, for a CLI tool or something else. Like how would we make that work? And at first it’s kind of tricky because if you want to add a lot of advanced, authentic features, [00:25:00] like how do you do web authen to a CLI tool that’s then going to make API requests like this that’s pretty hard.

But we were able to come up with a solution using a serverless function Lambda, that was able to sort of implement. CLI off capability or this API access capability for all of these services sort of at once. It was like a really good validation of, okay, if you can build these sort of centralized components that are reusable and somebody can just like flip a flag in their Terraform and then suddenly this Lambda gets deployed and it allows them to do this sort of CLI off to their to their app.

So yeah, the motivations were really. To make the act of spinning up a new service and exposing an engineer or employees in general. Not a horrible process. It’s something that like maybe security would be vaguely involved in that we’d like to give some advice, but it shouldn’t be like, oh, I have to spend a week doing this thing.

It should be, I’m going to go take this terror from config. And I’m going to change the name to my app’s name and then apply that. And then cool app is on the internet. Now at the same time, we can have all these controls around it about who can log in [00:26:00] and that sort of thing. Like I mentioned before, the sort of.

Representation of user identity down to the underlying services really important there. And so fortunately the ALB plus Cognito plus Okta flow on does allow you to send a lot of information down in sign JWTs. And so the underlying applications can see signed information about who is accessing them and, make decisions based on that.

So we use that for permissions in a bunch of places.

Ashish Rajan: Okay. And I’ve definitely, I’ll what I’ll do is also linked to. The talking, cause it’s supposed to talk at fwdcloudsec I’ll definitely. encourage you to read the blog posts as well as to

Max Burkhardt: talk. Cause I guess I just ran through a 30 minute talk in five minutes.

It’s probably like right there. I appreciate

Ashish Rajan: that because I think it would be a, it’s probably dark fair for me to kind of explain the whole dog. And she’s a 30 minute talking to like five minutes or even not even fighting. But I think that w the few things that I took away from it, that’s hard to read really interests you, and obviously tones the components we’ve already spoken about in this episode so far where it’s about I guess, the underlying principles and how do you implement them?

Something else as well, because maybe after this people get pumped, they read the blog, [00:27:00] they watch the talk that he gave it fwdcloudsec. And. One of the things that people would, I guess, think about is, Hey, how do I start building this in my team?

And I’m curious to know about initial thought process that you had, like what kind of like things, people who are looking for, what kind of skillset should they be thinking you might as well in their team, like could be a team leader or tech lead or a CISO here thinking that should be, we should do that. Cause I’ve alwaysthought about implementing the zero trust principles like what kind of skill set are we thinking?

Max Burkhardt: Yeah, I think

Max Burkhardt: the. Well, one of the things that I remember from sort of the beginnings of this project was sort of a consistent push to see, like, could we make this part better?

And we thought about that a lot from a user experience perspective, as well as the security one on. So. There was a lot of early design iteration where it’s like, well, , right now, , as it’s currently configured, like he’s asked to click this button to continue the awful, like, can we get rid of that button?

Can we make it as seamless as possible? Can we, can we make it so that people don’t realize authentication is happening? And that’s actually been like a fun thing that we’ve noticed in a few cases is that there were a few cases where we set this up. It was pretty seamless. At one point people were similar size cause they log in.

Cause they had like [00:28:00] a non device tested thing and they had not actually realized that auth was ever enabled on this service. They did, it was just totally transparent to them. And they’re like, oh, I didn’t, there was not there. So th that was kind of a cool validation. Some of that user experience work we did.

So first off, like what do you need a team? Do a lot of mock-ups and tests and try to keep thinking about like, how can we make this more pleasant to use, because that’s how you’re going to get adoption. Right? You can build a cool secure platform, but if people don’t want to use it or it’s too hard to integrate with, it’s not going to provide you a lot of security value.

The other thing that I’ll note, which is just sort of a fact of the design here. Like a lot of what we were trying to do was use these core infrastructure components that come from things like our cloud provider and SSO provider and connect those up so that we didn’t have to build a lot of our custom code.

Right. We’re a small security team. We’ve got a lot of stuff to do. We can’t spend all day like keeping the site up basically. So a lot of what we’re thinking about is how do we connect these various components that weren’t written by us. But another impact of that is that when you’re trying to combine various [00:29:00] components from multiple different, , technology providers, you’re going to run into some interesting bugs.

It’s going to be sending compatibilities. You might need to like try a lot of configurations, dive deep into undocumented Terraform options, like all that sort of stuff. And so there’s a fair amount of. I guess you could say like resilience or willingness to like, just keep trying stuff and tweak it until it works nicely.

So somebody who I knew who kind of described a particular class of technologists has sort of the tinkerer, which is people who can take a system and then, , attach a new thing and figure out how to get those things to work together and tweak this and like customize that and like, just keep iterating until you get a really nice product.

And I think that was the sort of spirit that made this project. We had high goals for ourselves from the beginning about how it was going to be to use and the security properties you’re going to get from it. And then we just kind of kept iterating until it got better and better. Something that I.

It’s not immediately clear from like the blog or the talk stuff. It kind of seems like it was all one project. This was a work that was done over a period of about nine months as we like built an MVP that did the authentication. And then we realized [00:30:00] like, oh, we could add on this. Or like, oh, we can now do better stuff with JWTs here.

And like, as we learned more about the platform and learn more about the use case that we were trying to fulfill, we’re able to make it better and better. And so it kind of gets presented as like, oh, we built all of this. Right out the gate and that’s not really how it works. And I suspect that’s not really how any products work.

Right. You build what people need from the beginning. And then as you understand that use case better at you add on and on and, , make it better.

Ashish Rajan: Yup. Yup. And I think a couple of mentioned correctly, skillset in himself can be another podcast because we can get, you can talk about it on a very different driver toward at that point, I’m actually thinking, like, say of someone who gets inspired by this talk as starts building on applications, on zero trust principles.

That was the security like there. Cause you come from a pen testing background as well. And obviously there’s offensive pieces. Like we spoke about the cookie replay as well. From a defense point of view with the application built on zero trust principle. Does that change dramatically? How would you defend it or it becomes easier?

Like , what were the thoughts around that?

Max Burkhardt: Yeah, there’s things to get easier and some things get harder. If you’re at the [00:31:00] point where you believe an incident has already happened and you’re trying to investigate, I think that’s where you actually got. A lot of the gains of this system is that you have some logs from a service that you think are incorrect.

And like that service had more information about who is accessing. Then it would have otherwise, right. These gone are the days where it’s like, oh, well, this service was hacked by 10 dot nine dot three dot 42, who knows who had that IP two weeks ago. Right. I’ve seen very few systems that keep like a really good track of like all historical IPS in their assignments over a long period of time.

Anyway You have more information like, oh, well there’s a suspicious request, but like here’s the user that sent it. And we’re like pretty sure that it was in fact that user other things like, , source IP addresses now much more meaningful indicator than it was previously because now. Yeah, especially in a COVID work from home environment.

Like we can actually build , a pretty good profile of employees and where they connect to our systems from right. A lot of people are working from home. And so we kind of know what their home IP address is. And so suddenly when we see a new IP, that’s totally outside of that. I mean, sure.

They might have gone on vacation or gone to a coffee shop, but [00:32:00] that is another signal that we can use to, to build a better picture of what might’ve happened during a security event. I think a lot of that investigation can kind of get easier because you’re enriching more information about what was going on at the time more throughout the stack.

And you don’t have to do these sort of tricky correlations of like, okay, well, my VPN server had these logs. I think it gave out this IP address that user. And then I see this IP address later. Like you don’t necessarily have to do as much of that hop stuff. And you also don’t have to think about like, oh, it was 82, 1 X working in my office, like our Ethan at Ford security, you can kind of like get rid of that, that whole can of worms.

I will say though that from a defense perspective, ruling this out is tricky because as you are going from this model of like, everything kind of had this protective shell around it, cause it was on the internal network and you’re bringing things out. You have to be pretty meticulous about understanding what your attack surface is and making sure that everything is ready for that new prime time of like being on the internet.

Totally. Right. There are misconfigurations you could end up introducing in your role out here that might expose certain [00:33:00] paths even to authenticated users or might skip certain authentication requirements in certain cases. So you have to have a really good understanding of like, How sure are you, when you deploy a system in this model that all of your guarantees are being enforced?

Something that we’ve been able to have a lot of success with? There is even when we add a rule, does that. And the pay is required for the service. Adding alerts for even cases we think should be impossible. And that’s been very productive. It’s like, okay, well, even though the October rule says, you must have web authen under all circumstances, we’re still gonna write alert rule for somebody logs in without web out then cause then clearly our assumptions , are in variants are incorrect and we need to go back and revisit those or understand what was missing in that rollout.

And so, That’s really been a strategy we’ve used as this sort of combination of configuration, but also alerting and , high signal, high priority alerts for things that we think should be impossible. And those are really good at pulling out things. Misconfigurations are places where you missed something and things like that.

So, I’m not going to lie. There’s a lot of challenge here on the front end of like, as you’re going to the [00:34:00] small, and as you are taking that, , 12 year old PHP web app, that is really important and you want to like go put it on the internet as opposed to this internal thing. I think it’s going to get more secure, but you’re going to have to acknowledge that it might be a little bit of a challenging road to be confident in that deployment.

Fair

Ashish Rajan: enough. Now I think it actually explains part of it as well. Although there might be a bit added complexity, but there are definitely some benefits as well. Yeah.

Max Burkhardt: Yeah. It actually also briefly touched on something that I had not really appreciated before I was, I was doing this work is that I’m writing good alerts can actually, it’s not purely a reactive thing.

It can actually help you build a better system. And , I think sometimes detection or response is sort of pushed off into this other function of like, oh, we write alerts and then. Those people will often their office. They look at them and they like deal with them appropriately. But I think it’s a feedback loop that really should be involving the people who built the system as well.

It’s really like DevOps all over again. Right. If you are implementing a system, you write some alerts and you see what those alerts say, and then you like learn more about the system that you just built.

Ashish Rajan: That is true because you’re the one who’s building. It kind of like the way the QA testing kind of [00:35:00] disappeared in the development land.

The developers now expected to do QA testing. There’s a whole separate QA person. So similar to that, I imagine this is the , same concept you’re building it. You probably know what a negative test

Max Burkhardt: is as well. Yeah. And also it encourages you not to write flaky or like inconsistent security alerts.

Right. Because you’ve ever done that detection response role. Like those are the worst. Right. And you have like 300 alerts and he’s no, they’re probably 299 of them don’t make any sense or like aren’t necessarily. It’s important to make sure that everyone throughout the stack of writing that alert and building that system is sort of held accountable for that and that the actual security signals that you ended up acting on are really high quality ones.

Ashish Rajan: Yeah. Yeah. No, fair enough. I appreciate that. I think yeah, I think it’s a good one to kind of, well, I was going to say that was kind of my last question as well, but what I’m going to do is for people who are kind of jumping into. You got that going into way deep into the app application security metrics.

I’m going to pop some videos that are later on after the stream for people to kind of refresh application security and threat detection. So this is kind of like the final section and it’s just basically fun questions about the [00:36:00] three questions not to it’s just for people to get to know you a bit.

And the first question is what do you spend most time on when you’re not working on technology or cloud

Max Burkhardt: or zero trust? Yeah, I know. So something that I am really passionate about is a tabletop role playing games and collaborative fiction. So I played D and D for a long time, but I’ve recently been really enjoying the Indy role-playing games space.

And so I spent a lot of time designing and inventing worlds and then inviting my friends over. On a zoom call to go play through those and like build interesting stories together. So that’s something that I really love doing. Yeah. Well,

Ashish Rajan: I’ve played quite in depth. Like, do you guys dress up as

Max Burkhardt: well? We generally do not dress up.

It’s just words. But that also seems fun. And if I were better at the sewing machine, then perhaps I could get into that.

Ashish Rajan: Fair fair enough. I think I, I think it would be a really interesting to have a story play out and see where it lands, but cool. That’s a great answer, man.

The next question, what is something that you’re proud of, but it’s not on your social media?

Max Burkhardt: Well I think over the, so this is kind of a recent one, but , I I have been trying to. Start the [00:37:00] journey to becoming a plant dad. So when I moved during the pandemic, I got a place that had a little bit more natural light and I was like, think I can replace all of these.

Garbage Ikea plastic plants of like real ones that are alive and, and, , try and make my living space a little more more pleasant. I really like spending time outside as well. So I wanted to bring some of the outside in. And so I think that I’ve been able to make some gains on some very healthy houseplants and some nice window boxes.

And it’s, it’s it, it gives me some joy, but none of my social media cause. Anywhere near the level of many other people. It’s just sort of like a personal pride item that I’m like, oh, I grew up. Yeah,

Ashish Rajan: fair enough. Fair enough. I tried being a pant plan that, but that did not work out. I killed like three plants after that.

You’re like, nah, I don’t want to kill anymore. So it’s definitely a journey. It’s a good luck with that. And I’ll, I’ll definitely make, maybe take some tips from you once you’re done with that. One more final question. What’s your favorite cuisine or restaurant?

Max Burkhardt: My favorite cuisine Technically cuisine, but I just love burritos.

, I’m from San Francisco bay area, the mission burrito is very close to my heart. And my favorite mission burrito is of those from LA [00:38:00] talk, which is that 25th and mission, I think. So if you’re ever in San Francisco, that’s the place to go. It’s cash only big line, but it’s totally worth it.

Yeah, it’s a fantastic, fantastic reader.

Ashish Rajan: Oh, I feel like burrito is another one, which is like, they kept trying to make it. So how can you make it different that like there’s so many varieties of burrito as well? I think, I don’t know if you guys have this, but we have this something called a deconstructed burrito, but you kind of making your own burrito, which is that a dial thing is in a bowl.

But I just wanted to burry though.

Max Burkhardt: Well, the, the thing that I often joke about here is I like to think that I have taste but not standards so I can identify when a Brita is really good. But even if it’s like a, , $2 frozen thing, I’ll still enjoy it. Yeah.

Ashish Rajan: Yeah. But this was, this was really awesome conversation, man.

Thanks so much for coming in. And where can people find you if they want to connect with you? What more. Yeah.

Max Burkhardt: I’m on Twitter at max B. And yeah, I think that’s, that’s probably the easiest way to get in touch with me. Our max speed RFM FM is my website. Awesome. All right.

Ashish Rajan: I’ll put those in insurance as well, but thanks so much for coming in.

Well now, thanks for coming in with our men for [00:39:00] everyone else. I will see you next weekend. It’d be, this was the last episode for the Clarksville friends month. We’re going to go into another month of you actually talking about serverless next month. So that’ll be good. That’ll be going to interesting.

Awesome. Thanks so much for coming in. Everyone. Talk to you soon. Peace.