Building Platforms in Highly Regulated Industries Explained

View Show Notes and Transcript

In this episode recorded at HashiConf Boston, Ashish spoke about secrets management, platform design, and scalability challenges in highly regulated industries with Barnaby Robertson Hurst, a Hashi Ambassador and consultant at GlobalLogic. Barnaby speaks about transitioning to centralized secret management, building scalable platforms for financial and public sectors, and mastering tools like Terraform and Nomad.

Questions asked:
00:00 Introduction
01:15 A bit about Barnaby
02:28 IaC in Public Sector and Regulated Industries
03:48 Why do people build platforms?
07:39 What is a platform team?
11:23 Should you build a platform before doing secrets management?
20:24 Where to start learning about Terraform?

Barnaby Robertson Hurst: [00:00:00] And then the sort of the final step of the way is you remove the agent, you rebuild the application code and the application will make the native call to vault. But it's quite a gradual process. Oh, so you don't switch directly from, you don't have to go straight from unmanaged secrets to everything's dynamic right on time.

You can gradually step through the various maturity stages and build it in slowly. And that's the way most people would recommend to do it. I don't even know how this is standing anymore. Oh.

Ashish Rajan: If you're trying to work with the UK government or tried building capability in a financial organization for DevSecOps, I got a chance to speak to Barnaby Hurst from GlobalLogic around what does it take to deploy DevSecOps capability like secret management across a regulated organization like a financial organization, for example, and overall, we spoke about the challenges you may face and how you can unlock that capability within your team and what some of the things you may require beyond just doing secret management, which probably is a good place to start if you're trying to do DevSecOps at scale in a large regulated organization.

[00:01:00] Overall, it was a great conversation. If you're watching this on YouTube or Linkedin you can support the work we do by subscribing or following us on the YouTube and LinkedIn of the world. Thank you to HashiCorp for inviting us to HashiConf and I hope you enjoy this episode. I'll talk to you soon. Thank you so much for getting on the show.

Could you share a bit about yourself and what do you do in that infrastructure as code space and what's your story like?

Barnaby Robertson Hurst: Yeah. So my name's Barnaby. I am a Hashi ambassador. So I release content about HashiCorp products and stuff. So stream Mondays for something called DevCast Ops. So go and explore that.

And then outside that, I work for GlobalLogic as a consultant, consulting on platform engineering, cyber security, infrastructure as code, DevOps, so they're all blending into one. So across that field.

Ashish Rajan: When we were talking before this, you spoke about your history with public sector and now primarily in the financial sector as well.

Having built platform or infrastructure or environment in some of them and all of them, what do you see as the difference for people. And I guess it's because we have [00:02:00] an audience, which is a mix of cybersecurity people, but platform engineers who are interested in cybersecurity as well. And some of them are in the regulated space.

Some of them are not. Financial and public sector, they come across as regulated industries. And a lot of people may not even be aware of what is the state of infrastructure sometimes in these environments. How would you say it's different creating infrastructure as code or even building a platform environment in a public sector versus building an environment in a financial institute?

Barnaby Robertson Hurst: Yeah, so they're both highly regulated sectors, and they're fairly similar in the things they care about. But the difference from building it in a firm that is not highly regulated is data ownership is a massive sort of concern because both in financial and public, so if you're working for government agency, the data you own is sensitive, whether it's personal identifiable information or people's financial records.

So often they'll have that [00:03:00] data barrier. You don't really want anything to leave. So it's building a platform in a way that data can't leave it. So often you'll see places that aren't really that keen on going into the cloud. Or if you go into the cloud, just be very sure where your data is.

There's a lot of banks are less certain about cloud and they'll have security and secret sides on prem. Yeah. And then they'll maybe run infrastructure in the cloud for applications that don't need to have all this data in them. Public sector, from my experience, they were happier to go to the cloud, but they'll often use government regulated clouds.

Fair. Generally work the same as the cloud that you and I would have access to. You can get access to the government regulated data centers if you want to. They're more expensive, so Of course, if you can afford it, yeah, fair.

Ashish Rajan: Would you say, is that why people end up going on the platform path? What makes people go down the, even in the regulated industry, and you've been working on building platforms, or at least helping people enable with [00:04:00] platforms.

What's the main goal behind people building platforms?

Barnaby Robertson Hurst: The really nice thing about a platform is you're trying to remove replication of work, is the goal from it. You don't want every application team to spin up a secrets manager or their own CICD pipeline or databases or anything like that.

So if you have a platform team, you have a team that's responsible for building those products or designing the patterns to deploy those products. They will work on those. They can bake in security to the patterns when they're defining them. And then they can deploy those out to the different application teams or the application teams will just come to the platform and consume them.

So it saves the application team time because often they won't have someone in their team who is necessarily a security specialist or a networking specialist or something. There's lots of bits that goes into a platform.

Ashish Rajan: Ah, yeah.

Barnaby Robertson Hurst: So it means they can focus more on the development of their application and then sort Armon talked a bit about it this morning in the keynote, the plumbing is abstracted away [00:05:00] from them, and they can just focus on the bits that are important to them.

Ashish Rajan: Interesting, yeah, and talking about building and breaking platforms, I've got Jenga in front of me as well. And you've informed me of the new rules. This looks like it should be loose. Damn, that is

Barnaby Robertson Hurst: I've seen people, they've gone for something that's really like one of the bottom ones at the lower, at the start.

Oh, really? Yeah. And it falls over. Yeah, the way it's like an unstable table and that's just the end of it. I feel like it's actually You can go for a middle or the other side. Yeah,

Ashish Rajan: I

Barnaby Robertson Hurst: think I feel like I

Ashish Rajan: should Oh, jeez. Damn, this is definitely not my best game. Oh my. Take

Barnaby Robertson Hurst: all the bricks out at once. Does it still count as a go?

Ashish Rajan: Yeah. What if I just do this? Oh, there you go. All right. Sorry, I moved it for you. But I made it a bit challenging for you.

Barnaby Robertson Hurst: Yeah. I've got a nice UF tower.

Ashish Rajan: Yeah. It's like already a Leaning Tower of Pisa by now. See, Barnaby knows what the trick is. He's trying to feel which is the middle one. That is just.

Yeah. If

Barnaby Robertson Hurst: I [00:06:00] push it and nothing happens, I'm probably not gonna get it out.

Ashish Rajan: So if you just keep pushing, poking around

Barnaby Robertson Hurst: Yeah, you can feel which ones, depending on how, strict you are playing it, some people frown upon Oh, really? They frown upon this?

Ashish Rajan: It's quite flexible, man.

We're, like, platform team over here. Building a platform which is, at the moment, it's just patching coming out

Barnaby Robertson Hurst: over here. This is, Yeah, we put something in that's not quite working at the moment. Yeah. It probably needs to be redesigned. Yeah, I think There you go.

Oh, thank you.

Otherwise it gets real messy when we eventually get back up to these layers. Oh, okay.

Ashish Rajan: Clearly someone's experienced, man. You're like fat fingers nowadays. That's what makes you realize you have fat fingers. What does that mean? I don't really know. It's a new rule. We just have two. Stack them both. Oh, dude.

You're [00:07:00] brave. Awesome. So now that we have been smashing through this platform, which is going to go above a certain layer after a point, we were talking about the financial institute having their own unique challenges with platform and making sense that, hey, you know what?

It makes sense for have a platform and you have lots of application teams. You have people who are trying to build on the concept of standardizing, for lack of a better word, what infrastructure as code is like, what vault or secret management is like. Do you find that in terms of building it for a financial organization is what kind of skill set, because I imagine not every financial institute is that friendly with moving to cloud and that friendly with doing infrastructure score.

What is a skill set they should think about in their team? If they were to do this themselves now, it can be challenging in whatever way I'm sure has their own unique flavor to it. Yeah, what are some of the unique or what are some of the skills that people look at when they're trying to build their own cloud? I don't want to say cloud platform, right platform team. See, how do you define platform team? Because I feel like [00:08:00] everyone has a different definition of it. How do you define it?

Barnaby Robertson Hurst: Yeah, it's really difficult. What we call a platform team is essentially the people responsible for building centralised infrastructure solutions.

The board infrastructure can get very vague here because it will often encompass like monitoring and alerting and all of the systems that

Ashish Rajan: And the on premise cloud and all of that as well.

Barnaby Robertson Hurst: But it's essentially, a platform team is there to support developers, so they're building a platform that developers can deploy their applications on to whether those be financial applications, machine learning, AI based applications.

If you build your platform you just have one centralized platform that can handle all of these workloads. What skill sets you need to build that will vary depending on what kind of applications you're running on that. So that will influence certain things. So if you're looking at running AI, you probably want to bring some data scientists into your platform team because they understand the data science workflows.

It was having sorts of your basic [00:09:00] infrastructure guys are really useful, especially if you don't necessarily want to transition to the cloud and you're going to continue to use resources that you have on premise. Their knowledge of what you already have is helpful into integrating the cloud into your on premise solutions because it's rare that you're going to be in a situation where you're going to go, I'm going to build a platform.

And you don't need to interact with anything that you've built before. Not everything moves as often, you start building a platform, application teams migrate. But they might not even migrate the whole application at once. So they'll talk back. Having the sort of core DevOps skills is also really important for that.

Because a loss of the platform is what would traditionally be DevOps. Whoops, as it has before, and so it'd be your CI pipelines, writing infrastructure as code and kind of knitting that all together. Something that is also really important when you start building your platform is think about how big it could possibly get if everything in your organization [00:10:00] migrated, and if that is very large, look at building it in a way that's scalable, because there's a lot of times you'll find you go, Oh, I'll build this.

And it works well for the first few application teams. You're not putting much load through the system. There isn't much going on. So you'll have either manual steps or things that are hardcoded here and there. But when you're trying to scale that to maybe hundreds or even thousands of application teams, it's all different, like microservices, those things that are hardcoded or manual, it doesn't scale and you need to work out how you're going to sort of build in sort of structures in it that everyone can have their unique area, and they're not going to start overlapping bits like that. So it doesn't necessarily take that much longer at the beginning to sit down and think actually this sort of area, this is going to need to be repeatable and variable eyes and trying to get it so as much of it can be variable eyes and it can split in.

So then when you later down the road and you have several application teams already on board and you're helping [00:11:00] support them, you don't need to re architect your platform while people are already on it cause its a lot easier to do the work before anyone's on it because you can play around, you can tear things down, rebuild them, and it's not going to affect anything.

If something's in production, you can't really tear down your platform.

Ashish Rajan: No, you can't. I love the fact that you are asking people to plan ahead before going in and blending or not planning ahead for thousands of applications you're going to build. I think recently you said you were working on the secret management side of things.

Is that an easier part to start? Or people should think of building a platform before they go down the secret management part?

Barnaby Robertson Hurst: Secret management, you want to bring it, you'll bring it into your platform. It'll naturally happen as you build. You'll have secrets that you handle in the build process. Whether it's credentials, TLS certificates for application communication.

You'll very quickly hit the point where you have secrets. Yep, okay. Bringing a secret manager in at an early point is quite a good idea because then you have a centralized place for secrets. You can potentially build your secret manager before your platform, so if you [00:12:00] start migrating things to Vault, for example, beforehand, you can get application teams used to using Vault for their secrets.

Yep. And then, when you build out natively supported as part of the platform. Ah yep. Secrets are also often easier to transition So there's a lot of focus that's been put on trying to make it as seamless as possible because companies that are doing secrets management tools know that people handling secrets is a very important part of their business.

Often if though what's being passed around, the data in those is lost, it can cause outages that sometimes can't necessarily be recovered from. If you lose root credentials, it's an important system, you have to start again. So there's a lot of security.

Ashish Rajan: Would you say what Like, with the secret management part, yeah, so there seems like there's flexibility to start either before you build a platform team, start with secret management, then move to a platform, depending on, obviously, every company may have different business units that may be at different stages, so some applications may be more ready to jump [00:13:00] on that before anyone else.

If you are looking to deploy secret management as a standard, is that a, I think that what I'm looking for is. A lot of people don't know what's involved in it. And I guess, I imagine there's a lot more find the right application, because you might have some legacy application. What's a good starting point to even start having this conversation about?

Barnaby Robertson Hurst: When you bring the Secret Manager in and you've got your access to it set up and segmented so people can't access things they shouldn't do you can then replace things that are already in place. So if you have an application that's reading just a VARS file off a server, You can often deploy something like a vault agent to that server, which will then template out that VARs file for you.

Okay. The application still has the same access to secrets, but instead of them just being stored on the server, they can now be centrally managed. And, you can then start looking at moving to dynamic credentials, if it supports it, and still use the same end way of consuming it.

You'll just have your vault agent instead of calling the static secret and vault, it can then call a dynamic endpoint and inject those into the [00:14:00] file for your application. And then the sort of the final step, the way is you remove the agent and you read the application code and the application will make the native call to vault, but it's quite a gradual process.

Directly from, you don't have to go straight from, Unmanaged secrets to everything's dynamic right on time. You can gradually step through the various maturity stages and build it in slowly. And that's the way most people would recommend to do it because you don't want to do all the change all at once because you lose track of what's going on.

So if you have that little stepping through you, it makes it easier to adopt. If you feel confident that you can just go straight to dynamic,

Ashish Rajan: yeah,

Barnaby Robertson Hurst: go for it. But if you're unsure about, all these things that are going on in the background, the bits you need to set up to get dynamic working, you're not sure if you can trust it, you can just move your current static credentials to a centralised management, and have the same credentials that are just pulled from elsewhere, and then when you feel more comfortable, you can then upgrade to the sort of more secure, [00:15:00] shorter lived credentials, limiting that sort of attack service later on.

Ashish Rajan: People have application onboarding kind of teams as well when they're going down like a big project, whether it's digital transformation or building platforms or building a secret management platform as well. Are there applications that are easier to start on that secret management journey, whether they are on Kubernetes or with their microservices.

I don't know if there are nuances to it, and I know I'm throwing a curveball at you on this one. But does that make a difference in terms of picking what's the right one to start with first? Instead of, to your point, not every application will be ready. They might have hard coded static credentials and mainframe has four digit passwords,

Barnaby Robertson Hurst: yeah. It's generally, it'll be what you use the most and what you're most comfortable with. So you'll understand that technology the best. So you'll feel the most comfortable importing something new into it. Stuff like Kubernetes, if you're very You mature in your kubernetes life cycle, you probably will have some kind of secrets in kubernetes already, which will just be potentially just in the cluster.

[00:16:00] Yeah, they can very easily be replaced by some injectors that just inject secrets into the cluster for you. So that's a fairly low effort change for you just inject the secrets from externally and come in. But it there's the easy level of all of them. And the bit when it gets complex is when you're starting to thread multiple systems together and tie it all in.

Ashish Rajan: And if they all share secrets, which needs to be, managed to a point, have the right access, so no wrong person gets access to it, but at the same time the system that must be able to access it, should be able to access it seamlessly.

Barnaby Robertson Hurst: Yeah, the tricky one is, yeah, as you said, when you have shared secrets, and often it's a visibility thing.

If you don't know where your secrets are being used, it's very difficult to go to rotating the secrets because if you don't know all the places that needs to be rotated and you miss one something suddenly stops Working. There's tools that are coming out to do that. There's various scanning tools coming out I think Hashi's version [00:17:00] is Vault Radar.

Okay, they will scan your sort of estate. It'll find secrets and get repositories and stuff like that and it can tell you about them so that I'll give you visibility and then that helps with the adoption because you then have a better visibility of where things are.

Ashish Rajan: Yeah.

Barnaby Robertson Hurst: So it gives you more confidence to go and replace them.

If you're in a highly regulated sort of industry, you may have issues with this because it is scanned externally. All the reports go to HTTP cloud. You can run an agent with inside your own environment. So the agent does the scanning, but the metrics are still reported through the cloud platform, which I know is a deal breaker for certain people.

Because even though there's nothing sensitive leaving the environment, it's still data leaving the environment.

Ashish Rajan: Yeah. And to your point, it goes back to data ownership. Where is the ownership? Where is the data going? Clearly, my producer won't be able to just lose this game straight away. That's why she's on to me.

Oh, there you go. I love the trick that you just gave me, which is I just [00:18:00] flick it once just to see if it actually moves. Just give me a little

Barnaby Robertson Hurst: tap if it doesn't move.

Ashish Rajan: Yeah. I love that. Oh I didn't want to touch it. Am I the one who's breaking it for Barnaby? Oh, not that one. Yeah, some of

Barnaby Robertson Hurst: them you feel and they're just completely static.

Yeah, it's oh, there you go. It's funny, some of them look like they shouldn't move.

Ashish Rajan: Yeah, but they somehow are moving as well. Hitting the table is not a good way to do that. Oh wait, would that, if hitting the table and it fell off, would that have meant that? I would have

Barnaby Robertson Hurst: counted that as a loss to me if I punched the table and knocked over the tower.

That's purely my own fault for just being clumsy at that point. Yeah, fair. Oh. I can't even see where that was going. We're really going for the bottom, so we want it to be unstable. Yeah. That was going to definitely go off. Oh. Oh, everything's Oh, that one's nice and loose. Yeah. [00:19:00] So keen. Yeah, I'm like,

Ashish Rajan: this needs to be moved faster because the

Barnaby Robertson Hurst: producer is go.

I'm like, okay. Those ones I want to take out in my head, but they're just not gonna go. Oh. Go for a middle one, mix it up.

Ashish Rajan: I said, why don't you have good long fingers so I can just poke it all the way through. Just

Barnaby Robertson Hurst: cheat and get a chopstick and just push it out the other side. There you go, that should come out.

Ashish Rajan: Oh, thank you. It's going to fall. It's going to fall. Oh, I don't even know how this is standing anymore.

That was lucky, man. Ah, geez, this is like we're not far off. Yeah, definitely feels like we are really close to the foreseeable end. See

Barnaby Robertson Hurst: Oh, one of our made layers. We've got high enough that we can take from what we built. Oh, actually that's a good point because after a point, yeah, so these layers are untouchable, [00:20:00] but from here down, you're golden.

Ashish Rajan: Oh, okay. Oh, yeah.

Barnaby Robertson Hurst: So this can go on forever as well. Eventually you get to a point where it's just one, one brick on each layer and then a stack of three at the top. So you do have a limitation. Don't worry, we don't need, we don't have a

Ashish Rajan: need for a question. Oh, maybe we do have a need for a question.

This is going to be unstable for sure. Now that we are making this more unstable, for people who are on the engineering side, and who are looking at Terraform and probably being terrified by it as well, what's a good place to start learning about Terraform?

Barnaby Robertson Hurst: So there's lots of tutorials on HashiCorp's website, which are very much aimed as entry points. You can go in, you can start messing around. It essentially gives you the code that you need to run. And a lot of them will focus around free tooling that you can run locally. So if you have Docker, you can run it. It would deploy Docker containers for you just so you get used to the syntax, [00:21:00] which is nice.

And then going forwards, you can expand out and start going into. The AWS and the cloud environments and start deploying there. There are official courses if people are looking to do more structured training. So Hashi do courses where you can go learn about the various tools and have sit in a couple of days There's other online resources like udemy classes and well as well.

It depends how much you're happy to sit in and send the time yourself teaching it and getting hands on and playing with that way or would you prefer to go into more structured learning where someone explains to you and you maybe spend a couple of days with an instructor and they'll go through labs and stuff with you so if it's something that you like we want to start deploying this really soon.

Yeah. And you haven't had any sort of hands on experience, maybe structured courses would be better for you because you'll learn faster with someone who's a qualified instructor. Yeah. Talking you through the labs and you can ask them questions if you have a bit more time and have played around with various different texts [00:22:00] before, you can just teach yourself from the docs, which are pretty well written and the tutorials online that talk you through various things.

Another thing that's probably worth mentioning here, if you're coming more from an application developer side of things, into Terraform and don't necessarily want to learn HCL, they have something called Terraform CDK, which you can essentially write Terraform using regular programming languages. So I think they support Java, TypeScript, Python, Go, and there's one other which I can't remember.

Maybe, I can't remember off the top of my head, the docs tell you exactly which ones, but then you can essentially define Terraform resources using standard programming terms, so if you're more used to that and don't want to learn language, you can come in that way as well. So that's the lowest bar of entry for a dev.

Ashish Rajan: What would you say is a good example? So HCL for context is the template that defines what would be created?

Barnaby Robertson Hurst: Yeah, so HashiCorp programming language, It's an extension of JSON, so you can do some functional things on [00:23:00] top of JSON to compute values, so you have four loops in there, you can do if statements, so you can do some things that are programmatic.

When you start off with Terraform, you'll probably do simplistic things and everything will be like a Hello World. Yeah. Be defined in your code. You won't have anything that's dynamic. It'll be static config. So you deploy and start off learning the structures of the objects like that.

Yep. And then you can move to spinning up more dynamic resources, starting verbalizing things, and it will naturally evolve.

Ashish Rajan: Oh. So what would be like a, I guess what's a Hello World equivalent Terraform world.

Barnaby Robertson Hurst: I guess the one that we would traditionally do is so you spin up a VM and a cloud is often what will give someone that's new to like here's Terraform, spin up a EC2 box and then you can log on to the box and see the thing that didn't exist without actually logging into the box, but from Terraform itself, you can't log onto the box, your Terraform, depending on how fancy you want to go, you can set it up.

So it automatically register the host and boundary. Yeah, and [00:24:00] then you can use boundary to connect to the box. It works, but for a more basic Hello World, what you probably do is you can give Terraform an SSH key and that will dump that into the box for you as part of the provisioning and then you can use your private key to log on to the box just with regular SSH if you open it up to the internet and as this is just a Hello World application, there's nothing sensitive running on it, you should really deploy it in its own kind of cloud account that's not your account with all your resources that are running on it.

Get a dev cloud account. I think AWS does like free tiers. Yeah. Then they give you some credits when you start and most of the other cloud vendors do. So if you haven't touched it before, they give you, I think it's about $150, which if you're sensible is plenty to play around with. Just don't spin up giant boxes and forget about them.

Don't start building AI models. No, but if you're doing it through infrastructure and code, the nice side of it is it's all tracked in your infrastructure as code. So when you're done playing with it, you run Terra from destroy. Yeah. And it will destroy everything that you've created. So [00:25:00] you have the confidence that you don't have things left around because a lot of the times when, especially I found it particularly when I was learning AWS.

It's very easy to have things left when you're trying to delete things, because you go, I want this. And it quietly in the background creates a load of other stuff to support that resource. And you go, okay, delete this. It deletes what you tell it to delete. It leaves the things that you quietly created in the background.

So you'll have a few things, you might have a public IP address that is costing you a few cents every month. But if you have loads of those, it starts to add up and you're like, why am I being charged? Five bucks. Just for an AWS account that

I don't do anything with it.

Ashish Rajan: So to your point, the Hello World was to build an infrastructure and try and log in into it.

The next stage, would you say, is more, I'm thinking, people who want to go down the platform engineering path.

Barnaby Robertson Hurst: You want to start building modules at that point. Reusable code. Your Hello World, you'll define everything in your Hello World files. Yeah. And that will be All it needs, but that's not particularly [00:26:00] useful if you give it to someone else.

If you give someone else the exact same thing and your Hello World is deployed, they'll go to deploy and they'll go, this already exists. So you want to start taking out various bits that need to be unique, or you'd want to potentially change when you do it. So that might be the size of the instance, the network groups that it's in and things like that.

And you put those into variables. And if you have a standard stack that you're deploying, that maybe needs two EC2 boxes, and then maybe a database and some. Certificates from ACM or something like that. You start building these resources into a module, take out the bits that need to be defined when creation, and then they're in variables, and then you'll often have outputs of the information that you want from the resources that are created, so if we are still logging into those boxes, you want the IPs, you need to be able to route to them.

Ashish Rajan: Yeah.

Barnaby Robertson Hurst: And then you have packaged this up neatly into a module, and then you can consume that by calling the address to your module, And you can publish that to GitHub and pull it from Git, or you can [00:27:00] internal registry, or you can even publish to

Ashish Rajan: Oh, you can go all the way to your point., almost like the third layer would be the fact that you're just pushing a code into GitHub.

It triggers the Terraform. And that Terraform is the one creating a Hello World with the module that you may have defined.

Barnaby Robertson Hurst: Yeah, so we've gone really deep into this and some of the stuff that I do on the streaming and the rebuild out. So we're essentially trying to manage a lot of stuff through Terraform and our Terraform itself is managed by Terraform.

Oh. So we want to get to the point where we have a button and or like a script and we just go run and everything. You have nothing but like maybe some cloud credentials and some Terraform cloud credentials. Yeah, you hit run, and then you'll get a whole working platform at the end of it. Oh and it will get there, but what this does it will trigger a module that will build in Terraform cloud.

It creates workspaces, and then it's got run triggers, so it's gone. Okay. I've created these workspaces it then triggers those workspaces will then deploy. And the way it's linked up is we have various layers of our infrastructure and each one [00:28:00] consumes a bit of the previous layer. So you run the Terraform module that builds that.

And then it goes, I built and updated this and it fires a little web hooks to the next stage and goes, Hey, the infrastructure that you were using, maybe you've updated or has now been created, go and run your plan again, and it will take the next module, take the outputs that it needed from the previous one.

Plum those in as the variables and then start building it. You can chain through, you can jump out of Terraform and back in as well, if you want to use webhooks. And I think they're looking at using stacks now to help define this. So it's slightly cleaner than all these little run hooks that are behind the scene.

Ashish Rajan: So is this the DevOps playground you were talking about?

Barnaby Robertson Hurst: So DevOps playground, we do talks like this, also streaming through DevCastOps. So we are building our own platform, which we're trying to do for as cheap as possible, because it's our own money. Of course. And we don't want to spend loads, but we're using that, we essentially on Mondays we stream, in the evenings, us building [00:29:00] the platform there.

And then we were like, we're building a platform, we've got to do something with it. So we're like, we'll spin up game servers on it. We have, I think we've done Minecraft, we're doing Factorio at the moment, and so we'll spin those up as jobs, containerized jobs in Nomad. And then connect them through a VPN that we've got configured.

We're looking to maybe move to something like console.

Ashish Rajan: What are the components behind the scene then? Amazon account.

Barnaby Robertson Hurst: So we use GCP for this just because none of us had used GCP. And we're like, we're doing something to teach people. It'd be nice if we learn along the way. So we picked a new cloud account and went with that.

But then behind that we have GitHub. I think we use for this one Terraform cloud. And then the, we use Terraform to provision our infrastructure. We then build our images with Packer. And the config for that is in Ansible Collections, which we've made public. So it pulls those Ansible Collections in, builds those, and then we deploy and we get a Nomad server, which is also a [00:30:00] client in our case, to try and reduce costs.

If you were doing this at a slightly bigger scale, you'd have the client separately. And then within Nomad, we have a job that will Take or deploy additional Nomad nodes. So we go, we want a game server. We go to Nomad, we hit spin up a client. Yeah. It spins up a client, it tags it with the relevant tags.

Nomad then goes, oh, there's a server that's tagged Factorio. I have the Factorio job here that's looking for that node point. It just deploys the job, mounts a disk to that client with the Factorio save data on it. And just spins up the server. So it's seamless when it works and it sends a ping of the IP address to our discord channel so we can then log into the server.

Oh, eventually we want to get it so it automatically updates DNS and then we would just go to Factoria at devcastops. com and that would boot us into the game. That's pretty awesome. And it's, yeah. [00:31:00] It's a pretty cool project as well. be seamless behind the scenes. Yeah, it's nice. It's fun. And we're playing with it and experimenting with stuff.

And we're setting up things with console to get it to auto configure. And it's just a place to play. But it's interesting because it's now got to the stage where it's bigger than the tutorials. So we're starting to run into sort of more real world issues of We want to do TLS, but this TLS doesn't talk to that TLS, and how do we get this to trust?

Ah, fair. It's interesting to see, because the tutorials are good, and they're great to get started, but after a while, you if you start doing too many of them, you'll, one will want one configuration, one will want another, and it's trying to work out how to merge all of that together.

Ashish Rajan: We'll come back to the financial institution with hundreds of applications at that point, and now we are the same circle as them.

Where you have a lot of applications, each requiring its own unique set of configuration that you have to manage and work with.

Barnaby Robertson Hurst: Yeah, it doesn't take that long to get there. Yeah, wow.

Ashish Rajan: Are you guys scaling it across multiple Google [00:32:00] projects as well?

Barnaby Robertson Hurst: We don't have it that big at the moment. In theory, we could scale it to go across multiple Google projects and have different areas.

The way we differentiate is we have different node pools which are used for different jobs. We could use different namespaces in Nomad to segment off different departments. So you start to get that sort of segregation of concerns and boxing in applications to their own space so they can't reach out and accidentally talk to somebody else.

Oh, awesome.

Ashish Rajan: That was most of the technical questions I had, so I've finished the game now. Finished anger. Yeah, and I'm going to try and not rock the table, but I have to feel, All of his feeling is going to fall, you might as well grab your glass then. Just going to put this on the floor so we don't smash it accidentally.

This is definitely, top three are a sacred. Oh, there you go. Found a soft one. All right.

Barnaby Robertson Hurst: I feel like this layer, there's still three, one of these.

Ashish Rajan: We're going to be here for the afternoon, basically. [00:33:00]

Barnaby Robertson Hurst: Ah,

Ashish Rajan: okay.

Barnaby Robertson Hurst: I don't know. I feel like we haven't got many blocks left that we can get. Oh, yeah.

Ashish Rajan: Oh, good job, man. Good job. Can I pull this one out as well? I'll just try that. I feel like that's stable enough. If I do that one move, I'll just do this like that.

Barnaby Robertson Hurst: I feel like when you're going for one that's already been outer layer, you're Like you're doing the hard job of putting it outer layer.

Oh, is this going to go?

Ashish Rajan: Aw, okay. That was a strong defeat. Played. I appreciate you coming on and letting me win this one. Because the other one was really bad. Where can people find you on the internet, man? I can get to know about the playground you're playing with as well.

Barnaby Robertson Hurst: Yeah the playground is a meetup, so that is It's through meetup. com. That's UK based. We do stream online. I think it's through GlobalLogic's YouTube channel.

Ashish Rajan: Sure.

Barnaby Robertson Hurst: Those are last Thursday of every month. And if you sign up for Meetup, you'll get a lab account where you can play with the infrastructure that we're talking about. So [00:34:00] if we're doing a talk on Terraform, you will get your own little web IDE and you'll be able to spin up some Terraform resources.

The other content that I create is on Monday evenings, UK time, we stream for DevCastOps, that's on Twitch. Yeah. The series has been pushed to YouTube and it's under the same name, DevCastOps. And then on Saturdays we do our gaming streams, which are a little bit more informal, but if anyone ever wants to come along and chat about, technical stuff while we're playing games.

They're more than welcome to come along.

Ashish Rajan: When do you not stream? You're like Monday, Friday, Thursday,

Barnaby Robertson Hurst: Monday and Saturday are our regular ones, and then the DevOps Playground is only once a month, so that's Thursday. Wow,

Ashish Rajan: it's streaming on a weekly basis, man. That is that is one commitment, but I appreciate that.

I'll put that in the show notes as well, but I appreciate you coming on. Thank you so much for coming on the show, man. Thank you. Thanks everyone. See you next time.

Thank you for listening or watching this episode of Cloud Security Podcast. We have been running for the past five years, so I'm sure we haven't covered everything cloud security yet.

And if there's a particular cloud security topic that we can cover for you in an interview format on Cloud [00:35:00] Security Podcast, or make a training video on tutorials on Cloud Security Bootcamp, definitely reach out to us on info at cloudsecuritypodcast. tv. By the way, if you're interested in AI and cybersecurity, as many cybersecurity leaders are, you might be interested in our sister podcast, which I run with former CSO of Robinhood, Caleb Sima, where we talk about everything AI and cybersecurity.

How can organizations deal with cybersecurity on AI systems, AI platforms, whatever AI has to bring next as an evolution of ChatGPT, and everything else continues. If you have any other suggestions, definitely drop them on info at Cloud Security Podcasts or TV. I'll drop them in the description and the show notes as well.

So you can reach out to us easily. Otherwise, I will see you in the next episode. Peace.

No items found.