James Clough: [00:00:00] I think another thing though that goes even beyond what technology you're using is at the end of the day, a lot of customers, particularly in regulated environments, it doesn't matter kind of where you're processing the data. They still want to know the answer to a simple question, which is, are you training models based on my data?

It probably costs, I mean, all of these things probably cost a little bit more than if you did it all yourself or hosted it all yourself. But as a startup, you're going to be limited by your sort of your focus and your engineering capacity. And so it's worth paying 10 percent more or whatever it is 50 percent more maybe in order to not have to reinvent the wheel and kind of rebuild MLOps from scratch.

Ashish Rajan: Hello, welcome to CloudFugitive podcast. Today I have James. Welcome James to the show. Yeah, thanks for having me. Great to be here. If you can start with a bit about introduction, what got you into the space, where you're in today, where you are at at the moment as well.

James Clough: Yeah, sure. So yeah, my name's James Cluff.

I'm the CTO and Co-founder of a company called Robin ai. We're a legal AI company, so we're building a LLM driven assistant to help with legal work and [00:01:00] in particular work relating to contracts.

Ashish Rajan: Oh,

James Clough: nice. So my sort of journey into Robin, I started out. Uh, as a physicist, actually a theoretical physicist, um, really enjoyed that work.

It was, you know, lots of smart people solving lots of hard problems, but I was finding that, you know, as sort of indicated actually by the Nobel Prize recently for physics, going to someone in AI, Geoff Hinton, it is a space where there weren't so many problems that were really high impact and that were sort of tractable at the same time.

And I was getting more and more interested in AI where it felt at the time, sort of, And even more so now that there were these really high impact problems that were becoming solvable as the technology was getting better and better. And so I sort of transitioned from physics into AI. Quite a lot of people have done the same thing in the AI sector now and then moved from from that sort of physics academic work to AI.

to working in AI, did a post doc that was applying machine learning to medical image analysis, so lots of computer vision problems there, and then co founded Robin, so moving from the vision space into the NLP space, almost [00:02:00] five years ago now, so end of 2019.

Ashish Rajan: What makes people from a physics background move to AI?

I would have thought, there's

James Clough: no

Ashish Rajan: correlation, or is there a correlation?

James Clough: I think there's a very strong correlation. Lots and lots of AI people, I think you'll find, uh, ended up studying physics often early on, and it's, you I think there's a lot of similar maths there, it's just a lot of linear algebra there, so you learn a lot of the same tools, and it's a A similar sort of type of person I think ends up attracted, uh, into the space.

So, uh, yeah, I think a lot of, a lot of AI leaders now probably were doing physics in their undergrad at one point.

Ashish Rajan: Just to double in on the legal space. Before the whole AI world, what was being done by, I don't want to say traditional assets, traditional sounds like a long word. What were the current legal companies that had not adopted AI?

What were they doing? Yeah, I

James Clough: mean, there was a lot of legal tech before AI. I think Really say it was sort of traditional B to B sass. So I think the biggest category was this thing called CLM, which sounds for contract life cycle management. So it's kind of like the equivalent of a CRM, things like sales [00:03:00] force.

But instead of managing all of your customers is managing all of your contracts. So there were these big databases that big companies would use toe manage. You know how many contracts have we signed with this person? When do they expire? When do they renew? You could build these work flows. You know, if The contract's worth more than X, it has to be signed off by the director of finance or whatever.

So that's mostly what legal tech in the contracting space was. There are some big companies in that space for sure, but it had a bit of a reputation for being quite clunky, sort of hard to implement, just sort of happening around the time as people were transitioning to the cloud, so a lot of them were still on prem as well.

Is that world now moving towards more API driven then? It's an interesting space where there's some Of those companies who are transitioning to being more AI first, right, trying to get AI into their sort of old non AI software. And there's also, I think, a new generation of more AI first companies in that space starting to appear as well.[00:04:00]

Maybe too early to tell which of those stories is going to work best yet.

Ashish Rajan: Oh fair, and it could be Robin AI. The question that I have for you next then, I guess, if the whole CLM space, contract, Lifecycle management, I'm sure I'll get used to the acronym by the end of this. Cyber security is a lot of acronyms.

So I'm pretty sure I'll get used to this. So the contract life cycle management space, uh, how does AI enable it? And yeah, I guess why that use case? Cause I feel like legal, I'm sure there's a lot more to legal than just the contract life cycle management. Why that particular space for AI?

James Clough: Yeah. I mean, I think there are lots of challenges that That previous generation of software had an AI helps solve some of them, but also introduces some new ones.

So what are those challenges? I mean, one sort of a really obvious one that lots of businesses contracts are just stored. I mean, on paper at first, right there printed out and then scanned in later. And so, you know, You have all of these scans in of like 20 year old documents and, you know, you've got to read [00:05:00] people's handwriting.

There's still a folder somewhere as well. Yeah, exactly, you know, half of them are scanned in upside down or whatever. They're in different languages, the quality is really bad. So you have to OCR in all of this. If you want to actually know what's in those documents, you know, sometimes they have like complicated tables in and stuff like that, which can be quite hard to read.

There's obviously lots of security concerns. So that introduces a whole collection of challenges and not just the kind of standard, this is confidential data. You need to keep it safe, but complications like there might be different parts of the business, like your customer's business that have kind of Chinese wall or firewall between them, right?

Where people from this team aren't allowed to read the data that is being This other team are using, but then there's the team who needs to see both of them and so complicated permissioning and that kind of thing that you need to be able to handle and the consequences of getting that wrong can be really high in the legal sector.

It's not just that someone seen something they shouldn't, but it might be really commercially important data that leads to someone. You know, getting in trouble for insider trading or something, right? So [00:06:00] there can be actually like criminal complications of getting that wrong and not just sort of those commercial ones But also they're just very discerning customers, right?

it's a space where You need to get things right and people are willing to spend lots of money in order to Go from something being like 99 accurate to 99. 9 accurate because that last percent Can really make a huge difference and what that means is that It's hard to sell software sort of really early on in the life cycle of your company and then just kind of Get better as you go.

You have to start from a really high sort of quality bar and in part, I think that's because just because of the nature of the work, but also in part there are there have been lots of people who have been burned a little bit before by buying really expensive legal tech software that kind of hasn't really worked for them, you know, took a really long time to implement and then, you know, They don't want the same thing to happen again, basically, but now AI has come along and has changed quite a lot of that.

So one thing that's changed to really help that transition is that it's a [00:07:00] lot easier to show the value of the software now with AI. It used to be the case is really old clunky on Prem. CLM software, you know, sometimes it would take a year or two years to implement it. So, and the thing would cost millions of dollars.

Yeah. So you can see why if you're buying that software, you're going to be really risk averse, right? You can't afford for that to go wrong. Yeah. To spend millions of dollars and only two years later when it's implemented, you find out it doesn't really work. Yeah. So, that makes people really, really risk averse and makes you, Um, really hesitant to buy stuff.

Whereas now, a lot of AI driven software in the legal sector, you can show value really, really quickly. And so you can do a proof of concept that proves to people that this is going to work in days or weeks instead of years. And that's one way you can kind of get over that risk aversion by delivering the value.

really fast, often quite cheaply because you can just start with one small problem and focus on that instead of trying to solve every single problem at the [00:08:00] same time. So that's a way that the AIs help. There are some ways in which it's made things a little bit harder. I think one big challenge is that that older generation of software isn't as fast.

Those CLMs. They're really sort of managing like the the workflows around the contract rather than the data inside the contract itself All right, okay And that was just because it was too hard right in the past if you wanted to really understand the words in a document Yeah, you know without AI the only thing that could do that was a person basically, right?

There wasn't really software that could Understand plain text that was written by someone else and do stuff with it. And that meant that there was a lot of that software could be very sort of security focused because it didn't need to read the text in the document, right? So you could encrypt the document with a key that only the client had, for example, and that was fine, right?

That that was a really secure way of doing things. But now you're saying, well, my software needs to read the document. It needs to know what the words in the document are so we can help you automate stuff, right? So really powerful functionality, but it means that you have to. [00:09:00] approach security a bit differently.

You need to be able to read the text and document and maybe send it to an LLM somewhere that's not on their on prem installation. And so a lot of that extra value that's really exciting comes with different security concerns as well.

Ashish Rajan: All right. So, but this is best maybe for people who are in that, uh, for lack of a better word, old school world of still copy about not scanning and.

Copy, copying, scanning and copying the data or from a physical paper onto a digitizing it kind of a thing as well. I think we, before we started recording, we were talking about the whole SageMaker and the bedrock of the world as well. So how are these enabling an organization like yourself, uh, to be able to be more, for lack of a better word, use Gen AI in that CLM space and what does that really look like?

James Clough: Yeah, so those are great technologies that we're using at Robin to help address some of those problems. So. So SageMaker is the AWS service that kind of lets you train and do inference on, um, machine learning models and Bedrock is a service that lets you kind of [00:10:00] access large language models like, like Anthropix, which we use sort of as if they're in an AWS managed service.

So the benefits of that are that. Looking at like Bedrock, for example, you know, we want to be able to use the best LLMs that there are best in terms of, you know, the biggest models with the best reasoning ability, but those are developed by a handful of companies right by OpenAI and Anthropic and Google and whoever, and most people access them just by hitting an API, right?

You send the text over the internet to some API. You wait a little bit and then you get some answer back. And our customers, pretty understandably, they don't really want the text in their contracts, this highly confidential, really commercially valuable text to be sent to an API that we don't control and they don't control over the internet in, in California or something, and then get this answer back later on.

What Bedrock lets us do is basically have the best of both worlds. We can have the most powerful LLMs available, which for us is is Claude 3. [00:11:00] 5. But instead of having to access that via API, we can basically access it from within our AWS ecosystem. So it makes it a really easy story to kind of explain to clients as well.

You know, we can give them a diagram that has an ice cream box with a little icon of a padlock in the corner. And we can say, look, this is our AWS environment, we control everything here and the LLM is inside that so instead of sending your data to the LLM Which is scary. We're bringing the model to where the data is, which is inside this kind of safe ecosystem

Ashish Rajan: So are we saying that if they were to use bedrock they could do the same thing But it's just too complicated from an engineering perspective

James Clough: Yeah, I think it's the the you need more than just the LLM core, right?

so if you just wanted to make an LLM API call then bedrock could be a great way for them to do that. But what we need to do is combine the the llm calls with all the other Functionality that we're building in our product that allows you to to automate more complicated legal work, right? So for example, you know [00:12:00] if we're you know Part of our product is a is an add in in microsoft word that helps you review your contracts and in order to do that you need to be able to You know, build these complicated workflows that lawyers have, for example, checking whether all the defined terms in the contract to use consistently.

So there's lots of domain specific logic there, kind of legal work logic, which we have to build. And some of that involves making calls to these LLMs, sending certain parts of the text, the document, or finding certain key clauses, and that's where those SageMaker models will be used, and then making decisions based on that.

So it's a combination of kind of bespoke logic that we're building in the software, but also having access to this, you know, LLM, um, as like a component of that.

Ashish Rajan: And what would you say are security considerations for SageMaker and Bedrock that people have to consider? And I guess, because I imagine people who are listening or watching this are two sides, I guess.

One, people who may be from the same space as you are, or someone from another industry who's looking at Bedrock and [00:13:00] SageMaker, and for whatever problem that they're trying to solve. What are some of the security things in those two that made you guys pick that over anything else out there?

James Clough: Yeah, so I think, I mean, things that were useful for us that, um, have helped make that decision.

I mean, lots of customers have requirements for, Data to be stored or processed in certain geographic regions, right? Whether it's regulatory reasons or just their own internal policies. So having a lot of options for where you can host the models is, is really valuable, right? It means that you can tell the customer, yes, your data lives inside the EU and it's only processed inside the EU as well, because we're using this bedrock instance based in Germany or whatever, right?

And instead of saying. Oh, we just send it over an API and it gets processed somewhere, but we don't really know by who, we don't really know where, which is what you'd otherwise have to do. So that's, that's a pretty important consideration. I, I think another thing though that goes even beyond what technology you're using is at the end of the day, a lot of customers, particularly in regulated environments, it doesn't matter kind of [00:14:00] where you're processing the data, they still want to know the answer to a simple question which is, are you training models based on my data, based on the text that we're processing We're sending you and that's a question that you just need to have a good answer to regardless of the technology that we're using.

So the approach we take at Robin is to, to not do that. Basically, we'll say we'll train predictive and classification models based on customers data, but not generative models that write new text. And the reason we made that distinction is basically that, you know, you ask the customer, what are you actually worried about here?

What they're worried about is that someone else uses this technology and then. The words from your contract appear on their screen. Yeah, that's basically what you don't want to happen. And that can only be that that can only happen if there's like a generative model that was trained on your words and then somebody else uses it.

So that's basically the big risk that you want to avoid. And so, you know, that means you want to be in control of the process of using the models and training the [00:15:00] models as much as you can. And kind of Bedrock helps us do that as well.

Ashish Rajan: The two that you mentioned, which is not the Gen AI part, which is, uh, so what are they?

Because I guess it's worthwhile calling out, not everyone would be familiar with those words as well. Could you just expand on that a bit?

James Clough: We sort of make a distinction between two types of machine learning models. And the first is what we would call predictive models or classification models. So these are models where The input is some text and then the output is a, a decision.

Maybe it's a yes, no, or it's a sort of classifier into a number of pre existing classes, right? So we might use that kind of model, for example, to find an important clause in a contract. So maybe you're looking for a clause called an indemnity, right? This is a really important clause in a contract. And to find that you would use a classification model.

You would look at every paragraph and you would say, is this an indemnity? No, is this an indemnity? No, is this an indemnity? No, is this an indemnity? Yes, right? So now that's a predictive model. It's predicting yes or no. Yeah, for every clause is it an indemnity? And then you might have a second step of the process where you would use a generative model.[00:16:00]

So a generative model is one where the input is text, but now the output is new text, right? So in this case, we might have the input be that indemnity clause we found. Yeah, and the new text might be What do we have to do as a business as a result of this? Right. Uh, so then text is going in and text is going out.

Yeah. And because in those models, the output is text, in principle, you could be worried that the text is going to contain confidential information. But that first set of models, the output was just yes or no, right? Yeah. So yes or no, that's not confidential, right? You already know what all the options are.

So it's kind of safer to train, Those first class of model is predictive models on confidential data because you can control the scope of the outputs whereas generative models There's more risk that you could produce something

Ashish Rajan: in terms of because you mentioned you mentioned using cloud Cloud 3. 5. So that's for the gen AI piece But the other one is your proprietary model that you guys have built

James Clough: That's exactly right.

Yeah. So those, you know, predictive models tend to be proprietary models that we've [00:17:00] built and trained on our own data sets. SageMaker is the tool that lets us, it's basically the infrastructure that we use to train and then deploy those models. Okay. So I think SageMaker is kind of analogous to lots of those other AWS managed services.

So you have, for example, in AWS, you have RDS, which is the relational database service. And that sort of, That's you just saying, look, the relational database setup I need isn't that bespoke, right? Lots of other people need databases. And so AWS have sorted all that out for me, right? I don't want to have to host it myself and do all the backups myself and whatever.

It's all sorted out for me. Oh, and SageMaker is a similar thing where you're saying if your ML ops requirements are, they can still be complicated. Yeah. But as long as they're not really bespoke and really unique, someone else has had this problem before. Yeah. And lots of AWS customers have had that problem before.

Mm-Hmm. . And so AWS have built this service that captures a lot of the things that you. You need to catch a lot of your requirements, and then it just means that you can [00:18:00] focus your engineers time not on rebuilding the stuff reinventing the wheel, you can say, Look, there's a service that already exists, has all the basic functionality we need.

It probably costs. I mean, all of these things probably cost a little bit more than if you did it all yourself or yourself, but as a startup, you're going to be limited by your sort of your focus and your engineering capacity. And so it's worth paying. 10 percent more or whatever it is, 50 percent more maybe, in order to not have to reinvent the wheel and kind of rebuild MLOps from scratch.

You're almost increasing efficiency by doing that as well. Exactly, exactly. You're making your business more efficient and you're trading off perhaps a slightly higher cloud services bill in exchange for just not having to worry about loads of stuff. And

Ashish Rajan: when, for people who probably don't. Or have not used SageMaker, um, would I be right in saying that if I want to build a model, what you said, a predictive model based on, yes, it's a yes, it is indemnation or not, [00:19:00] indemnity or not.

So for me to build that model, I'm sure the background components that may be required, there might be a database for me to store the data, it will train on the whole CI, CD pipeline that it might have to build on, all of that is taken care of by SageMaker.

James Clough: Yeah, exactly. There's, there's a kind of out of the box version of all of this stuff.

So if your requirements are not that unique, then you can say, great, I'll just, you know, have an S3 bucket where I put all of the training data and another one where I put the validation data or something like that, you know, you can take some off the shelf model architecture might not be the best you could get, but it'll be pretty good to get started with.

And then in, you know, a few hours or a day or something, you'll be able to get started rather than having to do all of that basic stuff yourself.

Ashish Rajan: Interesting. For people who are, because obviously we're at the Gen AI loft, there's a lot of Gen AI companies here. What is, and you're obviously leading the tech side, what is the team skill set required for building a Gen AI company?

I mean, because that's a big mystery as well for a lot of people. What happens? Are you, are you guys

James Clough: all physicists? [00:20:00] Yeah, I mean, there's lots of, there's a lot of cross functionality I think you need. So you still need the kind of traditional, I mean, traditional is still a pretty new job, but the traditional machine learning engineer and traditional data scientist, right?

I think people who. understand, um, how these models work and how to train them. I think a lot of that has got easier with this tooling, but if you want to debug a system like this, it really helps to kind of understand how it works in the first place. And so that sort of skillset, good engineers, you've got.

the knowledge of how models are built, how they work, how they're trained and what kind of failure modes that you might expect to see, that's really valuable. But at least for us, Robin, we also have a team who have got a background in the domain of the problems we're trying to solve, which is legal, right?

So we have team if we call them legal engineers who are sort of at the intersection of that legal work and the software as well. So they help us do things like build evaluation pipelines for these various tasks that the models are doing. You need someone with a bit of a legal background to be [00:21:00] able to assess.

You know, is this output good or bad? What kind of failure modes should we expect to see in these LLM driven outputs to certain bits of legal work? Yeah, um, they can help us, you know build data sets They can help us do a lot of the prompt engineering that's necessary as well. So that's a pretty new concept job.

You know, you have people starting to train as prompt engineers now, but I think for us, it's more than that. It's that intersection of prompt engineering, a bit of software engineering and R and D, but also deep domain expertise in the thing that you're trying to do your work in. And those people, you know, there aren't a million of them applying for jobs because yeah, it's, it's a new intersection of skills.

Um, but that's why for us, I think the trick is to, you know, maybe you find people who've got. 70 percent of the skills required, and then you go and give them the other 30%. Um, as they're, as they're doing the work. Because you're never going to find loads of people who have this intersection of brand new skills, right?

That's just impossible. But, you can [00:22:00] find people who are really smart and ambitious and want to skill up in those other, you know, whatever the missing section is for them. So,

Ashish Rajan: for people who are listening in, I guess, uh, you're a younger version of James, for lack of a better word, who wants to build a Gen AI or work in that space as well.

Do you actually have some thoughts on where can they even start being a legal software engineer kind of person?

James Clough: Yeah, it's a really good question. I mean I think one thing to say is that it's got a lot easier with Gen AI tooling to kind of hack around and build, build things for yourself, build little prototypes and build new projects because, you know, you can use ChatGBT or Claude or Copilot to help debug your, your project, right?

Help build your, help build your project. Yeah, they're, they're really, really good at writing code now. And if you want to just build a little. Widget or a little app to test some idea that that's a lot easier than it used to be. And so people who are interested in just sort of building something, then, you know, you don't need to have a whole team hired to help you do it.

You can build it yourself [00:23:00] with your LLM assistant a lot easier than you could before. So that's one thing if you want to build it. I think if you're interested in those sorts of new emerging roles in these kind of businesses, then I think the main thing to say is that you can. You know, I mean, obviously you can go and go and try and learn what you need to learn from the outside, but don't be worried that you don't have every single skill that's necessary to go and do those jobs, because nobody does, right?

It's impossible to have all of those skills at the same time. I mean, yeah, the field itself is

Ashish Rajan: what, I mean, I guess the ML field has been there for a long time, but the Gen AI has been kind of just like explorer of the last, what, two, three years? Exactly,

James Clough: right? Like there's, there's nobody out there who's got, you know, 10 years experience as a prompt engineer.

They're lying if they're saying. Exactly. They're lying if they say they do, but, but that's a good thing, right? Cause that means that you're only two years behind the world's most experienced prompt engineer. Yeah. You can catch up pretty quickly. Yeah. So, so go and go and start catching up.

Ashish Rajan: Oh, awesome. Uh, dude, uh, this is a really good conversation.

Thank you so much for coming in. Where can people find more about the work that Robin AI and you guys are doing?

James Clough: Yeah, there's [00:24:00] one place, which is our website. So www. robinai. com and you can see what we're doing, see the roles we're hiring for and go and try the product as well.

Ashish Rajan: Awesome. Thanks so much for coming

James Clough: in.

I'll leave that in the show notes

Ashish Rajan: as well, man. Thanks so much for coming in. No, thank

James Clough: you for

Ashish Rajan: having me. Thank you for listening or watching this episode of Cloud Security Podcast. We have been running for the past five years, so I'm sure we haven't covered everything cloud security yet. And if there's a particular cloud security topic that we can cover for you in an interview format on Cloud Security Podcast, or make a training video on tutorials on Cloud Security Bootcamp, definitely reach out to us on info at cloudsecuritypodcast.

tv. By the way, if you're interested in AI and cybersecurity, as many cybersecurity leaders are, you might be interested in our sister podcast Podcast, which I run with former CSO of Robinhood, Caleb Seamer, where we talk about everything AI and cybersecurity. How can organizations deal with cybersecurity on AI systems, AI platforms, whatever AI has to bring next as an evolution of chat, GPT, and everything else continues.

If you have any other suggestions, definitely drop them on info at cloudsecuritypodcast. tv. I'll drop that in the description and the show notes as well so [00:25:00] you can reach out to us easily. Otherwise, I will see you in the next episode. Peace.

‍