Hi, this is Hoseppin Bhartiya and today we have with us Arun Gupta, VP and GM of Open Ecosystem at Intel. Arun, it's great to have you on the show. I'm super excited to be here, Swapnil. Yeah, and today we're going to talk about Intel's, you know, donation or contribution of Open FL framework to Linux foundations, AI and data foundation. And of course, we will talk about Intel's open source strategy. I mean, I have been covering Intel and open source from a very early on, even when I was a journalist in India because Intel, though it's known as a hardware company, you folks do a lot of open source either way. So for me, I am not a newcomer to Intel's open source, but a lot of people will be. So I'll talk a bit about Intel open source, of course, Open FL.  Talk about your own journey and what led you to join Intel? Well, I've been always like an open source citizen, you know, for over two decades now. My history goes back to Sun Microsystems where we took the company, people process technologies out of the three key elements, you know, brought that cultural change at Sun Microsystems that many years ago, almost 20 years ago. And then I've always stayed on sort of the open source path. That's how the common thread across all my beads. So I went to Red Hat for a couple of years, couch base for a couple of years after that, spent Amazon a few years over there. And then my latest gig was Apple, where I started the open source program office over there. So when Greg reached out that around we want somebody, Greg Lavender over CTO reached out, that around we want somebody to build the open ecosystem team and say, Intel, and what do you do in open source? I'm not fully aware of it. And then when he started telling me the story, I said, wow, that is a beautiful gem. And why is it hiding? So that's the story we wanted to tell. And essentially that's how it came about to be. So if somebody asked me, yeah, the VPGM is sort of the formal title, but really the main title here is the chief storytelling officer. How do we tell the story? What is Intel doing in the open source? And why is it doing it? And how is it doing it? So that's sort of my main element here. And that's how I came about to be just about in a month, I will be about a year at Intel. And I'm super excited, super thrilled. I would say other than maybe Red Hat, this is the only company where everybody in my management chain truly understands the value and the premise of open ecosystem. Thanks for talking a bit about your journey and also highlighting the work Intel is doing. Actually, when you said that, you know, your job is to tell stories. I mean, I am also kind of a journalist, but I'm also an open source advocate. Even if folks know it's very important to repeat the message because sometimes repeating a message is as important. I remember my early days of journalism, it was back in 2005. We are telling people why you should use open source today. Almost everybody is using open source, but a lot of people don't even know companies don't know how to become a good open source citizen. So it's very important to repeat that message. And that's why I'm going to ask the next question, which maybe a lot of understood in the open source community, which is that talking a bit about Intel is technically seen as a hardware company. What does software mean for Intel? And why does it do open source? Well, Intel fosters an open ecosystem strategy to build trust, choice and ensure interoperability across the industry. I mean, yes, Intel is primarily a silicon company, but the way our customers consume that silicon is through software. And as they say, software enabled silicon, software powered silicon enabled. So that's the reason, you know, Intel has been the top corporate contributor to Linux kernel for over 15 years. That's the reason we are one of the top contributors to Kubernetes. That's the reason we are one of the top corporate contributor to open JDK. We are the number three contributor to PyTorch. So those are the projects how our customers consume Intel silicon. So making sure those projects are run in an optimum and efficient manner is super critical. We have over 19,000 engineers that contribute to software. A lot of it internal, but a lot of it external as well. So we have hundreds of community managed projects where we contribute. We have hundreds of our own projects where we contribute. We also play this open innovations. Also plays a very key role in the company success. These community led projects, not just benefit Intel, but it benefits everybody. And the more we more inclusive, the more diverse feedback we get on these projects, the more it benefits the community and us as well. We also look at like, you know, how do we champion the code of conduct? You know, such as contributor covenant set up by Github and CNCF and all. So those are all critical element. And I would say the last point that is super relevant for us is how we are contributing across every layer of the global technologies to act. And I would say in that sense, Intel is very uniquely positioned being a silicon company. We can tune down to the wafer level and all the way to the application that is running in your application. So within my team and across Intel, we talk about how we are doing silicon to server less messaging that is relevant for the developers. So we contribute to entire layer of the stack and making it much more impactful and relevant for the customers. Thanks for sharing, you know, this and you're right. Also, we kind of live in a software driven word where everything is software defined. Now let's talk about this project open federated learning or open FL a framework. Talk a bit about what is this project all about. And also if you can share what is the origin of this project. Yeah. So if you think about federated learning, let's understand the concept of federated learning, why that is required. Now, if you think about any global problem that we need to solve, let's say you want to build a model that is really required to do the cancer detection in brain, for example. And that's a real example that I'm talking about. So this is where Intel was working with Penn medicine and identifying when we initially built the model, that model was built with the three states, California, Massachusetts and New York. Now, this is a global problem. If you were to deploy that model in a rural part of India or somewhere in Africa or somewhere else in the world, how would that model work? That is the problem exactly that federated learning is solving. Now the challenge over there is because of the regulatory environment, HIPAA and you know, all these GDPR compliance, these hospitals cannot really share their data. They said, take my data and go build the model and then give me the data back. It doesn't work that way. So essentially what federated learning enables you to do is you have an aggregator and then you have these multiple collaborators. You send the model to the collaborator. The collaborator runs the model in their own compute. They generate the model and send it back. So the collaborator is sending the model to all the aggregator. Sorry. The aggregator is sending the model to all the collaborators. They run the model. They send it back and then the accuracy improves and then that model is then sent back to everybody. So the security and the privacy of data is super important. Very much stays with the collaborator, but the model is the one that is exchanged back and forth.  But how do we make sure that is data is fully protected? So that's the whole premise of how Intel was really working with University of Pennsylvania, VMware, and flower labs on how do we build this? And that is the project that was created out of Intel labs and then eventually contributed to LFAI and data foundation.  Talk a bit about what it will look like. There is it going to be a project within LFAI data foundation? What will be the structure also? There are no open source community. There are open source communities. So we'll also talk about what kind of community you're looking around open, you know, FL. Well, first of all, it's a project as part of LFAI and data and there is a GitHub repo. So if you go to github.com slash secure federated AI, there is a open FL GitHub repo over there. And it truly follows all the governance model that is proposed by LFAI and data. There is a contributing.md. There is a governance.md and all of those standard things that you would expect from an open source project. So today you may be just a consumer of the open FL project, but tomorrow if this becomes critical for your organization in a very classical meritocracy based way, you may want to rise up to be a maintainer of the project. And boom, you know, you can just look at that governance.md rise up, become a maintainer of the project. And honestly, this is how when I was talking to the open FL team, I understood VMware got involved because, you know, if you think about generative AI, those models, large language models are trained using 175 billion parameters. If you have to ship that back and forth between the collaborator and the aggregator that we were talking about, it requires a lot of bandwidth and the security element of it. And that's sort of the functionality where how do you ship those large models back and forth, was scratch their own itch element for VMware. And that's the whole element. So it's both a project and the GitHub code base where people can look at it. And they're very simple.  So there's very easy way for you to get started with open FL as well. You talked about VMware getting involved. Can you also talk about what kind of community you're trying to build around this project? Yeah, no, absolutely. So I mean, if you look at, you know, Penn Medicine is super deeply involved. I mean, the initial study that I talked about, that was done around three states, but then we expanded to 71 different sites. So we were really working with, you know, a lot of these medical institutes all across the world. So from three states just based out of the United States versus the 71 sites all across the world, that truly really allowed us to make this data that much more diverse. So now you can start thinking about what kind of communities we want to build around it. We want to work with the academics. We want to work with the medical researchers. You know, we are bringing the concept of federated learning and the whole concept of federated learning, if you think about it, was created by Google when you are typing your text message, you don't want your text message to be sent back to the server and that's where it was leveraging the whole, they published a paper of federated learning. So really academics, medical institutes, you know, companies, you know, who really see the value in this, who really value that the data cannot leave the jurisdiction and also anybody and everybody, developers, you know, how can they improve their entire developer experience, getting started experience, all of those are really valid folks that we would love to invite and participate and I would encourage, you know, literally it took me half an hour to just say pip install openfl and it installed the thing for me, then I have the FX, the CLI, which got me started. So I think I would just say, get started with it, open source is all about scratching your itch. So if you can say, you know what, this is what I don't like about it, submit a PR. You know, very well said open source is scratching your itch and once when I was talking to Linus or Greg Covertman and they're like, yeah, we do like a contribution but we don't like contribution for the sake of chatting. We want selfish users who are also using the same code base because that will make it sustainable. Otherwise, once you drop the code base, you're done. I don't even care about that. So which brings me to another point is that when we do look at today's open source word, there are two kinds of open source. First, you know, of course, a lot of companies, they put their code base in neutral foundation, like Linux foundation, which breeds a lot of confidence among your potential competitors. They don't worry about tomorrow, you'll change the license and suddenly log them out of it. At the same time, there are a lot of companies who still do a lot of open source but they keep control over the code base. Contributing has its own benefits because you have a lot of your community but you also lose control. You cannot direct the flow because that depends on the community or the whole ecosystem around their projects. So talk about what does company like Intel gain by not essentially releasing, but not only releasing but also putting their code base in foundations like Linux foundation. Yeah, I mean, I think what I would say there is the concept of Nora, right? No one right answer. And also people do open source, how it matters to them. A couple of weeks ago, I gave a talk at scale, which is the biggest community run conference in North America, about 3,000 open source developers and I was talking about fostering an open source culture. And one of the key elements over there is that the open source that you're doing, it really needs to be tied back to a sustainable business goal. That is super important. And as I mentioned in the beginning, one of the main reasons we contribute code out there is because we want to see that diverse community helping us make it better. It is the moral imperative for us that we want to create that horizontal competition where we create an equal leveled playing field. We want to break down those world gardens where one or two vendors are really defining and controlling what that agenda looks like. We want to enable that more diversity, more inclusion across the industry and more partners to make that solution that much better. Now imagine if that open FL case that I was talking about, if we were only running that within those three states, that model would never be successful. That's why bringing that richness of data across all the world is so much important for us. And as I say, a rising tide lifts all the boats. Sure, that's how we see it too. The reason LF becomes a really good home or any foundation for that sake, because it's vendor-neutral. As you mentioned, there is no copyright trademark issue. It's a neutral governance model. It allows folks to join in. That's exactly why University of Pennsylvania, VMware, Flower Labs, they are all joining the part of the technical steering committee because they know they will have an equal say in the vice. Sure, the project was originally created by Intel, but by being as part of the LF AI and data, there is more than 20 years of legacy of LF doing things the right way. So I think that is a part that really excites me. That is where Intel is part of what? Almost 800 foundations and standard bodies. There is a reason why we are so deeply involved in open source for over 20 years. And we believe that is the right model. We believe that is the right way to engage with the community. And we have contributors. We do job would carry water work. We are part of the sake leads. We are part of the foundation boards. So I truly believe that is the way, how we make the world a better place. Arun, thank you so much for taking time out today. Not only talk about this project, but also, I love the way you talk about the low-law organization Linux foundation of low-law computer like Intel by putting their own code base into open source. I love all this discussion. Of course, we talk about this new project as well. And I'd love to see you again soon. Thank you. Thank you. I really enjoyed it. Thank you for the opportunity.


So, Steve, welcom to Bradford, first day here at the training ground. Nice warm day, nice easy start for you. Yeah, yeah, it was warm out there this morning. It was nice to get in and to get to know the lads and get me first session done. I meet you guys. An old sign of steel now. You must be delighted to join Bradford. Definitely, I spoke to the manager at the back end of last week and he obviously showed an interest. I was speaking to one or two other clubs at the time and I went on holiday and had a good think about what I wanted to do. Having spoken to the manager, I thought that this was a club with the ambition that I wanted to be a part of. You know, you look at the squad, that's already here, they were a little bit short last year and hopefully with one or two additions that the manager is going to bring in, we can be up there.


Good morning. Good afternoon. Good evening wherever you're hailing from. Welcome to the first ever data services office hour featuring data scientists. Holy smokes. What's going on? Smart people early in the morning. So I'd like to welcome two new faces to the channel. Sophie Watson and Chris Chase Sophie's favorite algorithm is linear regression and Chris Chase is in love with dogs apparently and has treats for his dog in his desk. I've never thought of doing that but I might in the near future so please introduce yourselves for the audience. Everybody. Thanks Chris. Hey, I'm Sophie. I'm a data scientist. I've been working at Red Hat for the past kind of four years. I've been working at Red Hat and we've been thinking about how can we make data scientists lives easier and how can we make the Kubernetes and OpenShift experience fantastic. I'm Chris Chase. I'm an application developer by training and for the last couple of years I've been trying to take all the good work that data scientists do and incorporate them into different applications using lots of different methodologies. My dog just decided to strut in so now she laid down the doorway. Never mind. Sorry. So for folks that don't know, my dog is a German Shepherd corgi mix and people always want to see it when I say that so yeah. So we were talking before the show and one of the topics we were covering was, you know, there's a lot of AI, there's a lot of ML there's all kinds of, you know, things happening out there but those models are trying to get into applications or actually running in production, as I call it. And, you know, I've spent some time in the research land in my career in DevOps, and I know the struggle of, you know, that researcher trying to get their model into an application actually working kind of deal. So the idea here is to kind of help you the audience, you know, whether you watch it now live or later. You better understand like how you can integrate your models into your applications and actually get all that working together so that your actual work can go to work for your customers or your organization, for example. Right. Yeah, what we're talk talk more Sophie please. But, you know, I think the thing that we've got to remember is that data scientists on application developers or infrastructure experts or Kubernetes experts, we, you know, we don't, we don't get excited about running things on X platform versus X platform. We just want to go off and train that machine learning model explore that data and do what we're comfortable with. And although there's programming involved in that it was certainly not, you know, application developer level of programming mastery. I think in general data scientists work in a way which is not conducive to getting that machine learning component into a larger intelligent application. And so that usually means I just do my bit I'm like yeah I've got a machine learning program I've trained a model I'll just flow my stuff over the fence to Chris, he gets very Yeah, and, and usually when I get it this is not Sophie by the way but just like sometimes from a data scientist who hasn't worked in a software engineering firm before or you know they used to stay in that Jupiter notebook all day every day and that's kind of their world. And so they say hey, I've got this awesome working model that does XYZ. Here you go. It's a notebook where totally detects whatever I want it totally works my fantastic and then the first time I go in there and I try and get it working.  And then, usually for the app all I want is like the prediction, like I'm not going to train in my app like that's a long process right I just want to know if this thing is this five is a five or if you know this linear aggression, you know what it's going to spit out what this prediction is so I just want that prediction and I just want. I just want its dependencies I don't need all this exploration and you know training and all this other stuff I just need to find that like one function and it's dependencies and that's not what I get I have to tease it out. And it's really hard to do that. And dependencies is a good one as well you mentioned that Chris like yeah. Like all what what all dependencies would come with a train to model. So, talk to me. So, my favorite is when when a data scientist has been working on his local machine and he'll do a pip freeze is like here's your dependencies and it's literally every dependency on his computer. No virtual environment.  Like there's a process that we have to follow to get it in the prod and downloading the Internet's not part of it usually. Yeah, I think so I think that's true. I don't want you know data scientists to look at this and think, oh gosh I've got to change everything about how I work and I'm the problem here. But because that's, you know, it's just a completely different skill set and a different appreciation for what it takes to put something into a larger application. And I think at the end of the day as data scientists we want to we want to find the insights in the data and we want to share that with people we want to be as useful as possible. So, the way that we've been kind of approaching it here at Red Hat is to try and you know teach some DevOps best practices to data scientists. So still allowing them to work in Jupyter notebooks, not assuming that we're trying to turn data scientists into application developers, but just trying to, you know, educate on what it takes to put something into production. No one, no one teaches you about dependencies when you go to data science school, data science class isn't like. Right, it's like this is the log loss regression function for the third level of your 7000 level neural net and if you integrate this then you're good and like you know you never use that in a day job I'm sure someone somewhere does but the kind of the practical and engineering best practices. I think we as data scientists can learn a lot from. Yeah I mean just something like putting it in a container is like a huge win. You know normally. So if you tell a data scientist how just put this in a container right like you have to get them there first but also why why should they care whether or not it's in a container. So it's kind of like well if it's in a container is more repeatable it's more portable it's more scalable all of the benefits of containers when they translate beautifully to you know machine learning and data science models and workflows and pipelines. But at the end of the day. I don't want to take it like this stuff to write a docker file. Right. And that's the thing where like it teams and your dev ops or sres or, you know, all the people and functional parts of your pipeline of getting applications to production come in and say. Hi, let's work together on this so that we can make something usable for all. Right. Like makes the dream work in the sense, like for sure, like, sorry to be cliche about it but you know it's in my experience working with, you know, data teams, it is all about like, hey, I can build you this thing to help you make this more portable just run it like this. And you can put anything in it you want, and it'll be fine and giving them the context to do that helps tremendously, as well as you know any guiding points and kind of being their help desk for lack of a better term like that is definitely necessary. Yeah, and I don't think we're asking too much from the data scientists like we don't even really need for them to understand containers, even know, right, because if they just give like a fun like if they just are able to separate their functions, you know, write a Python function which they know how to do very well. If they and they understand their dependencies way better than the app devs guy right like oh I need this version of TensorFlow and this version. Like that's really all we're asking is just like just, just know it's important for that app dev to take your work and take it to that next level. Yeah. Cool. So, we do have something we can show off I'm assuming right. Yeah. We're up for it. I mean we can show you what this kind of process of getting machine learning into an application is looking like at the moment on OpenShift. We can use Red Hat OpenShift data science for the data scientist environment so you'll be able to see that that's kind of it's what a data scientist likes it's got the tools data scientist likes is the environment they like to work in. And then I think Chris is going to impose some structure on my work. Yeah, I mean, we can start off with a little project here I guess if you want to see like what what what I would kind of need for you to do your work and so that we can like take it to the next level. Yeah, let me show a little like a sample project I guess. Is that all right. Yeah, whatever you want. Cool. So here is like the most dead simple thing I can think of because like in OpenShift we have S2I and the data scientist has their notebooks and stuff. And so, how do I, there are a bunch of really complex methods of getting data science work and you know we can come cover that I think the future with like CI pipelines and you know, servers like Selden and KF serving and stuff. But really, we could just the two of us could still get our work done just by using OpenShift and like Jupyter notebooks and the OpenShift data science platform. So here you can see this little template which would make that easy for a data scientist like so to get started, it's got some notebooks here for her to get started. And then when she's done there, she can just take out her prediction function put it in a Python file and take out the necessary requirements for her. Her prediction function and put it into the requirements that text. And if she does that for me, like I can take this this already builds because it's already got an application file in it really a very small application file. So we could just like use this template, create a new project out of it. Right. And then like she's ready to go I'll give this get repo to her and she'll just this will function is basically storage version control, and it'll notify OpenShift that hey, you need to rebuild and redeploy so it's doing some heavy lifting there. It's still simple. And there's almost nothing to it as an application developer I'm already used to using up, you know, as to why on OpenShift, and she's already used to using her notebooks and writing Python functions. Right. So it was a dead simple and I would give that to the data scientist to go ahead and get started writing. I think that, I think by starting with already a discussion between the data scientists and the app dev like right this is how we're going to serve the model so this is what we need your work to look like this is what the end goal is. It's nice. It makes me think that, you know, maybe my model will deploy this time. Well yeah and pride in your work is everything right like Deming going back to Deming, you know, having pride in your work means if you're a data scientist you want to see your models out in the world right like that's going to give you the most pride and and make you fill the field which we all want to kind of, you know, feel like we're being productive at our job I hope you know and having that desire to get your models out there I understand that I feel that for a lot of people because I often feel there's this wall of confusion that sometimes data gets thrown over and just doesn't work out somehow and we're trying to, you know, educate people so that it can work out. The show is for our ops people and our data scientists and our app devs all together all in one. So cute. And okay so I will go ahead and show you what. Let me see if that works so you're seeing sweet. So I'm in Red Hat OpenShift Data Science so this is a managed service it's running on top of OpenShift dedicated. And it's an environment that like has all the data science things so I don't have to go to the OpenShift console and look at all those dashboards and find a route or a route as you North Americans say and expose it and then go. You can say dirty Americans that's fine. So yeah so this is Red Hat OpenShift Data Science you can see that the only thing we've got enabled here at the moment is Jupyter Hub because that's all admin has put in our cluster for us at the moment but there's a load of other stuff that you know you can add into here. It's like Chris talked about other ways of serving models you know we could be using a pipeline maybe we're using sold and deploy to serve and manage monitor those models. So that could all be integrated. I think later we're going to hook up to OpenShift Stream for Apache Kafka right Chris maybe. Sure yeah if we have time we can we can do that. So I'm going to launch Jupyter Hub and we can see all these Jupyter notebooks that we've been talking about so completely self service for me as a data scientist I can get my environment I can set my container size. I've got my AWS secrets in there so I can access my data and I'm going to start my server. Suspense. All right so now I'm just in you know standard Jupyter lab environment you know I can launch notebook and start coding in here I'm not going to because Chris sent me up that Git repo. I actually forked it and already put a load of stuff in it otherwise this would be a very slow very boring hour for everybody whilst I typed some slow Python code so I'm going to copy my fork and come and clone it in the environment. All right so I know we've been talking about Jupyter notebooks like if you haven't maybe you haven't used them before maybe you haven't had the absolute delight of trying to interact with them and see them or being sent one and trying to get it to run. This is what they look like so it's essentially like a real notebook it's like you can just pull your thoughts down it's a combination of posts and markdown and code and you can actually execute that code in line and then output is printed below so here you can see we've got these standard Python imports and then we're asking it to print out the TensorFlow version so I know we've talked a bit about dogs Chris you've got dogs as well right. Yes. And other Chris you've got dogs as well right. I do Franklin's here actually maybe we'll try out your model on this dog here. So what we're going to do today is just we're going to use a pre trained model for object detection. And I think the thing I want to emphasize here is that a lot of machine learning models have already been trained for you are not good enough for the job at hand. And there's you know there's some use cases where you're doing something completely new and pushing the boundary, but in general there's things out there. And we want to create cute little dog detector today so that we can, you know, point our camera at dogs and confirm that they are dogs. And I know there was questions earlier about whether dogs are actually humans.  She has human like needs in the whole nine yards. Yeah. Several words that she understands in the English language and the whole nine yards. She's ignoring me though. So, no, that's typical. Hopefully we get to see her later. Hopefully she might come in here. She's like right outside the doorway to the point where it's like obnoxious to go grab her. So cute. So yeah we're going to put together an object detection app that you know you can show a photo of a dog and hopefully it will tell you yes there is a dog and you can show a photo without a dog and it will say there's no dogs here. So, in order to, you know, start this and test this out, I'm going to need a photo of some dogs. I'm using Max and Margot here today. This is Max. This is Margot. They live in Bristol, England. They're very good dogs. That's all dogs are. That is true. And so I'm using TensorFlow as well. So TensorFlow is like a standard library that's often used for image processing, which is what we're doing here because, you know, machine learning models can't actually take in a photo. They don't know what to do with that. So we're using TensorFlow, turn this into a tensor, which is something more that our model can understand. I said I'm using a pre-trained model. So there's like these image detection models are good and they're very big. So to train them yourself, you need tons of data and you'd need tons of time, you know, load of compute power and there's just no benefit when someone's already trained a model on dogs for me. So we're using this SST MobileNet data model and it's been trained on Google's open images dataset. So it's got 600 types of objects it can detect. We just want dogs in there. So we'll deal with that later. But yeah, we load in the model. We pass it in our tensor here and we can go ahead and ask it to detect and it spits out all of this code, which is essentially telling us the objects that it's seen in this image, how likely it thinks it's seen them. So this is, you know, 85%, 79%, these are much smaller. So it's not very confident that it's seen whatever object it thought this might be, maybe some more classes corresponding to the type of object. Detection boxes. So this model is going to put a box around the thing in the image to show us where it is. And finally the moment of truth, the things that it's actually predicted. So dog, dog, footwear. And so for each of these we've got a detection box. So we're going to be able to impose those back onto the image so that we can see. So we can go ahead and draw bounding boxes on our image containing those predictions. And okay, we've got a dog here. We've got a dog here. The font isn't very clear, but I can tell you that it thinks that their paws are footwear, which I think is really adorable. That is adorable. It is. But for the purpose of today, we just want to detect dogs and not footwear. So there's a couple of things we're going to do. We saw that first off, the model was making predictions that wasn't very sure about. Right. So we're going to fill all of those out. We're going to put a threshold in and say, hey, if you're not at least like 30% sure about this, then don't wake me up in the middle of the night to tell me there's a dog outside that I can go and pet.  Yeah, awesome. Yeah. I like the gnarly stuff. Yeah. Yeah. Or Coyote. Could it tell the difference between a coyote and a dog? That would be fun. I don't think I know what a coyote is, but really just out later. Yeah. It's more than loony tunes. I'll just leave it at that. All right. And then the other thing we're going to do is filter out any predictions that aren't dog because we only want dog. So if we're only going to get this app to identify things that are a dog and that we're sure are a dog. So we go ahead and, you know, encode that here if label equals dog and score is greater than 0.3, nothing too extreme going on. And now we just got our dogs and no footwear. Fantastic. So, you know, Chris, I'm basically done, right? I'll just throw this over the wall to you and tell you put it in an application. Right. Do the thing. So right now is actually usually where I end up getting it, but that's why we had that definition earlier of like, hey, could you separate out your prediction function into its own Python file, right? Because you don't want me fishing through that and trying to figure out what is important because I don't know. I'm going to break her code. Like I have no idea which function is important. And the same thing goes for dependencies. I don't know if TensorFlow is just for her training or if it was going to use it all of it. There's a bunch of those. You know, Matplotlib is probably not going to be used for my prediction function, right? So she puts that requirement that I'm going to need in the requirements.txt and she puts that prediction function and prediction file. So I'm going to need from screwing up her code and I'll be able to incorporate it. Keep that much easier. Nice. Stuff like bo2 3 that we use to connect to S3 standard Python library to connect to S3, which is where my dog image was stored. Like Chris doesn't need that because Chris is going to be trying to predict new images that are taken on the app. He's not going to be connecting to S3 buckets. Let's start with the requirements. Chris, tell me about these first three. So I added that to the template. You know what? I should have put a comment in like this is mine and this is yours. So that's my bad. I probably should have done that. But six below is kind of her dependencies for like the data science stuff. And up there was my application dependencies, right? So we're kind of keeping them separate. And so that'll allow me to change the application up and work while she's changing her data science without messing with each other too much. S3. And then this notion of putting the things that we need to make the prediction in the prediction.py. So our notebook had everything in it had. Let's look at an image. Let's see how we turn an image into a tensor. Let's look at all the output from our prediction. Let's plot all the bounding boxes including footwear. Let's now amend the code a bit. Okay, now we can see that we can just detect the dogs. So there's kind of a lot of stuff going on in there. It doesn't look like Python code that you would want to share with the class, I guess. It's a communication tool. It's a nice story. You can see how my brain was working and what we were doing to get to the result. I think it's really good for teaching, learning, understanding, but it doesn't make for an application. So in this prediction.py, we've just got all of the things that are important for doing the prediction. We've got three functions in here. And that's all you need, right, Chris? Yeah, and that is so much easier for me to read and get started with and then going through all that exploration stuff in like the notebook, right? It's just going to, like, I can pick it up and go. Yeah, I can show you that actually. All right. All right, let me see here. What do I got? So let's see. Are you finding the mythical tab that you want? I am not finding the mythical tab. I'm just terrible at this. It's okay. Can you see the dog detector service screen? Yes, we do. Okay, yeah, so you can see this is her project that she uploaded. You know, it's got her new prediction file in it, you know, which is what we wanted, right? It's got her requirements in it. My application file was ridiculously simple. This is kind of a dummy project. It just calls her prediction function, right? And this is a buildable S2I project. Like it started off buildable, right? She didn't have to do anything. I could have built it right away, right? So we were already ahead of the game. So if you don't know what S2I is, source to image. Source to image, which is really nice. Like, so I'll show you what it does. So your OpenShift can just read from that Git repo and create an application from it. So let me go to the developer screen and show you how easy that is. I mean, this is really good for development. I wouldn't use it for obviously production systems, but it's really nice for iterating on it, right? If she pushes up a change to Git, or if I push up a change to Git, you know, our service gets automatically redeployed and we can take a look at it. This here is my little app that I'm going to connect to her service, right? So let's go ahead and create her service here. So create a new S2I from Git. We'll put in the Git repo. We can find that. So it's already going to automatically detect as a Python application, because that's what it is. We'll go ahead and create a new separate for this, the dog detector service. And then so my app will call into the dog detector service and request some predictions. Now there it is. So it's building now. So it's going to automatically build from Git and create a containerized deployment and put that in OpenShift for my application to call. And my application is already configured to know where that is. While that's building, because it takes a couple of minutes. Now I wanted to automatically rebuild every time she pushes up a change to her model, because her model is going to get changed quite a bit as she, as we test it out in the real world and see that coyotes are coming back as dogs. We're going to want to retrain that model to leave out coyotes or give you some special super alert. So it's really, so that's really easy since we're using Git and as to why we'll just copy the URL here for the GitHub web book. And we'll go ahead and create a web book into the dog detector service. So we'll do that do Jason. And that's that's pretty much it. And so every time she pushes up a new model that build will go ahead and rebuild and redeploy and we'll get a new deployed service that my Apple call. Every time she updates model, we'll see that right. So let's see how that app works real quick. Let's see. All right, so here's my little app. Come here, buddy. The science school is how to use Git. I had never used Git before I started working at Red Hat and I remember that first week was quite something I think there was a lot of Well, it makes you feel better as if it makes you feel better. My first time, like this is a long time ago, but switching to Git from other version control. I swear to God, I think I screwed up for six months. Like I think I forced pushed a master and I never heard the end of that for like the rest of that job. Okay. Yeah, I think I mean the thing about this is that it's like, because you set up that repo for me. All my files were already there. I just had to edit them. Kind of it feels like it makes it a bit neater. And then the benefit of working with an application developer is that if I do something tragic on Git, which doesn't happen anymore, but it definitely has, then I can just call you up and say, Chris. Although I will say no matter, no matter what happens if it's Git or something else, I do hope that data scientists save versions of their old stuff. Like I have been in that situation is like, oh, hey, can we get that model and he's like, I don't have it anymore. It's like it was in my Jupiter have no book over there and I don't know what happened to it. I can't get it working again. That is that has been frustrated for me too. Yeah, and I think not just the, I mean, for me, it's not just the model as well, you know, we talk about the model as being like the entity that's going to, but it's really like, it's a whole pipeline. It's what data train that model. It's, if we think about getting these stuff into applications, then you've got to add that extra discipline of like, you know, this pretend that everything you do might be audited. Hey, which model were you running on dates, you know, yeah. And the notebooks are super important for that too because that was the notebook usually sometimes they use to train that model right like that they can reproduce that model as long as they have that notebook still there. So, yeah, saving that stuff is super important. Yeah. It's true. It's also the case sometimes that, you know, with notebooks you're never, you're never able to reproduce the model because you know we're talking earlier about running all the cells out of order and kind of you have to impose some discipline to know like what, what happened where things came from and how to how to keep going.  Right. So, I mean, it's crystal here. Do we have time to show I guess we could show Kafka might change that app up to use like a Kafka queue instead. Yeah, let's go for it. Okay. How's it going. Yeah, sorry everyone Chris is having internet issues right now. So I'm not sure if we'll be back but please continue the conversation. All right, we'll pick up where we left off so right now this app is fun but it, it only like works whenever you are clicking on it so if I want to do a monitoring service like if I wanted to monitor something outside for coyotes or for dogs, or whatever to make sure, you know, if it gets on his own I need to be notified about that. So let's see. So, let's go ahead. So instead of doing a single snapshot and uploading it let's go ahead and like put a constant stream of images onto a Kafka queue, read that Kafka queue, and then do the predictions on that Kafka queue. So I went ahead and made a Kafka queue called object detection, right, and it's going to take the images, do the prediction and put the objects on this other queue and then I can read from that objects queue and see if anything was detected I can read those predictions right and so hopefully what we'll get is we can put up a camera, and it'll take that stream of images and it will run the prediction on those images. And like I said, you already done your part. My goal is to redo this app without you having to do anything right. So, so I'll go ahead and see what I can do so go ahead and open up a Jupiter hub app of my own. So just like you did s3 I'm going to go ahead and talk to this Kafka queue. So I need to get the connection information and get that here. And that will go as one of my environment variables which I'll use in there. So I'm going to go ahead and spawn my app. And this is just to show how easy it is to work with Kafka you can do it from inside a notebook and then we'll just see change this app up a little bit. Right, so we'll go ahead and do a new project a new service that Kafka consumers which will go ahead and use from inside Jupiter hub, and we'll see what that looks like. So this is a Kafka consumer which will read from that second queue so we'll go ahead and get it working here. So you can tell I've installed the Kafka Python library. I've read my server information. And this is me creating a consumer that's reading from just a notebook test queue to see if it works so it's subscribed. And then here in another notebook, you can say, go ahead and just send some messages for it to read. And this is a producer. Right, so this is going to put messages on that queue. And so my app was going to go ahead and read the images from one queue and then produce a new message on another queue and you can tell this one is sending some hello messages. And this one is going ahead and receiving those and consuming those. So we've got a consumer and a producer worked out. Let's go ahead and change that up instead of doing the rest for rest service we did before we'll just go ahead and create a consumer to consume those images. That the app is feeding into Kafka and then we'll produce the predictions on there so just do a prediction and just stick that on that other queue. Right, and so you've already got your prediction hasn't changed it's still exactly the same. The requirements your requirements are exactly the same. Mine I just changed to use Kafka Python because it's a Kafka Python consumer. So we can go into the app again. And we can create a new service the same way we did for the dog detector service we can create one for this application right so we'll go ahead and create from gets the same one. Right, and then I'm going to have to put my Kafka information here in my deployment of our variables in my, let me go ahead and hide that from you on my screen because I don't want to have to show you. And there you go so now instead of just talking to your rest, the rest service, I can go ahead and talk to this Kafka consumer. Sorry, the coffee consumer is going to be talking to Kafka and it's going to be adding messages onto that queue, which my app is going to pick up. And once that's done building the same way like the other thing is really the same thing. We can take a look at what that does. All right, let's see this work this is on video mode is going to push up an image every so often and see if that works. All right. All right, if you're walking across along for my flower bed. So the nice thing about this was like the app is totally changed, but Sophie didn't really have to hardly do anything at all. Right. And just by that little bit of structure upfront that, you know, that that lends itself to me being able to change everything without having to bother her over and over again you know she's got her prediction she's got her requirements and it's really easy for me to pick up and move with it. But yeah, like, so all this transition from data scientist app dev I feel like you just talk a little bit beforehand to get a little structure, you find out what I'm going to need and what she can, you know, do just makes everything so much easier. The shared understanding the shared knowledge right like that's what DevOps is all about right like helping others succeed. Like is everybody's role in DevOps and ops in general, you know, especially application development right like we should be helping each other. And your dog still in the background looking at you like why aren't you letting me out. Just one point that out. Shout out to my dog. So, yeah, like, this was an amazing, you know, kind of handoff process that we just witnessed, you know, so if he did the work, train the model. Handed it over to Chris, some shared understanding about what things are needed and Chris implemented it, you know, for lack of a better term so this is the dream right. And it's not prescriptive like this is just kind of what we did and this is a really easy way for us to do it between an application developer and a data scientist right you know there are lots of great ways with you know CI and pipelines and stuff that we can do if we have go a little bit further and have a little bigger team, but this is drop dead simple. Exactly. I think that's the advantage of it.  Yeah, just like explainability services. And it kind of automates that roll out of the models. You can route some of your traffic to new models and tell very canary to plumes and stuff. Yeah. Yeah, and it's pretty easy like you can just tell it where your TensorFlow model that serialized model is and it'll be able to deploy it. So it's real, it's real nice and easy you can cover that and then for the building of the model, you know, sometimes you want it to automatically build or build outside of a notebook. And so people will use like, you know, so pipeline of some sort, whether tecton to get to flow pipelines, you know, Argo, whatever it is people will build those things in all kinds of ways. I think there are all kinds of possibilities, whatever's right for your team, but definitely in the transition from data science to SAP dev is really nice if the data scientist knows upfront what to make. Awesome. If you were, you know, somebody that had just watched this and were, you know, and like your Sophie or training models and such. Do you think going to that app deployment team and saying hey, or app development team and saying hey, you know, do you feel like you've given people enough to go with here to take to those teams and be like hey we can work together. Like, somebody has to reach out and build that bridge, and it's not always going to be the operations team or the app dev team because oftentimes they have their head down but if the data scientist needs the model in prod, how would you build that bridge. I mean, I think the process you can work ahead time. Yeah, right like, you know, at some point in time you need to have like a sit down with you know representatives from all sides to kind of say hey, this is the goal we're working towards. How can we facilitate that as a team. Right, I mean you kind of have to have whatever because it also depends on skill set right so if you like how much is that data scientist willing to take on because there are some data scientists who feel pretty comfortable actually in flask and this that and the other thing. And they don't mind doing understanding some of that and there are others who don't want to touch it. So, right. I think there's got to be an agreement between teams like hey this is what we can produce. You guys can pick it up from that point onward. Right, like at the hand off the takeover point. Yes, absolutely. And then you can try out a different product and see what happens works for you whether it's, you know, cancer and keep low word custom apps. Yeah. And I think just like, you know, a real understanding from kind of the upside and the upside that a lot of this is outside of a data scientist comfort zone might know if there's data scientists that are watching this they're just starting out or, you know, in school like data science like nobody, nobody seems to teach you, you know, recent like package control that library control like you do just work on a computer and install every Python library into the same environment like no one. The kind of the architecture and engineering stuff is like definitely a completely new level. And I hope people see that like you don't, you can learn as much or as little of it as you want, you don't have to become an expert, like we're not trying to make data scientists engineers. Right. Yeah, that's the biggest thing right like we got to keep people close to their disciplines, but still integrate right and that's, that's the struggle that's where teamwork comes in right to figuring out where the boundaries start and in for each individual team or person or whatever it may be. But just get get everybody in the room right like have everybody, everybody needs to see to the table, everybody needs to have that you know kind of equal footing in the room, as it were to say like, this is what our capabilities are. And this is where we need to be and figuring out how to get there together is the key, I feel like. I feel like the containerization helps a lot too especially with running those notebooks because like instead of everyone having their own set of things on their local machine, everyone having that common starting point platform for where they're working in Jupiter or whatever helps a lot and me being able to look at that, you know containerized image, even if it's just a Jupiter thing and seeing what dependencies are in that container, you know, helps towards production, a lot to house and pick up and understand what's going on. I thought I heard my dog work but no something else. All right, awesome. So, despite the train wreck of my home internet and you know all that I appreciate everybody coming on and joining us today is there anything else we want to show off before we sign off today or talk about you know future shows maybe we'll be back in two weeks, two weeks or four weeks, four weeks as well. You tell me how often does this repeat, let's see. Remember these things and I've separated it from the whole thing. Every four weeks. Okay, scientists. But basically, every two weeks, the data services office hour will feature storage or science. Right. So, when you think data, we're going to have every other week is going to be alternating between like how to set up the storage, how to build the environment to do these things and then have the actual data science process happen on the show so if you're looking for Chris Blom or bloom and Michelle to Palmer they will be here every other week and Chris and Sophie will be here on alternating weeks basically so it's every two weeks for the show, but every two weeks that's a different crew. Does that make sense. I probably explained that the most awful way possible just now I feel like right because I'm so flustered but anyway. It makes sense once every four weeks. Right, like. I think for for next shows I think I think we got some stuff that people want to see I think we can look at some data exploration from inside notebooks and really nifty tools to do that. I think we can show people some of the, you know, the training pipeline stuff that people use to build their models. And I think we can show them some of those serving things I was talking about we can show off like sell dinner, okay, serving and see, you know, just to show people the different options because this was the drop dead simple option. Like I say if you're willing to go into those other offerings, I think there's a lot of value there, your fun to use. Correct. They are fun, and I wish I was a data scientist, because it seems like a lot of fun. So, thank you Sophie. Thank you Chris. Thank you Bobby for helping out the today Bobby the intern saving me yet again. I don't know what we're going to do when you're gone Bobby, I'm going to be sad. But tune in in two weeks for more data services office hours, but later today we're going to have in the clouds with Kirsten newcomer and hopefully Andrew play Schaefer talking about DevSecOps and shifting left. We are also going to have Dev nation on the air will have get off sky to the galaxy this afternoon. We're going to look into into it into it's coming on to give us a deep dive into Argo. And we're going to wrap things up with the another, you know, kind of series premiere of the stack rocks community office hours today so stick around stay tuned subscribe to the calendar if you're not already which I'll drop in chat. Sonny's back. Hey, Sonny, come here, buddy. Come here. Come here. Come here. Come here. Everybody wants to see. And dog. Oh, she looks terrified. Hold her wrong. It's okay, buddy. You want to go get Julie. Okay, go get you. Oh, don't have me. All right, so now that everybody's seen Sonny, I will should sign off for the day and we will see y'all later stay safe out there folks. Thanks, Chris. Thank you.


Good afternoon, everyone. Thanks for coming to my talk. Today, I'm going to talk about security in modern apps with zero trust and next generation web application firewall. I'm so happy to be here in this conference. I came with my entire family.  So I appreciate that. Before we start with the topic, let me introduce myself. I am Jose Carlos Chavez. I am a software engineer at The Trade. The Trade is a service mesh company. We do security features on top of Istio service mesh. And I am an open source enthusiast. I have been doing open source for seven years now. I really enjoyed it. I'm an OWASP, Coraza WAF co-leader. This is a WAF project at OWASP. I'm a SIPCM core member for those who are in the world of distributed tracing or observability. And I'm also a loving father. And I am from Peru. So web application firewalls, WAF for family and friends. What is a WAF? Well, traditionally, a WAF helps you to protect web applications from incoming traffic from the internet. It protects you from known attacks like cross-site scripts, cross-site forgery, SQL injection, this kind of malicious traffic that I aim to compromise your systems. Usually it's deployed in the layer seven as a reverse proxy in front of your servers, listening and monitoring all the traffic and deciding whether this is a malicious request or not. And the proxy basically does the analysis and then decides whether this request goes upstream or not. This is how you traditionally deploy a WAF. There is the outside world with mobile applications, web browsers, devices, legitimate users, and then attackers as well.  And then if you see it, some sort of becomes a bottleneck in the middle. So evaluation can be that expensive. It cannot be that expensive. Also, as you see, you have it decoupled from the server. So any knowledge about the server is an overhead. But yeah, in the old days, you have the WAF like this as every other firewall. Some of the features are IP fencing, which is you can deny a specific IP through a deny list or allow a specific IP. You can do geo fencing and geo blocking based on geo IP databases. Basically, you can create a virtual perimeter and say, OK, requests from this region or this country can go through. Requests from this region or this country are denied depending on the geo IP database, right? Another quite useful but complicated feature is request response inspection. Basically, the WAF will buffer your headers and your body and the query strings, run analysis and examine them, and compare against no malicious strings and then decide whether this is a legitimate request or not. It's useful because it allows you to prevent zero days attacks if you remember log for self, for example, which was an attack through a query string parameter. Well, WAF could prevent these kind of attacks when you know what the problem is or how the attacker is approaching your service but you don't have the solution yet in place. It avoids client-side attacks, of course, both attacks, various files, et cetera. Then you have a security rules, which is a curated list of rules that will block known attacks like SQL injection, cross-site scripting attacks, local and remote file inclusion, remote execution, common injection, size restriction, these kind of rulesets. If you have heard about, for example, a WASP code ruleset, it's a curated list of all these rules for preventing these kind of attacks and they can be deployed in a WAF. Anomaly scoring is another interesting feature where you will analyze the traffic and then if a rule matches, depending on a misspelling, for example, in the URL, maybe someone trying to attack your server or if they're actually trying to run an SQL injection, you will give a score and then based on the threshold, you will decide whether I should block this request or not. It's a well-known feature where you will block a request or you will only allow a certain number of requests in a window of time. Right? I guess the most, let's say, the most famous use cases blocking by IP but you could pretty much do on every other input, right? It could be user agent, it could be IP, it could be, yeah, well, those are typical use cases. And then you have both mitigation which is basically analyzing the cookie sent by the browser and then check them to find whether they are legitimate bots or not, right? Because there are different kind of attacks when you have a bot, then you launch a CAPTCHA challenge or you can have a bot pretender which is a bot that pretends to be a good bot like those index, those bots that index your web and web scrapping protection and so on, right? Those are the features that are from the classical WAF, let's say. And I know security has become like a strong concern these years but before it wasn't that hard, security was easier, everything was easier, I guess, but security specifically was an afterthought. There was a time when security was just about, I need more security so I just deploy a new WAF and then I feel protected, right? Unfortunately, now the way we deploy applications and the way we operate applications and the amount of dependencies we are tied on when we build an application are so big that this is not anymore valid. And the reason is because the traditional WAF was focused on perimeter security, right? Basically, when you were inside the network, inside the perimeter of security, then you were safe, right? You will even have unencrypted traffic because, yeah, they are deployed in your servers, right? So why would you not trust them? And the problem at the moment is that with all this cloud native madness that we live on and because of the scalability, the scalability needs that we have now and the way we deploy stuff, if you think about deployment nowadays, it's not something that you could draw a perimeter around, right? Because there is no single easy identifiable perimeter because you have now cloud, multicloud, on-prem, third-party services, lambdas or functional services, artifact registries, et cetera. So what would you draw their line whether this is trustable or not based on a perimeter, right? Also now in the time of microservices, a request traverses those enough services when you do a call, for example. So the majority of the traffic is West, right? Across fellow workloads, whereas traditional WAF and perimeter security is more like a single getaway and you receive all the traffic from the internet to the internals. So yeah, that's problematic. Let's say the more of the traffic happens inside your network, not from the outside. And then what I was mentioning before, like you have getaways in front of your services and they have to have a lot of knowledge about internals because they have to protect the PHP applications, the Go applications, the Java applications, deploying tons of rules for different languages and having to know a lot about implementation detail of your components, right? So that leads to operational complexity, misconfigurations and you need to have an effective way to deploy all these configurations in a timely manner and all that, right? So it becomes much more complicated. If you read security reports from the last year, one of the biggest source of vulnerabilities in the systems are misconfigurations. So we should be avoiding this. And then finally, the guiding principle of perimeter security is trust but verify, right? But that means that you will first trust and then verify when the attacker is already inside, right? And this is basically crazy now because the attacker is most of the times inside already, right? Then is when zero trust erases, right? Started by NIST, zero trust is enabling the right user under the right conditions to guide the right permissions to the right date, right? Zero trust is now very popular. Everyone talks about zero trust. Everybody sells zero trust. You might buy lots of zero trusts. So let's give a step back because you probably hear about a lot of tooling about zero trust frameworks and services, blah, blah. But let's give a step back and think about the definition. It's a term for an evolving set of cybersecurity paradigms that move defenses from static network-based perimeters to focus on users, assets, and resources, right? Basically, every actor in the communication internal of your deployments becomes a first-class citizen, right? This is the definition from the regional papers, zero trust architecture from NIST. And what are the driver assumptions? Like what is the shift in the mindset that turn into this kind of approach? Well, trust can no longer be based on a network perimeter as perimeters can always be breached and will be breached for sure. Policies have to be defined based on the assumption that the attacker is already inside of the network, right? That's opposite of what we saw in perimetral security where we first trust and then verify, right? Here we are more like in a paranoic mode where we always not trust anyone. All access decisions have to rely on least privilege per request and context-based principle, right? And no, and on identities associated with users, services, and devices, right? You have now user accounts, service accounts. So all the actors participating in your model in a request-response model or whatever, are first-class actors. They should be identifiable. We should run security assessment on all of them and we should be able to know which one is and what permissions they have, right? It's not anymore like, okay, I'm logged in or logged out, right, which was back in the days. And security and access state constantly change over time, right? So this is a shift in how you see your system in terms of security, where before you have an assessment of security and it was static. Now it's more like over the time, when my service is not secure, right? When my deployment is not secure because from one minute to the other, a new CD could be disclosed and then your system is not secure anymore. But nothing changed, right? Your system is the same as one minute before but then the state has changed because of external conditions, because of the internals of your system, right? So if you were granted permissions yesterday, that doesn't mean that you will grant permissions today, even with the same payload, with the same request, the same service, with the same actors, right? These are the driver assumptions. Then we have the tenants. First, well, tenants are principles but not in a dogmatic way, more like in a consensus of a group of people. All data sources and computer services consider resources, as I said like, before you had like, your deployments were basically a rise, an array, or an arrangement of servers and endpoints. But now you have more dynamic elements. You have scaling groups, you have lambdas, function as a service, this thing that in a non-deterministic way will be deployed in your system at some point, doing something. But not necessarily you have control of them in terms of security because they will have to get access to your components still, right? They might have specific permissions to your resources in your environment. Second, communications are secure regardless of location. This is the main shift from perimeter security, right? You have access policy, which are by default to deny and then you have to grant specific access, like questions like, why am I accepting this request from these other service? Although they are in the same cluster, should I trust them or not? So this service in the other, when you have a tenancy model for example, is this service deployed for this tenant? Should I accept request from? Should I grant permission from? These kind of things are new questions that you should ask yourself and then define the least privilege based on that. Then access to individual resources is granted on a procession basis, right? Permission shouldn't be extended beyond a session, right? This is what we were talking earlier about. Okay, the fact that you have permissions, now doesn't mean that you have the same permissions or you will grant the same access in five minutes because security might change over the time because even though the rate is the same, the input is the same, context might have changed, right? Policies might have changed, deployment might have changed. So all these things should be considered. Access to resources is that they're minded by dynamic policies and other behavioral and environmental attributes. That's what we were talking about. Like access policies now should consider a lot of not known or unknown attributes that come into play. You cannot anymore define a single model with these are the fields or these are the attributes that I should consider. Like now there will be a lot of attributes that you might not have beforehand that will be part of your network at some point. So all these policies should be let's say elastic enough or flexible enough to accept this kind of new attributes at some point. Monitor and measure integrity and security posture of ONET unassociated assets, right? This is very important because we say that you should learn from your errors. Well, you should learn from your logs as well, right? When you have added logs about access or denials of access in certain endpoints, you should be paying attention on what happened, what triggered that denial or what triggered that access or what triggered that error that made your service behave differently. And then based on those information to be able to provide new policies that will cover these potential risks, right? And so you should be constantly recording logs and then auditing them, monitoring changes and then you will be able to learn from that to create new policies to improve the security assessment in your system. Dynamic resource authentication and authorization is strictly enforced before access allowed. Yeah, this is a key principle or a key point in Zero Trust that every access should be audited by a policy enforcement point. Actually, in the next slide, we will see the diagram where you have a policy enforcement point in front of every service that is analyzing whether can you access this, can you access this, right? And that will connect to a policy decision point which holds all the knowledge and all the information about which component has access to watch or what are the policies, right? Granting access and trust is occurring in a dynamic and ongoing fashion, right? So you should be able to analyze on every time. You cannot, let's say, access decision can be cash but shouldn't be cash or should be cash under certain conditions with certain policies. But you cannot just say like, again, if I grant you access before, I will grant you access now. Collecting from your current state of asset, network infrastructure and communications to improve security posture, right? This is more or less what we were talking also before, like you should be monitoring everything in terms of security, what are the error rates, what are the, let's say, the anomalous scores that you collected, what are the things that changed in your system that you should be aware of and you should be analyzing to determine is this something, let's say suspicious, is this something malicious? Should I take an action? Should I create a new policy, right? This is what I was talking about, the diagram, which specifically here at Trust Talks about a policy decision point and a policy enforcement point that are those that are guaranteeing that your resources are protected. There is, here it says untrusted zone, implicit trust zone, but implicit trust zone is very local, like it's basically the workload, right? Because you're not trusting anyone inside. If you think about Kubernetes, for example, inside the pods, the sidecar and the actual container and maybe another sidecar, they trust each other because they are part of a logical unit of work, although they are three separate applications, but that's the implicit trust zone. Outside the pod, a pod from another name space or whatsoever, that's untrusted zone, right? So then, why if zero trust crushes all these perimeter security, we are still talking about WAF, right? Because we said already that it doesn't work and that we should throw away everything around security in the perimeter. Well, in life, I like to say that less is more. That's my mantra. In security, less is less and more is more, right? The less you have insecurity, the less protected you are, the more you have insecurity, the more you're protected out. That doesn't mean that you won't get caught, but you have more chances to be protected. So web application firewall is still a valid thing in the zero trust days. And pretty much other security measures like VPN or other kind of firewalls, perimeter security is still a thing, right? Zero trust is not incompatible or the enemy of perimeter security or network security. They are complementary, they are just different layers. So if we look at the, for example, the tenants that I just mentioned, there are two that are specifically feeding the WAF in the WAF law firm, right? You have the integrity and security posture. Every resource request should trigger a security posture evaluation. That's what a WAF does. Every time there is an incoming request, I will analyze the header, the body, the URI and then decide whether this request is legitimate or it's malicious. When you identify an attack, apply network patches and vulnerability remediation. That's also important because once you detect that you are in a vulnerability, it's been exploited based on different factors, right? Based on that your services are now throwing lots of 500 or based on that there is now traffic from one service that wasn't there before, right? Suddenly one service started reaching another or because you have things like Falcon and then you see a lot of course from one service to the internet, right? Then you put in places or put in place new policies that will block that. First you will probably recreate a service, kill that service or even kill that machine because it's compromised already and then put policies in place to mitigate that problem, right? Sometimes you even not fight the root cause but fight the symptoms, right? Start blocking access rather than until you can figure out how this guy got into my system. And then collect the info on current state of communications, right? You should be doing continuous monitoring from the audit logs and traffic and then improve the security posture by right limiting is one option or put new policies, put new security rules that will block this malicious traffic. So your application firewall is still fitting here in zero trust days and then indeed there is an opportunity. So how a zero trust application firewall would look like, right? Well first it should protect workloads by filtering and monitoring traffic between within board logs at the policy enforcement point. Then it should protect workloads from attacks from the outside like cross-site forgery, cross-site scripting, as I said, file inclusion, SQL injection, the typical ones. It should leverages wide network patches for zero day vulnerabilities, right? Sometimes you don't, you know that there is a vulnerability, you don't know how to fix it or even if you even know how to fix it like the, let's say the trip from the vulnerabilities fix at the library that library is fixed at the consumer and that library is fixed at the consumer and so on until it gets your system is a long path, right?  And that's why with WAF you can deploy, you can patch your network basically. You can say like all the, like if you think about log for sale for example, it was in a query parameters, you could say like, okay, I will block all the requests that the query parameters follow these rigs and then you don't solve the root cause of the problem but you have time to wait for it, right? It should allow to onboard legacy applications in a lift and shift fashion. Lift and shift fashion is basically getting your old legacy application and, oh sorry, getting your old legacy application and putting in the cloud, in the cloud, right? No changes because probably it's a legacy application you haven't touched for years or you don't even know how that works or you don't even know how to build it, you don't know the language, you don't want to update anything, you don't want to rebuild. So you just move it and put it here and then you need protection, right? Because that application maybe is vulnerable already. It was just that you have internally in an internal network and now you will put in a cloud where it's external and server somewhere that you don't know and people could potentially grant access there.  You have, you mitigate the risks, right? And that's very popular nowadays, right? Imagine a bank moving to cloud, right? They are not rebuilding whatever cobalt they brought. Flexible rule set based on application internals because as we talked about before, we said like, okay, we have the gateway that protects all these servers, but some of them might be PHP, some of them might be Go, some of them might be Java and then they get, we have to conflate all these and put security rules for everything. That's, as we said, it's problematic and it will probably add a lot of latency because you will have to evaluate a lot of rules. If you put in different workloads you said, you put different rules sets then it will be much easier and much more easy to maintain and not much easy to understand and digest, right? And also provide audit logs for further analysis and improve security posture through adaptive rules, right? If you run anomaly analysis on your audit logs, you could find new rules arising, new attacks, right? And then you can create new directives that will be back to the application firewall and then it will be a security measure, right? In an adaptive way, right? Yeah, and as we said, like there are a lot of people doing lift and shift, people in service mesh is requesting for web application firewalls, PCI compliance, which is the standard for the electronic payments is also requesting for a WAF. So there is a lot of opportunities for WAF in the cloud days, right? And this is why we believe that WAFs also plays a role in the zero trust days. And now I will briefly talk about this open source project that I work on, which is a WAF project. First of all, this open source, so yay. It's inspired in more security. And you might know that more security is going in of life in July, 2024. So KORASA is a good candidate for replace because it's compatible and supports Eklang. It's also focused on the Waspcore rules that before, which is like the newest version. It's gonna be released very soon. It has multi-platform connectors. It supports native Go, of course. Caddy, we have a connector for hub proxy. Then it supports Envoy, Istio, KONG, using AP6 as well. And they just released a new version supporting proxy wasm as well. Proxy wasm is a standard for web assembly and proxies. And it's fully compatible with web assembly. That means that you could run in different platforms without this Go library and this Go engine in different platforms. For example, KORASA Playground, which is a website where you can test rules, is running entirely in the browser because we're compiling in the web assembly and then run in the browser through web assembly. So that's also open up a world of possibilities. Plugable architecture, we have a plugins API for extending functionality. A month ago, one student from Google Summer of Code wrote a plugin for rate limiting in KORASA. So that was cool. And it's focused on high throughput and performance, because as you might know, when you have the WAF in your gateway or in your ingress, then it will only happen once. And then performance is, of course, a concern, but it's something you could accept. But when you run on every single warp row, then you need that performance is really good because then you will add latency every time, every hop. So it's performance driven. It's focused on memory consumption, on CPU consumption. And it's aimed to be running the critical path because it will appear in the critical path many times. For example, in the policy enforcement point. And finally, conclusions. Well, first, Serotrust isn't incompatible with other network perimeter-based security approaches.  They can work together and provide you more security. No single component or function will be sufficient to achieve good level of security alone, but they work collectively as one layer over the other to achieve protection. And then WAF still plays a role in the cloud days, that's for sure. If you have any question, I will be around. Thank you for coming. Thanks. Any question? I uploaded my slides to Speaker Deck and also the platform for the conference, right? Question? Yeah, I would say it's more like a trade-off, right? When you choose to run a sidecar along with your actual workload, let's say, you decided already that this is trustable, although maybe it might be compromised. Who knows? Maybe tomorrow they disclose a vulnerability in Envoy and we are done. But at that time, you can never trust that. It's like testing. You never finish testing, right? It's the same. You cannot guarantee that this is 100% sure or safe, but you make a decision that, okay, I will trust this. And then, yeah, there is no security measure between your sidecar and your workload, that's for sure. Because otherwise, the latency will be huge, right? Also, because of the nature of the sidecar, it has a lot of access to your actual workload, right? Or to your actual workload. So, yeah, you should trust, you should trust. You have no other choice. But that's why you call the implicit trust zone because everything there is implicitly trustable. But it's a moral decision. I mean, nobody can guarantee this is 100% sure. Safe. Any other question? Sorry, I couldn't. Yeah, I mean, you could use KORASA as a library and put it in a middleware in your code. Then there is another concept that plays, that comes into play, right? You want to leverage WAF as a wide organization policy. And then the easy way is to use a sidecar, right? Because otherwise, you will force, not force, but you will request that all teams will include this middleware, which, for example, here, KORASA is written in gold. What if you don't have gold services or you have different service, multi-language service? Then I think that's one of the key points of having service mesh, right? That you want to leverage wide organization policies for everything without being intrusive about them, without requesting them to do this or that, right? So teams can just focus on the service itself. You had a question? Why for the sake of WAF you need another software? Why just is not part of the Apache or NGENIX or, like, yet another model? Yeah, because, yeah, for example, more security was like that, yeah? You will put more security in Apache as a plugin. And that happened when Apache was the main orchestrator of your endpoints, let's say. Right now, what you have is that you have one gateway, but then in the days of Apache, when service A communicated with, or endpoint A internally communicated with endpoint B, the communication will be direct. With zero trust, you don't want that. You want that they both are encrypted, mutual TLS, for example, that the traffic is also analyzed, for example, imagine that you have service A and service B, and someone gets into the container of service B and starts doing CUR, or start modifying, if it's an interpreted language, start modifying the call to do certain malicious calls to service B, so you don't want that trust. And at that time, the security that you put in the main proxy is already gone. It doesn't come to play, right? That's why you will put that on every single workload, let's say, that's the main thing. Yeah, that's, yeah, that's, I would say when you have, let's say when you start a green field, and you decide, okay, we are using this language and whatever, then things are easier, but as soon as you move to, or you have legacies, or it's a bigger organization, or you have high throughput, so you need to decide, okay, for this application, I use this language, for this, I use Kafka Streams, for this, then everything becomes heterogeneous, and then if you don't have a consistent way to deliver these policies, then it's like not having policies at all, that's the main problem. Any other question? Thanks for coming, I really appreciate it. Thank you.