Greetings, everyone. Today, I'll be giving a talk on Open DataLogy, which is an open source data license compliance project. But fair warning, this is my first in-person talk after the whole pandemic. So I'm really excited, also slightly nervous. So before we start as a note to the past two years, can everybody see my screen? Great. A little bit about myself. So my name is Gopi Krishnan Rajpahadur. And I started my career as a programmer analyst in CSC in India back in 2012. And then I started doing cobalt programming, actually, for two years. And then I got tired of cobalt. And then I went into data science. I was an engineer doing data science pipelines. And then I moved into a role that was senior software engineer. But I was mostly developing data science pipelines. And then I did a bit of research for four years in Queens University. And then now I work as a senior researcher in Huawei, where I look into AI engineering, data set licensing, and all of this fun stuff. So I've been working with data for a while. And that's why I feel comfortable giving this talk. So it's a few disclaimers, which my company really wanted me to put in there. All of these views are the views of the contributors, not of the company. That's it. I'm going to rush through this. Open Dataology. Like if there's one thing you take away from the talk, this logo, this name, that's good. Like I won if that happens. But why should you care? And I'm going to convince you over the next 20 odd minutes, why should you care. And the next 10 minutes, I'll tell you a little bit about what we did, which you can possibly find. So the recent, like software at the world, past tense, AI is eating the world, present tense. And this whole revolution is kind of fueled by the availability of massive amounts of data. As you can see from a lot of evidence, news articles, research that it's a huge market. And this is a graph that was put out by Andrew. And when deep learners started becoming popular, the performance of deep learners was purely attributed to the availability of large swaths of data. And almost nothing else. And how do we get this data, typically? There are only a few ways, actually. One, we can go buy the data from someone, procure it from a third party, they sell the data. Or one can go ahead and create it from scratch, which is actually neat, like you control end-to-end everything, it's nice. But both of them are expensive, time-consuming, and frankly, too tedious. And so what do people mostly default to? They use an existing dataset. They either download a dataset that's already available, nicely published like ImageNet or SeriesKF, CIFAR, or they go hit these different search engines like Google, Flickr, and curate their own dataset for their custom purpose. However, as most things in the world, they have licenses. And each license has rights and obligations. So for instance, rights are what, rights tell you what you can do with the dataset. Rights in a license tell you, these are the things that you're entitled to do with the dataset, and these are some things that are not allowed. And obligations are, if you use that right, or if you cash in on that right, what are the things that you have to do so that you can continue to enjoy those rights? So for instance, down in the here below, I have the license and obligations of some common datasets, sorry, famous datasets. And so all of them, at least most of them, have some sort of license out there. And in general, we can abstract it out to license as rights, license outlines rights and obligations. And in this project, we, sorry, yeah. So our project is about making sure that the rights and obligations outlined by the licenses are indeed valid and they can be followed. And I'll tell you why this may sound very familiar to most people who have been working with open source for a long time. You have, all open source software has license and why am I making a fuss about it? Why do we need a whole project? Yes, this is why. Unlike open source software, conducting license compatibility analysis for datasets are a little bit hard, primarily because the provenance, lineage and license related challenges differ from open source software because in software, the IP is clear. The copyright is fairly clear. However, when it comes to data, it is not. Like who created the software, who created the data, who owns the data and who curates the data can all be three different parties. And they all have an action that they perform, which means do they own the IP? We don't know, it's an open question and we don't make any claims here. We are just saying there are risks. And doing this analysis is hard. And I'll tell you why is it hard? So some of the provenance related challenges are like unclear licensing range. I'll go through each of these challenges. This is just a slide that kind of gives you an overview, gets you hooked in and we'll go through it one by one. So provenance related challenges. One of the first provenance related challenges is unclear licensing range. What do I mean by that? Let's consider this data set C for 10. And you wanna use C for 10 in 2022, which is now. However, this data set was created in 2009 and this is all they say about license. They say one line which says that please cite if you use this data set. And this was created in 2009, which was actually created from another data set, which was called 80 million tiny images. And that was in turn created from a lot of these other search engines, which were hit, we don't know when, because none of these papers documents that. So the license from an arbitrary range there should apply because licenses of many of these other search engines have evolved now. And we can't use the license currently. So we need to find the ideal licensing range, which is a very specific challenge. And then unclear licensing locations. In at least in source code these days, it's fairly straight, there are only a couple places where you can find the license. It's either in the source package or in a read me file or in a license.txt. However, for data sets, it's still very wild out there. There are sometimes the licenses are present in a completely different webpage or sometimes it's only present in the paper with which it was published or sometimes it's not at all present. Like sometimes there's just a note that says to do this if you're using this data set and that's about it. And the other problem is many of these data sets because of the pervasiveness of machine learning models, they're available in so many different platforms that each platform can sometimes even slack the license of their own. For instance, because C410 license is not very clear. Some of these platforms say it's licensed under MIT. Some say it is licensed under Creative Commons. And that's a problem because for anyone who downloads it, they might just think that that is the license that applies while the source is not clear. And for lineage-related challenges, this is actually interesting. For C410, if you just read the paper, they say, oh, we created it from 80 million tiny images. However, 80 million tiny images data set was actually created from several different sources. And if you're using C410 and if you're not doing a deep dive on it, there is no way you're gonna find all the sources that are associated with it. Which means that all of these license actually applies to C410 data set, but it's removed. And like nobody knows unless we go through and analyze it. And another challenge which we are not tackling as a part of this project yet is identifying the minimum licenseable unit. So many of these image data sets or many data sets have different levels of abstraction. So for instance, these have, so C410 was created from 80 million tiny images. 80 million tiny images was created from all of these different search engines and each of them have an image which has a license of its own. So which unit do we consider? Do we just say, okay, let's just go till Google and after that it's unmanageable. So we're just gonna stop there. Are we gonna say that? Or are we gonna say every individual image license should apply and then final license should be a representation of all of it? We don't know and identifying this is a huge task. And finally, the license interactions are not very clear. Like how MIT license and LGPL interacts is fairly well studied, but how please site if you intend to use this data set and this custom license interacts is not straight forward or clear. So these are some of the common challenges. So open data algae. Now I hope I have convinced you that this is a problem and our project tries to solve this through a few different things by a license compliance analysis process and putting all that metadata that we get out there in a portal so that people can come and use the results and if people do this process, they can share the metadata and they can contribute and as a community, we start having a repository of what are the actual final rights and obligations that are associated with each publicly available data set out there, eventually, but at least to begin with some of the common ones. And I'll tell you how we kind of can do it with this project for first publicly available data set. Let's to make this easy because I usually understand things through examples. Let's take an example where an AI engineer wants to use this HIFATN data set for either to commercially distribute it or to release a product with AI model or to commercialize the output. And because the license starts VE as I've been repeating just says please cite it, he is not clear. So he wants to go through this, we suggest he goes through this process and that would enable him to analyze the final rights and obligations. For instance, the process looks something like this and I'll walk us through it. So first one extracts the license and the key thing here is they do their best effort possible to identify it from the source, but if not wherever they download the data set from they capture the license there. And then in the provenance extraction step, it's key to identify the official source of the data and see if the data set source, sorry, the license that they originally obtained and the license that was provided in the source are the same things. If not, we suggest them to file a PR or do something to change it. But if not, at least now, you know, and kind of mark it as an untrusted source and use this official version of the data source. And we have some templates with which we suggest that one should extract the metadata associated with the official source. This helps us kind of mitigate the non-standard license location problems. And then the fun part, lineage extraction. This, right now we do it manually through just elbow grease going, we trace the data set creation process by reading the paper, identifying the sources. And then to do the licensing range, so far in the project, what we recommend is set a, like find the top level source, say for instance, if it's three for 10, for 80 million tiny images was published in 2006. So hopefully the license range of all the data sources around 2005 to 2006 is what applies and we recommend one to use wayback machine or something like that to extract the license from that time and decompose the rights and obligations. And once these details are extracted, most AI engineers are able to do this. We suggest that they put this in a standard format which I show, which I will show, and in the data provenance table and the data lineage table and pass it on to someone with legal expertise. Eventually we hope this is a database where someone can submit this metadata and ask if what is the final rights and obligations and there is enough atomic pieces in this database and the interaction analysis is done that results can be automatically provided but for now we have six data sets. So that's not a lot. So we suggest for the license compliance assessment, the lawyer or someone with legal expertise takes this license of the different sources and puts it in a format like this. This is a format that we came up with which is based on Montreal data license which was proposed by the Mila group, the Ayrshire Benjia group in the University of Montreal and we extended it for few more use cases. We blew kind of represents where we extended it but we are promoting this format because this acts like a good intermediate between the AI engineers and the legal team. So AI engineers have enough expertise to populate these fields and identify or consume these fields and identify the obligations that are outlined here and legal team can always generate legalese from this template and this acts as a good intermediate representation and a means for customizable licenses for the future. And once that is done, the lawyer puts all of those different licenses of different data sources in this format and then does a first order interaction analysis with legal expertise of course and finally ascertains what are the rights that are actually allowed. So this is an interesting analysis because if we just decompose Ciferton license like the top level license you would see all rights are allowed. However, after all the interaction analysis that we did we find that commercialization of the output or commercialization of the model or tagging, distributing and representing the model commercially is not allowed and so many people use these data sets currently and here are the results of our analysis. So our process can be adapted and retrofitted for pre-curated data sets at one curates two. The only change that they have to do is now if someone's creating their own data set they know what the provenance is they know the sources that they're hitting and we suggest that they use our templates that we provide to record this provenance details the lineage details and rest of the process remains the same. So that is the overview of the technical stuff or the crux of our process. I'm going to take us through what we have done so far and how we could use the community's help. Yeah. Our perspective is the, then after two or three months the same page or something is missing. So how can you... I'm hoping that the analysis that we do will always be present. We'll make sure that we maintain it but we can't speak for the data set creators. I'm hoping the papers that they publish are in archive and archive is immutable, like it's always there, right? Like and this is... Once again, this is... I don't know a good answer to that. That, but as the problem... The source of the problem is the fact that a lot of the people who created these data sets have not been very diligent at assigning a license or maintaining it. And we hope it changes with projects like these. So, yeah. So as I was saying, the current progress, we put it into four pillars. One is the processes that we need to establish. We work on that. And then we develop some tools and automations to enable those processes because if we say you have to manually do all of it, like beyond three months no one's going to stick around. So, and then we want to build a community around it as you guys are already probably aware, this is a massive, massive undertaking. It's not possible for one company or one research group or any one person to do it. And this is the need of the hour. It's a need of the hour. And we hope we can recruit the community, get them excited, spread the awareness that there are risks associated with using these data sets and can get people in. And we also want to develop standards around it.  They did a lot of work, a lot of nights went into it. Thank you. So current progress is this process, this beautiful process. We tried to publish a paper around it. It's still on revision and you can find it in archive. And the results. So we always six commonly used data sets. And if you have been developing AI for a while, you might recognize most of these data sets. And this is what you can do commercially with it. But I can tell you, if you go to certain pre-trained model sources, they are charging money for some of the models that are trained on these. And I won't name names, but yeah. And to enable community building, we put all the metadata that we collected in a portal and all the final rights and obligations that we have analyzed in this portal. So it's free, you can go look it up and contribute to it if you do any analysis. And we decompose the final things into this nice, you can do this, you cannot do this, these are the limitations and these are the obligations. So it's easy to consume and nobody gets confused hopefully. And yeah, this is the provenance metadata schema that we have developed. So all the provenance details can be captured and the lineage details can be provenance details of different sources that is involved in the whole dataset lifecycle can be captured. And they're open for collaboration. I have the GitHub link both in the first slide and the last slide. And I'm happy to share it with anyone. We are actively looking for volunteers. And as for tools, I showed you that nice dataset license template. So we came up with the very alpha version of a tool which asks you certain questions, saying that, oh, do you wanna do research on this dataset? And it guides you through it to create a license for your dataset. So we hope the datasets that are being created will use this tool and they can use this nice extended Montreal dataset license so that it is easy to consume and it becomes a customizable license. And this is our tool. And we currently have rudimentary capabilities of generating the SPDX to work with SPDX in the future, a SPDX-like markup of our metadata. And we are actively working with the SPDX community kit here to have a data bomb. And once the data bomb is there, all this metadata would be represented there and there should be full compatibility with tool support. And these are the formats I've been talking about for a while now. The provenance, we suggest that right now be recorded in this format and the license be recorded in the Montreal Data License format. And once the data bomb is there, we would ideally want all this provenance detail represented in a data bomb so that it plays nice with the rest of the world. And we welcome feedback. Through all of this, any time if you have feedback, reach out to me. I'm online most of the times. And yeah, for the standards purpose, we've been developing an initial data bomb in collaboration with SPDX. So this is what we have done. And now this is the fun part. This is what we are trying to do there. We don't want to stop at licenses. We wanna extend this project to capture all the problems that come with copyrights, privacy, ethics. Yeah, it's a massive one to take. But hopefully enough people realize that there is a lot of things to be done if we have to use these data sets. And these data sets are going to power the next decade or so for machine learning unless we are going to radically move away from supervised and unsupervised learning models as a thing. We are going to need data on a lot of them. And all of these have to be answered, especially with regulations coming in and all the government starting to put the screws. And we realize that most of it is manual right now and that's hard. But we had to start somewhere and that's where we started. But we wanna start developing tools that tools like data clone detection which will help us analyze the lineage. So to identify and to automatically extract provenance lineage and do the final interaction analysis. For instance, for automatic provenance extraction and lineage extraction we have, we are looking into NLP methods to read through these websites, understand the semantics of the website and identify where the source is. Or read through the paper to identify where the official source should be located and stuff like that. And for lineage analysis where multiple data sources are involved and this has been around for a while now, we can use data clones, we can use data clone detection which kind of goes and finds that is my image a part of another, is the image in my data set a part of another data source like Google or Facebook and then say that, oh, this might be a potential data source for it. And there are many different types of clones if anyone's worked with code clones, it's like actual exact duplicates, slight modifications, or is it just a, is it something that someone took it and modified it so that it's a different source. So there are many different types of duplicates and the challenges that we have to deal with here, but we are currently working on tools that will help us detect at least type one and type three clones which is index data. So type one is clones that are just duplicates and type three is clones that have modifications or new additions to it. And this is hopefully the timeline we have, but we wish, we are very open on all forms of contribution. If you wanna come work with us on some research if any of this sounds very interesting, we are very happy to do it. Or if you wanna help us write those NLP codes to extract provenance or extract lineage, we'd be super happy.  And the licenses are hard. Legal language is clunky. We don't know enough and we can use the help. And that's my next pill. Like we wanna establish a governing policy which we are trying to do with the help of the LFAI and Data Foundation. I'm happy to announce that our project is sandboxed with the LFAI and Data Foundation and they are helping us with some of these policy creation and wiki creation. However, we can't officially put it up yet because the paperwork hasn't gone through. So and finally, we are also trying to take all of this knowledge and create awareness and at least from this point forward, make the world a little bit of a better place by working with standard communities to create standards. And I've been working with Kate Cairne to form this AI bomb profile where data bomb is also a part of it. And hopefully this creates more awareness. So the goal is SPDX will eventually capture all the details that are associated with data provenance, lineage and the license decomposition. So that all the sources are tracked, all the official sources are marked, all the licenses are available. And once it's all available, it's just hopefully a matter of doing the interaction analysis which I hope the platform grows enough to have enough interaction analysis done so that it can be codified into a first order logic or second order. So this is the look ahead. We have a timeline. We wanna do all of this. We need help. And if you wanna contribute in any way through your time, if you wanna just analyze some of the data sets, provenance, lineage, just mark the data sets up or join the discussions, just shoot us an email or join our Slack. We are very friendly bunch. We respond promptly and we will be really grateful. Thank you for coming to my talk. I know it's the last day of the conference. Thank you, super grateful. Yes. I wanted to say thank you for the presentations and for the community, what you are doing. It's actually great and what is needed for the environment, let's say, and also a good question apart from this data set licenses question can be a question that was actually raised by Daphne, their co-pilot. Yeah. Is it polite to use the code itself in this highly connected with the open source itself? And it also can be a good question That's almost one of the motivations why we started this project. Like, I don't like, I'm not a regulating body. We are not regulating body. So we don't know the answer. There are different countries can decide differently too. Also with co-pilot, I don't know the full case, but some of the things that I've read about some of the license terms were violated and that is not okay. Copyright is a different problem, but contract. That's a different problem too, right? And we are right now dealing with contract. And so your question was many of these data is being used and the copyright is not clear and what happens then, right? And that's why we need to do copyright compliance issue. We need to tackle copyright compliance issues, but also we don't wanna just rely on one government body to give a regulation, right? Like what is in US might not apply in Canada. So our project aims at identifying the risks. It may or may not apply to you, but it's always good to know the risks, right? And that's at least the goal. Yes. So first of all, we had presentation, I mean. Thank you. For something which... You should contribute. Join our Slack. Definitely. So there, are you also considering this? So are we also considering the... I think copyright and contract together should address that, but we're not explicitly considering patents right now, especially because that database is not accessible. Yeah. So hopefully enough lawyers get excited about it and they can help us with this. Are they different? Is ethics should be common everywhere, right? No, I understand. I think the ethics itself is the same ethics. We will hopefully be able to codify it to look what are the problems that are happening in datasets. But right now, as I said, there are a few contributors. We have limited expertise. We need people. And once there are people, we want to tackle ethics. We want to tackle everything. We want to make the world a better place, but we don't have the people right now. Please, China's. Thank you. So what's the relation between the standards and open data logic? Which standard again? Yeah, yeah. Oh, so, okay. Your question was in Q4, there's potentially a standard, what is the relationship, right? So open data logic relies on a lot of metadata to conduct these data license analysis. So we are working with the standard communities to create the standard so that all this metadata can be captured going forward and open data logic can easily use this data to compute the risks associated with the datasets. Because without a standard form of capturing this metadata, right now we are having a hard time collecting this metadata. Yeah. Because standards take time. So it starts creating now, but it will finish. Yes. Okay, thank you. Yeah. This is part of those PDFs. Yeah. Yes, sir. Yeah, it's a bit of a side question, but I wanted to connect to the co-pilot question. This is something that I'm actively working with OSI to develop a new license around. Because right now, legally speaking, you cannot sue someone for their code unless you demonstrate intent. And generative code has no individual to type intent to. So we need a new process to define where the legal agency exists within that space. So if you're curious about that, I would love to chat about that so we can get that done. Yeah, just to repeat to the audience, OSI is trying to work on a license that addresses the concerns associated with generative code and sales looking for people to work on it. Any other questions? Or if you, yeah, please take a picture, shoot a message, and if you know lawyers, bug them into joining us. Well, once again, thank you so much. The project's name is Open Dataology. I'm Gopi. I'm around. Come talk to me. And let's make the world a better place one day to set up time. Thank you.


There is so much fun. Oh, hey! Give me a minute. Hello friends! Today we have the new MatePad 2022 by Huawei. That's a 10-inch tablet which looks like it's been made for fun, but it can do much more. We've totally waited for you, so we could open it together for the first time. Let's do it. Inside the box you find more than the standard stuff, with the device itself, a cabling charger and a keyboard attachment that also acts as a case. The charger is branded as a fast one, but sports a limit of 22.5 watts. The tablet has a pretty standard design, with an aluminum enclosure for the back and sides, while the display is covered by a slab of Gorilla Glass. Despite that, this isn't the most durable tablet out there, so if you were you, we'd look after it. The bezels on the display are moderately big, while the tablet itself weighs only 460 grams and has a profile of 7.5 mm. The camera array includes an 8 megapixels front-facing camera and a 13 megapixels rear-facing center, with autofocus and a LED flash. On the sides we find the I.O., which includes a single USB Type-C port, along with four speaker cutouts, which hide Harman Kardon Tuned's tweeters. Only 2 per 100 people watching this video are subscribers. If you decide to just start following us, we'll be able to reinvest more in our laboratory, thus making even more helpful videos for you. Thank you, you're awesome! The keyboard attachment has a dedicated USB Type-C port for charging, along with an impromptu kickstand, which essentially transforms the tablet into a tiny 10.4-inch laptop. The laptop runs the Harmony OS 2.0, which looks a lot like Apple's iOS while being built on the Android kernel that is open source. Since Huawei devices can't use Google services, you're left with the App Gallery, which started off strong but has stagnated its progress as Huawei's market share keeps dropping and dropping. The OS looks good, but lack of apps can hinder even the best-designed software. Thankfully, for the more tech-saving consumers, you can download APKs from the internet and install them manually. The battery life is good enough with the tablet managing to last through a whole workday on views. Sadly, the Kirin 710a SoC on the inside isn't the most powerful, sporting 8-arm Cortex cores in a big little configuration. When compared to other tablets, this one is at the bottom of our benchmark tests. The same can be said about the Mali-G57 GPU. However, while performance isn't the best, the stability is through the roof. With there being no thermal throttling or increases in temperature, the hotspot of the tablet reached only 31 degrees Celsius, which is fantastic. In our opinion, spare yourself the hassle with the main part and get yourself thumbed in more reliable and not under-regulations or political restrictions from the entire Western Hemisphere. While the tablet is totally usable, most of the time you will have to go out of your way to look for solutions for problems due to the lack of Google services. Thanks for staying with us through the review and leave a comment with your opinion on the MatePad 10.4. If you want to see more tests and details about the device, you can check out our in-depth review. The link is in the video description below.


Using filters in CINNL. CINNL is a nursing and allied health database. So in addition to general filters common to most databases, it uses filters that are specific to nursing and allied health resources. In this video, I'll show you how to access the filters in CINNL, and where to locate some noteworthy filters in a list. In order to access a lot of specialized health sciences filters in CINNL, you actually need to select your filters before performing your search. Start by clicking on Advanced Search under the search box to display all of the filter options. As you can see, there are quite a few options for filters, including some of the more general ones, such as the full text filter and the publication date. There are also quite a few health sciences specific filters. I encourage you to take a look at the full list to see what there is available. I'll show you a few of those specific filters now. So let's scroll down a bit and start with first author is a nurse here on the right. Use this filter to search for articles written by nurses. If using this filter returns too few results, try switching to any author is a nurse right below. This filter will search for articles where a nurse was part of the research team but not necessarily the first author in the study. Another useful filter is evidence-based practice. Use this filter to limit your search to evidence-based practice research articles, including sources like meta-analyses, systematic reviews, articles about EPP and commentaries on research studies. You can also filter for a specific publication type or types. Use this filter to limit your search results to particular types of materials such as systematic reviews, randomized control trials, and commentary. If you want to search for more than one type of publication, hold down the control or command key and then select multiple types. When I do my search, in this case, I'll search for privacy and choose the subject heading privacy and confidentiality. Now my filters appear along the left-hand side of the page. I can click the little X beside a filter to remove it from my search. In the list of results, I can see how many results I'm getting. In this case, I'm not getting very many, so I might want to remove one of my filters. I hope this video helps you use search filters in CINNL. Thanks for watching.


Okay, welcome everyone. So we'll go ahead and get started. Thank you for attending our session here where Nathan and myself will give you a rundown of building security frameworks into your CI CD pipeline. So a little background, my name is David Brock. I'm the product manager for the Compose platform here at Allstate. Hopefully you guys were able to stop by our booth in the foundry at some point over this conference and get to get a rundown of Compose. I'm here joined by Nathan who is a senior manager of application security and cloud security with an enterprise group at Allstate. And what we're going to talk to you today is how we are partnering together to kind of bring the enterprise security into what we're doing here with the cloud and how that partnerships allowing us to automate things and build a stronger platform. So with that I'll hand it over to Nathan. Alright, thanks David. Everyone, thanks you for coming here at the end of the conference. The last session where security belongs right at the very end. Appreciate it. And you know to actually talk about here in a little bit how we're gonna basically turn Jenkins over to security and and put you know laws into Jenkins. I'm joking. We're actually going to talk to you about that partnership where security is actually embedded but the product team is responsible for being a delivery organization and actually delivering product to market. So we're gonna go through some of that at some level. We're gonna get technical and actually look at some code. I won't go into too much detail on that but I want to show you guys conceptually how it's done. But we're gonna kind of keep it a high level to show you the full process stripe. So security is a huge domain of topics to get through in about 40 minutes. So as Dave mentioned I'm the senior manager of application cloud security at Allstate. My background I've been in security about 16 years and I've been primarily focused in application development and agile environments both scrum and XP methodology. So as we go through this you're gonna see some stuff that's kind of you know that it looks like a scrum environment. Some of it looks like an XP environment. A lot of it can be applied in either way. So just kind of keep that context at a high level that this is an agile methodology. So okay so high level this is a process flow with kind of a security perspective right. You guys are familiar with this in an agile world we have you know product managers who are coming up with you know requirements and acceptance criteria and kind of adding that to a backlog and then you have the various phases of portfolio prioritization, user story, breakouts whatever it may be where we're determining what's important and what we're gonna work on. And then all the way down to the developer level down here where they're planning and actually coding in five and six and then seven and eight. This is where we're really gonna get in the fun of it. That's where Jenkins at right and that's where we're gonna get in the meat of continuous audit and continuous inspection. So this is really about how do you get continuous audit added to your continuous integration pipeline. So we're gonna go through the stack basically the journey throughout the presentation. All right so the first thing is defined security right. That's all we hear that's what I hear a lot from developers you're just like you know what are my security requirements and security is like ah you need to be secure. What does that mean you know what does that mean at the operating system level what does that mean at the infrastructure level what does that mean at the application level right and as we move more and more into a platform society and DevOps some of those line site to blur on who's responsible for that right. So a lot of times you know requirements are assumed to be taken care of by your enterprise you know infrastructure guys because that's the operating system but security has no context of containerization they don't know what RunC is they don't know what Docker is right and that's another layer of abstraction that still needs to have security requirements kind of applied to it. So that's why what David said the partnership is important because you can't secure something if you don't understand it and what ends up happening is you have requirements that were originally intended for a traditional environment that don't make sense but somebody says I you have a firewall in there that checks the box on the compliance but it doesn't necessarily mean you're more secure and that's the biggest difference there is a difference between being compliant and being secure right and I'm interested in being secure so for security to partner with the developers actually understand the environment what controls need to go where so we don't end up with something like this right and this is you know traditional we run into this all the time of you know we don't allow our operating systems to talk outbound but you know within the containers have the ability to you know have Jenkins talk outbound and you know grab that that information pack it into a container and deploy it anyways bypassing the security controls which doesn't help the developer or the product or security right all right so this is how we define security I've been in security 16 years you guys ask me how do you define security I'll say I don't know much like your world it changes every day in my world you guys have new technologies new you know products and services you want to try out that meets customer needs the same thing with security our threat landscape is constantly changing our vulnerabilities are constantly changing and I can't keep up with it all these are my problem statements these are my user stories right these are what guides us and if you guys really think about it at the end of the day where do these standards come from NIST National Institutes of Standards and Technology it comes from a law called FSMA Federal Information Security Management Act let's think about where laws come from right the legislatures legislators are voted on by the people and the people themselves at some point in time were concerned about the security of their data and the companies that were you know supposed to be entrusted for taking care of their data and that wasn't happening so the people not through a product manager but the people through their legislators came up with these various standards or asked the legislator to work to come up with these various standards these are user stories these are user acceptance criteria they're not compliance so if we change our mindset a little bit and think about these is this is the voice of our customer it's an unspoken voice it's a expected voice you're not going to directly ask for the security requirements but that's all they are they're still requirements from our users and we're going to take you down into turning these into code alright so the first step is up here where we have product managers you know kind of looking at what that acceptance criteria is what the feature of a minimum viable product is right I'm going to take you the key with viable for a customer include security right so how do we define security within that viable MVP and the definition of that and that's what's missing and a lot of what I see in the enterprise is securities at the tail end when you're just getting ready to go to production you're just getting ready to release you have all your commitments you have everything lined up you think you're set then you have your pen test then you have your vulnerability scan then you have whatever it may be and you're that's when you find out that there's a whole bunch of stuff missing we need figure out how to get that to the front end of the pipe and how you do that is through partnership having security be a product delivery organization be there at your inception that's your user story breakouts defining scope for minimum viable product and that's the first phase we're going to talk about this is a user story right here given when then right Gherkin syntax so those laws those standards and this code but whatever it may be those are problem statements that you can actually translate into that Gherkin syntax and put into a given when then statement and here in a minute you're going to see how really powerful this is if you couple this with test driven development but in this case we have a basic you know privacy system message everyone saw that you know I've seen that you have to have privacy policy whatever it may be that we're going to collect X Y and Z information from you and down here this thing doesn't have a laser and I'm sorry for the resolution but down here you'll see that this is mapped to NIST controls so in this particular case it's NIST AC 3 4 5 and 6 right so we're actually giving user story mapping it back to a compliance requirement so when that gets down to the developer and the first thing I want to talk about I guess we'll step back we have user stories for the application we can have user stories for the application or for the operating system or the container right in this case we're talking about actually you know providing a denial services hack story at the network level and then you can have one for the database so sorry for resolution the point here is not to go through every NIST control because that's super boring I enjoy that but I also get to turn into the code so that's what makes it fun for me but in this case you have all kinds of acceptance criteria for the definition of a secure database right inside these NIST controls all right so now the other big thing is when you get into documentation right how many people heard where's your governance documents where's your standards where your policies your procedures right those aren't you know traditionally of things that XP your agile scum teams are delivering one would think but in reality you guys are creating a lot of readme's you're creating a lot of documentation for how you go about doing things to make sure it's predictable repeatable that's the name of agile right is you want to be lean fast automated and a lot of it's predictable repeatable well those documentations whether they're developer docs readme whatever it may be those actually constitute governance and compliance documents so your API docs is another one right those are standards for how to interface with your application in a secure way and a lot of people don't think that those map directly to those NIST controls which are usually the first two in every NIST family is a policy or a procedure or a standard you're creating those you just got to you know get a security person that's part of that product delivery team to put a pretty wrapper around that actually communicate that to internal audit to your security team whoever it may be all right so we're gonna get down into the actual paying attention time we're gonna get down to the actual iteration planning right so this is where the developers they have the user stories from all the product managers and we've mapped that back to the NIST controls or COVID or PCI or HIPAA whatever it may be right all these laws are just user stories we're gonna get into TDD so if we actually take the features that comes from NIST that the user has requested through us it just came through law and actually map it back to a NIST requirement and start to identify the feature the background scenario this is part of test driven development again this is business language right so this is something that an auditor can look at this is something that your internal security people can look at and if you guys how many guys are familiar with fitness or cucumber BDD driven framework so that's what I'm about to walk you guys through so this is business driven development here we're actually taking those those laws and those standards and turn them into BDD so this is the next phase right so we're getting down into actual executable code and the next slide we'll talk about it and I just want to give you guys a chance to look at this so we went from this high-level business statement we're taking it a step down right working on the step test at this point and then finally what you end up with is an actual functioning code that's mapped back to a NIST standard so how often how powerful do you guys you think this is from a from an agile perspective if I'm a waterfall organization actually the last time you had a NIST audit you have a compliance audit they're going to say oh last quarter say oh maybe last year last quarter best case scenario because those things take forever guess what the scrum XP agile guys get to do had a build that ran five minutes ago here's all 635 tests that are associated map back to NIST or PCI or HIPAA or whatever it may be and here's my build that it's green and more importantly your standard audits those important point-in-time pieces of paper they do nothing to prevent insecure or uncompliant code to get to production but in this scenario if somebody writes some code later on that you know happens to violate some security tenant that we had previously defined that was an acceptance criteria whatever it may be it'll break the build and prevent that from going to production that's powerful that is a super powerful thing to be able to attest to and to be able to talk to oops sorry about that this is just another example of a unit test it's kind of map back to a NIST control up top using tagging that's the other thing you can do is kind of search your code and actually report on how many tests and what NIST controls that they're mapping back to so continuous audit on the platform have you guys heard for CAS benchmarks so I'll give a little bit of background yes I'll give a little background on that so we get a lot of questions from a security what's the definition of a secure Ubuntu image what's the definition of a secure windows image you know what's the definition of a secure red hat or secure engine X whatever it may be right and there's a group of professionals in the security industry I'm one of them that kind of participate and collaborate to define what those standards are now are they perfect no they're meant to be more of a guideline but at the end of the day you have a CAS benchmark standard for you know defining what a secure image would look like or a secure you know web hosting platform may look like and I've heard rumors that there may be some stuff going on specifically Cloud Foundry on that but we'll see in the future how that goes but in this case you have centers for information security that have actually defined those benchmarks and what's cool is with the advent of chef puppet Ansible people have actually taken those those PDFs for those XML files that are published by CAS and put those into executable Ruby or Python or code whatever it is that you're using chef puppet Ansible and you can automate the deployment of a secure operating system and couple that with something like test kitchen you can actually again provide test-driven development again sure operating system infrastructure in your platform infrastructure as well not just the application so I'm going to pass it off so I've talked a lot I'm going to pass it off to Dave a little bit to talk to you about on our Cloud Foundry platform how we're actually putting this stuff into play yeah thanks Hapen so in this scene here you see a red build but what you're seeing is a automated scan of a stem cell that was pushed to PivNet was pulled down pushed to an old build pack in Bosch and we have some configuration to give a user and then using a Ruby gem that we partnered with Nathan to provide us we're able to do a scan of that stem cell and actively see the results now this you see there are some failures but the other thing that we're partnering with is not everything that it's identified as a failure makes sense in this cloud environment so we're work to define a filter so that as these builds come up we're notified of the things that matter in our environment and then we can choose to push to the rest of the environment and at that time we now have documentation that this was done in a past or it failed and we know why so this is something we're actively doing and we're expanding because not only are we interested in doing this for the applications that run on Cloud Foundry but the platform itself thanks Dave and I'll talk to you a little bit about how we're doing that and again Dave the Cloud Foundry expert you can kind of pull that together but the security tools in the security industry are evolving just like your tools are right so they have APIs security department may not know that or know what APIs are but as we move into this more continuous integration continuous automated you know continuous deployment environment you can leverage Jenkins to call your static analysis tools to call your vulnerabilities tools to call it metaspoil automated pen testing frameworks right and produce artifacts on the fly and give developers real-time feedback and also break the build obviously if new intro new vulnerabilities are introduced I will caution on that to select you guys know as these tools are also getting feeds obviously from the vendors themselves so a developer may check in code to add you know a widget whatever it may be that they want to add to the site and at the same time a new public vulnerability has been detected and published by the vendor right and the developer happens to commit the code just at that time as the signatures came down and the bill breaks but no fault to the developers because of maybe a third-party library you know that's sitting in their code some vulnerability that was detected in spring so you need to have the framework on the back end to provide for exceptions and those can be as simple as you know a JSON template that is documenting exceptions within the source code that you know just requires a simple pull request for a developer to update to get their build green and just make sure you add something to the backlog that can be prioritized like everything else to go back and fix that you know vulnerability that you found or that the the security tool found so those were kind of some of the more difficult things because your security people traditionally oh you have a vulnerability no don't deploy and be like you have a 30-day patching cycle I have a two-week patching cycle because I can get into my next sprint so you know I'm still going to deploy this code because I didn't introduce that vulnerability so those are the type of conversations you need to have and it's just an educational journey is really what it is so so we're going to get into a little bit box seven and eight this is where the continuous audit comes in it's really about hooking these various security tools up to Jenkins one of the things I want to talk about and Dave will talk to a little bit about is how we're utilizing deployed Actal which I'll let Dave talk about that and coupled with security standards such as OOSP and NIST to actually provide this automated gating protocol as well as many other things in our CICD pipe so talk a little bit about deployed Actal. Yeah so I know it was announced last year but if you have not checked it out you should. Deployed Actal is a tool that we've written and contributed to open source check out our GitHub project that allows for automated deployment blue green deployments to multiple foundations now our architecture is very complex so having a developer have to deploy to each of our foundations and check them and we're about change records and all that is just a lot of overhead and burden that they don't need to worry about deployed Actal is our solution to that allows for blue green deployment across multiple foundations so that they know that they've got the environment running exactly how they should across the entire production environment so as Nathan's talking about is we can leverage that and extend that have deployed Actal or conveyer and interact with other components and and other security tools to bring those in and provide that validation as well. Yep so internally just to make sure I don't get the names across we call deployed Actal conveyor internally so as you hear me talk about conveyor that's essentially what deployed Actal is but so my team came in and kind of you know worked with the platform team to kind of identify and really understand again the environment and how things are being deployed so we don't create you know a gate that just kind of has things going around it and identified that deployed Actal really looks like a central centralized point that is responsible for deploying over their cloud boundary infrastructure right so how powerful would it be to leverage deployed Actal to basically go off and check to see if the appropriate security artifacts were in place either required from a compliance perspective again I don't like that because I'm more looking at risk or you know from a security perspective that that the minimum acceptable risk has not been exceeded before deployed Actal or conveyor pushes that out over the production instance so in this case we're we're hooking deployed Actal up with with various tools so the first thing I want to talk about is licensing right so how many people dealt with in an enterprise you know legal and different people you know concerned about a pati versus MIT things like that right it's a very difficult conversation developers you know need to move fast they don't have time to sit through and read every piece of code so we've got some open source technology I'm sorry we've got some commercial vended technologies like black duck that has the ability for deployed Actal to actually reach out and validate that there are not any licenses that you know violate the existing licenses for the enterprise and that can be hooked right up with Artifactory Pro as well to be used when the developers trying to pull down that software the other thing that can be used if you want to do more of an open source cheaper out we have things like Maven that is already doing a lot of this information right so through some creative scripting whether Python or whatever you like to write we can actually start to produce those artifacts and storm in a central location and ask conveyor deployed Actal to check that that exists and parse that whatever it may be right so that's really how we start taking these traditionally manual processes where legal whoever had to review these things and you know manually approve or sign off on some form and try to and start to automate that within your CI CD pipe the next thing is automated pen testing so anybody familiar with W3F yeah so W3F is a pretty powerful dynamic analysis tool it does require a little bit of upfront configuration for each application because each application is a little bit unique but the developers themselves and the scrum team partnered with a security person can basically create a configuration for their application and stored in their source code and when they go to deploy to production again you can have W3F running inside a Docker container that you basically pass command line to along with that configuration file and say hit my application let me know if I introduce any new vulnerability and if I do give me immediate feedback so this is where you start to really get automated pen testing built into your CI CD pipe and again you can have deployed Actal check for the results of this and either say hey they did it or hey it found vulnerabilities don't deploy so there's any number of scenarios gradually over time that you can introduce you don't have to break the build day one in fact I encourage you not to break the build day one right hook these things up start gathering some stats and some data it should be a development tool and useful for the developers before you start breaking the developers pipeline because they really need to understand what's happening and more importantly how to fix it or who they can go to for help the next thing is a vulnerability scanner and this one's probably one of my more passionate things because people traditionally think a vulnerability scanner is just going to find vulnerabilities but it's much more powerful than that a vulnerability scanner has the ability to actually get into your box and really profile it and create artifacts on a number of different things that are important to security and compliance and the product manager you can create and automatically profile the users that are on the system and create that with every build and maybe do diffs in between them to identify if new users you know have been added for whatever reason you do it for the groups and the users that are in the groups a list of all the installed software you know RPMs that be in package or whatever it may be and bad configuration again your vulnerability scanner those CIS standards that we talked about your vulnerability scanner can really measure you and compare you and score you against those CIS standards and give you information on what ones you're meeting and what ones you're not okay good okay the next thing is not many auditors not many security people are not gonna want to go to Jenkins you know to view all these artifacts right in fact they get questions all the time what is Jenkins you try to explain it it's just you're not gonna get anywhere these are tools that they're used to seeing things in so RSA Archer is a what they call a GRC tool governance risk and compliance tool where from a security standpoint we can load those templates for those laws that we talked about NIST cobit whatever it may be and it's an easy way for them to map requirements from a legal compliance perspective over to solutions and then also measuring those solutions effectiveness right what are artifacts that we're testing proving that they're in place and they're not being exploited whatever it may be again tools like this are coming out with API's your security people may not know that your compliance people may not know that and and again we know what API's are and how to integrate with that so all these artifacts we talked about we've been talking about giving a developer view for right now but if the tail into your pipe right before you deploy you can work to deploy that artifact or test or whatever it may be out to RSA Archer again you're giving them a real time feed into residual risk something that they've never had before they're used to every quarter doing an assessment or some sort of penetration tests and that's the best case scenario a lot of times it's a year because pen testing teams you know are not very big and they're usually backlogged in an enterprise environment so if you can start to automate some of the stuff that they do and then take the results that they usually manually upload through a UI interface and start to automate that with Jenkins now you're starting to give them something they have never had before and it's a real time view into residual risk at the enterprise at multiple levels not just the application but the infrastructure as well as the operating system layer this is just a view from RSA Archer and again this is what they're used to seeing with their compliance and their scores and everything like that as you guys begin to add those artifacts they're starting to get a dynamic view which will really change their operations at that point and to be honest they're going to start getting a live stream of data that's going to force them to figure out how to be a little bit more agile right at this point they're saying hey we're not getting data they're not producing artifacts but as you start to give that more and more it's like okay now I have too much information how do I deal with this how do I start to do agile risk management agile risk assessment which that is a really cool place to be for an organization so something I'm working on and a couple things I forgot to mention this earlier so the the gym that they are using for hooking up into deploy deploy-a-dactyl it's specifically for rapid seven expo so if you guys have rapid seven expo within your organization there's a gem out there called the next post runner that you can actually download and it's basically command line access that allows you to do automated vulnerability scans in this case it's another one we've taken the actual whole NIST 853 which is about 385 security controls and actually you know jsonified that whole thing and that becomes really powerful if you guys are in a situation that you have to create system security plans and different things like that you can actually start automating Jenkins to populate the different controls which are essentially your artifacts from your build and producing automated system security plans so I know what's happening here well it's just jumping so anyways that's where we're at if you guys have any questions we're welcome we've actually got a full crew all state people here that can talk to you as well so thank you for your time Dave so thanks for joining us we'd love to hear your questions or talk to you see what you got yeah yeah so just to repeat everyone if he didn't hear what he said basically he's saying that the the timing of some of these scans take a while to run especially like W3F and sometimes our static analysis and how do you keep up with the pace of velocity continuous deployment while developers have to sit and wait for a scan to get in place and he's absolutely right that's a challenge that we deal with and one of the things I guess from our perspective we're looking at is what are the tests that we can run upfront that are really really quick and have those be part of basically you know automated gating to get into production and what ones take a little bit longer that we're going to run after the fact right and and things like a fuzzer for example I didn't talk too much about a fuzzer but sometimes a fuzzer can take days to run depending on the size of the application that isn't necessarily needed for all applications one of the approaches you can take is if you have you know one of the things in this is to kind of classify your systems you know this system is a highly critical application to the organization you may have a different policy and process for changes for that application saying it may require a fuzzer just because the sensitivity of that application where systems that maybe not are as sensitive as that one we do that after the fact after the deploy and basically true up if it finds anything at that point I think that's individual with each organization and kind of the risk tolerance so and then within the company you're gonna have a different risk tolerance based off the type of application so not always are you gonna run every single test I guess is the answer it's up to each group so yes sir I love it that's where good ideas come from spider doc I may not be oh oh swagger documentation wow I don't know the answer that question from a fuzzing perspective so traditionally what what a fuzzer is going to do is look at your inputs either through API's or or on the application itself and just nail it with an obscene amount of garbage to try and get 500s and stuff like that so yeah I had not thought of that so I'd like to follow up with you on that I guess so yeah I don't yeah I'm learning every day and to be honest you know everything that you guys see up here is because of partnering with developers sitting next to them and being open-minded enough to say yes I've done this this way for 10 years now and throwing that away and just saying okay guys teach me Jenkins teach me you know whatever it may be and you know conversations like this is where this has been derived from and it's not perfect it's still a work in progress their third times were you know still today that that we're challenged together and getting better at it so yes so yeah we're obviously working on that that journey and figuring out some of those things as we speak right now but what you're getting into is kind of like software-defined infrastructure at that point right and in this case Jenkins could and should still be used to control that that RDS right Amazon has API's just like everything else and that should be a software-defined deployment and if you do that you can actually you know create test-driven infrastructure that point to look for certain configurations for my sequel.com that's stored in the source code that that your Jenkins jobs uses to create RDS so the same concept you see here apply for the the database you know user stories and everything if you're using software-defined infrastructure deployed of the cloud this should still work and it has I can speak to it that I've done it before now I'm relatively new at all state and we're going through that journey here so I'm gonna learn a new way to do it I guarantee it won't be done the same way that I've done it past but yeah that test-driven infrastructure is really powerful yes sir yeah yeah so it's definitely balanced like you said security's got certain objectives and I think one of the challenges more of a communication education on what are the problem statements we're trying to solve with that and then security at the same time what are your problem statements so outcome-driven stuff not solution statements like this is what we've done all the time so that that challenge is real so to speak right and it's really going back and when you see people talking about solution statements or you know specific implementations to drive it back to a problem statement and at the end what you'll end up with is most of the time some hybrid in between that most people aren't either side is gonna be comfortable with but over time they get more comfortable with it because it's a fear of the unknown once the unknown becomes less of an unknown then people kind of buy on board and continue to evolve and grow that but in the beginning yes it's very challenging but it's really about problem statements and acceptance criteria more than solution statements and expectations I guess so okay well thank you guys for coming I appreciate it