th them I didn't write them but they're useful so so what we can use my tools for is to figure out is to try to figure out just how robust these things are to noise you know because that's if you're going to use these things that's actually pretty important how dark can you make the image so let's so let's do that so they have a lot of demo applications so this is the April tag detector I can ask you to produce V-LOGs I'm giving it an image so this is the image that I showed you guys and I can run it and bam there it is so so it produces a log file you know a whole bunch of stuff so I should say that in my day to day now I use these log files for everything I routinely make log files with hundreds of columns and you know I can't read them but the tools can they can pull out whatever they need to but this isn't that bad relatively speaking so you get the path to the image so here there was just one image you get how many detections so there were seven here and these are various metrics the hamming and the margin are the various metrics the detector gives you the ideas which tag there is which tag it is and these are the various pixel coordinates of it so right I don't need to say in that okay so one tool I have so one kind of problem with this is that as a human this is not very easy to read because the columns are unaligned so I can feed that to this tool and that will keep the data the same but just add some white spaces and align it so if I did that you got something like this so same data but you know I can sort of see what's what now which is nice okay uh useful okay so the so another thing I can do as a human is that I can tell this is right so you know there's some numbers with coordinates they sort of look fine you know I can manually open the image and sort of poke around in the coordinates and sort of see if it's correct but it would be nice to just be able to visualize the whole thing at once you know and then you can see that you can you'll be able to directly see if it's doing something incorrect so I pull out just the data I want to visualize in this case is the the ID and the coordinates and the center coordinates of each tag um there's a reason I'm doing a disorder um I'll get asked me later if you want to find out it's all in the documentation but trust me this is what you want uh so when you do that these are the results again you can see the you know some X coordinates some ID you can imagine I think you can see where this is going and then I can take this thing I just did I can pipe it through the plotter you know same tools before I tell it to I give it that image which will overlay the data spotting on top of the image so I can clearly see that it did the thing that I wanted to do and a couple of things that don't matter but the result is this so this is this is the image these circles are the things that plot it so they're kind of hard to see and actually turn this thing off you know so these are the circles that are the center of the detection the colors are which tag it is so for example yellow is tag number ten so this is kind of funny like why are there two of these and put the image back ah you can see that these things actually are mounted on this cube and there actually are two of the same tag on the side so this actually did the right thing and here you can actually see that it did and identified properly put it in the right place so April tags are good um yeah and you know and another thing that these sets of tools give you is that if you didn't have them if you were doing this manual you would maybe manually check two of the detections but here I checked them all right so if some of them were wrong if I was doing this without the tooling I might miss them but I didn't miss anything here I know that they were all right so it's usually you know often to clouds um okay so I don't want to get too deep into the details it doesn't matter so all right so let's see how robust this thing is to noise so in real life what I if I was doing this for real I would have gathered a whole bunch of images into different lighting conditions so if the dark thing is I can look at those things together so here I'm going to fake it I will just I will write a little loop to use image magic to actually change the brightness in the contrast of the thing and just add some noise so I will write my thing right and so here's the images made and you know I can go through them so I just I just made these right looking at this as a human like this looks like a pretty terrible image if it failed in this I wouldn't hold it against it but we can check so um so I just I have a separate image for each each one of these guys and the actual contrast number is in the file name so like this guy has a contrast of minus 30 and again since I'm faking it what I can do is I can I can write a little bit of a of a contrast so when you run this little thing you get this vanilla which just tells you the path and the contrast number so you realize I would have a little thing to actually look at the image look at how bright it is but for this for this talk this is fine um okay so I have that and let's also actually run the e-platform texture over all the images you know all my images I'm gonna do this and I could make a little loop and have a separate file of detections for each image but you know it's a choice the choice I made here is to just make one file without with everything in it and I can do this and it's gonna crunch for a while there we go so this is long this detects this tells me about detections for in my whole dataset and fortunately I have tools so as a human I don't actually need to read this because that would and yeah okay so I have two logs one has detections one has contrast numbers I want to be able to to compare them together in some way so I am going to join them uh GNU Corbytels has a join tool how many of you have actually used it nobody awesome anyone one person I have three people three once yeah so before before I did this I knew the tool existed because it's kind of a pain and you have to match the columns yeah so now I use it all the time because I have this guy Vino join so it's a wrapper around join so 90% if you want to learn how to use this tool you will 90% of what you need to know will come from the join man page which is not mine thankfully um and as with all these kind of wrapper tools the main difference is that any place where you would give it a number give it a name instead of saying take this file join column 3 and column 7 you would say here join on time whatever you're doing um another nice advantage of that is that you know if your log file's changed and then time used to be in column 3 and now it's in column 5 my command can stay the same and like that actually turns out to be very important because you can it's very easy to pass up so anyway so I'm going to call join I'm going to join the thing that tells me where my image is here are my two log files and then I will send it to this file so if I do this bam so the pad detection and if it goes over end I have one more column contrast so that data came from the other file so now I have one log file with all my data in it so now I can visualize stuff together um yeah this is an inner join for all your database people the join tool can do outer ones and various other things read them on page it's surprising and powerful actually um yep and you know as before the output is also a valid vanilla log so you can chain it and send it to other tools and all that all that wonderful stuff uh yeah so just because I can let's run it through a sorter so same thing it works the same as the other one except I'm sorting in contrast not on dollar 2 or whatever it is um and there you go so I'm not actually using this phrasing but this is just to show that that you can do this and I do this all the time now which I did not anticipate doing for um alright so let's actually let's see how well the detector works so let's look at the contrast so here I'm doing a filter so I'm going to pull out just two columns the contrast and the detection count um so the plus means throw out all all rows that don't have any data in the column um yeah and send it to the plotters so we can see it so if I do that I get this so there we go so this so the x is the contrast and the y is the detection count so it looks like here for contrast values either way too low or way too high you see only six tags but for the other ones these seven so it means that actually mostly works across the whole range you know even in the darkest image you didn't find zero or one tag you found almost all of them which is kind of shocking so so this already gives us an indication of how good this thing is but we have we have a lot more data so we can do more we can look at more things uh so I'm not going to get into details so first of all let's just confirm that it really did work in the darkest image because that image is awful so let's let's take a joint file let's pull out the detection coordinates but let's also so this pulls out particular columns but let's also pull out only rows for which this is true so you know I'm looking at the darkest one which is minus 40 so this guy you know will just take a file and give me only data about the darkest image and then just send it to the plotter so this is just a sanity check so there we go so indeed we have a terrible image and it actually looks like it kind of works and you can see that the the missing detection was this horrible one in the corner the one that looked a little ridiculous that it was able to find in the first place so if I zoom in on this guy like with the with the original image it was able to find the detection here as well so that's the difference between the 6 and the 7 and that looks awful but I'm okay with it not being able to find it okay so apparently it worked so one potential question is that how consistent are the hits so these are all really the same image so if this thing worked perfectly when it gives the coordinate of the tag they should all be exactly the same right so there's noise they won't be but we can we can visualize how bad it is and I can do it this way so this is pretty much the same guy as before except here I don't care about which tag it is I just care about where it is and upon I'm taking all the images together so if I zoom in on an arbitrary one ah so there's a spread you know for this particular tag so it's not perfect but you know this isn't terrible so the this resolution is awful so you know if I want to measure like the distance from let's say from here to here that that is 0.2 pixels you know that's pretty good and yeah and here I can turn out the image if I want yeah so you can actually see the color tell you which tag it is so one thing you might discover if you did this you might see an outlier you might see that there's one detection that's way off to the here that all crosses so you can't really tell but I can plot things with I can plot things with labels so if I do that and it helps if I do this okay sorry yeah this is a screen thing you don't need to worry about that so if I do that this is the same plot as before only except for X's it will print the idea of the thing so if it just will happen to have an outlier just tell me which one it is and then it can I know what to do so right so my mom was done so the okay so we have we saw that across a pretty wide range of contrast the detection counts are pretty good we saw that the spread looks reasonable and we can do more analysis to actually draw histograms and look at the distributions if we want I think that's fine for this so one last thing we can do is that if we look at the actual if we look at the actual output of the detector we can see that it actually gives you some error metrics like it gives you hamming distance and the margin so we can look at like we can ask it how well it thinks it's doing by looking at its own metrics and we can do that with the contrast and the margin and then I'm just plotting all together and you get this so you know this looks at all of the images and all of the all of the tags so the x-axis is the margin so how well the detector thinks it did and you can actually clearly see that there's a parabolic trend so in the images that weren't too dark or too light you can see that it thinks it did a little bit better and that's it so just sort of demos as as before there's these things more powerful and there's more things you can do but you guys feel free to ask me stuff or look at the documentation there's demos that are online and that's it any questions? let me get you the mic if you have questions will you share your org mode file on a will you share your org mode file on github I can where should I maybe in the repo yeah yeah let me do that I'll put it in the repo yeah do you have any examples where you're doing like top 10 in a column or in counts or any you know averages something like that I don't have any examples but you would do it exactly how you would do it so like if you didn't have a nilog how would you do it so if you wanted top 10 you would do a sort and then you would do a head that should do right if you wanted averages so what I would do without this I would write a little bit of org to actually do running some at the end divide so you I should say that the vnl filter tool all the all the modes you've seen using it so far are I've only showed you how to pick out rows and columns but since internally it is just it generates org and just runs it that's what it actually is doing you can just run it in that mode if you say desert eval then you quite literally just give it a bit of org where the only difference is that instead of say $5 you say time or whatever it is right so that's how I would write an org but inside the vnl filter so I don't have to remember which columns which could you display the top of the document that shows where the repo is just that that is a great idea right there can you do histograms and oh yeah oh yeah every day and I mean you know this is a trivial thing I would just do a stupid one for you sequence 100 pipe I can't see both sides at once that is a histogram so you tell which column is the histogram so columns here that's what we have and say bin with one so what do we expect yeah boom great here we go and actually while we're on the subject let's make it slightly more interesting let's say I want to say $1 times $1 I'm not totally sure what I expected let's find out probably I don't know $5 what did I do yeah wrong quote there we go okay so this is not interesting lots of accounts I need a bigger bin with hold on $50 ah okay I don't know why not anything else well thank you very much oh my god just kidding I've I'm kind of a plant I've heard that this is also in Debian oh yeah oh yeah yeah so both in Debian for a few years it's stable when the log is in said we are going to do a new release in a few months probably and it will be in that one yes yes yes yes thank you very much thank you the last talk for this track will be in 50 minutes back in this room it's swimming in the countywide data lake it is how LA County gets all of their data ready to process one two hello okay can you hear me okay good awesome we'll give just another minute or so here I don't know if you want to give anything special to say at the beginning or you want me just rock and roll however you want to go we can rock and roll all the other all the other speakers when I asked if they wanted to be introduced they looked at me like I had grown a third head and said don't why would you do that so me too I didn't bother to ask John so everybody this is John Neil he's going to be talking to us about the countywide data lake and I will let him proceed he did update these slides to be technical as opposed to governmental so hopefully this will give us the perspective we need yeah thank you yeah my name is John Neil I work for the county of Los Angeles as a program manager over a data science analytics team as well as electronic forms enterprise content management DI all kinds of stuff like that I'm going to talk a little bit about a journey that we're on with trying to provide an analytic platform yeah so I've done this presentation it started super technical and then over time I've been doing these presentations as a an executive type stuff and so I had to kind of put some of the I mean it's not like super duper technical but for the executives I normally talk to when I start talking about product names you can see so I'm going to kind of walk through this stuff and kind of give you an idea of what we're doing and what our challenges are and hopefully you can get some interesting information out of it so first of all just really quick about L.A. County L.A. County is really big and even I sometimes kind of lose sight of the fact that we service 10 million constituents we it's 4,000 square miles 88 cities so we had a conversation yesterday with some folks from the city of Santa Monica and then they were like oh but then we're working with the city of L.A. which is so much bigger and both L.A. City and L.A. County L.A. City and Santa Monica are completely absorbed by all of L.A. County so it's big we have a huge budget we have a lot of employees etc the department I work for actually is it does technical stuff development we have a data center for the entire L.A. County we also in my particular apartment also are the ones that have the department of public work so we fix potholes we are the pay master for the entire they're not pay master they're the contract person for the entire county so there's a lot of different areas that we that my particular department does but and it's really important that we are trying to become the the trusted partner and provider of choice and that ties back in because about what I'm going to talk about because if you don't offer something that people can use and want they go someplace else and the places that they're going aren't always the best places so so let me just talk a little bit about before I dive into this the nuts and bolts of what this is let's talk about the challenge of what we were trying to solve here and I think if you start with the technology first because we like to because it's cool you miss the point the technology is only there to serve and service and solve problems and so I have to keep that in mind all the time because I just want to dig right into the bits and bytes but so this was the issue that we dealt with at the county and still deal with at the county and that is that we are that we're data rich we have data everywhere but analytics poor and I point this out because if you look at the variety of types of divisions, departments and so on that are that we have to service we have family and well-being we have public public works parks and recreation beaches and harbors we have mental health facilities we have our general government operations the treasure tax sector the registrar recorder who does voting systems we have our public safety the sheriff district attorney fire department I mean you name it across the entire gamut it's crazy and every one of those is silos of data and problems that can be solved with data so this was the problem statement how do we solve this problem so see sorry I there we go so one of the ways you do it is think big that's a big problem with big things and this is sort of the agile you know manifesto approach is think big start small and learn fast and iterate quickly so that's a challenge at the county of LA I'm just going to tell you right now I mean think big I'm going to tell you that and the start small is kind of sort of what this is so this we have we have sort of a vision statement if you will and that is we wanted to create a flexible scalable and cost effective analytic platform to meet our customers needs and the unwritten piece of that it's not in there is I didn't want to do it with a one vendor solution or a we were going to try to do this on open source it's not written into our vision but it it informs it quite a bit so our vision is to take all these different areas every area has its own silo of applications data and so on along with their own data warehouses their own analytics if any their own reporting all this stuff sits on top of what it does or most of it does but that's this is what things kind of look like now and what we're looking to try to do is and again this is a vision is to have one place for all this data at least for the purposes of analytics and some sort of a sandbox or an area that you can do exploratory data science data analytics and so on and not disrupt the existing data ETLs data warehouses and so on that's the vision here's another view of that vision essentially bring data into and by the way you know having something called a data lake offers you all kinds of opportunities for every water-based metaphor you imagine so I've managed to find as many graphics and pictures I could that have water in them or in some way and of course you know oceans, harbors, reservoirs it never ends so the idea is to have a place where you have a large you can adjust data of all kinds a confidential area that you can do exploratory data science and so on try things out and then a place where you have published results stuff that your customers or consumers of the data can rely on so that's the vision here's the reference architecture that we use to try to meet this vision you're going to see this one a lot in this presentation I'm going to kind of point out all the different bits of it and so let me just kind of lay out the different pieces kind of high-level pieces of this and then we'll dig in so you have sources of data and by the way just about any data platform is going to have this these general pieces you have data sources you have ingestion you have a way to get the data in you have a place to put the data you have visualization and analytics on it you've got to have security on it you need to be able to govern it you need to have a catalog so you know what's in it and ultimately this stuff needs to be consumed both by modern tools and of course the county's got nothing but legacy crap everywhere so that too so that's and by the way this model it doesn't matter you know I haven't said where physically this is it could be in your basement if you want but these general pieces are transcend where it is so this is an eye test I'm not going to you know try to read every piece of it but basically we came up with 13 functional requirements in order to deploy this and we're going to talk about most of these in the ensuing slides but here they are all in one fell swoop the green first ones we tackled so the question is what do you need of requirements for I mean heck this is starting going it's so that you can have this and not that you don't want a dump a place where you just dump stuff without any any kind of no wow wow west kind of situation so let's talk through some of these individual requirements and our individual pieces and what we used or are using and and we'll kind of walk through this and try to not be too fast on this part usually I skim through these technical details or the the specific requirements pretty quickly because I have to do this talk in 20 minutes since I had more like 40 45 minutes I just pushed a bunch of more slides in so I still have to talk fast so the first one is this part right here of that diagram data storage so that's a key component and so data storage it needs to need to be able to put data sources in a single location for analytics use you have to have the infrastructure and flexibility to scale this out quickly and it needs to be cheap and plentiful I mean we're talking about not you know we're talking about potentially terabytes and petabytes of data and you need to be able to scale that quickly and it so what we did with that is essentially we have we use HDFS Hadoop it's kind of the big data this distributed file system with and you could use I mean some of these are what we use and other ones are just what you can use so open source Apache Hadoop direct the task storage with commodity hardware option so super cheap hard disk and lots of them Hadoop has built in it expects you to be using crappy hardware that's going to fail all the time so it uses triple redundancy on the data it's kind of it's self healing in the way it works but let's be honest there's cloud storage too and we have to address that you can't ignore it so in each of these cases I try to call out the common options where I can and where I can speak to what we're using specifically for our particular case we use Hortonworks Hadoop and you know it's going to be interesting to see what happens because cloud era and Hortonworks just merged and you know so but Hortonworks is probably the best distro of HDFS and you know Hadoop distributions but Hortonworks is to Hadoop like Red Hat is to Linux or Ubuntu is to Linux right nobody goes out and downloads the stuff directly from the Apache servers and stuff people go they get a somebody has built the distribution and you use that one because it's just easier and that's the same way with Hortonworks and the reason we use Hadoop is because we don't know what is what is in and you know we are not knowing what is what is what is what is what is what is our our everything is our the to be able to see my data, and I don't want to do that. And oh, wait a minute, you're going to put that in the cloud? Oh my god, you know, everybody's going to see it, right? So governance helps tackle this problem. There's several things in this presentation that talk about that. But you need to be able to provide governance on these data sets. We want to use our existing data stores. We want to be able to define tags that can tell what users can access what data, what information is accessible. You want to be able to create a catalog that has user-defined tags in it so that I can make up my own pieces of this as a user. And you want to be able to have data lineage. So a lot of what happens in a data lake is you take some data from over here, because you can put anything you want in here. You take some data from one type of data set and another type of data set, and maybe you get, I don't know, external information. And you've mashed that together and create value out of all that data. But at a certain point, when you look at that and you get your response, you want to be able to say, well, where did that come from? How did you put these things together? And what data lineage allows you to do is that you can still be the root place of where it came from you can track back to. That's really important when you start expanding the value of the data within there by enriching it with additional data. So that's kind of the governance aspect of it. The pieces that we use for that in the open source pieces we use in the data lake are Apache Atlas and Apache Ranger. So Atlas lets you define metadata about all your data, about your data. And Ranger gives you access and entitlement. So it lets you in to the data lake. It lets you know what you're entitled to. And those two things work together so that you can tag the data and say, I only get to see data from these three sources. So when I log in, even though all the data is there, all I see is what the data's been tagged for me to see. And it gets even more granular than that. Because, and I'll talk about this. Actually, I'll wait because I talk about that in another slide. But your ability to have three people see the same pieces of data or the same records but only see certain parts on it based on their entitlement. OK, so you've got to get the data in. So you have to have an ingestion. So first, let's talk about the sources. So prior to this construct that we built, we were pretty good with structured data. The county has a lot of structured data that we have databases, data warehouses, extracts. We have data all over the place. But it's almost exclusively structured data. So operational data and our ERP, our data warehouses, all the departmental apps, even external data that comes in from places like the DMV and other external entities. But we really weren't really good with things like video images. We store a lot of PDFs and documents. But we don't really have any good way to do stuff with that information. And when you get into more modern types of data sets, like social media interactions, mobile apps, no way to deal with that. And no way to deal really with streaming data. So sensor data, Internet of Things data. We run hundreds of buildings in the county of LA. And we have no visibility into the environmental systems that are on those because we don't really have a way to collect and deal with that data. So this platform allows us to do that. So those are the sources. Then the ingestion piece. So this allows you to onboard and bring in data. It allows you to set up a continuous ingestion of data streams. So you can bring in video feeds. You could bring in Twitter feeds. You can bring in regular batch updates from your data warehouses, from your data sources like your ERP and so on. But the other part is this ability to deal with large and viewer real-time data. So big data kind of is defined by, they call it the three Vs. So it's variety, velocity, and volume. And the data, the county has volume. We're okay on dealing with variety. We, our ability to deal with stuff and volume maybe not so much. But that's what those three things is what helps you define what big data is. So you need a way to ingest that. For the purposes of our particular use case, we use a scoop, a lot. Scoop is open source. Really great for batch ingestion, for setting up workloads and so on. We have the capability to support Flume and Kafka as part of our ESB, our enterprise service bus technologies. I don't know if anybody's really using it yet. So, yeah, let me just do a quick disclaimer.  In some cases, it's a matter of, I'm providing the capability of doing this. We have the tools, we've tested them out, they work. Do I have use cases that are really using Kafka or using Flume? No, not yet, right? That's the key, not yet. But without it, without this, you can't do it at all, at least on our particular and inside the county. The one we use a lot is NiFi. It's also open source. Originally, I believe developed by the CIA, I think. But it's really good with creating visual workflows of data. It's open source, but really supported mainly by Hortonworks. And that actually is a theme you see a lot in the open source world is, and I'll talk about it in a minute when I start talking about cloud stuff, where stuff is quote unquote open source, but literally the only contributors to it are one company, right? NiFi is not that way, but you do see that a lot and it's a caution, cool. So data security and controls. I talked a little bit about this as well, but you notice that this is kind of the security zone. So data comes in from these sources and the people who own this data and the people who wanna get access to this data need to feel comfortably secure or feel comfortable with the fact that their data is secure. So all of the security in this, in our platform, it sits over the top of all of these pieces, the data storage, the analytics toolkit, the consumption zone. We're gonna talk about data obfuscation in a minute and the other two, but I think it's super important because there is a perception you're throwing all my data in here. I need to make sure it's kept safe. So we need to be able to define specific roles, so role-based security. So this is some specific stuff to the county, so we have single sign-on. People don't wanna have to sign in 15 times, so whatever we do, we have to be able to support single sign-on on our existing active directory. So the county, again, there's a little county inside stuff, our employee IDs all start with an E, our contractor IDs all start with a C. So this slide is still left over from when I use presentations to say we're using existing credentials. So if you look at these slides later and go E number, what's that? So that's what that is. So, provideability for security, for view modify, for the data they only have rights and access to.  But you need to set up rules and roles and rights, just like any other data structure. So, and this administration needs to go over the entire stack, this entire, the entire middle part that the circle went around it. That's what, when I showed this, it needs to control all of this. Obviously audit logs, tag-based access for user interfaces and row-based security. So, the example of this is like PII and PHI, right? So, health information, personally identifiable information like social security number, that's information that we have about different organizations, about different people. So how do you deal with that? And one of the ways you do that is on ingestion, you can tag different pieces of data as certain data type. So, you can say PII and you can say, these are all the fields that are PII, social security number, birth date, and then based on those tags and the tags that you're allowed to have. So, our payroll people are certainly allowed to see somebody's social security number for various reasons. But Joe Blow, who's just doing a report on all of their employees, they don't get to see that. And the tags are what define who can see what. In a sec, we're gonna talk about obfuscation, but there's a way to do it in such a way that you can see that it exists, but you just can't see what it is. So, soup tag-based security is a big component of the Hadoop ecosystem and the tools that sit on top of that. So, essentially, you've got Apache Knox, which is a single point of obfuscation into Hadoop. You have Apache Ranger, which obviously we talked about before, which is what provides the access and entitlements within. And then, we have data at rest, and we have Kerberos for our cryptology, cryptography, to keep our platform secure. So, yeah, that's our security. So, obfuscation, a little piece right here. This was really important to us. One of our first use cases was around, we wanted us, we were doing a fraud use case, and one of the things was to find out was, was a county employee also a vendor, right? So, in some cases, that's actually a perfectly legitimate thing to do, but in other ways, if they happen to be working in the procurement area, and they're also a vendor to the county, that could be a problem. So, how do you deal with that? We still want to be able to search to see if an employee is a vendor. You know how you do that is, we have their taxpayer ID, we have their social security number, and taxpayer ID is social security number for a person, right? So, one of the searches we did was to match, and this is not like massive data science here, we're matching social security number with the taxpayer ID number. But, the people were really concerned, oh my gosh, you're gonna let this, you're gonna let the programmer, or you're gonna let somebody who's working on this see that information. And so, what we did was we used obfuscation, which basically took these keys, these numbers, your social security number, and turned it into an unreadable one-way hash. But it always hashes in the same way every time. So, that way, it sits in the data lake as your social security number, but as you're searching it and looking at it, it's obfuscated on the way out. So, that way, you can do searches and match that way without exposing that information. You could also choose to say, I'm not gonna make this available at all. There are ways to build in to do only the last four, so, but that capability is super important. So, and that is done straight into in ranger. So, row level filtering and dynamic masking is part of ranger. And then also as part of the Hadoop file system. Okay, we're pretty close to the end of the different components, just a couple more. Data consumption zone, this is one of the bigger pieces because this is where your data ends up after you've done all your fancy analytics on it. You have to have a place to report on it. So, basically, easy connectivity for analytics software. So, this is really some of the things I had a hard time explaining to my customers and partners at the county was the data lake itself, honestly, it's not an end user system. It's not a place that you go and wander around in freely and so on. It really is an expert system. The place that most developers, most BI people, visualization reports up, they're dealing with this data consumption zone. This is what comes out of data science. It's the end product. So, ability to get into whatever data you need, that you need to get access to. Access zones for different roles. So, again, what you can see, what you can't see. Self-service access to various areas. So, and you're gonna find these pretty familiar probably because, so this is this world, right? Your data consumption zone is Postgres, Mongo, MySQL, Apache Hive is basically a SQL front end on top of Hadoop. But, I mean, this stuff is pretty familiar, right? I mean, hopefully MySQL and so on. So, this is the world most people live in. And then finally, the analytics toolkit. So, this is where your, the actual applications that you write live. Multi-tenancy, so we've got 37 different apartments. So, if they provide a tool kit that lets them point their tools at the data. A sandbox, as I mentioned, sandbox really, this is a place where you can play around and make, you know, figure things out. So, provide analysts the ability to select from a variety of county-wide approved analytical tools. So, this is one of these ones where it gets a little interesting because, because I don't think it's my job to dictate what tools that you use to access the data on one hand. On the other hand, it can't just be every tool, any tool you can think of, because at a certain point we've gotta draw the line and say, here's what we're gonna support, here's what we're not gonna support. But, I want that decision to be made by the people that are using the platform. Not by me. So, this is, again, if your tool can talk to Hadoop, you're good. That's my feeling about it. So, and that's what we're trying, that's what we basically have built here, is it's a place for people to come, point your tools at it, have at it. If you can deal with Hadoop, and you can deal with this platform, you're good. And I would say most tools can these days. So, this world, I'm not even gonna try to like go too crazy, but this is our Python, and the various libraries that go with it, MathLab, PsyLab, Numbee. We've got a whole bunch of visualization tools. GGplot, plot we, we're also using Jupyter Notebooks. Zeppelin and or Jupyter, I haven't really let people, I haven't forced people down one pathway. Our particular team is using Zeppelin, but you could use whatever tool you want for packaging up your analytics and using Notebooks. This is, this slide could change every week, seriously. These tools, I think R and Python are probably, if I was gonna give you a directive to go learn something you don't know, and if you don't, and you wanna do analytics, if you learn R and Python, between those two things, for now, you could probably take over the world. So, let me just talk, so that's the architecture, how am I doing on time? Yep, perfect, come more minutes. Let's just talk a little bit about the, so we talked about the architecture and the functional requirements and the pieces of it. I'm just gonna show a little bit about what we actually did in our particular organization. So this, so of all those tools, here's what we have and what we selected. Again, this chart, this is our reference architecture. So, so we built this platform. It, we use, it's all, we support all the open source tools on it. We essentially bought a eight node cluster. It's kind of a big data starter kit from Cisco. You know, in the, depending on what world you're in, it was super cheap or God, it sounds kind of expensive, but I think the whole thing costs us like $80,000 for this. And it's bare iron hardware. It's not virtual. Literally, it's eight, like nodes. It's eight nodes that add up to 90 terabytes for like $80,000. And you can add terabytes to it at like a $1,000 a clip onto it. And we'll talk about the cloud in just a second because that's the question that comes up. Like why would you do it there? There's a lot of questions around that. But it's actually, it's an eight node cluster, five compute nodes, 160 cores, 512 gigs of RAM. And another thing about the way this works is it's, I mean, an analytic job that runs has access to all of that computational power if it needs it.  If a job needs 160 cores and wants to load a 500 gig database in the memory, it can to do its processing. So this is a little bit of a, this is kind of the diagram. Here's our Hadoop cluster. We have data sources coming in. Our key components. So we use Hortonworks. We're using a NiFi and Scoop for ingestion. We're using Atlas and Ranger for governance and security. Nox for security around the whole thing. The whole thing is perforized. And then the way this works is you've got this data in here and then you just, we have these VMs that attach to it to do their work. I called out these tools because these are some of the ones that are being used. There's actually people using SAS and we have a SAS connector for the crap load of SAS that we have at the county, which is kind of frightening. So, so here is our analytics portfolio. Again, Hortonworks Hadoop. Our studio and Python. We're using both Zeppelin and Jupyter Notebooks. We use Apache Spark. We're using Hive for just straight database lookups and SQL type stuff. We're using a lot of Power BI right now. Microsoft is, the county uses Office 365 and Power BI kind of comes along to the ride. A lot of people are using it. We're having issues with it though because Power BI is in the cloud and this stuff is not. And so there's some issues there. Again, Apache NIFI for ingestion. And then I have the SAS cognitive web focus. I have to put these in here because the, there's a bunch of this still at the county. A lot of legacy stuff. So the ability to figure out how to support that's super important. So what about the cloud? And so my question is, all right, well which one? And this is the problem we run into at the county is, who's gonna win this discussion, right? And so the approach of going with an open source platform is that we could, whatever, pick which one you want. And so by using tools that are open, you can, you're not locked into one cloud or another cloud. And so our, we haven't done it yet but in our roadmap is the ability to burst and scale into the cloud where we need to. Because it does take me a while, if I need to get more than 30 terabytes right now, I mean I can get it in a month or two, a month or two, right? If I need 30 terabytes like tomorrow or I need it this afternoon, I can scale up into the cloud to do that. And that's kind of the idea behind our particular approach. And, you know, part of the concern is we need to start in a spot where my internal customers were comfortable. And initially they were not comfortable on the cloud at all, at all. And so okay, here's an option for you. But now it's time has gone on, they're a little more comfortable and, you know, we're probably gonna have more and more people pushing that direction. So you can do all of the stuff that I described in Azure, right? So you have Spark, you have, they have what they call HD Insight which is basically Hortonworks with a Microsoft wrapper on top of it. So all the different pieces of this, you can do. The functional requirements that I laid out, that does not change. Regardless of where you do this, you've gotta have that. Amazon's the same way, right? You go to AWS, they have a data lake, they have machine learning, then you can move data in and out. And you can do analytics, of course. I mean, obviously all this stuff is super huge. But, let me just put a butt in here. And I kind of talked about this earlier. Open source, right? That's what this is all about, right? Just whole conference basically.  But what they are not is interoperable, right? So you can do whatever you want on Amazon and just recognize, well, this is Hotel California, right? You can check out but you can't ever leave. Once you get your data up there, once you start running your analytics up there in their structures, that's where you are. And that may be okay. I'm hedging my bets a little bit because we've got a religious battle going on back at the office around Azure versus AWS. Google isn't really in the play right now. Neither is IBM, neither are the other ones at the moment. But those two are like kind of shaking out to be the winners. And I'm trying to keep us open enough so that at some point we may just give it all up and go in there. Okay, so challenges. Number one, this is driven by use case, right? The technology is fun but you've gotta make sure that you're solving a problem. Otherwise you're just messing around, right? Again, wary of monolithic platforms and lock-in. All the vendors are now, they all have their data lakes, they all have their own analytic platforms. I'm dealing with a company right now who has basically stitched together a bunch of open source tools, packaged it up and is selling it as a piece of software. And I tell you right now that it's like, what about the analytics? Oh, that's proprietary special stuff that's our secret sauce and all that stuff. And it scares me because once you're in that world it's really hard to get out. Data sharing agreement. So challenges I had before work, it wasn't technical, it was, I'm not letting you see my data.  It's already in the open data portal. Well, I'm sorry, I'm not letting anybody else see my data. I mean, getting people to put their data in there is a big challenge and the part of that is, can you share it? That's a whole other talk, by the way. Data quality is a problem, right?  The county was talking about licensing cannabis. We tried to grab data about cannabis before legalization and the data's terrible. I mean, it's just terrible. We built beautiful dashboards and they're completely live, right, so you gotta have good data. The cloud, it's an opportunity and a threat. I mean, if you're in a big organization that already has a data center and already has infrastructure, why not? Why would you, why not use what you have? If we're just folks in this room, we're gonna start our own company, why the hell would we build this on-premise? Why would we go buy servers and manage this? We would totally do this in a cloud, right? So it kind of depends, it's easy to say it's an opportunity and a threat for us anyway. Data science is iterative, our customers are not. And so I read into this as well, where it was like, okay, we did a run through your data, can you tell me what you think? Well, we'll meet with you all in that in about three weeks and we can talk about it. Okay, three weeks, here's your data. Okay, a month from now, we'll meet again and it's like, if you want to be able to iterate and go fast, your customers need to be able to do that too. And the county is old and stodgy, so. So think big, learn fast, et cetera. So my takeaway is think big is easy, start small is easy-ish, it's important to find a champion, somebody who can help drive this. Moving fast, I started this project two years ago and so I'm kind of where I was a year ago right now so it gets a little disheartening. But it's there for people that want to use it and I think that's the important thing. So, I found this this morning, I had to put it on here. Have a Super Mario, March, March 10th, you only get to do this today, I thought, sorry. So actually before I do questions really quick, I want to do, you know, if there's some donut stuff in here, I want to point out data and donuts. Nina Kin and I work with these, with data and donuts. It's a monthly speaker series. Also, Hack for LA, Map Time and LA Counts. So check them out, there's fantastic LA resources. So with that, woo-hoo, all right, I took my 20-minute presentation and turned it into 40. We're good. Well done. Thank you. Any questions? Yes, my questions, are you nuts? Yes. I don't want to put you on the spot. Go for it, go for it. Do it, no, it's nothing crazy. But do you see potential in this tying into the open data portals to have closer to real-time updates of government open data? And have you seen any interest in county departments who would want to do that? So the answer is the open data portal, I look at that as a source for us, like one of the data sources. And a really, it could be output, right? It's a matter of finding departments that want to do that. And so the answer is that we have had interest in people loading their data in, and you can tag, that's where the tagging comes in, right? So the data comes in, you can tag it as being open, right? So now all you gotta do is expose all the open data and push it up to the open data portal. It is in the roadmap, is it being done? I would love to change the approach that your department, Nina works at the county, that's why she sends you her notes up, does for their open data and do it exactly what you just described. Put all the data in the data lake and pull just the open data out and expose it to the world, so yeah. Department of, they want to do it? Possibly Department of Public Works, yes. So, yes, exactly. I just want to get your opinion on where the project stands as far as maturity and adoption, like where you are, where you want to be. So the program right now stands, I'm trying to figure out how I want to describe this. It's, so the fact that it exists has opened up the conversation for us, right? So before they were like, man, nobody understood it, nobody knew what it was for. Now I've got that part, and as we've shown value in it, you're getting more and more customers. So we're starting to use it as a sort of as our sandbox. Well, it's more than a pilot, it's sandbox environment. And because everybody's pushing like, well, why don't we just use Azure, Azure for all this stuff? And that's kind of where my challenge has been, is that in the time that I stood this thing up, I really want to move off to the cloud. And some of the folks that are looking for security are now looking at us going, oh, wait a minute, we can do this on premise and it is actually secure. So I would say that we are in the adoption phase right now for a couple of departments, and it needs a lot more. So I wish I could say it was like a massive success, but I'm not gonna just, oh, I'm not going to lie about it. And like, oh, no, it's fantastic, but most of it is because of people which is not understanding what it's for. You mentioned that the county is data rich and analytics poor, so are you hiring for analysts and data scientists or other positions? Yes, so there's a two part answer to that. The problem we have is if you look at the counties, all the jobs in the county, and trust me, they go from accountant to scuba diver, I mean, to, you know, undertaker, really. There's no data scientist in there, there's no data engineer, there's not really even a business analyst which is also kind of part of this. So we're in the process of creating new positions to do that, so we are hiring for those roles, we're hiring in and sitting them against positions that really don't describe that's what they do. If you look at the goals of the county, one of the biggest goals is to become a data-driven organization. Data-driven metric, data-driven justice, data-driven, we're doing the homeless initiatives, so the major initiatives, homelessness, justice reform, child care, child endangerment, and stuff like that, and it's all driven by data. So they recognize the need for it and now, because the county's just big and slow, they're actually figuring, oh, I guess we should have positions that actually would tie into that. So the answer is yes, we are hiring for that. Unfortunately, the hard part is that you wouldn't see the term data scientist or data engineer or data analyst in any of the job listings. And so that's the hard part for us, so it's messaging. Does that help? A lot of them are under apps developer, IT specialist, systems analyst roles, right? And DBA, right? We do have a DBA position that we're turning into data analyst positions. Most of them though are systems DBAs, which is good, but I'll be honest with you, people who can tune a database, we appreciate them, but the way the world is going, we need application DBAs, people who can mung data, who understand data itself, not the inner workings of the database, so yeah. With such a wide set of departments to serve, like where are you on, kind of like where your organization has to partner and do a bunch of upfront work to get somebody on boarded, or they're self-serve and then you have tools that look for social security numbers and flag stuff. So I wonder how you were gonna scale to onboard. Right, so the, it's use case driven, right? So it's not like, hey, we're gonna go get the department and do all their stuff. It's like, what specific problem are we trying to solve? So with the auditor controller, we did a fraud use case where, again, we pulled in procurement data and accounting data, and now we have that data, right? And we secured it, that was a use case we had to match on vendor's taxpayer ID, and so when the use case needs it, then we deploy that piece, right? So we're working on some risk management analytics as well, so we're loading our CEO risk management, so it's like data from our, from, again, a lot of them are old sources. So the answer really to that is the onboarding comes with a use case, as opposed to like, hey, come dump your data in here and see what we do with it.  I mean, we did a really high level survey and I came up with like 40 or 50 that you could just fill this thing with data on. So we're picking those use cases off with people who are willing, you have to be a willing customer, and we tried forcing some people and they don't wouldn't, they had a hard time with it. So, anyways. Sorry, miss, but there was one question I wanted to ask you even before you got to the talk. As different departments are producers and consumers of data, how do you guys calibrate your interaction between departments? So there is a, the way the structure of the county is, there's clusters, right? So you kind of saw it on the earlier slides, but there's like a justice cluster, there's a general government, there's a public safety and so on. And the CEO's office kind of sits over the top of that. We do have a CIO and an office of the CIO and we do have a data management, information management program which kind of lays the rules of the road for that. And we're running on like a data maturity model, but that interaction between departments, it has to be use case driven, right? So in cases like homelessness or I'm trying to figure one of the ones. Well, so child safety is a big one, right? So you have to deal with folks with mental health, you got folks that are in the, our Children and Family Services program, like how do you put those things together? And there's a lot of pushback on sharing data and people will go, well, by code, we're not allowed to share that. And what's happened is we have to bring County Council in to sort of be the arbiter and our key data officer to be an arbiter of what's shareable and what's not shareable.  And so we don't want to, it's my data. I'm afraid if you look at it, you're going to find something that I didn't find but there's a lot of reasons why they don't want to share. But in a lot of those cases, what has to happen is our CIO, our CEO and our County Council has to come in with a hammer and go about shout, right? So we get data sharing agreement on our use case basis and that's actually happening with our homeless initiative right now. So I don't know if that helps. It's a very contentious, can be contentious. Some departments are more open to it, and other ones, man, it's like World War Three over this stuff, so. I see Nina has your hand up, but I have a quick question about that. Do you share with the state because I can immediately imagine, like in your broadcast case, if I file an LLC and I get that taxpayer identifier and you would need to cross-reference right through the state in order to catch that kind of fraud, right? So how do you guys do that or do you do that? So we don't. I mean, we, so the. Now I know how to make money. Yeah. Well, so no, the answer to that is, at least for the purposes of this, is we don't currently, but until now we really didn't have the capability, right? So now we actually have the capability and now it's a matter of bringing the business processes along for the ride, right? So I mean, there are, I'm not saying that doesn't happen at all, but it doesn't happen in any systematic way. So there are, we have a fraud investigation unit that would find that kind of shenanigans. I just want to let you know. But it's not automated and it doesn't do it proactively and there's, again, there's so many potentially really cool use cases around that because one of our things we're trying to do is we're trying to promote local small business.  We want to bring in small minority owned, veteran owned, women owned, businesses, local small. And so we want to use data from places like you just described to help drive that.   So when the use case dictates, it makes sense. So there's been some local governments, as well as state governments who have codified data sharing and open data into actual policy. Do you see anyone pushing to do that for LA County? I haven't actually, unfortunately. I know our chief data officer or our acting chief data officer is that has awareness of it. We are actually hiring a chief data, I think there was an opening for our chief data officer right now to hire. And I'm fairly certain they're gonna bring somebody in from the outside to do that. But I think it needs to be brought up more. I wish it was, to my knowledge, I don't see that. I don't see somebody driving that as a function. Maybe you could do it. No, but I think our chief data officer, he's got his hands full right now just trying to get our information management program laid out. I have a question about healthcare in particular. I just joined a small startup that focused on medical records. So I've heard the data's very, the whole, it's kind of all over the place of like mumps and fire and HL7 hasn't really been very widely adopted yet. So what's your experience been like with that in terms of integrating with all those? Second part of the question would be, I've heard a lot of the data tends to be, you mentioned having very well-structured data, but I heard when it comes to medical records, a lot of the important information is unstructured. Doctors' notes, reports, lab results, so I'm curious about those two. Well, and again, this is where it gets a little aspirational on my part in that, until now we had no way to deal with that. And now at least we have a platform that we can do that stuff on and people starting to think that way, which before it was just unreachable.  I mean, a lot of times they fall back, they fall kind of hide behind HIPAA, right? Oh, it's HIPAA, oh no, no, and I always like to point out in our meetings with them that one of the A's in HIPAA is accessible, right? I mean, so it's all about accessibility. It's not about we're gonna put giant walls up in front of our stuff. I have not been dealing with the healthcare area at the county so big. I mean, we have the largest public healthcare system on the planet, I think, a hospital system as well. That it's just, it's one of those, like, you know, think big but smart, start small. Our healthcare area is not small and so we've been focusing on more biteable chunks, to be honest with you, so, yeah. Cool, all right, cool. Thank you very much. Awesome, hey man, thank you. Thank you very much for sticking out to the end, man, I appreciate it. I don't know where the local bar is, but that's where I'm gonna be.


Hi everybody, I'm Jen with opensource.com and I have a brand new top five for you this week, March 25th. At number five, director of Google.org on challenges of unconscious bias. Nicole Engard traveled to South by Southwest this year and reported on some events she attended at South by Southwest Interactive. First up in her series is a conversation between the director of Google.org, Jacqueline Fuller, and Hugh Forrest about diversity in tech, including what Google is doing about it. At number four, Ozcon moves to Austin. Will the 18th Ozcon be the best one yet? Nicole Engard is back with this interview with the Ozcon chairs where they discussed the conference's move to Austin, Texas from Portland, Oregon during this their 18th year. At number three, can we talk about ageism? Deb Nicholson explores what negative bias against age for the young and the old is like in tech. At number two, how are open, how are high school replaced IRC with Mattermost? Charlie Reissinger and his high school students bid farewell to IRC as they replace it with Mattermost, a new open source chat service. And finally at number one, for open source alternatives to Dreamweaver. If you're looking for an open source alternative to Dreamweaver or another proprietary HTML or CSS editor, Jason Baker's got you covered with this roundup. Also, we have an honorable mention this week. It's we're giving away a Raspberry Pi 3 Ultimate Starter Kit. We announced our Raspberry Pi 3 sweepstakes this week where we'll be giving away a Raspberry Pi 3 Ultimate Starter Kit. Enter to win by March 31st at 1159 p.m. EDT time. All you need to do is complete the official entry form in the article. Good luck.


Hello everyone, I'm John Furrier with theCUBE. We're here at Palo Alto, California for exclusive news around generative AI. Of course, we've been covering it like a blanket for many months now, and it's been a surge of activity. We're here with two CUBE alumnus, the CEO and co-founder of OctoML. Luis says who's been on theCUBE many times before, going back before the craze started, we were chatting a lot about AI. Welcome Luis to theCUBE. And of course, the investors were drone adventures and John Tru is on the board of directors of OctoML, also an expert in AI as well, former AWS, now a VC partner. Matt McClain did the original investment, giving props there. We're here to discuss the hot news in generative AI that's being released today. Luis, John, thanks for coming on theCUBE. Glad to be here. Okay, let's get into it. Luis, we've talked many times about what your role has been in the industry, obviously with OctoML, hot startup. Generative AI has been going crazy. You have news that you're announcing today, first optimized compute service for AI, which just sounds like a great product. I want to hear more about it. A platform you guys say that's going to help developers have a fully managed cloud infrastructure to make it easier to write AI apps. Basically, take all the heavy lifting and abstract away all the complexity. Let's get into it. What's the news? This is the hottest area. Generative AI, people want to do more. GPUs are on allocation. You can't get any. There's all these different services out there. A lot of confusion, a lot of noise, but it's hot. What's the news? All right, so we are releasing today the OctoAI compute service. So it's for self-optimizing compute service for generative AI. It offers freedom because it lets users to go choose their model or bring their own custom models. Second, offers efficiency because we optimize the models and we choose the right hardware and we make sure that it gets the right performance efficiency trade-offs. And then second, it's very easy to use. So we make it very easy for folks to get started. We offer a collection of super-optimized models like stable diffusion, Nalama-based LLMs, Whisper 4, audio transcription, and so on. So with freedom, efficiency, and ease of use, offering this service that makes very easy for folks to build really amazing generative AI applications. There's a lot of confusion out there around how to get into the gen AI business. Some people are leaning into it, have been doing well with it, but the rapid pace from idea to code to product have never seen this kind of acceleration. What problem are you solving for the developers? I'm asking this, interest is high. What problem are you solving specifically? Yeah, so several problems. First, abstracting away the complexity and helping clear this confusion, right? So we offer the ability for developers to come to the platform, select a use case, for example, text to image or text to text, and very quickly get started with state-of-the-art models ready to go and ready to be integrated into their environment. And then we also abstract away all of this incredible complexity that is involved in putting a model into production because once you have a model to deploy, the path from there to actually deploying in a way that first has the SLA that you need to make your application usable, and then two has the right cost and scalability properties is a lot of work today. And we completely abstracted away and make it fully automatic. John, you've been covering this area. We've been on theCUBE talking about this before. Madrone, obviously early stage investor here in the company. The world spun right in the front door step of OctaML. What's your assessment? What do you see in this new platform? What's the outlook? What should people think about it? What's your perspective? Well, what's really exciting about the world that we're in right now is there's almost sort of an Android moment that Luis and I have written about before that look, we have very exciting models. You might call them the iPhone kinds of models, things like GPT-4 and model from OpenAI and chat GPT, Anthropic, Cohere, lots of other models like that that are genuinely really powerful, really exciting. Good. There is also a world of open source AI models that are really useful in their own right and together with the closed models. And the open source models are special because they let developers and scientists push the boundaries on functionality, on data types, on cost and on latency. And you can assemble these models into what are called ensembles. Some people call them cocktails of lots of models that work together. And while that is good and we've seen great companies like Runway ML and Mid Journey and some others built on top of open source AI, it's been really difficult to get started with that until now and to manage it and to run it all. And so what's exciting about what Okto and Luis and the team were doing is that they're going to be able to give for the first time the kind of ease of use with open source AI that you're getting with the closed models. That's going to unlock lots of new innovation. Let's get into the open source content here, Luis, because, you know, John, we were talking before this interview on text and also in person about the long tail, there's a lot of long tail stuff coming out. You got the power law. I've never seen power law applied to this kind of phenomenon before, but open source is making a real impact. More people are coming in, a lot of experimentation. People are looking at different scopes of the size of the models. They don't need to be huge big models. They can work with existing. There's a blending cocktails, if you will. Why is this important? What's the big thing that's going on, Luis? Why is this so successful? Well, I mean, the pace of progress in these open source models is just absurd and they're showing to have value, right? So I'm talking about literally on a weekly basis, there is new models that are released and these models actually they're useful for problems that businesses have today, right? So now the big problem here and then the anxiety that this generates is that how do you ride all of that? How do you actually, you know, how do you harvest value out of these models and into your business in a way that's sustainable? It's exactly your goal. I mean, I talk to a man with extremely focus in making these open source models that are moving so fast work for you to work for the business, right? So in that, you know, the application creators focus on what experience they'll offer their users and abstract away all of the complexity infrastructure and details involved in getting these models to actually function in production, right? So that's our laser focus here is like let businesses extract value from these models that are moving really fast. John mentioned proprietary Android iPhone. I like the analogy, but it's interesting. OpenAI is runs on, doesn't run on AWS yet. Anthropics does, OpenAI is on Azure and yet cohere out there, they interplay. So it's, and there's interaction between them. What is this, is this part of the equation because I see lock in there. I don't see a lot of choice if it's something runs only on AWS or something only runs on another cloud. What does that mean? Cause there's consequences of that. I mean, OpenAI is not going to go and run on any cloud. What good is it? Or is that going to be proprietary? We back to the old network protocol stacks where, you know, you had to pick something and stay with it. It's just, how do you guys talk about this and how do you see it unfolding because it doesn't make any sense to me. I think there's a very early moment that we're in where a lot of these questions are open. You can use OpenAI from AWS.  And that's the moment that we're in, but the clear movement from enterprises and from developers that I speak to is they want to access this powerful technology and they want choice of the best model and the most useful model and combination of those models for a use case that they're building. Luis, what's your take on the thing in theCUBE that developers in OpenSource are the de facto standards bodies these days? They're the ones driving the change. If you look at just what's happened in the past four months alone in OpenSource, it's been pretty incredible. Explain the phenomenon there. Yeah, so let me answer a mix of the two questions that are in the air here, right? So I'll start with, first of all, it's something valuable that these closed proprietary models are offered by, say, OpenAI, have incredible functionality, right? So, and it takes a lot of money, a lot of effort to actually build those models and understand, and gradually they're building a business, a business out of it. And these models, models like GPT-4, they do truly surprising things that are very, very powerful. But what John said is really important, which is like, it's important for users to ask the question, what is it they want to solve? Do they want to solve a question answering service over a set of documents? Do they want to do a tax summarization service? Do they want to do tax classification? Do they want tax image? And once you understand your use case, it became really clear that, there's a large set of open source models that are quickly progressing, that can solve those specific use cases and address them very, very directly. Now, and the end user advantage here is the following, right? So you can, for the use case that you know, they can be fulfilled by an open source model or by a customized version or fine tune version of those models. You can control the deployment of that model. You have control of where it runs and how it runs, what data it's gonna touch. You have full transparency, what it does. And you can assign, you can use that in components of your application that makes sense. That said, it's also important to realize that once you need a functionality that today is only available in the large preparatory models, you can still call them. The reality today that John alluded to is that people are building with model, building with an ensemble of models where you combine a collection of open source models that do things that you know that they do well, we can validate that. But then once you need this functionality only offered by the closed models, you can call them, right? So all that said, I think like if you zoom out on the market at the timeline too, it's interesting to see that the gap between, this is the point that John and I made before and supported by others in the community, that the gap in functionality between close preparatory models and open source models as a functionality is shrinking, right? So, and then if you project this out over time, I think it's easy to see a world where yes, you can probably fulfill all of your, you'll probably be able to fulfill all of your AI needs out of open source models. And that means that the big challenge then is like, do you have a platform to run them on, right? So. Yeah, and I think this is what I like to do. Let's get back to the news. So GPUs are on allocation, they're hard to get, they're jacking up the prices, NVIDIA stock is soaring. Although they just invested in Coheir recently last week, we saw that news, NVIDIA is a player, okay? Amazon has cloud and they have great footprint, carbon neutral they say by 2025, and these large language models aren't helping the world. I mean, talk about carbon footprint impact. You know, I was talking to a customer, they said the sustainable budget, their budget and their metrics are exploding off the charts because the sustainable goals are being not met because they've been blown out by all the compute and that. So it's a lot of pressure. At the end of the day, the developers just want to code. What do you guys offer the developer? What's the pitch of this platform? Why use you guys? What does this mean for me? I want to get in, get my models nailed down. I want to understand how it all fits. I'm tinkering, I'm kicking the tires. How does it work? Yeah, absolutely. So, and this topic's obviously near and dear to my heart because of the OctaAI compute service, but also because I'm a computer architect by training. So I like seeing chips being so important in this new phase of the world here. So what we offer to the developer again is the ability to not have to worry about infrastructure. What does this really mean? It means that really making the deployment viable, which in the first instance is you have access to the Silicon, right? So because of our ability to deeply optimize the model and how it runs on the actual hardware, we can actually offer a choice. You may not need an A100 and you might actually use an A10G, for example, or maybe even a T4 that's readily available. So giving, offering the optimizations as such that use less computes coupled with the ability to move the actual work around and abstract that away from the end user point of view gives you access to more Silicon fundamentally, right? So, and then that directly leads to more cost efficiencies, right? So cost efficiency will come from literally making the code faster and run better, that's one. And then the multiplicative factor there is the ability to run this on Silicon that has better pricing, right? So and more availability. And then all that said, of course, we're supporting and working with Amazon and supporting in France. And then we're going to, you know, we're going to expand that offering to other providers as well, right? So abstracting away the choice of hardware such that users can focus on building their application which is what really matters is a significant part of our mission. And the use cases, you guys, what do you see coming out of the gate? Customization, is that the purpose, finding their differentiates? Yeah, I mean, one clear use case that we see extreme need for, and it's very much featured in our launches, text-to-image, you know, it's just interesting how text-to-image is a great example of an open source model like stable diffusion, deep frogging what was around there that was closed before and then showing that it can offer amazing results.  We support that. Users can come build on with stable diffusion 2.1. We also guarantee you have the latest and greatest optimizations, the ability to fine tune that model and the ability to change the model weights, you know, when we place them such that it can customize for a broad set of users and quickly switch between them. So that's one clear use cases, text-to-image. Another use case that we're supporting today, ready to go from pre-optimized models is large language models for, say, text-to-text, right? To do summarization, to do text expansion, text classification and so on. The third one is audio transcription, turning, you know, audio into text. You know, we support that as well. But we also support, you know, customers bringing their own models and building their own custom containers, right? So go and run it on the service. So we have an authoring service for that. Anyways, yeah, so you can, that's, those are some of the key use cases. I also want to point out that we've been releasing steadily some exciting demos built on our compute service. One of them is the InkMM multi-model. It's the first open-source commercializable multi-model model that you can upload. You can offer images and ask questions about it. It's called InkMM, you know, and we show how that was built. There's other demos, cool, you can make your photos look like it's been cartoonized and then we show what the code looks like there. So these are some examples of use cases that we see that users are really excited about. And then just like an example of what people can build with, build with, you know, very, very quickly. John, please compliment that. John, way in here because you're watching open-source. There's innovation happening. What are you seeing in the landscape that supports this need to get tinkering, get building, get experimenting, you're mixing those cocktails? I mean, you're seeing performance enhancements, all kinds of, what are you seeing? Yeah, so you're going to, it's about pushing the envelope on one of a few dimensions. Customization to push the envelope on either new features, new types of data, lower cost, higher performance or privacy and, you know, sensitivity of data. And when you speak to, when you speak to developers or CEOs or CIOs, what you're going to hear is some combination of those items become very, very important. And this is not to diminish the amazing work that OpenAI and Coherent, Anthropica and others are doing, but there are cases where a CEO wants to take her own fate in her own hands and open source makes that possible. That's awesome. I think too, the support that's going to come out, open source, you know, everyone said that we never see another red hat. Maybe there could be because you got to get in there once there's momentum, you're going to want support. Enterprise is going to want support. Another big part of that, Luis, isn't that true? Absolutely, yeah. We have world-class customer success team here and they're ready to help. And, you know, some of our early access users are ready, you know, enjoy that benefit and now we're opening to the world. Very easy to help folks to get started and have a world-class team of ML engineers and solutions engineers to help folks build with it. Well, congratulations on the big news, a platform infrastructure managed service. Awesome, get people standing up their models, testing them, customizing them, tuning them up, you got to helping them do that. Well, in the last 30 seconds we have left, Luis, give the pitch for the developers watching, for the press that are watching this video. What's the big deal? Why Optimal? Why this platform? Give the pitch.  And finally, ease of use, we make it very easy to broaden the set of creators to go build with this, you know, with a set of world-class models ready optimized to be used and then, you know, very, very clear instructions and quick start guides here. So that means it'll be more people building with it, more innovation and better life here, so. Great service, helping the developers, getting more innovation, getting that flywheel going, faster, smaller, cheaper. That's the way we like it in open source, Luis. Thank you for coming on, John. Congratulations on a great venture. Say hello to Matt for me and congratulations on the news, Luis. We look forward to following up. Thank you, John. It's right out, everyone. Octo AI, complete service. Bye. We'll be checking it out. This is theCUBE exclusive news from Octo ML. I'm John Furrier, host of theCUBE. Thanks for watching.


Welcome back. Let us look at a few more exercises are from E04 to E07. In E04 we are asked to derive an expression for partial derivative of Cv with respect to v at constant t in terms of pv and t. Let us proceed. We have to start with a definition of Cv and we start with the expression which represents Cv which we have shown to be t into partial of s with respect to t at constant v. Now we have to differentiate Cv partially at constant t with respect to v. So let us do that. What we want is partial of Cv with respect to v at constant t. Let us differentiate the right hand side with respect to v at constant t. Because the differentiation is at constant t, this t which is a factor will come out of the process of differentiation. So we have t and then we have, let me write in big brackets, partial of s with respect to v at constant t of the second term which is partial of s with respect to t at constant v. Now we will change the order of integration because this is a second partial derivative. So the order of integration should not matter. So when you interchange the order of integration we have to be careful because the variable which is maintained constant also gets flipped. So we end up with t. Now the inner differentiation was with respect to t at constant v. So we have that as the outer. The outer one was with respect to v at constant t. So that becomes the inner. Now we notice that this particular partial derivative appears in our Maxwell's relations. If we do the exercise, partial of s with respect to v at constant t, the Jacobian form, it will be Jacobian of s t with respect to v t. We want to convert this t s to p v. So we write s t same order as this but instead of t s in the numerator we write p v in the same order s t s. Now this s t here and this s t here sort of cancel each other and you will end up with Jacobian of v and t with respect to Jacobian of v and t which means partial derivative of t with respect to t at constant v. Substituting this back into our previous expression we will get partial of c v with respect to v at constant t is t outer bracket partial differentiation with respect to t at constant v and the inner differentiation now replaced by partial of p with respect to t at constant v. Now you notice that the inner differentiation and the outer differentiation are both with respect to t. The inner differentiation is at constant v, outer differentiation is also at constant v. So we can write this as simply the second partial differentiation of p with respect to t both at constant v. This is the expression which our exercise E04 expects us to derive. Now in the next exercise we have to show that for a van der Waals gas c v is a function of t alone. Before going to the van der Waals gas do the following exercise. In the right hand side of this expression here which is a function only of p v and t. We substitute say the ideal gas equation. If you substitute p v equals r t or p equals r t by v. You will notice that the second differential of p with respect to t at constant v is 0 and you will get partial of c v with respect to v at constant t it is 0. This is the ideal gas and that means because there is no variation of c v with t when volume is maintained constant that means c v is a function only of t and not of v. Now use the van der Waals equation of state. Remember that the van der Waals equation of state is linear in pressure. You can straight away solve it for pressure p equals some simple function of t and v. And you will notice that variation of c v with respect to v at constant t is 0 even for the van der Waals gas and that means even for a van der Waals gas c v is a function of t alone. It is not a function of v. The next exercise asks you to determine c p minus c v for a van der Waals gas. That is easy to do. We have an expression for c p minus c v in terms of p v t. So substitute the van der Waals equation of state and you will get an expression for c p minus c v and check whether that expression is a function of t alone or depends on v or p also apart from t. If you find that c p minus c v for a van der Waals gas is an equation only of t then well we can say that since c v for a van der Waals gas is a function only of t, c p for a van der Waals gas is also a function only of t. Otherwise we will have to come to a conclusion that for a van der Waals gas although c v is a function only of t, c p is a function of t as well as either v or p. Thank you.