Thanks for coming everybody. So yes, I'm fortunate enough to have about a year at POV to experiment and build things and try different things. So I'm going to show off some of the things that I've built and we'll ask for your feedback. There's a couple of themes that you'll see kind of throughout the work. One is trying to take the kind of expressive aesthetic power that we have with kind of traditional media, whether it's through, you know, the software tools like After Effects and editing tools and things like that and sort of all the kind of work you put into something offline to make it truly polished and expressive, you know, get the artist's voice out there, trying to bring that same level of ability to interactive, mostly on the web. So my first couple of projects, we'll talk about that. And the other is kind of meeting the audience where they live, whether that's adapting to the device they're on, the time commitment that they're prepared for, the sort of level of experience that they may have with, you know, whatever the interactive kind of components are. And finally, I think I'm interested in finding an analog to the traditional media have these sort of conventions that help orient the audience to know where they are in space and time. And I think we don't quite have that yet in interactive. So I'm trying to kind of develop some of that as well. So the first project I'm going to show off is this is one I've been working on since before POV. It's a few years old, but I've integrated into some of my work at POV. And this is a JavaScript library called Seriously.js, which is, it aims to be kind of the web equivalent of something like After Effects, it does video post processing and effects like that. So it starts with this sort of basic library, this is just a, you know, a color pattern of different effects that we can apply. So this is an example of a bleach bypass, right? And you can kind of turn it up and turn it down and you do things like, you know, exposure adjust, which is, you know, it's simple effects and stuff like that. I'm trying to use the camera working here. The camera is not working. So anyway, we'll show that off later, but it gets more interesting when you start to kind of apply it to video and combine the effects. So this is an example. This is a music video that was put out by the band OK Go. And the actual video that's coming into the browser looks like this. This is what gets downloaded from the network. The band put out their sort of raw green screen footage, and I can process it in the browser. So by combining different effects, I can create these different moods. So what this is doing is it's all kind of node based. If you have ever seen like Quartz composer or Nuke, some software like that, every sort of node in the, you kind of create this graph of nodes and each one is either a media source or some kind of effect or an output point. And you can kind of combine these and chain up these different nodes. So what this is doing is it takes, starts off with this video of this band doing things in front of a green screen. And it runs, first it runs through a chroma key filter, then it runs through, we kind of composite that on top of this background, then convert it to black and white, and then run this kind of TV glitch effect on it. And each one of those is a separate effect and it runs one on top of the other. And then if I change over here to a different effect, it's a similar, it takes that same chroma key node and puts it on top of a different background and then runs a different color effect. It's kind of night vision effect and some scan lines and some vignetting around the edges. I know when I kind of go between them, there's that kind of Star Wars wipe effect. As it's doing that, it's kind of running the two different chains of effects in parallel and combining them up at the end. So this is using a technology called WebGL, which gives access to the graphics processor, which allows you to do a lot of really kind of advanced things in parallel and gives you these, the ability to basically edit high definition video at 60 frames per second in real time in the browser. So the idea is that you can rather than having to render this stuff offline, which if you can render it offline, you may as well render it offline, but you can render it as it goes and react to what is the user doing or I can bring in some new data or new video clips or new images kind of right up to the second that the person is watching it. So it's mostly kind of experimental and it's a cool technology and we're kind of continuing to see what uses does it have. So I'll show you something, another kind of experiment that we built with it at POV, which is to do these transition effects. So this is running a video clip. Again, this is all in a browser, right? And I can switch between one video and another and it runs this flash. Well, let me, this should be playing. So it runs this kind of flash effect and we see that sometimes on, well, the videos pause because of network issues, but sometimes we see these flash effects on, I think they do it on like ESPN and reality shows and things like that. So what this does is you switch between the videos whenever the user wants to switch between the videos or you can kind of pre-program that in based on preference or let's say you want to have maybe a short version of your video and a long version of your video and it adapts to what the user is doing. So maybe you say, okay, if I'm on a mobile device, maybe we show them the short version because it's likely they're on the go. Or if I'm on a desktop or like a giant screen, maybe we show them a long version, you kind of splice it as you go and say, we're going to build in these transitions and you can have the same level of expressiveness that you would have if somebody had done this in Final Cut or After Effects offline and you can have them online. So there's a couple of other different transitions that we built in here. So there's the whip pan, which I like. And of course the channel change, which is my favorite always. So yeah, the channel change uses the same TV glitch effect that we used in the other demo. The flash, what it does is again compositing multiple different effects.  So it's the same kind of stuff that you would do in traditional editing, but we do it in code and it makes it much more adaptable. So that's a cool toy. What else? The next one comes back into the area of adapting to the user. So this was an idea. I was very frustrated with the way the experience of watching videos either in a browser or especially on a mobile device where we've ever seen where somebody wants to show you a video they took on their iPhone and they hold it vertically and it's got these giant black bars at the top and the bottom. I'm one of those people, I always have to grab it from the person and rotate it and say, please, please just hold it right and fill up the screen. It drives me nuts. Or even if you're doing a web experience and you have a video and you want the video to fill the screen, that question of what do you do with it, it's tricky because you either have to have black bars on the top or the bottom or the side or you cut off chunks of the video. Now if you're subject to the videos in the center of the video, that's fine.  But the one thing that to me does not work is this assumption of, well, I just think whoever's going to watch my interactive video web page should just always have, let's close all your other browser tabs and maximize the thing and show it on a big screen like that. I think that's disrespectful of the user's experience. I think somebody, maybe they read about your work on a blog and the browser window looks super narrow like that, right? And then you say, oh, well, sorry, you have to maximize it. I think that's rude at best. So I came up with this idea of, well, I was remembering back years ago with the little experience I have shooting film and remembering the time, especially around sort of when we were transitioning to the four by three TVs, to the widescreen TVs, and you used to kind of have to shoot for both, right?  And I thought, well, what if the guides could be different every time for every shot? Because maybe one shot, you want to, your center of the screen is, or your center of focus is on the left side or another shot, your center of focus is on the right side. So I built this software that kind of, for each shot you get that data is specified along with it and the cropping of the video reacts to it. So this shot, for example, I thought this building on the right, I want to keep that in frame. So when I, so if I turn off the effect, right, just the traditional, and we get these bars, right, on the sides, and if I go too skinny, then I get them on the top, right? But if I turn on the effect, it fills the screen. And as I close it, as I shrink it, it keeps the right side of the frame in, in frame, and it crops on the left side, because the left side is extra, right? And if I play into the next shot, the next shot I thought, well, this bridge part here is the most interesting. So as I narrow it like that, I keep the bridge in the frame. Or if I go kind of up like that, I figured this wall kind of would, that tells the story best. So this video clip has a bunch of shots. Again, with this one, we keep, we keep our subjects in the frame, whichever way I do it. Hang on, I lost. So we can do this shot by shot. And the next, let's fast forward a little bit, we can even kind of animate it, which I think gets interesting. So here's this bit where this, this is from a POV film, Cutie and the Boxer will be coming out in our next season. Keep an eye out. And so he's going to kind of walk across this frame, filling up this painting as he goes, right? Which, so if I'm going to watch this in full screen like that, this is just more or less the original shot. It's fine, it's great. But if I narrow it down, if I think, all right, I'm watching this on, say, a vertical tablet, right? It's actually going to pan with him as he goes. You can see it moves really slowly. So something I learned from this, which is interesting, is that I do have to pan really, really slowly. Otherwise, you can kind of see that pan and scan effect. Because if I had, let's say I had the camera on a track and we were moving the camera, there'd be this sort of very subtle kind of parallax difference that you would see. And if I moved it quickly, you would really see it. So if it's not quite as good as being on a track, but if you move it slowly, it's not going to, those differences aren't going to jump out at you. So this, you know, and this adapts to a tablet. It adapts to not quite every phone, but you know, it works pretty well. Now something's going to happen shortly. So I don't know if you see the flashes on the side. So this is the boxer and Cutie is taking pictures of him. And it should, in a second, it's going to cut back to her. There it goes. So it cuts back to her, right? And she takes the picture and then it's going to cut back to him. Okay, great. So, but they're in the same shot, right? So if I were to do this, if I had the frame like that, and we wanted to cut back to her, it's not really enough of a cut, right?  So it's intelligent enough. So I tell it for that shot, only apply that cut if it's within a certain aspect ratio. So if I go back and play it again, it just, it just continues to kind of pan across. So again, this is another scenario where, you know, I think that sometimes I get some resistance from filmmakers on this. They say, look, I frame the shot the way I frame the shot, you know, deal with it, watch the way you want to watch it. And I think that's kind of the traditional approach is we assume that people have nothing to do on a Friday night and they're going to go to the cinema because, you know, what else are you going to do? But we know they have other choices. And I guess the kind of statement I'm saying is like, yes, it is ideal if people watch it the way you want them to watch it. But audiences can be picky now. So if they're going to watch it the way they want to watch it, you should at least have some control over that, right?  So I think this is a good kind of compromise. At worst, it's a good compromise of making, serving both the audience and the audience. And I think it's best if you can shoot the film with this in mind. I mean, this is, we happen to be lucky that we found a cool clip that it works with, but it doesn't work with every clip. And it's, I'm distracting. No, no. This is the beginning of the film. And the filmmaker was cool enough that yeah, to let me do it. This is, this is actually an example where it gets a little bit tricky because we don't want to cut off the title. So I do, I sort of, so for every shot, I essentially draw a rectangle around what is the most critical part of the shot. And it will keep that rectangle as close to center as possible or area and frame. So, you know, outside of that, sort of temporarily outside of that, it'll let you be right. So here, so, yeah. So that part, that's one of my favorites. Okay. Do you have any questions while we're going or anything? That's about three minutes. That's the longest one I've done. So I would love to see that. Yeah. Yeah. I mean, would you do that to a like an hour and a half long or an hour long documentary? I mean, I don't see why not. We haven't really built out the authoring tools for it. The authoring tool is basically I edit the JavaScript by hand. I had to go through that shot, you know, frame by frame. Yeah. It was the kind of thing where, you know, this was an experiment to see if people liked it. And if somebody wants to use it, then, okay, we look into building the authoring tools for it. But yes, you could do that. Yeah. Yeah. There are there are algorithms for taking an image and sort of finding parts of it that are of interest. And I think the way that would work is you could say, I would do it once offline and kind of run a first pass at it and say, you know, here's my video, let's find all the interesting stuff, and then output the kind of first draft, and then I would go back and have a human tweak it, I think.  I think, I think, you know, if this were to, if people were to want to do this, I would think you would probably want to do something like this in an edit suite. You'd have an editor doing it. And it would be part of that, you know, maybe like a final cut plug in or something, and it would output the data file and you'd upload it to the web. But I think ultimately, if it was the sort of thing that was going to become standard, I think you'd want to have this in mind as you're shooting to whatever degree is possible. I mean, if you're in the field and the documentary, I think it would probably be easier in a studio or in a sort of fictional film, you know, where you have a little more time in your shooting schedule. But ideally, yeah, if you were going to shoot something and you had time for a setup, I would keep that in mind when composing the shot. The same way they did when, you know, when you had to shoot TV for four by three, and they said, okay, but the DVD is going to be like this. So you keep that in mind when you're shooting it. You know, you kind of build in, you know, if you're doing like a talking head, for example, you say, all right, I want to, I know that somebody's going to watch this vertically on a tablet and somebody's going to watch it horizontally on a TV and who knows what for a browser, right? Like a desktop browser. Yeah. Do you have another question? Yeah, that's interesting. I mean, shooting a closeup that fills the whole screen, that would be, that would be challenging. I think it's up to the sort of artists to whether or not they want to do that.  So if somebody's watching it, holding their iPad vertically, you're going to get the black bars at the top and the bottom. And if that's the best way to do it, that's the best way to do it, that's okay.   And they, so they, they put it in the middle and they kind of blur out the sides, right? You've seen that. Right, they have a second image behind it. And that feels sort of hacky. And it's like, okay, but what are you going to do? Because this is primary source footage of somebody else who holds their phone vertically, which I beg people not to do all the time. But you know, you work with you got what you got, or, you know, if you have the, if you have the time, then, then I think it's best to kind of play it that way. Yeah. To be like a really interesting A-B test to do, like if you were to push out like a single phone or like half the users just get the standard version, half of them get the smart crop version, and then just like, if you can get feedback on like whether they're like, how much does this actually qualitatively improve at the end of the day? Right. Okay. Yeah. Talk to my agent. He's over there. He'll set it up. Cool. Yeah. I've also thought that I haven't tried this yet, but one idea was, was also even if the aspect ratio doesn't change, maybe taking into account the size of the device to, you know, you might have, for example, if there's a shot that requires a certain amount of detail, right? And if you're showing on a big screen like that, you know, I can have maybe a small thing in the middle with a lot of detail in it. But if it's on a very small device, maybe even though the aspect ratio is the same, you might want to zoom in more. So that might be an area for future research as well. Do you have a question? Well, I have not, but so this uses, so I think if you're going to do something like this, you would, you would render the subtitles dynamically. So this is something that I had done in other work. So this uses, so it's totally custom software for managing the aspect ratios, but it uses popcorn.js for the timing, which is a JavaScript library comes out of Mozilla for running different effects at different times on video. And, you know, I've used that for subtitles as well. So I think, you know, this is the sort of thing. Yeah, once if you're going to do something like that, I would absolutely dynamically render the subtitles in HTML. And that will allow it to adapt to that. It does get a little tricky with, like, you know, the contrast between the subtitles and the picture. And so you have to do something that's going to be, yeah, that's going to be adaptable to that. And there's, there's different approaches you can take. But yeah, I think I would not want to do it with something that has burned in subtitles. Yeah, right, right. But the dynamic subtitles has, has so many other benefits anyway. You know, like I did a project a few years ago where we had, it was like, part of it, people were speaking English and part of it, people were speaking German. And part of it, people were speaking, I don't know, French or something. And so we had, like, if your audience speaks German, like we would, the browser would auto detect your language. So if your language is German, we would only show the English subtitles and the French subtitles. If your audience was, you know, Spanish, we'd show all the subtitles, right? Or if your audience was German, but they could turn it on like captions for, you know, hearing paired or whatever, we could turn it all on. So you get a lot more sort of finer grain control. You know, you could also adjust the size of the subtitles if the audience wants it bigger or smaller and things like that. So there's a whole lot of other benefits to that as well. Cool. So why don't I move on to the next one? So let's see what we got here. Okay. So the next thing, the next batch of stuff, we get into, we start to get into virtual reality. So this is something else we've been experimenting with over the last several months. So you guys probably all, I've heard some of you talking about it. Has everybody, everybody fairly up to date on what's going on with this thing? Yeah, who's tried on one of these? Okay, we got a few. Not quite everybody. I recommended trying it. You have one at home. So virtual reality is hot right now for the last year or two. You know, the Oculus Rift is a work in progress and there are some other devices coming out at much lower prices than they've historically been. We're talking in the area of hundreds of dollars as opposed to tens of thousands of dollars. And cardboard. Yes, we'll get to cardboard. Don't worry. That's coming. And right, so it's getting better and better and better and a lot of work is going into the visual quality. But a couple of things that are sort of left out, among them that people are really struggling with is how do you interact with stuff?  And people have been sort of starting off with some of the same interaction models from kind of like first person shooter video games, things like that. And I find that they, I'm really frustrated with them. They don't work at all. So the first experiment, which come on browser. So I'll give you, while I'm dealing with this, I'll give you another kind of bit of talk about this. So a lot of people are using video game engines to build work for virtual reality. So there tends to be sort of two approaches. You either have, you build something like a video game where you model this 3D world and you put the user in the middle of it. That's one approach. And another approach is you get these fancy camera rigs with many, many cameras around in a circle and you kind of, you shoot video in synchronization, in synchronized in all these multiple cameras and then you stitch them together in this horrible complex format that supposedly makes it 3D. So that is interesting and there's some good stuff happening there. It's got some limitations and the process for both is difficult. Do you have a question already? Oh, sorry, I'm just, I subscribed on as it's visible. Oh, hi. Sorry about that. Okay, that's all right. Let me restart. So the reason I'm having so much trouble with this browser right here is that there's another approach to delivery of this content, which is delivering it in a browser as opposed to a game engine. And where this has been interesting is that the native game engines, they're pretty fast and you get these really high frame rates and low latency. So when I say latency, I mean as you move this thing, there needs to be like less than 20 milliseconds for each frame. Each frame needs to be less than 20 milliseconds about behind where the actual position of your head is. A little bit more than that, it starts to not quite look real. A lot more than that, you start to get sick, which is a whole other interesting thing is that I've been working with bugs in software for years, but this is the first time where a bug in my software can make you car sick for like two hours. So that's just a new thing I'm learning about the process. So anyway, so the browser is an interesting delivery platform because it does a couple of things. One is that it, so yes, the latency is a little bit longer and the frame rate is not quite as high. But one is that it's a really great delivery form because if you're going to get something that's built in a game engine, you usually have to go and you download some file and it's 500 megabytes, a gigabyte, whether it's on your phone or your laptop and you got to like wait and you download this thing and then you run it and who knows what software you're running on your computer. It could have security issues, it could be buggy, it could take over your laptop or do whatever.  If you have the available bandwidth. That's the big issue. Yes, yeah. Because you're going to have the boot. Right, but what you can do, and you could do this in a native app as well, but it's a lot harder, is you can build it to sort of gradually display. You can bootstrap your basic scene and your head tracking immediately and then fill in the scene as it goes, which we do some of that in another demo. And the other thing that's good about it, and I'll get more into this in a little bit, is that it's a lot more adaptable. A browser runs on so many different platforms and we can build something that, if you say, all right, I need to run some Unity game engine thing and it's going to be geared, okay, I have to have the Oculus Rift and I have to have a fast computer and it has to be on this and you have to send out the PC version. It's like, oh, I didn't make a Mac version. Let me see if I can make a Mac version. So this can work. If I have the headset, if I don't have the headset, if it's a phone, if it's a tablet, if it's a laptop, as long as it's a relatively recent browser, it works fine. So the first interesting experiment we built with this is to go back and address that question of how do we navigate and control stuff. So let me make sure I'm on the Wi-Fi here. So this is a fun trick. So I can look around, right, and I can even, I can look around in 3D, right? Okay, so that's all good. But how do I move around? So one of the things that happens, so Unity game engine, for example, has this kind of built-in controls for moving around based on what you would do in a first-person shooter. And the way it usually works is whatever direction you're looking, use the keys, the arrow keys on the laptop or desktop, whatever, and whatever direction you're looking is forward, right? Now, this is passable, but it stinks, right? The majority of all the, any virtual reality piece you've seen out there that lets you move at all does this. And it's totally unnatural because nobody moves like that, right? So imagine Sean and I, right, right, exactly. Oh, you know who does it? Batman. Batman does it because if you ever look at all the Batman movies before the dark night, he can't turn his head. So he always got to do, he's got to do that. Next time you watch Batman, any Batman movies, you're going to see it. Although Michael Keaton was, all right, anyway. So, right, so that's, that's the thing. It's like, if I'm, if Sean and I are walking down the street and we're going to talk to each other, I'm going to turn and talk and look at him as we're walking and I'm going to check in every so often, make sure I didn't step into traffic. But for the most part, I'm kind of turning my head. But imagine Sean and I are walking down the street and I look at him and I just start walking into him, just like that. It's totally unnatural. It really just throws you off. So I was like, maybe we can do better. So I figured I've got this thing, right? And this thing has an internet connection. It's got a touchscreen. It's got an accelerometer in it. So what I do is I scan this. I know QR codes are cliche, but bear with me. So I look, feel like I'm connected. So now let me get centered. I can point at things. And so I can, it's tracking where I'm pointing it. So I can grab stuff. So I grab this cow, right? And I just tap on it. And now I'm moving the cow as I move my phone. Right? So that's kind of cool. So I tap it again. I can, I can grab any of these objects and manipulate them, which is pretty cool.  So it's a little bit of kind of feedback that way, which is neat. And the other thing is, so now how do I move around? So I can just kind of drive like this, right? And wherever I point my phone, that's where I go. So I can either do it by pointing my phone or by moving my thumb in any direction. So if I put my thumb down and I move it forward, I'll move forward. And if I move my thumb to the right, I move to the right. But then if I rotate it, my thumb is still to the right. So it's sort of, it works whichever way you do it. And it's better if you're kind of looking around as you do it. And it's pretty, I think the alignment got screwed up, but it's pretty natural. Let me see. There you go. The other thing that I do with it is kind of neat. After some experimenting, so there's, there's this whole best practices guide that Oculus put out that tells you what things make you sick and what things don't make you sick.  So what this I had to do is if I'm, if I'm moving, let me restart it here. And so if I'm moving forward, I can move forward pretty quickly. But if I look to the side, it slows down quite a bit. Or if I look backwards, it's going very, very slowly. So it's always kind of tracking the difference in angle between where you're moving and where you're looking. And it's, and it's the fastest as if you're walking straight ahead and it slows down as that angle increases. Which if you think about it is mostly how if you don't walk, I encourage you to walk that way. Anybody doing full sprint backwards, just please be very careful. So yeah, so I feel like this was a pretty cool experiment, fairly successful. And that was cool. So I took the same thing and I used it in my next experiment, which I'm also kind of, kind of psyched about. Let's see. So let's get, I gotta get this set up again. Okay. So this was getting into data visualization a little bit. So where this comes from was again, so if I go back and talk about the way that people are making virtual realities, like you either have kind of this, these expensive and difficult to use camera rigs, which limit you to one point of view to one spot, or you have to construct a 3D world more or less by hand. You know, you can scan some things, but it's difficult and you need, you need to get the lighting right and you need to have artists really constructing these things and that's difficult. And I had neither an expensive camera rig nor a good 3D artist. I'm certainly not one. So we, I worked with some data here. So what I got was I used these open street maps has these giant databases of buildings and maps of most of the world, like every major city in the world. So I can drop you into Tokyo, London, wherever you want. This happens to be Manhattan because that's where I live and it's awesome. And so we can fly around and this works with this thing too. So I can get aligned. So yeah, I can kind of fly Superman like all over New York City. And here's where this is, so this loads up progressively as well. So as I go, I didn't load up the entire world as much as I would like. But as I've screwed up my, I've got to get my alignment right here. Sometimes the, the orientation of the phone gets out of whack. So, but anyway, yeah, as you, as you fly around the city progressively loads up. So that lets you keep a pretty high frame rate without, sorry, every time this is a thing, if you're sitting in a chair, it becomes difficult to turn all the way around. So I just did a thing where you shake the phone and it does, it rotates you 180 degrees. So that's kind of neat. So anyway, yeah. So, right, so it loads up progressively and it's cool. What's that? Yeah, yeah, yeah, with, with boxes over their heads. Yeah. Okay, so, so the first part, the first part was just getting the city rendered. And that was, that was tough work. And it's pretty cool. And if you can fly around it, it's super fun and weird. But now what kind of data can we bring in? So the first piece I got was income data. So this is from the United States census. So we can go anywhere in the United States and look around. And we charted income data. So here on the upper east side, we've got some lot of income happening. And if you look up in the distance up in the Bronx, some of these bars are not so high. So that's kind of cool. It's kind of excited about that. We did the same thing with bubbles. So instead of bars, we've got bars and we've got bubbles. Okay, giant bubbles on the upper east side. It also happens that this is like the best spot in the world to see like wealth disparity. This is uptown Manhattan. What else? Okay. Here's where I think it starts to get really interesting. So let me go back to the beginning here. So here I got, this is the database of, excuse me. This is, each one of these is a personal injury claim against the New York City police department in 2013. So there's something like 3,800 of them from that year. And I plotted them just all over the city, wherever they happen. So here I am on the upper west side and we fly uptown. They're from different databases. So this one, right. So the buildings and the maps, that's from open street maps. The income data is from the US census. This is from the New York City, some office of the controller. So there's various data sources. The census was a good one. Got a lot of stuff from that. So as we fly up the upper west side, we see, okay, there's a couple of incidents here. The upper west side is a pretty safe place. A lot of wealthy people and I'm surprised that there's even as many as there are. And then we start to get uptown and of course what happens when we get to the Bronx is a lot. There's a lot in Brooklyn as well, but the Bronx is just, it's totally lit up. So where this gets interesting, I think, is we get uptown here. We have, this is kind of flying up over the city a little bit. And I can do another view where I'm sort of lower down, right. Get closer to the ground. And as I kind of play around here, so I also, we move a lot slower when we're close to the ground because it just makes you more sick. If you're faster at that height, you're just crashing into buildings, which is super weird. But as, you know, I played with this a little bit. And I found myself, as I was sort of roaming around the city, I find myself imagining that, and again, this is just, these are just these big red cylinders, right? You know, it was this quick and dirty visualization I had in mind. Like I wanted to make it into like these fires or like lit up spaces and make them more maybe a little more expressive and elegant. But it's just, it's just this clunky example. But I find myself trying to avoid them, trying to think, can I walk through a neighborhood without running into a spot where this violence supposedly happened, where somebody was, presumably somebody was injured by a police officer. And you can't, like it's just, think about how many Starbucks there are, and it's like multiply it by 50, they're just, they're everywhere. You walk around. And this is where I think the kind of VR pays off, where it's different from, you know, plotting this on a map. Because this somebody did plot this on a map, right? And that's how it was inspired to, to try it. And okay, yeah, it's cool on a map, but this feels very different. And, and I discover there's sort of this balance, right? When you start off, if you think about the map as sort of the God's eye view, right, from space of any data set, your, you get this, you get just tremendous overview, you see all of it, right? And you can say, okay, there's a lot going on here, but there's not a lot going on here. And as you kind of come lower into the space, right, and it's sort of with the high flying high up as sort of the middle ground, as you come lower, the, you sacrifice that overview because you can't see what's going on even three blocks away because there's buildings in the way, at least if you're in a city with tall buildings. And so you sacrifice that, but in exchange, you kind of get this a little bit more of a feeling of intimacy and presence. So I think there's a lot of potential here. I think, you know, what I would ultimately like to see is rather than just say, okay, here's a city, go explore it, is a narrative project where we kind of alternate. You say, okay, we're going to take you on a tour, we're going to fly you around. There's going to be a voiceover or something presenting. Here's what the city looks like. Let's fly down. And, you know, as that goes and say, okay, now you can explore when you've been through that. I want to show you one more data visualization here, which I think is also pretty interesting. So this is population data sorted by race, also from the US census, also inspired by a 2D map that somebody did this. So this is interesting. Let's get back up to the upper east side. Actually, I'll just do upper east side. Okay. So here we go. So this actually looks even denser. This is what it looks like in 3D. So the blue dots are white people. If anybody's ever been to the upper west side, upper east side, this is not surprising. And as you fly through, it's sort of, you know, people are represented kind of like, you know, I think of them as like kind of pollen in the air. And you see what happens when we get uptown towards Spanish Harlem. You can imagine what will happen. Sorry, I'm crashing you through all these buildings, which is super weird. And people. And it starts to look a little bit different. And once we get up to, and now it's pretty different. You know, this comes from some of my own personal experience sort of traveling the world.  And then, you know, when I went to Copenhagen a few months ago, it was a little bit different. So I was trying to figure out like, can I replicate that experience a little bit? You know, so it's not quite rendering actual people and they're still abstracted, but I think it does give you the sense of like being there. So that's kind of neat. Do you have any questions about that? No? Okay. So we can move on, I guess. The next thing. Okay. So as I was presenting this work in Europe, there was a, I was on this panel, some, a couple of you were there, and there was this discussion of diversity in the field of virtual reality. And we were kind of worried that there was not enough so far. It happened to be a panel of five men, and you know, we were looking around at the industry and who's making what, and it's, it's not great. And that got me thinking, since somebody asked on the panel, they said, well, what about, you know, how expensive is this stuff to make?  And I was like, oh, well, you know, this is only $350. It's not so bad. It used to be, you know, you'd have to be thousands of dollars and you'd have to have like, you know, super computers everywhere.  And now you can get this and you can get Google cardboard. Do you guys know about this cardboard? So this is fun. So Oculus came up with this thing. And then Google came up with this thing. This is usually sells for about $25, but you can get it for as low as four and a half.  And it uses the gyroscope in the phone to track where you're looking. Right. So now it's not as, you know, the latency is longer, the frame rates are lower, but you can at least get in it. And it's really not that bad. It's pretty good. It's pretty cool. So, you know, so I started looking into it and I was like, well, what kind of assumptions are we making? When I say, oh, don't worry, this is only $350, but I'm missing something here. I'm missing this. This is like a $2,300 machine. Okay. I'm missing that I have to know where to get this thing. Right. I have to know where to get the software for it. And if I want to build something for it, well, that's like a whole other thing. Right. So I started thinking about the first thing I did was some research into what would it take to just like view this content at a minimum. Right. So I found out, I did some research and I found out that something like 77 or 80% of people even under making under $30,000 a year in the United States, people under the age of 30, about 80% have smartphones, like not just flip phones, but like actual smartphones. So it's not that bad. Could definitely be better, definitely be better. But even, you know, I found that it is possible to go buy a used, not this one, but like a kind of low end smartphone for about $45 that would be just enough $45 and one of these for $4, you can get in VR for under $50. Right. So yes, it could be better, but it's a start and that's good. And I think we need to be thinking about that. But that also raised the question of, okay, what if you want to make something? What if you want to build content? And I still wasn't really satisfied with that because, you know, it requires, again, thinking about like, what are the assumptions? So instead of saying, okay, well, I can build something really easily because I have this fancy pants MacBook Pro and I have hosting accounts and I have a computer engineering degree and time, you know, at POV to work on this stuff. Like what if I didn't have any of that.  They don't have a hosting account. They don't have a computer engineering background. What do they do? Right. So maybe they go into a public library or the school library and you say, okay, well, I can get on the public Wi-Fi with this thing. So I don't have to use my data plan. That is an issue, by the way. As I was, the last time I was giving this presentation, a buddy of mine was downloading the Sundance set. Sundance, they released a bunch of VR pieces and he was downloading clouds over Sidra, which is one gigabyte. So he's, he's talking to me and he's got his cardboard. He's like, oh, this'll be cool. I'll set that to download and he puts it down and I give him my presentation and he goes out to take a walk. He sticks the phone in his pocket and he gets off a Wi-Fi, right? And he comes back and so he's from Canada. He comes back, he's got a text message that he's cost him $100 in roaming fees for his five-minute walk and he's capped out his plan for the month. So this is something we try to be careful of. This is why we do this adaptive streaming stuff. Like, so, you know, some thinking about all this stuff. So this kid can get on Wi-Fi and they can sit down at the library computer. Now, what do we assume about these library computers, right? So you can install any custom software on these things. You know, you can't put the Unity game engine on there, right? You can't, you can't put like a development, you want to build an Android app and you got to put on all the Android development software, right? You can't do that. You can't even really save files on them, right? You can save it for five minutes and as soon as you get up and log out, it wipes off all the files, right? So this is, this, you know, even if somebody, even if somebody, Santa Claus comes up with one of these things, like, the kid can't use it. The kid can't make anything with it. So what can we do? So I started with, so this is a JS bin. Does anybody know what that is? It's a, so it's a, it's a kind of a HTML JavaScript sandbox tool. So it lets you, so if I make like just a new one here, jsbin.com, right? Well, it helps if it doesn't crash your browser. That I can't promise you. So this, I'm, by the way, I'm running an experimental version of Chrome that interfaces with the Oculus Rift and because it's an experimental version, it crashes all the time. So hopefully they'll fix that one day. So, right, JS bin. So JS bin is this kind of sandbox. So it gives me some HTML on the left and I can write, hello, right, maybe, hello, and it shows up on the right and I can make it into, I can put some tags in there, right? So I can type in HTML and CSS and JavaScript on the left and it builds a web page on the right. So pretty easy. So what do we do is I start off with this one and let me zoom in a little bit so you can see it and all I got to do is I paste in this script. So this is one script file that I built and what it does is it bootstraps a VR scene, an empty VR scene with nothing in it on the right side and we can pop this out like that. Okay. And so far I got, I got nothing going on. If anybody wants to bring this up on their own machine, you're welcome to. It's JS bin.com slash W E M I P O slash one. So let's come back to that. So now, so I've got the output. So I get rid of the HTML. So now with that scene, I can start making some basic stuff. So first, I'm going to make a box. So I type VR dot box, right? And there's my box. And actually, if I reload, it should show up on here. No, not so much. Oh, it's just number two. Yeah. So, yeah, JS bin. So now as I even, so now I've got this box in my Oculus and 3D, right? So it's a, it's a sample. It's a, this script that bootstraps this scene, it works, it works on the, it works just on a browser, right? If I, let's say I turn this off and I can just look around with my mouse like that. If I don't have the device, I can use a device. If I have it, I can put this on my phone and drop it into cardboard, right? If I have cardboard, it'll do it in 3D.  It works on any of those. So this kid can, this 14-year-old kid that we imagine in this library can bring their low-end smartphone, sit in the library, build something. There's a bunch of commands. So I did a VR dot floor, right? So I've got a floor and I can say, I'm going to make the box, set material, wood, right? And let's move up two meters and move x negative one meter, move it over. So there's a bunch of commands for building simple things that this, what you can do is you, you don't have to have a hosting account. You don't have to have anything except for the phone, maybe. You don't even really need the phone. You go to a library, you look up the reference for these commands which are much more simple than, much simpler than most of the 3D libraries out there.  And as long as you have that URL, you can come back to it and edit again later. Anybody can fork it. So now this person is, yes, it's really basic shapes. Yes, it's lower quality because it's on this dinky little cheap phone and this piece of cardboard. But now we at least have this other range of voices making content and getting it out of the world. And if they do get it out of the world and they get it to somebody like us, and I have this thing, I can look at it in here. And now this person is, is building things in full proper VR. So that's, that's the latest project. And I'm finding that as a sort of nice side effect of looking at it from a point of view of an audience that's very different from me. It happens to be, I think, a pretty good prototyping tool as well.  They can use it for a quick prototype, you know, or if I'm sitting in a cafe with somebody having a meeting and I left my Oculus at home and I'm like, okay, what would this look like? And let's sketch it out. It's like, okay, I can get a floor over here and a box over here and a sky over here. And there's, there's animation things I can get them moving around. And there's a, it'll play audio in 3D space. And I can look at something and it triggers the audio or, you know, I can shake the phone and it triggers something else. You can get something really basic going in a couple of seconds even. So I'm pretty excited about that. And, you know, hopefully get some workshops going, teaching people to do it. So that's it for that one.


I'm Prasant, I'm a senior engineer slash data scientist slash sometimes I don't know what I'm doing engineer on the AI center of excellence at Red Hat. So as you see from the slide it's machine learning for developers and QE. So you can see from the title like I'm going to start rambling on and on about machine learning but why developers and QE, why disturb the poor souls and not let them live in peace. So what do I mean by machine learning?  So but here I'll refer machine learning I'll use it as like an overarching term like and relate that to anything related to statistics like analytics data anything that you use with data and try to get like meaningful information from there. And coming back to like why developers and QE so they're like several personas out there like users like data scientists like rocket scientists the mighty Thor and even Ant-Man like they all have data what do you do with it like and what do you do to like get meaningful insights into the data so machine learning is one popular technique to do that and it lets you get meaningful insights not into the not only into the data that you have but also lets you foresee the data that you will have in more technical terms that's like the prediction. So I'm gonna focus specifically on developers and QE like explain certain use cases that's specific to development and QE so let's see I forgot what I practiced. But so I mean when you talk about I mean when you want to learn a new language or when you are introduced to something new you always look for a hello world example well is there like a hello world example for a machine learning that's the field is too broad to ask for that but think of this as like a hello world presentation or a hello world like template where I'm gonna show like a tool or a framework that lets you turn prebuilt machine learning algorithms into a service and kind of provide an easy interface for you to access these machine learning models through the service and play with your data and if you want to move further and kind of advance the code like tailor it to your own use case you could do that and and then you can follow the same fashion of like how do you turn that into a service and make it more user user friendly. So I'm gonna explain the tool or like the framework that I'll be using for this it's called AI library so it's a part of a bigger project called Open Data Hub I don't know so this Open Data Hub there was a session yesterday about Open Data Hub so that's like machine learning as a service platform that has all the bits and pieces like ranging from streaming services like Kafka like how you saw in the previous slide I'm sorry in the previous talk and bunch of other tools like Jupyter Hub and like data data handling services like plenty of other stuff so AI library is just a part of it and it has like an open source collection of AI components which ranges from like simple statistical algorithms like regression correlation analysis to all the way to like natural language processing models and so so and it also provides like support around like bringing up the infrastructure around machine learning models like data handling and let's you turn that into a service so having this environment set up now you can quickly play with the algorithms without like trying to develop it from scratch and kind of like lets you do rapid prototyping so let me jump into the architecture so once deployed like the AI library sits on top of like the container platform that's we usually order that open shift and on top of it is going to be Selden so Selden is an open source platform to serve models so anytime you have a model like a Python code or a Java code anything that does machine learning you want to package that into a container and kind of serve it through like a REST API or a GRPC so Selden lets you do that so and we kind of like wrap all this in a nice package so that it's easy for you to do it and of course for the store data handling part like it's compatible with any S3 storage but here we've used Ceph but if you have any other S3 back and it will work too like AWS or MINI or any S3 compatible ones so the top most part is where like it shows what models are there in the library right now so like association rule learning correlation analysis so I'm going to focus on four of those because I have data set specific to QE or development environment so that's like association rule learning correlation analysis flake analysis and duplicate bug detection detection so those four I'm going to explain in detail as we move on but to give you an overview of like the rest of the stuff so let me start with fraud detection so fraud detection was the one that was showcasing the last talk so where they were streaming data through Kafka and kind of like trying to predict whether certain transaction was fraudulent or not typically this is used in a financial setup like where you're trying to detect fraud in a credit card transactions so that's the model and it's based on I think random forest regression and sentiment analysis and topic modeling they are pretty common natural NLP use cases so sentiment analysis you want to get gonna classify the sentiment around like a certain text whether it's like positive negative or neutral and topic modeling is kind of like a sub part of like sentiment analysis where you kind of like try to condense any given natural language text into like short information so you have like a huge document you want to just get a bunch of topics that actually tell like what the whole document is about then that's a simple use case for topic modeling or if like there are a bunch of like tweets and you want to say like okay what is it being what are they talking about and then use like topic modeling on the whole and just get condensed information and regression that's plain simple linear regression or could be like multivariate or simple and matrix factorization that's that's a use case that's game from the DevOps team so you might remember this algorithm from the Netflix price challenge where they were trying to get a recommendation system built so that's the algorithm that's underneath the hood but in in our IT environment it's being used to like recommend like packages for certain build environment you might have anomaly detection and so anomaly detection is basically very like you try to look for anomalous records in a data and in this case like it was used on a build log data so they were trying to like find if there was any anomalous data and it's based on an unsupervised learning technique called isolation forest so here's a typical workflow on how you would interact with the AI library so you start with deploying it so there are like two ways to deploy it one is it's available through the open data hub operator or if you want to like get hands on it's more you have the ansible roles they use like the ansible playbook to install it and I'll show it in a bit and once you deploy this like the models are gonna show up as service on the OpenShift platform so the model itself they exist as Python core and a Git repo so what does the deployment do it basically pulls down the code like builds the container pushes it to the Docker hub or any repo that you have and then tries to spin up the container on the OpenShift platform and then Selden also lets you kind of like expose a rest interface so any further interaction with the model happens to rest API calls and once you have everything there like you need a trained model so the trained models you basically save the train models to your back end there are pre-trained models which are available but if you want to have your own model do your own training like use at this moment you need to use like Jupiter hub or like any external environment that comes with Jupiter sorry open data hub train the model and push it to the set for like the S3 back end and once you have the model story basically start interacting the prediction endpoint like through a rest call which so once you send the rest call it Selden forwards it to the specific model or the part that's hosting the model which in turn fetches the train train model does the prediction and it doesn't see the result so I'll kind of show how the deployment is done through Ansible so this is a can you guys see the font or do you want me to increase it okay so this is an OpenShift environment we have and so I've pre-deployed all the models in this namespace so the last three you see here is Selden the last three components belong to Selden it's the Selden Core Redis server the API server and the cluster manager so the Redis is more for like in-memory data management and the API server is the one that kind of takes in your rest calls and forwards it to the part that kind of hosts your model and the Selden cluster manager kind of keeps a watch on all your deployments and manages that now let me go back so I've cloned the repo and I'm gonna try to deploy one model because I'm gonna by default like when you run this play Ansible playbook it's gonna deploy the entire set of models like all the libraries and everything but now I'm gonna just restrict the deployment to one specific model so as you see like I've just turned on like the fraud detection now before you run it you might want to configure like certain parameters that the model kind of acts just like what's the namespace that you want to deploy it to like what's the base image that you want to use to build up the containers and where does it code live and where do you want to push the built containers and when it comes to model specific things you just give it a name and like resource constraints like how much memory do you want to use like how much CPU you want to use and all this is available to the operator but I'm gonna just show the Ansible version and the other thing that you want to define is like the S3 backend details like the endpoint the access key and the security it's defined down I forgot to delete it so I'm not gonna show my secret key here yep now see so because I already had the fraud detection model kind of cleans up and then tries to create the part that hosts the fraud detection model let's go in and see so the parameters that you actually define there like in terms of the CPU memory you can see like it gets reflected here and when you go into the logs you can see that it's ready it's running on and it's ready to receive the request if you come to the seldom part because that's the one managing the thing so you can see that so it added the deployment for that fraud detection model and it's watching out for its request similar as the case for the cluster manager so it's gonna look watch all these deployments and manage that so now I'll just briefly explain what this happens so basically like very first that you have everything in the configuration and clones repo then kind of builds the template to spin up the container and creates the container and then pushes it up and host the model now this specific part it says like update graph details so seldom not only allows you to like host like single models but you can kind of like host like multiple models kind of do like a sli a b testing like kind of like pipeline stuff there but at this moment like a library supports like hosting model as a standalone but if you want to have your own deployment file modify that way it would definitely work so now let me jump into the models itself so that's specific to the development or QA environment so let me start with correlation analysis so what is correlation analysis so it's it's very in simple terms it tries to get the relationship between variables you might have like so you have X and Y it says like is there a possible connection between the two like does one influence the other like if one increases does the other increase or decrease so depending on like how it which direction it goes whether like if one increases the other one like then it means that it has a positive influence but if it's in the other way in the other direction it means negative and it also tells you the strength of the association like whether it's strong or weak so that basically gives you a value between zero to one saying like how you move between zero and one it tells you whether it's strong or weak so how do you use this in a DAV QA environment so I'm gonna the data set that I use is focuses specifically on the bugs reported so in a typical DAV QA environment there's like the bugs reported and the bugs that's are fixed so it's basically less like a think of it like an incoming QN and outgoing so it needs to balance itself like the number of bugs that come in and the number of bugs that go out or get fixed it needs to be kind of like equal so that you stay balanced and you don't build a backlog to it this is not any made-up data so it's from the OpenStack Neutron team and I'm gonna show like the data from that team so you basically plug in the data that you showed in the regular format like and then try to send in the cold request this is the one yep so now what happens here is like so you send the data to the model that's already hosted on the OpenShift platform and it basically gives you back the result saying okay what is what is what can I say about the data so in this case the two data sets have like positive small correlation and it's statistically insignificant so which brings back to the question like the OpenStack Neutron team is not keeping up with the bugs that come in so which definitely leads to a backlog and the other one is like in our model we don't save the plot but we can also save the plots and kind of look at it so so you can see like the red is where the incoming bugs and it starts creeping up and they don't keep up with fixing the bugs so this is definitely a backlog that's happening and you can look at analyzing the data so the next data is more from a QE endpoint so like once a bug is fixed like it goes to multiple stages like it goes to on QA where like you ask the QE to verify if the fix really works and then once it's verified it moves from verified to released where you actually release the fix to the customer so that's a critical process to like the once the the fix hits the QE pool it needs to be verified quickly and also it needs to be released quickly so in this case we are trying to like wave the bug fix verified which is the released now does that like tally and they keep up in that sense so this gives a more insight into the QE process typically like a QE manager would want to know this like the QE team is actually keeping up with verifying bugs as well as like and if it's being released so here's a query that we ran and it came back saying the result is statistically significant and there's a strong correlation but if you look at the correlation it's like 0.5 it's just over like 0.5 6 it means like it's a 5050 it's more than a 50 chance that they're keeping up with the process but you can dive more deeper and see like if you want to look at specific trends then you go into like actually modeling the behavior like can use regression if there's a strong correlation so now let's move to the the next model that's association rule learning so this is another model that's let's you find relations between variables but it does in the sense of in the form of rules so this is this model is widely used in like market-based analysis like when you do like shopping and do like market like grocery transactions or things like that you want to see like is that like an association of a certain item with another so an example would be like anytime people bought butter they ended up buying bread so that's like an association or a rule that was discovered within the transaction now what what measures does it come with like there are like it's confidence support and lift but to explain that in plain terms confidence basically tells you like saying the above example like what are the chances of someone buying a bread given that there's butter or things like that so you're looking at the chance of one happening given the other one and support basically tells you like how frequently does this association happen and lift is another lift is a measure that's more like in purely technically terms it's like the conditional problem probability of an event happening given the confidence because it's that's already a probability now you're like conditioning on the knowledge that that probably that thing is happening but in simple terms lift just tells you like what is the effect of the rule the rule body likes the in this case it's the bread on butter if it says like does it have like positive is it positive negative or there's no relevance at all it's like the more you buy bread so the more you buy butter we buy more bread or like is it the other way around or there's totally no relevance here now how do you tie this to a software development or QE environment let's see okay so here's here's some data that I took from the same OpenStack neutron beam does anyone work on OpenStack neutron here okay I think please don't sue me as such my life runs on free paying mortgages so this is true data so basically you look at the developers and the QE pool and you say like what is the severity of the part that they worked on it's like urgent high I mean it's just a sample data but there's like a bunch more like medium severity or low and how much time did it take for them to fix the bug and how much time did it take them to verify the bugs so if you look at it like there is definitely an association here because I'm gonna when you hit a bug it needs to be explained in a certain way that both the developers and the QE need to understand and once the developer fixes it and provides the fix he needs to give the steps to reproduce it or like steps to verify it so and that's kind of like flows into like how quickly the QE can verify so if you kind of think about it as like you can derive like an association between the developers QE and the time it takes for them to go through all this process so when you run this let me show you it's so when you run this like you're gonna get a bunch of data which makes no sense but this is separate out in columns but once you export this to a spreadsheet and start looking at the different columns then you can start asking questions there so as I showed in the previous slide it does come with like those metrics like confidence frequent like support and lift basically tells you like the chances of one happening with the other or like how frequent they are and things like that so here are some of the questions that I asked and I'm certain that I'm gonna end up in trouble but I'll still present it so so you take a developer and kind of try to associate what is the time taken to fix bugs right now you can you can look at the confidence level saying like you have the time taken to fix bugs and you have the developers now you look at the different time until say take a week so you can ask the question whether a developer will be able to fix an urgent severity bug in a one week's time and let's see what the analysis came out with so all these names so I think this would be more useful from the management side or like when you're assigning the box so here Tim Roger has like a hundred percent chance of fixing that bug within a week and so does Steve Hillman has like an 80% chance and if any of you are on the bottom there please erase that or like that tells you like how confident can you be with the developer in fixing a bug within that week within a single week and you can start asking more questions like say if it's not an urgent severity bug but it's more like medium severity and your process lets you go for like a two-week interval you want to you can study that relation too now see you move one step up in the management chain you want to look at the development team as a whole and says like what's the chance that how's the team doing with level with respect to like addressing like urgent severity bugs in a single week's time so that's is like there are different percentages that and with respect to the time intervals and the same you can ask for the QE teams so will they be able to verify the bug fixes or will they be able to reproduce and there are so many ways you can ask questions but basically tells you can look at the individual data even not if like the association rules can also look at the individual metrics that's thrown out there and get meaningful insights so next is the the flake analysis so this came out as a came out as a use case from the Fedora cockpit team so they're basically trying to find test flakes test that fails but shouldn't have failed so how do they how do we do it so we use clustering and classification so you basically take a bunch of test logs and say like you try to group them into clusters like something that's they're pretty common to each other and you start forming these clusters and once you have clusters and of course like within these clusters you have you would have like test cases that have failed but where like I mean there were like flakes so you would basically kind of like compute the probability of a test being a flake within that group so now you have like within a big pool of test logs you have these like small logs where you can narrow down your focus now when you get a new test failure you can kind of push this into a classification problem where you're taking the test log and trying to see like which group can I go into or in another case like we use the k nearest neighbors say like okay which is the nearest cluster that I can associate myself to and once you do that you go back to the compute the probability that you computed saying like what's what is my chance of being a flake now that I belong to this group and that basically tells you like what's the probability of a certain test being a flake yep so this is a sample execution so give the data what goes into the request is like you give the model the pre-trained model and then you show a sample log so this is the whole log and say like hey tell me if this is a flake or not okay so it came back with seeing like there's a 33% chance that this is a flake now based on your experience or you might want to set the threshold and decide whether you want to like take report this as a flake or not typically like anything over 50 or like 75% that's that's a good chance it's a flake yep so that's the result that you saw in the demo so next we have a duplicate bug detection so that's a classical case that you would see in any development QE environment where like when you're submitting like a bug there's chances that might already have been submitted by someone else or it already exists in your bug database so you might want to detect duplicates there's plenty of techniques and the one we have here basically is topic modeling so it runs through your existing pool of bugs and does topic modeling in the sense like it condenses all the bugs and all the information related to each box into much shorter information and once a new bug comes in it goes to the same step of like condensing its information and then running it through similarity measures basically runs through the previous set of condensed information says like hey who do I match so closely and it basically shows you like top few matches and that's configurable like if you want to look at like the top five matches of ten you could do that so you could know this these flake analysis or the duplicate bug detection you can eventually turn that into a software bot you kind of like doing it automatically for every request that comes in okay so finally come to the conclusion here so we I introduced you to AI library like how do you deploy it and how do you use it so I'll show you like how I'll show you the repo and like what the contents of it and so what do we have in plan in future so right now like everything all the models having happens to seldom so if you want to train a model now when I explained in the earliest slide you have to like save the train model to the set back end but what if you want to train a model using this framework so we're going to like start incorporating Argo project into it so Argo project is it's kind of I think it's a part of the Kubeflow community and it lets you manage workflows so the workflows typically nothing but like a set of containers that gets executed so we're going to use Argo to handle a training of models and the reason being is like seldom is more of a synchronous because once you send in the request it waits it needs to come back immediately so that's more suitable for like in like real-time processing or really fast models and training is typically training a model kind of takes a long time within hours or like days or things like that so Argo lets you submit the workflows in an asynchronous manner and then come back and get the results so that's one thing we are looking into and the other one is right now like when I showed you the Ansible Playbook like you are kind of like deploying the models through I mean through the code that's already in there but if you want to just do the I mean once you submit the code to the Ceph back end if you want to turn that into a service automatically without even like going through the ODH operator or the Ansible Playbook so model runner it's already there in the library but it's still in the testing phase so once you save a model to your Ceph or an S3 back end it kind of like detects that okay a new model has come into this pool and let me spin up a service for it so that it's being done automatically and last is the dashboard or the user interface for this AI library you know like I've been showing you like a bunch of curl commands and it's it's really hard to like start forming the curl commands you know like but once you get a hang of it it's easier but still you would want a much more easier interface to interact with AI library so we started putting together like a dashboard whatever I showed you like you could do it through a graphical interface I'm gonna show a slight demo so this is what you're kind of like started testing out with linear regression it basically tells what the model is about like what's the API here so this is an interesting model here for once like we'll have to look into the health of developers and QE as well so just to have some fun element here so we took data set that's from people in various professions with their height weight and their health index and it turned out to be a simple linear regression model so you input your height your weight and it basically throws you back an index between zero to five zero meaning like extremely weak one is your weak three is your normal for you need help and five oh god you need help right now so so that's it and so let's try here there it came back with a 2.2 minutes it's between the normal range so does anyone want to try here at life I pretty much pissed off my other colleagues saying sending this to them they don't like me anymore so it's set to 50 Zac is still my friend so okay okay yeah for good so that's the UI and that's a simple model to play but we're gonna start putting in all the other models that be sure it's yeah that's the thank you slide so I'm done so any questions no questions from the OpenStack Neutron team thank you good presentation I just had a question about so so all the model servings gonna happen through seldom what if I have a machine learning service that doesn't have a UI or doesn't have something that you know something like a back-end thing like a function do you guys have plans to have something like a function as a service type of thing so when you say it doesn't have a back-end or a function like is it no no UI it just runs as a as a back-office job runs at a particular interval and there's no serving it'll it'll run its predictions and it'll store its predictions in in a data store so yeah I mean it's up to you to like so the model behavior is something that's decided by you so what we show here is like you send so you have the model in your back-end you send the data that you want to run prediction on and it'll send you back the results but if you want to want to alter the behavior no I don't want to send it through the rest API back but I just want to like store it in some back-end that's certainly possible too so all seldom just submits the request and sends back the results whatever the model sends back to seldom if you're not going to send anything then it's just send back another string but on the other hand like your results are going to go to a back-end or where you want to store it so I think in that case how do you kind of like can monitor that maybe like if you want to put that into an Argo workflow where you can get the logs within that container and say like hey what happened to my workflow and I want to see that so probably want to move that into like an asynchronous rather than a synchronous request yeah so this the synchronous when you're interacting with the cell then you can tie that to like really fast predictions like when you're doing it on the fly like in case of like the previous talks they had like data streaming in through Kafka and they wanted to run prediction instantaneously so in that case you can like hit the cell then rest endpoint and get back the result but in some cases like you want to run the analysis like on a weekly basis you don't care coming back immediately you just run like on a daily job or things like that that's possible to do. Thank you it's a great talk. I'll give you a beer for that. Sorry I'll give you a beer. So like we discussed four models I think those already trained models that's what you send the prediction on or that's the thing I missed the part. So so linear regression so like correlation analysis and association rule learning they are kind of like doing they're not models as such but they are like doing analysis live on that data but flake analysis more is a pre-trained model so something that was trained on your data set and it was stored in the self-packing I didn't show the training part because training takes a little while but when you're moving into more complex space like clustering classification or like neural networks you want to store the model ahead of time and typically like how the process goes is like you get a bunch of data to say like you stop like a week on a week timeline train the model and try to run the prediction for a week and see if it performs better and then start retraining on a plus one week timeline so the retraining part is something that would be handled as part of the Argo inclusion that's going to come up. Okay thank you. If you start with a great talk I'll give you a cake. Yes sure. Thank you Mr. Embaligan for your presentation I appreciate that and so I mean thinking back so what we have done so far is the evaluation of developers and QE using machine learning techniques that's what I understood and the data scientist I think he or she falls under both the categories a developer as well as a QE person right no in terms of QE what do they do with the model part. So here let me rephrase the question so you're asking like how is it relevant to the QE space versus the development space ask against how a typical data scientist would do or is that am I right or what is the role of QE in data science. Okay so as I said like I mean so traditionally like it all started with data scientist as a role being associated with anyone who want who had wanted to do analysis on the data but we've moved away from that point and transition so much like with so much tools techniques available like now like everyone has data like you have data and I have data so we can start using the tools and start doing the analysis you don't have to like develop like all the algorithms from scratch but as long as you understand like what the model throws back at you like in terms of the metrics or what is the result and then like the the differentiation starts to disappear you know so in this case like QE's devs or like anyone else like they can just start using machine learning algorithms and tools rather than like trying to develop them. Thank you.


Julian Leporeau, he's from France. He's based in the USA. He was a 2009 Eclipse award winner as the most outstanding jockey in the USA. He was also an Eclipse award winner or champion jockey in USA in 2006, both a champion rider and champion apprentice as well, based on the number of wins he had. But he wrote confidently there aboard Rowan Crichton's Rough Entry. And Rowan, congratulations on this victory. Congratulations on how well you've been doing at Goldstream Park. But this victory here in Jamaica must be really pleasing for you as your home country. Elated. Elated. What a great ride. The guys at the bar did a fantastic job. Coming out of quarantine it was tough. But the horse handled it well. And I tell you, I was very confident. Very confident. Can you talk to us about the exercise gals? Because there were some onlookers in the mornings who were a little bit tricked by what the horse did on the exercise track. In the sense that the horse did a lot of running actually after it passed the winning post, we're told. So some of the splits that were recorded weren't genuine splits. Can you talk to us about that? No. Well, you know, you don't get paid in the morning. You get paid in the afternoon. So we were trying to, you know, not use them up, bring them to the race day as happy and as kicked as possible. When you read the race, the local horses, Mahogany and Atomica, who were both the favourites last year and lost, we said before the race that these two horses, when they run bad races, it's because they aren't able to relax in the races. And that's exactly what happened to them again today.  Well, yes. And Julian and I spoke about it. And Julian noticed that Ruff-Enter is able to relax in his races. And that was exactly what he wanted to do and he did a great job. Talk to us about your career. You have won over 300 races at Gulfstream Park. You were number two trainer in the spring-summer meet in Gulfstream in 2022 behind Safi Joseph Jr. So a lot of success in Florida. We hear that you may be having some thoughts of setting up a stable here in Jamaica. Well, we have. We have about six horses here. And you have six now, but is there a plan to build it? Step by step. Okay, Rowan. Congratulations on the big win here by Ruff-Entry. Splendid performance by the horse. And will he be staying here? Absolutely. That's a good story. All right. Thank you very much, Rowan Crichton. So he has confirmed that Ruff-Entry will not return to Florida and will remain here in Jamaica to continue his racing career. What a stunning victory. And what a brilliant ride from Julian Leporeau. Very confidently ridden. The horse ran 138 even for eight furlongs. Not a terribly fast track. So that's a decent time.


This algebraic geometry lecture will be on some more examples of projective varieties called projective space bundles. In particular, we will do the special cases of hits of brook surfaces and scrolls. So let me just stop by recalling what a fiber bundle is. A fiber bundle is something that looks locally like a product. So a fiber bundle is going to consist of a total space mapping onto a base. And the fibers are all going to look like some fiber F. And if you take a little open neighborhood of B, so if we select little open set in B sufficiently small, then this will look locally like the projection from F times U down to U. However globally, it might not be a product. So a simple example of this is suppose we take the base to be circle and the fiber to be the real numbers S1. And obviously we could just map S1 times R onto S1 and this would be a product which would be a special case of a fiber bundle.  On the other hand, we could twist this construction and have a Moebius band mapping onto S1. So if we've got a Moebius band, well, you can think of the Moebius band as being projected onto a circle. And then the fibers are all going to be copies of what you could make more copies of say the real numbers. And locally it looks like a product.  So locally it looks like a product. Globally it obviously isn't a product because for a start it's non-orientable. So when you go around the circle, the fiber gets twisted. So fiber bundles are sort of twisted products. Quite often the fiber is a vector space. And in this case we talk about a vector bundle. All the fiber might be a copy of projective space, in which case we talk about a projective space. So what we're going to do is to give some examples of projective varieties constructed like this. So the first example will be here at Sebrouk surfaces. So what we do is we take the space a squared minus zero times a squared minus zero. And what we're going to do is we're going to act on this by the group gm times gm. So gm for the moment can be thought of as let's work over the complex numbers. You can think of it as just being the group of non-zero complex numbers. And if we take the quotient of a squared minus zero by gm, with gm taking lambda of x, y... Sorry, gm takes x, y to... lambda x, lambda y for lambda in gm. Then the quotient of a squared minus zero by gm is of course just the projective line. So we can write a squared minus naught over gm is just the projective line p1. And similarly, we could let the other gm act on this a squared minus naught by the project... act on this a squared minus naught in the same way, and the quotient be another projective line. So if we take the obvious action of gm times gm on this and take the quotient, we just get p1 times p1, which is not terribly exciting. However, we can let it act in another way. Suppose we pick a point lambda mu. This is gm times gm, so we are just non-zero numbers. And we act on a point s, t, x, y. So this is an a squared minus zero. This is an a squared minus zero. And we're going to make the action slightly different. So this is going to take s and t to lambda s, lambda t. And then I'm going to take, multiply x and y by mu. So I get mu x mu y. Well, that's just the action I've mentioned up there, but I'm now going to triddle it slightly by putting the factor of lambda to the minus a here, where a is some integer. So now I'm going to take the quotient of this space by this group action. And that's going to be called a hit-so-brook surface. So f is going to be this copy of a squared minus zero times a squared minus zero, quotient by this action gm times gm. Well, there's a map from f to the projected line, taking s, t, x, y to s colon t. Because if we act on s, t, x, y by any group element, we just multiply s and t both by the same constants. This is a well-defined element of p1. And similarly, the fiber at any point is also isomorphic to p1. And you can check that locally this is actually a product, but globally in general, it's not a product. So what we've got is a surface that is a fiber bundle of p1 with fibers that are also copies of p1. So it's almost a product of p1 times p1. Only it's kind of twisted. Well, there's no reason why we have to do this in two dimensions. We can increase the dimension of things that are actually called scrolls. So this time you take a squared minus zero times a to the n minus zero. And I'm going to act on it by gm cross gm, where this time lambda mu acting on a point s, t, x1 up to xn is now going to be lambda s lambda e. And then we get lambda to the minus a1, x1, lambda to the minus a2, x2, and so on. Where a1, a2, and so on are all integers. This time, again, we get a map. So the scroll is defined to be the quotient of a squared minus zero times a to the n minus zero by this group action. And just as before, we get a map from the scroll to p1 by taking s, t, x1, and so on to s colon t. And this time the fibres are copies of n minus one dimensional project of space. Incidentally, as growth and Dick showed, these come from essentially all possible vector bundles over the projective line. So if you've got any vector bundle over the projective line, what you can do is replace each vector space that is a fiber by the corresponding projective space, and that gives you a projective bundle over the line. And this construction essentially runs through all possible vector bundles over a line. Anyway, strictly speaking, what we haven't done yet is shown this as a projective variety. So I've constructed some quotient, but I haven't actually yet embedded this into projective space. I mean, fibres are projective space, and the bases are projective space, but we've still got to project the whole thing into projective space. Let's map this to projective space. Well, let's assume that all the AI are greater than zero, which is fairly harmless, because if you shift all the AIs by a constant, it doesn't really make a whole lot of difference. And what we can do is we can then consider the monomials s to the i, t to the aj minus i, xj, where j is 1, 2, opt to n, and i is less than or equal to aj. So there's a finite number of monomials. You can see that you can take these as the coordinates of a point in projective space of dimension, sum of all the aj's plus 1, minus 1, because this is the number of different variables we have. And if you do this, you find that each of these fibres is mapped to a linear subspace of this large projective space. And strictly speaking, classically, when you talk about a scroll, you don't talk about this fibre bundle. What you talk about is its image in projective space. Well, it's a little bit clumsy to have to map everything into projective space. So here you see we've constructed a perfectly good algebraic object that's fibred by p to the n minus 1 over p1. And there's no real reason why we should be forced to embed it into projective space. For this reason, it's sometimes useful to talk about, to use abstract algebraic varieties. So abstract varieties were invented by André Vey. So the reason André Vey wanted them is improving the Vey conjectures for curves over finite fields. He needed to do a construction known as taking the Jacobian of a curve, which he wanted to be an algebraic variety. And unfortunately, it wasn't at all obvious at the time that these things could be embedded in projective space. And that turned out later they could be. So he came up with this concept of abstract varieties, which meant he could construct varieties without having to embed them into projective space. So let's summarise how you do this. First of all, let's look at differentiable manifolds. So differential manifold, an old view, might be that it's a subset of Euclidean space defined by some equations. For instance, a sphere might be the set of points x squared plus y squared plus c squared equals 1 or 3. However, this isn't really a differential manifold. It's really a differential manifold together with an embedding into Euclidean space. And nowadays, you don't think of a differential manifold as coming with an embedding into Euclidean space. So you can say the new view. We might, for example, cover S2 with charts. For instance, we might take two copies. If we've got a copy of S2, we might cover it with two pieces. We might cover it with a sort of top piece. And we might also cover it with a bottom piece, which might look like this. Now, each of these two pieces just looks like an open disk in Euclidean space. So we might construct S2 by taking two open disks in Euclidean space. And then we take this sort of ring-shaped bit and we glue these bits together. So we glue this ring to this ring here. And we can do exactly the same thing for an algebraic geometry. So we have an old view. A variety is a subset of project and PN. Well, the new view, Andre-Veys' view, is that you take some affine varieties and we glue them together. For example, in the old view, you might just take projective space. Say, let us take the projective line. We might consider that to be a subset of projective space. Well, that's not very difficult. We can just take it to be a subset of itself. Alternatively, we can glue it together as follows. We can take two copies of the affine line. So we take a copy of the affine line A1 and A1 and we glue them together. So let's see how to do that. Well, if we look at projective space with coordinates x, y, we have one copy of the affine line of the set of points 1, y with x none 0. And another copy of A1 consists of the points x1 with y none 0. So how do we glue these together? We've got two copies of the affine line. Well, the first copy, we look at the subset A1 minus 0 and the second copy of A1, we again look at its subset A1 minus 0. So here we're going to have points denoted by x and here is a point denoted by y. And we glue these two copies of A1 minus 0 together by mapping x to 1 over y, which you can see if you think about it. You can ask actually, what happens if you glue them by mapping x to y? Well, then you get something really funny because the first copy of A1, you've got one copy of A1 which is A1 minus 0 and a point 0 and you've got a second copy of A1 minus 0 together with a point 0 and you're gluing all the none 0 elements together with each other, but then you've got these two points left over. So if you sort of carelessly glue them by mapping x here to y there, what you get is something that looks like a line with two origins. That's a famous example of a non-Hasthorff manifold if you're doing topology. It also gives an example of an abstract variety that isn't a projective variety because projective varieties can't have this funny phenomenon that there are two points in the same place. So abstract varieties really are more general than projective varieties. Actually, this example is so annoying that people usually modify the definition of abstract varieties to exclude things like this and eliminate these non-Hasthorff sorts of varieties. It turns out that all reasonable abstract varieties, in other words, if you eliminate this sort of phenomenon in dimension one are automatically projective varieties. In two dimensions, if they don't have singularities, they're also projective varieties, but there are some three more examples of abstract varieties that are non-singular but not projective. All abstract, at least all complete abstract varieties turn out to be quite close to projective varieties. There's a lemma called Chow's Lemma saying that if you've got an abstract complete variety, it's pretty close to being a projective variety. So next lecture, we'll give a few examples of abstract varieties called Toric varieties.