to myself, to Paige, to Chris, to Ed, or others. Today we actually have a very short agenda and a very welcome guest. So with that, I'd like to hand it off to the man who needs no introduction, Jeremy, to talk about SDI V2. Thanks, Brandon. So this actually comes out of my enthusiasm for when Adam presented a little bit of Haskell Torch code a couple of weeks ago, which I thought was super cool. And so mainly my goal here is to kind of encourage other people to present cool things in other languages and libraries, because I think it's a great way for us all to learn what cool stuff you can do. But as tends to happen when you say, can somebody please do X? Somebody else says, hey, why don't you do X first? So here I am doing X, where X is telling you about the library that Silvia and I have been working on. Basically, since Chris Latner and I finished our last Swift and Fast.AI lesson. So for quite a while now, I will, it's a library for PyTorch called Fast.AI and it, I think there are things we can learn from it regarding stuff, cool stuff we can do in Swift. But I'm gonna focus on trying to sell you on Fast.AI rather than on the Swift bits, but where I think of Swift-y things, I will mention them as we go. So Fast.AI is a library, as I said, that sits on top of PyTorch. And a lot of people kind of think that a higher level API is this kind of like small little thing that you slap on top of the serious business of TensorFlow or PyTorch or whatever. But hopefully you'll be convinced when I show you actually what's involved in a truly modern high level API that there's actually quite a lot going on. If you wanna check it out, I put a link to it in the meeting notes and that will link you to the notebooks, the development notebooks. So that's the first weird thing.  Well, this is an example of what Fast.AI V2 source code looks like. It's written, as you see, in notebooks. We are just having a little troubles actually doing that seeing part. Okay, so that probably means I failed to present my screen. Shall I endeavor to do that? That would be great. Should present your entire screen. Yeah, that explains a lot. There you go. Nope, nope. We don't. There we go. There we go, victory. All right, sorry about that. So yeah, so here is an example of what the Fast.AI V2 source code looks like. It has links, it has titles, it has pictures, it has code. And this may seem like a painful way to develop because these are notebooks that are designed for interactive stuff, not for normal development, but actually you'll find that also this pixel shuffle appears here in a standard layers.py module which you can import in the usual way. So we've developed a new literate programming system that allows you to write code and have it automatically turned into nice modules, which even do things that most people don't bother to do because they're annoying if they're automatic like setting done to all, so it only exports the things that you want. Also coming out of that is automatically documentation. So all that gets turned into hyperlink documentation, including links directly to the source code and automatic doc strings and parameter lists. Also you'll see tests and the tests are used both to document the behavior expected. So if you're not sure what pixel shuffle is, this is actually, this test is a very good description of exactly what it is and also ensures that our code is working and those tests can all be put in continuous integration and so forth. So that's the first interesting thing about Fast.AI V2 is it's the first kind of truly literate programming based system I've worked on and it's been, it's been an absolute delight. Now, so we've written our own framework for every part of this, which is kind of a theme for Fast.AI V2. Basically every time Shilpa and I found something that didn't quite work the way we wanted it at any part of the stack, we wrote our own. So it's kind of like building something with no particular deadline and trying to do everything the very, very best we can. So the layered API of Fast.AI V2 starts at the applications layer, which is where most beginners will start and it looks a lot like Fast.AI V1, which is the released version of the software that people have seen before. But V2, everything is rewritten from scratch. It's totally new, there's no code borrowed but the top level API looks quite similar. The idea is that in one, two, three, four lines of code you can create a state-of-the-art computer vision classifier, including transfer learning with nearly the same one, two, three, four lines of code, five lines of code in this case, because we're also displaying, you can create a state-of-the-art segmentation model. And actually, like when I say state-of-the-art, like for example, this segmentation model is to the best of my knowledge, still better than any published result on this particular Canvas dataset. So like these five lines of code, super good five lines of code. And as you can see, it includes a line of code which if you say show batch, it will display your data in an appropriate format, in this case showing you segmentation, a picture, and the color-coded pixels overlaid on top of the picture. The same basic four lines of code will do text classification. So here's the basis of ULM fit, which is a system that we developed and wrote up along with Sebastian Ruder for transfer learning in natural language processing. And as you can see in here, this is working on IMDB on a single epoch in four minutes. The accuracy here is basically what was the state-of-the-art as of a couple of years ago. Tabular or time series analysis, same deal. Basically a few lines of code, nearly exactly the same lines of code, and you'll get a great result from your tabular data and Ditto for collaborative filtering. So the high-level API for FastDIV2 is designed to be something where regardless of what application you're working on, you can get a great result from it using sensible defaults and carefully selected hyperparameters, automatically largely done for you for the most common kinds of problems that people look at. And that bit doesn't look that different to V1, but understanding how we get to that is kind of interesting and involves getting deeper and deeper. This approach though does work super well and partly it's because this is based on quite a few years of research to figure out what are the best ways to solve various problems along the way. And when people actually try using FastDIV2, they're often surprised. So this person posted on our forum that they've been working in TF2 for a while and for some reason they couldn't figure out all of their models are suddenly working much better. And the answer is basically they're getting all these nice kind of curated best practices and somebody else on Twitter saw that and said, yep, we found the same thing. We were trying TensorFlow, spent months tweaking and then we switched to FastDI. Couple of days later, we were getting better results. So these kind of carefully curated defaults and algorithms and high level APIs that do things right for you the first time even for experienced practitioners can give you better results faster. But it's actually the other pieces that are more I think interesting for a swift conversation because as the deeper we go into how we make that work, the more stuff you'll see which will be a great fit, I think, with Swift. So the mid-layer API is something which is largely new to Fast to... Actually, I guess the foundation layer is new. So the mid-layer I guess I'd say is more rewritten for V1 and it contains some of the things that make those high level APIs easy. One of the bits which is the most interesting is the training loop itself. And I thank Silver for the set of slides we have for the training loop. This is what a training loop looks like in PyTorch. We calculate some predictions. We get a loss. We do a backwards pass to get the gradients. We do an optimizer step. And then optionally, we run time to time, we'll zero the gradients based on if we're doing... When we're accumulating. So this is what that loop looks like. Run the model, get the loss, do the gradients, step the optimizer, do that a bunch of times. But you wanna do something interesting. You'll need to add something to the loop to do keeping track of your training statistics in TensorBoard or in Fast Progress or whatever. You might wanna schedule various hyperparameters in various different ways. You might wanna add various different majorization. You may wanna do mixed precision training. You may wanna do GANs. So this is a problem because either you have to write a new training loop for every time you wanna add a different tweak. Now making all those tweaks work together then becomes incredibly complicated. Or you try and write one training loop which does everything you can think of. This is the training loop for Fast AI 0.7 which only did a tiny subset of the things I just said but was still getting ridiculous. Or you can add callbacks at each step. Now the idea of callbacks has been around in deep learning for a long time, APIs. But what's very different about Fast AI is that every callback is actually a two-way callback. It can read absolutely everything. It can read gradients, parameters, data, so forth. And it can write them. So it can actually change anything at any time. So the callbacks, we say infinitely flexible. We feel pretty confident in that because the training loop in Fast AI has not needed to be modified to do any of the tweaks that I showed you before. So even the entirety of training GANs can be done in a callback. So basically we switch out a basic training loop and replace it with one with the same five steps but callbacks between every step. So that means, for example, if you wanna do a scheduler, you can define a batch begin that sets the optimizer's learning rate to some function. Or if you wanna do early stopping, you can write an onEPOC end that checks the metrics and stops training. Or you can do parallel training, set up data parallel and be happy at the end of training, take data parallel off again. Gradient clipping, you have access to the parameters themselves so you can click the gradient norms at the end of the backward step and so forth. So all of these different things are all things that have been written with fast AI callbacks, including, for example, mixed precision, all of NVIDIA's recommendations, mixed precision training will be added automatically if you just add a 2FP16 at the end of your learn call. And really importantly, for example, all of those mixed precision things can be combined with multi-GPU and one-cycle training and gradient accumulation and so forth. And so trying to create a state-of-the-art model which involves combining state-of-the-art regularization and mixed precision and distributed training and so forth is a really, really, really hard job. But with this approach, it's actually just a single extra line of code to add each feature and they all explicitly are designed to work with each other and are tested to work with each other. So for instance, here is mix-up data augmentation, which is a incredibly powerful data augmentation method that has powered lots of state-of-the-art results. And as you can see, it's well under a screen of code. By comparison, here is the version of mix-up from the paper. Not only is it far longer, but it only works with one particular dataset and one particular optimizer and is full of all kinds of assumptions and only one particular kind of metric and so forth. So that's an example of these mid-tier APIs. Another one is the optimizer. It turns out that it looks like there's been lots and lots of different optimizers appearing in the last year or two. But actually it turns out that they're all minor tweaks on each other. Most libraries don't write them this way. So for example, Adam W, also known as decoupled weight decay Adam, was added to PyTorch quite recently in the last month or two. And it required writing a whole new class and a whole new step to implement. And it took, you know, it was like two or three years after the paper was released. On the other hand, FastAI's implementation, as you can see, involves a single extra function containing two lines of code and this little bit of gray here. So it's kind of like two and a half, three lines of code to implement the same thing. Because what we did was we realized, let's refactor the idea of an optimizer, see what's different for each of these, you know, state-of-the-art optimizers that have appeared recently and make it so that each of those things can be added and removed by just changing two things. Stats and steppers. A stat is something that you measure during training, such as the gradients or the gradient squared or you might use dampening or momentum or whatever. And then a stepper is something that uses those stats to change the weights in some way. And you can combine those things together and by combining these, we've been able to implement all these different optimizers. So for instance, the lamb optimizer, which came out of Google and was super cool at reducing pre-training time from three days to 76 minutes, we were able to implement that in this tiny piece of code. And one of the nice things is that when you compare it to the math, it really looks almost line-for-line identical, except ours is a little bit nicer because we refactored some of the math. So it makes it really easy to do research as well because you can kind of quite directly bring the equations across into your code. Then the last of the mid-tier APIs is the Datablock API, which is something we had in version one as well. But when we were porting that to Swift, we had an opportunity to rethink it and actually, Alexis Gallagher in particular helped us to rethink it in a more idiomatically swifty way. And it came out really nicely. And so then we took the result of that and kind of ported it back into Python and we ended up with something that was quite a bit nicer. So there's been a kind of a nice interaction and interplay between FastAI and Python and SwiftAI and Swift in terms of helping each other's APIs. But basically the Datablock API is something where you define each of the key things that the program needs to know to flexibly get your data into a form you can put in a model. So it needs to know what type of data do you have? How do you get that data? How do you split it into a training set and a validation set and then put that all together into a data bunch, which is just a simple little class. It's literally, I think four lines of code, which just has the validation set and the training set in one place. So with a Datablock, you just say, okay, my types, I wanna create a black and white pillow image for my X and a category for my Y and to get the list of files for those, I need to use this function and to split those files into training and validation, use this function, which is looking at the grandparent path directory name and to get the labels, use this function, which is use the parents path name. And so with that, that's enough to give you MNIST, for instance. And so once you've done this, you end up with a data bunch. And as I mentioned before, everything has a show batch. So one of the nice things is it makes it very easy for you to look at your data, regardless of whether it's tabular or collaborative filtering or vision or text or even audio. If it was audio, it would show you a spectrogram and let you play the sound. So you can do custom labeling with Datablocks by using, for example, a regular expression labeler. You can get your labels from an external file or data frame and they could be multi labels. So this thing here knows it's a multi label classification task. So it's automatically put a semicolon between each label. Again, it's still basically just three lines of code to define the data block. So here's a data block for segmentation. And you can see really the only thing I had to change here was that my dependent variable has been changed from category to pillow mask. And again, automatically our show batch works and we can train a model from that straight away as well. You could do key points. So here I've just changed my dependent variable to tensor point. And so now it knows how to behave with that. Object detection. So now change my dependent variable to bounding box. And you can see I've got my bounding boxes here. Text and so forth. So actually going back, I have a couple of questions if you're, if it's a, yeah. So if you, the code you've got sort of the X's and Y's and these both, these sounds like these different data types roughly conform to a protocol. Yep, we're going to get to that in a moment. Absolutely. That's an excellent way to think of it. And actually this is the way it looked about three weeks ago. Now it looks even more like a protocol. So yes, this is where it all comes from, which is the foundation APIs. And this is the bit that I think is the most relevant to Swift. A lot of this I think would be a lot easier to write than Swift. So the first thing that we added to PyTorch was object-oriented tensors. For too long, we've all been satisfied with a data type called tensor, which has no semantics to it. And so those tensors actually represent something like a sentence or a picture of a cat or a recording of somebody saying something. So why can't I take one of those tensors and say dot flip or dot rotate or dot resample or dot translate to German? Well, the answer is you can't because it's just a tensor without a type. So we have added types to tensors. So you can now have a tensor image, tensor point, tensor bounding box, and you can define a flip left, right for each. And so this is some of the source code from we've written our own computer vision library so that now you can say flip LR and it flips the puppy. And if it was a key points, if it was a bounding box, it would flip the bounding boxes and so forth. So this is an example of how tensors which carry around semantics are nice. It's also nice that I didn't just say dot show, right? So dot show is something that's defined for all fast AIV2 tensor types and it will just display that tensor. It could even be a tuple containing a tensor and some bounding boxes and some bounding box classes, whatever it is, it will be able to display it. It'll be able to convert it into batches for modeling and so forth. So with that, we can now create, for example, a random transformation called flip item and we can say that the encoding of that random transformation is defined for a pillow image or any tensor type. And in each case, the implementation is simply to call x dot flip LR or we could do the dihedral symmetry transforms in the same way. Before we call, grab a random number between zero and seven to decide which of the eight transposes to do and then encodes call x but what's dihedral with that thing we just got.  So a lot of these things become nice and easy. Hey Jeremy, Maxim asked, why isn't tensor a backing data structure for an image type? A tensor image is a tensor, which is an image type. Why isn't, he says, why isn't tensor a backing, why not have a different type named image, I guess that has a tensor inside of it? Do you mean why inherit rather than compose? Apparently, yes that, yeah. So inheritance, I mean, you can do both and you can create identical APIs. Inheritance just has the benefit that all the normal stuff you can do with a tensor, you can do with a tensor that happens to be an image. So just because a tensor is an image doesn't mean you now don't want to be able to do fancy indexing to it or do an OUD composition of it or stack it with other tensors across that axis. So basically a tensor image ought to have all the behavior of a tensor plus additional behavior. So that's why we used inheritance. We have a version that uses composition as well and it uses Python's nice getAtra functionality to pass on all of the behavior of tensor but it comes out more nicely in Python when you do inheritance and actually the PyTorch team has decided to officially implement semantic tensor subtypes now. And so hopefully in the next version of PyTorch you won't have to use the extremely ugly hacks that we had to use to make this work. And you'll be able to use the real ones. And hopefully you'll see in TorchVision some of these ideas will be brought over there. Can I ask you, so how does that, the type propagate? So if you do arithmetic on a image tensor do you get an image tensor back there? So Chris and I had a conversation about this a few months ago and I said I'm banging my head around this issue of types not carrying around their behavior and Chris casually mentioned, oh, yes, that thing is called higher kind of types. So I went home and that was one of these phrases that I thought only functional programming dwebs talked about and I would never have to care about. You have to care about it because it actually matters a lot and it's basically the idea that if you have a tensor image and you add one to it, you wanna get back a tensor image because it should be an image that's a bit brighter rather than something that loses its type. So we implemented our own, again, hacky partial higher kind of type implementation in Fast.av2. So any of these things that you do to a tensor of a subtype, you will nearly always get back the correctly subtype tensor. Yeah, I mean, I saw that PyTorch recently started talking about their named indexing extensions for their tensors as well. And they seem to have a similar kind of challenge there where when you start doing arithmetic and other things like that on a tensor that has named dimensions, you wanna propagate those along. And I don't know how they type that. Yeah, so we haven't started using that. Yet, because it hasn't quite landed as stable. But yeah, it's, we talked to the PyTorch team at the DevCon and we certainly are planning to bring these ideas together. They're orthogonal but related concerns. Yeah, I just mean that I assume that that feature has the same problem, the same challenge. I assume so, yeah. I don't, it would be interesting to see what they do. Yeah, yeah, it would. Yeah, so it's kind of nice. Not only do we get to be able to say dot show batch, but you can even go dot show results. And in this case, it knows what the independent variables type is, it knows what the dependent variables type is. And it even knows things like, hey, for a classification task, those two things should be the same. And if they're not by default, I will highlight them. So these like lower level foundations are the things that drive our ability to easily add this higher level functionality.  We had to write our own type-to-spatch system. And so we can annotate things with types and those type annotations are actually semantic. And so we now have the joyfully modern idea of function overloading in Python, which has made life a lot easier. And we already have that. Do you have many users that are using this? Yeah. So it's still pre-release. It's not even alpha, but there is a enthusiastic early adopter community who is using it. So for example, the user contributed audio library has already been ported to it. I've also built a medical imaging library on top of it and I've written a series of five notebooks showing how to do CT scan analysis with it. So it's kind of like, it works. And... Well, I was curious what your users think of it because there's this very strongly held conception that Python folks hate types. And you're kind of providing a little bit of typing. Yeah. And I'm curious how they react to that. The extremely biased subset of early adopter class AI enthusiasts who are using it love it. And they tend to be people who have gone pretty deep in the past. So for example, my friend, Andrew Shaw, who wrote something called Music Auto Bot, which is one of the coolest things in the world in case you haven't seen it yet, which is something where you can generate music using a neural network. You can put in some melodies and some chords and it will auto complete some additional melodies and chords or you can put in a melody and it will automatically add chords or you can add chords that create melody. And so he had to write his own MIDI library, fast.io.midi, he rewrote it in V2 and he said it's just like, so, so, so much easier thanks to those mid-tier APIs. So yeah, at this stage, it's easy. I was just gonna jump in quick. I've been helping with some of the audio stuff and it's been really awesome. So it makes things a lot more flexible than version one. So that's probably my favorite thing about it is everything can be interchanged. Nothing is like, well, it's gotta be this way because that's how it is. Yep, that's cool. Cool, thanks. Another piece of the transform of the foundation is the partially reversible, composed function pipeline dispatched over collections, which really rolls off the tongue if we call them transform and pipeline. Basically the idea is that the way you kind of want function dispatch to work and function composition to work in deep learning is a little different to other places. There's a couple of things. The first is you often want to dispatch over tuples. And what I mean by that is if you have a function called flip left right and you have a tuple representing a mini batch where your independent variable is a picture and your dependent variable is a set of bounding boxes, if you say flip left right on that tuple, you would expect both the X and the Y to be flipped and to be flipped with the type appropriate method. So our transforms will automatically send each element of a tuple through the function separately and or dispatch according to their types automatically. We've mentioned type retention, so the kind of basic type stuff we need. One interesting thing is not only encoding, so in other words, applying the function, you often need to be able to decode which is to kind of de-apply the function. So for example, a categorization transform would take the word dog and convert it to the number one, perhaps, which is what you need for modeling. But then when your predictions come back, you need to know what one represents. So you need to reverse that transform and turn one back into dog. Often those transforms also need data-driven setup. For example, in that example of dog becoming one, there needs to be something that actually creates that vocab automatically, recognizing what are all the possible classes so it can create a different index for each one and then apply that to the validation set. And quite often these transforms also have some kind of state such as the vocab. So we built this bunch of stuff that builds on top of each other. At the lowest level is a class called transform which is a callable which also has a decode. Does the type pretension hire kind of type thing and does the dispatch over tuples by default. So then a pipeline is something that does function composition over transforms. And it knows about, for example, setting up transforms. And like setting up transforms in a pipeline is a bit tricky because you have to make sure that at each level of the pipeline, only the previous steps have been applied before you set up the next step. So it does little things like that.  And then you can do those in parallel to get back an independent variable for instance. And then finally we've built a data loader which will apply these things in parallel and create collated batches. So in the end, all this stuff makes a lot of things much easier. For example, the language model data loader in FastDIV1 was like pages of code. In TensorFlow, it's pages of code. In FastDIV2, it's less than a screen of code by leveraging these powerful abstractions and foundations. So then finally, and again this is something I think Swift will be great for, we worked really hard to make everything extremely well optimized. So for example, pre-processing and natural language processing, we created a parallel generator in Python, which you can then basically pass a class to that defines some setup in a call and it can automatically paralyze that. So for example, tokenization is done in parallel in a pretty memory efficient way. Excuse me. But perhaps the thing I'm most excited about both in Python and Swift is the optimized pipeline running on the GPU. So all of the, pretty much all of the transforms we've done can and by default do run on the GPU. So for example, when you do the flip left right I showed you earlier, we'll actually run on the GPU. As we'll warp, as we'll zoom, as we'll even things like Crop. So one of the basics of this is the affine coordinate transform which uses affine grid and grid sample, which are very powerful PyTorch functions, which would be great things to actually write in Swift for TensorFlow's new meta programming because they do a lot of the work in TensorFlow's new meta programming because they don't exist in TensorFlow or at least not in any very complete way. But with these basic ideas, we can create this affine coordinate transform that lets us do a very wide range of data augmentations in parallel on the GPU. For those of you that know about the Dali library that you created, this provides a lot of the same benefits of Dali. It's pretty similar in terms of its performance. But the nice thing is, all the stuff you write, you write it in Python, not in CUDA. So with Dali, if they don't have the exact transformation you want, and there's a pretty high chance that they won't, then you're stuck. Whereas with Fast AI v2, you can write your own in a few lines of Python. You can test it out in a Jupyter notebook. It makes life super easy. So this kind of stuff, I feel like, because Swift is a much faster, more hackable language than Python, or at least hackable in the sense of performance, I guess not as hackable in terms of its type system, necessarily, I feel like we can build even more powerful foundations and pipelines and a real Swift for TensorFlow computer vision library, leveraging the metaprogramming and leveraging Swift numerics, stuff like that, I think would be super cool. So that is the end of that. That was great. That was excellent. Thank you very much, Jeremy. My pleasure. So just sort of thinking through, so as you're propagating along this self type amongst the transformations, that seems relatively straightforward for Swift to handle. Are there other sorts of things that you think we should start thinking about now?  Having them as a value type the way they are now makes some things hard or impossible. So the generic optimizer is a thing that I really, really want you guys to look into and build properly. Currently it uses ugly keypath hacks and it's only kind of partially doing what we needed to do. So I talked to Alexis about this idea quite a bit and we kind of thought maybe there could be like some type that represents the actual block of GPU memory in a way where we can easily share that. Like in practice, we've realized the vast majority of the time we want to refer to that exact piece of memory on the GPU, not this idea of a tensor which may magically copy itself if I change something, you know. And so for example, with the generic optimizer, we need to be able to say like, oh, this layer is part of this layer group and this layer group has these things that need to happen to it. So I actually said to Ed, like, hey, could you please have a look at the Swift AI generic optimizer because it looks, it's trying to be a similar design to the fast AI V2 optimizer, but it's currently pretty unattractive. The second is, I feel like creating a really good computer vision library is something which could be done now-ish. When I tried to do it, I was getting kind of race conditions and freezes inside Swift and I don't have the Swift skills to know where they were coming from or how to fix them. It'd be nice if folks could like, I think all of my answers is like, go back to the stuff that we all built together back in March, April, May and try to start using it in real life and build models with it and put them in production and see the bits where it hits, where you get stuck because you'll find things like, oh, there's no grid sample and oh, there's race conditions in the interaction of OpenCV and the optimizer doesn't quite work properly and that stuff. That makes sense. I think we're also trying to figure out right now what the right path is with the runtime. So we've historically been building on top of the TensorFlow runtime, which is great for a lot of reasons. It has a lot of functionality in the box. It does pretty much everything. On the other hand, the performance particularly in eager mode is not great. I think one of the things we're kicking around is the idea of going more directly into XLA. Yeah. Well, I think that's a thing that's- And XLA being a stepping stone towards MLR and the bigger future, which is also coming. I mean, I think that's the thing that's been stopping us all from using stuff like Swift AI to actually build models because the Autodiff has memory leaks and the TensorFlow runtime is, I don't know if people like some Autogurgles, slow as molasses and implements everything in six different ways and six different places and so forth. So yeah, it's, I think everybody's gonna be thinking into these higher level APIs a lot more once the foundations are working better. Yeah. And so, I mean, the trade-off there is if we go with that direction now, XLA doesn't provide all the things in the box, but I think that's probably fine. We haven't fast-forwarded something, I'm just looking for that stuff that we need it. And so I think we're talking about that, trying to decide what to do there. We're also investing a lot in AD and finishing that off. Yeah. I mean, all the right works being done. It's just, you know, it's just early days. Yes, yeah. I think the challenge that we're really struggling with is this decision to stick with the TensorFlow runtime or to move on to something else. And that I think is complicated, but I agree this is one of the major blockers for adoption of use. Yeah. I mean, especially if you want to take advantage of Swift, which we do, you need something where, you know, the kernel launch time is tiny or better still kind of non-existent because you can write everything in Swift. Otherwise it's, yeah, you don't really get the benefits. Yeah. And one of the, so I'll answer your question a second, but one of the trade-offs there is that XLA doesn't have really fast kernel launch time because it effectively jet compiles things before launching it. On the other hand, there are a lot of opportunities to do, for example, Fusion and other things like that that can offset it. And one of the nice hybrid models you get is this combination of tracing plus compilation, which I think could be really interesting. Yeah. Which is good to explore. Said asked, what's going on with MLNR? There's tons of stuff going on, it's really exciting. Just yesterday there was a really fantastic talk from some folks at Intel talking about their code generation algorithm that are bringing over to MLNR, which I'm really, really, really excited about. And so there's tons of stuff going on. Getting the ideal code gen for NVIDIA GPUs, for example, is probably still six plus months away. And I don't know how much plus that is, but what I'm encouraging is the community come together and collaborate instead of the different teams and the different companies, like kind of being in front of me. And the Intel stuff that they presented yesterday is super, super impressive. And so we'll see what happens with that. The other thing I might. The other thing I might mention a lot in this stuff as well. The other thing I might mention in terms of, tales from the other side, what's life like in the Python world. Things that are and aren't working well over there. The kind of the answer to switch for TensorFlow in the PyTorch world is JIT. So it's basically to trace your Python code and attempt to figure out what it's doing and create a what they call TorchScript, which is a dialect subset of Python or else to actually parse your Python code is also an option and turn it into TorchScript. It has reached the point now where it can actually be used for good.  It's a very nice activation function that's outperforming everything else that anybody's trying it on and it was pretty slow. And when we just took me half an hour to create a JIT version and it ran at the same speed as somebody else's hand created code. So for like small things like that where it's like two or three lines of code, that's working pretty well. Although for bigger things like a new batch norm implementation we tried to do during the last course, the performance wasn't there or if we actually tried to take, like one of the big problems at the moment, not just for Python, but the whole world of non-Google people is that the best computer vision models by far are largely those that have been coming out of Google, like Efficientnext, Nix Nets, like Cochle's team. They run very slowly and with a lot of memory on GPUs. And so we tried wrapping an entire EfficientNet and MixNet into a Jitted thing so it wouldn't be so slow. The MixNet didn't work at all and the EfficientNet was a little bit slower. So that's kind of the status of JIT in PyTorch is, you know, bits of it are useful. The way I look at this from the compiler is that every cogeneration eight pieces that I think the MLR pieces are all going the right direction. They're just going to take a while to get here. XLA, as far as I know, is state-of-the-art in cogeneration. For the things it does, it does quite well. The challenge with it though is that it does have sort of limitations like static shapes and the number of offset supports. And so you kind of have to be within its world for it to be useful. But it has a very useful, it has a large subset of the world that it covers very well. It has a pretty useful world. My understanding is that the base model of TorchScript and the interpreter they have, I understand that's quite nice. But the kernel fusion pieces is still fairly early on, it's mostly Elm-wise operations, for example. I don't find them that quite nice. I mean, like simple things, like they're partly a limitation of the Python type system. So like you wanna be able to write things that can work with different nums of channels while you're out of luck, because they use Python height limitations, which have no way of saying it's a tuple of size n, you have to set a tuple of size three, so you then have to hard code all these assumptions into your code. Lots of stuff I find pretty frustrating. I see it interesting. Well, so I mean, I think there's other spaces that I'm eager to reevaluate as, I mean, this isn't the highest priority at this moment, but in terms of our APIs, there's still very legit questions around, should we encode D-type in the static type system, or should we just say tensor, right? And if you just say tensor, then you get rid of all the generics everywhere, cleans up tons of code at the cost of losing some of the checking. But then I think if you go with more semantic tensor types that Jerry was pushing forward, you actually really don't even want the D-type, if you want is the semantics and that you're actually in a better spot. Right, like for big decision, we're switching stuff from one type to another all the time. Depending on whether you're doing a loss function or a gradient calculation or whatever, you need to be changing between half and single. So like if we went that direction, I think that would be really interesting in terms of ergonomics, but also simplification, which I think would be great. And also like your point about the optimizers, the key path, I have all kinds of weirdness because you have multiple D-types and you want to be generic over D-type. And so that's really unclosing right now. I think also like for Swift, wanting to bring over big world of Python using data scientists, they're definitely not going to be wanting to put lots and lots of verbose generic type annotations in their Jupyter notebooks. So I don't know when we'll have cycles to re-evaluate those APIs, but I think we should go do a fresh take of this and combine it with an XLA-based approach that changes a lot of the trade-offs. Right. It's really interesting. Yeah, I mean, I think in my mind right, so a couple of weeks ago I presented the layering proposal to separate out LibTensor from LibDeep Learning so that we can then get the freedom to then iterate at that level and have multiple explorations on top. So the progress update on there is that I've started, we have the two different packages now in Swift DPI's so you can depend only on one as opposed to the other. And Dan helped fix all the issues that I caused while doing the initial move of the random number generators out of, but we'll become LibDeep Learning. That said, it's still very early and I have a lot more code to move. Well, I think the Jeremy's like fundamentally right, that we need to spend more time with Swift AI and the optimized designs and re-evaluate the training with callback systems and things like that. Yeah. As each of these variables change, like it affects other parts of the system and different trade-offs, I think should be re-evaluated just between that. But I think that getting AD like full of proof is like super important. And performance. Yeah. We have to get those two things right. Well, and upstream and integrated in Swift so that we can build on it and take it for granted. Yeah. Quick question about tensor.d type. Yeah. I wonder if we would add any type assertions in any functions. I think the Python model is to not check things and to let things crash at runtime, if I understand. I don't know. So I don't know. I mean, I think that there's a couple of different options there. I don't know what the right answer is, but again, one of the things that PyTorch is doing is they're doing more coercions with d types. So if you take an intate and add a 10 and 32 it will actually promote then a 10 and 32, for example. I mean, rocket science, but you know, that's the kind of thing that it's just very nice and it just eliminates a certain kind of error. On the other hand, it's kind of like broadcasting where it makes certain things just work at the cost of potentially getting surprising in some cases, I don't know about that. I think a few things that don't make sense like you try to do a put a point operation on an integer then you would want it to be a runtime error. I think that our model is turning towards a much more runtime centric approach. I think ironically, Swetford and Swet started out very static, but now like throwing a very dynamic. Yeah, for me, like I'm realizing like one of the major benefits of having a fast fast language is like dynamic is free. And so now you can have super dynamic abstractions that you can do these things in a nice way. If PyTorch, you do get a pretty clear runtime error if there's a type mismatch, it doesn't just crash, it will tell you what it's better than what it got. Yeah. And one of the nice things about eager motors is that then you get a snack trace. I think there are other ways where instead of encoding things into the static type system that you have to adhere to, right? I think Adam's work on just fitting perfectly shows that you can still get a lot of benefits of static analysis without necessarily coding into the type system. That said, I think it's still an open question as to how far we can really push that where we end up landing. Yeah. I think it's just a really great, really great opportunity to re-evaluate these things as other pieces are coming together. Maxim asks, why is runtime tracking preferable over static analysis? I think it's more that we're still trying to figure out what dimensions you want to be flexible on. And so doing things dynamically is sort of the ultimate in flexibility. And so as we're trying to iterate on the programming model, making sure that things are as dynamic as you want them to be is sometimes nice. And then we should think about how static analysis can help catch errors sooner. Yeah, exactly. And so this is just a spectrum. And it's not that one end of the spectrum is better than the other. It's about where in the spectrum you end up. And this Nicholas's question, Nicholas asks, how are MLIR and XLA related? That is a super complicated question because we're actively re-implementing pieces of XLA in terms of MLIR. So that's actually a lot more complicated than it sounds. I would just say that MLIR is a broad scale compiler technology that solves lots of problems. XLA, as a name, is typically thought of as the thing that turns sensors into efficient code. And so I wouldn't overindex on the number of letters, I guess. And once Swift has one of those on top of MLIR, we'll still use XLA to target TPUs. Yeah, so we're, I mean, this is internal work, but we're doing a lot to change and enhance the TPU software stack in XLA. And things that are XLA are changing in their implantation as well. And so there's a big investment going on in all these pieces right now. And I think that more generally, again, if you ignore which letters get attached to them, the effort here culminates in a much more flexible cogeneration stack, support for dynamic shapes and custom ops and things like that. It's just that different pieces in this very complicated technology come together at different points of time. I don't know what the marketing, the crack compiler marketing team will end up labeling the resultant thing. Excellent, we're slightly over time. So I just wanted to, you know, unless there's any pressing questions, thank everyone for joining and see you all next week. I think next week, Mark will be up talking about some of his work on testing the auditive system to ensure that it's really reliable. There's some pretty good things that Mark's been up to there. It's also exciting that AD is getting upstream to master too, which is really cool. Yeah. Thanks everyone, have a great week and see you all next week. Thank you.


Radius Gold must single-handedly have the absolutely best about paragraph. You say right at the onset, you've got projects located in Mexico, the US, and Guatemala, and then you clearly explain what you're doing, which means you're seeking out additional investment and project acquisition opportunities across the globe. And congratulations, it looks like we have one today. Yeah, I mean, my job is to find minds. That's what I do and what I've done forever. So I just try to find great projects and we're constantly on the hunt for those. And I think we have found a new great project. So it's called Tropico. It's in, I can't imagine a better place actually. There's a few places in the world that are outstanding, Nevada and Central Mexico for discoveries and Tropico is new, undrilled, right in the right place. Okay, so first of all, let's go back to the name Radius Gold. And I think I read in this news release, which I love it, there's photos of the actual project and you standing on the project property. You've resigned from another project so that you can dedicate yourself to the Radius Gold deal. Is that correct? Yeah, well this new discovery, not many people understand that Fresno district is the world's oldest mining district. It's been going since 1470, no 1550, so 470 years constant mining. That district has produced over 30 massive deposits and most of them buried hidden. And just down the road we've found a great target that's undrilled. It's a gold breacher pipe in one of the world's biggest districts. So we're pretty excited about that. And of course Radius is a member of the Gold Group of Companies led by Simon Ridgway. So I just want to stress this is gold, gold and I think we've got some silver too, yes? Yeah, well the systems in Fresno are silver dominant but with gold, lead, zinc, copper they're zoned. So at the new discovery it's gold at surface. We don't expect to find silver at surface but they're zoned over depth. So I'm sure that it will change as we go down and we'll get into silver and base metals at the bottom of it. But these things go down a kilometer or more. And so the gold rich part is right at the top. And Bruce, like the Gold Group of Companies, you guys are very old school. You don't just raise lots and lots of money but you're constantly finding projects, vesting, making announcements and moving forward. Would you like to talk about your agreement terms for instance? Yeah, as you say Gold Group is very frugal with our budget. We don't put out stock. Radius hasn't done a capital raise for an excess of 10 years. So we're very frugal with our money. Our shareholders with me, Simon, our core group of shareholders, we don't like to dilute. So we have some cash available to drill this property. Eventually we'll have to raise money but not today. So the deal is that we've paid 200,000 cash up front. I've been chasing this property for two years. It's taken me an enormous amount of time to get them to sign. But we've paid 200,000 but we can now go ahead and drill and test it and figure out what it is before there's any further payments.  And the big payments only come if we make a discovery. If we find a world-class mine, there's some significant payments. But we've got four or five years to do it. So Bruce, we absolutely love Radius Gold and we're so delighted to get an update from you. I think the last time I spoke to you I said love the Radius Gold model, would like to hear regular updates and that was what six months ago? Yeah, about six months ago. We tend to find all of our projects are grassroots discoveries. We made them all through prospecting. So we've got five projects, six, and we don't usually go to other companies, buy them advanced projects. We go and prospect and find them. This recent one is owned by a couple of Mexicans. It's taken me a long time to get them to sign. But all of the projects that we have have never been worked on by anybody else. And I tend to find one once a year, once every two years. So we have a constant flow of good properties. We don't take them on because we think we're going to join venture them or we're not a joint. We take them on because we think there's going to be a discovery. And sometimes market conditions, financing, we decide to bring in partners. But usually my only concept for taking on a project is discovery. And that was the point I was trying to meet Bruce, which is how lucky everybody out there at Investor Intel Land is to even see you and hear how it can be done. Bruce, thank you so much for joining us today and congratulations. Thanks, Tracy.


Welcome to MOOC course on Introduction to Proteogenomics. In last lecture, a guest scientist Dr. Ratna Thangudu, he started providing you an overview for the large scale data sciences. Today, he is going to continue his talk and mainly focus on the Clinical Proteomic Tumor Analysis Consortium or CPTAC. I think it is really important to know that there are some resources publicly available, which are sharing large amount of data set. I think TCGA, the Cancer Genome Atlas from National Cancer Institute was once such initiative, which has really provided large data set to the broad community. Thousands of patients entire genome for various type of tumors were sequenced and then that data made publicly available. It was so interesting to note that many of those data set while the original data was published already in nature, but from the same data set, many people started probing specific questions that what could be the effect of let us say survival for based on different genes and then you know they did metadata analysis, published papers based on that. So, not only the original papers which generated data, those are published, but also many associated papers came out solely based on the data available in public repository. So, the Cancer Genome Atlas really made a huge impact to the broad community for sharing the data and making the publicly available. So, in this slide, the CPTAC provides a very good resource for the community to look at the proteomic data of different tumor types. So, Dr. Ratna will talk about the Common Data Analysis Pipeline of CPTAC and Cancer Imaging Archives. Various omega data repository such as even imaging data for various cancers, genomic data from the NCI GDC portal, as well as the DB GAP some of these are good resources repository is available for obtaining large data set which is already available from the you know very nicely done experiments using next generation sequencing as well as mass spectrometry. The huge amount of resource have been already put in to obtain these data and now the data is made available to public for doing further analysis. So, Dr. Ratna will actually eliminate you, provide you more information about how these repositories these resources could be accessed and what kind of features are there and how you can make use of them for your own research in which way you can analyze the data or use the data for your various comparisons. So, let us welcome Dr. Ratna for his second lecture on the Big Data Sciences. So, the interesting thing with CPTAC program is we do read all of the data even before the publications come out that is a pretty interesting way to look at the data right. So, if you are generating the data you are most of the times you are already worried if I put my data out there somebody else will publish, but that is actually hindering the progress of the research in terms of how the government sees. So, there is some guidelines that come put forth saying that we will give you the data, but there is an embargo date. So, I mean I am just showing like we are pretty active like every other month we will release some one study we harmonize all the study and at least on the top these 1, 2, 3, 4 those studies they do not have any publications yet. The groups are working Dr. Mani is working on one publication, Dr. David Fenyo is working on another publication right now and Bing Zhang is working on another publication alright here. So, there is something called an embargo date. So, the expectation is that all the data is free, but we expect use it, but you wait until that date before you publish because that means basically we are giving credit to the data generators from the consortium to publish first. So, this is the common data analysis pipeline that we done. Every piece of data that comes into CPTEC data portal we have all of the Amazon based, Galaxy based infrastructure to run a pipeline on that. This is basically as you can see here it is MSGF search and then we have a custom built protein report summary generation and at the end the files that I am pointing out earlier is basically identified peptides, identified proteins and the quantitation data and also the study sample consistency report which is basically QC matrix. So, there is a lot of information out there. So, without downloading any of these raw files you can just use this information, but if you are not satisfied with the results you can always get those raw files and generate them. And another thing you would appreciate if you notice is basically all of the mass spectrometry files they are proprietary that they come from the instrument. So, you need to have a windows machine there is a lot of complications how you convert them into open source. So, most of the pipelines run very smoothly and seamlessly in Linux environment. So, there is a bottleneck that we have to do the first steps in the windows environment. So, what we do we follow open standards and we make all of the we convert all of the raw data into MZML. MZML is the re-representation of the proprietary formats in the open data formats. So, all of that information is available as MZML also. And all the other information that comes as a pre-spectral matches and other things we convert them into the open source formats MZML, MZID, and so on wherever it is possible. So, this is once you are on a particular study page you see a lot of 43 studies listed there you can click on a study once you go there. So, the first thing is because it is all cancer related patient derived. So, we take utmost care to actually curate all of the data the clinical information and also the experimental design. So, we put all that information in these files they are just simple excel files you can just download, but it has a lot of information that connects you to the files in here. And then we have also metadata packet. So, that has all the protocol information past protocol information and also we have the protein reports that I mentioned earlier. So, they are all packaged into one single packet that you can just download those two for example, we do not need anything else the bottom ones are raw files. And as you might have noticed here it is like R dot R1. So, what that means is we have we version all of the protein report results what happens sometimes we find something interesting that we need to update the pipeline itself. So, then we will read on the pipeline and we report the results as a new version. So, always you can trace back what that means alright. So, I talked a lot about all this mass spectrometry data I showed, but where is all this other omics data, where is the genomics data, where is the clinical data, where is the imaging data. So, that is all the CP tag is producing where is it sitting let us see. The imaging data sets in cancer imaging archive the genomic data is sitting in GDC data portal. The other part of the genomic data that is a SNP aerodata it is sitting in DB gap that is the database of genotypes and phenotypes and the proteomic data is sitting here excellent. So, we have data from one single patient sitting in 4 different areas. So, how do you connect all of them? So, we are generating data it is extremely difficult even for me to connect all the dots and it is not helpful right. So, a lot of information being lost. So, what to do? So, Dr. Henry actually mentioned about a precision medicine initiative that came to light in the last 4 years you say Joe Biden's cancer moonshot initiative. So, as a part of that the National Cancer Institute it is developing a cancer research data commons. It is a big ecosystem in theoretical ecosystem where all the repositories here these tags are basically kind of repositories. So, genomics proteomics imaging so on they all coexist together. So, physically they are at different locations on different servers, but in the ecosystem. So, they are together and then we provide the ecosystem provides analysis of all of the information. So, now I talked only about earlier the common data analysis pipeline for proteomic data, but we will have tools the expectation that we will have tools to analyze the proteogenomic component. And then we will have data models and dictionaries to represent the data. So, that I when I call a patient I am calling that patient from all different resources at one time alright. So, then we have visualization we have quality basically if you go to any portal you have all these features. And the kind of users that we are expecting is you see that wide variety of users we are looking at. The patients, clinicians, computer scientists and tool dollar person and biomedical researchers. So, everyone's export is different everyone's expectations are different. So, such an ecosystem trying to support so many different kinds of users it is a it is made known as F4 it is needed alright. So, and then there are cloud resources. So, I have all these data sitting there, but I want to analyze the data myself. So, there are 100 data sets sitting I will pick and choose 3 or 4 different data sets and I want to analyze by myself how will I do it. So, that is where we have cloud resources. So, Dr. Mani has talked about the fire cloud the other day. So, fire cloud is one such resource. So, you just need to have a login you do not have to take anything other than your credit card alright.  So, already there is one NCI genomic data commons which came to write 3 years ago. So, most of you might have heard about the TCGA the cancer genomic class and all of the data used to be available in a TCGA data portal that is very specific to TCGA. So, but now there are so many programs coming up NCI third we have to bring all of this genomic data together at one resource it is not just one program specific resource but rather a common resource where all the genomic data is needed. So, that is what they did and right now there are about 40 plus programs there are among us program that there is so much data there or there and this is a free resource there is no login required and you can see there are so many different cancer types available already. So, this is all about the genomic data and then I talked about the NCI cloud resources. So, Dr. Mani already talked about the broads fire cloud. So, cloud is it is a public cloud see somebody else is actually offering you services. So, you do not have to do anything you do not need to have a cluster right. So, cluster yesterday I mean you are asking me I have this much data. So, I do not have any disk space what do I do. So, I buy a new disk I will attach it reconfigure it and then 10 days later more clients come and you generate more data. So, what do you do and there is no solution. So, people are slowly moving towards public clouds these are Amazon web services and Google cloud I can I mean anyone can actually have a free account you can log in try to explore the good thing with that is you do not need to have a lot of informatics expertise to begin with do not be scared just go out there see what is there lot of them are made as services. So, I want to do this.  So, so broad institute of system biology they have cancer genomic cloud and then 7 bridges 7 bridges is a private company is a commercial partner with NCI in developing, but they did determine AWS. The fire cloud and the 7 bridges I would recommend you to take a look at those 2 resources. Institute of system biologies is more of a program programmer centric approach they took. So, they made all the data available in a certain way and you can access through APIs programmatic. So, you need to have little expertise in understanding that, but they do have some kind of a UI where you can actually look at the results. So, now the genomic data commons and the cloud resources are the only things from the ecosystem I they showed they are existing as of today. So, last year this proteomic so, NCI thought all the CP tech data is sitting just like the ways TCGA used to say it is in its own specific resource. So, what do we do about that? That is why they came up with why do not we have a proteomic data commons because there are more programs that are coming out it is not just a CP tech, but there are Apollo and ICPC that I will talk a little bit later on and Dr. Henry talked about that. So, we need to have a similar resource, but we are asked to actually have both the both the GDC and the cloud resources combined. So, we have the data and also analysis tools at one place for the proteomic data that is pretty ambitious yeah that is what I mean. So, now that is the proteomic node, but it needs to do both the things that the GDC and the cloud resources doing together. So, the very high level goals of the PDC and the proteomic data commons, unsilow the mass spectrometry data everybody is storing on their own local spaces right do not do that share it publicly and the move from a situation where people move the data to the local tools. So, same thing I am telling you do not do not download the data you bring your tool to the data because data is so humongous it is not just your local storage it is about how you transfer the data does IITS network allow you to transfer so much of data right I do not think so it is not worth it. So, instead you bring your tool go there to the cloud and analyze and then this is an interesting thing I always shift from a data graveyard model to a data workspace model what this means is graveyard the one I talked about earlier you go you deposit your data after the life cycle of your research that is like you are dumping it somewhere I am done with this I do not want this anymore right that is data graveyard. So, nobody looks at that kind of data, but the workspace model that we are proposing with the proteomic data commons is you connect your instrument directly to the workspace on the PDC. So, the data directly moves from the instrument directly to the workspace. So, there you attach all your metadata when you when you are ready to analyze the data at that point you actually attach all the metadata all the samples and the study design and which tools you want to use and you click a button and on the pipeline. There is no such I mean there is no. So, in English case it depends very much like you first store the data on the cloud then you to download it analyze and again upload it back. So, it takes you back to the same problem. So, is there any initiative that has been taken to make more and more company based softwares like to develop any such cloud system where the analysis could also be done and then there is a need for a physical drive where they can actually explore. Yeah. So, that is the cloud thing actually doing. So, you can whatever the tool that you develop you can dockerize that. So, dockerization maybe you have to learn a little bit about that, but basically it is not just the tool, but the environment the computer system where it runs you package the whole thing and put it on somewhere. It is like a another database it is called dock store. So, once you have the tool you can take that particular tool to the cloud and run it. And like here I said these are the high level goals they are goals. So, we are not there yet there are a lot of we have to cross a lot of hurdles to reach to those goals. And in PDC we are starting small. So, we will make a couple of pipelines available initially and based on the users interest we will add more. So, that way we know what is interest in the community just I mean I cannot we cannot have a resource that tries to do so much and and then give it to you and you see are the users come and see like 90 percent of the stuff ok we are not interested in this. So, that is not the good use of money or time or therefore, right. Did I answer your question? And then we have to improve the metadata annotations and ensure the data is annotated well like the standards I was talking about. All right just to give you an idea of what kind of data you will see in future. So, CPTAC data that is I already talked about then we have Apollo and the international cancer proteogenomic consortium that is Dr. Shiva's way is part of. And then human tumor at last and last I said user generated data that is your data to start generating the data you can approve. All right. So, what we did about 6 months ago we started building a prototype that is what we call proteomic data commas pilot or in other words we call MVP or a minimally viable product. So, what we did is like I talked about a lot of things right now. What we did is like build a minimal product and put it out there. It has some basic features that I am talking about. It does it does it will not do everything, but bits and pieces of everything I talked about. And then see take the feedback from the community and develop on those right. So, rather than I developed the whole product and I put it there like I said and nobody is interested in that it is rest of our effort. So, we will the portal like the commons will have a data submission system a portal where you will see all the information and a workspace that I talked about where you can actually upload your data and run pipelines. And then programmatically or the computationally access that information to the problems. Yeah data processing and harmonization is the common data analysis pipeline that we run all of the data. So, what happens like within CPTAC. So, each of the cancer types is being the data is being generated by different groups and it is analyzed by different softwares and different protocols. So, when we start putting out the data and the through the portal. So, it is we are one single consortium, but we are actually providing results from so many different kinds of pipelines. So, what the even when you go to the genomic data commons. So, we have both these portals what we try to do is this called data harmonization process. Any data that comes through these resources we will use a one single common pipeline and all of it. So, it is not an ideal way, but it is one way to get all the information on one single. So, that is something what the pride is doing what preparatory class are doing. So, you are depositing all the data in one resource of your publication is done and they are running through the one common pipeline. Because when you have so much data if it is not harmonized as per one single pipeline it is very difficult to make sense out of it. And then that is that is what I call processing and harmonization and the processing workspace is basically you would be able to run the pipeline by yourself right. We run the pipeline as a starting step, but you want to change something some parameters. So, I am not very happy with this you want to just change some parameters you should be able to do that. So, as an example we will have encyclopedia as a one DIA pipeline and also the DDA pipeline that I described earlier which is just showing the software architecture of our system. And you do not have to actually spend a lot of time here, but just to give you an idea of this is called S3 this is a hard drive you can say the cloud hard drive where you put all your data. So, it is scalable.  So, there is no limitation here the same thing like you want more compute power with a click of a button you can add as many processors as you want. And this thing it is called authorization. The idea of this is basically because there are so many ports I talked about genomic data commons, proteomic data commons, imaging and the cloud pilots there are so many things are there, but the user has to remember all his user IDs and passwords and how do these commons actually talk to each other if it is not a single sign on. So, the purpose of this box is basically telling that we will have a single sign on once you sign into any of these these resources. So, you will be able to access the data from the other resources seamlessly. So, in conclusion today you have learnt about one of the large initiative from NCI about the CPTAC which has contributed the scientific knowledge for the cancer research immensely. You also learnt that to obtain omic data for a single patient you need to search for different repositories. Hence, the NCI has taken initiative to make a common data portal. So, that you do not have to look for variety of you know different lists to search for data and all data could be commonly accessed from the cancer research data commons or CRDC. You also got a glimpse of how CRDC has different features such as visualization, analysis, query and many more features. We have also learnt that how cloud platform which is being freely available to everyone can be very useful in handling and analyzing big data or even meta data analysis. In the next lecture, Dr. Ratna will continue his talk about large-scale data sciences and inform us more about different publicly available portals. Thank you.


Hey guys, Dylan Schumacher, Citadel Defense, and today we're going to talk about people who don't know what they're talking about. So I'm really surprised that people still make these videos and put them on the internet where they're talking about things they don't know about. And by the time this gets up I know I'm going to be late to the party and several other people will have already posted corrective videos of this guy's actions, but I thought I'd take a swing at it anyway. So Grandmaster Jay from the Not Fucking Around Coalition, the NFAC, posted this video and this is an excerpt, this is like a two or three minute excerpt from a larger video of his, which I've not watched the whole thing, so go ahead and take that for what it's worth. But this clip is just bizarre and asinine. So for those of you who know about guns, this will be comical, for those of you who don't, I'm hoping to provide some education so we can avoid not being educated. So here we go. Let me demonstrate something for y'all. Y'all hold on a second. We're here on the morning mental.  Give me a weapon. Is the weapon ready? Yes. This is the bullpup. Here's the bullpup. It's earlier. In Germany. This is an XR-15 shorty.   Okay, first thing, that's clearly a lefty AR, right? Because my bolt release and bolt care group is on the right. His is over here on the left, so it's a lefty AR, which is fairly unique. You really don't see those very often. But secondly, that's not a bullpup. I don't think he understands the definition of a bullpup is or maybe has never seen one. A bullpup, ladies and gentlemen, is where the action of the gun, usually where the magazine is going to be and all the bullet shooting is going to happen, is behind the trigger. That's what a bullpup is. That means the mag will be back here, the bolt carrier is back here, everything is going to be back here. That's what a bullpup is. That action is behind the trigger. A bullpup does not mean that it kicks. That has nothing to do with a bullpup whatsoever. And also, again, he's holding an AR-15, he's holding a 556 gun, and they don't kick that much. So, let's keep going. The interesting thing about this weapon is if I pull back the bolt, see that? See the bolt? There's no bullet in there. Pay very close attention. If I'm not paying attention and I take this weapon and I bang it, look what just happened. The bolt goes forward, the weapon has fired. I didn't touch the trigger, I just simply banged it on the table.  It happens in basic training every day and you all get a letter and a flag saying your son was killed in a training accident. An old shot done. If you drop it, it will fire. So don't believe these motherfuckers running around talking about you shouldn't have your hand on the trigger. You don't have to have your hand on the trigger. Let me show you again. Okay, so let's lock our bolt to the rear, shall we? Bull lock to the rear. Okay, let's take a magazine with bullets in it. Okay, magazine with bullets in it. Now, this is just a pretty weak table. I don't know if I'm going to be able to get the same effect here. Hopefully you can see that. I do not have a hole in my ceiling. The gun did not go off. Open bolt guns like machine guns will of course fire when the bolt goes forward because that's how they're designed. Close bolt guns like AR-15s will of course not fire. Watch for the bolt to come out. There it is. Will not fire when the bolt goes forward. If that was true, if that was the case, then AR-15s would fire every time you loaded them. Every time I put a new mag in there and I closed the bolt, the gun would go off. Or every time I put a mag in there and I rack it, the gun would go off. That's what would happen if AR-15s fired when the bolt closed. You can do this with your AR-15 at home. I'm sure there's going to be a ton of other videos of people doing this with AR-15s. AR-15s do not fire simply because the bolt goes forward. Same thing on a handgun, same thing on a shotgun, unless it's like a slam-fire shotgun, which we'll talk about that in a minute. There's just not... Unless the shotgun has slam-fire, I should say. Again, I can't understand that people actually put videos like this on the internet because this is so asinine. So it does not happen in basic training every day because, one, guns don't fire like that. And two, I don't think there's 365 people a year or more that die from training incidents. It just doesn't happen. So let's keep going. You just committed an incident. Your hand was not on the trigger. It doesn't make a difference whether it was on safe. Unless you understand weapons for real. Don't you listen to these motherfuckers who have no intention of ever picking up one and who don't understand that weapons go off by themselves all the time. I find it ironic that he says, unless you understand weapons for real when he clearly doesn't, I also think it's really interesting that he says weapons go off by themselves. No, they don't. They don't. If they did, they wouldn't be reliable and we wouldn't use them the way we use them. Weapons do not go off by themselves.  That's called an accidental discharge and we'd sue the manufacturer because guns don't do that. Now, I will say he mentioned shotguns here. Now, I'm not 100% sure on shotguns, but I do know that there's been cases of shotguns going off in car accidents and things like that because they have floating firing pins and things are a little bit different, which is why, by the way, police typically carry their guns cruiser ready, which means the tube is full but there's not one in the chamber, so you have to rack it when you get it out of the trunk. So there's stuff for that. I get that and that's a side note. He probably doesn't know that as my guess, but he's so incredibly wrong on so many things. It's funny and I can't believe that he doesn't know he's full of it. He must actually believe it because I don't understand why else you'd put that out on the internet. You got to know if you don't know what you're talking about like that. You're going to get crucified. So I just wanted to try to help educate you in light of some misinformation in the world. Air 15s do not go off by themselves. They're not dangerous magic death machines.  That's it. That's all I got. Leave a comment. Do brave deeds and endure.


Hi everybody, welcome back to UiPath Forward. Six, theCUBE has been every forward except the first one, two through six. We're excited to be here at the MGM Grand in Las Vegas. My name is Dave Vellante. And we're really excited to announce the AI 10 award winners at UiPath. I'm here with Maureen Fleming, who's the Vice President of AI and Automation at IDC. Maureen and I were judges in the competition. Sid Shah is here. He's an automation leader at Intel Jeff Rittner, who's the Chief Trade Officer at Intel. And we're going to talk about their use case. First of all, folks, welcome to theCUBE, all three of you, and congratulations to Intel for one of the top award winners. And we're going to get into your challenging use case, but pleasure to have you here. Thank you. Thank you. Thank you very much. It's great to be here. Very exciting. Yeah, so Maureen and I were judges along with Adrienne Bridgewater, who is an independent journalist. And then Dr. Ed Chalice, whom you guys know, and Palak Kadakia, who's the VP of Product Management at UiPath. And essentially, Maureen, describe what the judging entailed.  And we had to look at their objectives, what we were trying to get out of the automation, and then the details about the implementation so that we could understand what was being done. And then the metrics are the outcome of the automation. So we had to score on each of these different categories and the scoring was interesting because we had to compare it with everyone on almost like a relative scale. So it was a great, the use cases were all really great. Yeah, it was quite simple. You had this, it was basically the objectives, the implementation, and then the results. But then in a scale of one to five, and then as you know, Maureen, you would go through it and say, hmm, I actually think this one was more interesting so you'd have to go back and jury rig the scores. But- Well, no, it just changed them to weight them. Yeah, we did self-weighting. That's what I'm telling myself. But so the use case here was international trade, the international trade function. Yes. So, Jeff, given that that is your area of expertise, can you explain what that is and why it's so challenging? Absolutely, yes. I run Intel's global trade organization. We have 130 people around the world. And our job is to make sure that we get the freight, whatever we're shipping across the border quickly, with the least amount of cost, and efficiently. And it's a huge challenge if you think about on a given day, the number of transactions that cross a border in every country, every port around the world. And at Intel, we're moving freight all the time. And so I have a team that have to look at everything and figure out what's the right classification, what's the country of origin, the valuation, and so on. And so we have people that do this all day long and it's very manually intensive, working with our systems and so on. So the use case really was an attempt to automate what we spend so much time doing manually. And so that's the exciting part. And is compliance part of that?  Is that part of the deal? Sorry, you guys, had to get that in. But yes, but that's an important piece, right? You can't be shipping things that aren't allowed to be shipped to various countries. Absolutely, there's the efficiency piece, but there's also the compliance and making sure that what you shipped, it's accurately declared. It said, so how does, how did AI and automation fit into this? So basically what we did is took our finished goods and non-finished goods products and we had historical data for what we call them as HS codes or it's a tariff code to import any merchandise into a country, right? And that's decided by the government. We took 100,000 finished goods products, 100,000 non-finished goods, put it in a machine learning model and then we started our metrics to answer your question. We set it at the bar really high to 95% threshold should be our minimum requirement. For finished goods, putting it that in the machine learning model, we got 99% accuracy and for non-finished goods, we got 97% accuracy. I think the key challenge here was the HS code keeps changing. So the government keeps changing the codes. The minor updates are two to three months, major updates six months and a complete overhaul every four years. So now I think I have historic data, I build a model, great. But the codes change, how do I take care of that? So we build a retraining pipeline where the model gets retrained, we have an ingestion layer where we put the new codes, the model gets retrained every quarter and that's how we maintain that accuracy. So the goal, the objective if I got it right was 80% of the products get auto-classified to 95% accuracy, so smart. You sandbagged a little bit, under promise, over deliver. You came in at 99% after 16 weeks and that was 56,000 products. The current numbers are, we've been live for 25 weeks and in 25 weeks we've done 35,000 products at 99% accuracy. So in a week about 1400 products is what we are classifying. And this is main transformation instead of, like Jeff said, instead of manual work, it's machines, we're transforming how we're doing the work, so. Now Maureen, you and I were talking off camera and you were saying that you really like this use case because you actually had some domain knowledge about this space and so your degree of difficulty radar was on. So explain why it's so difficult and why this resonated with you. So first off, it was more tied to volume, the scale of what was going on. The other piece was the difficulty of clearing customs and then on top of that, I also was thinking about sort of lead time, you know, the metric, you know, I don't think that that was part of the thing but I was thinking about lead time. Anything you can do to speed up and get as many goods to the destination as possible on time has an impact on the business, the cost of the business. So I felt like, you know, that was a pretty good use case. And are there other sort of examples or patterns you're seeing in your research, you know, trends in AI that are sort of relevant to this discussion that you can share? Yeah, another winner, I think it was Omega Healthcare, they were doing this, it's the same sort of thing but for medical recording, medical records coding. And so for them that had, again, I gave them a really high score because they were able to improve the speed of doing the coding by I think 97% and improves accuracy and they also improve customer satisfaction. So to be that whole idea of taking a big reference database that has to be done manually because you have to have knowledge and suddenly it magically is done in an automated way. It doesn't lend itself to rules, it has to sort of be AI centered. So, you know, that I'm seeing over and over again are like ingesting documents with regulations and then tying those to financial performance. There's a lot of use cases that are tied to complex codes, complex regulations, that sort of thing. Let me read off the winners. Banco, Progetto, hope I'm saying that correctly, Cox Enterprises, Equifax, Expeon Health, Flutter UK and Ireland, Intel, of course, Lazard, Omega Healthcare, Omers and USI Insurance Services. So quite a mix of industries that probably all Intel customers. So, Jeff, where do you want to take this? Expand the number of countries, products and what's next for you guys? There's several areas that I think we could really utilize this technology. One of them is in the area of export controls. So because of export controls and sanctions, every single customer, you have to make sure they're not on a list, that they're not sanctioned. You have to sometimes even look at who owns the company. And so we do all that through human glue, looking at lists and so on. So being able to take this technology and have the machine do it, we know it'll be accurate and it'll go quickly. And so that's one area. Another one is export licenses. We submit export licenses routinely. And to have a program that would take all the data, put it together for the actual license and submit it, I think we might be able to take advantage of this to do something like that as well. So those are just two examples of things we can do. And so where are you guys, thank you for that, Jeff. Where are you guys in your automation journey? It's not like you started in November of 2022 when ChatGPT was announced.  But automation and AI are sort of converging. Explain sort of where you are in the maturity model. So in the trade group, we initially started with just doing unattended automation. So our core is SAP GTS. So this is Global Trade Services. And of course, people are doing resources of performing transactions. So the first go-to, the easy one was hey, replace a human by virtual human. So we started there. Then we started expanding into AI. So we did machine learning project to predict the HS classification. The next one is more on digitization. So we have a lot of supplier invoices and we need all that data. So using UiPAD document understanding, we are extracting that data and we've done a pilot where we've extract that data and accuracy of that is also 86%. And we tend to use that data for customs declaration. After that, our next journey is on a GenAI POC. So we are looking for newer use cases for GenAI. And where we tend to see a rare use of, I wouldn't say rare, but not so known use of UiPAD is in audit space. Because like Jeff mentioned, we are so heavy on compliance, all post transaction audit, we tend to use automation there. So it's in different zones that we've done and it's a journey. So we're still evaluating different use cases. And the classification prior to this was done predominantly manually or all manually? All manually. Wow. So, give you an answer.  I still had you guys a high score, but it's off the charts now. To classify one request, a trade analyst would take five minutes and we get around 1400 a week. And there were 11 people sitting manually and doing this and think the volume's increasing. Now that this machine is there, even if the volume keeps increasing, we don't need to put more resources to do that work. It's the machine doing it. So that's, I think, was a real transformation for us. But so, Jeff, I got to ask you, were you like the cobbler's kids? Because there were certainly ways to automate classification. Admittedly, I mean before AI, admittedly it wasn't that great, things like SVM and probabilistic latent cement, math basically, and you chose to stay manual for a while. Why? Because you needed that level of accuracy, I presume, right? Yeah, accuracy is important. I want to be clear, we do have SAP GTS and there is classification that's not manual then. We also have what's called an integrated shipping memo. It's all the non-revenue junk, right? And that's got to be, you got to have the same classification and we are able to, we were able to automate that about 80% because you can pre-define descriptions and then attach a classification. But then there was always 10 to 20% that kicked out and that's where you have people spending time. Because it's an odd item or something that's not clear, so you have to go look at the book, figure out what it is. So I think this is the value of taking that percentage from maybe 80 all the way up to the 98%, 99%. And the astounding thing is the timeframe in which you were able to achieve it. And we're just, early innings is sort of the bromide, but it's true. I'd love to hear from each of you where you see the potential of this type of technology, specifically as it relates to sort of automating some of these, forget the revenue generation for a moment. I mean, this is profit generating as it drops right to the bottom line. But it seems to me, Maureen, that we're going to first see sort of improved labor productivity. You heard Bernie Olson today in the keynote. I was blown away by his stats. He said 3% productivity improvement in the relatively near term, and he'd be disappointed if it wasn't higher, 4% or higher. If that's true, there's hope. What do you think is the future? The future. I mean, I think that we're going to just see this evolution of capabilities. And so people will, in some sense, some people are trying to think through what can we do quickly? And other people are saying, how can we do something super high value right away? And so as it progresses, different organizations will be harnessing in different ways. And then the foundation models and all the things that you can do with this stuff will change. I think one other thing that we're looking at is the multimodal aspect where you do, you combine image and language, English or the text and things like that so that you can start having a more holistic way of looking at how you apply AI to different types of use cases. Right now everyone's scouring for use cases and proofs of places for their proof of concept. And so some of them are stumbling on amazing results. That demo was pretty powerful today. Basically reading some really illegible, because it was Kanji characters, they're illegible to just a mono language-speaking American. You just ingested the itinerary and said, this is the hotel you're staying at and had all the data in seconds. That was quite remarkable. But Sid, what's your vision for, specifically where you want to take this at Intel? I think more and more generative AI because we've been there, done that with automating operations. And that's where if you look most of the unattended automation, 75% of automating work is in operations. We've done that. And I think every company's doing that. But how do you take it to the next level with Genai AI? And I think that's where we're trying to figure out. And that's all dependent on data. Like unless you have clean data, you have modernized your legacy, I think even Genai won't be able to help if you don't have clean data. So focusing on where we can take these use cases in different areas of trade, going forward, that's where my vision is. Yeah, it's definitely one of the top use cases you see for Genai AI is helping improve data quality. Jeff, we'll give you the final word. Yeah, take us home. You know, I'd like to use an example of trade like a coin. And there's two sides to it. One side is all the work that it takes to get prepared to actually ship. And that's what Sid has been working on, automating that. The other side of the coin is you've got to get all that data to the government. And if governments aren't ready to receive it, then you're still going to be subject to paper. So my vision as I look at this, I'd like to see a day when from the moment it's ready to ship to the moment it gets to the actual end user, it's digital. And there are no fingers, nobody's touching it, everybody gets access to what they need to see, when they need to see it, it's powerful. That's what I envision. That's a fantastic vision. Guys, again, congratulations and thanks so much for telling your story and spending some time with us. And Maureen, thank you for all the work that you put in. I mean, as you know, when you saw the results in terms of the submissions rather, very impressive. You guys put a lot of work into this. I think, you know, I'm not a big fan of participation medals for kiddie sports, but everybody should get a participation medal in this one because there was some really good, fine work being done. It was hard to judge. Yeah, it really was. And I'd love to take another shot at it. Well, thank you again, guys, good to see you. Thank you so much. Okay, thank you for watching. We're going to take a quick break here, but we'll be back at UI Path Forward 6 from the MGM in Las Vegas. You're watching theCUBE.