So Rachel and I started a fast AI with this kind of idea of make neural networks uncool again It is a grand plan to be sure because currently they are terribly cool But really there are some there are some things that we want to see improve and that's why we're doing this course We're actually not making any money out of this course We're donating our fees both to the diversity fellowships that we're running and also to the Fred Hollows Foundation I would like to briefly maybe give a quick pitch for the Fred Hollows Foundation for those of you who aren't aware of it Because as you know deep learning is fantastic for computer vision It's basically allowing computers to see for the first time What you might not realize is that there are I think it's something like three or four million people in the world Who can't see because they have something called cataract blindness? Cataract blindness can be cured for twenty five dollars per eye And actually the group of people who got that price down from thousands of dollars to twenty five dollars was Fred Hollows who was the Australian of years some years ago He's passed away now, but his legacy is in this foundation where if you donate 25 bucks You're giving somebody their site back. So as you learn to teach computers how to see You know, we're also Rachel and I are also donating our fees from this to helping humans to see so we think that's a nice little touch So we're doing this both to help the Fred Hollows Foundation But more importantly to help something we care a lot about which is making deep learning more accessible. It's currently terribly exclusive As I'm sure you've noticed Resources for teaching it tend to be quite mathematically intensive. They really seem to be focused on a certain kind of Ivory tower type audience. So we're trying to create training in examples, which are for non Machine learning and math experts dealing with small data sets Kind of giving rep role models of applications. You can develop quickly from today We're going to see how to create a real useful piece of deep learning code in seven lines of code We want to get to the point where It's easy for domain experts to work with deep learning. There are a lot of domain experts here whether you're working with getting satellites in the air or whether you're working with Analyzing the results of chemical studies or whether you're analyzing fraud at a bank and I know all those people here in this audience You are domain experts that we want to Enable to use deep learning at this stage the audience for this course is coders Because that's as far as we think we can get at this point We don't need you to be a math expert But we do need you to be coders and I know all of you have been told of that prerequisite We do help that with your help we can get to the point where non coders will also be able to participate in this The reason why we care about this is that there are problems like Improving agricultural yields in the developing world or making medical diagnostics available to those who don't have them or so forth These are things that can be solved with deep learning But they're not going to be solved by people who are at these kind of more ivory tower firms on the whole because they're not really that familiar with These problems the people who are familiar with these problems are the people who work with them every day So for example, I've had quite a bit to do with a lot of these kinds of people such as through the World Economic Forum I know people who are trying to help cure TB and malaria I know people who are trying to help with Agricultural issues in the developing world and so forth and these are all people who want to be using Deep learning for things like analyzing crop imagery from satellites or my most recent startup Which was analyzing? Radiological studies using deep learning to deal with things like the fact that in the entire continent of Africa There are only seven pediatric radiologists. So most kids in Africa That most countries in Africa No kids have access to any radiologists and therefore have no access to any kind of modern image-based medical diagnostics So these are some of the reasons that we're creating and running this course And so we hope that the kind of feel of this community It's going to be very different with kind of the feel of a lot of the deep learning type communities that have appeared before that Have been all about let's trim 0.1% of this academic benchmark, you know This is going to be all about let's do shit that matters to people as quickly as possible Sometimes to do that we're going to have to push the state of the art of the research and where that happens We won't be afraid to you know show you the state of the art of the research The idea is that by the end of part one of this You will be able to use all of the current best practices in the most important deep learning applications If you stick around for part two you'll be at the cutting edge of research in most of the most important research areas So we're not dumbing this down by any means. We're just refocusing it So the reason why we're excited about this is that we have now The three pieces of kind of this universal learning machine We now have This kind of the three critical pieces like an infinitely flexible function All-purpose parameter fitting which is fast and scalable. So I'll tell you what I mean about these three pieces The neural network is the function We're going to learn about exactly how neural networks work, but the important thing about a neural network is that they are Universal Approximation machines. This is there's a mathematical proof the universal approximation theorem that we're going to learn all about Which tells us that this kind of mathematical function is capable of handling Any kind of problem we can throw it whether that mathematical function be how do I translate English into Hungarian or whether that Mathematical function is how do I recognize pictures of cats or whether that mathematical function is how do I identify unhealthy crops? It can handle any of these things. So with that mathematical function then the second thing you need is some way to fit the parameters of that function to your particular need and There's a very simple way to do that called radiant descent and in particular something called backward propagation Back prop which we will learn all about in this lesson and the next lesson The important thing is though that these two pieces together Allow us to start with a function that is in theory capable of doing anything and turn it into a function that is in Practice capable of doing whatever you want to do as long as you have data that shows examples of what you want to do The third piece which has been missing until very recently is being able to do this in a way that actually works with the amount of data that you have in the time that you have available and This is all changed. Thanks, particularly to GPUs So GPUs Graphics processing units also called video cards. That's kind of an older term now also called graphics cards and these are Devices inside your computer which were originally designed for playing computer games So it's kind of like when you're looking at this alien from the left-hand side And there's light coming from above. What would each pixel what pixel color do I need for each place? That's basically a whole bunch of linear algebra operations a whole bunch of matrix products It turns out that those are the same operations that we need for deep learning and So because of the massive amount of money in the gaming industry that were thrown at this problem We now have incredibly cheap incredibly powerful cards for figuring out what aliens look like and We can now use these therefore to figure out how to improve medical diagnostics in Africa. So it's a nice handy little side effect GPUs are in all of your computers But not all of your computers are suitable for deep learning And the reason is that programming a GPU to do deep learning Really requires a particular kind of GPU and in practice at the moment It really requires a deep learning GPU from Nvidia Because Nvidia GPUs support a kind of programming called CUDA Which we will be learning about There are other GPUs that do support Deep learning, but they're a bit of a pain They're not very widely used and so one of the things we're going to be doing is making sure that all of you guys have access to an Nvidia GPU The good news is that in the last Month, I think Amazon has made available good quality Nvidia GPUs for everybody for the first time They call them very excitingly. They're P2 Instances, so I've spent quite a bit of the last month making sure that it's really easy to use These new P2 instances have given you all access to a script to do that Unfortunately, we're still at the point where they don't trust people to use these correctly So you have to ask permission to use these P2 instances. So the data Yes Okay Oh, yes Amazon, so the data Institute folks For anybody who does not yet have an aws P2 instance or their own GPU server They are going to collect all of your aws IDs and they have a Contact at Amazon who will go through and get them all approved They I mean they haven't made any promises. They've just said they will do what they can so They're aware of how urgent that is so if you email your aws ID To Mindy She will get that organized and we'll come back and look at aws in more detail very shortly The other thing that I've done is on the wiki I Have added some information about Getting set up Here we are installation There is actually quite an interesting option called over H I'm sure by the time this is a MOOC. There's going to be a lot more But this is the only company I've come across who will give you a buy the month Server with decent deep learning graphics cards on it And it's only 200 bucks and to give you a sense of like how crazily cheap that is if you go to their page for GPU servers You'll see that this GTX 970 is 195 bucks a month and then their next cheapest is 2,000 bucks a month it just so happens this GTX 970 is Like ridiculously cheap for how good it is at deep learning and the reason is that deep learning uses Single precision arithmetic in other words it uses like less accurate arithmetic these higher-end cards are designed for kind of like fluid simulations and tracking Nuclear bonds and stuff like that that require double precision arithmetic. So it turns out these GTX 970s are only good for two things games and Deep learning so the fact that you can get one of these things which has got two GTX 970s in this is a really good deal So one of the things you might consider doing in your team is maybe sharing the cost of one of these you know 200 bucks a month is Pretty good, you know compared to worrying about starting and stopping your 90 cents per hour AWS instance Particularly if AWS takes a while to say yes So AWS how many people here have used AWS before? Maybe a third or a half so AWS is Amazon web services. I'm sure most of you if not all of you have heard of it It's basically Amazon making their entire back-end infrastructure available to everybody else to use So rather than calling it a server you get something they call an instance You can think of it as basically being the same thing It's a little computer that you get to use in fact not necessarily little some of their instances cost 14 or $15 an hour and give you like eight or 16 graphics cards and dozens of CPUs and hundreds of gigabytes of RAM The cool thing about AWS is that you can do a lot of work on their free instance You can get a free instance called a t2.micro And so I should drop that down t2 Dot micro and you can get all everything set up and working on a really small data set and Then you can switch it across if you want to then run it on a big data set Switch it across to one of these expensive things and have it run and finished within an hour or two So that's one of the things that I really like about AWS Microsoft also have something a lot like AWS called Azure Unfortunately, their GPU instances are not yet publicly available I've reached out to Microsoft to see if we can get access to those as well, and I'll let you know if we hear back from them So One of the things actually that Rachel's done today is to start jotting down some of the common problems that people have found with their AWS installs Getting AWS set up is a bit of a pain So we've created a script that basically will do everything for you But the nice thing is that that script is very easy for you to like have a look at and see what's going on so Over time you can kind of get a sense of how AWS works Behind the scenes AWS is using their command line interface or CLI, which we've given you instructions on how to install As well as using the CLI You can also go to console.aws.amazon.com and use this graphical interface in General I try to avoid using this graphical interface because everything takes so much longer And it's so hard to get things to kind of work repeatedly But it can be nice to kind of look around and see how everything's put together And again, we're going to come back and see a lot more about how to use the graphical interface here As well as how to create and use scripts Okay So these are some of the pieces that we want to show you I wanted to talk a bit more before we go into more detail about some of the interesting things that we've seen happening in deep learning recently And perhaps the thing that I found most fascinating recently was when one of the leading folks at Google Brain presented this at a conference at Stanford which showed the use of deep learning at Google and You can see from this is just 2012 To today or maybe two months ago It's gone from nothing to over 2500 Projects now the reason I find this interesting is because this is what's going to happen to every organization And kind of every industry over the next few months and few years So they've kind of described how at Google it's getting used pretty much everywhere And you can imagine probably if they redid this now today two months later, it's probably going to be somewhere up here So we kind of felt like it would be great to Help kickstart lots of other organizations to start going up this ramp. That's another kind of reason. We're doing this I Really like looking at applications and we started seeing some examples of the kind of these kind of like Deep learning amateurs applications if you like This is an example of it, right? What these guys did is they're not Machine learning or deep learning experts. They downloaded a copy of cafe They ran a pre-existing model. This is what we're going to learn to do today So run a pre-existing model and use the features from that model to do something interesting And in that case the thing that they decided to do that was interesting was to take data that they already had Because they're skin lesion people and analyze skin lesions. These are the different kinds of skin lesions that you can have and They found for example that the previous best for Finding this particular kind of skin lesion was 15 point six percent accuracy When they did this off-the-shelf cafe pre-existing model with a simple linear thing on top They could ruple it to 60 percent, right? So often when you take a deep learning model and use the very simple techniques will learn today You can get extraordinarily upticks compared to non deep learning approaches Another example of that was looking at plant diseases there's been at least two groups that have done this in the last few months and again Very successful results from people who are not deep learning or machine learning experts Similar results in radio modulation. So these folks Who are a lack of computer engineering people? Found that they could double the effective coverage area of like phone networks and stuff like that So like this is massive result and again, they used them very simple approaches It's being used in fashion It's being used to diagnose heart disease by hedge fund analysts So there's a particular Post which I found really Inspiring actually and trying to put this together, which is that Keras, which is the main library we'll be using the author of that put together this Post showing how to build powerful models using very little data I really just wanted to give a shout-out to this and say, you know This this work that friends wire has been doing has been very important in a lot of the stuff that we're going to be learning over the next few classes The basic environment we're going to be working in most of the time Is the ipython notebook or the Jupyter notebook? So let me just kind of give you a sense of what's going on here when you have a Jupyter notebook open You will see something which let's open out today's one Okay, so this is a good time to show you about starting and stopping AWS instances So I just tried to go to my notebook on AWS and it says it can't be reached So my guess is that if we go Back to my console You can see I have zero running instances. So I've got zero servers currently running So if I click that I will see my all my servers Normally I would have one P2 server or instance and one T2 because I use the free one for kind of all the Getting everything set up and then use the paid one once everything's working Because I've been fiddling around with things for this class. I just have the P2 at the moment. So having gone here one way I could start this is by Going start here But like I said, I don't much like using this Gouy for stuff because it's just so much easier to do things Through the command line Let me make this a little bit bigger But that yep, okay So one of the things that I showed you guys that you could download today is a bunch of aliases So making starting and stopping AWS really quick If you haven't got them yet, you can find links to them on slack Or you can just go to platform.ai slash files And there's a bunch of different things here this AWS alias dot sh Is a file that sets up these various aliases? so You can the easiest way to grab stuff On your AWS instance or server is to use something called W get so I Would right click on this and choose copy link address and Then go W get and paste in that and that will go ahead and Download that file now. I already had one. So it's credit a copy of it We can take a look at that file and you'll see it's basically a bunch of lines I say alias something equals something else And it's created AWS get p2 AWS get t2 AWS start AWS SSH AWS stop I'm going to show you what these things do because I find them pretty convenient Basically if I run AWS get p2. So first of all, I'll say source AWS alias dot sh And that just runs that file. Okay, that's in bash That's how you just run a file and that's now caused all of those names to appear as aliases to my system So if I now run Alias get p2 That's going to go ahead and ask Amazon for the ID of my p2 instance Okay, and not only does it print it, but it's going to save it into a variable called instance ID And all of my other scripts will use dollar instance ID So I now want to start that instance. So I just type AWS start and that's going ahead go ahead and do this Equivalent thing of going to the GUI right-clicking choosing start. The other nice thing it does is it? waits Until the instance is running And then at the end it asks the queries for the IP address and prints it out Now this grip that I have given you guys to set up these instances Uses something called an elastic IP that actually keeps the same IP address Every time you run it. Okay, so you should find that IP address stays the same which makes it easier So there is the IP so I then have something called AWS SSH And AWS SSH will go ahead and SSH into that instance So all it does is basically use the username Ubuntu because that's the default username for this kind of image on AWS at and then Dollar instance IP. So that's that IP address. We just got the other thing it does is to use The private key That was created when this was originally set up Now in my case, I've actually moved that private key to be my default key So I don't actually need that minus IP. So I just type SSH Ubuntu at dollar instance IP But you can just type AWS dash SSH and you'll see bang here. We are Okay, so we are now inside that AWS image and One of the handy things about AWS is they have these things called AMIs Amazon machine images And an AMI is basically a snapshot of a computer at a particular point in time And you can start your own instance using a copy of that snapshot So in the script I've given you guys I've created and provided an AMI Which has all the stuff we want installed So that's why it is that when you use that script and log into it You can start running things straight away. So let's do that right now So I can I've created a directory already for you called nbs for notebooks So we can go ahead and type jupyta notebook And this is how we ask Amazon to set up Jupyter notebook server for us and when it's done it says okay, the Jupyter notebook is running at all IP addresses on your system 888 so what is our IP address? Well, it told us up here when we started it. It's 52 blah blah blah blah, so I'm going to go to my instance and I'll go 52 40 116 119 in my case Colon and it told me that the port is 8888 so colon 8888. So I'm just typing in that here and press enter Again, I put too many colons I've set up a password and it's just dl underscore course We can look later on at how to change that password if people want to but I just sort of be handy to have a Password there for everybody if you want to start looking at some of your own data And actually by default it's not going to show you anything So now we can just go ahead and say new Notebook and just say Python Converoo and this sets up a Scientific computing environment for you where you can type Python commands and get back responses. So one plus one There we go. Okay, so it seems to be computing things correctly. So The basic idea here is that over there on Amazon you have your server It is running a program called Jupiter notebook Jupiter notebook is causing a particular Port called which is 8888 to be opened on that Server where if you access it it then gives you access to this Jupiter notebook environment in your team You guys can all use the same Jupiter notebook if you want to or you could run multiple Jupiter notebooks on one machine It's really pretty flexible, right? So now that I've created one like I could rename this say and say this is Jeremy's notebook and so then Rachel might come along and Be like oh, I want to run something as well. So she goes new and her computer And it creates a whole new one over here and she could say file rename Rachel's notebook Okay, so if I now go back here, you can see both of these notebooks are shown to be Running okay, so this the server is running multiple Kernels, they're called and you can see back here. It's saying creating new notebook kernels started So each of those are totally separate. So from one of them I say Name equals Rachel And in the other one I say name equals Jeremy And then over here I say no, you'll see that they are not in any way Talking to each other. They're totally separate. Okay? so That's a super handy way to kind of do work and the other nice thing is that you can Not just type code, but you can also type markdown. So I could go Section I want to talk about Yeah, and so as I do that it allows me to mix and match Kind of information and code and every piece of code that comes out I can see where it came from and it also as you'll see let's just put in Visualizations and plots and so forth so Some of you may have come across this important concept called literate programming and literate programming is the idea that as you code You are documenting what you're doing in a very Deep way and not just for others, but maybe more importantly for yourself And so when you're doing data science work Worked like a scientist. How many people here are in some form scientists or have been scientists? So you guys will know The importance of your journal notebook, you know that that the greatest scientists They're all stories about the kinds of notebooks they kept and how their lab notebooks worked or their lab journals worked This is critical for data scientists to this idea that as you do experiments You're keeping track of like what did I do? What world what didn't work? I can see all the people who put their hand up as scientists are all nodding right now So this makes it super easy to do that right so be Helpful to yourself and to your team by taking advantage of this now in order to learn to use this environment All you have to do is press H And when you press H it brings up all of these keyboard shortcuts After not very long you will get to know all of them because they're all Extremely useful, but the main ones I find particularly helpful is you hit M To turn into markdown mode So that's the mode where you can enter text rather than code or why to switch it back to code again And you certainly need to know shift enter which evaluates the cell and gives you a new cell to enter into And you also need escape Which pops you out of entering information and gets you back into this command mode and Then enter to go back into edit mode again. So just to show you that enter to get into edit mode Escape to get out of edit mode Shift enter to evaluate Okay, and you can see as I move around it changes which one is I love it I've started to create some resources on the wiki for helping you with Jupyter notebook It's still Ready early, but you guys I'm sure can help by adding more information Here one of the things I particularly mentioned is that there are some good tutorials You know, I thought I had also mentioned My favorite book which I now can't see here Python libraries Yeah There we are Python for data analysis By Wes McKinney. It's a little old. It also covers pandas a lot which you don't need but it's a it's a good book for Getting familiar with this basic kind of Python scientific programming stack Okay so the last kind of Ingredient that I want to introduce is Kaggle how many people here have been to or done anything with Kaggle at any point Okay, I've got a third of you Anybody who was in the master's program here. I'm sure we'll have used Kaggle or be shortly to use Kaggle Mainly because it's just a great place to get Interesting all kinds of interesting datasets so for example if you wanted to test your ability to Do automated drug discovery you could go to Kaggle and download the files for the Merck molecular activity challenge Run some models and test them to see how they compare to the state of the art by comparing to the leaderboard So Kaggle is a place where various organizations run machine learning competitions They generally run for about three months It's super cool because they get archived essentially forever You can download the data for them later on and find out how you would have gone in that competition Generally speaking if you're in the top 50% that means you have an Okay ish model that is like somewhat worthwhile if you're in the top 20% it means you have a very good model If you're in the top 10% it means You're at an expert level for this type of problem If you're in the top 10 It literally means you're one of the best in the world because every time I've seen a Kaggle competition I used to be present with Kaggle, so I'm very familiar with this every time I've seen a Kaggle competition At least the top 10 generally all beat the previous best in the world and generally are from You know really good machine learning experts who are going beyond anything that's been done before it seems that the power of competition pushes people way beyond kind of what the previous academic state of the art was So Kaggle is a great environment to find interesting data sets and to benchmark your own approaches So we're going to be using it for both of these purposes Our first challenge Will be dogs versus cats So sometimes on Kaggle they run competitions that are not Done for lots of money, but sometimes they're done For free or for a bit of fun in this case It was actually done for a particular purpose Which was can you create can you create an algorithm that can recognize it has trained dog photos and cat photos? And the reason why was because this particular organization was using that problem as a capture in other words to tell the difference between humans and computers it turned out That the state of the art machine classifiers could score 80% accuracy on this task So really this group wanted to know You know, can you surpass the state of the art is this a useful capture? and then If you can surpass the state of the art can they can they then use this kind of dogs versus cats recognizer for their pet finding work, which is what they were doing so Really the goal here was to beat 80% now This is a great example of The kind of thing which you could use for a thousand million different purposes for example The work I did in cancer detection is this So if you take a CT or an x-ray Or an MRI and you say to a deep learning algorithm these people have malignant cancer these people don't It's the same as cats versus dogs if it's this is a Healthy high crop yield area from satellite photos. This isn't that's cat versus dogs If you say this is one kind of skin lesion and this is another kind of skin lesion If you say this is an abstract Painting and this is not this is a extremely valuable painting and this is not this is a Well-taken photo and this is not they're all image analysis problems that are generally classification problems and these are all examples of things people have done with this kind of technology so cats versus dogs it turns out is a very Powerful format and so if we can learn to solve this well We can solve all of these kinds of classification problems not just binary so not just is it this group or that group but Also things like that skin lesion example These are ten different types of skin lesions which type is it or the crop disease example Which of these 13 crop diseases are we looking at here? An example of an actual thing that I saw I can probably find it was Cucumber analysis Cucumber deep learning There can't be too many things with those particular keywords. There we go. So a Japanese cucumber farmer Used this approach to deep learning to do the automatic Kind of automated all their logistics and basically had a system that would put the different grades of cucumbers into different bins Automatically and made their cucumber workflow much more efficient So if that was your idea for a startup, it's already been done. Sorry, but there's many more Okay So they are All of our basic pieces So to get started here we are with this AWS server with a pretty empty looking Set of notebooks here so we want to go ahead and start Getting some Getting some work done. So to do that. We need to download the basic files that we need So I've sent you all of this information already All of the information you need is on our platform.ai website All of the notebooks are in files slash nvs.  I'm going to press control C twice that shuts down the notebook Okay, so the notebooks not running. Don't worry. It saves itself Automatically on a regular basis or you can just hit s to save it right now Okay, so shutting down the notebook as you'll see the Python notebook files are still sitting there and you can see actually that behind the scenes. They're just big bunches of JSON They're just big bunches of JSON text so you can stick them in GitHub and they'll all work perfectly well What I generally like to do is run something called T-Mux. How many people here have used T-Mux or screen before? Less than I expected. Okay, so those of you haven't you're going to love this trick T-Mux and screen are programs that let you run programs on your server Close your Terminal come back later and your program will still be running in the exact same way I don't remember if T-Mux is already installed it is okay So to use it you just go T-Mux Right, and it looks like nothing happened except a little green bar has appeared at the bottom But if I now hit T-Mux's magic command, which is control B and Press question mark you can see there are lots of keystrokes that it has ready for me to use And so one of the ones I like is control B double-quote which creates a second window underneath this one or Control B Percent which controls it shows a second window next to this one And so like I tend to like set up a little T-Mux session And get it all set up the way I want So I'm not going to go into detail about How to do everything I show you what I really want to do in the class is to make most of the time Is to say here's something that exists Here's something I recommend using here's what it's called and like during the week you can play with it You can ask questions you can use in your team and so forth right so here it is It's called T-Mux this is what it does and I'll show you something cool if I now go control B and then D to detach Close out of this all together Right, it's all gone So if I now go back into my server Okay, so I wasn't able to ssh in actually properly because currently dollar instance IP is not defined and the reason for that is that I have to This is good. I got to show you lots of steps rather than every time I start sourcing my AW AW s dash alias dot sh file What I should do is I should go vim dot bash RC dot bash RC is a file that is run every time you run bash Right, and if I edit my bash RC file and at the end I type source aws alias dot sh Okay, and just to show you I'm going to close it and reopen it If you're wondering why my computer is going so slowly It's because when you have these big Skype calls running at the same time as screen recording everything slows down So you can see now all those aliases are there so before I ssh to dollar instance ID I have to find out my correct IP address so I can say aws get P2 get my instance ID That's taking other it comes And then we can You know what I'm not sure I've got something here to actually just get The IP address so that's interesting As you can see I'm kind of playing with this a little bit as I go so I'm going to go ahead and show you How to do this so right now the IP address only gets printed out When I start an instance in this case. I've already got an instance running So I'm going to edit this script and I'll change it later on but basically I'm going to create a new alias called aws IP and I'm going to get rid of the bit that starts the instances Get rid of the bit that waits for it to be running And I'm just going to keep the bit that says instance IP equals something something something source aws alias ssh and I've got a new alias called aws IP and Now I can go ssh ubuntu at dollar instance IP Having said all that Because my IP address is going to be the same every time and I couldn't really be bothered waiting for all that I'm actually going to manually put my IP address in here So the next time I run this I can just press up arrow and just rerun that command So I'm kind of showing you lots of ways of doing things So you can kind of decide what your own workflow is like or come up with better ones But here's the cool thing. I am back in my in my box here And then if I say team up's attach I am exactly back to where I came from so whatever I had running and whatever it Said it was it's still sitting there. So the particularly cool thing is that any notebooks kernels. I had running. They're all still sitting there This is particularly helpful if you run something like those ovh servers or one of your own servers like with aws It's a little less helpful because you really need to shut it down to avoid paying the money But if you've got something you can keep running for all the msam students You all have access to the or will have access to the GPU server. We have here at At the University is particularly helpful for you guys, right? So I actually tend to use this little bottom right hand window to permanently have Jupiter notebook ready So that's that's kind of like my particular way of running and then I tend to use this left-hand window to kind of do other things right and in particular I'm going to go ahead and grab my notebooks So I need to grab I'm just about to yeah, thanks So the easiest way to grab things is with W get so if I go W get I now have a notebook lesson one and so if I go back to my Dupler notebook See it's appeared lesson one and if I click on it if you're using a T1, sorry a t2 instance the free ones generally speaking particularly the first time you run something it can take quite a long time To open you should find the second time. It's quite fast by the way So here is Our notebook and so hopefully quite a few of you have already got to the point today that you can see this Those of you that haven't will get plenty of help during the week This particular notebook uses two external scripts to help those scripts are called utils and VGG 16 so the last thing I'm going to do before our break is to grab those and So I'm going to W get and Just pop these all in this notebooks directory. So they're all in the same place VGG 16 Then unzip them Okay What's the other one utils And then the only other thing you need is the data so the data sits in the platform AI data directory and The data is all the dogs and cats now I've taken the Kaggle data and made changes to it which I'm going to be showing you so rather than downloading it from Kaggle I suggest you grab it from platform AI and I've sent you this information today as well. I think So I'm going to CD into data and W get that as well Great, and so that's going to run for a few minutes. And so while it does I think it's a good time to have a break let's have a 10-minute break During which you are welcome to keep going with your install Chat to myself or Rachel or Tara or yeah, if you've got any questions or get to know your team or just go and have a Have a coffee. So let's get back together at Five past eight Thanks The previous section I think for some of you and I was chatting to a couple of you during the break Was a bit of a fire hose of information Because it was like, you know, here's bash. Here's AWS is Kaggle.  You'll have the video tomorrow And by the time you're here again next week I want to make sure that everybody who has the time and interest to work hard on it Has got through all of the material if you haven't You know, like you're a couple of days, you know, maybe it's like early on the weekend And you're thinking I'm gonna get there Please let Rachel and I know like we will work with you in person to get you there Everybody who puts the time in I am determined to make sure can get through the material If you don't really have the background and you don't really have the time That's fine. You know, maybe you won't get through all the material, right? But I really am Determined that everybody who's prepared and able to put in the time can get through everything so between the community resources and the video and Rachel and I and me folks that taro and yad We will help everybody to those of you who are kind of practicing data scientists and you are familiar with all of these pieces You know, I I apologize that there will be a bit slow for you and hopefully, you know As we move along, there'll be more and more new stuff and I'm kind of hoping that for those of you who already have some level of expertise We will continually give you Ways that you can go further So for example at the moment, I'm thinking like can you help us with these scripts to make them better to make them Simpler to make them more powerful To to create Azure versions of them You know all the stuff that we're doing to try and make deep learning as accessible as possible Can you help contribute to that? Can you help contribute to the wiki? So for those of you who already have quite a high level of expertise I'm really looking to make sure that there's always ways that you can push yourself. So if you're ever feeling like a Bit bored, you know, let me know that it's perfectly reasonable thing to say Hey, I'm kind of a bit bored and I'll try and give you Something to do that you don't know how to do and then you have your board anymore Okay so at this point I Downloaded dogs cats zip and I And I unzipped it if you're wondering about the minus Q that's just because unzipped otherwise prints out every single file name as it goes So that's Q for quiet So Just about the most important thing for doing this kind of image classification is how the data directories are structured And in particular your notice that we have a Trading set and a test set and that's because when we downloaded the data originally from Kaggle It had a train dot zip and a test dot zip Keras, which is the library we're going to use expects that each Class of object that you're going to recognize is in a different directory So the one main thing I did after I downloaded it from Kaggle is that I created I created two directories one called cats and one called dogs and I put all the cats and the cats and all the dogs and the dogs Okay, because when I downloaded them from Kaggle They were all in one directory and they were called like cat dot one dot jpeg or dog dot one dot jpeg So that was pretty straightforward. So now if I LS train dog one Oh, sorry dogs Okay, for example Dog dog Okay, so there are Let's have a look Well, it's 11,500 one. There's maybe 11,500 Dogs in there. That's the on something weird and then cats Clearly something weird Okay so that's the number of dogs and cats that we have in our training set and so for those of you who haven't done much data science before There's this really key concept that you have a training set and a test set and Kaggle being a competition makes this really obvious The files in the training set tell you what they are here is a dog. It's called dog something But if I look in the test set They don't say anything. They're just numbers Okay, why is that? that's because Your job in this Kaggle competition is to say for example for 43.jpeg. Is it a dog or is it a cat? so there are 12,500 images in the test directory for you to score for you to classify Even if you're not doing a Kaggle competition, you should always do this yourself In fact, ideally you would get one of your colleagues to do it without you being involved to split the data into a test set and a training set and to not let you look at the test set until you promised you're finished And Kaggle kind of enforces this they let you submit To the leaderboard and find out how you're going But actually the final score is given based on a totally separate set of data that is not scored. So this is like I don't know like for me before I started entering Kaggle competitions. I kind of thought that my Data science process was reasonably rigorous But once I actually started doing competitions I realized that that level of enforcing the test training split made me a much better data scientist You know you can't cheat so I do suggest you do this in your own projects as well Now because we Also want to kind of tune our algorithm in terms of like different architectures and different parameters and so forth Which we'll talk about It's also a very good idea to split your training set further into a training set and a validation set We'll see a lot more about how this works, but you'll see in this case. I've created another directory called valid Which has Dogs and cats as well. So it's structured exactly the same Okay, and here you can see that there are a thousand cats and A thousand dogs so when I originally downloaded from Kaggle there were twelve thousand five hundred cats and dogs in the training set That's why in my training set. There are eleven thousand five hundred because I've moved a thousand of each of them to the validation set So that's the basic data structure we have now Other than splitting things into test training and validation sets that's the most important advice I have just as a data scientist The second most important advice I have as a data scientist is to always do nearly all of your work on a sample A sample is a very small amount of data that you can run so quickly that everything you try you get a nearly immediate answer to This allows you to very quickly Try things change things and get a basic process running So I always create a sample with you know a hundred or so Items in to to just get started with so you'll see I have a directory Called sample and in that I have a whole separate train and valid I Did not move things there. I copied them there right and the purpose of this sample directory is just to let me do things Really quickly so you'll see inside sample train Is again we have cats and dogs, but this time There are eight Okay Okay, I probably should have put more in there I think probably more like a hundred would have been good But I think at the time I was probably using a really low-powered computer my testing But it's enough to check that my script's working so Now that everything is downloaded you can see that I have In my Jupyter notebook is automatically noticed that it's changed I'll get rid of these zip files. I've unzipped them now and I'll get rid of the Rachel's and Jeremy's notebooks that I was just playing with and We're ready to get started with doing some deep learning so the goal for you guys during this week will be to Replicate Everything that I've done Initially just by making sure that this notebook works for you But then to replicate it with another data set And so one of the things we'll do tomorrow was we'll post some ideas of other Interesting Kaggle data sets you could try and maybe other people can also post other interesting data sets they found elsewhere So the idea will be to make sure that during the week You can run your own Classification process on some data set other than dogs and cats But first of all make sure that you can run this okay So as you can see in this notebook, I've used markdown cells if I hit enter you can see There they are how many people have used markdown before Okay, so most of you so that don't know markdown is what we use both in the notebook as well as on the wiki It's basically a way of really quickly creating formatted text There's not enough of you that aren't familiar with it that I'm going to go into it in detail If you're not familiar with it, please Google markdown. Okay, and you can experiment with it either on the wiki or in your notebook As you can see though, I basically created cells with headings and some text and during the week You can read through these in detail as we mentioned. We're going to try to enter the dogs and cats competition So 25,000 labeled Dog cat photos half of each 12,500 in the test set and the goal is to beat 80% Okay As we go along we are going to be learning about quite a few libraries not too many But enough that for those of you that haven't used Python for data science before It's going to seem like quite a few by the end of the seven weeks. Hopefully you'll be pretty familiar with all of them One of the really important three is matplotlib That's because matplotlib does all of our plotting and visualization and on the wiki We have a section called Python libraries and as you can see we have our top three listed up here At the moment, there are just links to where they come from I'm hoping that you guys will help us to turn this into a really rich source of information about You know places that you found lots of helpful staff answers to frequently asked questions and so forth But for now if you're not familiar with one of these things type the word followed by a tutorial into Google And you will find lots of resources because one of these things are widely used Care us a little bit less so because it's just a deep learning library and therefore relatively new NumPy and matplotlib and all these other ones scikit-learn scipy There's lots of books about them. There's lots of tutorials about them. There's lots of videos about them So matplotlib Creates plots and one of the things we need to do is to tell Jupiter notebook what to do with those plots should it pop open a new window for them Should it save them and so this percent matplotlib inline says please show our plots in the actual Jupiter notebook Okay, so that's pretty much the first line in every Jupiter notebook that I create and Here's the thing I told you about which is sometimes I want to run stuff on a sample and sometimes I want to run it on Everything and so I make it really easy for myself by having a single thing called path Which I can switch between the sample and the everything so for now. Let's just do things on the sample Okay, and so I as I said you should do all of your work on the sample until everything's working So as you can see each time I've done something I press shift enter and it says it puts a little number after the in Showing that this is the second input cell that I've run like every programming language a large amount of the power of Python comes from the libraries that you use and To use a library in Python you have to do two things You have to install it and Then you have to import it in Python I strongly recommend that you use a particular Python distribution called anaconda and if you're using the scripts and AMI As we provided you are already using anaconda You can check what Python you're using by typing which Python and it will tell you right You'll see that not only am I using anaconda But I'm using an anaconda that is installed into my home directory So no screwing around the pseudo or any of that business again if you use our AMI in scripts. This is all being done for you With anaconda Installing anything is as simple as typing conda install and the name of the thing and on anaconda Everything's been pre-compiled. So you don't have to wait for it to compile You don't have to worry about dependencies. You don't worry about anything. It just works Okay, that is why we very highly recommend using anaconda It works on Mac it works on Windows and it one works on Linux Lots of Windows users use it Very few Linux users use it very few Mac users use it I think that's a mistake because lots of Mac and Linux users also have trouble with like Compiling dependencies and all that stuff. I suggest that everybody uses them From time to time you will come across something that does not have a conda installer available In which case you have to use pit instead in our case, I think just Theano and Keras have In that situation, but no neither of those need compiling anything at all. So they they're very very easy to install Okay, so once that you've installed it by typing conda install whatever and most things are already installed for you with our AMIs You then have to tell Python that I want to use it in this particular session Which you do just by typing import and the thing you want to look at So I'm not going to go through all these libraries right now. I'll go through them as we use them But one of the big three is here, which is numpy Numpy is the thing which as our wiki page describes provides all of our basic linear algebra How many people here have some familiarity at all with linear algebra? Nearly all of you good. Okay, so for If you're somebody who didn't put up your hand I would suggest looking at the resources that Tyro added to the did you write them to the wiki or just the four at the bottom here. Oh Yeah, that's right. So go back to the home page and Go to linear algebra for deep learning generally speaking for for any math stuff My suggestion is to go to the Khan Academy site and Khan Academy has really great Videos for introducing these kind of simple topics We just need to know these three things mainly really the first two things for this course and Numpy is the thing which gives you These linear algebra operations in Python and as you'll see it makes them extremely easy to use Pretty much everybody renames numpy to NP. So that's what import numpy as NP does You'll find a nearly everybody's script on the internet. It'll be NP dot something not numpy something in general we try to stick with the same kind of Approaches that everybody else uses so that nothing will be too unfamiliar Okay, so we've imported some libraries that we need We also try to provide some additional utilities and scripts for things that we think probably ought to exist But don't exist that make things easier There's very few of them nearly all of it is in one script called utils There's a cool little trick which is that if you Using external script that you've created and you're changing it quite a bit So like now that you've got utils feel free to add and change and do whatever you like to it If you import it like this by saying import utils Reload utils and then from utils import. What if you need you can go back and rerun That cell later after you've changed utils.py and all of your changes will be there available you to use So for now, we're just going to use one thing from our utils of the library called plots, which you'll see used in a moment. Okay So our first step will be to use a pre-trained model So what do we mean by a pre-trained model? What we mean is that somebody has already come along downloaded millions of images off the internet and built a deep learning model That has learnt to recognize the contents of those images Nearly always When people create these pre-trained models, they use a particular data set called image net The reason that they tend to use image net or one of the key reasons that they tend to use image net is because image net has the Most respected annual computer vision competition So people that win the image net challenge tend to You know what fat nowadays people that win the image net challenge tend to be companies like Google and And Microsoft a couple of years ago it had to be people who immediately then got hired by Google and Microsoft Image net itself is fun to explore if you go to image net and go to explore You can check it out and basically there are 32,000 categories So for example, we can go okay image net. Let's look at plants and plant life Let's go to crops Let's go to field crops Corn type crops dent corn Okay, so here we have a number of pictures of dent corn there are 397 of them so The folks that create these pre-trained networks basically download A large subset of image net the competition has a thousand of these 32,000 categories that people compete on So nearly always people just build models for these are a thousand I would be remiss if I did not mention the shortcomings of the image net data set Can anybody tell me kind of something that they notice in common about Kind of what these photos look like or how they're structured Yeah, they're they're like They're just one thing, you know Like if you look at an arbitrary photo say from my photo album You'll see like there's a person here and a bridge there and something else here Image net is carefully curated to like be pictures of Flint corn, you know, these are like 312 pictures that are designed to be really good pictures of Flint corn whatever that is so This is a Easier problem than many problems that you will be facing, you know, so for example, I was talking to Robin from Planet Labs that break about the work that they're doing with satellite imagery Their satellite imagery is going to have a lot more than a picture of piece of corn, you know Like a planet lab satellite has where's Robin So how many pixels is a planet labs photo? Couple of million. Okay, so pretty big and in that couple of million pixels You're going to have like what kind of area with that cover 50 square kilometers 500 square kilometers, okay So there's going to be tennis courts and swimming pools and people sunbathing and all kinds of stuff, right? So when Robin takes this stuff to Planet Labs You know, he's not just going to be able to use a pre-trained network directly Right, but we're going to show you how you can use some of the structure of the pre-trained network Even if you are not looking at photos that are this clear Having said that if you remember the slide I showed you earlier of the plant disease Project each of those plant disease pictures were very clearly just pictures of one thing like it's this certainly happens as well But do be aware that when you're using a pre-trained network You are inheriting the shortcomings and biases of the data it was trained from and therefore you should always look at The data it was trained from Okay, so being aware of that I would say that for us This is going to be quite a suitable kind of data set and as we look at the data set you'll see why I say that So each year most of the winners of the image net competition make their source code and their weights available So when I say their source code and their weights The source code is a thing that defines do you remember I told you there are kind of three bits that give us modern deep learning It's the infinitely flexible function. It's the way to train the parameters and it's the fast and scalable The particular functional form is what is the neural network architecture? We're going to leave learning a lot about that, but that's the source code Okay, so generally you download the source code from the folks that built them all and the second is the parameters that were learned so they generally an image net winner has trained a model for Days or weeks nowadays often on many GPUs To find the particular set of parameters the particular weights that make it really good at recognizing those image net pictures So you generally have to get the code And the weights and once you have those two things you can replicate that particular image net winners results the one of the winners of 2014 Was the visual things for the visual geometry group. It's an Oxford University group And so there's this model called VGG. You'll hear about it. What's generally speaking every year's image net winners The particular models they used are so well used in the community that people call them by name So like the 2012 winner was Alex net 2014 was VGG 2015 was I guess our inception 2016 was res net so they all have like notes VGG is a couple of years old So it's not quite the best today but it's special because It's the last of the really powerful simple architectures We will get to the more complex architectures Depending on how we go it might be in this set of classes. If not, it will be a next year's set of classes Hopefully this year's set of classes But VGG's simpler approach is Not much less accurate and for teaching purposes, you know We're going to be looking at something that is pretty state-of-the-art and it's really easy for us to understand So that's one of the reasons we're using VGG another reason we're using VGG is it's Excellent for the kinds of problems that we were just talking about that Robin with his satellite imagery has which is it's a great network for Changing so that it works for your problem even if your problems a little different Okay, so there's a number of reasons that VGG is a really great thing for us to be using My strong preference is to start out by showing you how to do things that you can use tomorrow Rather than starting with like the one plus one and showing you how to you can do things that are useful in six years time after You've got your PhD. So I'm gonna start out by showing you seven lines of code that do everything you need and To get to the punchline the state-of-the-art for dogs versus cats in the academic literature was 80% accuracy This gives you 97% accuracy Okay, and you don't need to do anything else and so for you after this class to kind of See if you can get everything working Basically your job will be can you run these seven lines of code? Okay, and if you can Then you can rerun it on your own data set as long as you structure the directories the way that I just showed you So what I'm going to do is I'm going to go through these seven lines of code or something very similar to them line by line and Kind of show you some pictures of what we're doing along the way I Wanted to start by showing you the seven lines of code though because we're going to be looking at all kinds of things along The way in order to really understand what's going on And at some point you might start thinking Gosh, there's a lot to do to do deep learning But there's there's not there's a lot to do to really explain and talk about and visualize and think about deep learning But for you to actually do image classification You just need these seven lines of code Yeah So what does it mean to train a model that's already trained? Yes, you're getting a little bit ahead of us, but it's an excellent like it's great to answer these questions many times So let's try and make a starting answer at it in this case The VGG model has been trained to recognize photos of the 1,000 pipes that are in the image net competition There's a number of reasons why that does not give us dogs versus cats recent number one is That if we go into the animals section of image net Here we are animals but beings and beasts and brutes domesticated, I guess There we go dogs Hunting dogs sporting dogs pointers Visla, okay, so They have two thousand three hundred and thirty two pictures of Visla Okay, also known as the Hungarian pointer so You could go through and run it and go back and find all the viswas and the German short some things and the status Say, oh, they're all dog But that's something you'd have to do so that's one Kind of shortcoming of the VGG approach compared to what we actually want the second shortcoming is that sometimes It's going to get it wrong and like it might get it wrong for like really good reasons for example, maybe this one It comes back with snow right But in fact, it's going to come back not just with snow But it's going to come back with a probability for every one of the thousand categories So it'll be probability of point oh three that it's a mushroom and point oh two that's an airport and you know Point four that it's snow and point three that it's a German Shepherd Say and so we want to kind of take advantage of all of that information as well. So What this actually does is it does something called fine-tuning which we've had to learn a lot about and what fine-tuning does is it takes that pre-trained image model and says use everything you know about the thousand categories To figure out which one the cats and which one the dogs That's a great question. And we're going to go back and talk about that a second time when we get there So this code is going to work for any image recognition task with any number of categories regardless of whether those categories are in the image net or not and Really the only image kind of processing image recognition that they're not going to do is something where you want to recognize lots of objects Okay, this is specifically for recognizing, you know a class so let's see how it works so the VGG object let's run this from VGG import VGG 16 Okay, so When something's running it has a little star You will probably get this warning that coo DNN is more recent than the one Tiano officially supports So this is a good time to talk about some of the layers that we have going on in this example We're using our VGG 6.G VGG 16 class It is sitting on top of as we will see this in detail Keras, which is the main deep learning library. We're using and what talk a lot about Keras is sitting on top of Tiano Which we'll be talking about quite a bit that less than Keras The Arno is the thing that takes Python code and turns it into compiled GPU code um The Arno is sitting on top of a number of things broadly speaking Nvidia's CUDA programming environment and part of CUDA is the CUDA deep neural network library coo DNN for most Important things in deep learning. The Arno is simply calling a function inside CUDA in it Okay So one of the things that we've set up for you in the AMIs and the scripts is to get all your is to get all of this stuff stuck together and working Oh, thank you Rachel. That's a very good point. So Keras is it's all written in pure Python okay, and What it does is it takes your deep learning architectures and code and turns it into in our case The Arno code it can also turn it into TensorFlow code TensorFlow and The Arno are very similar. They're both libraries that sit on top of CUDA and provide really a kind of Python to GPU Mapping and lots of libraries on top of that TensorFlow comes out of Google and it is particularly good For stuff that Google really cares about and in particular running things on lots and lots of GPUs one of the Things you will hear a lot is you can't do anything with deep learning unless you have shit loads of data and shit loads of GPUs That is totally totally wrong as you'll see throughout this course It is true if you want to win image net next year You will need lots and lots of GPUs because you will be competing for that last 0.1 percent Against Google and against Microsoft and against by do. However, if you're trying to recognize 10 different skin lesions You know like the folks that have showed you where they were the first people to try to do that with deep learning and they quadrupled the previous state-of-the-art Using one GPU and a very small amount of data that they had hand-collective So the reason you see a lot of this stuff about lots of GPUs and lots of data As it's part of the trying to make neural networks call rather than uncall It's trying to make it exclusive rather than inclusive. It's like unless you're us, you know, you're not in the club and I really Don't want you to go in for that kind of thing You will find again and again. It's not true As I've just shown you in fact with seven lines of code You can turn the state-of-the-art from a 20% error rate to a 3% error rate And it takes about five minutes to run on a single GPU which costs 90 sets now. Okay, so I Am not going to be talking much about tensorflow in this course because You know, it's it's still very early. It's still very new It does some cool things, but not the kind of cool things that uncall people need access to Tiano on the other hand has been around quite a lot longer It's much easier to use it does not do multi GPUs. Well, but it does everything else. Well If you build something in Keras and You get to a point where you're like, okay, this is working great We've got a 400% improvement in the state-of-the-art. I want the extra 5% that comes from running this on a GPUs It's a simple configuration change to turn it into to change the back-end to tensorflow and specifically I want to show you that configuration change Your you'll find in your home directory So for those of you that haven't used bash before when you see tilde that just means your home directory Okay, so in your home directory, there is a dot Keras folder and in there There is a Keras dot JSON file And this is the configuration and you'll see here back-end Tiano Okay, so if you change this to say tensorflow and rerun it it's now using tensorflow Okay, and tensorflow will use all of your GPUs If you do that I also suggest changing this thing here that says th which stands for tiano to tf the changes that stands for tensorflow We may talk about that in the next course. It's a pretty minor detail The other configuration file to be aware of is the Tiano RC You'll find a lot of unixi things are something RC is how they name their configuration files So here is well exactly dot something RC. So here is dot piano RC. I Want to point out that there's a really important line here, which is device equals This is either GPU or CPU If you're using a t2 instance, you will find that the AMI we've created has changed the g to a c That's because the t2 instance does not support GPU Okay So if you want to switch from GPU to CPU just change the g to a c or the c to a g okay, so those are the two Configuration pieces that you may need to know about for this class You won't really need to know about those because everything's been set up for you But I you know I like to show you what's going on behind the scenes Okay, so this morning that coo D&N is too recent that says If you see any problems try Updating Tiano or downgrading coo D&N. I haven't found any problems. So you can ignore that morning It just means that we're using a more up-to-date version of coo D&N than the Tiano authors have tested Okay, so we create our VGG Object In doing so, there's a whole bunch of stuff going on behind the scenes. We're going to look at all of it Okay, so by the end of the next lesson you will understand every line of code in our VGG Script for now I Would just point out that you can look at it because you downloaded it and inside it You'll see there is a hundred lines of code. Okay, so it's not very big at all All right, so we're going to understand all of it by the end of the next class For now, let's treat it as a black box. Okay, there's a pre-trained network. It's called VGG 16 We now have a VGG object which gives us access to that pre-trained network With deep learning With deep learning We don't look at Images or data items one at a time. We also don't look at them a whole data set at a time We look at them a few at a time and the number that we look at all that little few that we look at a time We call either a batch or a mini batch We run this a mini batch is simply grabbing in this case images a few images at a time and the size of that is the size of the need that to the size of the batch and Computing on all of those at once Why don't we do one at a time? Okay, thank you the reason that we don't do one at a time is Because a GPU needs to do lots of things at once to be useful It loves running on like thousands and thousands of things at the same time because it can do all of them at the same time So a single image is not enough to keep your GPU busy and it's slow Why not do all of it or the whole data set at once? Well, first of all your GPU only has a certain amount of memory generally somewhere between about 2 gigabytes and about 12 gigabytes and generally speaking your data set is unlikely to fit in that memory and secondly There's just no need to do the whole lot generally pretty much everything we want to do We can do a small amount at a time So in this case, I'm just going to show you how we can look at the result of this BGG model And so we're just going to do four at a time So there's a get batches command which basically says in our BGG model Let's look inside the path that we've defined and remember that's path We made the sample path and grab four at a time Okay, so we've got an error. That's good No such directory Okay missing a trailing slash So it's good to see these kind of errors. So we'll just go back to where we defined it add our training slash And go back down and run it again Okay, so we're in the sample. So there's 16 images So let's grab one batch and so that's going to grab four images and four labels So here are the four images And here are The four labels and so you can see that it's labeling at zero one if it's a dog These are all dogs. That's rather annoying. So let's get a few more And it'll be one zero if it's a cat. There you go. It's a cat one zero. So now that we've done that So that's basically what our data looks like We can call BGG dot predict Passing in the images and so that's now going to ignore the labels of what it actually is and it's going to use this pre-trained model And tell us what does it think the four things are and in this case? We run it it thinks they're a Rottweiler an Egyptian cat a Toy terrier and a Rottweiler. So you can see it's clearly made a mistake here. Okay It's very rare that actually I find it makes a mistake I'm glad it did make one and you can see it must have been confused by all the awkward stuff going on in the background So It's also shown you for this one. That's toy terrier. Here are the probabilities of each It's only 24 percent sure that it's a toy terrier Okay So you can see that it does actually kind of know that it's not sure where else the Rottweiler It's very sure it's a Rottweiler How come it's not so sure it's an Egyptian cat? Well, that's because there's lots of cats that look a little bit like an Egyptian cat It doesn't quite know which one it is So we could have a look at all of those details and see exactly which which other ones it thought it would be And we'll be looking at that in the next lesson So the fight final thing I'm going to do Is to show you how we can take these probabilities and Turn them into a dogs versus cats model I'm going to do it quickly now and then I'm going to revisit This in the start of the next class since we're out of time so to take that Thousand probabilities, so we're just showing one probability from each But there's actually a thousand probabilities for each thing. This is the probability that it's a tench. It's a goldfish There's a great white shark so on and so forth take those thousand probabilities and turn it into a dog versus a cat prediction We basically do exactly what we did before just to say get batches But we call fine tune and what fine tune is going to do is it's going to build a new model and it's going to replace the thousand categories with The two classes that it's found and how does it know what the two classes are? Well, that's because We have directories called cats and dogs Okay, so the fine tune command has now created a model that checks for cats and dogs Okay, just creating the model is not enough. We have to actually run it Train it. So if we then go dot fit It will then Use that gradient descent method that I talked about earlier that propagation and it will attempt to Make that model better and better at determining cats versus dogs Now obviously doing it on just 16 data items is a fast but be not very accurate So I can run it a few times and you can see that the accuracy is actually getting higher and higher each time But the validation accuracy is not getting much higher Okay, and that's because I'm running it on the sample so If I ran it on the full data set it would take about five minutes to run and you would you try it when you Get home give it a go and see what accuracy you get if you want to make the accuracy higher just rerun this cell a bunch of times Great, so that's the end of today's class It's the class which it's kind of like the opening of a novel You know when you have to introduce all of the characters and their backstories and stuff And so a little bit less deep learning goes on in the first class a little bit more getting set up Your first week is likely to be for many of you the most kind of frustrating and challenging week Because many of you will find you've got some kind of configuration problem. We don't understand how some piece of the Stuff fits together Don't worry by the end of the seven weeks It's all going to be that stuff's all going to be straightforward and all of the interesting bit will be in the deep learning So I think the more time you can put into this week Making sure that you get all of that infrastructure stuff working and comfortable with what it is Take all of the things I've introduced today Look at the video and go and Google all the stuff that you're not already familiar with Understand how it works anything you're unclear about ask us and your colleagues on the slack channel on the forums Teaching is the best way to learn so go to the wiki and try and explain the things you've learned And make sure that you can run the code up to here That we've seen today For those of you who are pretty familiar with most of this already Make sure that you can run this code on a different data set And we'll talk about some possible different data sets that you can use tomorrow Any of you that want to go further Please let Rachel or I know we have lots of ideas for ways that you can extend this a long way Thank you so much all for coming and I'll see you next week and talk to you during the week And make sure that you've got your teammates details so you can all stay in touch. Thank you