So this is the Python Intensive Flow talk. Thank you very much for coming to this hour-long session right after lunch. I know many of you are going to start getting sleepy right around the 40-minute mark, 30-minute mark. So do your best to stay awake. I'll do my best to keep you awake. So let's kind of work together to get through the talk. So if the slides will advance, just to introduce myself, my name is Ian Lewis. I'm a developer advocate at Google. I work on the Google Cloud Platform team, so that kind of encompasses all of Google Cloud Platform, so a few people are not new people, but a few you guys are familiar with things like App Engine or Computer Engine or that sort of thing. That's what Google Cloud Platform is. So I'm on Twitter at ENM Lewis. I've been tweeting sort of throughout the conference, so you should be able to find me fairly easily on Twitter. And just a little bit more of background about myself. I'm based in Tokyo, Japan, so I've lived in Japan for about 10 years, and I've been kind of active in the Python community there as well. So I'm one of the four people who kind of founded the PythonJP conference, which is about a 600-person conference, just to give you an idea of the size. And we're going to be having the conference in September. In the third week of September, I believe, it's from the 20th to the 24th, I think. And if you look at PythonJP, you can find out how to register. I think there are, as of now, something like 20 slots left, so hurry up. I'm also pretty enthusiastic about other kind of communities, so the Go community as well as the other open-source kind of projects coming out of Google, like Kubernetes and Docker and those type of containerization type of things. So that's the type of thing that you can expect to hear from me if you follow me on Twitter. So first, just as a kind of a background, I want to go over kind of what deep learning is and kind of give a very high level, not necessarily a high level, but sort of a quick overview of what that is. How many of you guys went to the talks earlier than day about the deep learning? So quite a few of you. I'm going to try, as my best, to kind of build on that, but there may be a little bit of overlap. So what are we talking about when we talk about deep learning? So we're talking, in terms of deep learning, we're talking about a specific type of machine learning which is using neural networks. And neural networks are a way of doing machine learning where you build this kind of network of nodes that are interconnected. So you essentially give something like this cat picture. You change the pixels into a kind of numerical representation. You pass that through as the input layer into the network. And each of these internal nodes will take the values from your input and do some operation on them and then eventually give you the output. So these are typically organized in layers. So you can see this blue one is the input layer. The orange one there is what's called a hidden layer. So if you think of a neural network as kind of a black box, the hidden layers are the layers that are actually inside that do the operations. And so each one of these little nodes does some sort of operation on the input and that's called basically an activation function. And then each of these are kind of linked together using weighted connections. So each of these little lines connecting the layers will be weighted to indicate a strength between each of the layers. So what are the neural networks good for? So neural networks are essentially good for classification and regression problems. So these are very wide class of problems that you can apply machine learning to.  And regression is a little bit more, somewhat more complicated in that you get instead of like a probability that it goes into a single bucket like between zero and one, you get kind of a scalar output. So say you get some values into your neural network and output you want to say like is a temperature. So like from say zero Kelvin to some value or some temperature that's more like a scalar or could be solved by a regression problem. I'm going to be talking mostly about classification problems but regression is also something that neural networks are pretty good at. So what does that actually look like? So here's a little demo that's available at playground.tensorflow.org. This is like a little demo that allows you to kind of look into a neural network and kind of get an idea of what's going on. So here we have some input features. So these are some values that you add, they do input into the network and then you have some hidden layers in the middle and then you get some sort of output. So if you went to some of the earlier presentations you saw something similar to this where you have say a, my example I'm going to use is like say that you have like the weight and height of a person and then you have two different categories here like these orange ones are maybe children and these blue ones are say adults. So if you want to classify a new piece of data coming into your network you could say you know train a network to do this but this is really you know very easy to do like this is essentially a linear classification problem or a linear regression type of classification problem where you can just draw a line in between the two and get a way of predicting between, predicting between the two. But let's say you have something a little bit more complicated. Let's look at one where the one category is completely encircled by the others. So if we were to do something like this and then try to train using just some you know X and Y inputs this would actually never basically never converge it would never figure out how to do this properly. So we can do things like add a hidden input layer that will essentially do the this kind of linear classification multiple times so you can say let's do this one time we'll see that it basically creates one line here so everything on this side it will classify as orange and this side it will classify as blue but then when we add like new layers whoops not layers but new nodes we can actually see that it gets a little bit more complicated so we could say it now figures out two lines or uses two lines and then aggregates the result together so you can see in the one node it's done one kind of linear regression in one node it's done another and then when you combine that together it kind of makes this band here and then whoops not this one that one now if we do it with three we can actually combine the result three times and we get kind of a triangular type of structure so like as we add these kind of nodes and and hidden layers we can do things that are more and more complicated okay so that's great so like but how do we classify something that looks more like this so this is kind of a spiral looking thing and these this spiral is blue and this spiral is orange this is something that's quite a bit more difficult and we can't necessarily classify this using something like just x and y inputs so you can imagine maybe sine would be good or x squared but even still like we don't really get very very good output just by having a very shallow kind of network with just three nodes here so in order to actually make this a little bit more complicated or to or to solve these more complicated problems we need a much more complex network so for something like this this may or may not actually converge but it's getting there so so right now this is actually not terribly stable but it will stabilize so like so once you if you have these kind of more complicated networks that you can kind of put together you can actually start solving more and more complex problems and so I'll talk a little bit about why that's important a little bit later but you can see that each end of these individual nodes like has their own kind of addition to this to the to the final output and then each of these little lines here are weight show the weight so these blue ones are positive and these orange ones are negative so that's going to show us that the negative ones are actually a inverse inverse relationship so with these inverse relationships you can essentially just you know reverse the orange and the blue in order to get the right type of output but essentially you can have these kind of positive and negative weighted connections between the different nodes let's turn this off so it doesn't burn my CPU and then let's go back here by the way that that demo is basically just a way of getting to understand neural networks it's not actually using TensorFlow under the hood it's like all done in JavaScript in the browser but it's essentially a way of like kind of getting a familiar more familiar with neural networks so is a neural network so a neural network is essentially when you break it down is essentially a pipeline of basically taking something like a matrix what is essentially called a tensor and putting it through like this pipeline of operations and so you can imagine that each of these is say like a matrix multiplication type of problem or type of function where you take one matrix multiplied by another matrix multiplied by another matrix and another and another and another than another and then eventually you get out a tensor that represents the output for a particular for your particular problem and this is basically very loosely modeled after how the brain works and in how the the the individual nodes like have have a kind of strength or a weight in between the neurons in your in your brain have a certain weight between them but from a from a practical point of view you're essentially doing matrix operations on on a bunch of times in order to do some sort of prediction so I mentioned what a tensor and this is where the this is what a where TensorFlow gets its name but a tensor is not something that people necessarily think of very often or don't encounter it too often unless you're a machine learning type of person but most people are familiar with things like vectors and matrices and a tensor is essentially a generalized version of that so you can imagine like this kind of like 2d like Euclidean space or 3d space and then you have some sort of value out here in the space and so for something like a vector you know you would have like you know a 2d type of vector and that could be say represented by a a single array in a programming language or a matrix which is a you know a three-dimensional vector or two-dimensional vector but a tensor is essentially a generalized version where you have this n-dimensional type of vector so you could have like any number of dimensions so this could be one dimension for each type of feature that you're actually adding to the into the into the network and you can essentially do the same sort of operations on a tensor as you would on a say a matrix so like matrix multiplication matrix addition that sort of thing so how the how a basic neural network works is that you would have these kind of connected nodes so this is our input vector or input tensor with x1 x2 x3 then we have a weights tensor or that is represented that you can then multiply against and then finally we add the resultant result biases in the form of a tensor and then softmax it to get a the output so this is a very very basic one one-layer network but you you can think of these individual not these individual weights or whatever are not each matrix multiplications but essentially this this matrix times this matrix is this kind of interconnected makes this kind of interconnected pattern and so this is how like the the if you went to the some of the earlier talks most of the the operations are formed in this way where you have the input x times w which is the weights plus be which is the biases and then you can do that multiple times for for each layer in the network and so these are basically just the multiplications and additions and then we have this kind of softmax thing at the end this softmax is essentially just a form a way of normalizing the data once it comes out so you'll typically see these at the very end of a network so what happens is after you've gone through this network these outputs what it would be at this level is you would have some sort of value like say 50 this one's like 50 this one's like 20 this one's like 0.32 you know and like so you don't really get an idea of like what that actually means that's kind of these values are kind of a relative value for your actual network so when you put this through the softmax function this will actually normalize it to a value between 1 and 0 so you can get essentially a prediction output so and then these these individual values would represent whether the the percentage say of that's a particular value goes into a particular bucket say this is a cat dog and this is a human and we put in an image value the output might be like that it's 0.99 percent certain that it's a cat and 0.01 percent certain it's a dog and 0.001 percent certain that it's a human which essentially means that it's a cat so that's great so that's how like we actually like do predictions so like we say input and input we go through all these operations and we get some sort of predictive output right but how do we actually train a model so a model is trained in in this way where you have you use a method called back proposition which was talked about at some of the earlier talks but essentially what you have is do you have this this here is the neural network as we've been talking about it before so here's like say one layer here's a second layer here's our softmax and here's our output and we actually go through here and we do the prediction but then what we do is we use we use test data to actually as we put it through our network so we have some test data that says here's here's the actual data here's the cat picture this is a cat so you have the actual value or the actual output the expected output and the actual the test data associated with each other so you know which ones are cat pictures which ones are dog pictures and so what we do is we put this say that cat picture through here and then it comes out with it with a result and what we do is we take that result and we end the expected value and then we find essentially the difference between those two values so say if it came out that it was you know 0.86% certain that it was a cat but we know that it's 100% certain that it's a cat we want to be able to nudge our our network in the direction of actually determining with 100% accuracy that it's a cat so we'll take this output and use what's called a loss function to find the difference so a typical loss function might be cross entropy but there are a number of other loss functions that you can use depending on the situation and then you go through these other you kind of optimize the results by using something like gradient descent those were also talked about a little bit earlier I'll talk a little bit more in general about these kind of optimization or especially gradient descent but essentially what you do is you put through this this optimization function and then back propagate all the values into the weights and biases for each individual layer so these these this weight one weight two bias one weight two and bias two are actually the weights and biases from that are used in the network here and so what you're doing here is you're essentially back propagating all these values and updating the weights and biases and kind of nudging the the network in the direction of actually doing doing the giving you the proper output and then you do this essentially many many many times you know training it over and over and over and over again and it eventually nudges into the direction of being a very an accurate network at least that's the that's the theory so this doesn't always work but but in general that's the idea behind it so that's kind of like a relative overview of how the neural networks are like so why are we actually talking about this so one of the earlier talks mentioned things like that these like image net is a you know a famous open data set about for for machine learning would get like say like 25 percent error rate in like 2002 but so essentially the reason why we're talking about these kind of deep neural networks all of a sudden is because people have started to get very much better at creating these neural networks and this is because of a number of kind of breakthroughs in terms of training these networks to do things that are actually practically useful so you can think of the the quality of a neural network kind of like this so like kind of traditional deep or traditional learning algorithms would kind of as you give it more data would kind of increase in performance but would kind of level off at a very quickly and then you would have like small these small neural networks which would also kind of level off quickly and so essentially what people did was they would you know train the amount of data to about here or give it a certain amount of data to about here and then they would basically you know they wouldn't adding more data wouldn't actually make it much better so they would essentially be able to stop right here but we've since found these kind of these you know neural network methods that allow us to scale the the learning much better so as we throw more data at the problem they actually get quite a bit more sophisticated and have quite a bit better performance so we've been able to create these large deep neural networks that will continually get better as we give it more and more data and with that comes like other problems which I'll talk about in a second but essentially these these like medium and large neural networks have become possible recently and so here is a model of what's the this is a Google the Google net network that was used this is called this is essential the inception model that was trained on Google net and so what this is essentially doing is like labeling pictures or labeling images you can think that each of each one of these as being say say a matrix multiplication or some sort of operation on a matrix and then it goes through several different kind of layers and then eventually gives you out an output tensor that tells you the labels so this is what we mean when we talk about deep neural networks so networks that are essentially have like many many many many layers before they actually give you these this output and by adding these layers we actually can start getting more and more complicated you know solving more and more complicated problems and actually getting pretty good results about with them but this gives us a problem where we have you know you can imagine that each one of these is a matrix multiplication and these tensors might be you know a large image like a megabyte or something and you're changing that into a tensor and then doing a matrix multiplication on it you can imagine how many actual you know operations you have to do in order to actually train this or to do even prediction even just once so you have to do this many many many times over in order to actually train a network and so what people do is they use GPUs and these GPUs are very good and high-proud but still you're essentially waiting for like weeks or sometimes even months for the results of actually one one single training run so what people are starting to do is late they use like supercomputers in order to train models faster but still this is a problem because not everybody has access to a supercomputer how many of you guys have access to a supercomputer somebody does that's that's the most I've ever seen it was like three or four I think so how much do you pay for that by the way so those the those are supercomputers are something that you have to lease time on so they're like the old the main frames of old you know where you had to like lease some time you know 730 to 830 and you know like in the middle of the night or something like that and you pay tons of money for them so they're not exactly the easiest or the best way and we wanted like you know the ideal thing is to be able for everybody to be able to do machine learning so what you need is kind of distributed kind of training and so like at Google we've been able to do that and so we use it for a lot of practical applications things like Google photos and like detecting text in street view images so there's a lot of kind of exciting things that are going on and essentially recently we've these kind of breakthroughs allowed quite a lot of activity at Google so this is a number of projects internally at Google that use that use learning machine learning this is just number of directories that contain a model description file but you can see from 2004 we've got this kind of hockey stick growth and yeah by distributing it we've been able to do you know much much faster okay so now I'd like to talk about TensorFlow itself so TensorFlow is an open-source library it's a generic or a general purpose machine learning library for particularly for doing neural networks we are also kind of expanding it to encompass other types of machine learning but it was open source in November 2005 and it's used by internally at Google for a lot of our internal projects so it supports a number of things like you know you know this kind of flexible and intuitive construction you know do basically be able to do a lot of things in an automated way and you can it supports training on things like CPUs and GPUs etc but one of the nice things is that you define these kind of networks in Python so before I kind of dive into looking at what TensorFlow looks like some of the core concepts is that you have a graph so TensorFlow is the name of TensorFlow comes from the idea of like taking tensors and then having them flow through a flow graph or a directed data flow type of graph so a graph is a representation of that these the operations of the actual nodes of the the operations that you do and then a tensors of the data that's actually passed through the through the the network and then we have other types of structures so we have these like the idea of these constants which can be something that doesn't change but then you have things like placeholders these are basically inputs into our into our network these these variables so variables are things that I can actually change during the training so these are the things that you usually use for your weights and biases etc and then session is something that actually encapsulates the the overall like connection between TensorFlow's core and how you actually and the models that you define so I should mention that TensorFlow is a library that is based on the same sort of concepts as many other libraries kind of scientific libraries we have a Python interface or an API and then it has a kind of the C++ core that enables you to do these kind of very fast operations so when you're actually doing training you don't you're not actually going through the Python VM so these are you know a non-exclusive or a non exclusive list of all of the operations you can do with TensorFlow so things like math addition subtraction multiplication division of these tensors matrix operations stateful kind of operations etc etc so let's actually look at what this looks like so I'm going to run through so this is a this is a Jupyter notebook so how many people have heard of Jupyter use Jupyter etc okay how many people have been asked that more than five times at this conference yeah that's what I thought so I'm gonna just assume that you guys know Jupyter and just kind of go from there but let me actually just restart this kernel here yes this is a Python 2 one because TensorFlow also supports Python 3 if I remember right but this particular example is Python 2 yeah so TensorFlow is pretty easy to get started there's like this is this is just using the kind of MNIST example so one of the so mr. Rashid was talking about the MNIST example earlier today but it's essentially a bunch of images that are kind of handwritten numbers and then you just OCR on those to determine which type of which number is actually present in the image so the training images look something like this where you have 55,000 images and they're all in this big huge long array of and each one has 784 pixels and they're basically monochrome so like they're just black and white but the and so if you look at the shape of that you know it's a 55,000 size array with the 784 pixels but if you look at the these the images they're essentially each value in it is the this is each one of the images in a kind of a two-dimensional array and each of these values is a value from zero to one of essentially how dark that particular pixel is so some of these are like 0.23 which is kind of a light gray all the way up to one so that's essentially what the data looks like so that's how we've actually represented here like if you had a color image you would need to represent it a little bit differently but that's essentially how we're doing it in this case and then this is just using this is just showing an example value so using that plot wide so this is just one of the input images so that's essentially what the training data looks like but then we have these training labels that are associated with each image that says that's basically a 10 you know an array or a vector of size 10 with you know a bunch of zeros in it and a one in the right location that indicates the number of the for that particular image so for this image we have an 8 here so if we look at the training labels the shape of that it's a 10 size and then if we look at this the particular one for this 8 we can see that the one is in the what is it this is like 0 to 9 or something in this particular column so in the 8th column I think this is like the 0 is for is actually for 0 so it's from 0 to 9 so that's that's essentially what it this is these are actually called one-hot vectors in where you have 0 in all of the values except for 1 and this is used often in training data but the data that you'll get out of it is actually similar to this except for it will be a bunch of values from 0 to 1 essentially a probability and so here's some of the images but as your some images of that I've kind of shown earlier but so once you're training it you can kind of get these you can train it to show these different set these weights and biases so that like individual pixels like will indicate whether it's a a particular number so in this case we're actually using a very simple neural network which will kind of with with just one layer which will work this way but essentially like if you see pixels in these blue areas that's probably a zero and if it's in the if you're there's any pixels in here in the red area that it's probably not a zero and then it like basically aggregates the probabilities and we're to decide whether this is a zero or not and you can kind of see that in in many of the other ones so like this one to one so if you see pixels in this area with a two it's like in this area and three in this area so they look similar to the actual probabilities the numbers you're looking for so I actually execute all these okay good so the next one this is actually us defining our our network so here we're defined we're importing TensorFlow and we're using the placeholder that I talked about earlier this is our input into the into the neural network and we have it is a size of 700 e4 so this is the the size of the the number of pixels and then we have these weights and biases as variables which can be updated as we train the model and then here is actually where we define our our network so here is this is just a single layer network so what we're doing is you can define it very similarly in Python so the way that you would do it in in mathematics so here we can say that we're doing a matrix multiplication on the input times the weight and adding it to the bias variable and then doing a softmax on it at the end and then TensorFlow internally will take these and build our kind of our like data flow diagrams or our data flow you know kind of model representation so once we have that out we can then use that to to then train our model so this is actually our our our neural network and then we have a placeholder for the output with this y prime and we've defined a cross entropy function here's our loss function and then we can basically put it into this gradient descent optimizer and optimize it using cross entropy and this will then create our kind of training step so this is this encompasses the entire the entire neural network plus the training that we need to do and as great like like the some of the other explanations and some of the other talks gradient descent is essentially a way of kind of nudging our neural network in the direction that we want it to so I think one of the talks talked about using going down a mountain using a single little you know flashlight or a torch and then kind of just going a little bit of the time down the mountain but essentially that's the idea you're essentially going down moving it in the direction towards towards a minimum to actually minimize the loss so that the say that this altitude would be the the error or the loss generated by the loss function and then we basically nudge it in a direction where the loss gets minimized and then essentially do that just over and over and over again so each one of these would be a training epic as we're going down the gradient descent optimizer so here what we're going to do is we're going to train a thousand times on a particular piece of data and so what's great is that we can also do this kind of mini batch training which is a way of you basically pick just a small subset of the total training data so we're not actually training every single time on the every single piece of the training data we're actually training on a random randomly selected batch of in this case a hundred I think or yeah a hundred elements and we're doing that essentially a thousand times this is taking way longer than it usually does and so the what's good about that is that you don't have to train on the entire training data you can essentially do something that's you basically pick a randomized you know subset of the training data and that's essentially the same thing you do when you do like say a statistical survey when you ask a bunch of people something you you basically get a significantly or a a a representative sample of the data okay so this is actually done by the way okay so the so this I've actually run through the training and then at the end we can we can actually check the accuracy of our neural network so in this case we're actually got about 90% which is pretty bad but this is a very simple like one layer neural network so that's essentially like kind of how you can you can use TensorFlow you can basically create these 10 these these steps to to run through it but all of these steps are actually or all of the actual computation is done under the hood in part of the CPU or in the the the C plus plus core and that's also mapped to it maps actually two devices so if you have GPUs or CPUs available it will actually map the operations to those particular devices so in this case I'm running this on like say like I think a 32 core machine so it'll actually map that so I'll talk a little bit about TensorFlow later but let me go back to see if I can find the back button it's not the back button it's a whole new thing okay so here I'm going to look at a little bit more a little bit more complicated example where we can get a little bit better accuracy so here we're we're training or we're using the same exact data that we did before but we're going to actually use build what's called the convolutional network and this is this was talked a little bit about earlier and so this is this basically allows you to to look at the image like kind of in parts and and basically pick the specific features from each part of the image and this helps with things like like say if you write the way that I had it earlier you know you had the the image and you had like if you saw pixels in a certain location then that would indicate what number it was but what happens if you write this the the the zero or whatever but you actually translated it slightly over a little bit that would actually change the way that the you know the net that particular network wouldn't be very good at figuring out that that I just moved the zero over a few pixels instead so so stuff like that that's kind of convolutional looking at it helps a lot but in this case what we're going to do is we're going to initialize the weights and biases a little bit different I think this one is just doing this kind of I think this one was picking like kind of random random weights to begin with but here's the kind of convolution part so essentially what we're doing is we're going over the image and we're picking a particular these are these are what with the what are called kernels I guess over the image and then we're kind of building this other value or this other kind of tensor that indicates that has a particular value for each of the for each of the the picked kernels over the over the image and then we can actually work on these so this is just picking kind of features of each individual part of the image rather than looking at the whole image or the image as a whole then we can take take that those type of things and use what's called pooling pooling is another kind of a method that you use to basically kind of the most common example is max pooling where you take the the individual value from a part of the of the tensor and you pick the maximum value so this kind of like gives you a somewhat of a representation of a particular part of the image as well and then kind of put that all together into into a layer and then you can do that like create several layers that like that look like that. So here we're doing a set of full our first convolutional layer by building these bases the building the weights and biases and then building our layer here and then the second convolutional layer takes the inputs from the or the outputs from the previous layer and does the same basically the same sort of thing and then at the end we create like a this is a densely connected layer so as the some of the previous talks talked about like the convolutional layer is not 100% connected between the values because they're actually using this kind of translated kernel over the image but the final output layer is kind of a densely connected layer which allows you to kind of just do basically the exact same thing that we did in our previous layer but we're just we don't have we didn't have the convolutional part and this will allow us to you know get a much better kind of output. I'm not going to really talk about dropouts but and then we have basically the real layer and this is essentially just doing a softmax on the outputs of our of our last part of the of our last output from the previous layer and then you can kind of train to execute the model this in this particular one we're doing this using the same kind of cross entropy but we're using an atom optimizer instead of a gradient the regular gradient descent optimizer and then with those kind of optimizations you can kind of get a a much better output or a much better performance so here we're actually doing a lot more training on this particular one because it's a deeper network and we can we can train it or it scales a lot better our previous one if we continue to train it more and more it probably wouldn't get it wouldn't get very much better than 90% but in this case we can train it quite a lot more times in order to improve the accuracy so we actually train it about 20,000 times on mini batches of 50 and so they will go through this is actually me doing this because it said this takes about five minutes or something to actually run through but you can see from the output that we get about 99.2% accurate which is a good deal better than 90 90% right so instead of one in 10 you know it's like around one in 100 is is is classified incorrectly so you can do things from from very simple networks to more cut much more complex networks so let me go back to my so one of the other things that you can do because TensorFlow has this kind of internal representation or knowledge about how the graph and everything is working together is you can essentially law like write log output files as you're doing training and these can then be read what by a by an application called a tensor board board so we were obviously very unique nice with the names you know tensor flow tensor board you get the idea but what this is really what's really cool about this is that where is it can I make it where do I make this bigger there it is is that you can look at the the things like the accuracy the values of the loss functions and these look at these kind of graphs as you are training and going over the data to kind of see how your network is performing so in this case we're seeing the actual accuracy as we're training it so this is one of the this is data I think on an all on the simple version so once we get up to about 90 percent we get there pretty quickly but we don't really get very much better as we train the data but you can look at like things like the accuracy but you can also look at the actual loss functions this is cross entropy looking at the cross entropy value and that kind of goes down and down and down and down so this should actually be the close to the inverse of the the accuracy but you can also look at many of the other values and this these basically corresponds to the to the variables or the the the individual parts of your of your the basically the values that you have so here cross entropy was an actual object a python object that you can use or that was defined and then you can get this kind of log output data so other things like the max and the mean and min and stuff like that are all kind of part of that the part of that let's see these are like kind of input images that you can look at but one of the other cool things is you can actually look at the graph of the data itself so or of the the the model itself that you're building so here we have a two-layer network so if we have a two layer network we can actually just kind of like zoom in and look at the individual pieces of the network like the weights and the biases and things like that for for individual parts of the network look at things like the dropout values the the loss function as well so like this this basically gives you from the python that code that we wrote will give you a full kind of graph representation of the of the network so in the case of say something like a very complicated you know the image net thing that I was showing you earlier you know you would see this huge huge graph of for that was generated by that but this is really cool because it helps you visualize your your neural network that you you've defined so let's go back here I have about 10 minutes or so left or something right yeah well we'll get there so the the main difference between this between distributed training and or between TensorFlow and many of the other length libraries that are out there is that TensorFlow is built from the from day one with distributed training in mind so essentially TensorFlow is built in such a way that we want to be able to production eyes or to actually do practical work with our with our network with our live with the library and with our networks so we want to be able to do to train things faster and based on the kind of like hardware kind of breakthroughs and improvements that we've done in the past we've made in the past we want to be able to utilize those to be able to train models faster so TensorFlow supports multiple different types of of parallelism so multi model parallelism which is essentially breaking up your model so each one of these machines takes a different part of the model and you basically feed it through through here and basically break up the work that way it also supports what's called data parallelism which is hopefully on one of these slides no it disappeared so data parallelism is the opposite where you basically break up the data instead but each one of the the the machines has a full copy of the model so you're basically splitting up say like record one through a hundred and sending it to one machine and you know a hundred and one through 200 and sending it to a different machine and then breaking up the values or the work that way and there's a number of kind of trade-offs between these whether you use like you do like a full graph or a subgraph type of model parallelism or synchronous or asynchronous data parallelism this is going to help or like you know there's kind of these pluses and minuses to each of these so that it's kind of there's no like similar bullet to this but I do know that at Google that we use pretty much data parallels or use data parallelism pretty much exclusively so TensorFlow basically supports in a number of ways this these different types of model parallelism and and data parallelism so data parallelism is this one okay so this is where you take the data and you kind of split it up and each one of these replicas has a full has the entire model and then once you've done some training you can pass this to the parameter server so this is the thing that holds all the weights in the biases so as these are updated though it'll push those back to the model replicas and then there's like kind of asynchronous and synchronous versions of data parallelism where you're updating the model or updating the weights and biases in parallel or you're updating them in synchronously for each you know kind of iteration asynchronous is is much faster but can kind of add some some noise to your to your model because these the parameters are kind of changed midway through can be changed midway through a run whereas the in synchronization you kind of run the you split up the data and then you wait until all of the models are finished a particular epic before you go into the next one so this will reduce it but it'll actually make the make the data a little or make the training a little bit slower so this is a kind of an example of how that would run with TensorFlow where you have a bunch of workers doing the paralyzing and then you have some kind of parameter servers and then in between the servers they use gRPC to to communicate so why is this kind of data parallelism important okay so let's say that instead of a cat getting a cat out of our network we got a dog and we're like well we want to improve our network we want to make that better so where do we fix what do we fix in order to make that actually better I don't know maybe this this is probably a good idea I don't know so we do that we'd make our tweak and we run this again and we're like okay yeah like this is right nice and it's like running on my GPU and it's getting fine like and then it comes out it's like it doesn't make it better and you're like oh crap where do I do now like you got to go back and you got to start over again so in order to like run these kind of experiments like you want to be able to run these experiments over and over again like very quickly you don't want to have to wait a week in order to figure out that your tweak went well or not and this is this is a problem with with people who are even experts in in machine learning is like you basically you have a you have experience and you have literature that you can you can use to kind of figure out you kind of narrow down what things you might want to tweak but in the end you need to be able to run that and test to see if the data or if the actual tweak that you made improved or doesn't didn't improve so you basically have to test it and this takes time and so that's why it's very important to be able to do this kind of distributed training but one of the problems is that as you scale up these number of nodes like these number of connections the number of connections between your parameter servers and between your workers increases like kind of exponentially and so this doesn't essentially scale you essentially bottleneck on the network because these these guys are talking over TCP and you essentially get kind of like you know in the order of milliseconds latency between the between the machines so you essentially need to build this kind of like you need to have like a dedicated network or a kind of dedicated hardware network a lot of people use things like if in a band or whatever in order to make this go faster but this is actually something that's a really big problem at the moment so one of the things that we did at Google is like we're releasing cloud ML like internally what we do is we create we do our distributed training but we have a district a dedicated network that doesn't use TCP IP and basically skips the whole TCP IP stacks and is able to have the communication between the the machines run on the order of you know of milliseconds or nanoseconds instead of so nanoseconds or microseconds instead of instead of milliseconds so this is something that we are planning on making public as what's called cloud ML which will allow you to basically run TensorFlow graphs on inside of Google data centers we're also planning on exposing as part of the API the dedicated hardware that we are using for so instead of GPUs you can use what are these these are called tensor processing units I think is what they're calling them but essentially what they are they're dedicated hardware used for doing tensor operations so so we're basically being able to like expose those but to to other people so that they can use use that kind of dedicated hardware in order to like kind of do more experience and things like that so I think that's all I had so I want to thank you for coming and spending you know the last hour here how many of your store wake raise your hand so about like 70% of you so but thanks a lot for coming definitely check out the tensorflow.org website there's tons of really good examples like if you go here and then if you go up here there's like tutorials and documentations this is actually really really good and has like lots of good examples about how to how to use TensorFlow and especially if you're also a person you know there's different ones for different levels of people as well as how to use TensorFlow serving to kind of move towards actually productionizing your your models so thanks a lot for coming and I hope you enjoyed that presentation yeah two-ish questions I think we have like two minutes so that works your question has to be about 15 seconds and then you know maybe 30 seconds for me to answer it is there anything like profiling for this kind of models like to have an overview of how many multiplication how many parameters that does each block of a flow graph need so you're talking about like actual like time that it took to to run it I don't know if if tensorboard gives you that I think that it probably should if it doesn't I don't know if that offhand if that actually is but I think that that could be something that you could visualize with tensorboard as part of the out you basically log that that is a as a value that you can view here in tensorboard and then kind of see that you know how each part of your graph performed and sort of like things like that other question we got one right there behind you so the previous talks today mentioned that you typically have to do some feature extraction before you can actually apply neural networks will TensorFlow help me speed up my more manually designed feature extraction or is it designed only to do neural network stuff so at the moment it's mostly geared towards neural networks I mean obviously you can do like feature extraction using a separate neural network so you could do like a neural network that does feature extraction and another one that does the actual like classification and stuff but there's there is some work going on there's like I forget what it's called it's like like TensorFlow wide or something like that I think it's called that is essentially instead of like having deep neural networks the idea is that you have these like kind of more standard type of machine learning algorithms and so I think there's there is work going on there to like incorporate more standard machine learning algorithms so you can do that sort of feature extraction beforehand and stuff like that that's kind of ongoing work you might try to search about TensorFlow wide I haven't played with it personally so I don't can't really give you details yeah thanks a lot


From Bahrain, it's theCUBE. Covering AWS Public Sector Bahrain. Brought to you by Amazon Web Services. Hello, we are here at theCUBE in Bahrain, Middle East for Amazon Web Services Summit. Our second year, we're cloud computing and they're a region of a couple of availability zones are up and running. Big news with Amazon. We've got our next guest here, it's Cameron Mirza, head of strategy at the University of Bahrain. You guys, big news, announcing a degree, bachelor's degree in cloud computing. Yeah, certificate one year. This is going to rapidly put new talent on the market. Congratulations, welcome to theCUBE. Thank you so much, we're really excited by this announcement today. And what's exciting about it is that, first of all, it's the first cloud computing degree in the Middle East. And the other element to this is that the students from any background, any discipline, can get a really good understanding about cloud technology through the certification because the challenges we face in the region right now are we don't have enough skilled tech talent and we don't have enough skilled talent to fill the jobs that are available in the region. This is not just a regional thing as you know, this is a global issue. And universities have to adapt, be a bit more forward thinking, live in the future and we feel really optimistic with our partnership with Amazon today that we can actually fulfill the needs of public sector, employers, entrepreneurs, governments throughout the region and that's the exciting thing for us. I mean, let's just take, well take a minute to explain the two components. One's a four year degree, one's one year. Just give a little quick detail and I'll go into some questions. So the four year bachelor program is going to be delivered in a very different way than the traditional academic program. It's going to be heavily integrated with the needs of employers. So employers are going to be really involved in curriculum design. We'd like them to be part of the teaching faculty as well. The way that the program will be delivered will be very much in a kind of project based way. So it's about developing not just knowledge but the skills, competency, values, mindset required to be successful in the 21st century. That's the exciting thing about it. And of course, you know, looking at some of the detail behind the curriculum, you're looking at networking, security, machine learning, artificial intelligence, big data. So the fact that it's cloud-based is actually just a small component to what it opens up in terms of broader skill sets. I mean, one of the things that we always comment here in theCUBE is we cover Amazon's re-invent, their big annual conference. And the joke is, how many more announcements are going to make this year? Tsunami of new things coming. So certainly it's tough to keep up, many people say that, but for the young people in school, this is relevant stuff. This is like pathway to success, job, making some cash, making some money. That's what the purpose of education is. Well, I think there's a couple of, that's a great point. And the first thing is that education systems now need to live in the future. Living in a current or in many cases, the past is not acceptable. So it means taking some sort of calculated risk, but we're very clear in terms of the direction of travel in regard to technology and the future of jobs. The reality is today, that two thirds of the world's population already needs re-skilling. Those are the challenges we face today. Young people are purpose-driven, they know where jobs are going to be, they want to work for themselves. They understand far better than anyone else, the way the future is unraveling. Do they understand how relevant this is? I mean, that's pretty obvious, we're in the industry. We kind of obviously, you've been part of the plump. Are they getting that this is wave, this wave is not going to end for a while, this is going to be a great upward migration for opportunity, or is it still learning on the young kids part? I think sometimes in education, we do a disservice to young people. They're so well informed, they understand the market, the trends, the way that technology is shaping the future. And the reality is that what a student learns in year one of a university, 50% of that knowledge will be obsolete by the time they graduate. So the focus is not just around giving them a degree, this is also about re-skilling and upskilling people who have graduated, people in the workforce, so this is a far wider opportunity than just young people. Well, I tell you, one thing that gets my attention is that this reminds me of the 80s in computer science because I got a degree, I was a freshman in 1983. It's just at the beginning of the operating systems movement. Linux wasn't even around yet. Unix was just emerging on the scene. And it was interesting, what we learned as building blocks with operating systems ended up becoming obsolete in the sense that we don't have to use it anymore, but coding still happens. So this has had scale to it. With Amazon, you got, okay, EC2 and S3. Now you got, as you mentioned, machine learning. You got Lambda functions, server lists, and so much more stuff there for a variety of jobs. I think this is just the tip of the iceberg and I think for us, the way that the education's evolving is that we really believe that education will be more modular, as you say, credentials-based, lifelong, omnichannel, so some of it will be hands-on, some will be through other channels, and competency-based, and I think that's the thing. I think competency for us is about the mobilization of the knowledge, your skills, the values, the attributes, and that's the bit that's going to add value to economies throughout the world. So head of strategy, you've got to look at the chessboard in the future, you've got to live in the future. What are some of the feedback you've gotten? As you talk to folks in the industry, when you roll this out, doing some press interviews, I know you've got some feedback. What's the general sentiment right now? Really excited. I think that we talk to employers all the time. We talk to SMEs, we talk to big players like Amazon, and I think that in the region, I think when we talk about the scale of disruption, I think the way we talk about it in US or Europe is very different to the way we talk about it here. I think the Middle East region, like many developing parts of the world, still playing catch-up in a little bit, but what you'll find is that once they've caught up, the adoption rates go through the roof, and that's the challenge for us because what we see the uptake now, we see the uptake every year growing and growing, and now the next challenge is moving into government, moving into the private sector, and upskilling and re-skilling those, so we're just at the start of this kind of huge opportunity, John, and I see it being, you know, exponential over the next five years. You know, it's interesting, I live in San Francisco Bay Area in Silicon Valley in Palo Alto. You see what Berkeley's doing in Stanford. If you look at Berkeley in particular, the number one classes are the data science class and the CS intro. I mean, they're kind of hybrid, basically it's all cloud. If you do anything with coding, it's going to be cloud-based. Right, right. And Sayeel, who's the deputy group CEO, bank ABC, I just interviewed earlier today, he said AI, he thinks is the biggest thing that's going to happen, so it's not just racking and stacking, standing up infrastructure with Amazon, although great to learn that, and there'll be nerds who will geek, so we'll do that. Absolutely, absolutely. There's a huge machine learning AI field coming. Yeah, yeah. I think that's going to be something. As head of strategy, you got to keep your eye on the prize there. Absolutely. What's your view on that? Do you see that happening? I think you're right. I think, look, OECD have recently released some data to say that over 20% of jobs will be automated as a result of AI over the next few years. And I think our role is to prepare young people regardless of what they're studying for AI and the impact of machine learning. So I'll give you an example, medicine, you can make a diagnosis now for a patient diagnosis in a fraction of a second compared to what we used to be able to by using AI. Now, the reality is that although AI can give you that information, you wouldn't, as a patient, want a robot to give you that diagnosis, right? So our job, I think, is to look at the skills that will define what defines us as human beings away from robots, and that's empathy. That's the stuff around building connections, around team working, around collaboration. And actually, those are the things that education systems are designed not to deliver. So our job now is by embracing these types of new programs, it's to start to work on those softer skills and prepare this generation of students for the AI world that we're moving into. Cameron, I'm so excited for your opportunity, computer science, cloud, all kind of bundled together and software is powering this new generation. As we say, it's the keys to the kingdom. In this case, it could be the keys to the kingdom. Well, I think for us as a national university, and for many, not just Bahrain, but for many developing and emerging countries around the world, this is far greater than just technology or creating jobs. This is about sovereignty, because if you look at many countries, they import talent, they have to import hardware, software, computers, everything's imported. This is a great opportunity to help create a workforce that actually flips that on its head, becomes the innovators, becomes the job creators. So that's the exciting thing for us. It really is a generational shift. This is an opportunity for the younger generation to literally take the keys to the kingdom. Absolutely. Absolutely. Cameron, thanks so much for coming on theCUBE. We really appreciate it. Thank you for your time. Cube Coverage here by Rain, Middle East, AWS Summit, I'm John Furrier. Stay with us for more coverage after this short break.


It's from Saq Loa, Hemant, please. Thank you. Good morning, coaches, Professor Withit and Chris, if I may refer to you as the teacher. Good morning, everybody.  And as Brianna, you mentioned yesterday that you were standing between the delegates and the coffee break. Today I have to repeat the same expression that I am also standing between a coffee break and all of you. But there is a famous expression that there is always a gap between cup and sip. And let me be that gap today. And mind it, I said cup and sip, not cup and lip. Well, as you all know that I'm a lawyer by profession and I represent Saq Loa, a body of lawyers, as it's a Secretary General. And many a times when I address events, conferences, et cetera, I put a question. How can you tell whether a lawyer is speaking lies? And uniformly, answer comes from all directions when the lips are moving. So well, my lips are also going to move today. But I assure you that I won't be telling lies because all my lies end in the courtroom. The moment you exit the premise of the court, you're not allowed to speak lies. Lies are only meant for courtrooms. Well, let me give you a brief background or a backdrop about Saq Loa because this is Saq Loa's first interaction with many of you, though we have been associated with all the UN institutions for quite some time. Saq Loa was established in 1991 in Sri Lanka and it comprises of eight member nations from South Asia. I need not mention those countries, you would know. The constituents of Saq Loa are judges, lawyers, law teachers, academicians, law officers of the government, and people who have expressed interest in the development of law. It's a very uncomplicated constitution which Saq Loa embodies. It has two objectives. First being getting the legal fraternity of the region together so that they can develop cooperation, understanding, and exchange ideas. And second objective, the most important, being using law as an instrument as a catalyst for social change. We are not a human rights organization per se. We deal with every facet of law, be it commercial, corporate, economic laws, social laws, civil laws, insolvency laws, and human rights laws as well. And let me confess that until 2011, we were in terms of human rights, I'm talking because this forum is more directed towards that particular branch of law. As human rights practitioners, we were primarily focusing on child rights and women rights, ending violence against children, or child labor laws, revamping, or rights of women. It was in 2011 that we were introduced to UNAIDS by Saq Secretariat, person to their guidelines. In fact, Professor Witte, let me share with you that on these SOGI resolutions, the three countries were part of that scheme of voting, India, Pakistan, and Maldives. Pakistan-Maldives voted against the resolution, and India abstained. So abstaining also means neither here or there. So that is also as good or as bad as voting against. But if you were to see the guidelines of Saq, which is driven by India, Pakistan, Maldives, and other countries, Nepal, Bangladesh, the guidelines actually talk about protection of LGBT rights. So they don't practice what they preach. Somebody has to educate our sovereigns that please don't sign on dotted lines when you don't know the intent of the document. I think somebody needs to embarrass them through media or perhaps through some workshop or seminar. Once we were introduced to UNAIDS, we signed MOU with UNAIDS, UNDP, ADB, IDLO. And predominantly, the focus was response to HIV AIDS in the region. What were the barriers, legal or policy barriers, and how they were impacting the LGBT rights? They were needing a partner, a legal partner for the Saq region, and they found legal partner in us. And I think we came up to their expectations to the maximum, I would say, because in last four to five years, we have had several events, projects, studies, all are uploaded on our website, where we steered many initiatives in the interest of HIV AIDS and LGBT rights. And here I would also like to share with you a very practical sort of point of view which exists in context of Saq region, that parliamentarians in this region have always failed their electorate. They have legislated only to meet their selfish interests, never ever they have legislated to protect the rights of LGBT. So therefore, left with no option, Saq law started leaning on the judiciary. We devised our annual conferences in such a way that it coincided with Saq Chief Justice Conference. We ensured that all the chief justices of eight nations were present, not on the dice, but in the audience, to listen to the sentiments of the marginalized or the, let me say, vulnerable sort of group. And why we did so? Because there is one good sort of movement which is happening in South Asia that judges are legislating now in a way. Although the law making power was with parliamentarians, but through declaratory orders, they have started passing laws in a way. And this 377 clarification which was issued by the Delhi High Court had an effect of a law because you basically created a proviso under section 377 by a way of which you excluded a particular segment of individuals from the purview of section 377. And which was very doable because day in, day out, courts are passing judgments which have the effect of law. So we clinked on with the judiciary and we started influencing their mindset in a way that they started passing verdicts which they gathered from the deliberations of Saqlal. So we will definitely take credit of that. And the interesting feature was that the case, whether pertained to that point or not, they would somewhere in their observations try to get in the content of the deliberation which they picked up in our conferences and workshops and seminars. So therefore this orientation of judiciary became very relevant for us. And with all due respect to national human rights commissions, they are absolutely fundamental for our society and for protection of LGBT rights. But at the same time, it is very important to involve constitutional courts because human rights are part of fundamental rights and fundamental rights are enshrined in constitution which can be protected more effectively by the higher judiciary. The judiciary, I mean to say, are great courts or constitution courts. And there have been many occasions where UN AIDS and UNDP have invited sitting judges who were earlier quite skeptical about attending these open events, but they have been attending, sitting judges have been attending because we have nominated several judges on their round tables where they have come, they have expressed, they have aired their views very openly with regard to LGBT rights. I'll just take a couple of more minutes.  They have been former ministers or future ministers, they are members of parliament. We have sitting judges, our president by convention is always a Supreme Court judge or a chief justice. We have had several chief justices as our president and the current president is also the senior most puny judge of the Nepal Supreme Court. If all goes well, he's likely to be the next chief justice as well. The erstwhile president was chief justice of Bhutan who took amazing initiative with regard to LGBT rights. And we must admit that South Asia had a taboo about these expressions LGBT. But we were able to make inroads into their minds and hearts and spirits so that they at least became open. And that's how this 377 order came by Delhi High Court. And if Supreme Court overruled that, believe me Professor Witte, it was because of case badly presented before Supreme Court. Because the focus of the lawyers of LGBT was on what? That sexual orientation is immutable, which is not, which is incorrect. And secondly, the focus was too much on sex. Well, a relationship between man and man or woman and woman is much beyond sex. The lawyer was actually wanting Supreme Court to draft a operating manual for how to conduct sex amongst LGBT community. You can't expect Supreme Court judge to give you a judgment with regard to operating manual as to what you have to do within the four corners of your room. The argument should have confined to only one line which Professor Witte has beautifully written in the forward to the regional report of the NHRIs. It's a beautiful expression. He has said that they wish to be what they are. Just this one line should have been focused during the argument that you can't expect a person to be otherwise which he is not or she's not. So an approach should have been taken with regard to a human rights angle rather than a 377 focused corner intercourse against the order of nature. It is not, I mean, it completely deviated the judge. And he thought that it's more of a lust oriented petition rather than a genuine sort of argument with regard to a relationship, with regard to a person who or she, what she is or what they wish to be. It's a matter of choice. So you cannot sort of convert somebody by way of a judicial order. So I mean, that's what I felt because I went through the judgment and but then don't take this judgment as a negative judgment because it's a 100 page document which sums up the entire argument of anti-LGBT lobby. So you don't have to look at any other document. Just look at this judgment and demolish it by carrying out a campaign with regard to by, you know, perhaps an event of this sort. Because each of the expression of that judgment is objectionable because of he saying that sex and food is within the domain of sovereign. You can regulate it, which is absolutely ridiculous. And then the judge goes on to say that this western thinking cannot be transplanted into the South Asian thinking. Why? We are living in a global village. I mean, I don't consider myself an Indian because at the drop of hat, I could travel any part of the world. So on one hand, we are moving towards globalization. And on the other hand, we are saying that this is West ideology. This is North ideology. This is South's ideology or this is the ideology of the South Asia. So the judgment was quite primitive and backward looking, but I won't blame judges to that extent because the arguments weren't carried forward so beautifully. Let me close my address by referring to William Douglas, Supreme Court judge of the United States of 20th century. He said that to look for static security in law or elsewhere is absolutely misplaced and misfounded. The security in today's context lies in constant change. You have to get out of age-old ideas. You have to move on. You have to adapt old ideas to a newer development. That's what, you know, I mean, these were few sort of views of mine or thoughts I wanted to share with you. Sacklow website you must visit. And my email ID is also there. So if you would need any sort of assistance from my side, I would be more than happy to come forward. Thank you so much. Thanks. Thank you to all of us. Thank you very much, and thank you to all of us who contributed, and clearly we've had time for discussion.


Hey there lads and ladies, it is Petrifying Pumpkins here and I have got a new video today. This one is kind of a weird one. It's about super hot. If you're speaking of super hot, it is super hot right now in real life. 19 degrees Celsius at half one in the morning. It's ridiculous. It's ridiculous. It just won't cool down. I can't sleep. I can't. Well, I can eat, as you can see. Anyway, that's besides the point. What caught my eye today was this post over on redis or slash psvr and it is by a user called vencer and this person posted that all scenes alluding to self-harm will be removed from super-hot on all platforms. So I followed the link. It leads to the steam page where this update went live. By American time that will be today as of this recording. I do not have super-hot installed on my ps5 so I don't know if it's out yet on psvr, but it is out on steam. So this is what the update is. According to them, this is the patch notes. Skip disturbing scenes toggle was added in a previous update. Considering sensitive time we're living in, we can do better than that. You deserve better. All scenes alluding to self-harm are now completely removed from the game. These scenes have no place in super hot virtual reality. We regress. It took us so long. We're committed to shipping this update to all the your platforms from the super hot team. So the super hot team is based in some European country. English is not their first language. So some of this kind of sounds a little bit off considering sensitive time we're living in, for example. So that's why that sounds a bit like that. It is an official update from the developers. In terms of what I want to talk about, I'm going to go into a little bit as well, probably big spoilers I guess. For super hot VR, if you haven't played it yet, if you care about the story in that game, which to be honest, I didn't really notice the story too much when I was playing through that game. I know there's something going on, but it's kind of one of those things you have to piece things together yourself if you want to figure it out. But here's the spoiler warning. If you don't want to know what happens at the end of super house, click away now and I don't know, come back when you figure it out or looked it up or played it yourself, whatever. So three, two, one, spoiler warning, gone.   I think it's to wake yourself up because you're in the virtual reality world and killing yourself is the way out or something like that. Either way, it doesn't matter what the reason is. The point is this was part of their vision. This is what they made. And it's not enough that they added this toggle in a recent updates or previous updates, skip disturbing scenes toggle. So that's there already. If that was bothering you, you couldn't hit that toggle and then you didn't have to see that.  Now, I must say I have never seen anyone complain about that scene. So if there is backlash, I imagine it's from a very small minority of people. And for those minority people, they added the toggle already, but now the way it is, or now it's the way it's going to be after this patch for everyone, that's going to be removed. So if you haven't played it before and you update us, which naturally you would presume would be the wisest thing to do updates us, that's going to be changed. It's going to be taken away from you. Now, does that make the game worse? I'm not saying it makes it worse. I'm not saying it being in there made it better. But it was what the developers wanted. It was the original creative vision and it feels like they've been pressured into changing that to avoid backlash. So that doesn't sit quite right with me. I'm a little bit disappointed to see this and it's just so strange because I mean, you could go to any amount of movies where the character or any character in it kills themselves, you know, and you wouldn't know that beforehand. So why is that okay? Or there's other games out there where there's suicide in this. I know virtual reality is a little bit different because you're doing it yourself.  So it's not like, what, who's this affecting? You know, who's this affecting that other art pieces that include suicide isn't already being affected by this anyway?  I don't like this. It sets a precedent out or whatever you want to call us can only be met a certain way now. Obviously, in the grand scheme of things, super hot VR is going to this is going to fly under the radar or so many people, which is kind of why I'm making this video about us because it's probably going to get lost in the shovel. I just don't agree with this, you know? I mean, I understand it can be upsetting for someone to experience like a simulation of suicide in a virtual reality game. That's fine. There's the toggle there for you if you want that or just don't play us.  That's been taken from us, essentially, especially years later when it's already been done. I don't know. Anyway, let me know your thoughts on this in the comments below if you care about it at all.  That's fine, whatever. But I thought it was worth mentioning and that is it for this video. Thank you very much for watching. Before I end this video, let me thank my Patreon supporters whose names are on the screen as I speak, especially the top tier Patreon supporters tradition, Pete Hawkins, Daniel The Pumpkin Patchkid, and Crumb. Thank you very much for that generosity that's appreciated. If you'd like to help me out on the Patreon, you can do so the link to that will be in the description below. Let me also thank Decepticon for letting me use his music in all of my videos. You can check him out at Decepticon.com. Link to him in the description below also and with that I will end this video. Until the next one, please stay nice and moist, which isn't too hard in these hot hot times we're experiencing right now. My face is moist. Do you see this moisture? Moisture right there. I'm going to jar this. Who wants to buy my sweats? Who wants to buy my gamer sweats? Put your bids in the comments below. Thank you.


Should we, should we sally forth and continue to find out what's going on? Yeah, do you have any idea? You might be asking yourself, what did I just find? An actual fire dancer? Morgan is. As a sample of direct versus indirect measures, I'd like you to just watch this video. So what are they looking for? What are we looking for? The people? How? How are we trained in it? The people in there? Oh, in the tunnel? Or the people watching the video? The lights. Oh, well there's lights in here, right? You might be able to see them kind of flashing around. Have you got people in the tunnel? How many people are in the tunnel? Brad? Can't tell by the lights. Why not? Well, there comes what looks like three, four that don't have lights on. They don't have lights on? No. So you're telling me that the lights didn't represent the number of people? No. Oh, no. So the lights are a direct or indirect measure? It can be indirect. Weird, huh? Yeah. Because they're actually on people. Yeah. So it's kind of, it's a direct measure of the number of lights turned on. But not a direct measure of the number of people in the tunnel. That's a trap. Oh, not at all. No, this is great. Thank you. You're helping students learn. Yeah, you did. Exactly. All right. There we go. The boards will be at the front gate. So yeah, direct versus indirect. Now we've got a couple more. But we don't know how many people are really back there. Again, you're all the way back to that question. Yeah. So the lights can indicate there's a person. It might not be a person. It could be a dog. It could be a dog. It could be the light just taking from the ceiling. Yeah. Swinging back and forth. You don't really know. Ah, it's getting bright. So anyway, direct versus indirect. Sometimes things that feel direct might be indirect. There's a good example. Speak cautious. That's what you measure. And what you interpret from what you measure. What would give that away? Do you think it will be upsetting? Secondary assessment. Another assessment of something. Like if you're wondering how many people are in there, we should walk in and go find out. I mean, in the Jeep video, y'all have seen the Jeep video by now, we walk in, we stuck our hands in there to find out what the real problem was. It was the mud, right? Yeah. So sometimes we have to get in there and get the data. Like, you might actually go observe. You might actually have to work with people. Oh my gosh. You might have to find the people that you work with and find out what problems they actually have. Hi. I'm Dr. Sane. Are you an IVT? Have you been one for a while? Are you experiencing an AL? Have your MOs and SDs for studying gotten mildly confusing? Are you on a mixed schedule? Have you run into a generalization gradient, flat out fell off of it, and can't find your NKAs anymore? Join us for a cram session. We'll put your chains back together.


Ports of call! There beyond the thin blue line where sky and sea meet are distant lands, strange ports, exotic people. Out there lie mystery, romance, the lure of the far away. Come, let's visit them. Join us as we head for Ports of Call. Our steamer sails south of the equator into seas lit by the Southern Cross. We skirt the west coast of Africa past the steaming jungles of the Gold Coast, past the sprawling many-mouth Congo, south and still further south to the Cape of Good Hope. The Union of South Africa is our port of call, the Cape of Good Hope, the southernmost tip of Africa. Here, east and west meet as the monsoon-driven waves of the Indian Ocean spend the last of their fury in dying struggle with the turbulent waters of the Atlantic. The Cape of Good Hope, a friendly port to centuries of treasure-seeking mariners. Today, the gateway to South Africa. Here at the Cape, the early navigators bound east for the wealth of the Indies found safe anchorage in Table Bay, and here in 1652 landed a party of colonists led by Jan van Rebek to establish the first settlement in South Africa as a supply station for the merchant ships of the Dutch East India Company. Quaint Dutch houses arose beneath the cloud-capped top of Table Mountain. Sturdy Dutch burgers planted corn and grapes and traded for cattle with the natives and the little community came to be known as the Tavern of the Seas where the weary seafarer might pause on his journey half around the world and put aboard fresh water and provisions. Here under warm southern skies, round-headed Dutch boys and apple-cheek Dutch girls danced the dances of the Netherlands while dusky bewildered hot-and-tots looked on an aboriginal amazement. In a century and a half, the broad belts and the bright sun of South Africa had wrought many changes in the Dutch settlers, broadening their shoulders, heightening their stature, developing a new race of boars or farmers. From the narrow confine of the coastline, they had trekked northward inland to the wide solitudes of the high plateaus. They had become gentlemen farmers, owners of thousands of acres and hundreds of slaves. Then in 1814, events occurred in faraway Europe which profoundly changed their lives. The Congress of Vienna which repartitioned Europe after Napoleonic Wars granted the Dutch colony at the Cape to Great Britain. Scarcely had the news reached the boars than 4,000 British colonists landed at Algoa Bay and established the town of Port Elizabeth. The natural friction between alien peoples became too much to bear when in 1834 the British government proclaimed the emancipation of all slaves in the British domain. In scores of Dutch villages, hot-headed farmers met.  They have no right to take away our slaves. They belong to us. By only last year I paid 100 pounds for a book. The English will pay us for our slaves, Peter van Rijn. So they say, but how much? They have a lot of us more than a million pounds. Yeah, and our slaves are worth 3 million. Why not free our cattle and our pigs on chickens then? Are slaves no better than a horse? In my grandfather's day they shot bushmen and hot-headed pots on site. We forgot what is the world coming to. And then they make the money for the slaves payable in London. How are we going to get the London for our money? There are the agents. The agents, the robbers. Look what happened to Jan van Schlicht. He listened to the agents. He took 100 pounds for a claim worth 500. His slaves are free now and he can get no one to work his land. In my grandfather's day they had a solution. When the Dutch East India Company got unreasonable. What was that? They didn't like things. They trekked. Yeah, and that's what we must do. Let the English have their precious Cape Colony. North of us Burgers is a continent. Thousands of mergers of good land. It belongs to no one. We can take it. Just as we took this land a hundred years ago.  What of that? There were bushmen in this land when our grandfathers came and now look at it. Villages and farms everywhere. It's becoming too crowded. Give it over to the Englanders. Africa is a big place. We'll trek to a new country where we can find freedom in our own way. Thus began the great trek.  Northward they trekked across the Orange River through mountain passes to the high belt a mile above sea level. On further across the Ball River, into the country which soon was to be known as the Tronsfall. Northwest they trekked across the Dragon Mountains, the Drakensberg, whose forbidding peaks rising seer and brown 11,000 feet above the belt barred their way into Natal. They died by the side of the road. They were massacred by treacherous Kafer tribes. But still they trekked, indomitable, spurred onward by their birthright of freedom. In the end they triumphed. And between the Orange River and the Ball, they formed the Orange Free State. And beyond the Ball River, they formed the independent Republic of the Tronsfall. And here the British, weary of trying to bend the unbendable will of these rugged Dutchmen, were content to allow them to remain. Here perhaps they would have remained in independence. Had not a young boar child named Erasmus Jacobs been playing one day on the banks of the Orange River and found a curious white pebble which he brought home to his mother. Motor, look at the pretty stone I found.  I've got to get to my churning. But look, Motor, it shines blue and orange and red in the sun. Well, well, now isn't that pretty. Good day, Mrs. Jacobs. Good day, neighbor Vanika. How do things go with you? Oh, as well as a little woman can expect. And little Erasmus, is he well? Yeah, untroublesome too, as you might expect of a glowing boy. All this running off to play with the black children and finding books and animals and bringing them home. Ah, like this stone he brought me the other day. He would be a lot more good to me if he helped me with the cattle rather than picking up some pretty stones along the river. Let me see it, please. Here you are. Hmm, it is pretty. Yeah, it's pretty. So are the sunsets, but you can't buy a new dress with them either. I'll buy this from you, Frau Jacobs. You'll buy what? This little pebble. Pied it? Oh, my friend. Keep it if you want it.  And to your shulk for nacre. Who you in there, O'Reilly? Can I be selling you some ostrich plumes I'm just after bringing from the fell? No, I don't think so. Or does the good wife need some needles and thread?  Not this time, O'Reilly. Not for money, that is. What do you mean, not for money? Well, I'll trade you something for your kettle and some ostrich plumes. And what have you to trade? A diamond. And where would you be finding a diamond in this hot land? Down by the river bank it was found. Look at it. Just see how it shines. Get along with your man. That ain't no diamond. It's a chunk of glass. No, it isn't. It's cold to the touch.  It looks like glass to me. Well, if it's glass, it's the prettiest piece of glass you ever saw. So you're out nothing to trade your kettle and a few plumes for it. Maybe. And maybe I'm being stung. But Riley had driven a sharp bargain for Erasmus Jacob's pebble was an uncut diamond weighing 21 carats and the Irish peddler received $2,500 for it. A local diamond rush began along the banks of the Orange River. But nothing worthwhile was found until two years later, when Lucky Shalkman N'Kerk traded a black boy his barn full of livestock for a large opalescent stone the native had found on the riverbank. For this stone, Shalkman N'Kerk received $55,000. Cut and polished, it became the famous star of South Africa. And it was sold to the Earl of Dudley for $120,000. The diamond rush was on. From all over the world, the fortune-seekers poured into South Africa. Camps with populations of 10,000 sprang up overnight along the banks of the Orange River. Then diamonds were discovered on a farm called Furaitsik. And with them, the most astonishing mineral discovery in the history of the world. A load of diamond-bearing rock. A diamond pipe, as it became known. Stretching down as far as man knew to the very center of the earth. This was Kimberley, the Gulkhanda of the 19th century. And a mile away was another diamond pipe, the De Beers Mine. It was in Kimberley with the first rush of prospectors that a young Englishman named Cecil Rhodes, vacationing from Oxford, staked out a claim. Well, Harry, it looks though that you'll be able to make the summer pay. Rather, this claim of yours is one of the best in the field. But it can't last, Harry. What do you mean it can't last? There seems to be diamonds just as deep as you dig. That's exactly it. Look around you. All these claims, 31 feet square with roads between them and some of them dug down 20 feet already. The roads still high up in the air, like high fences. Those roads are going to collapse. The whole thing is impractical the way it is. It's got to be consolidated. Consolidated? Exactly. If you're interested in seeing that, they're all busy grabbing to themselves. I'm looking ahead. You mean you're going to consolidate? Yes, and I want your help. Circulate among the miners. Find out the ones who are discouraged in planning to leave the diggings. Let me know and we'll buy up their claim. It'll take time, but in the end we'll control the field. Then we can operate it on a money-making basis. It's an ambitious undertaking, Cecil. Of course it is. This is an ambitious country. We're going to do things down here on a big scale. Good afternoon, Mr. Loose. Like a box of cigars today? No, thank you. I've still got some left. Well, I'll see you the next time I come along. All right, Barney. We're fellow that Barney Bernardo, Harry, looking around the mines selling cigars on that little yellow pony of his. We're perhaps, but he'll bear watching what you mean. I have heard that that little yellow pony used to belong to a diamond trader. Nobody will tell Bernardo where the diamonds are because he's a Jew. But the pony knows the route and sees him for the abandoned claims. That little Jew, cigar peddler, may have the same idea that you have, Cecil. Let him have any ideas he wants. He won't stand in my way. But seven years later, Barney Bernardo, the cigar peddler, was standing squarely in Cecil Rhodes' way. He had become the largest single stockholder in the Kimberley field while Cecil Rhodes was consolidating the De Beers field. But Rhodes would not be stopped. He summoned his shrewd lieutenant, Alfred Byte, to his office in Kimberley. He said, I want you to whisper the story around that I'm raiding Bernardo's Kimberley stock.  That's going to be an expensive procedure. No, it isn't, because you're going to sell as fast as you buy. What? What do you suppose Bernardo will do when he hears about our raid? He'll try to buy in his stock. Oh, I see. You'll be buying in the same stock. You'll have to buy any more because of the price and we'll be unable to stop buying because the price will drop and ruin him. What will he do? The only thing he can do. Sell out to us. Well, Barney, it's unfortunate that you're in such a difficult position that I can help you out if you're willing to accept $25 million for your Kimberley holdings. Look here, Rhodes, this was a deliberate trick on your part. Trick? I'm sure I don't understand, old fellow. The stock market is a most unmanageable steed and when you ride it, sometimes you ride for a fall. You know that I'm against the wall. Well, I suspect that you are in need of assistance. I'm forced to accept your offer of $25 million for the mine. Exactly. And of course, you will enter the new organization in the capacity of an executive and as an important stockholder, but I will control it. Your times are stiff. $25 million is better than ruin, isn't it, Barney? Yeah, I suppose it is. It amounts to this, Barney. There isn't room in Africa for this. One of us had to absorb the other. Well, that's done. Now we can work together for the development of this great country. Let's shake on that. It's impossible to resist you, Rhodes. But although Rhodes and Bernardo made their peace to the tune of the drawing of the largest check in the history of the world and became partners, the former Jewish peddler found it impossible to accept his defeat.  Rhodes went on to greater heights. Rather comfortably situated on an income of $5 million a year, he became prime minister of the Cape Colony, dictator of the annexed territory of Matabeela land, which became known as Rhodesia and he became the most important figure in South Africa. But his ascendant star toppled as quickly as Barney Bernardo plunged into the sea and L.S. Jamison, administrator of Rhodesia, made his famous raid into the Transvaal Republic in an attempt to annex that territory for Great Britain. The raid was a failure, but the die was cast. Rhodes resigned his premiership and the Dutch republics of the Transvaal and the Orange Free State prepared for the Boer War. On the 31st of May, 1899, Sir Alfred Milner, British High Commissioner to the Cape Colony, met with President Paul Kruger of the Transvaal in an attempt to avert the impending catastrophe. President Kruger, Her Majesty's government has repeatedly pointed out to you the injustices your government inflicts upon British nationals who have vented your country since the discovery of gold on the Witwatersrand. We did not ask the outlanders to come to our republic. They are not welcome. We do not care for them to remain. But they are contributing to the wealth of your nation. Their industry in the gold mines has built your city of Johannesburg. They are pouring capital into your country for further development. Yet you place the utmost restrictions upon them, such as requiring 14 years' residence before permitting them to vote. It is the law. Then I asked you to have the law changed. And furthermore, I asked you to reform your police system so that such incidents as the shooting of my countrymen, Mr. Edgar on the nearest suspicion, can no longer occur. Sir Alfred, we did not invite your countrymen to the Transvaal. We moved to the Transvaal to escape the domination of your British. I was one of the four trackers. I walked from the Cape to Pretoria behind my father's wagon. I was present when Dingon, the Zulu, massacred my people. Sir Alfred, we have fought hard for this land. We have paid for it with our blood. We do not intend that it shall be taken away from us by the insidious immigration of Englanders. We will not yield one inch. You realize, President Kruger, that Her Majesty's government has a higher responsibility in defending the personal rights of her citizens? I do. You realize that your action might lead to war? I do, and I would welcome it if that war would gain for South Africa, complete independence and union under its own flag. The intense patriotism of the Dutch nationalists, which prevented them from budging an inch from their stand, finally resulted in an ultimatum issued on October 9th, and Great Britain was forced to go to war to protect the thousands of its citizens who were at work in the world's largest gold fields in the land. The war lasted two and a half years. It laid waste the country, ruined industry, and won for Great Britain the independent states of Orange and the Transvaal. Peace came. Men laid down their guns. But Dutch hated British and British hated Dutch. It has taken a generation of native-born South Africans to melt this hatred into a mutual understanding. Along South Africa's great Central Plateau and in the high mountains row many fantastic animals. Wildebeest and the Eland dashed fleetily across the belt to escape the leopard. Ostriches in great flocks proved to the observant traveler to hide their heads in the sand when frightened. But rather, run as fast as their long legs will carry them. Hyenas and jackals run in packs in the bushbelt and still in the Cape Colony may be seen an occasional elephant. No longer the prey of big game hunters but free to go his lumbering way protected by rigid game laws. In 1860 the sugar planters of Natal, finiting impossible to get results with caffer natives arrested and received permission to import Hindus as chief contract labor. The Hindus came and they stayed and they multiplied so that today there are nearly 200,000 of them in South Africa. Fearing the increase of the Hindus as they fear the multiplication of the Negro population the white man in South Africa grants the Asiatic no rights.  This attitude was to have grave implications for the British Empire. In 1891 there came to South Africa to practice law a young British Indian educated in London whose name was Mahandas Karamchand Gandhi. Three things happened to the young barrister Gandhi in South Africa which have cost the British Empire millions of dollars and years of worry. Upon arriving at the port of Durban the young Gandhi wearing the turban which the devout Hindi never removes in public attends a session of court to acquaint himself with the justice in South Africa. The case of Rin van Horst versus the government. Pardon me a moment council.  Yes, I beg your pardon. It is not permitted to wear a hat in this court. I beg your pardon sir. I am Mahandas Karamchand Gandhi. I am a barrister.  In my country it is customary for a barrister to plead his case in his turban. Then I shall leave the court. Proceed, council. The second insult. Barrister Gandhi buys a first class railway ticket for his trip to Pretoria for he is to handle a case. But before the train leaves the station a fellow passenger in his compartment has become outraged. Conductor! Conductor! Come here this instant I say. Right house sir. What is it sir?  I won't have it I say. I agree with you sir. Come along you. I beg your pardon? You can't ride first class. But I have a first class ticket. I don't care, you can't ride first class. The luggage vans for the likes of you. I will not ride in the luggage van. I have got a ticket and I demand that it be honored.  You demand. Nobody in New Ethan can say that to me. Get off the train. What? Get off the train. Go on.  One now. I ain't a man to be tampered with. Throw it in south. Throne off the train, Gandhi takes a coach. He is seated in the crowded smoker when a boar farmer comes in. Here you.  I beg your pardon?  I want to sit there. But I am sitting here. You can sit on the floor. Come on. Get up I say. I will not. What? Are you smart or devil? Such treatment made a profound impression upon the eager young Hindu. Formulated in his mind, his life ambition. Crystallized his whole political philosophy. This, then, is the civilization which the white man is trying to force upon us. Cruelty. Ignorance. Race hatred. Servitude. I will never stop struggling against it. I will never admit the hegemony of the European. I will battle for the rights of my people until they have been won or until in all wise God has stilled my efforts forever. As a result of the boar war, the goal for which the best people have fought, the war, the goal for which the best minds in South Africa had long been struggling was won. The Union of the two Dutch Republics of the Transvaal and the Orange Free State with Cape Colony and Natal. Hot British and Dutch heads slowly cooled off as the work of reconstruction required their attention. The most active leaders of the boar forces, Generals Hetzog and Smuts, became the outstanding leaders of the new Union government. In 1930, the Imperial Conference in London granted South Africa complete independence under the British Commonwealth of Nations. And today, with her racial differences slowly disappearing beyond the memories of the present generation, with her diamond mines, her gold mountains, her newly discovered platinum fields, her unlimited agricultural opportunities, South Africa faces a brilliant future as one of the important nations of the world. There's the warning whistle. The anchor is ready to up anchor and we must hurry aboard. Thank you, South Africa, for your hospitality. Now we know why you are so justly faint. Here we go, sailing away for another port of call where we will drop anchor next week. Next week, at this same time, our radio liner drops anchor in another of the world's famous ports of call. We hope you'll join us again when the ship sails next week.