ysbyrwagwr. teisana pab venir labi a i teisana merozu i teisana Opak Pwg i i attentive, a teisana teisana teisana e-siaidia rangālu hawkei. Rai que ida bwisi teisana merozu ida, Marcas teisana merozu i teisana, dan ida kai teisana a i teisana hulu hukumia. Teisana ma sprunau tiempu e paba leiratia tia asatia. So, hoffa yw'r gau te tach on some of this evening. Hoffa yw'r gau. Hi, thanks for having me. It's great to be here.  So, what's missing is a large country. It's a big place. I realised that I've thrown myself a hospital pass when I was asked by Bob what title I should give back in December or whenever you first approached me. And I say, oh, deep learning, let's say, what's missing. OK, that was my first mistake, because obviously I can't. If I knew what was missing, I wouldn't be missing anymore, so I've done a whole girdle thing straight off the bat. But I'm going to pick up on a couple of specific things that I feel are missing. I'm going to partially fill in one of them as just question marks to plant in your head. And I'll mention towards the end some other aspects that I think are just, I don't know, holes. They're really mostly holes in understanding in the current world of deep learning. So, what is this deep learning? Let's get right into it, eh? Something's happening in machine learning. If you can read that, it says, hype watch, big data saturating on the hype curve. And the red guy coming up there is machine learning. And I guess the question is, is this beginning of a, how big is this curve going to go? What's going to happen? Not just in hype, but in terms of application, success, stuff. Is it going to plateau or are we just in early days? And it's hard to know because there's a lot going on. This curve is, there's a lot of things behind it. There's some new insights, some wonderful ideas. And then at the same time, there's applications to those ideas that are very worthwhile doing, and therefore big corporates jumping into the space that used to be occupied by universities. And huge resources pouring in, and then more ideas flowing from that. So we're in that exponential phase there, and who knows where we're going. But I think part of the excitement is driven by, let's get rid of that hand, by recently, I'm talking to the last just couple of years, machines achieving near human performance on certain selected tasks, right? And those tasks being spread across the spectrum from image recognition to speech recognition, text translation, gameplay, dot dot dot, you'll have your own things you've read about in the press. And so these are exciting, and they're also slightly frightening. I love this sentence from the listener in April. I read it out, it's about to get worse. The job-sucking behemoth of artificial intelligence stalks the horizon. And I just think it's beautiful, you know. Stalking the horizon, this silhouette of the job-sucking artificial intelligence. And so that's scary, and there are aspects that we should be thinking about. I think that some things are a little scary. There's also, of course, tons of promise. There's enough promise that lots of academics want to jump in on the game. In particular, this phenomenon of deep learning, which is the subject of my talk, is the part of machine learning that's actually driving the red curve up, the hype curve. All these achievements that are close to human-level performance are deep learners. And everyone's saying, oh, yeah, of course. I've been doing deep learning forever. I don't know if you have colleagues like that, but... OK, so let's just get into this. What's a neural net? So deep learners, what it means is big neural nets.  But here's a neural net.  So that forms a vector. Let's turn them all on. A vector of input values. And some target output. In this case, there are two outputs. And in between, we're going to have a bunch of neurons. What's a neuron? That's a little unit that takes a weighted sum of its predecessors in the previous layer, computes the weighted sum, and then pushes that through a very simple nonlinearity. So if you like, let's make this thing go. Let's give it a problem, a hard problem. Make it go. OK, so what you see on the right there is that little diagram there, if you can see it. That's the loss. That's the error of the network as it attempts to try to get to grips with this learning problem I've given it, which is actually a pretty tough one, to learn to classify the data points on the spiral. All right, so what's happening here is there's a learning algorithm which is changing the connections between all these neurons. OK, that's all that's happening. The function, the nonlinearity is staying the same. The inputs are coming from some training set of examples and outputs, an output pattern that goes with each input pattern. And it's simply learning and mapping from the input to the output. OK, with some success, it's still going. OK. Fine. So that's what a neural network is. I should say there's another part of this red curve rising as a bunch of new fantastic tech being built, including TensorFlow, that demonstration there. But here's my more reduced picture. This is my shorthand for that last picture of what a neural network looks like that I'll use for other bits of the talk. So we have an input. That's a vector, that blue line, and an output, another blue line vector. I hope you can see that. Apparently we can't dim the lights to half-way. It doesn't work like that. But anyway, so input through to output and it's going through each of these parallel... Well, what is that? Trapezoid? Something four-sided thing is representing a weights matrix. So all the weights from one layer to the next. And then the sigma in a circle is representing that nonlinearity. So we have the weights matrix happening and then element-wise, nonlinearity happening. So all the neural network is.  It's deep if you make more than three layers. You're allowed to say deep, I think. In the old days you could say that. And now we can train things that are hundreds of layers deep. Okay, so how do we train it? We look at the error. So each input pattern has a desired output. You stick in the input, something else comes out, not the desired output. So there's an error. We can compute the gradient of the difference between them. A loss function. Compete the gradient of that and push that back through the network. And that's basically just saying, how would I like to change some parameter inside this matrix? Some weight value. How would I like to change it such that the performance on the training set has fewer errors? Error back propagation is often called. It's an older algorithm from the 80s. Okay, so that's a neural net. Game over, really? Okay, so here's an example of a neural net. A reasonably deep one. It's take the input. This blue vector here is an image. That could be just a huge vector, right? It's pretty big. Hundreds and hundreds of elements. But an image is a vector. It's going through this neural net. Don't worry about the details here. And out is coming out the far side at the output. In this case, labels, words, like car, truck, airplane, ship, horse. And in this case, you know, thankfully, it's figured out that this thing is more likely to be a car than a horse. So full marks. And it did that by training on probably millions of pictures of cars. All the weights in this big neural net have been trained by this algorithm. Okay, that's deep learning. Awesome. That wasn't too hard, was it? I should tell you about a particular kind of neural network which is an auto-encoder which is being used in very interesting ways these days. So an auto-encoder, what you do is you take the input and you push it through the neural net and you train it to reproduce the input at the output. So the target is just the input. It's called an auto-encoder. Why on earth would we want to do this? The identity map does this. It's not that hard to get the input from the input. But if we push it through a neural net and there's a layer in here that's very narrow, a little blue dot in the middle there. Sorry, I'm being very sighted. Rightest here. So if we go through that bottleneck layer, then what's the neural network trying to do? It has to push this picture of a car through the network and recover car picture at the far side. But along the way it's got to go through a really skinny layer in the middle. Imagine that was only two units just to be really nasty. That means it's got to push it down to a two-dimensional vector which represents that car. It's obviously pretty tricky, right? But perhaps you can imagine that those two dimensions are going to be pretty interesting features of car of the training set if you can achieve it. It's hard to achieve, but if you can do it, it's pretty interesting. So here's just another picture of the same sort of thing. How would you make use of this? Here's an example where this is a different kind of picture. Each dot here on the left is a data point. I'm just schematically representing a vector picture of a car. They're being pushed through this network just like before and out is coming an output and I'm learning to match that output to the input. The output is going to be really a distribution. And along the way it's being forced through right down into just two dimensions, say a very low-dimensional version. So each dot here actually has its own little dot down in the two-dimensional space, right? And then it recovers back and you get the car back. So each dot, the precise position of the dot, means a lot. Okay, so now I can imagine I can just pick a dot here and I can make a car picture and I could move on a line through that sort of central through this compressed representation. As I move along through that line what do I get? I get different pictures of cars and they'll change probably in some discernible way. So let's forget cars and do it with faces instead. This is my colleague Tom White from Victoria's Design School actually. Let's see if I can do this. Good, okay. So I'll just go straight to the horse's mouth. He's looking at... Oh, let's look at the earlier one first. So this is almost literally that two-dimensional picture of little dots. And as we move, you pick a dot in a branch all the time. Constantly rescaling and shifting them to keep them in the right ballpark. It kind of makes sense, right? Took a long time for someone to make it really work, but it makes intuitive sense. Those all help. They really solve that problem. Okay. So, here's the thing. I reckon there's another fundamental problem. And this is work with David Balduzzi who's really the lead creative on all this. There's another fundamental problem to learning in deep nets. It's not the exploding or vanishing that we call it shattering. David's term. So we've just written a paper called Shattered Grade Against Problem. The title says it all. If Resnets are the answer, watch the question. I'll tell you about Resnets in just a second. So, here we go. Resnets. So residual nets are, as of last year, the king. They are the state of the art in classifying images, things in images, for example. Very, very effective. And they allow you to learn very, very deep nets. Hundreds and hundreds of layers deep. This is great. What's the trick? We replace, instead of just having this weight-feed-forward connection through the weights matrices, you insert these little hop-over direct connections which are just the identity map. And why would that make any difference? Well, if you drew it differently, so you've just got basically the identity map as the short route going through, and these are all hopping out to do a bit more work. So you can imagine it's a bit like replacing a, well it is actually replacing the old feed-forward function with X, the input, minus the new feed-forward function. So the nonlinearities in the weights matrices are working on the residual. They're fixing up their errors. But there's this backbone of very direct connections. Okay, so for whatever reason, this is a really good idea. Lots of varieties of it, but basically let's call them residual nets or res nets. They work really well. Okay, so it was originally said that they were helping with this explosion or vanishing gradient problem. But if that's true, why are they working so well now in these deep nets? Because really that problem was solved by these other techniques. So they're doing something else, these skip connections. They're solving some other problem. All right. What do you think it could be? Maybe it pays to just have a look at some gradients. So here's a really simple neural net. It's got two layers. And this is what I'm doing is plotting. Ah, it's a really simple net. So it has one input. It's a scalar input. It has a every hidden layer has 100 hidden nodes in it. And they've all got these rectified linear relues in them. And then it ends up, the output is just a single scalar as well. So a scalar to a scalar, x to y, which is great because I can just plot it. And we're doing backpropagation on this network. So here's a picture of the gradient as I move along the x. I'm being rightist again. As I move the input value from 0 to 1 say how does the gradient change? The gradient I would calculate at the first layer. And it steps along. It's a Manhattan skyline. It's got flat bits. That's because of these value functions. But let's look at the overall shape. It jumps around. It's sort of a random walk, right? Below it is a random walk. Otherwise known as brown noise. A random walk is someone staggering down the road, right? Left, right. Oh, God. So one of the things we've shown is that in the limit of an infinite number of hidden units, this is a random walk. These are the same distribution. Okay, that's two layers. Here's what 20 layers looked like with this batch normalisation thing which is solving a whole exploding gradient problem. This is 20 layers neural net. That's white noise. Random white, totally uncorrelated white noise. Okay, so this is that's what we mean by shattered. The gradient is being shattered into tiny, tiny, tiny pieces that are completely unrelated to their neighbours. And you can start to imagine what they would do to a learner. How can you learn when the gradient information, which is the learning signal, looks like that. You can't. Here's what res net looks like. It's somewhere in between. It's not brown noise and it's not white noise. It's somewhere in between. It's got local structure but it's not it's not just a straight line either. All right. So here's our idea. Here's the intriguing facts actually. The gradients of a deep feed forward net look like white noise. Wow. It's only 20 layers. Imagine when you get to 200. And res net, don't shatter. The gradient does not get shattered. All right. So let's look at some more pictures. This is what I'm looking at is as I go left to right along one of these lines, the black and white lines I'm just saying is this neuron, here's a neuron in a hidden layer, is it's activation on I mean is it outputting a real value other than zero or is it outputting zero. I know that's what black and white means. I guess just don't worry about the coloured ones. Look at these hashy pictures there. Okay. So they all look the same at the two layer level. Fine. You go to 10 layers. Things are starting to change. Let's go all the way to the 50 layers. At 50 layers what's happening? We should be able to understand this. The the feed plan feed forward network pretty much all the neurons are either on all time or off all the time. That's what actually what the histograms are showing in the blue there. Neurons are either active or they're totally inactive. In other words they're just being linear, which is not very helpful, or they're being they're dead, they're outputting zero. Neither of these things is contributing any computation to the mapping that we're trying to learn. So that's no good. A feed forward network with batch normalisation the shattering has broken up the activity. So every neuron is pretty much uncorrelated to the different points of the input. So I have to vary the input. Any one of these neurons is flicking on and off all the time pretty much randomly. It's not random, it's deterministic, but it looks random. And there isn't it somewhere in between. So that's nice. We can prove theorems. So we can talk about the covariance as we move along in the x-direction watch the covariance in the gradients actually at different points. And you can prove that it dies away one over two to the power of the number of layers. So it's exponentially dying away. The covariance is zero means there's no signal left. You have totally white noise. No connection between neighbouring points in the input space basically as far as the learner is concerned. It makes the problem unlinnable. Resnets go the other way. They actually rise, but it's only a slow rise, so the square of the number of layers. Really different characteristics. So what's going on? Shattering is occurring as a feature of deep learning. It's a really important thing. It actually stops you learning things. It's solved. One way to solve it is to have these skip connections. So that's really what the skip connections are doing. Okay. So with this insight, it was a nice payoff immediately to come up with a new initialisation. Understanding something provokes further ideas. So we came up with a new initialisation. This is a student Linux leery which enables us to train really deep links of up to 200 layers and hundreds of hidden units per layer. So really deep. Without any skip connections at all. So that's the first time anyone's trained a network that deep without using skip connections. Skip connections are essentially short. Now I'll reduce the depth. Okay. So I like this. We've attracted that Reddit page, which is awesome. And one of the comments on that sums it up really nicely. So we really don't understand how to train neural nets, do we? It's such a basic thing. Fundamental aspect. It's just coming to light really. Okay. I want to switch tack to the other flavour of neural nets. So we've been talking about deep neural nets as feed forward enterprises. They go from input to output. People are really interested in modelling structure and time and modelling sequences. And you can do that with neural nets that are very similar with one difference. So the idea is we allow the hidden layer to feedback on itself. It makes a first-order Markov system out of the device, if you like. So now what I'm saying is the hidden layer we're still mapping inputs to outputs but the representation of the middle, these hidden layers I should say more than one but the hidden layers are receiving two kinds of input. They receive input from the actual pattern that's coming along at this moment in time but they also receive input which is a copy of themselves a moment ago. So in that way they're able to capture structure in time. In principle they could learn to use their weights to model temporal processes. It's not a done deal that this is going to work but that's the idea. Sort of minimal model of a system that could model sequences. So you can think of it like that or you can unroll it over time. I think this is a good way to think of it. So that's just the same picture unrolled over time and you can see that that is like a deep net. If I try to propagate it's gone wrong here and propagate that all the way back it's got to come back to change that weights matrix. It's got to propagate through a whole lot of weights matrices. It's a deep learning problem. So we're only unrolling here. These weights are all copies. The A matrix or whatever is going on in the A box is the same at all time. Alright So it has been pointed out that this really generalises the family of what sorts of things you can learn. So here's the old input goes to output. I'm sorry it's back the other way around now. Input at the top, output at the bottom. That's one to one standard old feed forward network if you like but with this temporal process each one's unrolled here we can imagine a one to many mapping that's like I say perhaps a signal a word making a sequence here's the opposite you could take a sequence and map it to a single value such as classifying a sequence who is the speaker or what are they saying many to many is also possible so this is what's going on in Google Translate which is taking in a sequence and then having had the whole sequence it's now got to generate a new sequence and the most general one I guess is many to many so a streaming input coming along that's the red squares all through time and we're having to produce outputs all through time as well alright so lots of possibilities lots of scope for fun here's one rendition picture of the fundamental element inside one of these recurrent nets so what we're taking is it looks more complicated than it is probably all we do is we take the input and actually this think of two paths through this so one is that the input is going to the output and on this picture it's going through two layers of neural net so that's a two layer neural net going from input to output and this is going to be replicated over time there's a whole bunch of these strung together if you like so the other path going through here is left to right and that's the hidden state which we might as well call a memory now because it's being propagated over time and the memory is able to pass from memory at time t to memory at time t plus one being changed along the way and it's changed in the light of the current input so you take the memory you take the input that changes the memory and here's the memory influencing how input maps to output it's got the bare minimum okay this has a bunch of problems so it seems like this is a sensible way to start unfortunately it doesn't really work very well one problem is it has this exploding vanishing gradients problem but there's more to it than that very early on in the 1990s an alternative architecture was proposed which is called the long short term memory or LSTM and here's a picture of it and it tells me how it works the interesting thing is it's pretty much inscrutable but incredibly successful so this is my second missing why is this thing working so well so LSTM long short term memory looks like this I'm not even going to try and tell the story but if you spend enough time with someone who really knows their stuff you can start to believe bits of this thing make sense but it's hard to that's not the same as saying it's the answer or I don't know there's something really weird about this that it's so successful it's been such a long time in 1997 so it's 20 years old incredible this thing is still the state of the art in recurrent neural nets varieties of it, slight tweaks admittedly but basically this idea so Google's translation engine runs with a whole bunch of these state of the art speech recognition all kinds of cool tech that it's either on your browser or on your phone or it's coming soon built out of these things and let's just look at some examples so here's a language model I feed one of these recurrent neural nets everything Shakespeare ever wrote and then so it takes, what it does is let's just jump back that picture back there okay I'm feeding it individual characters in this scenario more like that one individual characters you know C, A, T that's not a good example B, A, S and then I'm trying to predict the network okay it has a little window it's just got a sliding window and it predicts the next letter fine that's a predictor and then just for kicks I'm running that model and when it says K for the new prediction I'm just writing K in so now it's able to just write text forever okay so this is making up Shakespeare and if you have a look at it I don't know it's not that bad it's a bit hard to see the bigger picture but it's pretty good I mean it's making words it's making spaces grammatical sentences and look it has the feel right lots of fun this is trained on a few hundred essays by someone called Paul Graham founder of Yahoo Store you can tell just by looking at the fake text coming out of this generator the sorts of things that Paul Graham writes huge fun oh I if someone wants to ask me a question later I can show you some fake maths it's really fantastic so they got the whole latech of a big book and trained it up and then they ran the model they got a whole bunch of latech that actually compiled which means you can make a PDF out of it and look at the proofs they're junk proofs and you've probably seen Deep Drump this is a twitter account trained on all Donald Trump's tweets and it just generates tweets all day long you know stuff like this but they're pretty good actually fantastic okay alright what other things can we do heaps that's just the start that's just language stuff you can now take an image and push it to a sentence so instead of taking this image we'll take that representation the low level representation of the image and ask one of these recurrent neural nets to make plausible sentences from it incredibly this works you wouldn't think it would work but it works pretty well it's a man in a black shirt playing guitar it's true so these are slightly cherry picked examples but it's pretty effective and getting better all the time so it's taking the whole system end to end it's taking a raw picture raw image and it's generating the text at the far end and it's all just neural nets all the way along trained from lots of examples there's nothing else in there okay oh sentence to sentence this is just a picture to remind myself to say that the what's inside google's translate technology now is just a market cool multi-layered LSTM network that's what's inside it up until last year they used to have all kinds of clever linguistic machinery in there developed painstakingly over years and years and years huge thing and then last year they swapped it out because this worked better and this is just this is just one end to end thing it's just a whole bunch of those recurrent neural nets alright the one on the bottom right is oh taking a sentence and making an image out of it it is extraordinary that this works early days as well but if you can see them this has seen a lot of pictures of birds okay and then it's been given a sentence this small bird has a pink breast and crown and black primaries and secondaries and it makes some images how long has it been doing this? how long? this is last year I think it's either early to see or last year yeah okay so text to handwriting let me show you this this is fun this is someone needs to give me a sentence okay so it's taking text this was really came from someone trying to build a model that should recognise handwriting but in reverse I'll roll that down no can you see at the bottom? so someone needs to give me a sentence different samples you can mess with them all you like have a go a lot of fun so that's actually generated so the text comes in and then a recurrent neural net is actually directing the way this is actually done I believe is it's directing the drawing motion image with that sentence from scratch is actually controlling if you like a virtual hand alright oh and here's someone this is just the other day last week I think this guy's okay that's not great same idea as Shakespeare so you feel then it's going to predict something from the last few frames and we're just going to take that frame and go with it you can tell how this was trained it's trained on a train it must be completely new countryside it's not perfect but it's pretty amazing let's go to the town how was it kicked off I'm not sure for the Shakespeare one you can kick it off with a blank space and it just says okay so I'm not sure maybe it's kicked off with a picture but maybe it's just starts making stuff up I don't know okay where's that going like I say we're going to start mistrusting our eyes the world's going to get interesting so here's something that's missing I feel this is missing at least what's so great about LSTM they're almost impenetrable and that it's very hard to tell a story in a pub about how the LSTM works it's a sort of three-point explanation it's it's really hard you can pick out bits well this is kind of doing but there's not that very simple story that I was trying to tell about the basic current network there's two things going on inputs go to outputs through a neural net but they're mediated by the memory and memories go to new memories mediated by the latest input okay hard to tell that story so you have to wonder how special is the LSTM how unique is it there's an interesting story here that two teams ran very large trials in alternative architecture so they used genetic programming which is random search to try out all kinds of variations so this LSTM I've drawn it as a diagram but it's really just five equations so you can make up equations randomly you can try them out on a big machine they trained it on the Googleplex for the equivalent of I think about a century of CPU time equivalent and and found the LSTM and a few other things GRU if you know this stuff is the other alternative but not much right so basically this is a some kind of special spot it seems in the space that they were searching at least okay so one idea is to do genetic programming for 100 years of CPU time and another idea is just to think about it we've been trying to do that and this has worked with Paul Matthews, a wonderful honours student I'd ever well do as he can't get away from him again this is last year and this year we're pushing on with it so essentially we're trying to think of what's a minimalist model that's better than the old restricted recurrent neural net but it's understandable unlike the LSTM so and we're going tenses on this so there's a mapping from input the same story right so the inputs map straight to outputs in fact this is just an out of product so basically inputs are going to outputs through a one layer neural net it's just that we do want that to be mediated by what's in the memory so we involve the memory by taking out a product but this is another way to say this is a tensor mapping a three-way tensor takes the input and the memory in gives you a new output otherwise known as a bilinear form whereas a straight matrix is just a linear form so that's that story and memory is going to new memory how's that happening? well it's just going straight through really this is a little switch that says you either send this information straight through or maybe you take the other path and you take something from the input and this is an either or thing it's not binary but it's in zero to one a mixing between what's in the memory and what is coming directly from input it's as simple as that so how's this decision made about which one you want to keep? this is like a bit like a read write head right? you can think of it like in that simple way that this is saying should I keep what's there or should I overwrite it with a new with a new bit how do you decide what to overwrite which things should be overwritten or not we figure that's a job for both the current input and for the memory state so it has to be a combination of both let's do the same thing so take a tensor product of the current memory in the input and put it through a non linearity and that's your switch so that's a longer story but that's a story this thing has some nice properties so the gradients don't explode or vanish because basically the memory is propagating straight through here and we're just swapping stuff out or into it has the effect that the gradients don't explode we're avoiding this concatenation of input and hidden states which always happen in the other models you sort of push inputs and hidden together into one space where they don't really want to be in the same space they're different entities it's almost like a type error to do that so we avoid that and we use a tensor instead of concatenation you know it's a machine learning talk when everyone's frowning and there's a graph like this I guess there's a lot of talks to have graphs like this but machine learning you've always got one so there's the loss function training time going left to right the training loss that's the amount of error the amount of badness badness going away and as with most talks the badness goes away on the stuff we did and stays high on the stuff other people did so the top two lines are the LSTM actually and the GRU which was one of those other random ones found that it was as good as the LSTM so ask me about this task if you want to know later but I'll there are tasks very straightforward simple memory type tasks for which this new architecture seems pretty promising it's early days we're still chasing this around but there are tasks here's an interesting task which we also happen to do well on it's the task of well okay so here's the input pattern vertically happening over time okay what happens is pay attention to that that very last couple of rows if when this neuron comes on I want you to I want the network to look at these other bits so these ones over here and remember them and to spit them back on the output when that neuron goes off again so the last of the inputs is a sort of trigger and it says when it comes on remember something and here we're doing that with three of them right there's three memories going in store that store that store the third thing which overlaps actually and then cough it up at the end and it's not perfect this is our model trained on gazillions of these examples but it's having a good go so the first pattern comes out pretty perfectly second one's close everyone's actually pretty good not quite there yet so work to do but that's a that's to be compared with the best this is the very best of the competition where we try to store three memories we get three things out and they come out at the right times when these neurons here turn off you get a pattern back but it's the same pattern so this is the LSTM the state of the art inside in your phone doing all these awesome things it can't do this kind of task it's just a conceptual there's something it finds this very hard to do we would argue our architecture finds this really natural thing to do and it's very like variable binding so the trigger comes along you remember some arbitrary thing hang on to it and then when I release the trigger you've got to cough it up at the output so I guess what's particularly appealing about it is that it feels like the foundation for other processes any processes that are capable to bind one pattern to another and use like a key and then manipulate the key perhaps in some fashion and then get back the original bound item I didn't say that well but I guess we're optimistic and feel that this is the beginning something interesting bridging into a more symbolic realm for neural nets but it's still your standard neural net it's just got magic tensors in it it's just an even tougher task when this bit comes on you have to remember the bit pattern that immediately follows it on the other nodes and then when the neuron goes off you have to cough it up again same task but now there's all this distractor stuff in the middle that's what the hidden representation looks like and that's the output after training so if you're clever you can see that pattern there is the same as that pattern there I think one, two, three oh yeah it is great so it's learned to do that we never told it we just trained it and gave it lots of examples in which that's the pattern and this is a new example alright now quickly some more missing things we've talked about why one thing I said was missing was why the state of the art in feed forward nets is actually the state of the art why is it res nets what are they actually doing I argued that it's that they're solving the shattered gradient problem we said there is a shattered gradient problem and they're solving it and then I talked about why this missing explanation where it's LSTM I don't have an answer because we haven't found the better thing and pointed to direction for thinking about the better thing I think thinking about the problem and trying to gain insight building something that's got that is interpretable is a better way to go than randomly searching here's some other questions to throw out there missing things that I'm not going to answer they're very much missing why are deep nets better than shallow ones this is still up for grabs actually so there's a proof just this year I think it was last year but that relates actually to two versus three layers it doesn't generalise so that's why this architecture at all why neural nets at all what's so special about going mushed through a weights matrix why is non linearity and then back into a way just doing that and iterating that why does that work so well why not something else I think there's a coming something that is coming is various ways of making forays into more symbolic style processing but within this connectionist within this neural net world and hinted at that our tensor gate idea might be one direction to take lots of other people are doing things that are to do with for example having external memory having a neural net being able to read and write from external memory this sort of thing having neural nets able to implement programs that's a hot area lots of fun and my pet passion I guess is trying to understand generative models for really complex multi-causal data so you imagine you have some data my go to example is a face what you're seeing the pixels that are appearing on your retina are a combination of two completely independent things one is the physical structure on my face we're 3D and the other is the lighting environment and those two things are completely independent in fact but they interact and you get to see the interactions you never actually get to see a naked lighting environment you only see lighting environments when they hit things and you never get to see directly apprehend 3D structure you always have it in the context of a lighting environment or some other environment so there are there are pure things out there in the world but they get mixed and then we see the mixtures only can we build a system which is able to go the other way and having seen a lot of data disentangle that to discover the underlying causes that's a big tough problem and unsolved many many more things to come there's a lot missing missing is a huge country thank you very much thank you for your questions Kent Williams spoke about how to create a mind you agree with his sorry who's he's written a book last year it's called how to create a mind he discusses the nuance in the brain and that there's 12 layers etc I haven't read the book so I don't think I can comment on that interesting today the announcement at the Google conference today was that the future is to do with teams ooh no that's alright someone will get there I mean I do have a for me I have a skepticism about people the idea that you can study brains and that will if we study individual brains enough we'll understand entirely how they work I think that has to be coupled with computational standard thinking how do you achieve this kind of computation people have been it's a bit like the analogy with flights that you understand flights yeah you look at birds and that can help but then eventually you have to do some aerodynamics sorry yes you said that it's a good thing that you don't concatenate the inputs and the previous that you said can you elaborate this ok so I would direct you to David Balduzzi's paper called strongly typed neural nets is the key phrase so his point is when you when I map a hidden so there's a mapping from the hidden state to the new hidden state that's being achieved by this weights matrix so what's the weights matrix doing actually it's it's rotating and skewing directions right that's what it's doing there's a sort of native there's like a basis associated with that transformation and the basis for that is a different basis from that associated with the input pattern the interaction between the input and the hidden layer it's a separate it's a separate basis there in there in it's why would you expect the hidden units dynamics to be naturally mapped on to to be naturally altered by additive changes being coming in from the input it's a very limiting way to do things and have a look at David's paper about strong typing I think a lot and you can explain it yes this is the sentences to images the images are so there's a good in theory they're supposed to be brand new because they are projecting from this space of the sentence down to a much more modest latent space low dimensional space and from that space generating the image so in theory there are new images in practice have a read of the paper they're struggling so this is last year and it's actually look for GANs for the real networks another kind of generative model they they do seem to suffer from a problem it's called mode collapse where the generative model learns to only produce a very narrow range of what do we have, birds kind of all rather similar so that was a problem it seems to have been solved to some extent recently but it's not convincing demonstrations of that soon but no, the whole idea is that they're not supposed to be just stuff you've seen after all that would be kind of boring so most of these papers what they do and they should do is test so they're making up generating new things they should look at those new things thank you mmm hi sorry, Rishon let's go with you okay fine, sorry about my insufficiency last night the okay, I've studied CGS a couple of years ago we'll use launch budget okay, just a weird question that came up in the discussion with A that we used from medical background a few months ago is it ignore any ethical issues is it within technology to cut through the optic nerve of an animal and attach it to some device that can get the signal out of it ah, not my field of expertise I would say that can you chop the eyes or something and stick it onto the brain or something else not currently doable and not something I'm an expert on what was the question here admittedly a mechanism hmm I know it's annoying isn't it it's hugely annoying that there's all this progress and cool stuff and it's all built with that propagation and that propagation is biologically implausible so that's a damn shame for the idea that you might think about computation and make your way back towards brains and understand ourselves better bit of a blow various people have tried to think of alternatives which generally the efficiency goes down I think it's still a live question to what all this stuff is working very nicely it brings the pressure on so how's it done by real neurons then it's still an open question yes they understand that you have inputs you have a set of weights which you can tweak and tweak the weights so as to get the outputs to match some of the desired output yes what is the point of dividing the weights out of the structure of these weights dividing it out from layers but what do you get I think we're back there I think the same thing I don't know you can tell a story about very shallow nets like that so imagine a single layer it's not going to be up to much because mathematically what each neuron is doing is drawing a hyperplane in the input space it's going to be on for this side and off for that side and hyperplanes can't do much that's very interesting and then hyperplanes into that space so a two-layer architecture can make shapes that are much more interesting than hyperplanes there's a proof that if you make a second layer if you make just a two-layer architecture wide enough enough hidden nodes that are very wide it's not well understood what extra depth adds and I guess this how important is it to have it in layers probably not that important if I understand the question these res nets are really a break with that strict layer-wise thinking they have these skip connections and you can think of multiple paths through the network through many layers some of which jump over whole layers it's because a big complex image face has got to have eyes and eyes and mouth they have separate images and then they probably put together yes, well that's a good argument for visual examples at least last question I look at the sequence of slides and there's a kind of friend or something that I see I'm going to make a statement and you can tell me what you think I look at this and I see that you have this deep neural net and you've applied it to a variety of problems but the cases where it doesn't do like your Shakespeare example or your other example are things where it has to somehow represent some people calling it some sort of knowledge Shakespeare is not just about words it's cool to see it's pure Shakespeare in words but it's not making sense even for Trump because simply because this neural net deep net is not well suited for doing some kind of deep cognitive skills like humans are very good at this is just being connectionless stuff it is not sort of capturing say like you know how many reasons at multiple levels at multiple steps and stuff like that which is why in certain kinds of problems it doesn't work okay and what's the question I'm going to agree with you so you're pointing out this so deep nets they're for the language example the deep jump I would say that there is messing in the nearly silly beyond assume that it would do well because that's a wrong task to apply a deep network for okay yeah it's a demonstration of that the neural net correlation structure in temporal data well it's hard to say probably not worth exploring it's I would say it's a step towards that so there are things missing right for example there are symbolic style manipulations that humans achieve somehow but we don't achieve them we do it with the CPU and the register we do it with neurons and so I'd say these are not there yet there are lots to be figured out it's not just make it big enough train it on enough data and it'll become conscious and actually make sense as you point out the Shakespeare didn't make a lot of sense but I don't think that's the point sorry sorry I didn't catch the end of that perhaps yeah so currently with a memory in these neural nets is the knowledge is all being stored into the weight values and I didn't talk about it but there's a lot of interest in systems where the knowledge is they're also able to read and write from some kind of notion of an external memory an associative memory it's a hot topic it's yielding interesting things okay I think I'll draw up to a close now Marcus can probably still answer a few questions after we thank him so thank you very much for welcoming us to your country it does look very big it does seem like it's so much missing but it certainly shed light on some things that are happening in the area so thank you very much thank you we've just got one announcement very brief thing I'm on the committee of the Auckland branch of IT professionals New Zealand ITP is a co-sponsor of the Gibbons lecture series so thank you Associate Professor Freeman for your talk tonight quickly mention that there's two events coming up next week that you can register for on the ITP website one is of course the fourth Gibbons lecture on the Ethics the Ethics of AI so that's next Thursday and next Tuesday we are having the ITP event which is also free down at the Onwindyo Wharf next Tuesday so if you're interested in that it's a discussion on the future of money and we've got representatives from the banks zero event talking about issues including AI and how AI isn't affecting and then going to impact the financial system so you can register for that also on the ITP website


I welcome you all to this session of fluid machines. Today, we will start the discussion on reaction type hydraulic machines. The first introductory concept about the reaction types is like that as you know earlier that a reaction machine is a type of machines where both the change in velocity and pressure takes place while the fluid flows through the rotor. That means, the basic difference is that for an impulse machine the fluid the pressure of the fluid remains almost remains constant in the rotor of the machine. The fluid the energy of the fluid at the entrance of a fluid machine can be thought of to be comprised the kinetic energy and the pressure energy. That means, the fluid at the entrance to a machine comprises both kinetic energy and pressure energy. So, what happens in an impulse machine that the fixed part of the machine known as rotor which in case of a pelton is the nozzle that expands the fluid from this higher pressure to the ambient pressure. That means, the fluid coming out from the stator of the machine is in the form of a high velocity jet that means, it is fully in the form of kinetic energy. So, pressure energy is totally expanded the pressure energy is totally exploited in the form of kinetic energy. So, therefore, in the rotor it is only the change in the absolute velocity of the fluid or change in the kinetic energy of the fluid which is responsible for the work done by the rotor. But, while in case of a reaction machines the fixed part of the rotor of the machine does not exploit the entire pressure energy in converting it to kinetic energy. That means, a part of the pressure energy is being converted into kinetic energy. So, therefore, fluid approaching the rotor blades of the moving part of the turbine has both the pressure energy and the kinetic energy. So, therefore, fluid flowing through the rotor blades suffer a change in both its kinetic energy because of the change in its absolute velocity and also in pressure energy. So, how the pressure energy is changed then when fluid flows through the rotor passage then the flow area is changed and accordingly the relative velocity of the fluid with respect to the rotor changes or the pressure of the fluid changes. So, therefore, simultaneously velocity and pressure of the fluid changes in a reaction machine then the rotor of a reaction machine. So, that is the basic difference between a an impulse machine and a reaction machine of any kind. Now, the most popular type of reaction machines in the field of hydraulic machine is known as francis turbine which was first developed by an american engineer J. Francis this is the francis turbine. So, the first hydraulic turbine reaction type hydraulic turbine was developed by this is the name of J. Francis an american engineer who developed this known as francis turbine. It is a typically a radial flow turbine as you know that in case of a radial flow machine when it is a turbine that means it develops work then it has to be an inward radial flow. That means the inlet to the turbine should be at a higher radial location from the axis of rotation as compared to its outlet. This is because that the dynamic centrifugal head is released by the fluid. That means as the fluid passes through the rotor blades a decrease in its centrifugal head occurs which can be utilized in the form of work developed by the turbine. So, therefore, in case of turbine the radial flow machines will be inward flow type that means the inlet to the rotor will be at a higher radial location from the axis of rotation. So, here also it is a radial inward flow hydraulic machines. Now, the main components let us first see that this is the entrance this is the main entrance of the fluid which is having both the velocity and pressure high velocity and pressure. That means the energy at here which the fluid possesses is in the form of both kinetic energy and pressure energy. Then it flows through a typical space like that or typical projection like that where in the direction of flow the flow area decreases and this casing is known as this is a spiral type casing spiral type casing. This is a spiral type casing this is the first component of the machine spiral type casing which is known as crawl scro double lens crawl casing or volute can you see this volute casing volute volute casing. So, a spiral type casing known as crawl casing or volute casing why this spiral type because its area changes in such a way looks like a spiral as the liquid flows through it. Then there are certain veins or blades which are fixed and as the liquid flows through the spiral casing crawl casing or volute casing the liquid enters into the passages formed by this fixed blades. So, these blades are known as so these blades are known as these are the fixed blades fixed blades or veins blades or veins and this is the stator of the machine and this is the stator of the machine that means the fixed part. These are known as guide veins these are the terminology guide veins or wicket gates this is a peculiar terminology or typical terminology wicket gates in hydraulic machines wicket gates. I will come afterwards the function of each and every component before that let me tell you that what are the components these are fixed blades these are pivoted at those points. So, that these are they are usually fixed for a particular operating conditions, but they can be moved they can be swivel about this pivotal point to regulate the flow through these guide veins in case of governing of turbines which will we will come afterwards. Then the last next component is a series of moving blades. So, these blade is moving this is moving this is the shaft where it is mounted the rotor. So, this is the rotor disc and on the periphery of the disc you see the number of veins are attached. So, these are known as moving or this is the moving veins moving veins which moves that means these veins are mounted on the periphery of a disc which is coupled to the shaft which is rotating. So, therefore, these veins rotate these have got an angular velocity. So, these are moving veins are known as runner. So, the moving veins that is the rotor the rotor part of the machine is known as runner. So, machine is composed of now we will come afterwards let us see moving veins and runners. Now, if you see the flow of liquid you see the liquid comes here at this point this is the entrance as the liquid flow through the guide through the sorry scroll case or volute case while liquid comes here the liquid goes on entering through the passage of the guide veins. Now, you see this area is decreased because to make an equal amount of fluid to flow through the passages of the guide veins as you see as the fluid flows through the passage of the guide veins. So, therefore, some fluid flow in the in this direction. So, when the fluid comes in the scroll case. So, the fluid mass is reduced. So, the area is accordingly reduced. So, that the entrance velocity of the fluid in the passage of this rotor veins becomes almost same that means to maintain the uniformity in the entrance velocity to all the passages this area that means this scroll case area is reduced this is because the amount of fluid as it flows in this scroll case in this direction is getting reduced the flow rate because of the flow in the vein passages guide vein passages. Now, the guide veins are pivoted in a such a way that this area passage area that means if you consider one such passage through made found by two guide veins is such that it gives a converging area to the flow of fluid. So, that what happens when the fluid flows through this pressure of the fluid is reduced and the velocity is increased. So, therefore, when the fluid flows through the guide vein what happens the pressure of the fluid is reduced and the velocity of the fluid is increased. So, one function of the guide vein is to increase the fluid velocity some of the pressure energy is being converted to kinetic energy another function of the guide vein is to direct the fluid to the runner blade at an appropriate angle that means the angle which is equal to the angle of the runner vein at inlet. So, that a smooth shock less entry to the runner take place. So, therefore, we can isolate here the the purpose of the guide veins which I will tell afterwards. Now, let us see that after guide vein what happens the fluid strikes the runner vein and it flows through this passages found by the runner veins the fluids are channelized like that. Then ultimately flowing out from the runner vein at a at a radial location which is nearer to the axis of rotation it flows to another tube known as draft tube draft tube. Now, we should conceive all this components to conceive all the components. Let us consider the way that this turbines are set in a horizontal plane that is the shaft is vertical that means the shaft is vertical usually all reaction turbines of this type except for turbines of very small capacities have vertical shock that means the entire thing is this turbine is in a horizontal plane. So, therefore, you see the entrance is like this fluid comes in to the scroll case then it enters to the guide vein the passages found by the guide veins then it enters to the runner or the rotor of the machine. And this entrance is a mixed entrance that means it is a combination of the radial and tangential that means the fluid here has got both radial velocity and the tangential velocity well here also fluid has got both radial velocity and tangential velocity. So, the radial and tangential velocity both makes a plane which is the horizontal plane then while coming out of the runner the fluid is turned axially. Axially means in the direction of the axis of the shaft and it flows through a draft tube. So, therefore, if you see the plane view of it then a simple plane view like this you will see which this is the scroll case well. So, this is the runner that means this is the scroll case scroll case this is the guide vein guide vein. So, this is the runner. So, fluid approaches like this. So, throughout this so I can show like this then it comes out to the runner this is the axial direction. So, this is the radial direction. So, this is the axial direction. So, here what happens a tube or a conduit is attached which has a diverging section why it is done? So, this is the draft tube why it is done? So, draft tube draft tube. So, now before coming to all the components again let me tell that why such a tube with diverging area is attached for the final discharge of the liquid from the turbine. So, liquid is not discharged from the rotor as such at the outlet of the rotor, but it is allowed to flow through a diverging tube or diverging closed duct known as draft tube. Why it is done? And you know that for any fluid machines at the discharge of the machine for example, if the liquid is discharged from the runner outlet it has a velocity and that velocity corresponds to a kinetic energy and that is totally loss of energy.  So, therefore, you see even for an impulse machine we have seen in Pelton wheel for an impulse machine this is 0 this is 0 for an axial flow impulse machine. So, therefore, head is developed by virtue of the change in kinetic energy whereas, our initial head is 1 by 2 g v 1 square by 2 that means even in an ideal case without friction we cannot have an 100 percent efficiency of the wheel this is because some amount of kinetic energy is rejected. So, this is the head developed and this is the energy input the ratio of which we define as the hydraulic efficiency. So, therefore, the energy which is wasted at the outlet of the machine is the kinetic energy that means this kinetic head we cannot utilize if we could have made this velocity 0 at the discharge. So, the entire energy that is the kinetic energy in case of an impulse machine could have been utilized. So, which now I like to say that for any physical situation liquid has to be discharged with some velocity discharge means it has to be discharged with some velocity which corresponds to some kinetic energy which is a loss. So, therefore, this is a loss. So, this has to be made as small as possible. So, what happens when the liquid is coming out of the runner outlet it has got a high amount of velocity that means the velocity at the discharge of the runner outlet. So, if we can reduce this velocity at the final discharge then what we can do we can reduce the loss in the form of the kinetic energy of the liquid. So, therefore, if it is allowed to pass through a divergent duct as you know for an subsonic flow a area increase makes that it increases the pressure and reduces the velocity. So, velocity is reduced as the consequence of continuity for an incompressible subsonic flow. So, therefore, if we allow a divergent duct to be followed at the runner inlet. So, liquid after the runner inlet with a high velocity relatively higher velocity flows to the divergent duct and ultimately it is discharged at the final discharge level with a lower velocity. So, therefore, this is precisely the function of the draft tube to extract a part of the kinetic energy discharge from the runner outlet. So, that we can effectively gain in the head developed or the work developed by the turbine. So, therefore, we come up to very most important components of a Francis turbine is that important components are first is the volute casing volute casing volute casing next is guide vents or wicket gates guide vents or wicket these are the terminologies wicket gates wicket gates next is moving vents moving vents moving vents or runner and next is the draft tube. Now, we have to understand the function of each and every very simple this is the entrance point liquid enters to volute casing. So, volute casing initially guides the liquid to guide vents the function of guide vents is to increase the velocity partially partly partly part conversion of pressure to velocity and another function is to direct to direct to direct the liquid to direct the liquid in a proper manner in a proper manner in a proper manner to the runner in a proper manner to the runner what does it mean that means it decides the correct angles when the liquid flows through it the vane outlet angle are set in such a way that when the fluid comes out through this its velocity its angle should match with the angle of the vane at the runner blade at the inlet. So, that a smooth shock less flow occurs to the runner at its entry another function is there which will come afterwards that it imparts imparts an initial an initial angular momentum an initial angular or tangential you can call the same thing tangential momentum of the liquid momentum of the liquid momentum of the liquid. So, therefore, you see that there are three distinct functions of the guide vents it partly converts the pressure energy to kinetic energy by allowing the liquid to flow through a converging passages formed by two successive guide vents this is number one number two it directs the liquid in a proper manner to the runner that means. So, that the runner the entry to the runner becomes smooth and shock less that means the liquid velocity relative to the runner becomes equal to the angle to the runner at the inlet and it imparts that initial angular or tangential momentum of the liquid this we will come afterwards then moving vents moving vents is the vane where the head is extracted that means the energy of the fluid is extracted in the form of useful work that already we know there the same principle as happens in imparts machine except the case here as it is a reaction machines the most important part is there when the fluid flows through this moving vents you can see how you can distinguish a reaction machines if you are given a runner then you will see that it forms the passage in such a it is converging in this direction that means in the direction of fluid flow the passage through two runner vents is converging that means whether that means when the fluid flows through this passage between the two runner blades this pressure is change that means both the kinetic energy of the fluid and the pressure energy of the fluid changes that means in terms of velocity both the absolute velocity and the relative velocity changes what happens to the relative velocity relative velocity at the outlet increases from that at the inlet this is because of the convergent duct from which we get a release or a pressure energy is released in the form of the work developed by the turbine try to understand this part another very important fact that distinguishes an impulse machine from a reaction machine that since in an impulse machine pressure in the turbine rotor remains constant that means we can have free jets as the liquid in the rotor of an impulse machine and the liquid in the form of jet can engage one rotor at a time one rotor blade sorry one rotor blade at a time but in case of a reaction machine since the pressure changes while the fluid flows what happens the pressure changes when the fluid flows through the turbine rotor of the turbine therefore the fluid should feel the entire passage of the rotor that means entire passage formed by the rotor blades so it should not engage any particular blade at a time it should totally feel the runner and runner should not be exposed to atmosphere for example we have seen that pelt all runner runner is exposed to atmosphere and the liquid which comes out from the nozzle is a free jet because they are the entire pressure through the rotor of the machine is atmospheric so therefore the runner of a reaction turbine should be completely enclosed and the liquid at for example the water it should feel the entire passage of the rotor so that the pressure is always above atmosphere and it goes on changing while the liquid flows through the rotor blades or the runner of the turbine so therefore it happens so here so after that it goes to the draft tube the function of the draft tube is to reduce the kinetic energy at the runner outlet to a lesser value so that the final discharge from the turbine takes place with a very low velocity of the liquid corresponding to a low kinetic energy which is definitely the loss which cannot be converted in the form of useful work developed by the turbine now before going to the analysis for work developed very important thing is that we must have an idea about the different types of head and how the head varies across a reaction turbine and from here we can get a clear idea of providing a draft tube a clear idea of the working principle of the draft tube now let us consider a very simplified diagram which will give you a clear concept of the change of head across the turbine let us consider this is a reservoir at higher height that means the liquid is stored here which is at a great height from the ground level which is the main source of energy and this is the long pipeline known as penstock as you know this is known as penstock penstock is the pipeline leading to the turbine from the high head reservoir so this is the penstock this is the long pipeline which is tunneled to the rock in the mountain then it comes to the turbine this is the turbine you see this is the turbine is a flange coupling that this shows the typical turbine and this is the draft tube so this water level where it discharges finally this is known as tail rest this is the terminology so this is the final discharge level of the water for the turbine so turbine is set at a distance above this liquid level the tail rest level where the turbine is connected by a divergent duct known as the draft tube now this is the arrangement now we see the heads so if we consider this tail rest level that is this the water level where the discharge is taking place as the datum then you see at this point the liquid has got only the potential energy so head of the liquid corresponds to this height head means energy per unit weight if we consider the atmospheric pressure to create a zero head that means here of course liquid has a pressure rate which corresponds to atmospheric pressure so whenever we will consider the pressure energy of the fluid we will consider it above the atmosphere so therefore the liquid energy here it corresponds to this height that energy per unit weight that means this is h0 is the gross head to the turbine so h0 is the gross head to the turbine now as it flows through this pipeline the liquid comes here so this is the section 1 the entrance to the turbine so this head is lower than this h0 by an amount which is the frictional loss in the penstock the pipeline leading to the turbine from the reservoir so if we apply the Bernoulli's equation between this point and this point we see that the total energy per unit weight at this point which comprises both pressure energy and velocity energy is less than the total energy here which is h0 by the amount h a that is the frictional head loss so here the fluid has got an energy which is h0 minus h a and is denoted as h1 so therefore we see h1 is the net head at entrance to the turbine well so this head comprises both kinetic energy and velocity pressure energy this is because the liquid flows through this penstock develops a velocity so the fluid here is having high pressure and high velocity corresponding to a head h1 now you see when it comes out of the frances runner or the turbine runner at 2 so work is developed so work is coming out so then it has got a very low head again it is flowing through the draft tube where there is no change in the head first of all we discard the friction in the draft tube then we can tell that there is no change in the energy but the kinetic energy is reduced so therefore finally we discharge the water at the tail rest level so if we show this as the velocity energy at the discharge velocity energy sorry kinetic energy kinetic energy at the discharge v3 square by 2g 3 is the section at the outlet of the draft tube then this h this much is the net head producing work that means if we neglect the friction in the turbine so we can tell this is the head at h1 and this is the head which is being rejected that means energy per unit because the final discharge is an atmospheric pressure so this is the head h1 above atmospheric pressure so what is the head at discharge above atmospheric pressure this is only v3 square by 2g kinetic h okay so difference is h which is the net head producing work or sometimes it is referred at head across head across the turbine that means in absence of friction this is the amount which is responsible for the work that means this is equivalent to the work developed by the runner alright so here you see that this is the pressure energy I have shown you this is the kinetic energy so it is mostly in the form of pressure energy and a little is kinetic energy so these are the different forms of head so therefore we see that the head across the machine or turbine or net head producing the work is the difference of head at the inlet to the turbine and outlet to the turbine what is the outlet to the turbine outlet to the turbine is the point three now neglecting friction we can write that now first of all head across the machine or across the turbine head across the turbine head across the turbine is what head across the turbine is is inlet head h1 minus either h3 or h1 minus h2 considering h2 is equal to h3 neglecting friction in the draft otherwise neglecting friction in the draft then you can ask me a question sir if it is so then why do you provide draft you because h1 minus h3 is h1 minus h2 why do you provide the draft you draft you is provided this is the because the with the use of this draft you this h2 is lowered as compared to the case where there was no draft I would like to show that you understand that h1 minus h3 is equal to h1 minus h2 consider the case where there is no friction in the draft you so energy at 2 and energy at 3 remains same what happened to the draft you because there is no energy interaction with the surrounding a part of the pressure energy at point two a part of the kinetic energy at point two sorry is being converted to pressure energy at point three that means here pressure energy is less kinetic energy is more here the kinetic energy is less pressure energy is more so if you now apply you can see this two and the Bernoulli's equation at two and three then we can write if you apply the Bernoulli's equation between point two and three I can write p2 by rho g well plus v2 square by 2 g is equal to what I can write p3 by rho g is zero because we are considering the pressure above the atmospheric pressure sorry sorry plus z plus z z is this height vertical height zero plus v3 square by 2 g so you see that p2 well p2 by rho g p2 by rho g is equal to minus z plus what is there v2 square minus v3 square by 2 g now in this situation for a divergent duct as the draft you you know v3 is less than v2 so therefore this quantity is positive this quantity is definitely the height positive so therefore we see always we have a suction pressure that is pressure is below the atmosphere it is also a common sense intelligent student can very well gas gas speed that if there is a divergent duct and the final discharge of the duct is at atmospheric pressure so somewhere upstream the pressure will be always less than the atmosphere but divergent duct always goes on increasing the pressure and when the final discharge pressure is at atmosphere level at atmospheric pressure so any point upstream in the duct will be having a pressure lower than the atmospheric pressure and the minimum pressure or the maximum suction pressure will occur at the minimum area that is at the inlet to this draft that means at the outlet of the turbine so therefore you see by giving this draft you we can create a negative pressure here that means precisely this h2 is getting reduced that means if you do not give a draft you discharge the water there at the exhaust without a draft you this h2 will be increased because there the pressure will be the atmospheric pressure so you have understood this thing so therefore the h2 increasing means the head across the turbine will be reduced so therefore the purpose of draft tube is to create a negative pressure at the outlet of the turbine to increase the head across the turbine or net head producing work by making it possible by making the turbine to be set at a height from the tail raise so this is another very important point so you should not you do not have to place your turbine at the tail raise so that you can place your turbine at a greater height at a height above the tail raise and at the same time we do not drop you do not have any drop in the available head or head producing work or net head across the turbine by attaching a draft tube that means a diverging duct to the turbine so therefore it is clear that a draft tube reduces the kinetic energy at the outlet or in other way it increases the net head producing work or head across the turbine and at the same time allows the turbine to be set at a greater height at a greater elevation or at an elevation from the tail raise water so I think today that is all about the energy transfer head transfer across the reaction turbine and the basic introduction to reaction turbine if you have got any question please ask any question please you can ask the question in the session please yes this diagram yes good yes yes yes that the inlet to the turbine so at the inlet to the turbine the total head comprises both pressure head and the velocity head so v 1 square by 2 I have seen shown the relative proportion usually a small kinetic head but a large pressure head why because this pen stock this line is of large cross sectional diameter large cross sectional area diameter of the pipe is very high so therefore the velocity head here is not very high but due to the flow of the liquid it has got both the velocity head that it has got velocity head and the pressure head so sum of this 2 is equal to the h 1 that means h 1 is so what is h 1 h 1 is from this point to this point oh sorry plus plus because it is the reference data that means the energy at any point in a flowing fluid mechanical energy is the sum of kinetic energy plus pressure energy plus the data energy that means the potential energy that means velocity head kinetic head pressure head and the potential head so we have chosen this line that means this tail rest level as the data so therefore the total energy comprises of velocity energy which is v 1 square by 2 g plus the pressure energy p 1 by rho g plus this z so therefore this is the total head at the inlet to the machine while the total head at the inlet to the machine and this is the total head at the outlet where pressure is reduced to 0 that means atmospheric pressure head potential head is 0 because we have taken this tail rest level as the datum and the velocity is v 3 square by 2 g so this is given by this so this much amount is the net head across the turbine which is equal to the work done provided the friction loss is neglected so actual work done will be less than that because of the friction loss it is a very important question sometime I ask that a hydraulic machines there is a for hydraulic turbine there is always an hydraulic efficiency even if you have an inviscid fluid as the working fluid inviscid liquid as the working liquid which is not the case in case of pump which is not so in case of pump I will come across afterwards because in a hydraulic turbine you will have to reject some energy at the end of the machine as the kinetic energy so even if you remove friction there is a difference simple example that for a impulse machine or axial flow type as your pelton wheel it is a tangential flow type so the work done even for an inviscid liquid without friction is equal to 1 by 2 g v 1 square minus v 2 square that means the change in the kinetic energy that means initial kinetic energy when the final kinetic energy so final kinetic energy can never become 0 we can we have to discharge the liquid with some flow velocity whereas the inlet energy to the fluid turbine is v 1 square by 2 g or it is more little more than that if you consider at the entrance of the nozzle because of the friction loss in the nozzle but if you neglect the friction in the nozzle totally in the fluid so v 1 square by 2 g is the net head coming to the turbine in the form of pressure energy which was converted in the velocity energy kinetic energy at the outlet of the nozzle and that is being converted to work by an amount 1 by 2 g v 1 square minus v 2 square so therefore even for an inviscid fluid there is an hydraulic efficiency because we have to reject some amount of energy but this is again reduced because of the friction between the liquid and the solid parts of the machines and also because of the friction between the liquid layer that is the liquid viscosity clear any other question yes the height of the drop tube that means at which height runner should be placed what should be the cross sectional area of the how much divergence we will have to give what should be the cross sectional area we will be discussing after these are very interesting problems and the basic physical phenomena which determines these things are known as cavitation this is limited by or constrained by the physical phenomena known as cavitation which will discuss in the next class which will give the limitations for this height above which a turbine has to be placed or which should be the height of the drop tube and what should be the cross sectional area of the drop tube why we cannot reduce the discharge velocity from the drop tube to a very low value that means how much low pressure we can allow at the inlet to the drop tube the very simple that this pressure should not fall below the vapor pressure of the liquid at the working temperature liquid vaporize liquid you will vaporize it is very simple ok I will discuss it in the next class that phenomena is known as cavitation thank you please


When you low carb keto or carnivore, you'll never ever need to snack, but what happens if you screw up? What do you do then? Hey there, NJRoot22.com here with another low carb keto carnivore check and today It's about the general topic of snacking When you come before you Enter the low carb keto carnivore world snacking for this is most people on the standard American diet snacking is part of life you're always hungry and Your diet is the reason why you're always hungry because you're constantly spiking your blood sugar levels and that creates that hunger feeling I've been telling you I've said this before in a couple other videos my whole life I was like oh, I gotta I wish they the doctors invented a pill that would make me not hungry Well that pill is called carnivore or low carb Keep your blood sugar low and you're that hunger feeling just doesn't happen. So when you're doing like I'm carnivore Now for the most part animal based I'll eat Like a hard boiled egg Okay, or a heavy egg yolk omelette with a lot of butter for breakfast for instance or when I wake up lunch whatever and I might not eat the rest of the day or for another 24 hours. You just don't think about food That's the miracle of being low carb that that feeling of needing to eat just goes away and I've been in this phase for so long that that you don't Even think about food and that's why I always say food is not entertainment food is fuel And if you eat the right fuel you run lean and it's it's just It's a it's a fantastic thing. I wish I discovered this when I was in my younger days Because who knows life might be different now. However, you do make a mistake Especially for the newbies in the low carb keto carnivore world They're always looking for things to replace The foods they used to eat, you know, and they end up making mistakes and myself included you make mistakes You you'll end up eating too many peanuts for instance or or something like oh Let me have a gluten-free coconut chip and then you realize the serving is 20 grams of carbs and that even though that's not major You eat 20 grams of carbs. It doesn't matter Whether it's sugar-based or not carbs are sugar. They're equal in this world it Well, maybe not me. There's some carbs that get processed lower, but that that's that's minor.   I had Maybe a half a pound three quarters of a pound of ground beef. I Fried up on a pan. I didn't do it in the grill today for whatever reason I eat that for lunch with no condiments No ketchup never put ketchup ketchup should be in the garbage Mustard is okay. Mustard is low carb. I even know it's processed. Anyway, I eat this meat and I haven't eaten since So that said what happens when you screw up or how do you snack when you're low carb? Well, the the first thing you should do is always make extra low carb food Meatballs are great.  I mean even though I think If you ever just get a regular helmins, that's fine But the soybeans apparently are the soybean stuff you want to keep out especially for men, you know To make a good egg salad or have some hard-boiled eggs Bacon you could bake up a pound or two or three of bacon bake up some bacon Keep that in your fridge have nice crispy bacon strip if you're if you're feeling peckish or whatever And you just feel like eating or maybe didn't eat enough Low carb meat or eggs in the morning and your body needs fuel like when you're low carb the hunger is real Provided you need high carb stuff Side note here The way hunger approaches you when you are Low carb is way different than the way hunger approaches you when you're standard American sad diet like if you had a pint of orange juice and a bagel with cream cheese for breakfast You're hungry before lunch always that's why people go to the vending machine at 10 30 in the morning at their office And then at lunchtime they eat more hard high carb stuff And then they have another snack at three o'clock and then they're hungry at dinner and they get pizza It's this never-ending cycle of constantly being hungry I'm rambling here. Let me have a drink here.  I feel like a million bucks today was a really good day But what can you eat? Let's say you don't have your bacon your eggs or your meatballs in your refrigerator ready to go or some Nice roast beef or whatever it may be Here I got some stuff pork rinds All right Get some pork rinds here. These are these happen to be the spicy version. Okay, that those are pork rinds These are from Aldi. They're a Clancy's. They're 79 cents a bag They have hot and spicy and plain. Okay, and I got these chicharrones from a little little is that new Also a competitor of Aldi These hot and spicy pork rinds I got at Letal over in Union, New Jersey I don't see if you could see them if it comes out good These have to be see how it says here zero carbs per serving see that zero carbs per serving These hot and spicy ones and it doesn't say Before I go on here The thing you got to watch out for with with this flavored Pork skins or pork rinds or chicharrones, whatever you want to call them is MSG Because some of them that I forget which brand it is they have MSG and MSG is the worst thing you can eat Even though it's a it's zero carb I think it is MSG has the same effect on your body that eating like a bag of corn chips does an hour later It makes you hungry screws with your brain. It's just bad Avoid MSG at all costs, but I got these at Letal. These were the best tasting most enjoyable pork rinds I've ever had does that mean they're good and I recommend them. No, not necessarily These were also under a buck a bag, but I remember I That day I hadn't eaten I had gone to Letal and I said let me get these and I'll eat them on the way home I ate the whole bag on the way home They were the texture the crunch and the flavor on these Letal pork rinds was the best ever now I'm looking at the ingredients here and They don't have they have maltodextrin which which can contain MSG. They have yeast extract dextrose Ah, what whatever the case they have Torula yeast Extracts the paprika there. Yeah, that's for the color and they have hickory natural flavors natural hickory smell flavor see And same thing with these these Aldi Clancy brand they also have dextrose and Hydrol hydrolyzed corn gluten. I'm never gonna buy the spicy ones again I mean if I'm feeling I mean they didn't make me feel hungry. They didn't do that They didn't have the same effect So I don't think that whatever the MSG or add additives in here was so bad for me that because it didn't make me Obsessed it didn't make me want to eat like a madman afterward but As much as these are enjoyable I'm going to go ahead on the record here and officially never recommend them maybe once a year I'll treat myself To a hot and spicy pork ride from from lethal because these were absolutely amazing Better off getting the plain ones make sure it has nothing but pork rinds and salt Even though they don't have sea salt, which I highly recommend get the plain ones and what you do is Eat one or two the crunch you get the satisfying crunch. It's almost like a tortilla chip Sometimes they get they get stuck in my throat and I feel like I'm a throw up or I can't breathe It's weird how these pork pork rinds work, but I discovered I made my own little nachos out of pork rinds I wish I could I haven't I didn't document it But you put a couple pork rinds or you know 10 of them on a plate throw some sprinkle cheese on there nuke it for I don't recommend microwaving, but if when you're in a pinch Nuke it for 15 seconds melt the cheese on there You got it's almost like tortilla chips and you don't need to eat a lot You won't go back for more and more and more and more and more pork rinds are a very good in a pinch have them in your closet and You can have fun with the pork rinds, but other than that you shouldn't really need to snack when you're low carb And that's the truth anybody that has done low carb or keto or carnivore the right way Will be you know astounded with the fact that they'll never ever think about snacking Much at all a couple times here and there But usually it's legitimate your hunger. It's real and you need the fuel So just go back to eating your your low carb Or carnivore choice and have a couple bites and just move on And don't let food like overcome you or consume your life the way a lot of people do it Especially by the way, you should get rid of the TV Don't sit on the sofa and passively shovel food in your mouth while you watch like Manipulative and propaganda on TV Why it's just you're like a robot I mean I I could put some links somewhere on one of these days to some good videos that might shock you into Thinking twice about how you live your life You work all day and you sit there and you just rot away on the on the sofa Entertaining yourself when I do something productive, you know start a YouTube channel build something Read a book But but sitting there passively eating while you entertain yourself and get tired And then you just go go to sleep and and probably start the developing diseases that are gonna Require medical attention and where they feed you with more pills and all sorts of stuff And it's a vicious vicious cycle of how they've built this world we live in But break free, you know break free and live pure and free and we'll talk more about the organic crap And and that that's a little bit been been corrupted as well I think but I've rambled on over like 12 13 14 minutes here, so but You won't need the snacks and Later on I'm gonna talk about some snacks and the experiences we've had with them and why we don't eat them anymore I mean I might do one snack like do a two-minute video Every other week of telling you why I don't eat a certain snack that you may still like Okay, have a nice week ahead and enjoy the warm weather that's coming


This research paper examined the effectiveness of a new magnetic abrasive finishing, MAF, process for polishing medical grade nickel, titanium, needy, wires. The MAF process was found to be effective at reducing bacterial adhesion on the surface of the wires, which could have significant implications for medical applications. This article was authored by say Rim Jang, Illinois One Siu, and Lita Hang.


Gold contract down $2.60 straight at $19.45 an ounce. We have silver down 49 cents, $22.95 an ounce, light sweet crude up a buck 42, trading $72.61 a barrel, notes and bonds. Ten-year note, flat, $1.1310. The 30-year up for ticks, $128.06 in Kingdala. Kingdala, where'd I put you? Come on.  I gotta move Kingdala. There it is. Kingdala down 479 ticks, trading out at $1020.61. The euro is at a price point of $109. The yen's at $141. The British pound's at $127.01 the U.S. dollar. We get over and take a look at the spy folks bottom line.  You got to a low, low today, 70 million shares. Yesterday we did 76 million because that's pretty intense.  So, you know, there's a role. Tomorrow would be a big day now. The reason it's a big day is what I said at the beginning of the, what I was doing the update is that this is a hot shoe. And when you get a hot shoe at highs, particularly at highs after you've trended up for a long period of time, the straight side, say, if you take a hot shoe, turn it upside down, okay. And the straight side, we've already done the top of it, the straight side of it, and we'll be at it tomorrow. I've seen many good sized downdrafts. And if we don't get one, guess what, man, this market's gonna go higher like right now. Because that's how these things normally work. That's in the spy. NDX100 is the same setup. Bottom line, you take a look at the NDX, the Q's, same type of setup. That's doing the hot shoe. You did 51 million shares. Yesterday we did 49 million, and you're going into 57. Same type of setup, though. If we get a long bar tomorrow, it's gonna be, you know, it's gonna be different. Because what's also gonna happen with the long bar is that you're gonna be able to, you're gonna be breaking a trend, like an accused that started out at 329 or 361. So tomorrow's gonna be a big day out here. And start your day off right, folks. Come over to TFNN. Tommy starts live programming. Nine o'clock. Check it out. Great program. Have a great night, folks. Have a safe night. Thanks for being with us. Yeah! Look at him, folks.