In this practical, we see all the pieces we've seen in the practicals 2.0 and 2.1 coming together and playing nicely in Torch, using Torch NN package. We start with reviewing the Jacobian and the Hessian partial derivative formulation. This is quite essential to understand how partial derivatives are computed in Torch. We will create then a logistic unit with the NN package. We will perform the forward pass step by step, in order to apply the knowledge we have acquired the previous practicals. We will start with NN linear, which requires an input dimensionality and an output dimensionality. We will learn then how to use the curly brackets operator. We will see how to access the weight and the bias of a linear layer. And its own derivatives of the cost function with respect to its own weights and bias. We will see then how to zero these partial derivatives. And then we will introduce the sigmoid nonlinearity. We will see how to use the forward method of the module. Linear and sigmoid are modules. So by writing NN.module, I just represent both of them. Then we will also implement the back propagation pass step by step, again to verify our understanding of the underlying algorithm. We will start by defining a loss function with the NN MSC criterion, mean square error. Then we will see the attribute size average of the criterion and how to change its value. Moreover, we see that the forward method for a criterion now expects an input and a target, whereas for the module it just expects a single input. Moreover, we will see how to compute the gradient of the error with respect to the output of the network using the update grad input. And then how to back propagate the gradient computation through all the modules of our network, which also require the current input to the specific module and the gradient at the output of the given module. We will see also how to accumulate the gradient parameters, basically how to compute the partial derivative of the error function with respect to the parameters of a given layer. In this specific case, the linear is the only module which has parameters. Then we will introduce the NN sequential, which allows us to perform the forward and back propagation with just two simple commands. So we see how to add modules to our sequential container, how to forward an input through all the modules within the sequential container. Then we will forward the output of the sequential within our loss function, where we specify the input, which is the prediction of the network and the expected target. Then we perform the backward step, which is actually for the criterion most of the time like the same of update grad input. The backward step actually performs both the update grad input and the act grad parameters. Usually, the criterion don't have trainable parameters, but if they do, then it will perform that as well. Then we see how to get a specific layer out of a sequential with the get method. We see how to zero the partial derivative of the error with respect to the parameters of the model with the zero act parameters. And then we back propagate the gradient of the criterion through the network and with the backward method to which we have to provide the input to the network and the grad criterion. Finally, we can update our parameters based on the learning rate, etha. And finally now we will train a generic neural network by using stochastic gradient descent and mini batch gradient descent. We'll do so by using the forward, backward and update grad parameters together with the zero grad parameters functions we just illustrated. We can do the same by using instead an n stochastic gradient to which we simply provide the network and the loss function with the side. And then all we actually need is simply to ask the stochastic gradient to perform a training of our network based on the specific criterion on the data set that we provide. As last point, we will see how to perform a regression task with a three layer neural network input, a hidden layer of three neurons and just one output neuron. We will see this one at the following address, github.com, etcal, my username, torch, machine learning with torch. So far we have used the Jacobian formulation. This means when we have a partial derivative dy on the vector x, this is going to be a one row matrix. Where each component is going to be simply the y in the x1 up to the last one, the y scalar on the xn. If we have instead dy vector in dx, so it's only variable, which is a scalar on the denominator, this is going to be equal to the column vector of the derivatives of the components. So we have dy1 in the x and down until the last one, which is going to be dym in the x. And there we go. Finally, if we have a combination of the two, so we have dy vector and dx vector. This is going to be the Jacobian, so which is the y1 in the x1. And we go down to the last one, so it's going to be the ym above the x1. And then if we go on the other side, we're going to have still here the first row versus the last element of n. And then down we have the last one element here, the ym and the xn. And this is the Jacobian of y. The last case is when we have a scalar, so we have dy on the x matrix. And this one, since we use the Jacobian formulation, if we say that x belongs to rm times n, then the partial derivative with respect to the matrix x is going to be the... The dimensionality is going to be n times m. So we have that the dimensionality is the transposed basically of what it is on the denominator. So like it happened here in the first case, we have the vector column vector x. So since this is the denominator, we are going to have a row of derivatives. And the same way here, we have a matrix of the denominator, which is dimension m times n. And therefore the partial derivative with respect to that matrix is going to be the transposed in terms of dimensionality. So it's n times m. And for this very same reason, let's compute now the dimensionality of the partial derivative of the error with respect to the parameter theta, capital theta of layer l. So a hat of layer l is going simply to be, belongs to the dimension s layer l plus 1. On the other side, up to here, we have the error of the layer plus 1 is going to be of dimensionality s l plus 1. And therefore if we compute also the transposed, we are going to have 1 times s l plus 1. And therefore we have that the multiplication is going to be having as many rows as s l plus 1 and as many columns as s l plus 1, it's more.  And we can see of course here that we have the partial derivative with respect to a matrix of s l plus 1 times s l plus the bias is going to be in the transposed.  In Torch, we are using instead the Hessian formulation. And in this case we have that dy on the capital X, where X belongs to RMN. It's also belonging to RMN. In this case when we have to do the weight update, so we have our matrix theta, we can actually write this on entities legitimate theta plus a learning rate eta, multiply by de respect to d theta. And we saw just now that these all elements here will have the same dimensionality of this one. In the Jacobian formulation, we should have had a transposition to the second part. But as we said in Torch, we use the Hessian formulation where basically all the results of the partial derivatives are basically the transposed of the normal kind of notation. This is also called numerator layout. And this one, the Hessian, is also called denominator layout. You can read more about this on Wikipedia and look for matrix calculus. Let's run Torch. We would like to create our sigmoid unit. Let's require the package NN. Let's say our neuron has N equal 5 neurons. And then the final output has dimensionality 3. So we can create the linear, which is going to be creating the weighted input. It's going to be an NN.linear going from N to K. So if we print linear, we see it goes from 5 elements of input, which is the dimensionality of our x vector without accounting for the element x0, the bias. And it goes to our output. So we go from x, the input of size 5 to the output h theta of x, which has size 3. Let's see what's inside this linear. Linear has a bunch of things inside. Let's start with the things that are making more sense to begin with. So we have lean.weight. And this is the matrix 3 times 5, meaning it shoots to a dimensionality of 3, which is the output, and takes 5 elements, which are the elements of the size of the input x. And we have another one, which is lean.bias, which is basically our first column of our theta matrix. So we can call here now theta 1. It's going to be torch concatenate the first term, the bias, and the second part in weight. And we concatenate along the second dimension, so across the columns. And this actually creates a new tensor. So if I display theta 1, it's going to be simply the bias vector for 0.43, 0.44, minus 0.22, and minus 0.28. And then we have the rest of the weights. Let's see again lean. So what's this in here? So we have something that is called grad weight. So let's see what is that. Grad weight. And they are all zeros. And lean.grad.bias. And all these are zeros as well. So the first one, what is called grad weight, is going to be basically the partial derivative of E with respect to the weight of this module. And grad.bias is going to be the E with respect to the bias. So in our case, if we consider grad theta 1, this is simply going to be torch.cat lean.grad.bias stack in front of grad weight on the second dimensionality. And there you go. This is the vector, which is the partial derivative of the error with respect to the parameter theta 1. In case those gradients are not already zero, we can zero them with lean.grad parameters. And if they were not zero, now they are zero for sure. We will see later why we need to zero these parameters. And it's basically connected to the fact, connected to the way we can train a network if we use a batch or minion batch gradient descent. We have to accumulate the partial derivatives with respect to the parameters for several iterations. So this is the place where these partial derivatives are accumulated. And therefore we have to start with a clean accumulation point. So we want to zero them before starting. Then so the first part is done. So I think we have then we have an output, which is empty at the moment because we haven't insert anything in the module. We have a grad input, which is the derivative of the output with respect to the input of this block. And we don't have anything at the output and nothing in the input. So it doesn't it's empty at the moment. And the last part we haven't covered is the type, which is torch double tensor. And this shows the type of module we are using at the moment. Let's create the second module. After the linear, we need a sigmoid function. So we have sigmoid equal nn dot sigmoid. Let's see. I print sigmoid. Let's nn sigmoid. And let's see what's inside. So we have again a grad input, a gradient input gradient, which is empty. We have a type that is torch double tensor by default. And then also the output is empty. Let's see how this sigmoid looks like. So let's require new plot. And then we can do, for example, z equal torch linear space from minus 10 to 10 with 21 points. And then we can have new plot dot z and sigmoid to which we forward the z. And let's see. And here we have the sigmoid. It goes to 0 basically after minus 5 for numbers that are lower than minus 5. And then it hits the 1 half for x equals 0 here. And then after for x greater than 5, it approaches the 1. All right. So let's go on and let's start now with the forward pass. So we can clear the screen. And let's start. Forward, forward pass. So let's have our input vector x. Just a torch random normal distribution of size we said n. So this is our x. We have that a1. It's equal x by definition. We have that h theta, the final hypothesis, the output of our network. It's going to be the sigmoid to which we forward the linear to which we forward our input x. This is going to be our output. Let's have a look. So it's three dimension as expected. And those are the values. 0.36, 0.55, 0.29. Let's try to reproduce these values by ourselves so we actually understand what's going on. So we have z2. It's equal to theta1, which is the concatenation of the bias column with the weight matrices. For definition of how we saw in the previous lessons. Multiplier by what? By our input to which we put on top the plus one, so our a1 hat. So we have torch hat of torch ones, just one one. And then we have the a1. We concatenate in the first dimension. So z2 is the weighted input. And then we have to apply a sigmoid to the z2. Let's do this by hand. So we have a2 equal z2, which I clone. And then I apply. If I don't clone, I'm going to override its value. So I just want to preserve whatever it was there. So we are going to have a function. Function of the scalar z. Which simply returns a1 divided by 1 plus what? f dot x pon initial of minus z, the scalar. We close this one. And we end the function. And we close the parenthesis. And let's see our output a2. And there you go. 0.3613, 0.25510, and 0.2924. So we have just seen that our computations are, our network is actually computing what we expect, what we have seen before in the theory. Let's perform now the backward pass. This is a bit more elaborated. So backward pass. Oh, there's back propagation. Or back drop allocation. So we have to define first a loss function, which is going to be, in this case, the mean square error criterion. So if we print loss, it's going to tell us MSC criterion. And then if we would like to know what it does, we can press question mark. Then we have nn dot msc criterion. A description of how it works creates a criterion that measures the mean square error between n elements in the input x and the output y. So loss of x and y, where x is the input and y are the basically targets or the labels, is equal to 1 above n, where n is the dimensionality of the vector x and y, a summation of the square differences. We can also see that the division by n can be avoided if one sets the internal variable size average to false. That's what we are going to do because in our previous lectures we haven't performed any size average. Let's print the content of loss. And we see it contains a gradient, basically, a size average that is true by default and the output that is 0 to begin with. So we are going to say loss dot size average is actually false because we don't want that division. So we can check if it worked. Yes, it did work. So let's compute now the error based on our targets. So we have to define our targets. We have y, which is going to be torch rand, not rand n, because the output of the network goes from 0 to 1 since we have applied a sigmoid non-linearity. So rand of size k, so this is our y. Let's see the API of the loss function, the criterion. Basically, if we write forward, we are going to need an input and then we need our target or label. So we can do e, the error for this specific sample, the only sample we have here. It's going to be equal loss forward, to which we send the output of the network, which was called the h subscript theta, capital theta, and the targets are labeled y. And here we have e, which is 0.35. Let's try to compute this by ourselves. So as we saw in the description before, the criterion performs the difference of h theta minus y in the target on rise to the power of 2, and then summing all together. There is no one-half factor as we saw in the lesson before. So we had to actually keep in mind this one-half missing here, which is going to be coming back, of course, when we perform the derivative of the square. And here we go, we have exactly the same result. So we are still performing forward, but we are forwarding in the criterion to compute the error, which is required in order to perform the back propagation steps. So let's compute now the derivative, partial derivative of the error, with respect to the output of our network. So this is simply our loss. We can write update grad input, and we send h theta, which is the input. We have the same API of the forward. We have input and target. And then the targets are the y. And there we go. So if I print the e in the h, as we saw, as we said before, since we are using the hessian notation, it's going to be having the same dimensionality of h of theta. Let's compute this by hand, so we can see whether it is correct or not. So if we perform the derivative of this expression here, we are going to get, we had two times, simply the difference between h theta and y. And there we go. We had the same numbers. Minus 0.37, 0.19, and minus 1.10. So far, so good. What we are seeing here in Torch is exactly what we have seen before in the theoretical slide. Let's compute the error now at the output. So we have delta 2. It's going to be the sigmoid, to which we say to update grad input. And we send inside the input. We always have to send the input, which is z2. And then the output derivative. So the derivative which we find at the output of this block, which is going to be actually the input derivative of the criterion. And here we have delta 2, which we of course have an element of 3. It's going to be the size of 3. So let's verify this is correct so we can see. We start from d, e, dh. We do a clone. Then we have a sigmoid. And then we have a2, to which we do a sigmoid of 1 minus a2. And there we go. So we just apply the formula number 3. Now we can compute the partial derivative of the error with respect to the parameters of the linear module. So to do so, in tors we can do a lean, accumulate, gradient parameters, to which we provide the input, which is to the module, which is x, and the grad input of the following module or the grad output of the current module. So in this case it's delta 2. And then we can check what this has done. Looks like it hasn't done much. So inside lean we have grad bias and grad weight. So we can recall before they were zeros. So if I go back up, they were zero. So let's see if I print them again where they are. So if we print now grad theta, there we go. So these are the partial derivative of the error with respect to the capital theta matrix, which is simply the concatenation of the first column, which is the grad bias, and the other matrix, which is the grad weight. Let's verify also that this computation is correct. So we can do so by performing the column vector multiplication time the row vector. So that we have the partial derivative of the error with respect to the parameters of the linear module. Let's write down so it's more easy to explain. So we have delta 2 to which I'm specifying which I'd like to view as a column vector. So it's going to be as many rows as it needs and just one column to which I multiply the concatenation of a simply one and my x on the only dimension. And then this one instead I'd like to view as one row and how many columns it cares. There we go. We got the same result. Nice. Let's go one step further and let's compute the partial derivative of the error function with respect to the input of this module, which is actually the global input. It's not useful for training, it may have other utilities. For example, if we have multiple layers we are going to need that value in order to compute the delta at the previous layers. So let's just call this as lean grad input which is not delta 1 because delta 1 we take in account also the non-linearity in this case we don't have yet the non-linearity. There is no non-linearity at all. So it's simply going to be the partial derivative of the error function with respect to the input of the linear module. This is equal to simply linear, update grad input to which I provide the input and the delta, the partial derivative at the output. So grad output basically it's called this one, let's see. It's going to be of the same dimensionality of the input which was 5. This is still because we are using the Hessian notation or the denominator layout. Let's verify these numbers are also correct. Also as we recalled from equation 5 that we saw also today before we can compute this by using the matrix lean dot weight transpose to which I multiply which is applied to the delta 2. And there we go. So we just verified it. All the computations are correct and we obtain consistent results. It's pretty nice. We input so far so many commands and it looks like quite daunting to follow how it has been done so far. This is because we haven't actually used properly the NN package. The NN package provides us also and other amenities which make the whole training a breeze. And especially the forward and back propagations can be run in just two lines of code. Here we have typed down all the steps just in order to verify the correctness of these steps. Now we can instead see how we can type all these in a much more compact way and hide, of course, some intuition and some gatchas that will actually strengthen our understanding of the algorithm and how the package works. So let's define a network as being a container which is called sequential. Sequential simply allows us to have a sequential series of blocks one after each other where all the forward steps are performed automatically. So the output of the first block is sent to the second block, to the third, to the fourth, etc. And when we perform the grad input steps all the grad inputs go from the last to the first block. So we can add a net, add our linear module. We can add also our sick-moid module. And then we can print the network and it will show us that we have a simple sequential... We have an input which is sent to the first module which is sent to the second module which is basically the output of the network. To perform the whole forward pass we can write simply pred which is our prediction of the network which is our basically... which is our age of theta we defined before we call it now prediction so we don't overwrite our values that we have written before. It's simply network forward of variable input x. So now we can see prediction is this and if we check age capital theta it's exactly the same. So that's perfect. With one line we can compute the output of the network. We can compute the error, error equal loss to which we forward our prediction and the correct label y. So error equal this one that previous e is the same so far so good. And now the cool part so we can compute grad criterion equal loss and backward to which we send again pred and y. So we can compare grad criterion with d, d and dh before proceeding with the backward step for the network we have to actually clear out the grad bias and weight. Now that we have a network I can have net get first and we get the first module within the network. So if I'd like to see what are the actual derivative of the error with respect to the weight I can print them here as this net get one grad bias concatenated to net get one dot grad weight along the second dimension so these were the numbers we got before so we have to zero them otherwise they will sum to this one. So we can simply do this by typing net zero grad parameters so if we call again the same function we have all zeros that's lovely we have net backward to which we input our input x and the criterion and this one has performed a bunch of things let's see the last thing that has performed we have this minus 0.0 0.958 so this is exactly the input we have computed before you can see minus 0.0 0.958 0.1339 0.0826 if we go down we have the same numbers here so when we perform a backwards step return as the input gradient of the current network module whatever we are using let's call this guy again so we compare these numbers and we can see we have minus 0.0860 minus 0.0615 if you go second line 0.0461 third line minus 0.02284 that's perfect so we have computed the gradient of the error with respect to the parameters all the parameters of the linear module which are the only parameters in this case with just one instruction that was the network column backward to which we send x the input of the network and the gradient of the error with respect to the output of the network the age of capital theta so let's see how the parameters are updated so to update the parameters I can define a learning rate eta equal 0.01 for example and then I can say network update your parameters based on the learning rate eta bam the network has updated its own parameters based on the current parameters and the gradient of the error with respect to the parameters let's check that that is actually what it happened in reality we can have d e I think we have d e with respect to the age no so d in the theta 1 it's equal this thing here above right and then we have theta 1 minus eta which multiplies the theta d e the theta 1 so this one is this one and let's see there we go, theta that's called nu and let's print theta nu and we have what we expected so the new parameters where we have here the first is the column relative to the bias and here it's the the matrix the new matrix theta 1 it's exactly what we expected so this is actually this has been computed by torch because we actually perform now we just check inside the internals of the lean so linear dot bias and linear dot weight we have them concatenated across the second dimension so we have the first one here lean bias and then there is the matrix here lean weight which has been computed as the previous theta the full matrix here which includes the bias terms the bias vector minus eta to perform the gradient descent step multiplied by the d e in the theta 1 finally let's see how we can train a system with all we have learned so far so now we are confident that the torch works the library nn it works magically it does require really few instructions in order to compute forward and backward propagation basically there are two instructions one for forward one for backwards and then we have the two zero sometimes the parameters the grad parameters so let's have an example of working example almost working example so let's see how to train the full system with a script we have training we have our x it's going to be our design matrix and then we have our y it's going to be our labels or targets if you like matrix or vector the first one has size m times n and the second one is going to be of size m examples times capital K all right control u to actually clean the line that's cool so we are going to have again a for loop which is going to go from the first example so number one to the m example perform what so we are going to have our prediction which is equal to the network we have defined forward to which we send the ith example of the design matrix then we can have again a local error which is going to be equal let's call it loss forward forward to which we provide the prediction and the label of the ith example and then we can have the grad loss which is going to be those backwards of what we have to send the same prediction and y and then we have now to zero we have to zero the accumulated parameters so this is in this case we are talking here of this stochastic gradient descent in this case so we have to zero every time zero grad parameters and then we have network network to which we provide the input so it's going to be capital X of I and then we have grad loss and then the last one is going to be a network to which we go say update parameters because again we provide an update every time we provide a new example in the case of stochastic purely stochastic gradient descent update parameters and here we have our ith the learning rate and this is the full example working example for applying stochastic gradient descent given that we have a design matrix X and a design a label matrix Y capital Y and capital X if instead we would like to perform a mini batch a gradient descent this is a little bit more code to write but it performs better in terms of convergence in terms of speedup if we use multi-dimensional input which I'm not going to illustrate right now but mini batches provide some specific advantage both computationally and also optimization wise so we are going to have here I again that goes from 1 to the number of examples we have but jump with batch size so in this case if we go like from 1 to 1024 with batch size of 128 we are going to have I equal 1 129 257 and etc up to the last value so we can do we start by zeroing the parameters and then we do for j equal goes from 0 to the batch size minus 2 so at the beginning I the summation i plus j is equal 1 and for the last for the last value of j is going to be batch size minus 1 actually batch size this is going to be batch size minus 1 so it's going to be batch size so it goes from 1 to 128 then the second time is going to be 129 to 256 then it's going to be 250 7 and so on to what there is the output then we have if we exceed the index m so if i plus j it's greater than our index m then please don't do anything otherwise the same thing of before so prediction equal net forward then we are going to have local error and like before and then we have again local grad loss and then we follow with net backwards and then we end it here after when we finish to process the batch we can actually update parameters based on the grad with the learning rate and then we are actually done last part is actually so we actually don't have to write all this code neither so torch allows us to write even this code so that's why it gives us so many very nice treats let's say this way so the treat is basically if I have I have to provide a data set to start with so we have a data set which can be an empty table at the beginning and then this table has to provide a size which simply returns m the size of the data set so far nothing too complicated and then we have for i equal 1 to the size of the data set please do populate data set so element i is going to be a table of the first element is going to be the example and the second element is going to be the table and here so once we have this kind of data set which returns a size if required and then each each item is the table of example of the input and the target we can simply create a trainer so we have local trainer a local trainer yeah funny right so this is going to be a stochastic gradient trainer which we send the model and we send the loss we decided to use and all we really need to do is say a trainer please train my network on the data set i'd like to so basically to train your network you just really need those two lines of code of course sure you have to specify a network and a loss function and of course a data set otherwise you wouldn't be training on anything so sure you have a necessity of defining a data set a network a loss function which you know suits your needs and then you create a trainer which is going to be training your network in just one line we can see a full example of how to train a network with the trainer the stochastic gradient from torch at this address let's have a look so here we have the machine learning with torch repository and specific the first topic it's regression with MLP multilayer perception which is simply a neural fancy way of calling a neural network usually a multilayer perception MLPs are used for pattern recognition classification task in the field of image and speech recognition nevertheless they can be effectively used for regression check out MLP regression section to find out more about it so you can go here and then you can see what I'm trying to do in this tutorial so basically I will explain how to perform regression with a neural network so while I was reading the b-shops book pattern recognition and machine learning I got to the point in which a three-layer perception with only three hidden neurons and one output linear neuron, so four neurons overall was used to regress seemingly some contiguous functions and here is the image so we have the illustration of the capability of a multilayer perception to approximate four different functions comprising f of x equal x square f of x equal sine of x f of x equal absolute value of x and last one is the f equal h of x where h is the step function or the heavy side step function in each case n equals 50 data points so the x coordinates shown as blue dots have been sampled uniformly in x over the interval from minus one to plus one and the corresponding values of f, x evaluated so f of x are the y targets or labels and the x are simply the input one-dimensional scalar and also the output is a scalar so we have size of the input layer it's one size of the output layer it's also one and then we have three neurons which are the internal hidden neurons these data points are then used to train a two-layer neural network having three hidden units with a tan h which is similar to the sigmoid that goes from minus one to plus one instead activation function and linear output units so there is no non-linearity after the last layer the resulting network function are shown by the red curves here in the picture and the output of the three hidden hidden units are shown by the three dashed curves and so just go through the tutorial and then there is the algorithm where I explain how I use the trainer for training on the data set and I highly recommend to click here on the source regression and actually play with the code so read the description and use it interactively so change some values and play around to understand better how it works you have different modes just go through it carefully and you will get a lot of intuition and understanding I believe and that's the end of the overview of the NN package we will see more features in the next podcast stay tuned have a good night bye and good night because yes it's night now I'm working at night night


Book 10, Part 2 of Pharsalia, Dramatic Episodes of the Civil Wars. This is a LibriVox recording. All LibriVox recordings are in the public domain. For more information or to volunteer, please visit LibriVox.org. Recording by Phil Schemf. Pharsalia by Lucan. Translated by J.D. Duff. Book 10, Caesar in Egypt, Part 2. Pothonus did they pass, as though in peace profound the nightly watches. But Pothonus' mind, once with a cursed butchery imbued, was frenzied still. Since great Pompeus fell, no deed to him was crime. His rabid soul, the avenging goddess, and Magnus' shade stirred to fresh horrors. And a farion hand no less was worthy, as he deemed to shed that blood, which Fortune proposed, should be due the conquered fathers. And the fell revenge, due to the Senate for the Civil War, this hireling almost snatched. A vert, he fates, far hints the shame that not by Brutus' hand this blow be struck. Shall thus the tyrants fall, just at our hands, become a farion crime, reft of example? To prepare a plan, fated to fail, he dares. Surveils in fraud a plot for murder, but with open war attacks the unconquered chieftain. From his crimes he gains such courage as to send command to lop the head of Caesar, and to join in death the kinsmen chiefs. These words by night his faithful servants to a chilisbear, his foul associate, whom the boy had made chief of his armies, and who ruled alone, or Egypt's land, and or himself, her king. Now lay thy limbs upon the sumptuous couch, and sleep in luxury, for the queen has seized the palace. Nor alone by her betrayed, but Caesar's gift is ferocious. Dust the lay, nor hasten to the chamber of thy queen, thou only, married to the Latian chief, the impious sister now her brother-weds, and hurrying from rival spouse to spouse hath Egypt won, and plays the bod for Rome. By amorous potions she has won the man, then trusts the boy. Yet give him but a knight in her enfondling arms, and drunk with love thy life and mine heal barter for a kiss. Be for his sister's charms by cross and flame shall pay the penalty. Nor hope of aid, here stands adulterous Caesar, here the king her spouse. How hope we, from so'd stern a judge to gain acquittal, shall she not condemn those who ne'er sought her favors? By the deed we dared together, and lost, by Magus's blood, which wrought the bond between us. Be thou swift with hasty tumult to arouse the war. Shin with nightly band, and mar with death their shameless nuptials, on the very bed with either lover smite the ruthless queen. Nor let the fortunes of the western chief make pause our enterprise. We share with him the glory of his empire or the world. Pompeus fallen makes us too sublime. There lies the shore that bids us hope success. Ask of our power from the polluted wave, and gaze upon the scanty tomb, which holds not all Pompeus's ashes. Peer to him was he whom now thou fearst. Noble blood, true, is not ours. What boots it? Nor are realms nor wealth of peoples given to our command. Yet we have risen to a height of power, for deeds of blood, and fortune to our hands attracts her victims. Lo, a nobler now lies in our compass, and a second death, asperia shall appease. For Caesar's blood, shed by these hands, shall give us this. That Rome shall love us, guilty of Pompeus's fall, by fear these titles. Why this chieftain's strength? For shorn of these, before your sorority lies, a common soldier. To the civil war this night shall bring completion, and shall give to people slain fit offerings, and send that life the world demands beneath the shades. Rise then, in all your hardy-hood, and smite this Caesar down, and let the Roman youths strike for themselves, and logos for its king. Nor do thou tarry, full of wine and feasts, thoualt fall upon him in the lists of love. Then dare the venture, and the heavenly gods shall grant of Katoes and of Brutus' prayers to thee fulfillment. Nor was Achilles slow to hear the voice that counseled him to crime. No sounding clarion summoned, as his want, his troops to arms. Nor trumpet-blair betrayed their nightly march, but rapidly he seized all needed instruments of blood and war. Of Latian race the most part of his train, yet to barbarian customs were their minds by long forgetfulness of Rome debased. Rome's had it shamed to serve the Faryon king, but now his vassal and his Minion's word compel obedience. Those who serve in camps lose faith and love of kin. Their pittance earned makes just the deed, and for their sordid pay, not for themselves, they threaten Caesar's life. Where finds the piteous destiny of the realm, Rome with herself at peace. The host withdrawn from dread the sallia raves on Nylis's banks as all the race of Rome. What more had dared with Magnus welcomed, and legion house. Each hand a must render to the gods their due, nor son of Rome may cease from civil war. By Heaven's command our state was rent in twain. Nor love for husband, nor regard for sire parted our peoples. To as a slave who stirred afresh the conflict, and a chillest grasp to in turn the sort of Rome, nay more had one had not the fates at burst restrained his hand from Caesar's slaughter. For the murderous pair, ripe for their plot, were met. The spacious hall still busied with the feast. So might have flowed into the kingly cups a stream of gore, and in mid banquet fallen Caesar's head. But did they fear, lest in the nightly strife, the fates permitting, some in cautious hand? So did they trust the sword. Might slay the king, thus stayed the deed. For in the minds of slaves the chance of doing Caesar to the death might bear postponement. When the day arose, then should he suffer. And a night of life, thus by Pothinus, was to Caesar given. Now from the Cation Rock look forth the sun, flooding the land of Egypt with a day warm from its earliest dawn. When from the walls, not wandering in disorder, are they seen, but drowned in close array, as though to meet a foe opposing, ready to receive, or to give the battle. Caesar, in the town placing no trust, within the palace courts lay in ignoble hiding place, the gates close barred. For all the kingly rooms possessed. But in the narrowest portion of the space he drew his band together. There in arms they stood, with dread and fury in their souls. He feared attack, indignant at his fear. Thus will a noble beast, in little cage, imprisoned, fume and break upon the bars his teeth enfrenzied wrath, nor more would rage the flames of Vulcan in Sicilian depths should Etna's top be closed. He whom but now, by Hamas's mount against Pompeya's chief, Italia's leaders and the Senate line, his cause forbidding hope. Looked at the fates he knew were hostile, with unfaltering gaze now fears before the crime of hireling slaves. And in mid-palice trembles at the blow. She whom, nor Scythian, nor Alon, had dared to violate, nor the Moor, who aims the dart upon his victim's slain, to prove his skill. The Roman world but now did not suffice to hold him, nor the realms from furthest end to terri and gateys. Now as puny boy or woman, trembling when a town is sacked, within the narrow corners of a house he seeks for safety. On the portals closed his hope of life, and with uncertain gait he treads the halls, yet not without the king, in purpose tolemeus that thy life for his shall give atonement, and to hurl thy severed head among the sermon throng, should darts and porches fail. So story tells the colquian princess, with sword in hand and with her brother's neck bared to the blow, waited her sire, a venture of his realm despoiled, and of her flight. In the imminent risk, Caesar in hopes of peace, an envoy sent to the fierce vassals from their absent lord, bearing a message thus. At whose command wage ye the war? But not the laws which bind all nations upon earth, nor sacred rights avail to save or messenger of peace, or king's ambassador, or thee from crime such as befitted thee, thou land of nile, fruitful in monstrous deeds, not Juba's realm, vast though it be, nor Pontus, nor the land, the salient, nor the arms of Farnaces, nor yet the tracks which chill Iberius grids, nor Libyan coasts such wickedness have dared, as thou with all thy luxuries. Closer now war hem the men, and weapons in the courts, shaking the innermost recesses, fell. It did no ram, fatal with single stroke assail the portal, nor machine of war, nor flame they called in aid. But blind of plan they wandered purposeless, in separate bands around the circuit, nor at any spot, with strength combined, attempt to breach the wall. The fates for bad, and fortune from their hands held fast the palace as a battlement. Nor fail they to attack from ships of war, the regal dwelling where its frontage bold made stand apart the waters of the deep. There too was Caesar's all-protecting arm. For these at point of sword, and those with fire he forces back. And though besieged, he dares to storm the assailants, and as lay the ships adjoined rank to rank, bids drop upon their sides, lamps drenched with reeking tar, nor slow the fire to seize the hempen cables and the decks oozing with melting pitch. The oarsman's bench, all in one moment, and the topmost yards burst into flame. Half merged the vessel's lay, while swam the foemen, all in arms the wave. Nor fell the blaze upon the ships alone, but seized with writhing tongues the neighbouring homes, and fanned to fury by the southern breeze tempestuous it let from roof to roof. Not otherwise than on its heavenly track, unfed by matter, glides the ball of light, by air alone aflame. This past recalled some of the forces to the city's aid from the besieged walls, nor Caesar gave to sleep its season, swifter than all else to seize the crucial moment of the war. Quick in the darkest watches of the night, he leaped upon his ships, and Pharaoh seized, gate of the main, an island in the days of Proteus Sear, now bordering the walls of Alexander's city. Thus he gained a double vantage, for his foes were pent within the narrow entrance, which for him and for his aides gave access to the sea. Nor longer was Pothonus's doom delayed, yet not with cross or flame, nor with the wrath his crime demanded, nor by savage beasts torn did he suffer, but by Magnus's death, alas the shame. He fell, his head by sword hacked from his shoulders. Next by frauds prepared by Ganymede, her base attendant, fled Arseno from the court to Caesar's foes. There in the absence of the king she ruled as of legion blood. There at her hands, the savage minion of the tyrant boy, Achilles, fell by just a venging sword. Thus did another victim to thy shade atone, Pompeus, but the gods forbid that this be all thy vengeance, not the king, nor all the stock of Legos, for thy death would make fit sacrifice. So fortune deemed, and not till patriot swords shall drink the blood of Caesar, Magnus, shall thou be appeased. Still though was slain the author of the strife, sank not their rage. With Ganymede for chief again they rushed to arms, in deeds of fight again they conquer. So might that one day have witnessed Caesar's fate. So might its fame have lived through ages. As the Roman chief, crushed on the narrow surface of the mole, prepared to throw his troops upon the ships, sudden upon him the surrounding foes with all their terrors came. Intense array their navy-line the shores. While on the rear the footmen ceaselessly charged. No hope was left, for flight was not, nor could the brave man's arm achieve or safety or a glorious death. Not now were needed for great Caesar's fall. Caught in the toils of nature, routed host or mighty heaps of slain, his only doubt to fear, or hope for death, while on his brain Bravescae was imaged flashed, now vainly sought, who on the wall by epidemis' fields earned fame immortal, and with single arm drove back Pompeus as he trod the breach. End of Book 10, Part 2. End of Farsalia, Dramatic Episodes of the Civil Wars, by Lucan, translated by J. D. Duff.


Happy New Year Freedom Family! It is New Year's Eve and 2023 is just around the corner. We will be busy building Heartbeat and in this video I will show you what Heartbeat is in a sneak peek at our prototypes. But first, what has kept us busy in 2022? As you know, Freedom Games raised $10 million for indie game publishing. The link to this article is down there in the description below if you want to read it yourself. But basically we were able to raise $10 million to sell video games. Freedom Games is something we created two years ago guys from nothing. Obviously that took a lot of time but now that we've achieved this milestone it's time for the next one. What is our next big idea? That is Heartbeat. So without further ado, what is Heartbeat? YouTube as we know is the world's video library. Heartbeat is the world's memories. We are the world's memories. Any of you guys watch Game of Thrones? Remember the Three-Eyed Raven? That's Heartbeat. Kidding. But same idea. So ask yourself two questions. One, whoop, kind of scrolled too far there. What is the one thing missing from Wikipedia? First what is Wikipedia? Well this is Wikipedia. The world's encyclopedia. Here is the Apollo 15 lunar mission article on Wikipedia. So looking at all this text and imagery what's the one thing missing? Video obviously. There are many well-written articles but there's nothing to watch. So what's the one thing missing from YouTube guys? You get it right? Something meaningful to read. Lots of video. Some useful comments. Most descriptions full of follow me on Twitter links. Nothing meaningful to read. So what does YouTube plus Wikipedia look like? Well of course that is Heartbeat. We want to solve one core need. Save people time. We want to let you watch only relevant clips as you're reading articles or just watch the relevant highlights of the videos. You don't have to watch the whole 10 minutes. Just get to the key points quickly. We all have we will have one core function. Given any paragraph of text in a blog on a wiki wherever return a relevant video clip that's crowdsourced. That's from you guys. We will have a collection of something meaningful to read and let me give you an example. So I said this is the Apollo 15 article on Wikipedia. Lots to read. There's one particular section where they did an experiment. They dropped a hammer and a feather on the moon's surface to see if they would hit the ground at the same time. Well in my left hand I have a feather. In my right hand a hammer. And we're here because of a gentleman named Galileo who said objects of different mass will hit the ground at the same time if there's no atmosphere to slow down the feather. Oh great. Show it to me. On YouTube there's a video showing this experiment and on Vicky our video wiki at look at this URL guys vi.ki. That's our prototype. You can go there right now and check it out. On our video wiki we have the same text as on Wikipedia. See here. Copy. Paste. The difference is if you click this time code you will see and I'll drop the two of them here and hopefully they'll hit the ground at the same time. How about that? And here you get the idea. We jump right to the moment in the video showing the actual dropping of the hammer and the feather. So you don't have to wait the 37 seconds to get to that point. Now this is a YouTube video. Nothing special. It's a full video on YouTube if you want to go there to look at it. By the way links to all this is more down there in the description below. Check it out yourself. But there's no chapter. There's nothing that jumps exactly to that moment. You kind of have to look for it by hovering over this bar. Very time consuming wouldn't you say? So we're trying to save people time. You can just click this link and you can see the hammer and the feather dropping and hitting the ground at the same time. So that's the concept of heartbeat. Now let me show you the prototype. It's not just hammers and feathers falling. It's mission impossible. It's George's life as a journey quote. It's Elon Musk. This is all at vi.ki. Let's look at mission impossible. This is the biggest stunt in cinema history. Show it to me. Okay. Eight click. Here it is. Tom Cruise just rode a motorcycle off a cliff six times today. Popping his shoe. He's outdone himself impressive. Let's look at a few more. Here is George. Not so impressive, but still useful, right? Let's check out life as a journey of curiosity. Is that what life is? Let's find out. The journey of curiosity. That's what life is. I believe. What do you think? Tell me. So the point is, I jumped exactly to that moment and you will be able to write these articles. You will be able to identify these exciting moments that matter to you. These matter to me. Maybe they don't matter to you in the real heartbeat. You will be able to create these moments for yourself. And let's look at one more example, which is for gaming. If someone writes a review of Valheim and you made a video of getting killed by a tree in Valheim, you could submit your video clip into that article. And when someone clicks the time code. The tree falls down. And it hits me on the head. Death by tree. And you can see it in video. But the point is, guys, you will be able to submit your video clips to these articles so you can get more views when people want to see the paragraph they are reading on a heartbeat. Okay, you got the idea now. So I have many notes about heartbeat. Before I get to them, I would love to hear your thoughts down there in the comments below what you think of this idea. We will make it like YouTube where we will have a revenue share business model, which means we will pay you every time content you create on heartbeat earns revenue. We will share that revenue with you just like YouTube does. There will be support for networks and subnetworks. There will be support for music, music libraries, many of the things you're used to on YouTube. But the difference is there will be split payments, which means if 10 people contribute to your article, not only will you get paid, so will those people who contribute. And guess what?  You will set the revenue shares that your fans and friends will get when they contribute to your work on heartbeat. This will be a community effort. YouTube in general is a single player experience. You versus all the other channels in the world, yes, you can collaborate and invite others to make videos, but generally you make your own channel. You upload your own videos. Maybe you can hire a video editor once you've made enough money to afford his fee. Maybe you have some fans that'll help moderate your comments, but in general, you are alone creating your career on YouTube. Why? It should be a social experience. But to do that, YouTube needs to implement split payments where they can reward your friends and fans when they help you either editing your video, either providing music or collaborating with you. These things you have to pay yourself once YouTube pays you. Heartbeat will not be like that. Heartbeat will implement a point system and a revenue share system where you, the creator, define how much each point is worth to incentivize your friends and fans to help you build your video career on heartbeat, but not just video guys. We will have writing, text, we will have images, a click list gallery. Let me show you the click list gallery. Again, this is a prototype, but the idea is you have a bunch of images up here and then you can just hover over each one. Like show me the table in white now in black. Which one do I like better? I'm not clicking. I'm just hovering. These are 3D renders of different houses and you don't have to click anywhere. Let's say you want to zoom in. I want to look at the details of this microwave. Just press five. And again, don't click. Just move the mouse. You can explore these images as you would maybe on Amazon or other places that implement the same effect. But where have you seen a click list gallery? Most places you have to click to see the image. Then you have to click to zoom in. Then you have to click and drag.  So we'll have a click list gallery. All right. Let's actually go through the notes now. If you've gotten this far in the video, you probably want to learn more. So let's go on to, I mentioned split pay. Oh, start at the top. Be the platform. We want to be the platform instead of building on other people's platforms like Facebook or YouTube or Twitter or places you've seen Instagram. And then we will use the platform, which is the browser, everything in a web browser, even on mobile with one code base for you techie guys out there. This is called a PWA, a progressive web app where you have one piece of code that runs everywhere. Yes, it's possible and few people do it. I don't know why. Monetization. We will pay you exactly as YouTube does as a content creator that gets revenue share of all the content you create. Not just the videos, but the text, the images, everything. We will have split payments. YouTube will not pay both of you when you collaborate on a video, but we will. Let's say you write an article or make a video. You get paid for both if you add your relevant video clips to the article. And I showed you already the Valheim death by tree example where you can contribute your video clip of getting killed by a tree on someone else's article reviewing the game. Heartbeat is a shiny new idea. We are the world's memories. What I mean by memories? I mean moments that matter, the clips, the highlights, the things that you look for when you're watching a whole video and you want to go back to it and you remember some exciting moment, but you can't quite find it. That is what I mean by we are the world's memories. So we'll save you time. Yes, we will pay you for every piece of content you create. Replying to comments, bringing your friends to heartbeat. That last one is normally what YouTube MCNs pay you. Oh, I got a little too many capital letters there. Sorry. Okay. Get paid from the first piece of content you create. No 4k, 1k YouTube. You can upload as many videos as you like, but until you have a thousand subscribers, you make zero dollars. Why? I know why it's because of advertisers and it's YouTube's way of combating spam channels, but what about the legitimate creators who are doing it? You're basically choosing not to pay them, like have a system to separate the spam from the legitimate creators and pay them from the first video. Well, that's what we will do. We will have prototypes. You've already seen Vicki. I've shown you the Click List Gallery, but we don't have a public link for you yet to play with. We're working on that. You'll get paid for uploading images, not just videos and text. Yeah, images add value to a platform. You should get paid whenever you add value to the platform, I think. So revenue streams, upload videos, identify video clips, write articles, flex those writing muscles, guys. Presumably you went to school and presumably you learned how to write. You spent years learning how to write. When was the last time you've written anything more than a few emails or comments? Come on. If you don't use it, you lose it. So use it. What a shame and all those years invested in learning how to write if you don't write. So you can on Heartbeat and get paid for it. Upload videos. Whoops, that's meant to be upload images, right? Yeah, yeah, we already have an upload videos. I'm fixing my own mistakes here. Recruit your friends to Heartbeat, just like YouTube MCNs pay for recruiting creators to YouTube. We'll offer that. Does YouTube pay you for bringing new people to YouTube? No. They just pay you a revenue share for the ads around your own videos. So I feel that's not fair. If you help build YouTube, then you should get paid for it however you do it. And we'll support networks and subnetworks. We'll have early payments. You want to get paid early? Christmas is here. I forgot to buy a present for my girlfriend. I need 50 bucks right now. No problem. Here it is. Early payments. We've actually already built it. It's called freedom pay. And many of you in the freedom talent network already take advantage of this. So it's a service we will also include in Heartbeat. We will provide music, professional music, production music, commercial music. We'll have safety for your Heartbeat channel. If you get demonetized or terminated, we'll tell you exactly why and how to bring it back. Our policies will be clear and transparent and you will get a second chance, then a third chance, maybe even a fourth chance. We'll see. Even if you clearly did something stupid like upload the Simpsons episodes hoping to get more views from copyright content you should not be trying to monetize, we will let you learn from your mistakes and stay on the platform. How about that for a concept? And we will be community and creator driven. Heartbeat will not be built by a team hidden away from you making decisions that are really focused only on making advertisers happy. We want to make the creators happy first. Certainly we need to also make advertisers happy because that's how we will make money. But if we have to decide on a feature between advertisers and creators, we'll choose creators first, not advertisers because it's you guys who are building the platform so why should we not choose you first? So let's start collecting what you want. Actually, I don't have a Google form so I'll bring this out on another video. But for now, share your comments down there and I'll read and reply to every single one. We talked about how YouTube is a single player experience and okay summary built for you creator first clear and transparent policies with the second third maybe fourth chance even if you clearly abuse them commercial and production music for free not free we will pay for it but not charge you for it. Early payments get paid when you need it or want it not just when we want to pay you. Subnetworks are back and networks open to all to apply. You know with YouTube you can apply to be an MCN but they review applications only twice a year every six months. I think we will review them twice a day well maybe not quite that often but very regularly certainly more often than twice a year and of course we will save people time by letting you get to that exact moment that you've been looking for and we'll show you how all that works when we have more prototypes things you can try but here's a good example of one of our tools that we've built that you can try right now spam.tm it's where you can get your YouTube channel health check just paste your channel id and get a check look all of these channels are doing it three hours ago five hours ago six hours ago let's take one of these guys music channel so this guy did a health check let's run it again and it'll show us what the channel saw so he's just got no comments on videos which is actually a bad thing that means your videos don't have any user engagement so he knows what to focus on he's only one percent spam good for you so check out spam.tm put in your channel and uh check your spam score we'll pay you for every piece of content you create no 4k 1k get paid from the first piece of content you create on heartbeat get paid for uploading images in our new click list gallery and uh get paid for all these things so follow my heartbeat that's what we will call instead of subscribe to my channel or follow me on twitter just follow my heartbeat the things that matter to me if they matter to you then maybe we can have a shared interest connect strangers and make new friends that's how i met my wife guys i met my wife online not a dating website studypond.com which sadly is gone i don't know why but it's no longer there it's a language exchange site i wanted to learn chinese she was willing to teach i was in san francisco living there in the usa at the time she was in taiwan we met online to start these lessons once a week every friday for two hours and that's how we began our relationship and when i went to taiwan i felt a little bit more for her than just a student uh would and eventually we got married had four kids and uh are living are living happily ever after all because of a shared interest in language exchange maybe that will happen to people on heartbeat as well i hope so all right freedom family those are the key things i would like to build in 2023 come on come on come on there we go and i'd love to hear your thoughts down there in the comments below i'll be making more videos as we progress and i'll be engaging with you and i hope we will grow together as a family freedom family and now in heartbeat our own platform be the platform so who am i i am papa george george son pi obi one george nobby many other names you the beautiful freedom family has given me i am the ceo and founder of oh there is freedom for freedom and the freedom family where we all help each other grow together as a family that is the spirit of freedom till the next episode happy new year i hope you're bright eyed and bushy tailed for the shiny new 2023 like i am and we'll see you next year tomorrow take care i am george and you've been watching oh you're dreaming but love yeah click that i to partner with freedom and join the freedom family so we can all grow together you get many perks like position music you also get epidemic sound a lot of other access to royalty-free videos sponsorships and many things to help you grow just click the links down there in the description below to get involved in our community our forums our discord chat servers meet our graphics team meet our community team all of that on discord and the forums what are you waiting for get started and we will grow together as a family because this is the freedom family you are part of it we are all part of it and we're all growing together to get more george click that big f that will subscribe you to freedom central home of the george show and pewdiepie gave one of you freedom family a big shout out click that video to see the shout out and to see our new 3d sets for you and click that video to see what youtube recommends you watch next


All right. So my task was to talk about analysis and tools and portals. So we're going to be shifting gears a little bit because what I want to talk about is given the sequences in a central place, variance, recall, aggregated metadata, phenotype data harmonized, what tools are needed to make the data useful for the community. So I know conversation tends to come back what's possible given the data restrictions. But I think it's important to explore what's possible in an ideal world, how we could make the data, so who is our user base, what analysis we can do, and how to make this most effectively. So the first slide here is just an incomplete laundry list of the type of analysis we can do, given the variance. Steve already covered some of these. We can provide phased haplotypes, aggregate allele frequencies across populations and across the whole cohort, calculate variant burdens in various ways. We can provide functional annotations at coding and non-coding parts of the genome, lots of function analysis, including information from disease databases. We can redo GWAS analysis in various ways and go to higher order analysis, system biology approaches to network pathway analysis, and so on. Some of these analysis are well defined, some are pretty open-ended, some are easy, some are very hard algorithmically, and accordingly some tools are available for these, in some cases the tools are still emerging. Some require a lot of computation, some are very easy to do, and don't require huge computational burden. In order to prioritize which of these analysis are important to us, the most important is who are we trying to serve with all these analysis and the tools that we are developing. Because the analysis that were described in the previous slides clearly mean different things to different people and answer, and people are interested in very different aspects of the data. So David Altshuler made the point abundantly that statisticians and tool developers are already very well-served, people like me, we wouldn't have this project for people like us. Medical consortium project analysts and drug developers will benefit, but they already are fairly empowered. So really the target audience, in my opinion, again starting downstream from the variants is really biologists and small laboratory students, post-docs, and clinicians who don't necessarily have the expertise and the resources for taking advantage of the data. So in this vein the question is how to make our analysis accessible. First thing I think the key aspect is things have to be easy. Tools would have to be easy to install, easy to use, they have to be intuitive, preferably fast, and if they fast they can be interactive. I sort of highlighted web-based, which is not the only mode to accomplish this, but it's a very good way to accomplish this, and one that's for which modern technologies exist. HTML5 is making real impact on being able to access even a large amount of data and locally render images with new web-based technologies. This is only one side, the other side is even if these tools are very easy to install, there are some other impediments from small users to use the data. Some of the larger analyses require huge computational and storage resources, but users will have access to basically laptops, resources on this scale. Some of these, using these tools and running these tools require groups of analysts who are highly skilled in bioinformatics and computation. What we want is people whose expertise lies in other areas in experimental biology to be able to use the same tools. So with this in mind what are the ways we can and hopefully will in the future be able to provide the data and the analysis to the community? Well, rather to download is sort of the baseline. Everybody understands that's been the mode of operation, easiest to accomplish, hardest to go afterwards because now you have to store and compute the data, install the tools, all the stuff I talked about. Query portals, viewers, data slicers, Steve covered that in great detail. I think that's really is making an impact now. For example, the thousand genome data is now available through these data slicers from EBI and NCBI as well. I think that's a very good way to get the data to the users. Static variant annotations, pre-computing your resources. Again, Steve talked about that. That's also very important, but it's limited because many of the analysis will require more dynamic content and I'm going to come to that. So this fourth is I think for us to, for the way forward, where you have central data, for example, all the sequences from the medical projects, you bring tools next to them. Users can upload their own data, analyze them in conjunction with the central data set and you provide an environment in which the tools can be run. I think this is ideal. This is my opinion where things are going. So parsing out what are those areas where static analysis makes sense and there are a number of these. Clearly, variants, variant annotations, sample genotypes, fairly simple data types that are easy to compute and they general. Face haplotypes, a very well-defined data type again, basic per-variant annotations and I'm going to come back to this. Method data, phenotype information, again, things that are static content. In order to make these available, you need better query tools and again, Steve covered this I think very thoughtfully. There's a great need for being able to query and browse phenotypes and there's very little available in that area. Also need sophisticated viewers. One example is just looking at the variants. So current viewers are basically either genome browsers or seek or alignment viewers. But what's important about the variant is their structure and what they do to the genome. So here's an example. This is a sort of mock-up where a deletion introduces, so the structure of the variant that is a deletion. It deletes some of the chromosome and what it does to the genes is it makes a fusion gene. So this level of visualization and interaction, in my opinion, is essential. Also if we were to provide haplotypes, how to browse the haplotypes and phenotypes as I talked about. So what does are more dynamic that it makes more sense to provide tools to analyze these dynamically and provide customized content. Turns out that re-mapping and variant calling for users' own data. And this is ongoing. There are a number of projects funded by NHGRI. The Broad has one, Gonsal and I running one of these, is actually aiming to make these tools available in such an environment in an integrated analysis environment for users. Imputations, so we've been talking about the possibility of a central hepatotype resource and associated computing infrastructure that allows imputation of individual data into the central hepatotype resource. Customized functional annotations, these are not practicable to compute. You can't foresee any possible structural variation, genome rearrangement or deletion. The aggregate effect of phase variants, this is a slide from Daniel McArthur. So we often see in the 1000 Genomes Projects that consecutive indels, if considered in separation, they introduce frame shifts, but they always observe an individual in phase, in which case the phase is restored, the frame is restored. So these are the kind of things, again, that you can't really pre-compute. You need dynamic computations for these. Turns out that this category has the highest tool development cost because you're not just making these tools available as command line tools, but there's more infrastructural development. Do we need one tool or do we need multiple competing alternate tools? This is an example and the specifics are not particularly important, but these were comparisons from the 1000 Genomes Projects somewhere along the way, maybe a year ago. These comparisons are fairly common. If we look at the same exact dataset, process them with a number of different tools, in this case we were calling them with some experimental tools on chromosome 20, and then we compare these. And what's important about them is that if we use the results in combination, say, for example, take all the variants that are called by at least two different tools, the quality of the dataset improves. So the point here is that there are inherent advantages to having alternate tools available, centralized or distributed development. The point, and I think a very important point was made yesterday, I think maybe Mike brought this up, that tool development is iterative. Once you get an answer, you want to ask a new question. It's not possible to say, okay, we're speccing out, this is the information we want to get at. And then two years in a line, the tool appears and that satisfies everybody. It is really an iterative ongoing process which requires flexibility. So users are often better served by light, flexible tools for customized analysis, than with large, more monolithic systems. And this, so for example, I'm very partial to skiing and what you see here is all the available snow report apps on iTunes. I use some of them, some of them are great. Some of them I really don't care for. But the point is all these are looking at the same data. And the ones that provide good extra mile allow people to give up to the reports, they tell me, yeah, this is great. Come out or that's not where we're going. Those are the ones that survive. So providing tools in this way makes sort of a competitive mix for a competitive environment where tools are driven by demand-side economics, as we heard from Lincoln yesterday. Who would develop the tools? In our experience in 1000 Genomes Project, which is, in some ways, we can look at it as a very, as a large tech development project. We see a balance of tools coming from larger groups, large genome centers, and a large, also a large cottage industry, successful tool developers. And what we find is that even fairly small informatics tools can produce very sophisticated software, high-performance software, and respond to, namely, to user needs. So last slide before we get into the discussion. So if I look into my crystal ball, I would say we need to focus on the cloud because that's the right combination of things we need. Not necessarily for providing big central analyses, but for the biologist that doesn't have the resources, because it gives you all the components. I think we should build an open environment for tool deployment to pull in the widest possible developer base and let tools compete with each other and let these tools follow demand. And this is not impossible. That's where technology is going. Models exist, for example, the iPhone apps that everybody is using. And with that, I'll open up for discussion. Yes. Excellent presentation. I really like the analogy of the app world. My question in that case is that where is the platform? Is there one platform or multiple platforms and who is developing those platforms, right? Because you say iPhone. I think iPhone is in this case a platform. Who will be the genomic platform in this case and is it one or multiple? I don't necessarily think that this would be iPhone applications where we analyze genomic data, although you never know. There might be some questions where that's perfectly sufficient. David. Well, I had a comment or two questions. One question is, yesterday Mark presented the analysis server with apps and there was this generally negative response. And now I'm hearing positive ideas to apps. I just want to draw the line. In some sense, for the apps to run, there has to be a platform and there has to be data they can access. So is it the same or a different idea you're proposing? Well, I think the idea that here is that as long as we can put together a platform, I think what's key about this is that you have freedom to put in applications. Any developer can go and answer a question that may be of interest to the users. This is exactly what Mark said yesterday. And I'm just saying, I actually think it's a great idea. But there were other people who said it's intractable, it's impossible, we shouldn't try and do it. No, I think the difference is that there's multiple ways to look at it. A, you can say, OK, we're going to build something that will answer these specific questions. Then it's not. If we say, OK, we're just going to make a platform available, all the data that you would need to answer questions that may arise this year, or we already know, are some that arise next year. And let. So the apps have to have all the security and all the things. But the question I had for you is, you discussed a world in which the biologists and the clinicians want access to things like variants. But they want answers to questions. You didn't really talk a lot about analysis of phenotypes. You talked about how you think about that. Absolutely. I don't think to a clinician, variants are horribly useful because they find, they say, OK, I see this. They send off for a clinical sequencing test. They find, oh, there is a variant in this gene. And they may know that, OK, there's another variant in the same gene, and that causes phenotype. How do I alter treatment? I completely agree with you. So that's where the app comes in. So that's data integration apps, looking at it useful ways that can provide the clinician with specific information. These applications can be certified. They can be certified to satisfy various requirements to be used in the clinic, and so on. I think the reaction, David, just to clarify a little bit that the distinction between Mark's presentation and Gabor's a little bit, to me, was this issue of vetting the applications that come in. And the idea here is that, correct me if I'm wrong in this, but you presented this such that this would be for the average biologist, and there would still be an opportunity for people to develop, let's say, their own applications and download the data, move it locally, and actually do the compute. So that was said yesterday also. But there was this thing about, well, what's the criteria for vetting an application? No, but he just didn't talk about it, but it would have to happen because otherwise, you have the same problems everyone raised. He just didn't happen to talk about it. One word that was used, and I think Mark, you mentioned, it was scalability. So certain types of applications, discovering of certain types of structural variation are very computationally expensive. And if a decision was made that this is too much of an investment to have an app that would run on the central server, it would still want the option to be able to take that data and actually do the analysis locally, as opposed to having things vetted because it's not scalable for. No, no, so first of all, yesterday it was always said that, of course, the baseline, the data is always available for anyone who wants to do it. We're trying to enable people who aren't able to do it. But second of all, the board didn't talk about the fact and no one asked, well, if you had that app and it was on the machine, but it would be computationally so expensive, who's gonna pay for the server? Who's gonna ensure the security? It's just the two discussions differed in the focus on what it would take to achieve it, but the idea is exactly the same. Yeah, I think there's no, at least from my perspective, there's no disagreement. It's important to have some type of central analysis, as long as you don't preclude what might be considered more idiotic or more involved. The inviolable thing is that people can get data and analyze it. It's just the community people who can actually do that as small. The real problem is how do we help everybody else? Because already we're gonna streamline so we can get data. And there are ways to do the more heavy lifting style things. So it's possible that that's where centralized servers, say for variant calling, other people's data and mapping would come into play and that would not be an app. Would you also introduce this idea of this competition that would go on among the apps? So if you had multiple apps that were doing the same thing, that they would evolve, because whatever people used the most would ultimately win out. This competition as opposed to some group of individuals saying this is the standard that we're gonna apply. That's more than evolving. It has to be dynamic. That's what was said yesterday. You want that. That's exactly why you want to have the apps because you need diversity. I think the only question is just in an environment where apps could potentially access data that we need to protect. You have to have some quality control or some technical mechanism in place so those apps don't just open a connection to a server and upload all the data. You have to have some vetting, but it could be very fine grained of like, this is scalable. This is not scalable. It requires special infrastructure to run. I think you just, we have to articulate what tools require and what that vetting, what the barrier would be for running apps, say, for free. Yeah, I think the discussion of a sandbox, right, where people would actually do that and lots of the kinks would be worked out. It seems like a really good one. But what the opportunity here is that you can deal with data access on the level of the developer because you can control how the software, what the software access is so you don't deal with it on the user end. So it's not that the user, through this app, the user can't access anymore what the app allows the user to access. But that's the vetting question. That on the base layer, we can have the environment control which data sets and stuff a particular developer can access but what the applications use out the other side, that's gotta be vetted if it's public. Otherwise, the application has to be run under the security framework as well. I think that's what happens with security-wise but iTunes, that's the applications. You can't just put an application on iTunes. Right, but there's been a lot of experience that iTunes doesn't do a good job of that, right? There've been multiple cases this year where Apple's had to pull apps off after users have pointed out that the app is actually inappropriate. So I don't think we can keep using Apple as this. No, absolutely, but what we can in the sense that for the most part it works very well. I mean, out of thousands of applications. But from a security perspective, I don't know it's the best. The one thing I was gonna say, several people mentioned clinicians using these apps and I think that there's a whole different clinical infrastructure for decision tools and how they have to be validated. So I'm not sure that's the best model. I would get back to David's question again about there are thousands of researchers who work on these individual genes who have very important questions that we need to make sure they can answer. And many times they see that there's a variant and they have no way of knowing whether that variant has any evidence of being biologically important or not. David. This discussion really points out to me, I think the importance that we are very active in recruiting completely unrestricted data as part of this exercise. If we completely ignore the category of data like George Church's PGP data, then there's nothing that's in an unrestricted environment that the app developers can work on apart from this vetting process. And if we don't have that, we're just gonna have this very draconian vetting process and it'll be very difficult to encourage a creative community to get involved in this. So you have to have these things thoroughly developed in this free environment, because that's where the geeks work. That's where their most active and the most beautiful results come. And then you have to take the best of breed and go through an additional vetting process. But if you don't have that initial foment of activity in the open environment, you'll never get there. Yeah, I'm going to add to what David is saying. So I think there are really two technical issues that need to be set up at the beginning. One is access of the apps to restricted data. I think that it is much better, as David said, to start out with targeting the apps at either a completely open access tier or to a commons tier, such that the developers and users of the apps are certified researchers. And that removes much of the burden of detailed vetting of these apps, which I think everybody feels a little bit uncomfortable with the scale that we're potentially talking about. The second is the issue of the scalability of the apps, how much resources they take up, how much storage they use, how many computes they use, and who's going to pay for that. And I think that right at the beginning, we should talk about ways that the resources used by the apps is born, is charged back in some way to the users of the apps, so that there's a built-in supply, demand, and cost to the model. Otherwise, it's going to become a tragedy of the commons, that there'll be a thousand apps and none of them will be usable because they're really choosing too many resources. Yeah, actually, resourcing is something that we really didn't talk about, but it's a bigger discussion about how to use the cloud. Another advantage of that sort of environment is easy metering, easy way to meter use, pay for use, bill use, and so on. So with the cloud, NIH can just fund the analysis, fund the server, and doesn't have to set up many, many parallel computing environments in different laboratories, and so on. Same thing with the apps. You pay for the apps, $4.99. They use advertising also as part of their revenue, so maybe that won't be the case, so some other funds will have to pay for it. You want to? I just want to, I'm worried about, so I know this is not being suggested, but just to make it explicit, I don't think we should put all our apps into one app system basket, nor should we put app systems on the table, though, these solutions are not usually compatible. One can have direct downloads, one can have maybe one or two different ways of cutting more centralized analysis. Maybe some of the big institutes in Michigan will effectively run a private in-house systems that they use just for themselves, for example. So all of those are going to be different. And I think there's a lot of details in the details here, for example, IO between your storage and the app thing and something else, and when we talk about apps, do we, what really is it a Google-like API with very restricted access to the data, or is it more like you've got a file mount, both ways? And yeah, so there's lots and lots of details. This meeting is not the place to hash out those details. The place to hash out those details is using proposals to say we should do that to mine's app. But I wanna say that there's probably, there's quite a lot of implementation risk in this, and that's why we shouldn't put all the ones, the X and the ones in one basket for this process. Yeah, I- I just find it kind of remarkable how everyone seems to wanna hear one thing and then object because it's not all things. Like data should be, there should be publicly available data that can be operated on, but there's a lot of data that's valuable that's not publicly available. Users who are very sophisticated should be able to get all the data so that they can innovate and be serendipitous. But the vast majority of people don't actually wanna do that. They want somebody to actually provide them some answers. We should do all of the above. Apps are good, but it'll be a little complicated because we'll pay for them and vet them. Somehow instead of seeing the diversity in these things and saying we want diversity, we're saying you do this, you won't do that. But no one's saying do one thing and not the other. Yeah, different users will be served differently with different applications. Steve, have been waiting for a long time. Yeah, so I wanna put in a plea for something that hasn't been discussed yet, which is comment on reproducible research, that the NCBI BLAST server has been one of the greatest boons for democratizing access to computational analyses of data. In our hand, most of the result, a very large fraction of the biomedical literature belongs in the journal Irreproducible Results in that you cannot actually get back the same result that someone did on the day they did their query. Heraklitos of Ephesus said you cannot step twice into the same river because the waters are always flowing on to you. We were the problem that we can't twice query the same database because the data are flooding over us so rapidly. And so we need some sort of solution. So actually when someone does some sort of analysis, you can figure out what did they actually do and be able to figure out, what was that actual analysis they did and be able to verify that at some later date? And I think that if we're building a platform now to enable these types of analyses to a larger biomedical community, we should be thinking about this from the get go. So I agree with Steve. I wanna come back to David's thing. I think that everybody is interested in everything. And I think we just need to come up with a plan that steps out what are the most important thing, what's the most important thing we can do first? Obviously to get simplified access, right? Then what's the next thing? What's the next thing? I mean, I think that everyone's interested in all aspects of everything that's on the table. It's just that some are harder to implement and we'll take more time. So if we could just come up, I think at the discussion time of the workshop or sometime, a priority list and how difficult we think each of these steps will be or how easy and this is a no brainer, we gotta do this ASAP, this would be a good outcome of this workshop. I mean, I would say that that's definitely how we should move forward. But given the fact that there is a, the largest part of biologists is underserved by data, it would seem to me that we shouldn't prioritize the easy things first only. So we should give weight to impact. How about cost? I have a feeling that cost is gonna weigh into this and the things that cost a lot of money that would take the longest to implement. Fine, but as long as NIH is spending a huge amount of money generating data and a teeny amount of money to make it useful, we could question whether that's good use of the national investment. No, and I agree with you on that, David. And there are lots of ways to make it useful, but I think there are very simple ways to make the data more useful to the greater good. And I think this has also evolved over time and I think trying to put forward a plan that would have everything from start to finish at this stage is kind of a fool's errand. I think what we should be thinking about is the first two or three steps that we all can agree upon as opposed to, it has to go this way, this is the best plan. I mean, this will evolve and this won't be the only meeting I'm sure in terms of this question. Well, and also maybe deciding, maybe there should be additional workshops that talk about certain of these things and what it would really take to do some. I guess there are some clear things that we could do now, but I mean, some of us may decide that it makes sense to work on building larger scale infrastructure that'll solve sort of not today's problems, but maybe six months or a year from now. I think as long as we have a diversity and have those plans forward and support at the right scales for different prototypes or projects, it seems to be a great outcome. As long as we say, you know, these are the three or four things that we definitely wanna explore and have some support and some of those can be easy and some of them can be harder and it seems like that would be what we would want. We would be very enthusiastic about a specific area, you know, individually and we'd love to contribute to that, but that shouldn't preclude people from, you know, releasing data like DB Gap is talking about doing, which is also hugely valuable and can be done quickly. Yeah, so that brings up the question of what to pilot, how to move forward, how much testing we do upfront so we can learn for the bigger systems. And I know Mike wants to talk about that in the wrap up session. Thanks to all the speakers for this morning. Lunch start, yes, thank you. Thank you.


Hello! Welcome back to Exotic Wine Travel. I am your host, Matthew Horkey. Do you like mountain wines? Let me explain. First of all, especially in Italy, there are two regions that actually are known as being mountainous wine regions. First, we have Alto Adige in the north. That's right on the border of Austria. And another awesome wine region I was just in, Valle d'Osta. It's on the border of Italy, France, and Switzerland. This is Italy's smallest region by wine production, by population, and by size of area. It's a mountainous region. There's only about 120,000 people living in the entire region, 50,000 of those of which are living in the village of the town of Iosta. There's only about 450 hectares of vineyards there. So wine production is small. A lot of these wines are going to be drank locally, but you know what? The quality is extremely high. The reason I was there, actually, I was a judge at the Mondial de Vins extremes. And I have to say that it was a phenomenal competition, phenomenal trip. I want to thank them, first of all, for having me. It's a wine competition dedicated to winemakers that are growing vines in heroic conditions. Basically, to submit your wines to the competition, vineyards either have to be grown at high elevation, above 500 meters, above sea level, on slopes over 30 degrees, on small islands, or on terraces. So really some unique wines. There were not industrial wines at the event. A lot of small producers. The quality was extremely high. I think in total, we gave away lots of grand gold medals, gold medals. Fantastic competition. I will be supportive of it, even if I'm not a judge next year, although I hope I'm invited back next year. Needless to say, there were a lot of wines from Valedos to in the competition. You got to understand that the region has made kind of a three valleys. These vineyards are extremely steep slopes. There's vineyards on the north and the south sides of the valley, and you're going to find tons of unique grapes. You're going to find international grapes, and you're going to find local and regional grapes as well. I mean, you're going to find Pinot Noir. One of the first actually valios wines we ever had was a Alton Pinot Noir. I thought it was very impressive. We have this one from La Grande. You have yet to taste yet. You have Pinot Grigio, which does extremely well. Chardonnay, Muller Thurgau from this cooperative, also very good. Now, on for the local and regional grapes, that's where I think things get really interesting. They have a Swiss grape nearby Switzerland that's grown called Petit Arvine. This is the only place in Italy you'll find it. This is actually the winemaking school in Valedosta. It's called the Institut Agrico regionale. This is their 2018 Petit Arvine. This is very clean, delicious, well made. If you're in Valedosta, you almost will never go wrong with a Petit Arvine. You're going to get lots of floral notes, honey, ripe pineapple fruits. It has almost the perfume, the aromatics of a cooler weather grape combined with the fruit of a warmer weather wine region, brightest city, minerals. Petit Arvine, Cherine loves, I love, fantastic grape. Another historical kind of region you're going to find subregional Valedosta is called Torrette. This is a co-op, Torrette Sapirir. I also have a small producer here, Lassours. This is his Torrette Sapirir. These wines are mostly going to be made of a grape called Petit Rouge. These wines are really juicy. Kind of remind me a little bit of a mix, imaginary blend between maybe a San Giovese, Barbera. They're going to have a lot of red fruit, some sour cherry, juiciness. Sometimes you can get some tobacco and mineral-y notes. These are fantastic everyday, Torrette's can be fantastic everyday food wines and they can move up into the superior level. Torrette Sapirir can be more serious, structured, age-worthy wines. You're going to find some other unique red grapes there. Malellette makes light red wines. Cornelin as well, which can also be found in Switzerland. Two grapes that I find most exciting are Foumen. This is the La Grotta de Vigneron. This is a co-op. This is the Foumen Espirit Foulet 2017. This is a very unique grape. Not everyone's going to like it. It has weird, not weird, it has kind of more animal and gaminess type flavors. Some people could confuse that with a Brent with a bacteria. It's not. It's actually just unique, well-made. These are structured wines that can go long in the bottle. And then there's a grape that not many producers have, Volumen. This is one from the Institute Agricultural School as well. This is a fantastic red wine. It's peppery, it's got light red fruits. Reminds me a little bit of a Sera. The tans are a little bit softer. These are grapes that you should seek out. So where do you actually get some of these wines? It's not the easiest thing. Valle de Osta is, most of the wines are sold locally and there's a lot of tourism there because a lot of people go there for winter sports, for skiing, for hiking. You're going to have to scoop them up there. If you find them, you'll sometimes find them abroad. But the cool thing about these wines, the quality is really high for, the quality price ratio is extremely high. So it's something worth checking out. I can't wait to get back to the Valle de Osta. You have to check out my link in the description box. I will put an article up of tasting notes of some of these wines and check them out. Remember, drink adventurously, expand your pal, expand your mind. Guys, if you like this video, please subscribe to our YouTube channel, Exotic Wine Travel. I will see you in the next episode.