So, you need to connect the phone to the laptop, and then it's important to have a connection? Yes. To be automatic? Yes, you don't need to. Just use the phone. Yes, that's all right. Oh, nice. Then you have to use the phone to connect to the laptop. Oh, great. So, you have to use the phone to connect to the laptop. What's the problem with the picture? It's not very clear, if you look from the back, it's a little fuzzy. I can run a second projector, but with the focus I cannot do anything. This is the best projector. Can I run the second projector? I don't think we need the second one. The second one is more crisp. I think it's better because it's smaller, but sharper. We can try. Hopefully it doesn't break anything. If you enter the room, please try to close the door quietly so that we don't disturb the presenter. And please let me know in the short time who's going to be there in the talk. Please welcome. Welcome to this. I want to talk about the second edition of the conference. We'll talk about the short introduction to Loviat and all the components. Then we'll look at the difference in the tradition of the storage. We'll talk about few issues that we have to support. The future of Loviat. How many people are using Loviat or know about Loviat? I should know, I guess. I guess some are about SEPT. So SEPT is scalable, highly reliable, software-defined storage. Loviat was just only the locks for now. To use SEPT we are using Cinder. Because we had two goals when we wanted to accept support. We also wanted to support more storage in the future. And Cinder was to access to a lot of storage that we are not using now. We may be using the next version. And we support only SEPT back in Cinder. So we don't care what others thought in this version. To talk with Cinder, we'll be using OpenStack Java SDK. Cinder supports the project. And I will focus mostly on the SEPT integration. And less on the Cinder integration. Because this project was kind of a similar integration. I will focus on the SEPT side. I will talk more about the Cinder integration. So of the traditional storage, what is the traditional storage? This is a simplified view of the architecture related to storage. We have the engine controlling all the hosts. We have regular hosts like this. And the special host called SPM. As you know. The SPM is the only host that can change the shell storage. For example, if we have lock storage, the SPM will go and create volumes on this shell storage. While other hosts will consume this storage. For example, this host will connect to the iStudy server. And enable the MVs for the VMs. Or do mounts to exhaust the shell storage for the VMs. But only this host can change stuff. The SPM does a lot of complex stuff. And if the Cinder puts a fail, we are working on it. Hopefully we will do something. What is lock storage over the visual storage? We support iScazio's panel channel. And actually this is a video. The volume over the volume is the VG. Inside the VG we have special LVs for metadata. Like this one. We have special LVs for locking, like LVs and lcents. Inbox and outbox for messaging. Host can send messages to the SPM. And here we can see regular volume. And two other volumes that we use for VG. This is basically a block storage. What is block volume? On the bottom we have some SCSI devices. Which we use only via multiples. So in this case we have two passes to the same device. Both of these devices are actually the same remote device. This is the remote device that we can access via two passes. And we have another remote pass field. Both of them belong to VG. This is a block storage. Inside the block storage we have volumes using LVM. So we have one LV. And on top of the LV we created a QQ image. The last part is what the LVM is consuming. The problem with block storage is that we cannot create any amount of LVM. It started to become very slow. LVM was not designed to create huge amount of LVM. It's complex and how to manage. We have to do locking to do any changes in the VG. Only the SPM can do the changes. We have a complex way to roll back changes. There's a problem with timeouts. Static timeouts and multiple timeouts. It should be much better on PL-7.2. And of course we have the quality of service for storage operation. We are using the IO class. IO noise to run our operations. But we are not using the CFQ schedule. Actually we don't have any quality of service. The CFQ schedule is much slower. That is something that we should be working on. OK, how do we do the same provisioning? VPSL is not on the data class. We provide the device to the VM. And the VM is writing data. And we are promising about the event data. So we monitor the gives and what about the ideas. When we find that the disk is too full, we extend it. We extend it via the SPM. So if I'm a host running the VM, and the disk becomes too full, we send the message to the SPM. The SPM extends the disk, and we send the message to the disk from the standard. When operation is finished, we resolve the VM. It was too slow on the VM board. And we repeat. So this kind of looks... We're not going to do it, but this is how we do it in front of us. But of course it's limited. And because we are doing polling, we cannot scan to each amount of disks. It becomes costly. It's very complex. It's slow. The VM we post before we send the disk. And of course we depend on the SPM, which is a similar kind of failure. Okay, let's move on to file storage. This is Maxi Implea. We support a lot of shared file systems. File storage domain is just a directory, a meta directory. Here we see a file... a mounted file storage domain. And inside the directory we have metadata files and images. And file volume is just a couple of files inside the image directory. Here you can see the storage domain, image directory and image directory. And inside it we have single volume. This is the data of the volume, list file and metadata of the volume. And if you have a lot of volumes, you'll have a lot of files. Sync provisioning is easy. The file just go and leave it. Problems. We have operators of the file system. The VM wants to block the files, and we put it on the file system, so there is author if not leave it. We have bigger issue of timeouts, because NFS can block for minutes, and it's not accessible. So we run all the storage operation. We have 200 and set the same amount. In another process, we are using the IR process, which does all the IR for us. For example, to the directory in the list of files. And of course, we don't have any way to define quality of service. For example, maybe for locking, provide better quality of service, or for other storage operation, best quality of service, so we don't disturb the VMs. And of course reliability. It depends on the storage server. If you have one disk with NFS, and if this is gone, then that is one disk. If you have a smart server, that's the application or something, you're good, but it's not. Okay, how do we do snapshots? All snapshots doesn't have the same volume or same volume, they're the same way. We have a QCAD chain. The VM talks with the top layer, which is using QCAD format. And it has a backing file. The backing file can be all format with QCAD. And let's see an example. We start with one volume. We create the first snapshot. So we create new volume, and then we switch the VM to the new volume using the libger, if it's live snapshot. And the new volume backing file is the old one. Okay, so this is, I'll take a snapshot, we can take more snapshots. We can see that we have the volume of scale because it's meant to point out the page. So the issue is our performance. For a second, if QAMO wants to read the data, it will try to read it from this volume. And if it's not there, it will try this volume, and this volume, this volume. So it makes it read, sorry. Rights should not be affected too much because all the rights are going through the active volume. Again, the problem of scale, because we don't have a lot of scales, we're reaching the maximum number of scales that should be on top of it. So people, maybe, should ignore sort of domain so they don't have too many scales on the same sort of domain. And of course, if this parameter is too much, we have to keep the QAMO files in sync and the ADS, but it's not a lot of fun. Like in Edge. We know about it, it's really complicated. You can check Adam talking about it. Let's move now to something much more simple. Set storage. Set storage works completely different. On the host, VDSM is not accessing the storage. It's not organizing how to set storage. Only QAMO knows about the storage. Only QAMO can see the storage very, very deep. Engine uses single to create volumes at the same shop, to create volumes at the same shop. And single is... The set backend in single is doing the regression in set. So one important example, there was no SCM in this diagram because we don't care about the SCM in set. We don't use it. So this is nice improvement. Storage domain. There is no set storage domain on the host. On the engine side, there is a single provider which supports only set for now. But all the operations are done there. Here is the single. What is set volume? On the host, there is nothing about set volume. We set a piece of XML that we pass to the build. We get all the details from the engine that we get for the signal. For example, the pull and volume name, the host, the monitor, on this volume, and the authentication details. Same provisioning, the general thing to do, such as everything, snapshots. Nothing on the host, such as snapshots. The host does not know anything about the snapshots. You'll see that at the end you create snapshots. When we take a snapshot, we have a VM using the set volume. You can take snapshots or another snapshot. There is no tendency between the snapshots. The VM does not know anything about the snapshots. There is no way... I mean, there may be a problem of scale on the set side. I don't know anything about other real snapshots. But it's not our problem. So, for our site, this is the state. Live merge, there is no live merge. It's just a little snapshot on the signal and it does. The VM does not know about it. So, what are the benefits? Honey available storage, scale is very constructive. Everything is afforded to self. And we don't have to manage anything on the host. It's very little. Problem, it's not finished yet. There are some features that are not implemented. And where can we see that it can be tough? Because we see that there is a bit of an idea about storage. What are the options? So, we have to fight it sometimes. Self may be too complicated for the whole scenario. But it's important because it's only one set. Probably one set of view points. And scale is the problem. And in a way, the signal is our newest VM because now it depends on the signal. So, if the signal is down, we cannot do any source operation. Okay, now let's talk about how do we connect securely to self volumes. The standard way to do self-advocation is to create a keyring file and deploy this file on the host. I think the self-advocation is very useful for you. But you need something more secure. We don't want to deploy a piece like this on the host. So, what we are using is liquid secrets. You can register a secret in Lizard and we are using the general secrets and private secrets. It means that the secret is defined on the memory in Lizard. And if you get three stars, the secret is gone. So, it's never persistent on this. And private means that you cannot get the secret out on Lizard. Lizard will give it away to KMM, but normally you can take it out on Lizard. But of course, you cannot define this on Lizard and forget about it. Because when you get three stars, you use the secret. So, we need to manage this. So, what we do is we manage now the self-advocation key. You can add a set piece to engine. And engine will register the secrets for you along with the secret. And then the secrets will be out on Lizard. To allow this, we add new APIs. There is a register-secret API, new video center, and unregister secrets to multiple secrets. And engine will register secrets when host is activated or storage is activated or when we reconnect to host. This is the same way that engine is managing the storage connections. So, in the same way that we make sure that the host can talk to storage, we also make sure that the host has all the relevant secrets. And when hosting is put to maintenance or sold to municipal maintenance, you need to remove all the secrets that are not here.  Okay. Life's natural. Sometimes you want to capture the moment. To do this, we need consistent snapshots. We don't want to keep snapshots that are not useful to store later. So, to have consistent snapshots, we need to flip the guest's participants. Then we want to take the snapshot. And then we want to tell the guest for a system. And now we know that we have a good snapshot This, of course, requires the guest agent, the QMU guest agent. This is also for traditional storage, there is no difference. Although we do it with old storage, engine creates a new disk, and then we take a snapshot. Bigger sandwich, we have split bills to take the snapshot. We need to squeeze the guest's participants internally inside the snapshot valve of Libre. And finally, Libre will switch the VM to the new disk. Now, this is, we're not well processed because this phase should be here. So, we cannot use the old port for this. We have to make it different. And now, engine controls the time flow. So, it will not depend on Libre freezing. And we make sure that we freeze the guest file system before we created this snapshot. Because at the point we created the snapshot, the VM is not writing for the snapshot. It will write to the next one. So, we want to make the file system consistent and only then we have to snap it. So, we added the new PI to VDSM, freeze API, that is going to be a Libre FS freeze. And to the snapshot API, we added force and flag. So, when we take a snapshot we can tell VDSM if the VM is forced and or not. And finally, we added the tau API, again using Libre API. So, now the flow is like this. Engine first freeze the VM, create this snapshot via single, take the snapshot in VDSM, which may use, the VM may have set this in original VDSM and then you want to take the new snapshot or header. And when you finish, we target the VM via VDSM. And when we are taking a snapshot without set VDSM, we just tell VDSM take a snapshot with force and force and VDSM do the same. Any questions? If you make the whole thing agnostic and do the freeing and throwing in QA mode per se, you suffer from time loss and the guest operating system, wouldn't you? Yes, but there is no other way. You must use the guest agent. Yeah, that's what we do now because of such implications I just mentioned, was there some other design flow going down that path in order to avoid guest agent presence or more of the different guest operating systems? If you open the guest agent, we want that to consist of snapshots. You cannot do anything about it. If you want to consist of snapshots, you have to install the guest agent. Oh, we know you can use the VDSM. VDSM is the limitation of sections. I can answer the problem is that the guest does not just pass up, it does not give you memory and it doesn't drop down. Right, I know. It was, and I presume that we have issues with that approach with respect to timeouts and the guest operating system, that you essentially free the QA mode as a central place, presuming that you could do the whole cycle quickly enough that the guest doesn't hit a timeout an IO timeout. Of course, you wouldn't have the perfect, you know, say, content because the guest would have data and memories, but he'd probably suffer from the timeouts anyway because he can't rely on the particular guest behaving, you know, with respect to timeouts. If you went through it quick enough, you could actually avoid all these, you know, guest agents, like I said. Yeah, the problem is that the guest has the workflow and the crisis that the guest has to crash. So yes, it doesn't make possible for the guest to crash. Essentially, your snapshot will contain the half-written contents and your snapshot will receive the rest, it might be free, but that's just an idea because I was afraid of the timeout implications anyway. If your switch takes too long, right? It will continue to the edge, like it's sinking there. Okay, so there are a few future directions. First, a missing picture that we don't have yet. This is not a completely these are the most important things. Move this. It won't support moving this from safe to safe. From safe to another storage and from another store to safe. Should be impossible. I hope using liberal media without involving the mountain or such a disk. Life-sort negation. It's the same I do this, but I do it live, authenticated. Here, hopefully, we will support it to switch from a lot of disks in my projects. If it does, we can do it. Disaster recovery comes here. We obviously persist all the evidence in storage, but in such we do not create the obvious store volumes. We probably can do, but we don't know it now. And we cannot connect to such a storage domain with the cover of the evidence from the storage domain. Like we can in a stash or in a test or any other storage. Monitoring is an important point. In the regular storage, we have a lot of monitoring of the power of the storage domain. The monitoring is not perfect for me, but we can know what happens with this storage and if the storage is not available, engine knows not to try themselves here on this storage. We set, we don't have any monitoring. We open the whole site and we don't arrange this site. Everything is always up and operation will fail. Secret management we want to do the secret management that I that you see before is still complex. We don't want to manage secret or host. What we want to actually do is just send the secret to the host that needs to run again or needs to not to ingrate the VM to the source. Define the secret that we are leaving on the source start the VM or finish the integration and we are done with the secret. All the management that we are doing is just a waste of time and effort. So hopefully this is something that we can do in the next version. Of course migrating and other because we need to ingrate the VM we need to make sure that the same XML that the VM is using in welcome destination. For the secret that we should define should use the same UID but this is the UID that the this is what we look for when we try to connect to a single signal is having different views on storage for example like to decide for you what kind of storage you want so if you want to set storage signal you can decide why don't you take a VM storage or a data storage or NFS and we are not prepared to handle this we are not know why we don't use it we are not exactly our storage but sender will let us configure it so we must depend on the specific configuration and we can be at all with this specific configuration so one idea was to provide our own signal and control this sender instance with our agent so we can control the configuration and we can bypass sender a different view of the world and we can also hide sender completely behind the engine line which is something that customers really like simply why and simplify the line instead of this is one idea I'm not sure if we are going to do it because there is another idea right of going to talk sender we don't need a middleman we can use people with ease it's very easy to create volumes and snapshot and everything even if it's byte on byte things it's really great but the problem is that we want all the nice drive of sender support so we will have to think about this and if we think a few years ahead maybe we should go on a business storage it's so complex how to maintain easy to break so why do we need it if this storage is good because we can support the stupid disk we don't need any smart servers on the server side but we can use smart server like this this is the direction that the world is going smart servers not stupid disk I'm not sure that we will have this in case questions the sender service where does it run how many of those what the sender service where does it run where is it something that you should deploy in your setup you can have what you want you have to install you have to install a sender and whatever other services of the sender and the wires same like said you are responsible for installing sender and some such and we consume it or we may be not good at this like the high stack server I can go to install nice sender or infest like I said the sender so the open sender someone has to talk you don't know what the configuration is you don't know what the version is you don't know what the version is you don't know what the version is you don't know what the sender applies to sender or tail or sender that is the tool that we make it easier than you install your own sender sender you are thinking about making any that you want to install a little bit separate no, in the traditional story I think you can have content inside of an awesome sender sender sender sender sender back in the days that's kind of different places and ecosystem back as partners to give us all the space but it's not a black feature that we have out in the world so it's a little bit different you can also do it yourself because the rest of the time the question is the same so what is the difference between the two if you need to install a bit that's right you can do it with the same feature you can make a feature you can do it with the same feature where is the sender where is the sender what is the difference with the sender sender sender sender sender sender Everything alright? Great.


Helo! Thank you for hanging out so late on a Saturday night and there's lots of other things you could be doing in Bernal or at this conference. I appreciate you turning up. So today I want to talk to you about learning to rank. So raise your hand if you've ever used a search engine and not found what you're looking for within the first few items returned.  And next thing you know you find yourself clicking on that dreaded button to take you to the second page of Google. So contemporary search engines are often good at finding all of the results for a given query but less good at identifying which ones are the best and thus should be returned first. And search engines are everywhere. We're not just talking about Google and Bing here. They're just a subset of the bigger class of search. Looking at this sample of places where you'd find search you can see that good search functionality is going to lead to happy customers, better business revenue. And so the task of ranking documents quickly and accurately is very important. It was at a conference last year when I first started learning about learning to rank and from there I've just gone on to learn a little more. Today I want to introduce the topic to you. I'll tell you the things that I learned about and I'll point you to the resources that you need in order to get started and implement these algorithms in practice. We'll look at the standard machine learning algorithms for learning to rank and we'll look at implementing the algorithms in Python specifically using the XGBoost module. To begin with we're going to outline the problem at hand. So they're all on the same page. So the first thing we have is a query. I'm assuming that this query is text-based. So it's going to be the phrase or the words that you type into the search box. It might be in the Netflix case, Christmas films or Christmas films starring Hugh Grant. Everyone happy? Sweet. In addition to the query we've also got a set of documents. So each document might be a film title, a blurb, an abstract, a list of actors. It might be a web page or it might be a standard issue document. There's likely to be some sort of structure in that document. So I'm assuming that it's got a title, a subtitle and it's probably comprised of sections. There's also likely to be some data specific structure in there. So if we're thinking about web pages I'd probably count the URL as part of the document. So I'm using this term quite generally. The goal of learning to rank is unsurprisingly to rank the set of documents. Any learning to rank algorithm aims to return for a given query the set of relevant documents in order from best to worst. So an age old method for learning to rank would be full text search in a database. So in this case your documents would probably live in a postgres database. And as we mentioned they've got this fixed structure, title, subtitle, paragraphs and so on. So full postgres text search looks at where within the document the terms in the query are. So here I've circled them and then that is used to rank documents from best to worst. Something that's nice about postgres full text search is that the user can do some tuning. So if you know for example that in your case the words in the query appearing in the subtitle is for some reason more important than the words appearing in the title you can indicate that. But that's going to have to be done on a query by query basis and it's going to need some human intervention there. So it's fine for some examples but it's not really going to work on large databases or more complex queries. Another standard ranking algorithm is page rank. So page rank is great for many things and sadly it often downwards pages that have few URLs in them, few hyperlinks and so on. The algorithm also uses wisdom of groups, wisdom of crowds to learn and to make improvements. And I think we can all think of recent examples of where wisdom of crowds is not a good thing and a crowd making a bad decision leads to more people making a bad decision. So what we really want is for an algorithm to be general we want it to be robust and we want it to not need any query specific tuning. So that's why we're going to need to think about machine learning techniques in this case. All of the machine learning techniques begin the same way. For a given query document pair they try to capture the overlap between those two. So this is done by making a feature vector. A feature vector is a numeric vector of values that holds information about those two items. So how do we get this feature vector? Well there's going to be some features that depend on the query and the documents. These are known as query dependent. They might be things such as how many words in the query appear in the document, how far into the document do we have to count before we hit a word in a query and so on. And there's a nice statistic known as IDF or inverse document frequency which is often used to measure these kind of things. How often words appear in different parts of the document. We've then got some query independent features depending only on the document. So these are things like the length of the document, the number of times any user has accessed that document or the number of hyperlinks in the document. So for different applications some or all of these features may be relevant. The final set of features are query level features. These are things like length of the query and they depend just on the query themselves. So actually deciding what to put in these feature vectors is a very hard problem and it could be ten different talks. So for the purpose of this talk I hope we can all just assume that we have a nice feature vector that holds this information and we are good to go. Once we have our set of queries and documents and we've made our feature vectors we're nearly ready to learn to rank. All that's missing is a score which captures the relevance of that document for that query. So this score is going to be user imputed so we're sadly going to need somebody to go through the set of potential training query and features that we have and rate how good each of these items are. So this is a supervised machine learning technique. There are some ways to compute those scores on the fly but we're assuming that someone has gone through and given us a training set and said hey this document is good for this query, this one is not and so on. So the first class of ranking algorithms I want to talk about is point wise ranking algorithms. So they work by taking each document one by one and using the feature vector for that query document pair they go on to predict a relevant score. They predict how relevant that document is for that query. Once this has been repeated over all the documents we're able to order the documents by their predicted relevance from best to worst. Thus we have our ranking and we're done. So the missing piece of this puzzle is the model which is a function that takes us from that feature vector and takes us to the score or relevance. So for the purpose of this first illustration I've just simplified and assumed that we've only got one feature in this case but you can imagine this is multi-dimensional. Now we don't know what the best function, our model, will look like but we don't have to. So machine learning is the process of automatically generating the function given heaps of examples. And there are many machine learning techniques and arguably the simplest involves learning a way to add and multiply these features together to give us a relevant score. So you may remember linear regression from a statistics class and that's all we're doing here. Linear regression in this case is giving us a model and a way to turn our features into a relevant score. And we also have a way to evaluate the quality of the model so we could compare for the features where somebody has gone through and given us the true score. We can compare the prediction that we get from this model to the truth that was given. So once we have our model we can then take in a new feature, pass it into the model and get a score. There's lots of other ways we could have learnt this function so we could get a different model by learning a curve and I'm sure you can think of your favourite regression type to give you a different model. Given that there are a few possible models, how do we compare them? Well, since we're working with a training set where some human has given us the true score, we can just compute the difference between the truth and the prediction. That gives us a way to evaluate our model. So that's fine but that only really captures part of the picture because what we don't care about is the scores. When you go on to Google you never see those scores return to you, right? All we really care about is the order of the document. So here we're going to look at an alternative distance metric we could use which takes into account the ordering of the documents rather than the score. So to begin we take the predicted scores that were given and we use those to rank the documents. I'm then going to replace the scores with the rank. So the item on the left becomes rank one because it was ranked the best, the next one down is two, the next one down is three and so on. Now we know the truth in this case because those scores have been given to us and so I put the truth underneath. This distance metric is just going to be how many permutations do we need to make in order to get the true rank in order. So we're going to go from that top line being in order to the bottom line being in order. One, two, three, right? So a distance metric under this distance is three. So we've seen two different ways of evaluating the model now. One looked at the scores and one just looked at the order of the returned documents. There are two things I want to call out here. So clearly just looking at the scores doesn't answer the question that we care about. We care about the ranking of the documents. But just looking at the ordering arguably isn't perfect because I care much more about the first two documents being out of order than I do the last two. If we think of a searched engine, if we put something into Google, I care much more about the first page being nicely ordered than I do about what's happening on page 50. I hope I never have to see it so we don't mind so much. So the distance metric that is commonly used in learning to rank is called normalised cumulative, normalised discounted cumulative gain and it takes all of this into account.  It couldn't be worse. That's a disaster. We don't want that. And a 1 is perfectly ordered. Thank you very much. So it takes into account both the order of the documents, the score of the documents, and it penalises incorrect ordering near the top of the list more than it does near the bottom. So in order to compute this metric, we need three things. We need the predicted rank, the true rank, and the true score. And then in general it takes that rank list and it associates a weight with each of these returned documents. So the weights are returned as this ratio and here's an example of how we would compute it with sum it up and get a distance metric. Now in order to normalise it, we divide it by the optimal discounted cumulative gain is what you would have got if the list was returned in perfect order. Normalising is really useful because it enables you to compare across different queries and across different data sets. Another thing that's really nice about NDCG is that you can truncate and get a good result. So for example, we could just say, okay, what we really care about is the top 10 items being in order. That's reasonable. So we just compute it at 10 and we look at the top 10 items and get that result. Okay, so I normally, I motivated the use of NDCG by saying that we don't care about the predicted scores only we also care about the order of the returned documents too. But that point wise linear regression method that we were using before was just predicting a score and that's not really what we care about. We care about the order as well. So taking this idea one step further, the next class of ranking algorithms I want to talk to you about is pairwise ranking. So these capitalise on the fact that we care about the order and they take in pairs of feature vectors and estimate which of the pair is more relevant. So the first pairwise ranking method we'll talk about is rank net. If you care, it's a neural net and it has many more parameters than the regression model that we saw earlier. Hence it's more flexible, but that also means it's more complicated. So during training it takes in pairs of feature vectors and it learns the function so that the document which is more relevant is given an increased predicted score and the one that's less relevant is given a lower score. Now because this model training pushes relevant documents up and pushes poor documents down, rank net is good at getting documents into the correct region of the list and necessarily ordering them well within that region. So for example it's greater ensuring that the top document is perhaps in the top 1% of items returned but not necessarily right at the top. Furthermore this method is working by considering pairwise error. It says it's document A more relevant than document B. If yes we push A up and B down. And it doesn't take into account whereabouts in the ranking that is. So it would push both the top pairs and the bottom pairs by equal amounts. But earlier we said we care less about what's happening at the bottom of the list. So is there anything better that we can do? Well yes, lambda rank does better. So lambda rank is doing the same thing as rank net but it scales the change in the weights and the pushing in proportion to the NDCG. So if we go back to this picture, rank net was pushing all of these documents equally whereas lambda rank aims to give a smaller push to the ones down the bottom because we don't care so much about those. And later we'll see that rank net does much better than, sorry, lambda rank does much better than rank net in practice. So the final class of algorithms that we're going to look at is list-wise algorithms. So we saw point-wise where decisions were made on an item-by-item basis. We saw pair-wise, which trains itself by taking in pairs of documents and making decisions. And unsurprisingly, list-wise is going to take in the whole list of documents at once. My favourite list-wise algorithm is lambda mark and it uses decision trees to score and therefore to rank the documents. So a general introduction to decision trees. They're effectively a map that splits up the dataset at each node based on some feature. So, for example, we might send everything, every document with a feature vector such that feature one was less than 10 to the left and everything where feature one is greater than 10 to the right. And these splits continue until we reach the point where every document is at its own node on its own at the end. So the algorithm itself is learning these features to split on. And in this picture here, we've seen the example just using one decision tree. But in practice, the method is actually using an ensemble of trees. So we have multiple trees and each tree splits on a different collection of features. And so combining them, we're using a model that is much more robust and we get a model that's greater than the sum of its parts. So those are really the three main types of algorithm and hopefully you've got some intuition as to what they're doing. And now I'll just point you to the relevant Python libraries and show you how simple it is to actually implement these in practice. In order to do that, we're going to need a dataset. So the dataset we're going to use is this Microsoft Learning to Rank Web 10K dataset. I've selected it because it's open source. There's 10,000 queries in there and many more documents in there. So it's loads of data to play with. So each feature vector, each document query pair has been given a relevance value between zero and four. That's the score. So four denotes. It's really relevant in zero denotes. No, we don't want that, thanks. And we'll have a little look at the schema in detail in a moment. So there's about 2.6 million rows of data in there. So there's enough for you to mess around and do something dangerous. So if we take a look at one of the rows, it looks like this. So this first number here is the relevance. So they've been labelled by a human. Next, we've got our query ID. So the query ID in this case is 13 and it's preceded by QID and a colon in all cases. And then everything after that is the feature vector. The feature vector is of a form index of the feature and then the feature value. So here, feature number one takes value two and feature number 136 takes value zero. In order to use this data set with the Python library straight out of the box, we want to clean this data up a bit. So we want to remove the colons and everything before it. So we just want the raw feature vector itself and the raw query ID. So I'll point you to a little bash script that does that later so that you can try it for yourself. Now, the first algorithm that we talked about is linear regression. It's always good to have a baseline to compare to when you're using the more complicated machine learning techniques. So here you can run linear regression in Python. There's loads of different ways that you can do it. But what I've done here is assumed that we have our test set, which holds our data. So we set up a linear regression. We tell it it's going to take in the features and it's using those to predict. And we've got the score values. And then you just use the transform function. Nice and simple. All of the other algorithms are implemented in this XGBoost module. So XGBoost is an open source, optimised, distributed, gradient boosting library. Why do you care about that? Well, when we're talking earlier about the parameters of the model and learning the model, what's actually happening is some weights are being updated in those models. So XGBoost is fast and works nicely for this. The way in which XGBoost works is that you pass a parameter vector in. So max depth in the algorithms that were the neural networks, which were sort of sideways, then the depth is the number of layers we have in our neural net or the number of circles we have sideways. And when we're talking about the decision tree, the depth relates to the number of trees that we have. ETA is a parameter that prevents overfitting of the model. So you never want your model to just regurgitate the practice data set that you saw you wanted to be able to do better. And so by tuning ETA, you ensure that you're not just overfitting the data. But the key important bit is this objective function. And that's where you set which algorithm you're using. So rank net is by setting this rank map. Lambda rank is setting rank pairwise. And lambda marked is by setting this ndcg. So you can then go along and train a model using the XGBoost train function very simply. And prediction is equally as straightforward. So I have a little notebook that has some of this stuff in. So you can go ahead and try it out, mess around, and let me know if you hit any errors or you like it. So something nice about the XGBoost module is it lets you visualize these decision trees. So here we see an example of one. And you can see that first it's split on feature 65. So it said if it's less than some value, we'll send the document to the left, if not to the right and so on. It's always nice to be able to visualize what your model is actually doing. And another thing that it does is it tells you which features were most important. So it's a nice sanity check to be able to say, you know, I would expect that the query term matching strongly with the title of my document is really going to have a positive impact. And we'd like to see that. So this tells you how important all these features are. The two features at the top, 134 and 108, you'll have to take my word for it. But 134 is the frequency of the terms and the query appearing in the title of the document. That sounds good. And 108 is the number of outlinks or hyperlinks in the document. Well, that sounds good as well, because we know that algorithms like PageRank do work well for cases when there are such hyperlinks. And finally, we can have a look at the NDCG. So that was our distance metric. And we said that one means that your list is returned perfectly ordered. Zero is terribly. And you can see that as we move through the different models that we looked at today, the algorithms give better search results. So I hope this has motivated you to start learning a bit more about learning to rank. We started by looking at some simple methods that already exist that don't use machine learning. We talked about feature vectors, which are present in all machine learning techniques. And then we went on to look at the different types of algorithms and how to implement them in XGBoost. Please get in touch. So I'm at Salfwatts on Twitter or GitHub. And I'm Sophie at Red Hat because there's not enough women in tech. Thank you.


Okay, module five will be also talking about variant coding, but another type of variant coding, which is a structural variant coding, which I have to say as my preference, it's more harder, it's more complicated, it's more wider in terms of what they are covered, but it's more challenging, but I prefer it. So in that picture, I will try to make you understand what are structural variants first and how we can discover them using NGS data. And to that, the different strategy you can have, what are strong and what are weakness and a bit of how we can see these variants in IGV. So what are the structural variants? Genomic rearrangements that are usually more than 50 base pair long for indels, and could be so deletion, insertion, inversion, mobile element, modification, duplication, or translocation. So just to give you the details, at the beginning, since recently, structural variants, and especially copy number variation, were detected like that. So by fissure approach, carry typing, so it was not really evident, also a bit of some array, so it was not really an efficient way and a good resolution way to detect structural variants. So it's where the next generation sequencing has really been a game changer for that type of variant. So just to give you an idea of what's the impact of structural variants, a lot of people that are looking at structural variants are most of the time working in cancer, because in cancer, there are a strong, strong impact of a structural variant and then copy number. So this is an example of a carotid of a cancer patient. And you can see that chromosomes are totally break out reassembled and duplicated or deleted, and it's a big mess. So it's really interesting to study structural variants when you are working on cancer projects. So the class of structural variants, so we have the copy number of variants, so the syngvies, so deletion and duplication, so it's a large segment of the genome. We have the copy-neutral rearrangements, inversion, translation, and the other type of structural variants, so insertion of new sequence, like a virus or whatever, and the mobile element impact. So this is the same classification of structural variants, but what they represent is what they represent in terms of genomic structure. So what is really important to understand when you study structural variants is when we talk about structural variants, we talk about what we see in the sample when we compare to the reference. I mean, here we see deletion, but in fact there's no deletion in the sample. The sample is DNA like that, it's deleted when you compare to the reference. So the reference has something more, when we have an insertion, we have something more than what we have in the reference. So think about that, it's always referred to the reference genomes. So here you've got the insertion of genomic sequence or mobile element, tandem duplication or in-sponsored duplication, inversion, or transportation. So as I say, NGS was a game changer for that kind of analysis. So we did prototyping, then we did fish, then we do for the CNV, we do then CGS charade. Then we did steep array, and now we are working with isoput sequencing. Just to give you an idea of what type of information we have when we detect CNVs. So CGS array, you just look what the amount of DNA you've got at each position. When you look at steep array, you take the same information, but you also look at the analysis frequency. NGS, a lot of methods for NGS are similar to what they've done in CGS array, because you look at the read distribution for copy, but we now try to develop methods that are doing a mimic of what we have in the steep array. So when we detect method in NGS data, we can detect almost every type of variant. So the point mutation, as we did previously, but the indels, the copy number, or the other type of structural variant. So what are the strategies we use to do structural variant detection? So there are four major strategies. So the read pair strategy, I will go in detail of each method after that, but the read pair is to look at how you read maps between the two pairs of your reads, the read depth, accumulation of reads, split reads, how if you have a break point, how this will impact the internal sequence of a read. So read pair is really the two pairs of one of the other, the split read is inside the same read, or you have a separate method, which is the assembly. So read pair, how it works. So the idea of the read pair, you identify the break point of your structural variant by looking at your alignment between your two reads. So when you're done your sequencing library, you have created your library with a specific fragment side. You know you are around 400 base pair, for example. And if you span a variant, you expect that either your read will be, if you will be too close or too large based on what you have sequenced. So the idea is to look at this variation of the intersize to determine the structural variant. The main limitation of that is the sensitivity. And the sensitivity is based on how you do your size selection. If you have, so how you use size selection, I mean, what does standard deviation and the size of your read and your intersize. So when you look at your intersize, you look at the distance in your, the distance of your fragment, the size of your initial fragment, the size is supposed to be there. And you look after mapping. So you've got this kind of Gaussian distribution around the mean of your intersize, what you are supposed to have selected. And you take your distribution. So if you have the wider structural foundation, the wider your distribution will be like that, okay? So it's really important to have a tiny distribution. So what you will see is, okay, I've got my distribution of my read. No, I think that, for example, two standard deviations will be my concordant distribution, my blue distribution, and everything that is out of that, the two-tail will be my discordant reads. So what I do for that, I take this discordant read and then look where they are in the genome, and I try to find cluster of reads that are discordant and that are clustered at the same location. And that gives me several evidence of the same discordance. And then I will call my structural value. So how it works, if you have a concordant read, you've got your sample genome here, you've got your reference genome. So you expect to see in your initial fragment the same distance in your reference. Now, if I have a deletion, so if I lose part of the reference genome, my fragment is always the same. It's always the same time, because I select my set side. But when I map it, as I lose a fragment from the reference, my two-read will map really with a large insert size. Same for an insertion. So I've got my fragment which is the same size, but I've got a fragment that is not in the reference. So we see the fragment that will be mapped to close. So we can do it really easy for insertion deletion. Insertion is a bit more tricky, because due to the size of your reads, usually you have a really low resolution and it's hard to do insertion. It's better for looking at deletion. It's easier, because here you are limited by the size of your fragment. If the deletion is too big, you will not be able to map your read onto the genome. So now, if you have a tandem duplication, what you will see, you will have two copies in your sample. And you will have a fragment that will go over the two copies of your fragment. And what you will see, you will see this fragment that will go over the two copies, the read, the repair, will match in a larger way, but at the opposite direction. So you will have normal read there, because you have normal fragment here. You will have normal fragment here, but you will have some of them just pop up and tell you you have the tandem duplication. Inversion will be quite a similar pattern as this one. You will have read with larger inter-size, but only one of them will be inverted. But you should expect to have a set of evidence here and a set of evidence between that one and this one. You will not only have one break point, but you will have the evidence for the two break points. Now, if we look at tandem duplication that's already done, inversion. So you have the second break point here. Now, if you have a large insertion, it will be more complicated. Because if you have insertion from a other distance, you will have a signal that will be not sprayed on the same genome, on the same region. So it will be hard to differentiate between translocation and the insertion. So the way we do to detect that is we expect to see two sets of translocation that launch to the same region, and that's past two other locations. So this is how it works. So really simple, but with the limitation of the size of the reads, the limitation of the standard deviation of the size selection. So to do that, you have many tools to do it. Like Breakdown Third, one of the first tools that have been released, and a lot of the other tools. Today, we will use DELI to do that, which is a really great tool. You will understand what after that. So just to tell you, when we detect structural variance, it's in the population. When we do apply a method, before we find, we find a lot of them. So the method to detect structural variance with a read pair are really efficient, but they also provide a lot of false positives. So you will need to do a lot of filtering of your data. Another limitation with a read pair is when you have a complex region, like this one, you don't know what you have. You just have crazy evidence of many events, and you cannot really make sense of what you have. So just to summarize a read pair, the weakness of the method is it's really difficult to interpret when you have a complex region, which is most of the case in repetitive region. It's difficult to characterize early re-arranged region, and you have a right-high trait of false positive. The strength is, theoretically, you can detect all types of SVs. The second approach we can use is a split read. So the rationale of the split read is you take reads, and you look at reads that have been break into pieces. And then you try to make sense of these reads. So you try to see reads that break into few pieces, one, two pieces, and see how they map together. And you try to find cluster of evidence that show you the same pattern, and you can call the variation. One of the major advantages here is you are really looking at the break point. So you are really precise. And one of the people that are working with split reads really recommend to do longer reads, to exercise the longer read, because you have more chance to split your reads. Because if you split your reads at the one end of your reads, if you don't have enough reads in one of the parts, you won't be able to map it. So you will just cut your read and don't know where the other part of the read will map. So the longer you read, the better the efficiency of the split-read method will be.  Most of the pair align correctly, but some of them map usually one end correctly, and the other are breaking two pieces. And you take that to do the calling of the variants. So what are the signatures that you can observe with split-read? So here, it's the pair done mapping. So what we see in the previous methods. So if you have a deletion, you expect to see a larger indel, but you expect also to have a read that in your donor is like that, is one sequence. But in your reference, the first part of the read will map to the genomes, and you will have a break, and the second part will map later on to the genome. For insertion, you expect to see in pair done mapping to have, in read pair, to have the intersize, which is shorter. And in the split-read, you expect to have one part of the read that is mapped, one part that is deleted, and the third part that is mapped to the first one. So it's where you see here, for that kind of signature, that the longest your read will be, the method will be able to detect this kind of event. And it will also give you a more resolution to take larger insertion. Then when you do inversion, you will have the signature for the pattern mapping. So you will have the inversion where your reads are large insertion and inverted, and one is inverted. Here, you will be able to see read that map, a cluster of read that map at one end, and that are inverted, the read position in the other, and the second break point. So as you can see in that presentation for the split-read, I always compare, show you, split-read versus pattern mapping. So because now the tool, the new tools, is the way to analyze the data is to mix both signals, is how the new tools are working. So it's important to not only use split-read, but to use split-read and pattern mapping at the same times. Because if you do split-read, it gives you also a lot of false positives, but not the same as what you have in read pair, and it's really slow. So a lot of the tools will go first with read pair, and then add the split-read on the action as an additional layer to confirm the method. The advantage is you can have better detection and have the break point detection at the same time. So the split-read tools is this one. And you see, oh, we have the same daily, which is a tool that do split-read and pattern mapping. It's why we will use this tool. Another really good tool to do it is Olympies that do both of the techniques. So summary of the split-read strength and weakness. So in terms of strength, it works really well in pair with the read pair methods. It gives you best pair resolution, and it can detect a really short insertion that you cannot have the read resolution to detect in pair and mapping. So if you have a really short insertion of deletion, read pair you will be under the standard deviation. So it can give you a better resolution. Weakness, it needs more coverage than what you have in the read pair. And there's a lot of false positives. Third method is the read depth. So the read depth is mainly used to detect insertion deletion. It's the traditional method to detect so copy number variation. So how it works, it's based on an assumption that you have an homogeneous distribution of your read along the genome. And if you have a copy number variation, if it's a deletion, you will have a decrease in the amount of read you will see at the position. And if you have an increase, if you have a deletion, you have a decrease. So it works really well, except that this assumption is not always true. So how it works, you divide your genome into a bin of equal size. So you can choose your bin. It will really, this choice will have an impact on the resolution, but also an impact of the noise and the false positive. Then you estimate the depth of coverage in each bin. And then you look for a cluster of constitutive bin that show you either the significant excess or loss of coverage. So it's where the NGS meets the methods of CGH array, which are the measure of depth. And you do exactly the same. You do segmentation of your genomes. And then you look at the variation of coverage.  If you have a deletion, you expect to see almost no reads. If you have a notification, you see excessive reads. In terms of what you see in IGV, when you have a good event in a normal sample, a sender is really clear. You can see that you clearly see you have your general mean coverage and you have this excess of units. So it worked really, really well. What you need to do is to correct for GC because you have a GC bias of sequencing and use any segmentation tools like we do with CGH array. So just to give you an example, this is a cancer sample. We use it on cancer. It is a tool that I developed, which is called SCONS.  And you just do the ratio of the view and you plot. And you see it's really in case where it works well. Because in cancer, it's really often that it's not working well. The pattern are really evident to call. But here, as you can see, when we look at the data, what we tend to do is to look. I got my genome. I got my point. And I tried to look in a point that are different from the rest of the genomes. That's a major weakness of this method. It's because it's how you normalize your data. So most of the data makes an assumption, as I say, that you have a much-needed coverage. So this is what you expect as a signal of copy number variation. You expect your genome with a constant homogeneous coverage. And then you have a region with an excess of coverage for an amplification. Now imagine you have this point. It is really an amplification. But instead of having this homogeneous coverage, you have this coverage. Trying to find that this is an amplification becomes really, really more tricky. It's because we are not, as I say, we don't have the assumption that is almost homogeneous all over the genomes. Several ways to avoid facing these results, taking large-housebin. If you take large-housebin, you will normalize the coverage you measure. But you will lose some resolution. So there is another method that have been developed in the lab at C2G, which works fantastically to avoid that in one condition to have another sample. So the idea of this method, which is called Popesville, it's instead of normalizing your count over your neighboring bins, you take a population of sample and you normalize vertically. So you're looking for this particular bin in several set of samples in the 10, 30, 200 sample. What are the variations of coverage for this bin in all the sample? So you've got the local variation and the distribution into your set of control sample. And in red, you've got the distribution in your sample. And then you are able to take into account this local and non-nomageal use distribution of the bins. So if you have enough free, I would say that this method beat my method for sure. If you have enough sample to work on that, yeah. What's the minimum number you think that you have to be valid about? That's a tricky question. It's a PhD student that developed this method in the lab. And I still say, you need to assess the minimal number. And we don't have a minimal number, but from experiments up to 10 to 15, we're starting to have really good results, especially in a low-mappable region and in a repeated region. You said that the bin size, if you have a right bin size, it's going to work better, but you might miss some information. So if you do some kind of estimation with an automatic estimation of your camera size, does it improve the results? Yeah, it could. But the thing is there's really that local variation. If you look at the coverage, if I see the images of the application, you see it in every 10 basis, the coverage could be clearly. So the channel will probably have trouble to find the right size. Because it's a local region will have not some variation, some other not. Because you could have a local estimation of your camera size. Right? Yeah, probably. Right, your tools. Yeah. In hyperploidy, usually hyperploidy is when you're missing cancer, or you talk about hyperploid organism. So in cancer, usually what we do, we estimate the ploidy to be around before running two percent. Yeah, like we use a tool like Sequenza or Buttonberg. We estimate the look at how the variation, the data frequency of the region, we estimate the frequency of the ploidy and the ploidy of the sample. And then we use that to, as a parameter, in the copy number analysis. Is it the same sequence machine, exactly? Everything has the exact same parameters starting the sequencing so that any variation from the sequencing is from that sequence. Yeah. It just pulls available needles off. Yeah, you need to have homogeneous control, as usual. Otherwise, it will work for a lot of things, but some means will be crazy. So the different tools, so many, many tools, not so many tools that are dedicated to cancer. And most of them do a good job. So most of them do a good job in a normal population. It's where you go to cancer and repetitive regions that it starts to be difficult. So the summary of the read-depth approach. So it's a relatively low resolution. The breakpoints are ambiguous because your breakpoint will fall in a bin. So you know that your breakpoint will be approximately in that range of bins. So depending on what the size of the bin could be, if you have one kb bin, you know that your breakpoint is in one kb region. So it's not easy to detect exactly the breakpoint. And you cannot detect a balanced rearrangement. The strength, fast and simple, it's really easy to interpret. As I show you the graph, when you look at the graph, you can really say, OK, I've got to send it there and there. And there's a lot of methods, not a lot, some methods that do machine learning in normal to do that. It's work well in normal, it don't work in cancer. Because the problem of cancer, it's simple, it's specific. It's simple, it's on purity, it's on cellularity and everything. So the strength is to determine the copy number. So you can really tell you how many copies you have. And with new methods, you can have also information in the repeated region and low coverage region. The last approach you can use to do SV, Structural Variant Detection, is assembly. So the idea of the assembly, you have your genomes, you don't care about your genome, you take your reads and you reassemble them. You will see this afternoon how it works to do the assembly. The idea is to create a new reference specific to your sample. Then you take these long contigs and then you map this contig to the reference. And you see, oh my, I got one contig in one pieces and it's breaking a few pieces. So you know that you have deleted the region. So the idea is really simple and efficient because it gives you really large sequence, large fragments that are easier to match. And there are two different approach. The one is to do a world genome assembly and the other is to do local assembly. So world genome assembly works better, but in terms of computing, it's really intensive to compute a world genome assembly for humans. So just to tell you, for example, at BC, they do a file. They detect the Structural Variant. They use a world genome assembly, but they have a cluster that is only dedicated to that. So they have 3,000 CPUs only dedicated to do world genome assembly of humans. So you need to have a lot of resources to do that. So it's why this local assembly approach has been developed.   So it's really well because it really brings you everything that is the novel. How is the local genome assembly works? The idea is you do your mapping here and you will have different. So if you have, for example, an insertion here, you will have the reads from the fragment that come from your insertion that won't map. So it's what we call the orphan. So it's the two paired unmapped. And then you will have the read here in red in green, which is called OEA, which means one end on current. So that means that this read has one read that map correctly and the other read don't map. So you take this read, this two set of reads, orphan OEA, and then you do the reassembly on this subset of read. So you really reduce what you need to assemble. So you really reduce your compute needs to do that. Then you have your context. And what to do? You take your context, you take the OEA that should be at the end part of your context and you look where this OEA encore with the read that you map in the genomes. And that tells you where the sequence and the new SV is located in your reference. So it's kind of a signature you will see. And it works well. It's also intensive in terms of compute, but it's well. And there's a new tool that did that really well for the local assembly. I didn't update my slides because I found it one week ago. I will look in my notes, but there's a new tool that works really well to do this local assembly. Here's a typical denovo assembler you can use to do the assembly. So the tool is not there. So Cortex, SGA, Discover, Abyss, Ray, and so on. So what are the strengths and the weakness of this method? So weakness, computational, computationally very intensive. So especially for a world genome assembly. And sometimes it could be hard to resolve as a result of your blast. If you are in repeated region, you will have your contigs that could map different regions. So you don't know exactly what is the region. The strengths, you could have best resolution of the break point. And you can detect every type of SV. So just to give you a summary of the methods. So you've got depth of coverage, but then that leads to the treatment of assembly. So it's the resolution you could have. So in terms of speech read and mapping, have the higher resolution, but the higher difficulty and cost to do it. So it's always a balance to do to choose. A summary of the SV methods. So you have four methods that exist, either as its own strength and weakness. So if you are interested in the one specific type of structural variant, you could adapt and choose the methods that feature data. The most recent tool, no combine, several methods. So Bumpy and Delhi, I really recommend you to use this tool. And the major challenge to when we do SV is to understand complex events, because you will have several set of goals that will be in the same region and to really understand what you have. It's sometimes really, really hard, and you cannot really resolve that. To really find the break point, because when you find the break point, if you want to validate your data, you need to have the real break point to design kind of PCR or the kind of thing. Now in terms of what you could expect to see in IGV. So this is the type of deletion. I see we saw it one yesterday. This is what we saw in terms of duplication. So it's a ton of duplication as we saw the read are inverted and with large intersize and excess of coverage. This is what we saw for an inversion. So you have two clusters of reads, that cluster together. This one and this one and these two other. So this is your insertion. So you've got this read, some of the reads that map there. And you've got all this new term, new code. And what is a typically view that we do for a virtual variant is the circle spot. So usually you have different tracks that tell you the point of your events. And you've got the translocation of your genome. So if you do consider, you will probably have seen this time several times. So CirCos is a really good tool to represent your SV. But if you use the real CirCos tools, it gives you a really high quality graph. But it's a bit complicated to use. There's no new solution to do that with some package like CirCos to generate this kind of graph, which is way more easy to do and way more oriented to genomics. That's it. Do you have any question? Most of them could work with targeted data except the read depth tool. But not purposely, purposely will work because the problem with targeted data and the read depth is your coverage in targeted data in the example of the protein is linked to how many bits you have, how many probes you have. So the way you're here, so you've got your exam like that. And how you do the targeted, you do some probes that catch this region. OK. And some exam will have a lot of probes, some other really few. So you cannot. So all the methods that normalize horizontally in the same sample will be biased by the number of probes you have in each exam to catch this sequence. So it won't work. So some new tools have been developed to try to take that into account. But it's not working so well. It's only when you have exam, it's better to use a more a population approach. Like for this day or there's another tool, try to remember the name of the tool. I think it's better. So which one it is? No, it's not there. So there's another tool, I don't remember the name, that is doing also kind of population approach. The exam is better to normalize vertically your read depth because every sample for this exam has the same number of probes that I've used. For speech read? So you take your read. You look at the reads which have one read that match normally and the other which have been cut in two pieces. So you will have one part which is called primary alignment and the other will be secondary alignment. And then you look how it will be marked in the background what are the reads that are cut in two, three, four pieces. The longest one will be the primary alignment. And the other will be secondary. So just the way it is called. And you look where the secondary alignments are positioned regarding to the first one. If you are positioned at another location, probably you have a position. If you are positioned at the distance of x number of spaces, you probably have lost the seconds between the primary and the secondary. So why do you need to work with the two? Why do you force the approaches? It doesn't matter. Of course, to be in the middle of it, one, it's a two or three. And as I said, I watched you not for one read, but you tried to look closer of reads that show you the same pattern of reads in the same location. So you look closer of the same parameters. The primary could be at the two, at the two break points of your polynomial, which is a break point, arrive at a different location. If you read, the primary could be that one or that one. So you just look like the link between the primary and the secondary reads. And the cluster of reads that show the same jump. OK.


USS Ronald Reagan arrived in Yokosuka, Japan earlier this week. This is another milestone in the Navy's three-carrier home port swap. Reagan is the fifth U.S. carrier to be forward-deployed to the Seventh Fleet area of responsibility. SecNAP seeks to recognize individuals or teams who have made significant innovative achievements through his innovation program. Command and organizations can submit their nominations by October 31. For more information, visit the SecNAP's innovation page. Chief of Naval Personnel Admiral Bill Moran and Fleet Master Chief April Beldo will hold a worldwide town hall to address concerns from the fleet. This is your chance to get answers to questions from two leaders who directly influence decisions that affect sailors and their families. Send in your questions using hashtag AskCMP. The town hall will be broadcast live October 6 at 1100 Eastern on Shipboard Television and Navy.mil.


I say everything we're doing is preparing with smarts. We're doing studies on osteoporosis type things, muscle atrophy, immune system deficiencies and changes in our immune systems. Concepts, even our regenerative processes on Space Station are working towards systems that will be able to sustain and reuse air, water, urine, those type of things so we can continue and go long distances. So even the station itself is part of the process. Without doubt. Right now on the Space Station, Scott Kelly, another Navy pilot is up there and he's going to spend a whole year up there. He's, I guess, about two and a half, three months into an entire year in space. And the thing that I noticed towards the end of my six months is my body was still changing. My mind was still adapting. And I'll be interested to talk to Scott when he comes home to see if he ever felt like he had fully adapted in that one year in space. Because for me, I know at the end, I was still learning and still discovering new things. Yeah, current.


Going under the knife? Whether it's to have your appendix removed or a triple bypass, you'll need anesthesia. The powerful drugs temporarily render you unconscious, and eliminate any awareness of sensation, including pain, to make surgery possible. Scientists know a lot about how these chemicals work, but there are still mysteries about what happens at the molecular level. Where exactly does Propofal bind to produce its body and mind-numbing magic? What about barbiturates? Now, a team at Massachusetts General Hospital has studied four anesthetics in detail, revealing differences and similarities in how they work. Many general anesthetics work in part by boosting the activity of inhibitory receptors in the brain and spinal cord. These receptors are tiny channels that respond to the neurotransmitter GABA. When GABA binds, the channels open up and allow chloride ions to flow in, making it harder for neurons to activate. Brain activity and results in sedation and, ultimately, unconsciousness. But each anesthetic binds to different parts of the GABA receptor and achieves its calming effect a little differently. To map the precise points of contact, the Mass General researchers used two separate genetic techniques to modify the structure of the GABA receptors. The scientists selected six amino acids for testing. These were located in pockets between the five subunits that surround the central pore. In one set of experiments, the researchers changed these residues to bulky tryptophanes. At some locations, such as between the alpha and beta subunits, mutations reduced the ability of the anesthetic to enhance the receptor's response to GABA, suggesting the drugs normally occupy these sites. In a second set of experiments, scientists replaced the amino acids with cysteines that would be modified by a reagent unless the site was bound or protected by the anesthetic. Thus, the degree of cysteine protection provides a proxy for anesthetic contact. This method was compared with another technique called photolabeling, in which a light-activated anesthetic molecule sticks to the receptor near the drug binding site. The cysteine method was remarkably consistent with previous photolabeling experiments, indicating it is a reliable technique. It also identified new points of contact. In the end, the researchers found that the general anesthesia drug atomidate binds to the GABA receptor between the alpha and beta subunits, whereas a barbiturate binds between the alpha and beta or gamma and beta subunits. Propofol binds near, but not exactly on, all of these sites. On the other hand, a neuroactive steroid didn't bind to any of these locations, and none of the drugs made contact between the alpha and gamma subunits. The remarkable finding is that these anesthetics, all of which produce sedation and unconsciousness, bind to the receptor at different locations. Understanding these sites could be useful in the future and may lead to the development of better anesthetics, giving patients more options and potentially safer surgeries.


Live from San Francisco, it's theCUBE. Covering Red Hat Summit 2018. Brought to you by Red Hat. Hello, welcome back everyone. This is theCUBE's exclusive coverage here in San Francisco, California for Red Hat Summit 2018. I'm John Furrier, co-host of theCUBE with my analyst co-host this week, John Troyer, co-founder of the Reckoning Advisory Services. And our next guest is Arvind Krishna, who's the Senior Vice President of Hybrid Cloud at IBM and Director of IBM Research. Welcome back to theCUBE, good to see you. Hey, John and John, great to meet you guys. So you're just, you can't get confused, you've got two Johns here. Great to have you on because you guys are doing some deals with Red Hat, obviously the leader at Open Source. You guys are one of them as well, contributing to Linux. It's well documented in the IBM history books on your role and relationship to Linux. So, you know, check, check. But you guys are doing a lot of work with cloud in a way that, you know, frankly, is very specific to IBM, but also has a large industry impact. Not like the classic clouds. I want to tie the knot here and put that together. So first I got to ask you, take a minute to talk about why you're here with Red Hat. What's the update with IBM with Red Hat? Yeah, great, John. Thanks for giving me the time. I'm going to talk about it in two steps. One, I'm going to talk about a few common tenets between IBM and Red Hat, and then I'll go from there to the specific news. So for the context, we both believe in Linux. I think that's easy to state. We both believe in containers. I think that's the next thing to state. And we'll come back and talk about containers because this is a world, containers are linked to Linux, containers are linked to these technologies called Kubernetes, containers are linked to how you make workloads portable across many different environments, both private and public. Then I go on from there to say, and we both believe in hybrid, hybrid meaning that people want the ability to run their workload wherever they want, be it on a private cloud, be it on a public cloud and do it without having to rewrite everything as you go across. Okay, so let's establish those are the market needs. So then you come back and say, and IBM has a great portfolio of middleware, names like WebSphere and DB2, and I can go on and on. And Red Hat has a great footprint of Linux in the enterprise. So now you say, we got the market need of hybrid, we got these two things, which between them have tens of millions, maybe hundreds of millions of endpoints. How do you make that need get fulfilled by this? And that's what we just announced here. So we announced that IBM middleware will run containerized on Red Hat containers on Red Hat Enterprise Linux. In addition, we said IBM Cloud Private, which is the ability to bring all of the IBM middleware in a sort of a cloud friendly form, right? You click and you install it, keeps itself up, it doesn't go down, it's elastic in a set of technologies we call IBM Cloud Private, running in turn on Red Hat OpenShift container service on Red Hat Linux. So now I'm for the first time, if you say I want private, I want public, I want to go here, I want to go there, you have a complete certified stack that is complete. I think I can say we are unique in the industry in giving you this.  And first of all, it's not like, you don't have a cloud strategy, you have cloud products, right? So you have to deliver the goods. You got just a replay. The market need we all know is hybrid cloud, multi-cloud, choice, et cetera, et cetera. You take Red Hat's footprint, your capabilities, your combined install base is foundational. So, and nothing needs to change. There's no lift and shift, there's no rip and replace. You can, it's out there, it's foundational. Now on top of it is where the action is. That's what we're, that's what we're kind of getting at, right? That's correct. So we can go into somebody, they're running a, let's say a massive online banking application or they're running a reservation system. It's using technologies from masters, using Linux underneath.  You have a huge complex staff. It's all hardwired and rigidly nailed down to the floor in a few places. And now you can say, hey, I'll take the application and don't have to rewrite the application. I can containerize it. I can put it here. And that same app now begins to work, but in a way that's a lot more fluid and elastic. Oh, by the way, I want to do a bit more work. I want to expose a bit of it up as microservices. I want to insert some AI. You can go do that. You want to fully make it microservices-enabled to be able to make it into little components and digestible, you can do that. So you can take it in sort of bite-sized chunks and go from one to the other at the pace that you want. And that's game-changing. Yeah, that's what I really like about this announcement. I mean, it really brings the best of breed together, right? You know, there's a lot of talk about containers and legacy and we've been talking about what goes where and do you have to break everything up, like you were just saying. But the announcement today, you know, WebSphere, the battle-tested, huge enterprise-scale component, DB2, those things containerized and also in a framework like with IBM, either with IBM microservices and application development things or others, right? That's a huge endorsement for OpenShift as a platform. Absolutely, it is. And look, we would be remiss if we didn't talk a little bit. I mean, we use the word containers and containers a lot. Yes, you're right. Containers is a really, really important technology. But what containers enable is much more than prior attempts such as VMs and all have done. Containers really allow you to say, hey, I solved the security problem. I solved the patching problem, the restart problem. All those problems that lie around the operations of a typical enterprise can get solved with containers. VMs solved a lot about isolating the infrastructure, but it didn't solve, as John was saying, the top half of the stack. And that's, I think, the huge power here. Yeah, I want to just double-click on that because I think the containers thing is interesting to me because, you know, first of all, being in the media and loving what we do, we're kind of a new kind of media company, but traditional media has been throwing IBM under the bus and saying, whoa, you know, old guard and all these things, but here's the thing. You don't have to change anything. You could, containers, you can essentially wrap it up and then bring a microservices architecture into it so you can actually leverage at cloud scale. So what interests me is that you can move instantly, value proposition-wise, pre-existing market, cloudify it, if you will, with operational capabilities. And this is where I like the cloud private. So I want to kind of go there for a second. If I have a need to take what I have at IBM where there's web sphere, no, I got developers, I got install base, I don't have to put a migration plan. I containerize it. Thank you very much. I do some cloud-native stuff, but I want to make it private. My use case is very specific. Maybe it's confidential. Maybe it's like a government region, whatever. I can create a cloud operations. Is that right? I can cloudify it and run it. Absolutely correct. So when we look at cloud private to go down that path, we said cloud private allows you to run on your private infrastructure. But I want all these ability to just describe John. I want to be able to do microservices. I want to be able to scale up and down. I want to be able to say operations happen automatically. But it gives you all that, but in the private, without having to go all the way to the public. So if you care a lot about, you're in a regulated industry because you went down government or confidential data, all you say, this data is so sensitive, I don't really, I'm not going to take the risk of it being anywhere else. It absolutely gives you that ability to go do that. And that is what we brought cloud private to the market for. And then you combine it with OpenShift and now you get the powers of both together. So you guys essentially have brought to the table the years of effort with BlueMix, all that good stuff going on. You can bring it, you can actually run this in any industry vertical, pretty much, right? Absolutely. So if you look at what the past has been for the entire industry, it has been a lot about constructing a public cloud. Not just us, but us and our competition. And a public cloud has certain capabilities and it has a certain disaster city. It has a global footprint, but it doesn't have a footprint that's in every zip code or in every town or in every city. That's not going to happen to the public cloud. So we say it's a hybrid world, meaning that you're going to run some workloads on a public cloud. I'd like to run some workloads on a private and I'd like to have the ability that I don't have to pre-decide which is where. And that is what the containers, the microservices, the OpenShift, that combination all gives you to say you don't need to pre-decide. You rewrite the workload onto this and then you can decide where it runs. While I was having this conversation with some folks at a recent Amazon web services conference, I said, well, if you go to cloud operations, then the on-premise is essentially the edge. It's not necessarily, then the definition of on-premise really doesn't even exist. So if you have cloud operations in a way, what is the data center then? It's just a connected tissue. That's right. It's the infrastructure which you set up and then at that point, the software manages the data center as opposed to anything else. And that's kind of been the goal that we've all been wanting. So it sounds like this is visibility into IBM's essentially execution plan from day one and seeing and connecting the dots, having the ability to take either pre-existing resources, foundational things like Red Hat or whatnot in the enterprise, not throwing it away, building on top of it and having a new operating model with software, with elastic scale, horizontally scalable, synchronous, all those good things, enabling micro-search with Kubernetes and containers. Now for the first time, I could roll out new software development life cycles in a cloud-needed environment without foregoing legacy infrastructure and investment. Absolutely. And one more element, and if you want to insert some public cloud services into the environment, be it in private or in public, you can go do that. For example, you want to insert a couple of AI services into the middle of your application, you can go do that. So the environment allows you to do what you described and these additions. I want to talk about people for a second. The titles that we haven't mentioned, CIO, business unit leaders, how are they looking at digital transformation and business transformation in your client base as you go out and talk to them? So let's take a hypothetical bank. And every bank today is looking about at simple questions. How do I improve my customer experience? And everyone when they say customer experience, they really do mean digital customer experience to make it very tangible. And what they mean by that is, how do I get my end customer engaged with me through an app? The app's probably on a device like this. Some smartphone, we won't say what it is. And so how do you do that? And so they say, well, well, you obviously want to check your balance. You obviously want to maybe look at your credit card. You want to do all those things, the same things we do today. So that application exists. There is not much point in rewriting it. You might do the UI up, but it's an app that exists. Then you say, but I also want to give you information that's useful to you in the context of what you're doing. I want to say you can get a 10-second loan, not a 30-day loan, but a 10-second loan. I want to make an offer to you in the middle of you browsing credit cards. All those are new customers of this thing. So where do you construct those apps? How do you mix and match it? How do you use all the capabilities along with the data you've got to go do that? And what we are trying to now say, here is a platform that you can go all that, do all that on. Right in that complete lifecycle, you mentioned the development lifecycle, but I got to add to the data lifecycle, as well as here is the versioning, here are my AI models, all those things built in into one platform. And scales are huge, the new competitive advantage, you guys are enabling that. So I got to ask you on the question on multi-cloud. Obviously, as people start building out the cloud on-prem and with public cloud and the things you're laying out, I can see that going on for a while. A lot of work being done there. We've seen that Wikibon had a true private cloud before I thought was truly telling a lot of growth there still not going away. Public cloud certainly has grown, the numbers are clear. However, the word multi-cloud is being kicked around. I think it's more of a future state obviously, but people have multiple clouds. We'll have relationships with multiple clouds. No one's going to have one cloud. It's not a winner-take-all game. Winner-take most, but you're going to have multiple clouds. What does multi-cloud mean to you guys in your architecture? Because is that moving workloads in real time based upon spot pricing indexes? Or is that just co-locating on clouds and saying I got this app on that cloud, that app on that cloud, control plane, these are architectural questions. It's a data.  So there's a today and then there is a tomorrow and then there is a long future state, right? So let's take today, let's take IBM.  Well, all of these are different clouds. We run our own public cloud, we run our own private cloud and we have traditional data center. And we might have some of the other clouds also through apps that we bought that we don't even know. Okay, so that's just us. I think every one of our clients is like this. So multi-cloud is here today. I begin with that first simple statement. And I need to connect the data, I need to connect when things go where. The next step, I think people, nobody's going to have only one even public cloud. I think even amongst the big public clouds, most people are going to have two, if not more. That's today and tomorrow. Your channel partners have clouds, by the way. Global SIs all have clouds. Yeah, all have clouds. The Cube is a cloud for Crane Out Loud. So then you go into the aspirational state and that may be the one you said where people do spot pricing. But even if I stay back from spot pricing and completely dynamic and I'm worrying about network and I'm worrying about video reach, I just back up on two. But I may decide I have this app, I run it on private. Well, but I don't have all the infrastructures. I want to burst it today. And where do I burst it to? I got to decide which public and how do I go there? And that's the problem of today. And we're doing that. And that is why I think multi-cloud is here now. Not some point in the future. The problem statement there is latency, managing service level agreements between clouds and so on and so forth. Managing access control on governance. Where does my data go? Because there may be regulatory reasons to decide where the data can flow and all those things. That's a great point about the cloud. I never thought about that way. It's a good illustration. I would also say that I see the same argument in the database world. Not everyone has DB2. Not everyone has Oracle. Not everyone has, databases are everywhere. You have databases, part of IoT devices now. No one makes a decision on the database. Similar with cloud. You're seeing a similar dynamic. It's the glue layer that interests me. How do you bring them together? So holistically looking at the 20 mile stair in the future. What is the integration strategy long term? If you look at a distributed system or an operating system, there has to be an architectural guiding principle for integration, your thoughts. It doesn't mean a world that's 30 years in the making. So we can say networking. Everybody had their own networking standards in the, let's say the 80s, so it probably goes back to the 70s, right?  Decknet. Decknet, you can go on and on. And in the end, is TCPIP that won out as the glue? Others, by the way, survived, but in pockets and then TCPIP was the glue. Then you can fast forward 15 years beyond that and HTTP became the glue. We call that the internet. Then you can fast forward and you can say, now how do I make applications portable? And I would turn around and tell you that containers on Linux with Kubernetes as orchestration is that glue layer.  Similarly, you need all those here, I call those catalogs and automation. So that's the glue layer that makes all of this work. This is important. I love this conversation because I've been ranting on this on the queue for years. You nailed it. A new stack has developed in DNS. This is old internet infrastructure. Cloud infrastructure at the global scale is seeing things like network effect. Okay, we see blockchain and token economics, databases, multiple databases on structured data. So a new plethora of new things are happening that are building on top of, say, HTTP, and this is the new opportunity. This is the new platform which is emerging and it's going to enable businesses to operate. You said at scale to be very digital, to be very nimble, application life cycles are not always going to be months, they're going to come down to days and this is what gets enabled. So I want you to give your opinion, personal or IBM or whatever perspective, because I think you nailed the glue layer on Kubernetes Docker and this new glue layer that, and you made reference system, things like HTTP and TCP, which changed the industry landscape. Wealth creation, new brands emerged, companies we've never heard of emerged out of this and we're all using them today. We expect a new set of brands are going to emerge, new technology are going to emerge. In your expert opinion, how gigantic is this swarm of new innovation going to be? Because you've seen many ways before. In your view, in your mind's eye, what are you expecting? What do you share your insight into? How big of a shift and wave is this going to be and add some color to that? I think that if I take a, I'll take a shorter and then a longer term view. In the short term, I think that we said that this is on the order of $100 billion. That's not just our estimate, I think even Gartner estimated about the same number. That'll be the amount of opportunity for new technologies in what we've been describing. And that is, I think, short term. If I go longer term, I think as much as a half, but at least a fourth of the complete IT market is going to shift onto these technologies. So then the winners are those that make the shift and then by conclusion, the losers are those who don't make the shift fast enough. Of the market moves, that's huge. It's interesting, we used to look at certain segments going back years. This company's re-platformizing, re-platforming their lift and shift and all this stuff. What you're talking about here is so game changing because the industry's re-platforming. It's not a company, it's an industry. That's right, and I think the internet era of 1995 to pick that point is perhaps the easiest analogy to what is happening. Not the emergence of cloud, not the emergence of all that. I think that was small steps. What we're talking about now is back to the 1995 statement. Every vertical is upgrading their stack across the board from e-commerce to whatever. That's right. It's completely modernizing. Correct. Around cloud. What we call digital transformation in a sense, yes. I'm not a big fan of the word, but I understand what you mean. Great insight, Armin. Thanks for coming on theCUBE and sharing the press. We didn't even get to some of the other good stuff, but IBM and Red Hat doing some great stuff. Obviously, foundational. I mean, Red Hat, Tier One, first class citizen in every single enterprise and software environment. Now, open source runs the world. You guys are no stranger to Linux being the first billion dollar investment going back. So you guys have a heritage there. So congratulations on the relationship. I think 18 years ago, if I remember, 1999. Yeah, and I love the strategy. Hybrid cloud here at IBM and Red Hat. This is theCUBE bringing all the action here in San Francisco. I'm John Furrier, John Troyer. More live coverage. Stay with us here in theCUBE. We'll be right back.