I'm Jared Simpson. I'm a principal investigator at the Ontario Institute for Cancer Research in Toronto. My research group develops algorithms for processing DNA sequencing data of the type that you've all heard about today with a particular focus on de novo genome assembly. So this part of the course is going to be a little different than the previous sessions today in that this is only going to be a lecture. There's no associated practical session. But if you have any questions about choice of assemblers or how to run genome assemblers, feel free to ask either during the next hour after the course or email me. My email address is on the last slide here. Now in this talk, I'm going to go through really genome assembly from both a technical level in what's happening under the hood in genome assemblers, how they're taking a set of sequencing reads directly off a DNA sequencer and putting together that genome. That would be the first part of the talk. And the reason I go into the details of how the algorithms work for genome assembly is that it's still quite a difficult process to assemble a genome. And often when you'll take your own data, try to assemble it, you get results that aren't as good as you'd expect. And really key to troubleshooting assemblies and understanding how it's gone wrong is this understanding of what's going on under the hood. So the first part, I'll talk about exactly that, what's going on inside the genome assembler. I'll then walk through an example assembly pipeline of starting from the raw reads, going all the way to sets of context and scaffolds. And then finally, I'll go through some examples of these problems that can come up, what are different features of a genome, what are characteristics of the data that might give you problems when running a genome assembly. And hopefully this will allow you to have a better understanding when you go off and run assembly on your own. So just to put us all on the same page, what is genome assembly? What do we mean when we talk about genome assembly? Well, it means just taking a genome and here I'm just representing a genome as a set of colored bars where the genome is a mixture of unique sequences, which is this green, blue, light blue and purple bar here, and repetitive elements. And repetitive elements are these blocks of sequence that have been duplicated within that genome and these repeats are what cause trouble for the genome assembler. There are sequences that look identical and if your read link isn't long enough, you can't resolve where to place one repeat versus another. So I'm just going to depict a genome like this and I'll come back to this question of how it repeats affect assemblers throughout the talk. So we sequence a genome and from the standpoint of the genome assembler, the sequencing reads are just an unordered sample of fragments of the genome. So you just randomly select the positions from the genome, put them into a bag and then the genome assembler has to reverse this process and put the genome back together hopefully end-to-end into whatever that original source sequence was that you sampled with all these reads from. So first I'll talk about assembly graphs, which are the data structures that we use to work with sequencing reads, then I'll talk about this example assembly pipeline, then some of the features that make genome assembly difficult. And finally I'll end with talking about genome assembly with long reads like you get from Pac-Bio sequencers or like what you get from an Oxford Nanopore sequencer and how that makes the problem a little bit easier due to the read link. So key to understanding how genome assemblers work is understanding this concept of an assembly graph. Now an assembly graph is used to represent the relationships between the reads that you've sequenced. So if two reads overlap we might want to record the information that this pair of reads overlap and the genome assembler will use this to try to figure out what the correct sequence of the genome is. So I'll be coming back to this picture a little bit later on but when we're working with these genome assembly graphs usually a sequence read is a vertex in the graph and if two sequence reads overlap they make an edge in this graph. Now there's two types of graph that are used by genome assembly software. There's what's called the de Brown graph which breaks all the reads into short fragments called k-mers. If you sequence a hundred base pair of read the assembler might break that read up into sixty one base pair substrings, construct a graph of those substrings, I'll give an example of this in just a few minutes. And when we're building assemblies of luminate data most of the algorithms are based on this de Brown graph model. The reason being that it's extremely fast and efficient to work with these short fixed length substrings rather than complete reads and inexact overlaps between reads. So the first type of graph is this de Brown graph which is based on k-mers and then the other type of graph is called a string graph where we keep the reads intact. Every read is a vertex in the graph and if two reads overlap we connect them with an edge. So I'll give an example of this overlapping based graph now. So here we have a single read which we'll call read one, it's about forty bases at length. We're going to add a vertex into the graph for that read. If we sequence another read called a read two we'll add another vertex into this graph and since the start of read two is the same sequence as the end of read one we connect them with an edge here. So we're just going to progressively build up this assembly graph so now we've got two vertices connected by an edge. If we sequence a third read then the beginning of read three matches the end of read two and it also matches the end of read one. So we add this vertex into the graph and we connect it with edges to the previous two vertices. Then if we sequence a fourth read here we're going to add another vertex to the graph and connect it with edges to all the reads that it overlaps. So just this beginning of some reads matches the end of another read that's what we're talking about when we refer to overlaps between reads just that a suffix of a read matches a prefix of another read. Now we've labeled all of the edges in this graph with the overhanging string between the reads. So when we connected this edge between read two and read one the overhanging part of read two is this string TAC so we've labeled the edge as the string TAC. Now what the genome assembly software is trying to do is it's going to try to find a path through this graph which is just a sequence of vertices that are visited and concatenate all of these edge labels together just joining them together into a string that spells the original sequence of the genome. So all the assemblers trying to do is build up this graph which we call string graph and then find a path through it and then the string that that path spells is the sequence of the genome. So that's really the high-level technical part of how genome assemblers are running. Now I'm going to use an example assembly pipeline to illustrate these concepts further. So this is what an assembly pipeline might look like. The assembly software that you download from GitHub or somewhere on the internet it will implement these steps and take the data from your raw fast Q format all the way through to a fast file with contigs and scaffolds. So the first thing we might do is some cleanup of the data to get rid of sequencing errors, sequencing artifacts by either trimming the reads or filtering them out. We'll then try to correct sequencing errors in a read. We'll build one of these assembly graphs, clean up the graphs of additional artifacts, build contigs from the graph, scaffold them together and finally try to fill in the gaps in the scaffolds. And I'll go through all of these stages individually. So the first one and actually one of the more important stages is just cleaning up your data before you give it to the assembler. Often times when you get an Illumina sequencing run if there was some processing artifact like you've concatenated adapters onto the end of the sequence and the sequencer read into the adapter sequences, if you don't get rid of those adapters they can look like very high copy repeats to the assembler. And what will happen is that the assembler will just stop at the position of wherever it finds these adapters. So this is an example run so you guys all saw IGV in the last module. This is an example of a sequencing run that had a lot of adapter contamination. And all of these colored bars here, these are soft clip reads. All of these colored bars are an adapter sequence, it was on the end of one of the reads. Now I tried to assemble this data and the assembly completely failed because of the fact that 40 to 50 percent of the reads had these sequencing adapters on them. If you then run an adapter trimmer of lists, some of them on this slide here crack and traumatic and scythe, they will get rid of those adapters on the three prime end of the reads, present just genomic reads to the genome assembler and everything will work a lot better. So this is an incredibly important step if you do have adapter contamination. Another way that you might want to trim your reads is just if you have low quality bases at the end. You might have seen an Illumina air profile and I'll show it a little later where the error rate increases at the three prime end of the read. If that error rate increases quite drastically, you often want to trim the read back just to get rid of those bad bases. That makes it a little easier on the assembler. Right, here's actually a slide with the error rate for six different sequencing runs. So this was part of the assembly competition, we'll talk about it a little bit later. We sequenced a yeast fish which was Lake Malawi sicklet, boa constrictor, human genome, parakeet, and an oyster and this is the error rate as a function of the position in the read for these six different data sets. As you can see for this parakeet bird data set, they sequenced 150 base pair reads and the error rate increased to about 3% of the three prime end of the read. This is a little bit higher than we're usually comfortable with. So you'd probably want to trim that back to about 100 bases just to get rid of those extra errors. As an alternative to error trimming is we can perform error correction. And this is where instead of just getting rid of all of that sequence that might be bad, we'll try to fix it to be what the true genomic sequence was at that position. And how we do that is we use what's called camer-based error correction and illustrate this. Here's a single read where this position was miscalled. There's a sequencing error here where the sequencer said there is a seed. And how we're going to do this is we're going to exploit the redundancy in sequencing. Usually when you sequence a genome, you'll sequence it to 30 to 40x, which means each base in the genome is covered 30 to 40 times per read. So if we take a substring of one of these reads and count how many times it was seen in other reads, the substrings that are true genomic sequence will be seen a lot. The things that are caused by sequencing errors like this substring that contains the seed will be seen very few times. So what we want to do is we want to compare the number of times we see the genomic substrings to the number of times we see the substring with errors to identify where the sequencing errors occur. So an error correction algorithm will do exactly that. You will take every substring starting from the first to go into the last, and it'll count how many times that's seen. We'll get a coverage profile that looks like this for the true genomic sequence where it bounces between 20 to 30. And then when we hit a sequencing error, we see that the coverage drops to one. That's the algorithm will then say, OK, I think there's a sequencing error. It will then try to correct that error by just trying different substitutions of that position. And if it finds one that corrects the count profile up to this range of about 20 to 30 times, it would then make that fix and say, it's corrected that C to a T or whatever the true base was. So this idea of K-mer based error correction has been quite popular. And there's a lot of software that implements this idea. Some genome assembly packages that you can download will have algorithms built in to do the error correction. Others you can download an external package that only does error correction. Some of the most popular ones are listed here, like Quake, SGA is a software package. I wrote, SOPE DeNovo, BFC, Bless, Light, or Muscat, or all camera-based error correction programs. An alternative way of doing error correction is finding overlaps between reads like I showed before. So you find where the end of one read matches the beginning of another. If you find a lot of overlapping reads, you can then calculate a consensus sequence for the reads. This works very well and it can handle different types of errors in camera-based correction, but it's typically too slow to run on very large genomes. So if you sequence a human genome, you might have a billion reads. Finding overlapping pairs out of a set of a billion reads takes a lot of compute time. So typically we use this faster camera-based error correction for very large genomes. Right, so once we've cleaned up the data a little bit, we want to build the assembly graph. So one of the popular ways of doing this is building a brown graph, which I mentioned before, and again this is built on the idea of fixed-length strings, which we call cameras. So here's an example of what a brown graph looks like for a camera size of four from these five different sequencing reads which are length of six. So what we're going to do is we would take every four basis substring of these reads and we put it as a vertex. So the first four basis of this reads, C, C, G, T is a vertex here, then the second one, C, G, T, T is a vertex here. So we put all of the formers that are present in any one of these reads into the graph and then we connect them by an edge if they're adjacent in a read. So we've connected C, C, G, T with C, G, T, T because they come next to each other in this read. When we build up the graph from all the reads, it looks like that. And we see that this substring, C, G, T, T is present twice in the reads once followed by G, T, A, once followed by G, T, T, C. This causes a branch in the graph and it's the assembler's job to try to figure out what the sequence of the genome adds to end the counting for this branch. So here there's actually a unique solution. It can go from here to here, follow this branch around, go down, back up to here, and then follow there. And that's a super string that spells every one of these reads in this little example. Now, a quick detour into type of research that my group does. So for a long time I focused on trying to make genome assemblers very fast and memory efficient. When we first got Illumina sequencing in say around 2008, 2009, you could only sequence about 36 base pair reads when you try to sequence a human genome with that. You'd use a kamer size of about 27. And if you tried to store one of these the Brown graphs I just showed you with 27 MERS for a human genome, it would take maybe a terabyte of memory. Now, typically people didn't have a terabyte of memory to use on genome assemblers. You might have to go to a super computing center where they'd have very large memory machines to do your assembly. But since we've started working with Illumina data, we've developed new algorithms and new compressed representations of these graph-based data structures. And they've gotten that down to about 10 gigabytes of memory to do human genome assembly now. So this is after about seven, eight years of fairly difficult algorithmic work that we've gotten to a point where we can do fairly routine genome assemblies for humans off of Illumina data. If anybody's interested in the computer science of how this works, now we're able to do these assemblies so efficiently. I'm happy to answer any questions about that. Okay, now after we've constructed a graph, we're not quite ready to build contigs yet. And what we want to do is a step called graph cleaning. And this is an additional data cleanup step where we try to get rid of different types of artifacts that appear in the assembly graph. So the first type of artifact is what we call TIPS. And these artifacts are caused by uncorrected sequencing errors. Now, when I showed you the ground graph before, we had a vertex for every camber. Again, we're working with the same type of graph. I'm just representing the camers as these gray circles here. And when you get a sequencing error, you still put the camber for that position into the graph. But since errors tend to be rare and only occur say once or twice, what happens is that causes this spur off the graph where it has a couple of connected nodes that then ends up going no. And we've got an error here that causes this spur or what we call TIPS. And this error here causes this what we call TIPS as well. So if your assembler is coming or along and it's trying to assemble the genome, it might get confused by these two alternative paths that go nowhere. So what the assembler wants to do is get rid of these erroneous branches that are just caused by sequencing errors. So that's one type of graph artifacts. The other is something that we call bubbles which are caused by heterozygosity. So if you sequence a diploid genome, you're gonna sample camers from either of the alleles covering heterozygosites. So here we have CG heterozygos. And what happens is when you sample camers of those two alleles, it causes a divergence in the graph where you can either follow a path for alleles, one at the top or alleles two at the bottom. And this is a very distinctive structure where it's this divergence and then coming back together. So the assembler can actually efficiently identify these positions and say, okay, I think there's a heterozygous snip here. Do you have a question? Yeah. So what is the bubble showing to four if there's only one? Yeah, that's a good question. So let's say we're working with fairly short camer. So this camer, TCGAT, is shared by both two alleles. So that would be here. Then if you slide it one base over, there's a camer CGATG and CGATC. So that's the first point of divergence and that's this pair of nodes. If you slide it over again, we have GTGATGG, that's these two. And then it's just, you keep sliding over to get multiple nodes for that point. Thank you. It'll be the exact size of your camer. It's exactly the camer. If you're sequencing a haploid organism, would you expect to see bubbles? No, you don't. You do get them, you don't expect them to actually occur, but if you have a sequencing error that's recurrent, it happens four or five times, that might give you enough coverage such that you don't get this tip structure, but rather it rejoins and you can have an artificial bubble because of that. Also, if you have diverged copies of repeats, you'll get bubbles as well, just because of parallelogous sequence variance. Okay, so after the assembler's built a graph, it might look like this. And figuring out what the genome sequence is from this graph is probably fairly difficult. So the assembler's gonna try to identify these tips and bubbles to make the graph simpler. So it'll start by finding all of the branches that are only connected on one end. It will then walk those back to see where they rejoin the graph and get rid of them. That simplifies the graph quite a bit. It will then try to find these divergences where the bubbles occur. It will walk to see when they rejoin the graph and then collapse them down to just a single allele that will be represented in the final assembly. It will then try to build contigs, which typically is just trying to find the largest unbranching segments of the graph. So we'll compact all those nodes here. Then it sees there's a branch here that can't be resolved. It will compact all those nodes here, here, and so on. Intel is just merged. Everything can be merged together without introducing any misassemblies. Now how long are these contigs? Typically for an aluminum sequencing run of a bacterial genome, the contigs will be around 100,000 bases in length. If you sequence a large eukaryotic genome, let's say a human genome, they might be 10 to 20,000 bases in length. If you do pack bio sequencing, long read sequencing, you can get contig sizes in the megabases and that's something I'll return to a little later on. Now finally, once we've got our contigs, if we've sequenced paired ends, we're not done yet and we can build what are called scaffolds. And scaffolds are just a higher order structure where you try to link contigs together by paired ends that map between them. And you separate the contigs with a gap, which is just unknown sequence that you don't, that you can't figure out in between the contigs. And we call that higher order structure of linked contigs with gaps, scaffolds. So the way we build scaffolds is that we map paired end reads to the contigs and then we identify where a group of pairs maps the end of one contig and their mates maps to the end of different contigs. So a color code from here, so these blue pairs map here and then went here and we have red here and here. It's hard to see if it's purple here and here and then green here and here. So we would just link up those contigs by paired ends and we build what we call a scaffold graph, which is really similar to these read graphs I've been talking about before and we'd construct the scaffold from those. We can use the paired end insert size distribution that we learned about in the previous modules to estimate how far apart these contigs are in the scaffolds and then you just fill in the sequence with ends representing the uncertainty of that sequence between the contigs. There's a set of programs that are called gap fillers which will take those scaffolds and then try to do local assembly which is not as strict to try to fill in what the sequence is. Now those gaps are caused usually by two things either repetitive sequences that are too difficult for the assembler to resolve or just drops in coverage because you had either you didn't sequence your genome deeply enough or perhaps there was some GC bias in that region that didn't allow it to be covered by reads. If you have long reads to augment your short read assembly there's programs that will allow you to use the long reads to fill in the gaps which is especially helpful for highly repetitive regions. Okay, so I'm gonna, is there any questions about that part because I'm gonna move into now what are some of the difficulties for assemblers? Yeah. The assembler choice is six sequence for a graph. Is that random? Yeah, so it will select a k-mer size and then every k-mer will become a vertex in the graph and then when it's trying to do the compaction where it's finding the paths in the graph it will usually just pick a node at random but it actually doesn't matter. No matter what node you pick you'll arrive at the same set of context. And the second is that we might have some graphs that resolve equally well. Do you have some sort of scoring that helps to resolve two possible paths that could resolve? Yeah, that's a good question. There is somebody of research on trying to find the genome assembly that maximizes the likelihood of the read data. So this is like a very principled framework for trying to find the best reconstruction of the genome. Those algorithms tend to be quite heavy because you're doing a lot of these calculations you're looking at like a lot of different alternative path, a lot of different alternative paths through the graph and then pick whichever one's best. You have to use hidden markup models, things like that to score the sequences. So they tend to only be used on very small genomes. Usually when we're doing human genome size assemblies we're working so hard to get it to run quickly enough that we just do very simple things, take the contigs that are 10,000, 15,000 bases in length and then that's the result. Right, so in 2013 there was an assembly competition which aims to figure out what the best performing genome assemblers were. Genome assembly's been an incredibly popular research topic to work on. I worked on it for quite a few years. Many other researchers have worked on it and there's probably been 10, 15, maybe even 20 assembly software packages developed in the last decade. So the question was, would assemblers work best? What genomes do they work well on and why do some genomes assemble very well whereas others have problems? So this competition which was called Assemblathon 2 really aimed to address this question. And there's a quote here from the lead author, Keith Bradenon who really summarized what the field of assembly was in 2013 by saying, don't trust the results of a single assembly. If possible, generate several assemblies with different parameters and different assembly software and then select whatever the best one is for your purpose. And I think this really underscores the state of the field in that there's a lot of uncertainty in genome assemblers. We're relatively happy to read mappers now where you can use BWA and it's gonna give you a pretty good job. That's not quite the case with assemblers and it's much more specific to the genome that you're trying to assemble. Now on this slide I'm just trying to summarize some of the factors that make a given assembly difficult. So I've touched on the idea of sequence repeats as being one of the key factors. If there's high levels of heterozygosity, so if you have these bubble structures occurring in multiple places in genome quite often that can confuse the assembler. If your genome's not covered very well that causes breaks in the contigs because you just don't have sequence reads there that allow you to link reads together. If your sequencing is highly biased, if you sequence something that's either AT rich or GC rich, you might not cover the genome completely in records and that can cause problems with the assembler. Of course, if your reads have a lot of errors or there's chimeric reads where you've joined up different places of the genome into a read that causes problems with the graph. You've mentioned sequencing adapters. If your samples are contaminated or if you've sequenced multiple individuals rather than a single individual, these are all factors that make it more difficult for the assembler. So what I wanted to do after this assemblathon paper was put together a package of tools that could measure all of these factors about how a given assembly might go wrong automatically without any input from the user. So they could just take a FASTU file, it will then measure if your coverage is good enough to do an assembly, how repetitive the genome is, how heterozygous it is, as a way of giving the user information about how difficult that assembly is gonna be. And the next few slides, I'm gonna explain how this program works and go over some examples of the reports that it generates, which is just a PDF file, which allowed you to explore the properties of the genome you're trying to sequence and choose an appropriate assembly strategy. So I'm gonna go back to this slide, which I showed at the very beginning and essentially how the program works is it's going to analyze the structure of this assembly graph to estimate how difficult it is going to be to assemble. So what we're gonna do is we're gonna try to identify how often these branches occur due to sequencing errors, how often branches occur due to heterozygous snips and indels and repeats as well. And the way that we do this is we analyze how often these camers occur in the reads. So just like I was saying when we were talking about error correction, when you've sampled genomic camers, you usually see them 30 to 40 times if you've sequenced the genome quite deeply. And we can use that information to learn about the error rate and in particular the heterozygosity of the genome. So if we draw a histogram of the number of times 51 mer, so just a fixed length string of 51 bases occurs, we get a plot that looks like this for a human genome, peaks around .28x, and that reflects the fact that we've sequenced the genome fairly deeply. Now if we show the same histogram for a highly heterozygous genome, like the oyster genome, the histogram looks like this. And then what's happening here is that we have these bimodal distributions. And the reason for this is that the oyster genome has a snip around 190 bases. So a lot of the camers are coming from, are covering heterozygous snips and indels, it's from this peak, and then another population of camers are from just homozygous position that are present on both chromosomes. And this sort of distribution causes a lot of problems for genome assemblers. They want to treat the genome as being just a single string. They don't want to see a lot of these bubbles forming. So this oyster genome will definitely cause a lot of problems when you put it into a genome assembler just because of this high heterozygosm. And the level of heterozygosm is so high, it's readily apparent just looking at this camercount distribution. So now we can formalize this concept by just calculating how often these different types of branches occur within the graph. And in a paper I published in 2014, we have a statistical model for taking an arbitrary branch that's happened in the graph and classifying it into either sequencing error, whether it came from a heterozygous snipper indel, whether it came from a reed gate. And this is just based on these coverage counts that I talked about. I'm not going to go into the math of how this works, but essentially what we get out is a prediction of how often there's heterozygosity within the genome. So this is the six genomes I showed before, three of them were part of the semaphon competition. And we see that the branch rate for the oyster genome is predicted to be about one in 100 bases. And this is just reflecting this high level of heterozygosity within the oyster genome. For the human genome, which is in this color here, it's around one in a thousand bases, which is also reflective of the diversity we expect when we sequence a human genome. As a negative control, we sequence a haploid genome, we predict it about, I think one in 30 to 140,000 rates of heterozygosity in that. That's just these bubbles I was talking about that are caused by our current sequencing RFLACs. So that gives us a lower bound on the sensitivity of predicting heterozygosity by analyzing this assembly graph. We can also do this for predicting the frequency of repeats. Again, for the oyster genome, we see very high rate of repeats for the human genome. We see a high rate of repeats as well. This is what we expect. We know the human genome is larger and repetitive. And what we get from looking at these three pictures is a measure of how difficult the graph is going to be to resolve for this assembly. So what you want when you sequence your genome and you put it up to one of these plots is for its branch rate to be down here like this yeast genome, which is relatively simple to accept. This program will also predict the genome size for you. So here's just a prediction of genome size for these six test genomes. Human genome comes out to be about three gigabases what we expect. Yeast genome's tiny down here at about 12 megabases. The program will also assess the quality scores of your reads, so just plots the mean quality score by position, just like when we looked at the error rate of these six data sets. You see that the bird data set has problems at the three prime end. Might need a bit of trimming there. And we'll also predict the error rate as well for you. Now the question of GC bias came up a few times. I heard you in the previous module. And this program will also allow you to measure GC bias. So now we're looking at a 2D histogram of coverage versus GC concepts. Coverage is on y-axis, GC concept is on the x. So this is for the fish data set. We see that there's no real coverage is relatively dependent of GC in this case. This is what we want. We want to see most of the genome covered uniformly independent of what the GC content is. If we look at the yeast data, we start to see that there's this trend downwards in coverage for higher GC. So by looking at these plots, you can start to see whether your sequencing coverage was biased for this run. Again, we go back to this oyster data set, which is very problematic. And we see that this has actually split just because of this high level of heterozygosity, whereas all of this density has come from heterozygous position. This part of the density has come from homozygous position. And the program will also predict what the fragment size distribution is for your paired end reads, allowing you to check whether this matches what you expect from your library prep. So here, again, we have a variety of different data sets for the snake data to involve the constrictor. The intersize distribution is around 350, 360 bases. For the bird genome, it's around 500 bases. And finally, the assembler will actually run a simulated assembly for you in just a few hours rather than having to wait multiple days to run the assembly. To give you a measurement of how long it thinks your contigs will be as a function of this k-mer size for the de-brow graph. So we'll just sample pass through the graph, look to see how branched they are, and then tell you how long the contigs are. Now, the point I was trying to get at in these last 10 slides is that this yeast genome, which is very easy to assemble, you're gonna get very long contigs, no matter what k-mer size you pick. But for this oyster data set, which was extremely hard to assemble, you're always getting fairly short contigs. So this program will give you an answer in about six hours total runtime.  If your assembly then turns out to look very poor, you can go back through these other slides, say, okay, maybe my coverage isn't so good, maybe the error rate's too high, maybe my have too many branches in this genome. And that helps you troubleshoot your assembly. So here's the link to the code. I won't read this out since it's all on your printed out slides. It really just takes three commands to run.  Okay, in just the last few minutes, I'm gonna talk a little bit about the long read assembly. So this has become a really prominent research direction, probably the last four years, since long read sequencing has become more available. So the Pac-Bio guys really deserve a lot of credit for building this field, as maybe five years ago, it was considered that the error rate within long sequencers was too high to do genome assembly. So if you sequence with Pac-Bio, the error rate will probably be between 15 and 20%. And as I said before, trying to find overlaps between reads is computationally difficult, and that is just compounded by high error rates. You have to search very hard to look for reads that have a true overlap when there's such a high error rate in the reads. So like the engineers at Pac-Bio, mainly led by Jason Chin, they developed very fast ways of calculating overlaps, and they developed ways of error correcting Pac-Bio reads such that they can give very high quality genome assemblies. Of course, the reads are much longer than Illumina reads there. You can get 10,000 base reads on average, versus 100 base reads for Illumina, and this gives you a lot more information about how the repeat structure of genomes are compared to the short Illumina reads. So as I said, Pac-Bio really developed this field. More recently, Oxford Nanopore reads, which developed this, Oxford Nanopore has developed this miniatrized USB-powered DNA sequencer, which has similar properties Pac-Bio reads are now starting to be used for genome assembly as well. Now, the assembly pipeline for long reads looks very similar to short reads with a few notable exceptions. Here, instead of doing camera-based correction, we typically use overlap-based correction for long reads. It tends to be computationally much more expensive. The reason we don't use camera-based error correction is that the error rate's too high to have perfect cameras in your reads. They're usually interrupted by an error. And so you'd need to use a very short camera if you want to use camera-based methods. And finally, the last step is typically polishing the consensus sequence. So when you take your long error-prone reads through the assembly pipeline, the final contigs that you get will often have quite a high error rate because there's so many errors in the reads. So what we do is we do this step called polishing, where we use fairly sophisticated statistical models of how the sequencer works as a way of accurately correcting the final assembly and generating better consensus sequence. This slide has just a few papers that are noteworthy to read if you're interested in long-read assembly. This paper in Nature Methods, three years ago, is one of Stockholm Pac-Bio first demonstrated you can get completely E. coli genome assembled into a single content using their sequencing data. We're recently at, Philippi's group has done great work on speeding up overlap calculation for long-reads and never-pay-for-on-bio-archive. I think it's now Nature Biotechnology that you can read. And this is a paper from my group just showing you can do complete genome assemblies of E. coli using Oxford Nanoport data as well. Right, so final slide is just some recommendations for long-read assemblers. Luckily, if you do have long reads and I should caution that it is more expensive to obtain than short reads, but you tend to get much higher quality data. And one of the benefits of sequencing with long reads is that the pipelines are a lot more stable and they're a lot easier to run. Unlike Illumina where you have this laundry list of problems that can happen when you sequence a genome, a lot of those problems go away when you have very long reads. So I can make a recommendation of just two pipelines for long reads. So for Pac-Bio, use Canoe to build contigs followed by Quiver to polish the genome. And for Oxford Nanoport data, use Canoe to build contigs followed by Nanopolish to build a consensus sequence. And as I've said, typically the results are far better than when you get short-read assemblies. I think the best human genome assembly with Illumina data had contigs around 30 kb for Pac-Bio, they're around 20 megabases. So you just get much, much longer contigs that aren't really comparable. If you're doing serious genome assembly where you want to produce a reference genome for something that's very important, it's definitely worth it to pay for very long reads just because the quality assembly is so much higher. Right, so as I said, there's no tutorial, but if you feel free to ask me questions and hear now or just email me if anything else comes up. I haven't. That's a very good thing to bring up. So the question is whether I've tried 10x genomics technology. So this is a new way of doing library prep where you can tag very long fragments of DNA, hundreds of kb, such that you know that all of the short reads came from the same 100 kb molecule through this library prep. And this gives you much longer range information than just made pairs or any sort of paired ends. And it's becoming a very popular way of getting more information out of short read sequencing. So while I haven't done it, I know people are getting megabase size assemblies from using the sort of tagged information. So that's a good one to look into as well. I think it's cheaper than packed bio sequencing, but the drawback is you're still using, basing it off short reads. So if you have very locally repetitive sequence, it'll struggle to assemble that, but you will have a lot of long range information. Yeah, usually, so the traditional way of doing assembly is you would just do a sweep over the cameras. You submit 50 jobs to your cluster with different camera sizes and then you'd pick whatever one gives you longest context. That takes a lot of time.  And exactly why I wrote this program is that you could just narrow it down. So rather than saying, these cameras are always giving you fairly short values. You might not try them. You might just narrow it down to this ring, which is giving you the best assemblies. So while it's not gonna predict exactly the optimal one, it should narrow the choice down to a smaller range. Okay, and could you just re-explain the N25 and Sion can add your context length and then that I can never understand that? Yeah, so N50 is the length of the contig such that half of the genome is in that length or larger. Okay, so you can think of it like a weighted medium of the distribution of contiglates. You got one? Yeah, so we usually shoot for, when we do nanopore sequencing, 8KB for bacterial genome, that works very well. At least in E. coli, the longest repeat is I think 6KB, which is ribosomal RNA. Now, if you could only sequence, say, 3,000 base pair reads, just because you don't have enough DNA, it would probably give you a better assembly than an aluminum assembly, but it wouldn't be as good as sequencing the 8-10KB. Yeah, or at least the dominant repeat. Like for human genomes, ALU repeats, which are 400 bases, they cause a lot of problems within the assembly graph. So you could get pieces that are longer than that. You'd be in quite good shape. If you want a complete genome, which even for human genomes isn't really feasible just because the centromere is so repetitive, even you'd max out, say, megabase-sized content if you only have 3,000 base pair reads. But yeah, it's gonna be a trade-off. One thing that a lot of people do is if you're sequencing something like, say, Jusofla where you can't, it's very small, you can't get a lot of DNA. They might pool multiple individuals to get more DNA, but that's generally a bad idea because then you're sequencing the population, there's a lot more diversity, and it causes just all these branching structures in the graph, so as much as possible, you wanna sequence isolated bits, and then, yeah. Bias, really? Yeah. If you know that you see bias problem, would you pick that fact? Typically, you'd wanna try PCR-free libraries. So that just, so there's always gonna be this bridge amplification PCR that happens on the flow, so you can't get around that. But to make the library, you can remove the PCR step. Of course, this requires a lot of DNA if you're not doing the PCR, but that's the number one thing for getting rid of minimizing GC5s at least. Other than that, that's all you can do. Yeah, so the interesting graph-based assemblies is mainly when we map a read to the reference genome, we're just treating it as a linear sequence with no diversity, if that read came from something that's very different than the reference, it's not gonna map very well. Say we came from some novel sequence and there's a lot of novel sequence that's not present at human reference, that's where it's gonna go unmapped. If it has a very long insertion deletion in it with respect to the reference, it might not map very well. So the idea is if we can build these reference graphs, which incorporate all that known polymorphism, it would map to one of the alternative branches better than just the linear sequence. So rather than losing that information, we'd be incorporating it directly within the reference. That's really the cutting edge of algorithm design right now is graph-based reference genomes. It's not really in a state where people can use it, but it's an incredibly hot area of research. Yeah, I was just curious what the use is about. Yeah, it's mainly just so we don't lose. We wanna use it as much information as we can when placing reads into the reference. Yeah, we're quite good at accounting for just sequencing errors. When it's just single-based edits, if it's larger edits like insertions and deletions, if that information's in the graph, then it becomes, like Yeom said, a much simpler problem. But I guess there's not that. There's not yet, but it's growing, especially in the last year. There's an organization called GA4GH, Global Alliance for Genomics and Health, and they're pushing forward the use of these graph-based algorithms. There's five or six groups who are all actively developing methods around this idea, and everybody expects that to be the next generation of mapping and varying calling tools. To me, that sounds like using consensus data before assembly, is that a computational trick to make assembly easier? Because the same data in the end will be used to generate a consensus, right? Yeah, that's a good question. Something we've worried about a lot is the size of these assembly black graphs. If you have a human genome assembly graph, it can have 10 billion vertices. That's quite a lot. It takes a long time to work with, a lot of memory. So if you correct the errors up front, it just shrinks the size of the graph down. It makes it much more manageable to work with. So it's mainly a data reduction approach. So you do that more conservatively? Yeah, exactly. You typically be very conservative so you don't generate a lot of artifacts. You don't want to make the overriding goal in assemblies, like don't make errors. Because if you make an error in reference genome, it goes into some database and then it's there forever. At least that's how I've always designed my programs. If the genome is fragmented because you're too conservative, you can then add in long reads later, fix it up, get longer contigs. But it's typically a lot harder to fix misassemblies. Yeah. So usually what you'll do, so when you pick your camera parameters, say here, you want to pick something that's high enough such as you're getting around a lot of repeats, but you don't want to pick a camera that's so high that, right, this isn't my computer. You don't want to pick a camera that's so high that you're not covering the genome in these cameras. You need much higher coverage if you're using a long camera size. I don't know if that was clear. But because of the fact that there's this tension between using long cameras to resolve repeats and short cameras to have high coverage, if you pick a value that's too high, you just get coverage breaks. And what you can do is when you find that there's a gap, you can use a shorter camera to try to fill in that gap. If a gap is caused by repeat, you might use a longer camera, but it allows you to just adapt the parameters such that you can fill in that exact sequence. Yeah, that's something my group works on quite a lot, is that an alternative to using reference graphs is to just do everything in de novo assembly and then compare the assembled sequence to the reference genome afterwards, which tends to be easier. The main problem with de novo assembly based variant calling, which so to give you a concrete example, we try to do find somatic mutations, they're only occurring in a tumor, not in the norm in individual's normal genome by comparing the reads in an assembly graph. That works very well for long indels, which are a weakness for mapping based approaches, but unless you have very high coverage, you might not see the very, so the drawback of de novo assembly based variant calling is that you just need much higher coverage and you have slightly lower power in repetitive regions because of these problems that we've all talked about of having these branching structures. Yeah, so this program, which will do a simulated assembly to let you pick camers, that's doing a very similar thing to camer genie. So camer genie's trying to predict the best K. There's no more questions. As I said, my email's on the last slide here. Feel free to email me questions if you have any problems with your assemblies. Happy to answer those.


Aponius notes that the beating heart of Rome is not the marble of the Senate. Yeah, but do you really want morons rampaging through the marble of the Senate and defecating on it? All right. So your marriage is not your home. Your marriage is not your bedroom. Your marriage is not even your bed.  Why would that bother you? Like, why would you get upset? Your marriage is not your marital bed, bro. You can clean the sheets. What's the big deal?  Your home is not a physical building. Your home is a spiritual condition.   The beating heart of your marriage is not a physical place. It's a spiritual space.  Why would you care? You're at home and then a mob starts pouring into your home. Why would that bother you? Look, why do I always show for elites? Yeah, that's what I'm doing. I'm shilling for elites right now, suggesting that most people are not going to like it if a mob occupies their home. Most people are not going to like it if a mob just possesses them from their home. How would you like a mob to pour in to your workplace? How would you like that? And to go through your stuff and trash it?  How would you like that? And this is me just shilling for elites.  I'm not a big fan of mobs pouring into your home and dispossessing you from your home and trashing it. Here I go again, just shilling for the elites because I believe in the rule of law. I believe in people not smashing camera equipment that does not belong to them. I believe in people not murdering innocent people. Five innocent people are dead because of the rampage on the capital on Wednesday. I'm shilling for the elites because I'm against the death of innocents. I'm against murder. I'm against people pouring into your home or your office or your government's buildings. Let's say you have a medical emergency and you're about to go to an urgent care center to deal with your life-threatening medical emergency, but a mob has occupied your urgent care center. What's more, the mob is occupying the streets around you so you can't get out to get your life-needed health care. And here I am, shilling for the elites saying it's a bad idea when we allow mobs to block traffic, when we allow mobs to destroy medical centers, when we allow mobs to rampage through government buildings. Here I am just shilling for the elites saying, I don't want crazed mobs pouring into your home or your workspace. My God, I am such a shill. We got killed. The fact is, we have not seen the end of it. I think they're going to be major arrests. I think all of the people involved in this movement are going to, at the very least some of them might be on probation, but they're going to be serious arrest. For this, this is not fun and games. When you walk into a capital building, invades a senator's office and live stream yourself acting like a jackass, you're going to jail for that. Baked Alaska and company, whoever else was there, the America first movement. You're going to jail for this. This will be prosecuted as if it were treason. It doesn't matter if you were engaging in some hyper- So apparently Donald Trump has just been banned from Twitter.


In this video, I will show you how to live stream your Zoom meeting to YouTube. I will also show you how to use the new YouTube setting to automatically start your live stream the moment your Zoom meeting starts streaming. Let's get started. Please note, Zoom live streaming is available only for the pro account and above. The first step is to schedule your live stream on YouTube. To do this, go to YouTube Studio by clicking on your profile icon and selecting YouTube Studio. Make sure that your account is enabled for live streaming. You can check your account by clicking on settings, select channel and choose the option feature eligibility. You need at least the intermediate features to be enabled to live stream on YouTube. In my case, this feature is enabled. If not, you need to enable it and all it requires is for you to verify your phone number. Once you verify your phone number, it usually takes around 24 hours for the account to be active for live streaming. Now that I know that this account can live stream, let's go ahead and schedule a live stream. Click on this option which says go live and you will be taken into the YouTube live control room. You can choose to go live right now or what we want to do is we want to schedule a live stream for a future date. So let me choose this option start. Alternatively, you can schedule your live stream by using this button on top here called schedule stream. Since we will be using Zoom to live streamer event, you need to choose this option streaming software. Click on go. Let's go ahead and schedule a stream. Click schedule stream. In the first page, you will describe your event itself. Give a title for your event and optionally give a description. You can choose an appropriate category. Let's choose education. Upload a custom thumbnail that will be shown as the preview for anybody who logs on to YouTube. Scroll down and choose your target audience. If your live stream is meant for kids, you can choose the first option or you can say it's not made for kids. If you choose the option that it is made for kids, some features will be disabled such as live chat. In my case, I will say no, it's not made for kids. Click next. Here we can control certain aspects of your live stream. For example, whether you want to enable live chat and who can post chat messages. If you leave it to anyone, then anybody who's watching the live stream will be able to post a chat message. But if you select subscribers only, then only your subscribers will be able to post chat messages. Click on the next button. In this section, you will choose who can watch your YouTube live stream. If you select public, then anybody who accesses YouTube can access your live stream. You can make it private, which means only certain people will be able to access your live stream or you can make it unlisted, in which case the link can be shared with only those people who you want to watch your live stream. In our case, let me make it public. Select the schedule for your live stream, the date and time. Let's choose 20 seconds and let's choose the time as 2pm and click done. A live stream is now set up. One feature that you might want to explore is this option called enable auto start with this feature on by clicking enable. Now what will happen is when your zoom meeting starts streaming, YouTube will automatically start your live stream. You can also enable auto stop. Click enable. Now what will happen is when you stop streaming from zoom, the YouTube automatically will stop streaming. We now need three bits of information from here to use in zoom. Let me open any note taking program. For example, notepad first, I need the stream key and that is available here. It is currently hidden, but I can unhide it by clicking on this icon and then select copy. Let me paste it in my notepad. So that's the stream key. Next, I need the stream URL and that's available here. Let me also copy it and put it in notepad. So that's the stream URL. And finally, I need the link that I can share with the audience and that is available right on top here with this arrow key share. Click on it and then click this icon which says copy video link. Click close and let me paste the share URL also in my notepad. Now I'm done with YouTube. Let's move on to zoom. Sign in to your zoom account, enter your username and password and click sign in. Zoom has a few settings that enable or disable live streaming. To check your current settings, click on settings. Scroll down till you find live streaming. As you can see, if you have not enabled live streaming in zoom, you can enable it by clicking this enable button and choose the social media platforms where you want to allow live streaming. For us, the important part is here enabling the custom live streaming service. Make sure that this is ticked. Let's go and schedule a meeting in zoom now. Click on schedule. Give a title and description for your zoom meeting, which should be the same as the one you had given in your YouTube live stream. The date and time should also match your YouTube live stream. Select additional parameters of your zoom meeting as you normally would and click on save. If you now scroll down, you will have an option here called live streaming. And what we want to do is we want to configure the custom streaming service by clicking this button. As you can see, we need to enter the three bits of information that we had copied from YouTube. Let me open up my notepad and I know that this is the stream key. So copy it and place it here in the section stream key. Next, we need the stream URL which is here. Copy it and paste it here. And finally the link that will be shared with the audience selected and paste it in the live streaming page URL. And that's about it. Click on the save button.  Let's say it is the day of the event.  Zoom will prepare to go live and will immediately start the YouTube live stream. If you open the YouTube live control panel, you can see that YouTube has automatically started the live stream because we had enabled the setting to auto start. When you want to stop the live stream, click on the three dots again and stop live stream. If you check the YouTube control panel, you see that YouTube detected that the stream is no longer receiving any data and it will terminate automatically again because we had enabled auto stop. So there you have it. How to go live on YouTube from your zoom meeting. If you like this video, hit the like button and perhaps even subscribe to my channel because it encourages me to create more content like this.


Okay, I've been playing around a little bit with this two-stage amplifier circuit. I've done some slight modifications. I've not changed particularly the two stages here, but what I have done is, from the output, I've put a little driver transistor to drive these NPN and PMP transistors. Now, that there, even though it suggested that there's a resistor in the little circuit that I was looking at, all I did was I saw this circuit just from this simple stage here, and then I thought I could add another stage to it, and I put a 10-microfaric capacitor between them, and that made it sort of like no good, and so then I put a smaller capacitor there, and that made it good on the output. So then I saw this other way you could do this complimentary add-on, and I got that game, but I didn't have this little driver circuit here, and then I saw another part where you could add the driver, so I thought, okay, I'll see if I can just add these things together and get it to work. It's all, there's no practicality here. It's all just to experiment, to play around with, try to understand how the biasing works, and even though I don't, I can't fully describe it for fear of making mistakes.   It's just to get a bit more familiarity with, you know, building the circuits and pressing around, and that's what I've got, that's what I did, and we do get output. Now, I'm running this from my, it's got 10.5 volts there, which is getting into my little boost converter, three-thirds at the right time of day, where if I wanted 12 and a half volts out of my solar panel batteries, I could get them. I would be able to get them really, really, I mean, because you can see that there's 13.4 volts there, but by the time I go through the regulators to control the current and control the voltage, I lose a little bit of power. It's using 60 milliamps, not because it's, the circuit's using loads, it's just that, because I'm boosting the circuit, it's got to get the power from somewhere, it takes it in current and use it for this boost converter here, and I'm just monitoring it here. I was monitoring it there with this, but the problem with this thing is, although having this sort of dial is good, it sticks, so when you're adjusting your voltage, while what's happening yesterday was I was adjusting the voltage, it wasn't really doing a lot, giving this a bit of a tap, because it was up in 929 plus volts, and that's no good when you're trying to do your circuit, so there it is. Now, I've used the variable parts here, just so I could try and get the bias incorrect between the positive and the collector on here, and oh, sorry, is that to the emitter? No, it should be to the collector. Yeah, it is to the collector, and then I've also, no, sorry, sorry, my stand corrected, it's to the base, I should have looked at the circuit, it's actually there that it goes to the base, and I've also used the variable part here between the, what is basically between these two emitters and to the base of this driver circuit, this driver transistor, and that is here. Now these are absolutely terrible, these parts played around with them for ages last night, and that's why I've changed over to this, and I've changed over to that little one down there, which is a 200 ohm part. I don't know if that really needs to be there, but in the circuit that I was looking at, it has a variable part there, and so I've got it set and I've put just one ohm, it was really that you could have nothing, and it's still fine, but I've just got that there, so really that's just, I don't think it's really needed, but it was on the circuits, or the partial circuit is just out of a book and it's just showing you bits, it's not actually giving the whole thing. So yeah, this is what I've got on the output, and if I adjust, I'm not going to try and do this one because the wires are so loose here, it will just come apart. If I just describe what's here first, on the yellow is what's coming in from here, and the one underneath it is set to one volt and it just shows the difference between the input and the first stage, the pinky purpley coloured one, the next one down, that's set at five volts, you can see that, see what the voltages are there, and then the one underneath, I don't know why it sort of bellies out down the bottom, I don't know if that's the right terminology, I'll just make it up, and it's got like a very slight sort of clip on the top, that's just because I'm just pushing it right to the edge. But there's the differences in voltages, like I said, this circuit doesn't have any real practical application, I suppose the first two parts can be for headphone, the next part really, it's not even in phase correctly, if you look at the first three waveforms, they're out of phase with each other by 180 degrees, which is correct, but the next one sort of like, it's got a time shift on it as well, it's not being in phase, or out of phase, well I suppose it is out of phase, but it's just got that time shifting going on, I'm not quite sure all that's about, I twiddle and mess around, and craft around with it, what I was trying to do, to be completely honest, was I was trying to see if I could, if I could get it so I could put a speaker on it, and output, now I mean I've put this here, and I'm probing here, and I don't really need to, I suppose I'm really probing here, I just added that, because in the little description they were trying to say that this would be a complimentary power stage, and they've got a capacitor there, so I'll just stick it there anyway. I am probing on this side, of here, and I'm probing here, I'm actually probing on the, on the collectors, of each, so that's it, that's what I just thought I'd, I just thought I'd share that, like I said there's no real practical thing with this, because if I try, I have already connected my, music player into this, and if you have it on quiet, you know, real quiet, you can plug a speaker into it, and you can hear it there quiet, but you can't power out of it, this is only doing the very small amounts of power before we go into distortion, but it'll be okay for headphones, I suppose the first two stages are great, I'm not sure what that second stage is gonna sound like to one on headphones, I didn't bother, I'm just playing around, I'm just getting more familiar with using the oscilloscope, getting more familiar with just playing around and seeing what happens, trying to set them up, I tried doing more than the, in my circuit simulating software, but I couldn't seem to get it to work, probably even just one stage, so I'm not quite sure what was going on there, but anyway, Ramble, Ramble, Ramble, stop that, I'm gonna actually try and build something today or tomorrow, which is usable, so cheers for watching guys, if you watched, please don't just chuck us a thumbs down, this isn't a tutorial, this isn't supposed to be something that's gonna be great, that you can use and you can connect to them if you want to play around with it, and try and make that a bit better on what it is, great, but it's not really anything, it's just me just playing around and that's, I'm a novice or what was it called, novice, enthusiast, electronics enthusiast, I'm not an amateur hobbyist or professional or anything like that, or a tutor, I'm just doing this just to play around with and have a little bit of fun with electronics, so cheers for watching again, as I said, I'll speak to you soon, bye.


Hi, everybody, we're back. This is Dave Vellante with Stu Miniman. We're here at Edge Live at the Mandalay Bay Hotel in Las Vegas. Kevin Powell is here. He's the Flash Systems Business Line Manager for IBM. We've been talking Flash a lot. Probably, I don't know, Stu, at least a third of the content, maybe more that we've touched on has related to Flash. We talk big data, we're talking Flash. We talk Jeff Jonas. We end up talking about Flash. And he, Joe Suck, comes on. We talk about Flash. Yeah, Flash is changing the architecture everywhere and touching everything. So Kevin, welcome to theCUBE. Really appreciate you coming on. Thank you, Dave. Welcome to having you. So Edge, the discussion last year actually wasn't much about Flash. You had some partners here. You guys were reselling some other people's products. That's cool. Bang, you make the TMS acquisition and all of a sudden, in typical IBM fashion, you're smoking. Got us off and running. That acquisition, kind of the first start tipping point, if you will, tip of the iceberg in terms of getting the Flash ball rolling and really getting the value out to our customers. Yeah, and then April 11th, Steve Mills announces you're going to drop a billion into Flash. And he's done that before. He said that in analytics. He ended up putting in 10 billion. Right. Talked about that back in the Linux days. You know, I'm going to bet a billion dollars in open source. So you guys have a good track record there. And we've talked, we talked to Mike Hewn earlier today about, okay, what's happened since the 11th, but he underscored this big uptake, talked to Laura Gio, said, pipeline's good. You like that. So you guys had some announcements at this event. Talk about those a little bit and then we'll get into it. Sure, really the focus of this event has been extending the overall Flash portfolio. So on the April 11th event, we announced our IBM Flash system products, the standalone Flash arrays. But part of the focus of this event has been extending the fact that it's not just the IBM Flash system products, but the broader Flash portfolio across IBM. And so a couple of things announced in particular, some things around software. So some caching features, both in our DS8000 portfolio, we're actually taking the easy tier technology in the DS8000 that does tiering between disks and Flash today and extending that into the Flash in the server. So now we're connecting the Flash capability of the storage with the Flash capacity within the server and connecting that together with software. So that flash capacity in the server is a PCIe connect? Correct, that could be an SSD drive form factor, two and a half inch, one point inch, or it could be in a Flash adapter form factor, so the PCIe cards. Okay, and then so help me understand the connection between that layer and the DS8000 in this case. What are they doing? How are they talking to each other and how does that translate into business value for customers? Sure, so first it's kind of two layers if you will. So first there's caching and then actually with the DS8000 we're achieving cooperative caching. So first in the server at a caching, what the software will actually do is, as you do reads and writes from the server to the storage, the server will actually determine, well what am I most likely to read next and keep a small amount of that information local on the server within Flash within the server for a small amount of capacity, very high response time co-located right in that server. And so that's caching we actually could deliver. We did a statement of direction on software where we're going to deliver a system X that they'll do caching, as well as caching now with the DS8000. So sorry, that's a read cache. That's a read cache. Well, so it's not a persistent medium, is it? Correct, it's a read cache or a read cache right through. So any writes would happen, would write through to the persistent sand behind the data, behind the server. Which would go to the write cache in the array and then eventually asynchronously trickle back to the spinning disk. Correct, so it could be to a storage array that is all Flash or it could be to a storage array that was disk or even a mix of Flash and disk in the storage. So a storage array that's all Flash, obviously you don't have a destaging bottleneck. Correct, correct. Storage array that's spinning disk, you're going to back up at some point. Right. Okay, and in the old days you'd add more disk, maybe you're better off adding Flash. Okay. So we've been focusing a lot prior to this with the DS8000 on easy and store-wise portfolio and SSD and volume controller on easy tier, the ability for the system automatically to move data between a Flash sheet level tier and a disk tier for hot data. And what this announcement now enables is to extend that all the way to the server so the system can really prioritize what information is hottest, it shouldn't stay even on the server in Flash, what information is hot, but maybe need across multiple servers so it's on Flash within the storage and then what information's colder and I'll put on spinning media in the storage. Kevin, I'm wondering if you can talk a little bit kind of competitively. Last week, Dell talked about their Flash Cache technology doing something similar. Month ago EMC talked about how they've got their server-based solution going down. We've dug through a lot of the tiering solution. IBM tends to focus on really trying to fully automate that technology versus trying to put too many knobs in there. How does this fit into that and what can you say about the competitive aspect against what you've seen out there? Sure, and that's where, there are a couple of vendors out there, they're starting caching is obviously not a new concept. So customers and vendors out there with caching, which really key and different from our perspective is with the DS8000, it's cooperative caching. So what that really means is instead of just individual servers doing caching, the DS8000 and the easy tier functionality is aware of all the cache on the different servers and can act as a central intelligence to control all of that information. So if that centralized server needs to do a copy, a mirror, a backup, or it even knows one server has written the information and that's devalidated, it can tell another server, hey, your cache is no longer the latest, go update your cache. So it's taking the caching technology that you might deliver just on the server side and giving it kind of a centralized intelligence with the storage environment to coordinate multiple servers doing their caching as well as tiering within the storage itself. So do we get through all the announcements? Well, if we've gotten up to the announcements, if I look at kind of the Flash portfolios out there, there's the server-based solutions, there's that mixing between the server and the storage, you've got the Flash systems, which is kind of the point solution for all Flash. What about hybrid systems? It's been getting a lot of buzz, there's a lot of startups in that space, new architectures that has, not just the one to 2%, but more Flash in there, kind of the Flash first when it comes to the rights. Does IBM have a play there? What do you think about that? Yeah, so in the hybrid environment, that's actually, we've been doing that in our store-wise in DS8000 since 2008 and the new Flash systems are kind of all Flash array. So to put into context a little bit, step back, really the IBM Flash portfolio, it's about three different major categories. You have IBM Flash storage, which is our IBM Flash system, or all Flash DS8000 or hybrid DS8000 store-wise products. You have IBM Flash software with the capabilities we just talked about with things like caching, and then you have our direct-attach portfolio. So IBM Flash adapters, Flash drawers that attach directly to our power and our system X servers as well as disk capacity. So the hybrid environment, we're still seeing a lot of customers' interests where they're using the Flash, particularly with easy-tier technology of store-wise, or SAM volume controller or DS8000, to optimize, okay, I have a Flash capacity for my house data, I'll see 200, 300% performance improvements over with my current capacity by putting in just five, 10% Flash capacity in front of my current disk capacity. Okay, so do we get through the announcements or anything else that? That's the key really, the software is a new piece and then just accentuating the fact of the broader portfolio. So we talked a lot this week about customers putting a couple use cases. One is the application is doing the storage management, replication, data protection, whatever, and you drop a Flash system right in and it's pretty clean. The other use case was dropping behind in SVC, we had one customer, Sprint was on today, we talked about that with Kareem Abdullah. Where do you see that going? Is that, are you guys, question, a lot of people ask, is that a stop gap? Are you going to try to build out your own stack with Flash systems? Or is SVC the stack strategy? Can you talk about that a little bit? So there's two parts of that. There's the fact of current bundling buying the two products together and putting it and then there's what's the right software feature stack and while the bundle itself today is a stop gap that we're continuing to invest on, the core SVC code that we leverage across Sand Volume Control StoreWise is a strategic code base for us and feature set that we are looking to bring to the Flash system portfolio. So you'll see over time, and we're seeing the marketplace is the all Flash array marketplace really bifurcating the two aspects. One being customers focused on pure performance. They want the lowest latency, highest performance, they don't want any software features or anything getting in their way. They're purely performance driven. And then there's another set of customers we're finding that need high performance, need low latency, but they also need some of the enterprise feature stack, copying, mirroring, virtualization capability. And that's where today we recommend the Sand Volume Controller with their IBM Flash system products and in the future, that's where we're going to be integrating those technologies together. So as a single IBM Flash system offering, you have the choice of pure performance or performance with feature rich software stack. Okay, so when you say over time, you're going to integrate those, you mean is that a packaging go to market strategy? Or are you talking about extracting the function out of SVC and super gluing it into... I'm talking literally about taking that software code base and bringing in, doing further performance optimizations on the hardware. So right now we have a system where you can use a Sand Volume Controller in front of a Flash system. We've tested them, we've tuned them, they work great together, but we also think there's places in the future where we can actually bring that software code indirectly and optimize it for that high performance Flash hardware base and deliver even further integration. I see, okay, so make that code portable over time. And then of course, and then it fits into an SDS strategy, perhaps, it's behind an SVC or like you say, direct connect to where the application does a pure performance and nothing get in the way or something in the middle, which has a lot of legs as well. Right, right. So you have that choice and balance of absolute extreme performance, lowest latency, or I need high performance low latency, but I need some mirroring, some copying, some other feature-rich capabilities.  What were some of the decision points that went into that? First of all, how would you describe sort of the flavors, the choices that you had and why TMS? Sure, yeah, actually I have been involved since the due diligence and that was a great experience. I think one thing we see in the marketplace, as in a typical environment that was up and coming, new ideas, a bunch of small companies coming out, maybe some VC funding, some ideas, but not necessarily a lot of companies that had really market experience. And that was one of the things we fell in love with Texas Memory System. They had over 30 years in the marketplace, they had a product that was out shipping, proven with customers today, and had that real life experience versus just good ideas. So in some instances, we almost referred to Texas Memory System as a 30 year old startup, because they really almost acted in that fashion, small team, very energetic, very quick, from a development standpoint, but still real market experience, having been there, having sold in the marketplace, and dealt with DRAM based products originally, then flash technology, but solid state storage devices, really further duration. So what do you think's going to happen in this business? You would expect a lot of consolidation, right? The market's really not going to support whatever, I don't know how many vendors are out there, it's got to be 50, 60, some way, shape, performance, some sort of leading the pack, but you guys have made your move, EMC has made its move, see what HP does. Oracle has yet to make its move, although it's developing some stuff that appears internally, there's just, there aren't a lot of whales left, to buy these companies. So what do you think's going to happen? I mean, do you think it's going to be like the tier 1.5 was, where they all disappear and get absorbed, or is this market big enough and hot enough to support some startups going IPO, or do you have an opinion on that? Yeah, I guess my own personal opinion, I think you'll definitely see the consolidation to particularly the major storage vendors, particularly in the enterprise marketplace. I think there's still places where we'll see innovation and software feature stack and capabilities that will cause new ideas to come up and new startups. So to me, it won't just consolidate, there'll always be fresh ideas, but it's becoming a scenario to us where flash is everywhere. I mean, you walk around an event like this and pretty much almost everyone has flash in their pocket with a cell phone or some other device. So it's really getting a point of a matter of not are you going to use flash, but how much are you going to use and are you going to use it in your server, your storage, where are you going to integrate it in? So we do see that consolidation in the marketplace. Yeah, and we were talking before about the DS8000 and basically how that's incorporating flash and you're seeing that across the industry, but part of me just says, well, at some point, why not just drop in and all flash array and I'll be happy. Like my laptop doesn't have a spinning disk. I swear I'll never have a spinning disk again. I've sworn myself off spinning disk. Not that there's not a role for it. There certainly is, but the idea of short stroking spinning disks is sort of arcane. So am I overstating that?  Well, and tapes are a very good analogy. So just like tape, I don't think the spinning disk will go away completely, but as you mentioned, doing things like short stroking, 15K speed drives, trying to do the high performance disk segment as spinning disks, that is where flash is really taking over. And that's where we really see that market disappearing as a disk market and the high performance that moving to flash, but disk will remain as the larger capacity, you know, slower speed for the cold data. And just like tape's still around and still viable business for that coldest data, we really see that becoming the scenario with disk, where you'll have a mechanical spinning disk for the large capacity and flash really being your performance layer. What are your, I wonder if you also have an opinion if you could talk about the whole notion of atomic rights. We've talked about a lot about that at Silicon Angle Wikibon. We get excited about it because of the potential to change the way in which applications are designed. I know you guys have, you know, partnerships with guys like Fusion IO and you've looked at that very carefully. What are your thoughts on that as far as the viability of that? Is it problematic and that essentially you got to change the way in which you write software? But or at the same time it's appealing and alluring because you're bypassing the overhead of say the SCSI stack. I wonder if you could talk about that a little bit and share with us either your point of view or IBM's. Yeah, so I think that's one of the places where we are seeing innovation above and beyond just, you know, faster hardware, faster NAND capability with things like atomic rights. And I do think it's going to be disruptive to the software stack, but that flash overall is almost disruptive to the software stack. So we're already in a lot of conversation with our own DB2 teams and other ISVs and application vendors to say, okay, you've done all this work prior to now to avoid IO. You've tried to do in-memory database and do other things to avoid IO. Now you have high speed IO. Now you have all flash storage. How do you better optimize that? How do you start changing your application stack? So whether it's atomic rights or other innovations, I think there is going to be a change in the application layer and the OS layer to start to say, hey, we have to take advantage of this capability in servers and in storage because that IO bottleneck's not going to be. Yeah, and in-memory's cool, but it's expensive and it's not persistent. Correct. So those two things are problematic and presumably flash can solve that. But I wonder if we could have a follow-up question on that. So you've got a standard emerging NVMe, which looks like it's going to take forever to get out of the committees. It'd be great if we're here today and the industry could adopt it. So is IBM strategy to wait? Will you try to develop your own or no comment? Well, I can't comment specifically about that one, but I will comment. I mean, we're not waiting in overall, right? So we're investing a lot of money with the public one million, sorry, one billion statement in the investment, but our R&D team, even prior to this, has always focused on how do we better leverage flash technology? How do we better leverage the application stack? And we always as a company try to be a leader in terms of setting standards in the industry.  The problem with atomic rights is to get there, it's very disruptive. You can't get from point A to point B without essentially either ripping or replacing or completely changing the way you develop applications. Technically, do you think it's even feasible to have a technology that is not disruptive, but still gets the benefits? Or even something that's close, maybe even some kind of hybrid. Is that even technically feasible? Well, I think we are seeing a move where there's a lot more focus of the application stack on fully integrated solutions. And that's the easiest place for something like that to start. Not going out and trying to deliver a standalone storage device, and you're not sure what the OS, the application environment, but things like peer systems where our peer data for operation on the lakes today uses a large flash capacity. Places like that where we are delivering the application layer, the OS layer, all the way down to the storage, I think that's one of the places where you'll start to see that test out where you can deliver specific industry applications, leveraging that capability because you have the complete stack. And I think we're uniquely positioned to be able to deliver that complete stack solution and then through our customers. All right, Kevin, I'm going to give you the last word here. You're talking to customers. You know, they're sort of questioning, ah, flash is expensive. I'm not really sure where to use it or I can use it a little bits and pieces here, but I'm interested. I want to get started. What's the advice that you give to those clients? I think the biggest thing I would say is, you got to try it. So we have established flash centers of competencies around the world, benchmarking centers where customers can come in and get their hands on. I've been through multiple product generations across server and storage and I've never seen anything like this that has the success ratio. When customers actually get their hands on it, they run their production data on it. We heard from some of our customers earlier today saying, well, hey, I put it in our performance in our test environments, but then our production even performed better than I expected.  Drop it in and hang on for the ride. Kevin Powell, thanks very much for coming to theCUBE. Really appreciate it. All right, keep it right there. Everybody will be back with our last guest and then we're going to wrap. Vincent Su is coming on. He's the CTO of the IBM Storage Business. Storage is changing. We've been covering it here for the last two days, IBM Edge. This is theCUBE. Keep it right there, we're right back.


For 34 years, she dealt with her ears being muffled. She said during worship, all of a sudden, it went loud. She's been healed. You gotta tell me about that moment. I just remember as the worship team was singing, I literally thought that somebody turned all the volume up in the place. And I was able to hear the instrument so clearly I could hear my little sister singing. It was amazing. Wow! I'll tell you this right now. Jesus and Jesus alone deserves the glory, the honor, and the praise.


One of the real alarming things, but you see it over and over again for years now, from the left, from the kind of Marxist left, is the attitude towards work. So I've been getting these questions, right, from students or young kids. And the question basically boils down to, why should I work? I don't want to work. I want to pursue my hobbies. I want to have fun. I don't want to work. And it's like, what? It's so, it's so weird. Now, where does that come from? It comes from Marxist's vision of a utopia. So in the Marxist utopia, you don't have to work. In the Marxist utopia, all the material needs that we have and needs here for Marx would include the iPhone, would include everything you need to be successful, to be, to live successfully, quote, right, is just provided because, partially, because capitalism has been so successful, right, Marx would say, right. But capitalism has been successful in producing stuff. Then the parliamentarians take it all over. They automate it. It's all robots now. So this is the utopia. It's all robots. So all this stuff we need is being produced. Doesn't require human thinking and human effort. You know, unless you want to go into that field, and then it's fine. And then everybody gets to do what they want. So in a Marxist utopia, you don't, it's not like nobody does anything. What you do is you pursue your hobbies. Like everybody has a hobby and everybody pursuits their passion and everybody's happy and floating around and, you know, it's just wonderful. And the stuff you need just appears. It's like manna from heaven. It just drops and it arrives. And if you want to be a work, if you want to go and build stuff with your hands, you can do that. If you want to start a business, you can do that. You just can't have employees, you know, to each according to his ability from each according to his means. But you can do the activity, right, theoretically. And the one thing Marx never explains is how we get from what we are today to that utopia. Like the dictatorship of the proletarian, that party skips over, right. He describes it says, this is what has to happen. And then he goes to the utopia part. And so they take the utopia part as well. Of course, we don't want to work. We don't want to have a job. We don't want to have to produce stuff. The world is rich enough for stuff just to be provided to us. And it's like no sense that production requires effort. Production requires a mind. Production requires thought. Production requires, you know, people to actually go to work. Work in the broadest sense. And that they want to just skip to this utopia without having to do anything. And it's, I don't remember 10 years ago, people having that attitude. This seems like a new, like the left has adopted this new tactic of promising young people the utopia tomorrow. Like Marx, you had to go through a revolution. There was a whole process, right. And ultimately over a long time, which he never explained, you get to this. But they want to know right now, tomorrow. I remember a few years ago at the University of Texas, this kid came to the microphone and said, why should I work? There's enough wealth in the world just to provide me for the living. It was like, why should I provide you with a living? Because I don't want to work. It's my right. And it's one of the most bizarre of all the kind of questions, attitudes that I find out there. Because it just goes against what we know about reality, everything we know. It's kind of a metaphysical issue, right. For human beings to survive, we have to shape our environment. To shape our environment, we have to work. That work requires effort, requires thinking, requires changing the stuff out there. And some people don't get that. I think most people do. I don't think it's hugely popular, but there's a signal. The ones that are vocal, the ones that are passionate about their Marxism, that's where they are. They're at that level of women worshipping. I don't want to work. I actually do want to work. That's what part of my answer was, no, you should find something you love doing. And go do it. You know, work is not a burden. Work should be fun, exciting, thrilling. You should get up in the morning excited about going to work. It's sad that you're not. What we need today, what I call the new intellectual, would be any man or woman who is willing to think. Meaning, any man or woman who knows that man's life must be guided by reason, by the intellect, not by feelings, wishes, wins, or mystic revelations. Any man or woman who values his life and who does not want to give in to today's cult of the stare, cynicism, and impotence, and does not intend to give up the world to the dark ages and to the role of the collectivist roads. All right, before we go on, reminder, please like the show. We've got 163 live listeners right now, 30 likes. That should be at least 100. I figure at least 100 of you actually like the show.  There you go. Start liking it. I wanna see that go to 100. All it takes is a click of a thing, whether you're looking at this. And you know the likes matter. It's not an issue of my ego. It's an issue of the algorithm. The more you like something, the more the algorithm likes it. So you know, and if you don't like the show, give it a thumbs down. Let's see your actual views being reflected in the likes. But if you like it, don't just sit there, help get the show promoted. Of course, you should also share. And you can support the show at urunbrookshow.com slash support on Patreon or subscribe star or locals. And show your support for the work, for the value, hopefully you're receiving from this.  You'll know what shows are on, when they're on. You'll get notified, right? So yes, like, share, subscribe, support. Like, share, subscribe, support. There you go. Easy. Do one or all of those please.


Hi everyone, Joe for JaspeysCaseRakes.com, coming at you with a one box break of 2022 Panini Impeccable WWE Random Number Block Break Number 10 from a fresh case, 10, 11, and 12 will be from the same case. That's how the number blocks work. If you are not familiar with it, there it is. Big thanks to this group right here and there are the numbers right there. Let's do it. Let's roll it, randomized names and numbers, three and a six, nine times each. One, two, three, four, five, six, seven, eight, and ninth and final time. Got Craig down to chat. Ram not Org running fast tonight. I like that. Three and a six, nine times for the numbers. One, two, three, four, five, six, seven, eight, and ninth and final time. After nine, three down to two. All right, Craig with the number three, Chad with four, Michael with five, Josh with nine, Chad with six, Tim with eight, David with one, Shay with zero, Edward with seven, and Chad right with two. Sort that numerically. So, Shay, you get any and all redemptions in that one box break, including one of one redemptions. Dave, you'll still get live one of ones. There you go, gang. I'm gonna pause the video. Maybe we'll see some number trades. Maybe not, usually not, but I'm gonna go grab the case. We'll come right back and pop that baby open. Stick around, BRB. All right, welcome back, friends. No deals were done, so that list right there remains the same. There's the fresh case. This is also our last case. I don't know if we're getting any more, but as of now, we don't have any more to post after we finish these three boxes. So just letting you know. We've got Chris Jaspy in the background here. He knows a lot more about this up than I do, so he's gonna kind of lurk a little bit and see if there's anything fun in here. He'll let me know. All right, so let's mark these. That's box one, that's box two, and that's box three. So on the dice roll, we'll go one, two, three, four, five, six on the dice roll. And it's one, top box. Box one, easy. Next time we'll go one, two, three, four, five, six. Next one's up already.  Yeah, Mike, thankfully we don't have to play any Who Am I? Purely based on my WWE knowledge. Oh, that'd be fun though. That would be fun. Not for you. For you. For you. For you. I'd be like, this was the girl, that was one, so good guy, but now she's a bad guy. And she used to be like a candy raver, but now she's like wearing dark lipstick and is mean. Were you a Ripley? Maybe. Lita? Don't know. Bailey? That would be Bailey, actually, a good call, Joe. See, I know some things. Undertaker, 82 out of 99. That'll be for Chad Wright and the number two. We'll go by that digit right there in case you're new to the number blocks. Top 10 wrestlers who turn heel. Sometimes you watch those videos.  Sometimes you learn things. 17 out of 35, the almighty Bobby Lashley. Go to number seven. What can you tell us about Bobby Lashley, Chris? He's strong. He's actually fighting Brock Lesnar this Saturday. Wow, is he? Yeah, he's doing, that's like kind of a big match that they've been building to for a while. I know Brock Lesnar. Edward with that one. All right, next one is Edge. No, not the guitarist for you too. But that's the Edge, right, definitive article. This is just Edge. Although I guess no court would ever say people would confuse these two. That'll be for seven. It's also for Edward with Edge. We've got Santos Escobar, 15 out of 49. Michael Estrella with the number five. All right, now to our hits, I think. Yeah? Yeah, we got Hall of Famer, Tori Wilson. Hall of Famer in your book? She is in the Hall of Fame. She is in the Hall of Fame. So that Chris does not believe she should be. You can hear it in his voice. Controversy. Comment in the chat. I'm just kidding, don't comment. But if I was like that guy, he's like, hey, be sure to comment whether you agree with Chris or not. I'm by the Tori Wilson should be in the Hall of Fame. Actually, don't ask, I don't care. Yeah, don't ask, don't ask, don't ask. That's not number. Oh, no, it is number. I'm crazy talk. 15 out of 19. So that goes number five. That'll be for Michael as well. A plate auto, these are pretty cool. It is Finn Balor. Stainless steel star, is he a star? Yes. Oh, okay. It's quite good. All right. This is a purple parallel here. This is 16 out of 49. Chad Wright with the number six. Who would win a cage match between all Jaspie employees? You're looking at him. It would probably be Chris. I think both both in size and aggression. That combo is pretty dangerous.  He's like the Hulk. Zach Gibson, six out of 10. I would definitely not be the first one out though. I may not be the last one standing. I don't think I'll be the first one out. I'll put it that way. Six out of 10. Number six Chad Wright. Is that Gibson? Next? I mean, these are, this is like prospect league, right? Yeah, that's like their kind of thing before what's called the main roster. It's a former NXT UK. It's like the corn fairy tour in golf. It's kind of like triple A. Kind of, okay. All right, a couple more here. We got Mark Coffey, illustrious ink. 21 out of 99. There's a UK version of this? Yes, there was. It got kind of closed down, but they're going to do NXT Europe instead. So you make a bigger figure. Wealth of knowledge here. 20, so this one might be a little bit of a keeper. David Chan with the number one. Nick would win because he signs the checks. I mean, maybe if Nick got beat down, maybe someone else would be signing the checks, Kevin. Just, maybe it'll be just like Hunger Game style. And the last one here. Ooh, I know this guy. Long before his Hall of Fame induction in 2018, this guy skyrocketed at the top of sports, the sports entertainment world with an unprecedented 173 and zero winning streak. This superstar absolutely decimated anyone and everyone on the WCW roster, including legendary names like Kurt Henning, Sting, Diamond Dallas Page. You know who that is? It's Bill. Goldberg. Stingless steel. That looks sharp. Seven out of 75. That's Edward. That's Riffer with the number five, seven that is. Edward. Nice Edward. There you go. There you go. Too much luck before now. That's what he was saying. He said the pressure was on me and I said there was no pressure. Pressure makes champions. I've already been through pressure. That's why I'm a diamond. Exactly. I'm Joe for Jazzpiececasebreaks.com. Your diamond. I'll see you next time for the next one box break of WWE. I might not be without Chris's help and I'll be lost. All right. I'll see you next time for the next one. Bye bye.