triangle inequality. However, here, we have another caveat. All of this quantity is called the weak LP norm. It's not a norm. It doesn't satisfy triangle inequality. It's one homogeneous, but doesn't satisfy the triangle inequality. However, there is an equivalent norm for all weak LP spaces with P greater than one. There is a real Banach space norm, which is equivalent to this quantity. Moreover, if I consider P greater or equal to 2, which we can afford, then the coefficients of this equivalence are independent of P. So I can't treat it like the Banach space norm, and then I can write the norm of the sum J from 1 to 1 minus epsilon n y J weak LP is less or equal than a trivial triangle inequality. I'll multiply by the number of y J's. So each norm is C over epsilon n, and I multiply by 1 minus epsilon n. I'll get C over epsilon, actually C times 1 minus epsilon over epsilon, but 1 minus epsilon is a small change. We would not gain anything from it. And we got that translating it back for any positive, the probability that the sum J from 1 to 1 minus epsilon n y J is less or equal than tau over epsilon is less or equal than C over tau to the power epsilon n over 2. So here we used a functional analysis theorem. This is a standard fact, but it's not very trivial. You can look up this renormalization theorem, for example, in the book of Stein, Harmonic Analysis in RN, and the proof is, occupies some three pages. If you don't want to go this way to take this shortcut, you can argue on the level of the standard LP spaces, this is a little longer, and I showed the estimate in the notes. So before let's do one cosmetic thing, I don't like this epsilon n over 2, and I'll replace tau by theta squared, and then I'll have theta squared in the denominator, and this will give me epsilon n. Great. But all this was done because we, not because we want to estimate the distances between vectors and subspaces, but because of the negative second moment formula, and at this moment I can write that for any theta positive, the probability that the sum j from 1, 1 minus epsilon n as j of b to the negative 2 is less or equal than theta squared over epsilon is less or equal than c over theta to the power epsilon n. So the complimentary event now is likely. So omega theta b, the event that this sum, sorry, is greater, it was greater or equal, and here it was greater or equal, because I estimated the tails, and then I applied, so the triangle inequality yields the norm of the sum, and then this norm of the sum is translated to the probability that the variable is greater than something. So let omega theta b event that the sum j from 1 to 1 minus epsilon n as j of b to the negative 2 is less or equal than theta squared over epsilon, and we will choose theta in a moment. Now I know that the probability of omega theta complement is less or equal than c over theta to the epsilon. Assume that this event occurs, and then, of course, not many singular values can be very small. We can use Markov's inequality if you wish. The negative second moment anymore played its role. So I take theta to be delta square root of n, the probability that the sum j from, no, not the sum, the probability that the cardinality of the set j, which as j of b is less or equal than delta square root of n is, sorry, it's not probability any longer. I assume that the event omega theta occurred, so this sum is small, and then I play Markov's to the counting measure, so it is less or equal than, less or equal than, it's the number of all j's such that as j to the negative 2 of b is greater or equal than 1 over delta squared n, this is less or equal than delta 1 over delta, delta squared n over theta squared, theta squared epsilon. Now, delta is up to our choice. Let's choose, let's choose this delta to be 10 theta epsilon. Then theta square would cancel out and 1 epsilon would cancel out, so the cardinality of the set of all j's such that as j of b and b, I recall, is the matrix we want a minus lambda, so is less or equal than 10 theta epsilon square root of n is at most epsilon n over 100. So this is what we managed to squeeze from our argument. We were not able to show the bound for the minimal singular value, but we managed to obtain a bound for the number of smallish singular values. In general, the smallish singular values have nothing to do with the minimal one. Those who tried to, for example, to prove the circle or know that these are two absolutely different problems. So we worked hard. We got some bound, but the bound seems useless because we really need the smallest singular value. Actually, we are almost done. There is a trick which will allow us to finish the proof quickly. Let's redraw our matrix. So it's a minus lambda. This was n by n. Then I dropped a set i of cardinality, epsilon n, and I got 1 minus epsilon n by n matrix. And the difference of the number of rows and the number of columns is epsilon n. Let's cut here epsilon n over 2 rows and call this matrix this upper corner, say b. This will be our new matrix, b, and this will be the matrix g. So what are the sizes of these matrices? b is n on this blackboard. b is 1 minus epsilon over 2n by 1 minus epsilon n, and g is epsilon over 2n by 1 minus epsilon n. Okay, and let's look at the matrix b. The difference between the number of rows and the number of columns is epsilon over 2n. Epsilon n or epsilon over 2n doesn't matter. It's a constant coefficient. We can apply the whole story we told up to this moment to this matrix b. And we can get the bound on the singular values of this new matrix b. But then we saved this piece, the matrix g. And let's recall the dependencies. The entries of this matrix g depend on other entries of the big matrix, but these entries are here. We have thrown them away, which means that the entries of g are independent of the entries of b. We bootstrapped ourselves. We have the same estimate for the singular values of the sub matrix, and we have an independent matrix at our disposal. This would be enough to finish the proof. Let's see. Let's take a singular value decomposition of this matrix and let me call it say b hat. So b will be the big matrix, and b hat will be the top corner. And let's take the singular value decomposition of b hat, and we'll denote e plus to be the linear span of the singular vectors uj such that sj of b is greater or equal than whatever we put there, 10, greater or equal than 10 theta epsilon square root of n. Good. Then I know that the and the space c1 minus epsilon n will be decomposed as their orthogonal sum of e plus and e minus where e minus is the subspace spanned by the other singular vectors corresponding to the small singular values. So what can we say about these subspaces? First, we know the number of smallish singular values. So the dimension of e minus the smallish singular values is at most epsilon n over 100. Second, on the space e plus and the space e plus is spanned by the singular vectors, all the singular values are large, which means that for any z in e plus the norm of b z is greater or equal than the smallest singular value on this subspace. It's greater or equal than 10 theta epsilon square root of n. Great. So we need to estimate the minimal singular value of b, which is the minimum over z in c1 minus epsilon n norm b z 2. And on this event omega theta, which is the likely event, we estimated this norm from below, like we need something like something times square root of n, but not on the whole space on a big subspace. And we do not know anything about what happens on the orthogonal subspace. The output doesn't provide any information about it. But for this, we prepared the bottom matrix, g. Let's see. g is independent of b hat, and we will use g to get a bound on the orthogonal subspace. g is a short and fat matrix. It has only about, sorry, epsilon n. Not epsilon over n, of course. Epsilon n rows, and much more, many more columns. So its kernel is huge, but I'm not going to consider it on the full space c1 minus epsilon n. I'm considering it on the space e minus, and the dimension of e minus is epsilon n over 100, which means that efficiently I am considering it as a matrix with epsilon n rows and epsilon n over 100 columns. This is a tall matrix. It's very well invertible, and its invertibility can be proved by the straightforward epsilon net argument.  We needed this epsilon net argument. We cannot apply it for the whole space, but it comes handy when you want to control small dimensional subspaces. And then, just considering the norm of, so for any z in e minus, the norm of g z2 is going to be z, if I don't normalize it, is going to be large. It's going to be like the number, a square root of the number of rows times the norm of z2. Okay, and this is with high probability, with probability exponentially close to 1, due to the most straightforward epsilon net argument. And now I have a lower estimate on one subspace, and another lower estimate on the orthogonal subspace, and we can finish. Let's see, from this moment, I don't need any probability. I just write that for any z this time in the whole space c1 minus epsilon n, the norm of d z. Since this was a singular value decomposition, these subspaces e plus and d minus were created from the singular value decomposition, I can write that this is the norm of b hat z squared plus the norm of g z squared. And then I will be done by the elementary optimization argument. For any vector z, either the projection onto e plus is large, and then this quantity is large, or the projection onto e minus is large, and then this quantity is large. And then you optimize it. The only thing to take care of is the probability. And the probability here is defined by the event omega theta. So what we are requiring from this probability, let's go back to our, one would be enough. Okay, let's go back to our translation into the smallest singular value. We have to beat the binomial coefficient n over epsilon n. We need super exponential bounded n over epsilon n is about exponential, let's write this way, is about one over epsilon n to the power epsilon n. The power epsilon n I have here, so I just choose theta to be a constant times epsilon n and everything fits in its place. And we completely established the no-gaps delocalization. So next time we are going to discuss the applications of this result to random graphs. Okay, questions? Of b hat, of course. We repeat the whole story for b hat. Then these are defined by b hat and they become independent of g.


Okay, we're back live in Silicon Valley in Santa Clara, California. This is Stratoconference, O'Reilly Media. This is siliconangle.com's coverage of Stratoconference. This is theCUBE, our flagship program. We go out to the events, extract a signal from the noise, talk to the smartest people we can find, entrepreneurs, developers, CEOs, startups. Whoever can bring that signal, we will try to bring that to you with great questions. I'm John Furrier, the founder of Silicon Angle, and I'm joined by my co-host. I'm Dave Vellante of Wikibon.org, and we have an interesting guest right now from Intel. Boyd Davis is the VP of Marketing, and Intel made big news today, announcing a Hadoop distro based on a 2.x platform, made the announcements in San Francisco. Boyd made the trip down here to be on theCUBE at Strata and joined the show. So, Boyd, thanks very much for coming on. Hey, it's great to be here. One exit up from Intel, so we're close to corporate headquarters. Close, close to the mothership, yeah. Obviously, Intel is no stranger to innovation. Obviously, going back and Dave and I can show the gray hairs. Intel made the computer industry and extracting away complexity, and that was they made their bones on Moore's Law and making things faster and simpler and lower cost. And now, with the big data world exploding, where storage and compute is kind of blurring, but at the end of the day, software's driving everything. So you guys, although chip company, have a lot of software. So, one, tell us some background to Intel coming into the distribution of Hadoop. Obviously, it's a big data market. It ranges to analytics today, applications in the future, and certainly infrastructure will be affected. So tell us why and how you got here and then what you guys announced. You know, it's actually pretty close to our heritage. Intel has had a long history of participating in software projects. And historically, what we do is we engage with software providers and enable the software, contribute to open source directly, and really try to make sure that the software out there took advantage of the innovations in the underlying platform. And that's exactly what we're doing with the Hadoop framework. We believe that there's a huge opportunity to advance the Hadoop framework in performance and security and reliability. And the only difference this time is instead of just enabling, we think it's important for us to bring a commercial offering to the marketplace because we can provide some stability to the Hadoop ecosystem. You know, you have two different characteristics of players, some great venture startups that are innovating and then some traditional big companies that are integrating Hadoop into their products. And we're offering that middle ground of establishing Hadoop as a horizontal layer and creating a foundation for innovation and then making sure that customers know that there will be an open, open source Hadoop that's available for a long time so they can really remove the uncertainty and get those big data projects rolling on top of Intel's distribution. So we were talking off camera, David Fleur and I, who we talk about all the time, mobile, cloud and social. And we always talk about market share, consolidation.  Are we hinting at some consolidation? I mean, if Cloudera gets bought out by Oracle, that changes the game, look at MySQL, you know, things like that could happen. You just never know, but you guys are out there saying we're going to be a stable force. Well yeah, I certainly let you guys do the speculation on how this industry plays out. Our partners tell us. Or Cloudera gets bought by Intel, no. Our partners, we had partners like SAP and SAS today and Cisco and Savis. And one of the reasons they tell us that they want to work with us is one, the technology. You know, one is we really do know the underlying compute capabilities, memory and non-volta memory in particular, where leaders in 10 gig ethernet, that's one. But they also want to work with us because they know we're committed to this horizontal platform for innovation. We're really not seeking to go where everybody knows all the value is, is up in the stack where the vertical applications are. And that's where we want to enable our partners to succeed. We're really creating a platform, much like the Xeon platform in the data center is a platform for our customers to bring great infrastructure to market. This is an ingredient platform that allows people to bring big data analytics to market as a solution. So that stability and that notion that we're going to be committed to open source, keeping this as a horizontal layer, and that we're going to have the best underlying technology, that's why people seem to be migrating toward our distribution. Boy, tell us some of the things about where you see the market, specifically around abstracting away the complexities, because that's, Intel made their bones on that by extracting away some of the complexity and giving people just a simple look into a chip. And you guys have done that before with it. Dude, what do you guys look at as the key complexity that you're going to provide that reliable, hard and top or whatever you want to call it platform? Well, our focus on driving the Hadoop framework forward is really in three areas. One is interactivity and responsiveness.  And we'd like to bring that responsiveness down so that there's a much more rapid response time. The other thing we're hearing from customers, particularly in the enterprise is security. We're bringing encryption into the file system and cell level production into HBase. And we actually launched, just yesterday, Project Rhino to have a holistic open source view of hardening the Hadoop framework and all the projects for a consistent security, authentication and encryption approach. So that security is really significant. And then finally, reliability and enterprise quality of the stability of the platform. We've got a long experience in the enterprise. So ease of use is important. And I didn't answer your question directly because we are doing some work around ease of deployment with our manager, which is not part of the open source framework. The manager is a separate product. But within the open source framework, it's that security, performance and reliability that we're bringing to the table. And bringing Hadoop to the silicon level, obviously is a key for performance and presumably security as well. Absolutely, absolutely. If you take a look at one of the specific things we've done is take advantage of new instructions that we built into Xeon, actually, just a couple of years ago, that accelerate encryption up to 20 times. Now the reality is when you're talking about a 20 times improvement in performance, you really move away from having it be a responsiveness thing to having it be a possible or not. Because if you had that kind of performance overhead of encryption without that acceleration, people just couldn't afford to use it. So we're absolutely looking at the ground up. We built solid state memory technology into the Hadoop framework. We're taking advantage of advances in 10 gig ethernet and out in time, the future direction of fabrics. So there's a lot of this, not so much building Hadoop into the silicon, but building the silicon into Hadoop. So now you mentioned cell level security into HBase and I can't help but think of the Accumulo project when I think of that. So essentially it's for cell level security, you're talking about a more granular level of security for an open source database like HBase, which is widely distributed. Are customers clamoring for that? Are they at that point where lack of security is an inhibitor right now? Or are you- Absolutely, I think it's one of the major places we see in kind of two areas. One are those obvious organizations that have sensitive data, whether it be financial services institutions or healthcare institutions that really want to take advantage of big data. But the other big drive is for multi-tenancy. The industry really wants to drive Hadoop as a service. And as soon as you get into a multi-tenant environment, the first question enterprise customers want to ask is how secure is my data when I know that there are other tenants on it? So, you know, we're looking forward to working with the community around Project Rhino because security takes a comprehensive approach. So we've done some work around cell level protections in HBase, but Rhino really looks at security, encryption and authentication across the full range of Hadoop projects. And we're going to plan, we've submitted that to the community now and are going to look forward to getting feedback from people like that are working on Accumulo and other projects. Yeah, more of a holistic approach you're advocating there. Now, your point about multi-tenancy is interested and I noted you said Savas was at the announcement. And obviously the services component is huge in this marketplace. We've sized it in services is probably almost half of the opportunity. You mentioned the applications as the value there as well. So what do you see as sort of Hadoop as a service you'd be seen as Amazon attacking the enterprise? You've got partners in your ecosystem. What do you see in that whole landscape? Well, I think it's a way to accelerate the use cases for big data. So you have kind of one of a couple of different models. One is customers that aren't sure exactly what they want to go do. So it's much easier to do evaluations, prototyping in the cloud and to consume Hadoop as a service. Another one in Savas mentioned this morning that when you're already working with a service provider for your infrastructure, that's where your data is. So you really want to start to get value on that data. That service provider is the right place to deploy your big data infrastructure. And then the third class of customers are those ones who really know what they're doing and have their data accumulated in their own environments. And that's where we're seeing people deploy their own solutions. But in both of the first two scenarios, enabling a broad basis service providers is really, really critical. And we're excited to have Savas announce their collaboration with us today. We're in conversations with a lot of other service providers to make sure that all of the service providers are looking for that security, enterprise-class reliability and kind of performance. So we have a good fit for the service provider role. I just want to clarify the role. So you run a software group inside of Intel. Absolutely. And so I'm still a little unclear on the objective here. Is this an accelerant to the marketplace or are you actually trying to drive a large business out of this initiative? Well, you know, it's interesting. You know, I believe that our initiative in the Hadoop framework and the work we're doing around the Luster Parallel File System and some of the other software products we have will ultimately have a bigger impact to Intel in terms of driving accelerated growth for the data center in our Xeon business. But I also believe quite strongly that the only way for us to have that impact is to participate in the market with a commercial offering where customers have to vote on the technology with their pocketbooks. You know, we will continue to do enabling and we have a broad base of ecosystem partners that we're enabling, but in certain cases, particularly where that technology is very close to the underlying platforms, it's important for us to drive advances with an actual product. So we are absolutely planning to build a software and services business around both package applications and open source, typically lower level system software that's more close to our heritage, but we're going to build a business around it. But ultimately by building a business around it, we think we can influence growth in the market by accelerating these key architectural transitions. If we can make big data more accessible, more useful, more safe and secure, people are going to buy more solutions more quickly. So we plan to benefit both ways. Yeah, and obviously the security is a major thing, but also performance you mentioned with cell level security. That's one of the dings on cell level is the performance. But I want to ask you about some topics that have come up with respect to the kind of the data warehouse market that we heard from some of the vendors. With moving data around the network and transforming data sets, it's very expensive in terms of performance costs. So throughput and latency are really big deals in kind of the database world right now. So that's one question I want to ask you your thoughts on that. And then two, what use cases beyond normal, obvious data analytics are you guys seeing? Because Intel is a diverse industry set of solutions from data center to internet of things, to mobile, to everything. You know, I think one of the key reasons why we want to drive performance into the Hadoop framework is because ideally organizations can deploy their data into a single environment and then get value out of it a lot of different ways. You know, we have examples of telecommunications customers in China who are storing their call data records, billions and billions of records. And the first step is to make the data available to their customers online in a simple lookup. But then all of a sudden once that data is there, they can start to do analysis and figure out what's the most popular smartphone or what's the most effective data plan. So it's that versatility of the data that we find to be very important. And one of the reasons why we want to make this horizontal layer kind of a multi-use layer for many organizations. So that's our approach. Now by the way, you know, we're still engaged with some of the large data management vendors and I can guarantee you there are places for solutions like SAP's HANA and Oracle's Exadata and you know, Vertica and Green Plum and Matizah and the long list. We're by no means to a level where the Hadoop framework is capable of taking on all those tasks. But we can certainly drive it forward to take care of more of them.  Relative to some of the use cases around data, for example, data centers that flooded with big data that splunked on one end and kind of a niche there. And then you got all kinds of other data exhausts as Dave and I talk about out there for Internet of Things, probes, everything that's got thrown off data. Well, you know, I think relative to the Hadoop ecosystem and those usage models, I think most of the audience here at a conference like Strata are very familiar with the telecommunications, healthcare, public service and public safety, you know, marketing and use cases, fraud detection. You know, honestly, the list goes on but all of us in a community like this at Strata have a pretty good feel for those. Maybe one distinction for Intel is we are really focused also in our big data strategy beyond Hadoop on the Internet of Things and really the distributed intelligent data that exists at the edge of the network. We've created a framework called the Intelligent Systems Framework that really starts to focus on the connectivity, manageability and security of data coming from those sensor networks. And one of the things you'd like to do is make sure that you can go to the edge and filter out the noise and you have to be really clear on what's noise and what is data. And then also make sure that the data coming from the edge of the network has integrity. So the analysis that comes out of it is worthwhile. So we think we're just at a starting stage. Today, those solutions are very bespoke, kind of customized solutions for every kind of data and every kind of sensor, every kind of edge device. We'd like to move to a mode with the Intelligent Systems Framework where there's some commonality and extensibility so that as you particularly mix data of different types that you have some commonality in the data structures in the security profiles and so forth. So that focus on the internet of things and the Intelligent Systems Framework I think is a little bit unique to us because we really like to drive consistency and scalability into those edge solutions. Yeah, and you know, we're hearing that too because we also, in this market in Strata, we cover incumbent industries like data warehouse, business, intelligence, very low hanging fruit.  They're all doing that green plum. But beyond that, you mentioned the telecom example. I mean, this is really kind of how we see the new world. So the old way and new way, always transitioning. The old way never dies. It kind of hangs around like mainframes. We've seen that, right? So the new way, I want to talk to you, ask you a question about this new way. So the telecom company or other examples, they don't fit into a classic data warehousing model because you're now mixing data sets as you mentioned. How do you see that world evolving? I mean, it's simply transparent in the future.  I got a structured data set over here. I don't want to have to retrofit a data warehouse to a use case, say the telecom records, say, hey, I got fraud detection issues. I could do do lookups, okay, maybe a data warehouse app. But now I want to use that same data to do fraud detection or anything real time. Well, I think the, you know, the origin of a lot of the newer big data tools, things like Hadoop and even like Luster came out of either the high performance computing world who's dealing with computational density and capacity at scale or the big cloud services world who are dealing with, you know, computational scale of a level that most organizations haven't faced yet. And they've come at these with these new breakthrough, many cases, open source tools. And yet those tools are not, you know, going to fit all use cases without support and nurturing. So our point of view is for one, make sure that we drive for standardization and integration wherever possible. You know, we're a company whose business is built on standards and interoperability. We wouldn't exist without it. And then also to take something like Hadoop as a foundational technology and extend it. And think about how do you capture more and more use cases with the same framework to have that framework be more and more versatile. And we're going to be on a multi-year path, a decade long path at least, to get to the point where you can really start to standardize on a set of tools. But I think today we're a long away from that. And certainly, you know, companies really have to think about their data strategies to mix the old with the new. So I got to ask you the Intel Moore's Law question because obviously the history is the culture, you know, Moore's Law is inherent. So when you look at this new business case opportunity that you're forging, obviously it's a little bit, you know, mash up between software and analytics and big data and storage and the, you know, the hardware side acceleration with Silicon, et cetera, whether Silicon goes to Hadoop or vice versa. So what is the internal Intel Moore's Law conference room conversations like when you have to try to peg a number to performance increases? Do you say, you know, we want analytics to double every six months or I mean, how do you guys talk about Moore's Law in context to your market opportunity? Well, across the company and you asked this as a representative of Intel, it depends entirely on the usage case and the user experience. You know, we've made enormous strides in our technology for smartphones in the last several years where the focus is really on getting that rich user experience with a long battery life for a phone and getting the very best connectivity and the very best developer ecosystem. So performance and user experience for a cell phone user is radically different than it is for a data center user where we have seen in the data center, a shift from in the old days, we'd optimize around database transactions, TPCC. That was, you know, that's what you built your performance gains around. And then over time, we recognize that things like Java performance, floating point performance, you know, throughput performance for HPC that mattered. And now of course we're building energy, Hadoop benchmarks, of course, power consumption, you know, things like the spec power benchmark which we have actually led to. But wall clock time. I mean, you said today in your announcement that one terabyte of data could be analyzed within seven minutes versus say four hours previously. Well, wall clock matters, right? Yeah, performance is our heritage. I think that's one of the things that will remain true for Intel is performance is a heritage. But when you talk about how do we use that Moore's law, which is doubling the capacity of the transistors every 18 to 24 months. And we see the next several generations just like we always have. The way we use that transistor budget is all about user experience. And the user experience varies depending on the nature of the user. And increasingly we've gone beyond just thinking about that budget for only performance, which is still a part of our heritage and start to think about how do you use it for graphics? How do you use it for better battery life for a given application? How do you use it to secure identity or privacy or security? So we're really focused as a company now on from the ground up, building our silicon technology with the user experience in mind. And that's what's put us in a position. Where data is at the center of the value proposition? We're computing is at the center. Okay. And how many devices compute? Pretty much everything computes and that becomes the landscape that we're targeting. And as we think about user experiences, it gives us a chance to maybe deliver more of the solution. Well, I want to come back to security because I think it's so vital here. One of your former colleagues or I guess current colleagues, Pat Gelsinger has been in the Cube a number of times. And we've asked them before, is security a do-over? And he flat out, remember John, he said, yeah, it is. And many of our community have said, the way to solve this problem is the way Intel is solving the problem is really from the ground up, a bottoms up approach. And when you talk about internet of things, I guess I just make an observation. I mean, I think of like the Stuxnet virus and a virus going after a programmable controller. And essentially you've got a big task ahead of you to protect fundamental infrastructure of nations. Yeah, and we take that burden of responsibility quite seriously. Security is one of the pillars for Intel. And you saw our desire to take this to another level with the acquisition of Macapie of you years ago. We do believe that the only way to stay one step ahead of the bad guys, and let's face it, we're never going to be secure. We're only going to be safer. And the only way to stay one step out of the bad guys in our mind is to have an integrated solution of hardware and software working together so that you can really focus on the threats. And I think we've seen some good fruits of the Macapie and Intel collaboration on the device side. We're actually working much more closely with Macapie now on data center security topics. And we're looking at how do we make sure we extend that security out to that edge of those intelligent devices that may in fact represent the greatest threat surface of all. Certainly the most consequential when you take a look at where those devices are deployed. So I think customers can trust that we are going to stay on the forefront of integrating software and into silicon. It's not impossible to hack silicon, but it's a lot harder. And in that combination of hardware and software we think will ultimately create safer computing. Well, and it's a virtuous circle too because security gets performance requirements and that presents for the liability challenges as well. So you're hitting on all the marks here. Absolutely. Well, we love the edge of the network, we are really covering, we love it, we think it's going to be dynamic and diverse. I mean, the edge is not like it was before. It's multiple edge points, different environments. Big data is going to be key to analyze that and be real time, whether it's security prevention or user experience. Well, I think distributing intelligence out to the edge is an area that Intel is passionate about driving. We really want to not just view all of those sensors and the edge devices as dumb devices that kind of transmit data foolishly.  Well, boy, my final question is more of an ending question set up for our next interview when you come on theCUBE because we'd love to have Intel on theCUBE so you guys have a great track record of innovation and really make the market in terms of growing and helping everyone else grow in the industry. Pauline is in your group, right? Yeah, if you knew that. Yeah, we love Pauline, she's great content on theCUBE. My question is, what's your to-do items? Now you get the launch out of the way under your belt. What's your to-do list? What are you going to do in the next six to 12 months? What's your key task that you want to knock down? What do you want to nail down to the wall? What's your objectives for the next couple months, next six months to a year? Well, you know, and I'll take you asking me that personally because my scope is data center software at Intel and we're talking a lot about Hadoop and the Hadoop framework has an enormous amount of challenges to drive our next releases, support our partner innovation engagements that we talked to today and drive that business forward and get to a point where we're really helping the community go forward. But in addition to that, we're looking to make the Luster Parallel File System a lot more accessible to a lot more HPC users. That's a technology that Intel acquired a year ago and we have an excellent opportunity there. You know, we have a product called the Expressway Service Gateway that allows people to expose APIs from their legacy environments. And that concept of exposing APIs for organizations for developers is a hot topic all on its own. So I don't know if you guys have a conversation coming up around API management. We're participating there and we're participating in data center optimizations around better power efficiency, you know, more policy-based power management, better workload placement for the underlying performance and more efficient use of non-volatile memory. So our agenda from a software perspective spans all of those. And in the next year, you know, we hope to deliver solutions to the marketplace that customers can really help modernize their infrastructure and get value out of their data. Awesome, well we do love developers. We love data as a development asset for people and also with cloud and convergence with Flash and all these new modern architectures. The game's changing, so we'd love to keep in touch. This is theCUBE, Boyd Davis, Vice President at Intel. Going into the Hadoop market, all in. Intel is always all in and they're going to expand out to do what Intel does best, attracting the simplicity away from the users and developers for Internet of Things, Data Center and Hadoop. This is theCUBE, we'll be right back with our next guest after this break.


Hello and welcome. My name is Mallory Freberg and today I'm going to be walking us through a Galaxy tutorial for reconstructing a transcriptome without a reference using RNA-seq data. Today we are going to be analyzing RNA-seq data, aligning it to a reference genome, but then reconstructing the transcripts without a reference, which is called de novo reconstruction. We are then going to quantify the transcript abundance using our assembled transcripts in two different cell types and then comparing the gene expression across these two different cell types. So we're going to be walking through the tutorial here and we're going to start with the first step, which is uploading data to our Galaxy instance. Today I'm actually going to be using the public Galaxy Europe instance. You can use any publicly available Galaxy or even an instance that you've installed locally. And I've already logged in here, but if you aren't logged into your Galaxy already, you'll want to click the login button at the top toolbar and log into your account. So we're going to start by loading data into our Galaxy instance that we are going to be using to construct the transcripts and do the differential gene expression testing. The data we're using is from a publication that looked at reads or sequenced two different cell types in mice. And in order to do the tutorial more quickly, we're going to be using a subset of these data, which have been made available on Zenodo. So there's a link here to those datasets. And in the tutorial here, we've also already put in the URLs to each of these data types. And so we're going to start by copying, click the copy button here and copying all of these URLs. So you'll notice there are nine files here. The first four represent the first cell type G1E. The next four represent the second cell type Megacarya sites. And then there is a reference transcript annotation file, which we'll use in one step later in the tutorial. You'll also notice that our two different cell types contain two replicates represented by a rep one and rep two, as well as a forward and reverse file. So these are paired and sequencing files. So there's a forward read and a reverse read. So I've now copied these URLs. And back in our public Galaxy instance, we're going to start by uploading these files to our Galaxy. I'm starting with, oh, actually, first, yeah, I'm starting with a new history. And actually, the very first thing we'll want to do is to name our history. So I'm just going to call this to Novo. You can name your history, whatever you would like. And then on the left hand toolbar, I'm going to click the upload data button. And in the pop up that comes up, I'm going to click paste fetch data. And here I'm going to paste those URLs that I copied from the tutorial. I'm actually going to delete this last one. We'll upload the reference transcripts files separately. So what we have left are the eight fast queue files of the sequencing reads. So I'm going to set the type to fast queue Sanger for these eight files, and also set the reference genome to the mouse genome version mm 10. And then I will click start to start the upload. I'm going to reset and do this step again, this time keeping that ninth reference transcript file and set the file type to GTF and set the reference to mm 10. And click start to import this file. And then I'll close this box. So you'll see on the right hand side in our history, we now have these data files being uploaded. So they're indicated by the yellow. And while we wait for those to finish uploading, we'll go back to the tutorial. Now before we start analyzing these reads, we're going to first do a quality control check. And we're going to do this running a tool called fast QC. So we'll search for this tool in our toolbar by typing fast QC. And we'll see it comes up here fast QC read quality reports. So what this tool will do will give us a general assessment of the quality of the reads that we are going to be using. And so we're going to, we could run this tool one by one on all of the eight fast Q files. But if we click the multiple data sets button here, we can actually select all eight files at once. So these are the eight sequencing reads, read files. And we're going to keep everything else the default values and click execute to run these eight jobs. In the meantime, you'll see that all of the read import jobs completed, they're green. And I'll just mention briefly that these, I mentioned these were subsets of the full data files that were originally published. And they have been subsetted for one of the smaller chromosomes and mice chromosome 19, as well as some regions of interest, so that we can have some interesting results later. But if you want, you can view the data by clicking the I icon here. And you can get a sort of a snippet of what the reads look like. So as we wait for the fast QC to run, we're going to jump ahead quickly and run the second step so that it can be running in the background. And this is a step to trim off low quality sequence to bases from the ends of reads. There are a couple different tools to do this, but we're going to be using the tool traumatic. So again, we'll go to the search bar and the search bar and start typing traumatic. And we'll see it pops up here. We'll click on it. And now we're going to set some parameters. So in the tutorial, you will see if there are parameters to set for a particular tool, they'll be indicated here. So for this tool, we are going to make sure to set the tool to run for paired end sequencing, which is the type of sequencing that we have in these examples. And this will request us to select the forward or the R1 and the reverse or the R2 read. And then we'll make sure that this perform initial Illumina clip step is set to no. So again, back in the tool, we'll select paired end. And we will pick the two paired end reads that we want to analyze together. So we're going to first grab the G1E cell type replicate one forward read, which is the first item in our history. And the G1E replicate one reverse read, which is the second item in our history. And we see here also that the Illumina clip step is already set to no. So we don't have to change that. And we will click execute for the job to run in the background. Now, we want to actually repeat this tool for the other three sets of reads. And we could go through and so we're going to click the traumatic tool on the left. And again, set to paired end, grab the G1E replicate two forward reads, and the G1E replicate two reverse reads and click execute. And now we'll do this for the third pair of files. These are mega carrier site replicate one forward and mega carrier site replicate one reverse. And then the final time, mega carrier site replicate two forward and mega carrier site replicate two reverse. So while those jobs are executing, we'll now go back and look at our fast QC results in the history. So we'll just go down to the first fast QC job. And you'll see that the tool outputs two results, a web page and a raw data. And we're just going to view the web page results first. I'm going to also hide the toolbar so I can see I can have a wider view in my screen. So the first output from this tool is a table of some basic statistics about our file. So it has the file name. And importantly, what you'll see here is the sequence read length is set to 99. So these are pre trimming the read length, the length of each read is 99 bases. And the other bit we'll look at is this per base sequence quality. So this is a graph with the position in the read on the x axis from one to 99. And a quality score on the y axis with the higher quality scores highlighted are highlighted by the green. Okay, quality in this gold color and then poor quality in red. And the box plot here is showing the range of quality scores per base over all of the reads in the file. And one thing you'll note is that as we go towards the three prime ends of the read, the quality tends to drop off into getting this gold or even red area. And so these are the bases that we want to trim off by using the trimimatic tool. So we can look at another one. This is the G1E replicate reverse reads. And we'll see the same pattern with lower quality bases at the three prime ends of the reads. So we've run our trimimatic tool, you can see there's a lot of items in our history. And I'm going to do a bit of cleaning up just so we are less confused we need to bring these items to the next step by removing every history item that denotes an unpaired read because we're only going to focus on we're only going to focus on reads that are paired for the following mapping steps. So I'm going to delete unpaired and unpaired. And you can just delete these by clicking on the X in the corner. And if you make a mistake here, it's actually okay, you can always go back and retrieve these reads from your history. Unpaired, unpaired. So now we're left with eight unpaired files from our trimimatic step. And just to check that we actually have trimmed off low quality scores, I'm going to run the fast QC tool again. And this time for the sake of brevity, I'm just going to check on one of the files. So I'm going to scroll down and just pick the first trimimatic output and keep all the parameters, their default value and click execute. And this should run fairly quickly. And we're going to view the results. So in the table now, you'll see that the school, the sequence length ranges from one to 99. So in fact, we have done some trimming. And the per base sequence quality ranges are now almost entirely in the green or the high quality region. So we have successfully trimmed low quality bases from the ends of our reads. So if we go back to the tutorial, we've now done some quality assessment, trimming of the reads and checked with the QC that we have trimmed off the low quality bases. And we've done trimimatic on all of the read pairs. The next step is to take these trimmed reads and align them to a reference genome. We do this because we need to get the genomic coordinates of the aligned read so that we can then reconstruct the transcripts. Now, one important thing to note is that because we are aligning RNA reads to a genome, we are going to have to use an aligner that can enable spliced alignment because we are going to be mapping over introns that have been spliced out in the transcripts. So for this, we're going to be using the tool Hisap. There's also a box here describing how we set some parameters based on the library type. So if you remember, we've done paired end sequencing here. And we've actually done the type of experiment or the authors have done the type of experiment in the first row. So we're going to be setting Hisap parameter forward reverse indicated here. And there are some other experimental types indicated so you can use the appropriate parameters for the type of data that you were going to be aligning. So first, we are going to find the Hisap tool in our toolbar. So we want Hisap2 here. And we have quite a few parameters to set. So we're going to go through one by one. So first, we want to use a built-in reference genome. Again, these are reads from mouse cells. So we're going to be using the reference genome MM10. So the built-in genome parameter has already been selected. And we'll search here for MM10. Now, if your reference genome that you need to align to is not available in the galaxy instance that you're using, you can actually import it as its own history item and then align that by choosing the user genome from a history option here. If you remember, we're also using a paired-end sequencing library. So we'll change this to paired-end. And we want to pick the read pairs that are a result of the traumatic step. So here we want to make sure that we're choosing not our originally loaded FASTQ reads, and in fact, the ones that are a result of the trimming step. So we'll start with the G1e cell type replicate one, and we'll choose the forward reads for the file number one here. And then the reverse reads G1e, rep one, reverse reads for the file type two. Now, we also need to specify the strand information. And in that table, I noted that we are using the FR strategy. So we'll choose that here. And we also have some advanced options that we'll need to set. And these are specific because we do want to make sure that we're capturing reads that span splice sites. So we're going to change some of these options. So if we go down to advanced options, we'll click on this to expand. And then we want to specifically set some parameters for the spliced alignment options. So we'll choose specify spliced alignment options. Now, we'll leave the first item to default value, but we're going to change the penalty for non-canonical splice sites to three. So penalty for non-canonical splice sites, three. And we're going to use penalty function f of x equals b and set this to zero. And then the same here. And the reason we're doing this is because we want, we don't want to penalize mapping over spliced sites, both conical and non-canonical. So one of the main goals of doing this de novo transcriptome reconstruction is that we want to identify either novel transcripts or novel isoforms of known transcripts. So we really want to find as much as possible, not penalizing about whether a particular transcript or isoform has been identified before. So we're lowering all of these thresholds to not penalize against that as much. The last item we'll set is the transcriptome assembly reporting. So we want to set this to be report alignments for transcript assemblers using string tie, which is the assembler that we'll use in the later step. And we just double check all of the options. So built-in genome, MMM 10, paired end, FR, and then we changed all of these items. Yep. So now we will go ahead and do execute to execute this. So again, this is now going to align the forward and reverse reads from the G1E cell type, the replicate one experiment to the mouse genome, and not penalize against mapping across unknown or unknown splice sites. Now we want to do this alignment step for the other three cell types and replicates, but those were actually a lot of parameters that we set. And I want to make sure that I don't mistakenly set or forget to set some of those parameters. So I'm going to actually expand the job that's currently running that current high set job and click this run this jobs again button, these two arrows, which basically brings up the form again from the job that I already ran with all of the parameters set that I chose. And the only thing now that I'm going to change are the input files. So I'm going to pick the G1E replicate two forward reads and the G1E replicate two reverse reads. So this is the second replicate. And I'm going to keep all the other options the same. You'll even notice that under the advanced options, it has all of the items that I've changed for the splicing. I'm going to click execute. And I will do this two more times for the remaining mega care site reads replicate one and replicate two. So normally if I had run these alignment jobs on the full data set from publication, it would have taken a very long time. But luckily we've down sampled and so I'll just let these run in the background, but they actually shouldn't take more than a few minutes to finish running. So we'll just move on to the next part of the tutorial while those alignments are happening. And that is to do the actual reconstruction of transcripts from these aligned reads. And again, we're doing the Stenova, which means we are constructing the transcripts without a known reference of what transcripts are to be expected in these particular cell types or for this organism. So we're using mouse here. And again, the benefit of this is we can detect novel isoforms or novel transcripts that may be present in our cell type, but have not yet been detected in common cell types that make up the reference transcriptomes. And we're going to be using a transcript assembler called string tie here, which I'm going to go back to my Galaxy and search for string time. So we have here the string tie transcript assembly and quantification tool. And we're going to run this on each of the hysat to alignment outputs separately. So there will be four, we'll run string tie four times for each cell type in each replicate. And again, we'll need to set the strand information to the forward reverse FR. So we'll set that now. And I could run this tool individually on each of the hysat to outputs, but I'm actually going to use this function in Galaxy called batch mode. So I'm going to click this multiple data sets button. And that lets me select all four files that I want to launch a job on. And so this will launch four string tie jobs each on one of the input files. And we will keep all other options default. And we'll click execute here. So we'll see our hysat jobs did complete running. We can actually expand and see some information, see a little snippet or header of the output of running the tools if we had run it in the command line. We see that it does have a size, which is good. I'm always going to check because if this is zero, then you know that something had gone wrong. And while the string ties are running, we'll skip to, again, the next step, which is to do the assembly across those four different string tie runs. So what we're going to be doing is taking the individual output for running string tie on the four alignment outputs and combine them to produce one reference that includes all isoforms and all transcripts across the four experiments. And so we're going to use string tie merge. This is a function of the string tie program to do this. And the input is essentially the four outputs of the string ties. And in this case, we are going to provide the reference annotation for known mouse transcripts. And this is one of the input files that we initially imported into our history at the very beginning. The step isn't required for the string tie merge function. But because we do have this information, what we can do is actually annotate of all of the isoforms and transcripts we identified in our four experiments. We can annotate them as being known or novel. So that's one of the benefits of having an annotation. But again, the step is not required or this reference transcripts are not required for completing this tutorial. So we'll find string tie merge in our tool history. And we will select the four string tie outputs. So importantly, do not select the reference transcripts that we imported here, just the four string tie outputs. And then from the reference annotation to include emerging, we do want to select the reference annotation here, which is our imported history item. And we'll keep all other parameters the default and we will execute this tool. So you'll see our string tie outputs have completed. You can actually view these files if you want to just get an idea of the transcripts that have been identified. And this is a standard GTF file. So there's a chromosomal location, both the chromosome and the start and location. And these are annotated as either full transcripts or individual exons. And they have also been provided IDs. So these are the string tie unique IDs for the transcripts and an estimation of the coverage. So how many reads aligned to the reference genome? And there's a semi quantification done, although we won't be using this moving forward. We can also now see our string tie merge tool has completed. And we can also view this to get a sense of the different transcripts that have been identified, again, from the four outputs of our alignments. So we'll then run the next step, GFF compare, which is going to annotate our identified transcripts using the reference transcript file, again, to identify known and novel transcript isoforms. So we search our toolbar for GFF compare. We'll find it here. And we are going to select the output of the string tie merge. So that's just the first file here. And we are going to use a reference annotation from our history. And that's the GFF file. So we'll do reference annotation. Yes, from my history. And make sure we select that imported GTF file. And then we're going to go into use sequence data. Yes, from history. Oh, sorry, from locally cached. And we'll do the MMN 10, which has already been selected for us here. So we'll just double check these GFF inputs, output of string time merge, reference annotation, GTF, sequence data, yes. And we will click execute. And you'll see that this one tool actually produces six different output files. And we'll let this tool run. There's a bit more information about the output that GFF compare here that we're not going to go into too much detail, including what types of transcript classifications are done. So you can see, for example, a transcript that is annotated in the reference. This is for novel isoform of a known reference. And you can read more about these. So at the output of this step, this is actually a good place to maybe take a break while this tool runs, are going to be our list of transcripts that had been identified in our four experiments. So it's two cell types, two replicates per cell type, they have been aligned, the reads have been aligned to reference genome, the transcripts have been assembled, de novo without a reference. We've merged all the transcripts from the four different experiments to get a single set of identified transcripts. And then finally, we've done an optional step of comparing these to a known set of transcripts to get further annotations, although this step is optional if a reference transcript done is not available for the particular organism you're using. So at this point, we're now going to move to the last steps of the tutorial, which will be to now quantify these transcripts among the four different experiments, and then finally to do differential expression testing between the two different cell types. So we'll start with the quantification step. There are a number of tools that you can use to quantify transcripts using aligned reads and a reference of transcript features, and also multiple tools to do the differential expression testing. The two tools we are going to be using here are feature counts for counting the reads per transcripts, and then DEC2 for doing the differential testing expression. And these two tools work very nicely together and are used in other tutorials as well. So we'll start with the quantification. It's using feature counts. So in our galaxy, we will go over here to the toolbar and search for the feature counts tool. Meanwhile, you'll notice that our GF of compare tool has finished running, and you can feel free to look at the different output files. The one that we will be focusing on for the remainder of the tutorial is this annotated transcripts file, which again is our complete set of annotated transcripts and isoforms from our four experiments. So back for feature counts, we're going to be running feature counts on each of the alignment outputs, but actually, again, we can run this separately by using the multiple datasets feature here and selecting the four alignment output files. And again, we'll specify string information to forward. And then the last thing we need is to provide the annotation file to say quantify these transcripts. So we want to choose this from our history and want to make sure that we're running this on the GFF compare output. So we have multiple sets of GTF files here, but importantly, we want to use the GFF compare output. And the last item we want to update is under advanced features. And instead of quantifying on the gene IDs, we actually want to quantify on the transcript IDs. So we'll change this to transcript ID here. And I'll just double check we've done all of the parameters. So alignment files, string information, annotation file and transcript ID. And we will select execute. And you'll notice here that the feature counts has been initiated four times. And each feature counts will actually produce two output files, both a summary table, as well as the actual counts. So the quantifications on each transcript from each of the four experiments. And so those counts will be the ones that we take to the next step. So now we come to the differential expression testing part. And again, we are going to be using the DC tool, which does accept outputs of feature counts as one of its inputs. And then DC2 also performs a normalization step, which is important for removing any biases in the input files, such as sequencing depth. So in our Galaxy, we will go to the toolbar and define DC2. And we're going to set a few parameters here. Basically, we're going to tell it that we're going to give it the input files for the first set of cell types, the GE1 cells. And the second set, the mega carrier sites, because we want to compare the expression among the GE1 replicates to the mega carrier site replicates. So these factor names are basically cell types. So for the first one, we will call this Q1E. Actually, what we're going to do before I do this is change the names of these files in our history. Because as you'll see here, the feature counts outputs aren't really indicative of which cell types and replicates they are. They just use the history numbers. But we want to make sure we're selecting the correct files. So I'm going to change the name for the counts. This first one was feature counts on G1E, and we'll click save. And the next one is G1E replicate 2. Save. If you want, you can also optionally delete the summary tables if you want to clean up your history a bit. So now back to DC2. Again, cell types is our factor. We'll first look at G1E. And we want to select the feature counts for both replicates that can be done at the same time. And then I'm just going to call the mega carrier sites mega for short and select the two mega carrier site replicates. And then we'll want to make sure these two parameters are also set to yes. So visualizing the analysis results, we want to set to yes. That will provide us with some graphical outputs, which are nice to take a look at. And then output normalized count table, yes, which we will also use in a later step. So we now have all the parameters and we're going to click execute. So now that our differential expression testing is underway, we are going to take a look at the output in a couple of different ways. So first, if you remember, we chose to output visualization of the results. So we're going to take a look at the DC2 plots item in our history. And this produces a couple of useful visualizations for looking at our results. So the first here is a principal component analysis, looking at the different replicates. And we can see that along the first component on the x axis separates nicely between the G1E cell types on the right and the mega carrier site cell types on the left, which is as expected. There's also a clustering diagram here showing that the G1E cell types cluster cluster most closely with each other in terms of expression of transcripts. And similarly, the mega carrier sites also cluster together, which is great. And there are a few other plot types. So there's a dispersion estimates plot here, a histogram of p values. Again, there are quite a few significantly differentially expressed genes because we've specifically down sampled these, the input files from the original publication for regions of interest. And I'm a plot looking at the full change compared to the normalized recounts. So those are useful to look at. And also oftentimes what we'll want to do is subset all of the transcripts that we've compared for the ones that are significantly differentially expressed. And this we will use the tab, tabular output file from the EC2, which has seven columns, which are indicated here. And in particular, we will look at the seventh column, which is an adjusted p value for the difference between expression in the G1E cell type versus the mega carrier site. So that file is here, we can just take a look really quickly at what it looks like. So to filter for significantly differentially expressed transcripts, we're going to use a filtering tool. So if you go to the tool panel on the left under filter and sort, we're going to choose the filter data on any column using simple expressions tool. And we're going to select the result file. And we want the adjusted p value, which was if you remember in column seven, to be less than 0.05. So you could use whatever p value threshold that you would like here. And we will click execute. And then while that is running, so that was the first step here filter for significantly differentially expressed transcripts. We also as a second step, we'll want to look at the difference between which transcripts that are differentially expressed are either upregulated in G1E cell types or down regulated versus the mega carrier sites, which will be another run of the filtering. But first, I'm going to change the name of this file to differentially expressed transcripts, where the p value was less than 0.05. And I want to rerun this filtering step, but on my differentially expressed transcripts. And I want where the the transmits were upregulated in the G1E cell type. And this was the third column. So if you remember, the third column was the log full change. And to be upregulated in G1E, we want a positive log full change in G1E compared to mega carrier sites. So we'll run that. And then we will rerun this for genes that are down regulated in G1E, which is where the log full change would be negative a negative value or less than 0. Now, if we take a look at all of the transcripts, we can see there were just shy of 800 transcripts in our that we've actually tested the differential expression of. And of those about 800, when we run our filtering, we can see that there are 115 of these that are differentially expressed with a p value, adjusted p value less than 0.05. And you can see those listed here. Again, it looks like that adjusted p values were actually sorted from most significant to least significant. So we have the top 115 here. Under the first column under gene ID, you'll see different types of accessions. So the ones that are labeled with nm or sometimes you'll see nr. These are previously known and annotated transcripts that we've pulled from our reference GTF file. And the ones that are annotated with M string, these are the results of the string time merge that are novel or previously unknown in our reference transcript annotation file. You can also see the log to full chance here. So these negative values means that the transcript was down regulated in G1E compared to positive full changes, which are up regulated in G1E. And if we take a look at our filtered outputs, I can see that this first one, again, we'll rename this so that's more clear. These were differentially expressed transcripts that were up regulated in G1E. And there were, it looks like 57 of these. And then the second word, differentially expressed transcripts that were down in G1E. And it looks like there were 58 of these. So about 50% up and down regulated in G1E. These again are the plots that were output from the DEC2 that you can take a look at here. And finally, this leads us to the very last optional step of this tutorial, which is to do some visualization of the basically everything we've done so far. So this is looking in a genome browser, which you can use the built in galaxy browser called Traxtor, or we can do these steps. And then you can import the result files into a genome browser, such as the UCSC genome browser or IGV, which you can also run locally. And in these steps, we are going to produce some coverage files in the Bigwig format, which show the re-coverage across the regions that we've aligned to. And you can also import our annotation files, the GTF files of the merged and assembled and merged transcripts from our experiments to show those transcript models in the browser. So I'm not going to do this step live for you now. It does take some time to load the tracks. But you can see a screenshot here, which is something that you might expect to see. This is a region on chromosome 11. And this is using the Galaxy Traxtor browser. And we've loaded here in the first track is the output of the GFF compare steps. So these are the assembled and annotated transcripts from our experiments, as well as the reference track here. So you can see what was actually in our reference file. So this first transcript was a reference transcript. But this second transcript here was actually unannotated in our reference file. So this is a novel, a novel transcript. And then the remaining sets of tracks are the reads and assembled transcripts from the G1e cell type at the top and the mega karyocytes at the bottom. And these tracks are also color coded. So in blue are reads or transcripts that came from the plus strand of the genome. And in red are reads and transcripts from the minus strand. And you can see in this particular example, there is a clearly an annotated and known transcript that it was in that is expressed in the G1e cell type that is almost completely lacking from the mega karyocyte cell type, as well as this transcript in the middle, which was has multiple isoforms that have been detected in our experiments, but has actually been was not annotated in our reference transcript dome, but has quite a large number of reads aligned into it from the G1e cell types. And so you can use this visualization to actually see expression and corresponding transcripts of your experiments. So with that, that concludes this tutorial on de novo transcript dome assembling of RNA-seq data. You can see that we took our initial raw reads, which in this case, for this tutorial, we down sampled from a published data set. We did some quality control and trimming of low quality bases, which were followed, was followed by aligning or mapping these reads to a reference genome. From that, we did transcript reconstruction de novo, so without a set of reference transcripts, and assembled them into a single set of transcript of transcripts. We then did do an annotation step with known set of transcripts to annotate them, followed by a quantification of these transcripts and differential expression testing to look at the differences in their expression between the G1e cell types and the mega carrier sites. And at the different steps, we can also do some visualization to look at, for example, the results of de-seq2, or to look at our reads and transcripts in a genome browser. So I hope you found this tutorial useful. If you have any feedback to perhaps help us improve the tutorial in the future or to correct any mistakes, we would very much appreciate your feedback. And I hope you enjoyed the tutorial and hope you have a wonderful rest of your day.


Hey everybody, Dr. O here. We're going to talk a lot about DNA and RNA in the next chapter, but for this chapter we're just going to talk about the basic biochemistry here. So what is DNA and RNA made up? So real quick, DNA is going to be this genetics information storage device that's in the nucleus of your cells. And RNA is going to be how that message gets from the nucleus inside your cells to the ribosomes where proteins are actually produced. You have two meters of DNA inside of every cell of your body that has a nucleus, so some cells don't have nuclei, 3 billion base pairs, so about 6 billion bases. There are somewhere in the neighborhood of 23,500 coding genes that are found on your 46 chromosomes, 23 you got from mom, 23 you got from dad. About 1% of your cell is DNA, there's somewhere in the neighborhood of 6 picograms of DNA. That's what's amazing about DNA, there's 2 meters of it, but you only have 6 picograms worth of it. You know, if you stretched all your DNA out, it would go to the sun and back several hundred times. Not so much DNA is crammed into you, but in a very tiny package. Where it versus RNA, so RNA is going to be about 5% of your cell, it's going to be 10 to 30 picograms, so there is more RNA than there is DNA inside your cells. Any one cell can have somewhere over 300,000 different RNA messages floating around inside at the same time, thousands of different types of RNA. So both are very important, let's just go ahead and look at the structure here. So they both, what they have in common is they both are built on a sugar phosphate backbone and they both have these nitrogenous bases or nitrogen containing bases in them. So you see that there are 5 of these nitrogenous bases. You have the pyrimidines, cytosine, thymine, and uracil, and how you remember that is cut the pie, CUT, cytosine, uracil, thymine, cut the pie, Py is how you remember the pyrimidines, and you have the two purines, adenine and guanine, how you remember that, pure as gold, so purines, adenine and guanine. So these 5 nitrogenous bases, A, G, C, T, and U, are called nitrogenous bases. When you add them to the sugar phosphate backbone, they're now called nucleotides. So both DNA and RNA are made of nucleotides, they both have a sugar phosphate backbone. So the sugar that makes up DNA is called deoxyribose, which is why DNA is deoxyribonucleic acid, because it's the name of the sugar. The sugar that makes up the sugar phosphate backbone of RNA is called ribose, which is why RNA is ribonucleic acid. So those are the sugars that make them up. As far as the nucleotides themselves or the bases, DNA is made of A, C, G, and T. RNA is made of A, C, G, and U. So thymine is replaced by its cousin, uracil, when you make RNA. Last thing to note about the key differences between the two, DNA is double-stranded. You see this, this here is the double helix, looks like a rope ladder twisted on itself. So DNA is double-stranded, made of A, C, G, and T, and the sugar is deoxyribose. RNA is going to be single-stranded, made of A, C, G, and U, and the sugar in that backbone is ribose. I think that's all the important information for now. We will go into DNA replication, transcription, and translation in the next chapter. I hope this helps, have a wonderful day, be blessed.


The Indigenous People of Biafra IPOB group has asked security agencies to go after enforcers of a sit-at-home order. And the all-progressive's Congress APC has achieved more in seven years than the People's Democratic Party, and this is according to the APC national chairman, Abdulahir Dabou. This is PlusPolitics and I am Kofi Bartels. You welcome the Indigenous People of Biafra IPOB has stated that it does not have any problem with the military, particularly the Nigerian Army's killing of criminals and hoodlums still enforcing its long-canceled Monday weekly sit-at-home order. The group said it would never support criminals making the Southeast unsafe for habitation and that it would never tolerate the killing of innocent Igbo youth, according to IPOB, those who still enforce the Monday weekly sit-at-home order and not its members, but they describe them as criminals who want to rob the people and that they should be treated as criminals by security agents. So this was stated by IPOB's median publicity secretary, Imah Parfal joining us to discuss this. This is Loh Mefo, Loh Mefo is a public affairs analyst, and we'll be later joined by Chinadu Ohanda, Ritak Kahnel, the Nigerian Army. Loh Mefo, good evening to you and thank you very much for your time. Yeah, good evening my brother, good evening Nigerian. Are you surprised by the stance of the indigenous people of Biafra group on this Monday sit-at-home which has been on for some months now? Yes, I think their position is a particular one because all the while a lot of people believe that they are playing a double game that what is going on has received their tacit approval. And even when they said they weren't the ones responsible for the enforcement of the sit-at-home, a lot of people took that with a pinch of salt. Now that they are broadly asking the military to go after the criminals, I think it is only the right thing to do. And I have always known that IPOB to be peaceful, there is violence, being ascribed to the bodies is quite a surprise to so many of us, because IPOB has always been a peaceful movement. And it does appear that some elements that maybe broke away from IPOB. And those who believe that Biafra struggle must be given a push using arms, and criminals generally that saw an opening in that. These three categories of people, you know, are the ones that have been reconstituting themselves into what we call your non-agon men. I read one of the press statements issued by the spokesman of the indigenous peoples of Biafra in Mapawafo, where he broadly named some people, he named them, named their towns. And it was a very bold step, he took them, the men body, and we also have had from the lawyers of Namdee Kano, their leader, you know, their lawyers, Ijeofo and Ijeimako, have often relayed the message that Namdee Kano says nobody should sit at home, you know, because of his incarceration, and it should not be enforced also. So which means Namdee Kano does not approve of it, and they be grouped, especially the one that is from Simon Epa, who is resident in Finland, is the group that is behind this. So I really don't know how it will pan out, because everybody may be tired with the same approach. How do you expect the Nigerian military to know the difference between the various groups of Biafra agitators? And it is going to really pose a lot of a problem, and it is a delicate operation, and they have to be extremely careful, and my fear is that a lot of innocent people may fall and victims. I don't know how they will really go about this. And Professor Chukumasoludo, upon assumption of office, has also offered or extended the Olives branch to IPU be asking them to come out and let them work together to restore peace, and that should be the way to go. So I think just one more thing I want to add is that I think it is important that the branch of IPU be telling the military to go after the criminals should do more. They should work with the soludo, they should work with the military to be able to really ensure that innocent people are not victims of any oppression that they will launch in that election. Right. Interesting. So you are saying that this has been coming for some time now and it has been on for some time now, but I'm sure we can agree that whilst there have been calls by a section of the IPU be or Biafra agitators against the Mounted Seat at home, this probably is one of the few times we're hearing the group actually say the Nigerian Army should go after people. Is this surprising to you bearing in mind the history of the Nigerian Army in the Southeast, the history between the Nigerian Army and IPU be, the history between the Nigerian Army and the Biafra agitation, I mean we can go back recently to Operation Python Dance, you know, even when Kano was said to have jumped bail, the operation carried out in his country home in Abia State, where a certain person's amount of number of persons were said to have been killed. And are you shocked that the group is actually telling the Nigerian Army to go after people in the Southeast? No, I wouldn't say I am shocked. I don't think I was shocked when at first they had it. I believe that this is a problem that requires concerted efforts of all the sections of the society that need peace. And I think the IPOB is trying to demonstrate that they are not behind this killing and whoever is doing it is not doing it in their name. And I do also see that in their eagerness to prove their innocence, they went to the extent of asking the military to do the needful. Well, the military of course has not been in good terms with IPOB. But when you have a mortal enemy, you may have to weigh your options. You may actually combine with a lesser enemy to be able to deal with a more mortal enemy. I do think that the group that has constituted the unknown government, they have become a problem to everybody, including the IPOB itself. So if it would take the collaboration with the military to deal with the matter, I think they are right. I think I support their call. And that is exactly why we were shouting the time IPOB was prescribed, horribly, rather if you ask me, prescribed as a terrorist organization. And that has made it difficult for them to collaborate with the government agencies. Otherwise, it would have been easier this time around by IPOB to work directly with the military. But now there is that legal constraint that working with the military, the military will have to bear the burden of explaining why they are working with a group. They have tagged a terrorist organization. But be that as it may, a security operation of this nature, a lot can go behind the same. I recommend that they really have to work together, off record, behind the same and get the problem sorted out. All right. Is the IPOB current stance not going to go against what it says it's fighting for, what it says it's agitating for? Wouldn't the IPOB be opening the hornet's nest, as it were, telling or empowering or supporting military action in the southeastern part of Nigeria? I mean, the history is well documented of the aftermath, the casualty, when it comes to the military actions in the southeast. Is the IPOB not embarking on a journey that will ultimately lead to more casualties, more tales of wars and opening the hornet's nest, as it were? I don't envy the IPOB at this point. Because certainly there will be casualties when they occur. IPOB will certainly share the blame, not only in Nigeria, and it is going to constitute a bit of a collateral damage. But I still think it's a price, what a pain. They need to really cooperate with the state government. They need to cooperate with the governments, they need to cooperate with the security agencies to restore peace to the region. Anambra's case is the most pathetic, because until May last year, Anambra State was the safest state in Nigeria. So following the vision of these stranger characters, especially towards the build-up to Anambra governorship election, the state has no peace. Up till yesterday, the local government area was attacked and bombed. Two, three days ago, another local government area, my own local government area, which is in Emily and us, also bombed down. So I really do think that the oppression has to go on. It's very important. We need peace, whatever the cost. We don't need peace. All right. You initially mentioned Charles Solodosa. Let's bring him in at this point. What role do you think the visit to Charles Solodosa paid to Mars in Ndikano five days ago in DSS attention? What role do you think this has played in IPOB releasing this latest statement? I think what Solodosa wanted to establish was... We seem to be having a bit of a network issue with the Lord Mepho, who is a guest. He is also joining us from Abuja. We'll try and get back to him. He is Director of Public Affairs at the Igbo Leadership Development Foundation, and he's been giving us his thoughts on the development in the southeastern part of Nigeria. I'm glad to say we have a retired Colonel Chinadu Ohanda on the line. Can you hear me, sir? Yeah, I can hear you. Is this Colonel Ohanda, please? Yes, Colonel. Thank you for your time. Kofi Bartels here. What are your thoughts on the statement released by the Indigenous People of Biafra group calling with the Nigerian Army to deal decisively and to go after those who are for maintained violence in the Southeast? Also those who are trying to enforce the Monday seat at home. We've seen tales of carnage, you know, killings, attacks, destruction in different parts of the Southeast, like Imo and Nambra. What are your thoughts? Are you surprised the IPOB is actually asking the Nigerian Army to go after these guys? Well, I should tell the headquarters personnel to be, they are not the ones that are those who do the operation, that they are, if they are, would do this. Indeed, if this is a fan group of the IPOB, a POB leadership, very mentally denied it. In fact, it is not interesting, it won't be comfortable about most of it, that when you talk is half-dark as well, if you're a woman, well, yeah, there's this, this type of thing that you get. See, the IPOB, they are doing this to create a new front with them. In a situation this country step-by-step on these three kinds of guys Yeah, Kono, we're having an issue hearing you clearly. Please, if you're using EMPs or Bluetooth device, kindly detach it and maybe you can try again so that we don't have stand-interruption. Is that okay with you, sir? All right, we will take a short break, very short break, to try and get the audio issues sorted out so we can continue our very important discussion on this very important subject. We'll be back in a jiffy. Please stay with us. Bluetooth or something She's phone is using now, yeah? Hello? Okay, what about the Kono? Okay, tell me Yeah, Mr. Mef, are you there, please? Yes, I'm here. Okay, we're coming back to you, Jiffy, please. Yeah, Kono, please, are you there? What? Okay, Kono, can you hear me please? Yes, sir, I can hear you. Okay, all right, please stay with us. We have two guests, so please stay with us, we'll be coming back to you. Yeah. Welcome back, we still have our guests with us. We have Lord Mefo, who is of the Igbo Leadership Development Foundation. Mr. Mefo, are you there, please? Hello, sir. Lord Mefo, can you hear me? Can you hear me? I can hear you now. Fantastic. We also have Kono Chinadu Honda retired. Kono, can you hear me, please? I can hear you. All right, Kono, before we went off on that break, you were telling us your thoughts on this latest call by IPOB asking the Niger Military to go after those who are enforcing a Monday, sit at home, which they initially were the ones who started. So please continue with the point you were making. Kono, are you there? Okay, Lord Mefo, back to you. You've said you're not shocked. You've said that you're not surprised. But I was asking you if this call will not open the honoursness because we have been privy to the results and the casualties that I'm recorded each time the Nigerian Army backs on such an operation in the Southeast. We've had Operation Python Dance that we can refer to. Before we lost you, you were making a point, Mr. Mefo. Yes, I said that certainly it will come with some collateral damage. I know so.  What that means is that they will tell all the Biafra agitators with the same brush. They will label all of them IPOB, dissidents, ESN, and even in no set use. Many of them are likely to just go down like that. So it's a very terrible situation. But I do think that if they work closely, if they work closely with the IPOB, you know, regardless of their status in law, they should be able to pinpoint their targets more accurately. And because if they go all out, you know, in search of the enemy, you know, the way they have described them, they may not be able to really be too sure about those they have classified as such. We have seen that happen over and over again. What that means is that whoever becomes a victim, you know, is taken to be one of the unknown gunmen, which may not be true. And that is your fear. Yes, it is likely to happen that way. But we can reduce this collateral damage. Even the military will work in close collaboration with the IPOB, with the Anambra State government. It's not going to take all the problem away. But it is likely to reduce the dangers drastically to the point that may be tolerable. What role do you think the visit of Governor Charles Soludo played in this latest development with IPOB releasing the statement? Do you think his visit played a role at all? The role Soludo would play in any process. What impact do you think the visit of Soludo to Nnamdi Kano in DSS attention played in bringing out this statement from IPOB? Because of course, Anambra State, like you rightly said, since about May last year, has been the beneficiary of most of these attacks. Yes. Like, I don't know if I made that point before Network took me out. I said that his visit to Nnamdi Kano in the DSS facility was quite an important and symbolic gesture. He was suddenly done to show an olive branch and to reconfirm to himself that the leader of IPOB is still the peaceful man he has always known. You can recall that he was in Kujie prison in 2017. Some of us accompanied him on that trip to Kujie to see the Nnamdi Kano. So it's not actually the first time he was visiting him while in detention. He said about the second time. So it certainly played a role because he reconfirmed his position as a friend and as somebody who is sympathetic to their struggle. He understands them, no doubt. He understands what they are saying. Soludo has spoken about the Biafra struggle and Biafra factor in the survival of the Igbo Ethnik and even the survival of Nigeria. I have attended and listened to a number of lectures he delivered in that direction. So what he wouldn't want the struggle to be violent? He also wouldn't want the struggle to lead to this abandonment of Nigeria as an entity. He wouldn't want that. He would want Nigeria to remain an entity, to remain a country, but every section given its due, the country to be there for all of us, not only for the Igbo Ethnik group, but for all the 250 ethnic groups in Nigeria. That is what Soludo would want and he wouldn't want the Biafra struggle to become an armed struggle.  He didn't say do away with your vision or with the struggle. He said the strategy, something is wrong with the method and the method can only complicate what they are doing and put it further away to the point that what they want to achieve may be lost in the maze of uncertainty, of violence, of wrong methods that many people who are sympathetic to the struggle may even begin to draw away from it and they leave them offence and they are isolated and many more will perish in the process with that Biafra being attained and if you understand Soludo from that perspective, you can understand why he visited and what he has achieved. I think it was a wonderful move. Alright, so as far as you're concerned, that was a master's stroke. Lome, thank you very much for your time, sir. It's been quite interesting having you as a guest on Plus Politics. We wish you had more time because there are a lot of questions to ask you. For instance, who are these enforcers of the Monday seat at home? If IPOB is saying they are not behind it, there are more questions to ask. But we'll have you again soon, I believe, sooner than later. Thank you for your time. Thank you very much. Thank you. Lome Forrest, Director of Public Affairs at the Igbo Leadership Development Foundation. Thank you for staying with us. We'll take a short break now and when we return, we attempt to compare the achievements of the two major political parties in Nigeria. It's going to be explosive. Please stay with us.


What's up everyone take down here. Welcome back to another puzzle video today. We have the old loop the loop Now this one here is about 10 to 15 years old Um, a lot of people have done this before I've never seen it. I've never attempted it. I wanted to check it out So puzzle master sent me this one here If you guys want to check out other puzzles from puzzle master, for example the cast puzzles I've been doing on the channel then I'll leave my affiliate link links down below in the description Check them out. They're amazing definitely worth Checking out if you like these brain teaser type puzzles. So we're gonna open up this one here This is the only wooden puzzle that I believe I got I take the top cover off And the base just slides right out of the cardboard.  There you go Now you can see the full puzzle now the goal for this puzzle is to take the ring From one side and put it on the other ball Now, of course the ring cannot go through the wooden top part here And I don't know exactly how long this is going to take me So like always we're going to be pressing the timer here So you guys know the real time that it takes me I'm already a little stumped just because I've never done a puzzle like this before You guys thought it fell off there. Nope. It's still on the rope Hmm, this one's definitely one that's gonna make me think for a minute here And you definitely some of my other ones the cast puzzles Some of them you don't need to think outside the box. This one here You definitely need to think outside the box. So Now I know it looks like a knotted mess here I don't know it feels like it So if I follow the string it goes back down and it goes to this red one here So I need it somehow to get through This that was it. Whoa Five minutes a little less than five minutes, but I'll take that that was uh tricky How do I reset it now? Um, this is the first wooden puzzle that I got like I said this puzzle is at least 10 to 15 years old But still a lot of fun. I don't know We're gonna put this back onto the green side now Somehow I managed to put it back through and I think I'm supposed to pull on this Oh, wait a minute That was my fault So I did manage to get it from the green side over to the red side and back Getting it back did take a lot longer for some reason. So it's definitely tricky Definitely something I'd recommend if you guys want to check out puzzles like this And some others that I've done here on the channel go to the link down below for puzzle master I'm an affiliate So if you purchase anything you'll be helping out the channel because I will be receiving some money from them So I'll see you guys in the next video. Please take care. Peace