On Thursday, I was not going to be on campus this week before the midterm, so as Mushi said, I'm out here in California dealing with the intervention for our friend from Fat Faced Rick. He's somebody that used to know back in the day for slinging rocks, and then Mushi knows him from the trap circuit. So it's a mess out here. Who makes an intervention open bar?  So coming up for you guys on the docket, obviously this Thursday is the midterm exam. Again, that'll be in our main lecture hall at a regular time, and everything's posted on Piata with the practice exam and the study guide. Checkpoint two is the first check point to do tomorrow, so Tuesday, October 11th. And then the checkpoint two will be due the first Wednesday after fall break when everyone gets back. So again, post on Piata if you have any questions for any of these. So last class, we were talking about query execution. We were talking about how we can take these different operators that implement these algorithms that we talked about so far and we convert them into a high-level logical plan, which we then convert to a physical plan that specifies the exact implementation of the operator, like what join out we're going to use, what sorting out we're going to use. And so up until now, the entire semester, we've been assuming that the queries are being executed by a single worker or a single thread or process, right? There's sort of one execution thing that is processing tubals. So in today's class, we're now going to talk about how we actually execute queries in parallel using multiple workers in the same time within one single database instance, right? And this sort of seems obvious why we want to do this, right? Just make sure that it's concrete and understand why we're actually spending this time to talk about parallel execution. I'm going to go sort of three main points of why this all matters. So the first is that we obviously are going to get potentially better performance running in our database system for the same amount of hardware that we have to procure or deploy for our system.  So the way Intel gets better performance in addition to SIMD and some other, some of the acceleration stuff we talked about before is to now give you multiple cores on a single socket and for Intel, for most of their systems, most of their CPUs, the cores are all homogeneous. I mean, they have the same amount of power or computational capacity as one core versus another. The M1 stuff from Apple, they certainly have heterogeneous socket CPUs where some cores are fast, some cores are slow. That's sort of where things are going next. But in general, they're just giving you more cores. And so if we can take advantage of these cores that we provide on the CPU, then we're going to potentially get higher throughput. We can run more queries and run more transactions, complete more of them within a certain amount of time, and then we'll get lower latency, meaning the time it takes from when the query shows up in the system from the time it takes to produce a result, that'll get reduced as well. And so as part of this, you're going to get increasing responsiveness in the system. So because now when we have multiple threads or multiple workers running at the same time, when, say, one worker stalls because it has to go to disk and get something, get some page, and that's a really slow operation, another worker can keep on running and process the query, process another query that has stuff in memory. And so the system doesn't look like, it stalls all the time and this seems really sluggish. And then so related to all of this is that you're going to have what is called TCO or Total Cost of Ownership, this will get reduced for your system. Again, because now we can take full advantage of the harbor that we're given at the CPU or even the disk, then we'll need fewer machines, need fewer parts to put in these machines and maintain them, need fewer people to maintain these machines, although if you're running in the cloud, Amazon, Google, Microsoft handle that for you, but even then you pay them money to do that. And of course, all of this means that you're going to use less energy potentially because every machine has to maintain its own RAM, has to maintain its own disk, all that takes power, but if I can have my system run really, really well on a single box, then maybe I don't need to spread out across multiple machines. Now there'll be, we'll get into discussion later in the semester of like, there are some cases where you do want to have across multiple machines and this will be called a distributed database. We may want to do this for performance reasons, availability reasons, reliability reasons, but for our purposes today, we can ignore that. And so I want to make the distinction now between sort of parallel databases and distributed databases. And for the most part, I think this is my definition and so I'd like to do this sort of compartmentalize what we're talking about today and then when we talk about distributed databases near Thanksgiving, what I mean by that. And the idea is at a high level the same, like we want to be able to have multiple workers run in our database system and process queries and take care to run transactions. But for today's class, we're mostly going to focus on, or we are going to only focus on using them, having multiple workers within a single server instance, right? So for both distributed databases and parallel databases, to the application, although there may be multiple workers, it's going to appear as a single logical database. And this is sort of that logical versus physical independence that we talked about before where I could write one single query and I don't care whether the data system is a columnist or a roast or it's going to figure out the best way to execute that query and organize my data. So in the same way, we want to be able to have the application be unaware that the data is maybe split across multiple partitions, multiple servers, and that way when I scale the system up vertically or horizontally at bigger machine, more machines, I don't have to rewrite my application. So that's sort of the high level goal we're trying to do and you'd want to do this for both a parallel database and a distributed database. So the idea is that a single SQL query that we run on a single thread or a single worker database system will also work in a parallel distributed database system. So just define the distinction between the parallel distributed systems for this class. So for a parallel database system, we're going to assume that the resources, both the disk and computational resources, so CPUs and the disk, they're going to be physically close to each other and that the communication channels between them is going to be both very fast and high speed and also reliable, meaning if we send a message to another CPU core, another worker running on the same CPU socket, we're going to, for our purposes in this class, we assume that that message is not getting lost. It may show up in a different order and we use the latching to make sure that happens correctly, but it's not like we say, we pass a message from one thread to another, that message disappears. Because if that happens then you're having cataclysmic problems in your system and the whole node is going to crash and go down. So for our purposes and for the algorithms and things we'll talk about today, we assume that we can communicate between different workers reliably and very, very, very fast. In some cases, actually read the memory of another thread, another worker. So I keep doing this. I keep saying thread instead of worker. I'm trying to keep these words sort of separated. I'll explain why in a second, but just know that if I say worker, if I say thread, I really mean worker. And so in a distributed database system, which we'll cover later in the semester, that's where the resources could potentially be very far apart from each other, meaning different data center, different racks in the same data center, different data centers in the same state, country, different sizes of the globe and so forth, right? And therefore the communication between them is going to be slower than we'll assume in a parallel database, but it's also going to be very unreliable, meaning if we send a message, even if we're using TCP, there's no guarantee that our message is actually going to show up. And likewise, there's no guarantee that we'll get a response from the server. So we have to account for that in our, in our albums, in our, in how we design our system. So again, for today's class, we're assumed that we're the first one here, parallel databases, and it's just going to make our lives a lot easier. But then the same techniques that we were talking about today can get applied to the distributed database world. It's just we have to add extra safety protections to make sure that, to deal with, you know, lost messages and so forth. All right, so I'm going to begin talking about different process models you can have in your implementation of the database system. Then we'll talk about how to do query execution in parallel. And again, cover the high level concepts of how you organize a query plan to move data or shuffle data between, between different workers. And then we'll finish up talking about IO parallelism, which is an important problem. If you take the source system class, I think they cover this a bit more detail, but we'll talk about it from a database system perspective from about what can the database know is going on versus what is external to it. So the first thing that's fine is the database system process model. This is sometimes we call a process architecture in different systems, but basically it determines how the system is going to be organized to support concurrent requests from a multi-user application. And so by multi-user, what I mean is you could have multiple things or entities try to request different queries at the same time on our application. Now this could come from either within the application itself or the application taking its own input requests, like if it's a web server, taking requests from the outside and then passing this request over the database server. But for our purposes here, we don't really care. And as I said, I'm trying to be very careful in my language here, but we'll say that a worker is gonna be a computational unit or component in the database system that's gonna be responsible for executing tasks on behalf of the application, on behalf of a client, and then returning those results. And it may be the case that a single worker is responsible for getting the query request, and then it may split that query request up into different pieces in the query plan to execute in parallel and then submit those two separate workers. But the end of the day, something needs to produce a single result back to the application in response to that query. So we'll say that there's a sign sort of primary worker for the query, but it may use multiple queries, multiple workers at the same time. So there's basically three high-level approaches here of how we can organize the system. And the first two are basically very similar to each other. It's just one's gonna use processes, one's gonna use native OS threads. But then the last one is an embedded database. And this one is super common, and I just wanna cover that a little bit, but it's not, how do I say this? Whether this is not actually a process model or not, it depends on, in my opinion, this is a common use case of a database system, the embedded approach. So we're gonna cover that and understand the implications of it. So the process per worker is exactly as it sounds. So each worker in our database system is gonna be a separate OS process, meaning we're gonna call fork when the system starts up and starts a bunch of worker processes and there's gonna be some coordinator or what we'll call dispatcher and postgres they call this, the postmaster, but it's responsible for knowing here's all the processes I forked and keeps track of whether they're alive or not. So now in the application, assuming this is external to the database system, it submits a request to connect to the database system. So assume this gray box here is our database system process, at a high level it may contain multiple processes. And then for the database part here, I'm assuming it's running on the same box and the local disk, but doesn't necessarily have to be, it could be a remote disk, but we don't need it, we don't care about that right now. So the application says I wanna connect to the database system, it first hits up the dispatcher process and then the dispatcher process is responsible for picking one of the worker processes to say you're not gonna be in charge of this query or this connection. The dispatcher hands off that information back to the application who then then connects directly to the worker process and submits any query request to that who then executes that query on behalf of that connection and does whatever it needs to do on the database system. So there's a couple of things here about this approach. Why sort of I say it's less than ideal. And this is a sort of remnant of the days before the dominance of Linux and POSIX and P-threads and so forth, right? So the first is that we rely entirely on the operating system to do scheduling. We can play games with the priority flags in the OS so that maybe some one process gets a lower priority or higher priority than another but I don't think any system really does that and you really can't control. The dispatcher is not gonna control, okay. You guys, everybody stop executing because I want this process to take control. Everybody's sort of doing their own thing. And we have to use shared memory to maintain global data structures or do some kind of IPC or memory or message passion between processes in order to keep track of global state like a buffer pool, for example. So there's a bit more overhead to do that unless you're using shared memory which is what POSIX does and I think all of these other systems do. One advantage of this is that if now some process goes off the rails and crashes or if it does something wrong, divide by zero and kills itself and crashes, seg faults, you don't end up actually taking down the entire database system because that crash is isolated to just do that process. So that aspect is kind of nice but obviously if we just careful programers we would avoid divide by zero in the entire database system. So what I was saying that this is sort of a remnant of the old days, the 1980s and 1990s before Linux became what it is today because you typically see this in order systems like DB2 from the 80s, oracles from the 70s, Postgres certainly the original version of this from the 70s or sorry, the 80s. And so the reason why these order systems use the process per worker model is because it was a four P threads became sort of standardized. Like all of these different variants of Unix that was out there, BSD, Solaris or SunOS, Hbox, True64, there's all these different versions of Unix you may have never heard of and they all had their own different native thread packages. So it was a huge and had different semantics of what would happen when you spawn a thread. And so if your system needs to support a bunch of different environments you're better off using fork because that's something everybody pretty much implemented. And then you didn't worry about the threading stuff because it sort of complicated things. Because it was different one system than X. So what is more common is the thread per worker model or a multi-threaded database system instance. And that's where you have a single process with its own internal multiple worker threads. And these are typically implemented as preemptive native OS threads. And so for the most part this now means that our database system is going to manage its own scheduling because it has full control of what thread is doing what within some reason. And you may not use a dispatcher thread in the front you could have a single thread, you could have a single dispatcher thread take the request and then immediately forward it to another thread and then the outside application doesn't need to reconnect to something else or you could have do the similar approach that I said before where the application connects to the dispatcher, it gets assigned a worker thread and then the dispatcher communicates directly with that thread going forward. It's basically, you know, do you want a thread pool that any thread could potentially execute any query for any connection or do you want a dedicated thread per connection? There's pros and cons of each of these. So pretty much every database system that's been built in the last 20 years is going to be using this approach. SQL Server does this. DB2 actually supports both process model the process worker and thread workers because, you know, they have to support different these various, you know, these UNIX's and Linux versions that are UNIX versions that, you know, that maybe other open source system support mainframe stuff as well. Oracle added support for a multi-threading instances in version 12, which came out in 2014. My SQL does this. I'd say at CMU, we did hack up Postgres in 2015 to make it multi-threaded. Don't recommend it. I forget why we did that. But, you know, it can be done but there's still strictly a single process model or process for a worker model. The only systems in the last 20 years that don't do this approach is anything that is a fork of Postgres. The one system I want to talk about that's really interesting, I think, that does this approach is SQL Server and, in particular, I want to focus on how they do scheduling. So, for each query plan that shows up, and this is not just for SQL Server, this is for everyone, that is trying to do as much fine-grained scheduling as the cam workers, they're gonna have to, the systems are gonna decide like how many tasks should it use, how much should it parallelize and break the system, break the query plan up, which CPU cores it should use for the workers, and then where the CPU cores actually are. This is only if you care about NUMA, not every system does, certainly my SQL and Postgres do not, but the SQL Server definitely does. And then where should the output of the query go? Or, sorry, the operator go. We've talked about these inmate results that we'd write out to potentially a buffer as we potentially emit them up into the query plan, but it may be the case that, since we know that one, the next operator will be executed on this core over here, maybe we want to write to memory that's local to, for that core instead of our own local memory, these are some things we won't really talk about in the, for this course, we talk about in the advanced class, but I'll generalize, the data system is always gonna know way more than the, than the operating system, and so we don't want to allow the operating system to make any of these decisions if we can avoid it. And again, the commercial guys do much better at this than the open source guys. So I said the one I want to call out is on SQL Server, because I think their architecture is the, is probably the most fascinating of all of these. So in 2006, 2005, SQL Server rewrote their entire execution substrate into a new abstraction layer called SQLOS. And the idea was basically, they want to not have any part of the data system make direct sys calls to the operating system, in this case, in Windows. We don't want any part of the system make direct calls to the operating system. Instead, they're gonna make direct calls, or make calls to the SQLOS layer, and then it's responsible for then making the appropriate call down to the operating system to do certain things, right? And what they ended up did with this, as part of this rewrite using SQLOS, is that they switched the code base into a non preemptive threading model, where you basically allow the threads to yield up, yield their execution contacts back to this centralized scheduler in SQLOS that Microsoft implements in SQL Server. And then that SQLOS layer can decide who runs next, where do they run, and where's the data actually gonna be stored, or where the data is coming from. So this was a major undertaking for them. It's really, really fascinating. As far as I know, at least, there are some systems like Fauna that does a little bit of non preemptive thread scheduling, but not at this level sophistication that they do this here, right? And so you can think of like, in similar way we talked about that the buffer pool is a re-implementation of virtual memory inside our database system. You can think of SQLOS here as just like a re-implementation of almost the entire OS to do thread scheduling. And so they originally did this rewrite because they wanted to have control, where every time that they wanna have, find great control of where memory was being allocated, where memory was being written to. And they didn't wanna have to, as the hardware landscape changed over the years, they don't have to go back and rewrite, or add custom code in all different parts of the system to account for new types of hardware. If you just did it at the SQLOS layer, then everything above could take advantage of it and you'd use implement that complexity in a sort of single location. So the unforeseen advantage of this effort back in 2006, 2005, was that in 2016, 2017, when Microsoft finally added or ported SQL Server to Linux, because prior to that, it only ever worked on Windows, even though it's one of the most state-of-the-art data systems now, but if you could only run on Windows for Steve Ballmer and Bill Gates reasons, legacy reasons. But in 2016, they were able to switch over to Linux and now support Linux because of the SQLOS layer, they only had to change those syscalls that you would make to Windows to now to make to Linux, and you'd have to change anything else up above. Now there's the tooling ecosystem, like GUI's and all that kind of stuff, that's not affected by, that can't take advantage of this, and they had to port all those things over, but so the core kernel, the core engine of the system was much easier to port because they switched it to SQLOS. So I wanna talk about how they were doing the thread scheduling. So in SQLOS, they're gonna maintain an application defined, sorry, or a programmer controlled and defined four millisecond quantum, and then they're gonna have their own scheduler that's gonna say, okay, I wanna run things and task in four millisecond chunks. And again, this is not something we can enforce because we're doing non-pip and threading, meaning it's up for the implementation, the data system itself to say, okay, it's been four milliseconds, let me go back to the scheduler, let the scheduler figure things out, right? Because we can't, we don't have that kind of, we can't send the signals and do this interrupts the way the operating system can use a level for this. And so the way you implement this is you have to add explicit yield calls in the source code in roughly where there's four milliseconds or work being, have been done and then go back and yield to the SQLS schedule. Let's say we have a SQL query like this, select star from R where R dot val equals some input parameter. So as we saw last class, if you're doing an iterator model, it's roughly gonna look like, the query plans could look like something like this, right? For every tuple in R, evaluate the predicate, if it evaluates to true, then emit it. And so what, what now they have to do to do non-pip and threading is they add explicit yield calls in the source code like this. Like literally you get the right in the source code, it's not some compiler magic if they do this. And this is a gross approximation, but basically you get the current timestamp, check to see whether it's been four milliseconds since you've last checked the timestamp. If yes, then you've yield back to the, do the SQLS database and thread scheduler, otherwise do whatever the processing, do whatever processing you wanna do, right? And again, the idea here is that we have more fine gain control of how to intermix threads. There's also additional metadata that Microsoft can maintain about who's waiting on what latch and what lock and therefore when I yield, I can decide should I even try to run you again or do I know you're blocking on some lock up above? And again, the operating system can't know this because it doesn't know, it may know about mutexes and you're blocking on those, but it doesn't know about the high level locks that the database is gonna maintain, which we'll talk about after the midterm, right? So this is way better because now you have complete control of when threads run and when they don't run. And I said, the other two systems and do they do this or a font of DB? And as I said, all they really put in to do is just put a blind yield after like a disk read or before they do a disk read. And the idea there is just like, most of the time you're blocking, when you're getting something from disks because let me just yield before I go do that or do it again. The one that does do this in a more sophisticated way is Celia DB. We had them give a talk at CMU during the pandemics probably a year or two ago and they talked about how they were doing this non-patriot thread scheduling and they instrumented their system, the source code. So, you know, they had to do it at Microsoft. You have to put explicit yields, but then they would have infrastructure in place to know that if they found a portion of the code of a task took more than, you know, their five millisecond quantum time, it would then sort of send an alert to the databases and developers so that they can go look and see, okay, why did we just spend more than time we should have for this task? And then they can recalibrate and move the yields to the right location so that they have, you know, consistent chunking, right? Because otherwise without that sort of instrumentation, you're blindly putting yields and hoping it all works out. So I thought that was a really clever idea. All right, so the last process model I wanna talk about and which is again, is not really, how do I say this? I don't think the textbook mentions this as an explicit process model. I think it's as worth talking about because it's a different, it's a much different approach of how a data system would run than we've been assuming this semester. So, pretty much we've been assuming our data system runs like Postgres where it's a separate process either running on the same box or on a remote machine. Again, whether it's using a process per worker or thread per worker doesn't matter, but it's some separate process in the application. And that we're gonna connect to it using TCP or Unix Sock as whatever we want and then we can issue queries and get responses back, right? And then when our application dies or goes away, the data system is still running, right? Sort of a client server model. But another common approach is just called, what is called an embedded database system where the database system itself is gonna run in the same process or the same address space as the application. And therefore when the application makes calls to the database system, it's responsible for having whatever thread that it makes the call is for actually going executing the query or executing whatever the task it needs to do on the database system. Now, I say mostly responsible for thread and scheduling because there are some embedded systems, I think like DuckDB where it's an embedded system but it can spawn its own execution threads so you get parallelism that way and that are separate from the application threads. But for our purposes, the main purpose is like it's running in the same address space is the main thing we care about here, right? So the application could have multiple threads and each thread could just be responsible for executing some queries. So I mentioned DuckDB, the other two most famous data systems that work this way is SQLite and RockDB. When you connect to SQLite through the command line, that SQLite client is the thing that's actually the application that's actually running the system, right? You just happen to do it through their terminal interface but you can also embed SQLite in, it's better than in your web browser, right? So the web browser is responsible for the threads and it knows how to make calls in SQLite and then SQLite uses those threads to execute queries. Again, there may be additional threads that spawns for parallel execution, like I said, in DuckDB or background tasks to do garbage collection and other things, right? SQLite's probably the most famous one, the most widely deployed embedded database. It's actually the most widely deployed database in general but it's not the first embedded database. The first one probably is, I think it's BerkeleyDB from the late 80s, early 90s. Oracle bought them in 2006. Think of it like it was a embedded key value store, like a hash table, but they also had deep libraries. Embedded database is also common in systems that then build a more traditional client server architecture on top of that. So MySQL, for example, their embedded engine is called innerDB, that's their B plus tree index organized tables implementation. Then MySQL, the client server layer above it has the SQL parser and the optimizer and all that stuff. And so the innerDB doesn't fork its own threads, or spawn its own threads as far as they don't know, but MySQL up above does that and gives them to innerDB. Cocker's DB did this with RockDB. There's a bunch of systems that do this where they take an embedded system that doesn't have its own process model or maintain its own threads and they build a larger system on top of that. Okay, so I've talked about the pros and cons of multi-threaded versus multi-process architecture. And again, pretty much every system the last two decades, 15 years is using native OS threads. There's less context over switch between threads. You have more complete control over what thread one runs and where. And so if you're building a system from scratch today, the multi-threaded approach is the way to go. Now, just because your data system supports multiple threads or multi-threaded workers or start using multiple workers or multiple threads on multiple threads, it doesn't necessarily mean you support intro query parallelism, which we'll talk about next. Meaning just because you have multiple threads running, does it mean a query can show up and that data system will be able to break it up into multiple threads or execute it in parallel multiple threads. So MySQL is the best example of this. It's a multi-threaded data system architecture, but one query shows up and it'll be only executed by one thread, right? Because that's the model that they pursue. All right, so let's talk about the different types of parallelism we have at sort of query level. So I already mentioned intro query parallelism, this is where we take a single query shows up and we can break up the operations to multiple workers and run those in parallel. And this is super common in a lab systems where like, you know, when I want to scan a large table, I can break up the scan into separate workers and have them run in parallel at the same time. Intro query parallelism is the most common one. This is what pretty much everyone does. This is where the system itself can support multiple simultaneous active queries running on separate workers at the same time. And obviously, as always, we want to do this because we have increased throughput and latency. So I'll briefly talk about intro query parallelism, so we'll spend more time talking about it after the midterm because we need to worry about correctness of two workers for two separate queries, update the same thing at the same time, how do we deal with that? But we'll spend most of today talking about intro query parallelism. So in a query parallelism, as I said, it's multiple queries running at the same time, like two different requests show up and we want to run them at the same time. So if everything's read only, then this is really easy to do because they're not going to change the state of the database itself. They may change the state of internal data structures like our buffer pool, like you update the page table and so forth or any internal statistics that we maintain. But the underlying core database itself is for the most part is not going to get modified. So this is pretty easy to do. So we do use all the lashing stuff to protect the data structures, but we don't have to do any higher level locks to protect the database itself. But then if they update the database at the same time, then this is where things get really hard, equally as hard as maintaining the parallelism in the data structures. But as I said, we're not going to cover this right now. We'll cover this in lecture 15 after we get back from the midterm. Again, so there's not really much to say here that like I can have, as I showed before, application issues, different application instances issue different connection requests that dispatcher could assign them to different workers and whether it's a thread per worker or process per worker doesn't matter and those queries are running parallel and they have to contend for resources on the buffer pool and other parts of the system and we sort of make sure that they're doing this in a safe manner that doesn't corrupt our data structures. So intro carry parallelism is what people mostly will think about when they say parallel execution in a database system. And the idea is that for a single query invocation, we're going to execute its operations or operators in parallel at the same time. And so this is going to be similar to, you just think about the processing models we talked about last class, iterator model, vectorized model and then the materialized model where it's sort of this PubSub or produce a consumer model where we could have different operators are producing results to different parts of the query plan and there could either be a worker, multiple workers for each of those query operators running in parallel or the different operators could be assigned to different workers and they're feeding to each other. But the point I'm trying to make is that the produce a consumer model makes this all easy to do with the reason but to expand the or extend the query plans into do support parallel execution because we have that decoupling and a well-defined API of what one operator sends to another operator. Got a high level we're just saying it's emitting tuples and obviously when I talk about SQLOS you care about the location of where the results are being sent but for our purposes here we can ignore that. So the for all the query sort of the operator algorithms that we talked about so far there's gonna be parallel versions of all of these. And sometimes they're like completely parallel because I can run the scan in parallel on a table and I don't need to pass information from one scan invocation to another, right? Other times you do care about the results of the different operators like when you care about knowing what the global state of something like you're trying to find distinct keys maybe you need to know what are the distinct keys coming out from different operators at the same time so that you can coalesce them into a single distinct list or distinct set. But and we'll get to that in a second but like the basic thing here is just to point out is that the output of the operators can be done in parallel and they can either write to unique buffers or unique locations or they get right to a single data structure. We'll see how to do this in both cases but at the end of the day the result of the query itself needs to be a single result that's sent back to the client. As I said we write SQL, we don't know, we don't care necessarily that the query is gonna run in parallel. We just wanna make sure we get a result back in the single set or a single relation. We don't wanna get, we don't wanna be told here's the results, here's half the results but here's the location find the other results. Nobody would write their application code like that. So the data doesn't have to produce back a single call less result. So it's easy to conceptualize how to do parallel versions of all the algorithms we talked about so far. So I just wanna show sort of a high level how you would do this for the parallel hash join. So the, we talked about before that you would have this first phase where the build side and probe side they're both gonna scan through the result and hash it and produce the right, write the tuples into buckets based on the hash key. But then now an easy way to parallelize this would be for each level of buckets, I could just have another worker be responsible for taking the build side and probe side and either doing the nested loop join if it's in memory or build the hash table on the build side and then probe on the right side. So each worker could be doing this for distinct level of buckets in parallel. The alternatively, you could be when you hash everything together, if everything fits in memory, you sort of do the scan in parallel on the build side, build a single global hash table and then on the right side, you can then probe that in parallel with different workers. And because you're not modifying the state of the hash table on the probe side, you can run that in parallel without any latching and that'll be fast, right? So again, the tricky thing is the coordination between the different workers, but it's not hard to see how you could run these in parallel and the decision is whether you want to produce a single output result or split them up and then maybe coalesce them later. All right, so for intro query parallelism, there's gonna be three types. Well, in my opinion, there's two types. There's intro operator parallelism, horizontal parallelism and then intro operator parallelism, vertical parallelism. I think the textbook refers to a third type as bushy parallelism. And in my opinion, it's just a combination of the two. So I don't really see that as a distinct type of intro query parallelism, but we'll just, we'll go through and make sure we understand what's actually going on. And again, the advantage of SQL is because it's declarative, we don't have to write anything in our SQL statement to make it use any of these things, right? The data system response we're figuring out, okay, these are the resources I have, my data looks like this, my query looks like this. Here's the best, here's the optimal way to configure the query plan to use these different types of parallelism, right? And the spoiler I would say also too is that the first approach, the horizontal parallelism, this is probably, this is the most common one you see in parallel data systems. All right, so in the first one, the idea is that we're gonna decompose an operator in our query plan into what I call independent fragments or query plan fragments that are gonna perform the same function, the same operation or operator as the other fragments within the same operator. But they're gonna do it on a disjoint subsets of the data or whatever the data coming up from below it. Now it could be a table scan, so we're breaking the table up into shards or partitions and every query fragment's gonna operate on just its shard or it could be coming up from an operator below us and we can split that up and everybody operates separately. So to make this all work now, we're gonna introduce a new sort of synthetic operator in a query plan called the exchange operator that we're gonna use to coalesce and split results from either multiple children together and produce a single result or take a single result and split it up to multiple parents up above us. So again, this is not something that exists in relation algebra. This is something we're gonna introduce that Dave Simms is gonna introduce in the physical query plan to allow us to gain a reason about what our degree of parallelism and where the data is actually coming in and where it's going out. These are different steps, all right? And as the Postgres is gonna call this gather, they're gonna do the first type of interoper parallelism, intraoperated parallelism with something like SQL Server, a more sophisticated system is gonna have a support basically all three types. So let's see, I really see most of this. So say, I have a table with a billion tuples, a ton of tuples, and I wanna do select star from A where A dot val is greater than 99. And assume I don't have an index here. So the query plan is pretty simple, right? At a logical level, it's scan A due to the filter on value. And so what I'm gonna do is for all the different pages I have in my table, I'm gonna divide the query plan up for the scan operator with the filter into a fragment, right?  So the exchange operator is like a barrier that says, I can't proceed, I can't send anything up above the next step in the query plan, next level in the query plan tree until I get all the results from my children operators below me, right? And it's responsible for saying, okay, I have three workers and I have three plan fragments, so let me assign the plan fragments to these different workers and let them run at the same time. So now when I call next on the exchange, assuming we're doing iterator model with top-down execution, I call next on exchange that can call next on this one plan fragment on the filter, then calls next on the scan on A, and it's responsible for going and fetching the first page. And again, there's some bookkeeping that's keeping track of what page this plan fragment is responsible for reading. But unlike in the iterator model where it was a sort of blocking call as I go down call next on my children, the exchange operator has a lot of call next on the other children operators at the parallel at the same time. So now my three plan fragments are running in parallel, all processing different pages of A here. All right, so let's say that the first two plan fragments they finish, and we start emitting tuples of exchange, now exchange to say, well, I know I should have scanned the two other pages in table A, so it can call next on A1 and A2 again to have them start scanning four and five while the third query plan fragment finishes up scanning page three. All right, so again, the idea is that we're dividing the scan on a large table across multiple fragments that are each run in parallel so that we can sort of reduce the time potentially of the query execution of doing the scan because we're not blocking, waiting, work is not blocking, well, the worker does have to block and wait when it needs a page, but some other worker could have already the page it needs is in memory and it can process that and make forward progress. All right, so this type exchange is the most common one. It's called the gather operation. I think in SQL server, they may call this the parallelism operator. Postgres calls this gather. The original paper from the 1990s that described this approach, it's actually the volcano paper that we talked about many times before. They call this exchange and that's their term, but this actually distinctions between these different exchange operator types. This actually comes from SQL server because they as an enterprise system they support these different types. So this is the most common one gather. We're basically taking the results from multiple workers and you're gonna coalesce the results and produce a single output stream to whoever needs it up above. So going back to my example here, since I need to produce a single output result to the query or sorry, to the application for the query, the exchange operator is gonna combine them and then shove them up. So this is what gather does. The next one is to do distribute and this is we're gonna take a single input stream from our child operator and then we're gonna break it up according to some partitioning scheme. Round robin hash partitioning range partitioning we'll cover these later, but you basically take the single input stream and then you break it up to disjoint subsets and you send that out to different consumers up above. And then repartition is the combination of the first one and second one where you take multiple input streams, coalesce them and then produce multiple output streams. And the key thing to point out here in repartition is that the number inputs into our exchange operator does not need to be the same as the number outputs. I could have a lot of parallelism in my input, but maybe I only wanna have two workers process the outputs and let's put it up to divide it up to two workers. So again, as I said, the first one is the most common one, the second two are again, typically seen in the enterprise system and any system that's really focused on OLAP performance we'll do these other two. So let's say now we wanna do our scan on A. We had three workers that are doing this for this fragment here. We're also gonna push down the filter before they join for obvious reasons that we talked before because we don't wanna do, we don't wanna start doing hash table probes or building hash table for tuples that were never gonna match anyway. So we wanna get the filter to be close to the scan as possible. We're also gonna do an optimization where we push down the projection because we would know that these are the only tuples we're gonna need at some later point in the query plan. So rather than us copying the whole tuple, assuming we're doing early materialization, we just copy the subset of the data that we actually need. And then we're gonna build our hash table here. So we'll have exchange operator above because we want to coalesce the hash tables together into, or assume we're building a single giant hash table for these three plan fragments. So we don't wanna start doing the probe until we know our hash table is fully populated. So then this feeds into the build side of our hash join. So now we do the scan on B, same thing. We're gonna do the projection push down, the filter push down and the projection push down. And then now we're gonna probe the same hash table that was built by the other side.  And then now the output of these different threads doing the join, they're gonna write their own individual output buffers. So we can put another exchange operator up above that to then coalesce the results into a single output, right? And again, this exchange is just internal metadata that just denotes in the query plan that it can't do anything up above until it knows the task below it and completes. And where the metadata is is say, like well, what determines the amount of parallelism you're gonna use when you execute the query that's typically done before you start executing the query. Like the data system is not gonna say, oh, well, I could use more threads while it's running and start giving you more threads. Some systems do that, Postgres does not, SQL Server does not. Typically there's like, you define the amount of parallelism you want when the query plan is actually generated. Some systems like Snowflake can recognize it, oh, I'm running slower than I should and they actually can pull in, they can borrow workers and not only from your cluster they have this sort of global pool of available workers and they can pull in workers to speed things up for you. I think if things are going slower than they had anticipated, which is kind of cool. All right, so interoperable parallelism, this is vertical parallelism and as I said, this is not as common in what I would call traditional data warehouse OLAP systems. You typically see this in what are called streaming systems or continuous query systems, things like Spark Streaming, Apache Flink, Pulsar, KSQLDB, which runs on Kafka. And the idea here is that you're gonna have a sign a worker to an operator and one worker per operator and so that operator can just sort of run all the time on any tool that arrives and when it produces an output, it admits that to a next operator, there's some other worker that can pick that up and process that, because there's always gonna be some other work that each worker can have to keep doing. The reason why this is common in streaming systems because the idea is that I'm connecting to some event source or some source of data that's always sending me new data and so rather than run the query in a batch mode where I run, produce, resolve and then new query shows up and on a second let I run it again, these things are just running all the time. And this is also called pipeline parallelism because again, within a pipeline, you have multiple workers running parallel at the same time. All right, so let's say in a real simple example like this, we have this join operator here and let's ignore what we talked about before what we paralyzed that with multiple workers doing the scan, assume that there's just one worker that's responsible for taking any, any tuple gets from the inner table if you build the hash join or hash table and just keeps emitting it, right? So this thing is always running on one worker and it's emitting tuples like this. And then the up, up, up, up, it's gonna be another worker to do the filter and then it just sits and listens on some incoming stream and says any tuple that shows up apply my projection and then spit it up, right? And so the idea is that this incoming queue or input stream is, or socket is there's no new result and this worker is just idle and it waits, right? And again, the idea of a continuous query is that this thing just say you're trying to say, you know, do an alert on the stock price. So you want this thing running all the time as new ticks, new stock ticks come in. So each worker can be doing a piece of the query plan rule always running parallel. All right, the last one is the bushy parallelism that I talked about before, again, my opinion is just a combination of both intra and intra operator parallelism where you have different workers, excuse you, different multi-operators and different parts of the query at the same time. You still need exchange operators because you have to combine things and put things together but at a high level it works the same way. So now we have say a three-way join or sorry, four-way join between table A, table B, table C and D. So with bushy parallelism, I could break it up so that like the join in A and B is join in one worker, the join in C and D is join in one worker and then these are producing outputs that are then running to do the join on the intermediate results between all of them. Everything we talked about so far is about getting parallelism for across CPU cores. The problem there is that it may be the case that the CPU isn't the bottleneck, right? It's actually the disk. So if I have a CPU socket that has 1,000 cores and it's great that I can run using the different types of parallelism we talked about, I can run all my workers on different cores at the same time but if they're all in the day going to the same disk in that super slow, then all my additional parallelism is for not, right? It's a waste. It's actually I could get potentially worse performance because now everything is just, the system disk is just thrashing, bringing things in and out of memory nonstop, right? So what we also potentially need in our database system, actually what we do need in our database system is IO parallelism, right? So that we can split the database itself, the things that we're actually storing across multiple devices, multiple non-volta devices, multiple SSDs or running in the cloud, multiple S3 buckets because this is gonna allow us to improve our disk bandwidth quite a bit. And there's a bunch of different ways that we can do this. Like we can have one database could be comprised stored across multiple disks. One, you could have one database per disk and you have multiple disks and one per database. You could have assigned one table, one relation per disk. You split a relation across multiple disks, right? All the different approaches have different trade-offs and not only in terms of both performance but also reliability or durability. Like if one disk goes down, do I lose everything or do I lose part of it, right? And so we're not gonna get too much into that now. I just talk about from a database system perspective what we need to actually think about for this. And we'll talk a little bit more about this when we're talking distributed databases. So I will say that some systems will support this notion of IO parallelism natively inside of it, meaning the data system is aware that it has multiple disk devices that are unique and that it can assign tables or assign relations or sign indexes to different disks and it knows now in the disk manager or the dispatcher when you go fetch a page where you're actually going and getting it and then potentially also do back up some recoveries according to the topology of your disk layout. Other systems, this has no notion of different devices, right? It's just using the file system, the operating system's file system API to say, create me a file, create me directories and it doesn't know where, you know, what the actual physical storage is for that. Again, so PostgreSQL might see it work this way. And so it's up for you now in the, as like an administrator on the operating system level to then move directories to different disk devices or do syn links and so forth, right? And so that you'll get the benefit of the parallelism in the data system because the disk got potentially faster but the data system scheduler can't take, it's not aware that these are actually different storage devices, it can make different decisions based accordingly. So this is that a person I'm talking about here. This transparent parallelism we would have in data system, do you do I parallelism? And this is, again, this is probably the most common one. This is what most people do to get started. The enterprise things can do more complicated things. The basic idea is that we would have at the hardware or the OS level, this notion of multiple storage devices. So from the data systems perspective, I got a bunch of pages on disk or in my file, but I don't know where this is actually being stored. So this is what you would use something like RAID 4. So RAID 0 would be a striping where thing that was around Robin for every page that shows up, you assign it to one of the disk devices that you've set up in your RAID array. Storage appliances basically are doing the same thing. So striping is one way to get better performance because now within one logical disk view from the data system, it can run requests in parallel and there's multiple devices dispatching and executing these things. Of course, now the problem is in this case, if you crash and go down or one of these disk crashes, disk crash, you lose all the pages there. And so there's also mirroring in RAID where every page is stored on multiple devices in duplicate. So this makes reads go faster because now you can potentially have different queries read different pages from different disks. It makes the data more reliable because if one disk dies, you have backups on the other disks and then again, this is all transparent to the data system. Most systems are going to use some combination of RAID 0 plus 1. So you get the benefit of the striping because that's better for writes because now all the writes don't have to go to all multiple disks, but then you get the benefit of mirroring so that one disk dies and you don't lose everything. And again, from our purposes, the data system doesn't know anything about this. The way it isn't, doesn't need to natively support IO parallelism. The way you would do this is through some amount of database partitioning. And the idea here is that we are taking a some level of division in our database to then split the database up to destroying chunks and assign them to different storage devices. So you can get, as I said, you can do it on a per database level, per table level. Within a table, you could either do this at sort of horizontal partitioning or vertical partitioning. Again, it's up to the data systems to decide what's the best way to do this and get better performance. So if in some cases, if you're storing everything one database per file, one database per directory, this is really easy to do and you can just move directories around. But again, this has pros and cons. The one that is super common, especially in distributed databases, that we're gonna hold off on talking about until later is the lower level table level partitioning or tuple level partitioning where we can split a single logical tuple into disjoint subsets or segments that are then treated separately in the system. And then the query planner knows how to take the request of the SQL query and route it to the right location of where the data is located. And ideally, we want this partitioning to be transparent to the application because we always want it to appear as a single logical table, even though physically it's stored in different ways. But again, sometimes this information bleeds out into the application. Now we can talk about the implications of that later. So I don't wanna go too much detail of this later. Detail of this right now, we'll cover this when we talk about distributed databases because we don't really need to know this particular, we don't need to know this technique to get through query planning and transactions, but we need to know about this for scaling onto multiple machines. So again, we'll cover that later in the semester. All right, so that's everything I wanted to cover for parallel execution. As I said, the multi-threaded one worker, so one thread per worker is the most common approach. And then enter query parallelism, meaning running multiple queries at the same time is also in this common approach. The multi-threaded execution of a query, so intra query parallelism, that's more common in the OLAP systems because you wanna scan a lot of data and do things in parallel. So this is, again, easy to think about but hard to get right because there's a lot of things now, a lot more moving parts in the system and you have to worry about things that we don't have to worry about before when just, okay, one query shows up, we just run it on a single thread because now we gotta worry about what fragment's gonna run at what time, where do they send data to, who runs first versus others, if we're doing writes or we have to worry about updating the data at the same time, the same parts of the same data at the same time. We've already talked about how to do latch and protect the data structures. And of course now there's contention on the resources, everyone's trying to do, access the disk at the same time. I mean, how do you make that and not be a major bottleneck? And again, this is why we don't want the operating system to do anything because the data system always knows what's going on and it can always make a better decision for all these things, you know? All right, so coming up this Thursday is the interim exam for you guys. And as I said, it's gonna cover everything that we discussed in class up to including, up to including the lecture from last Thursday. So this lecture here on parallel execution is not covered, but everything prior to that, so query execution one, that is covered. It'll be the hour and 20 minutes that we normally have for lecture time and it'll be in the location where we're gonna have lectures. And then that link here will take you to the midterm study guide, so please go look at that. And there's practice problems in the textbook you can look at if you want additional more feedback, okay? All right guys, I gotta go. I gotta deal with Fabius Rick and this intervention that's not going well. And I'm flying back tonight and I'll see everyone at the exam on Thursday. See ya. He is you with me, what I'm speaking. Just grab a double duper two and then I'm good to go.


Nuh-nuh-nuh-nuh, nuh-nuh-nuh-nuh, nuh-nuh-nuh-nuh-nuh-nuh. All right, so coming soon, speaking of Arduino-shaped things. We've got a Metro ESP32 S3 with bucket load of flash and RAM. We're going to be using this for some of our own internal development, but I thought I would toss it in. It says S2 on the back, but it's because I just haven't updated the silk. This is a prototype, but if people are interested, you can sign up and be notified when we do the final silk screen, final revision for this board, still finishing it up. But I do like to have a Metro for every chipset that we use, particularly because it has a JTAG debug and all the GPIO exposed and it's just easier for people to do development on it. Like this would have been probably really handy for our team wearing circuit Python code for the S3, but it's never too late. Good enough. Okay, next up. Next up, we have a new leap motion. This is the leap to the second generation of it. This is a computer controlled motion detection, like hand detection user interface experience device. It's kind of explained like what is it? It's not a camera, but it's something that can detect your hand and where it is and how it's moving and then you can use the API and code that they provide. It's designed for people to interface with. It's not like a ready to go product just for using with, you know, an audio player or a video editing software. You can use it for whatever you like. The previous version was discontinued. We sold lots of them, very handy for hackers who want to do hardware interfaces using hands and motion. And we do sell some gesture sensors, but they're not like honestly that great. This is really designed specifically for gesture decoding and interfacing. To check it out, the previous version has been great. We just got these in and I'm going to definitely pick one up and try one out because it's like, if you're doing art interaction, you want to check when people are there or not and the prices, you know, compared to AI camera systems as much as possible. These are super fun. All right. And Starship, besides ULA data, our team, our customers, our community is. This is the new Pi CSI DSI adapter, which is why the code is CSI DSI. And it's basically a very simple thing.  This is the adapter thingy. We have an extender thingy. This is the adapter thingy. It'll take the 15 pin, one millimeter pitch connector and convert it to 22 pin, 0.5 millimeter pitch. That's it. It's only a couple of blocks. Solves a problem for you. We have an adapter cable, but it's like a fixed length. This lets you use any length cables you like.


Hello and welcome to theCUBE Studios in Palo Alto, California for another CUBE Conversation, where we go in depth with thought leaders driving innovation across the tech industry. I'm your host, Peter Burris. What does every CIO want to do? They want to support the business as it evolves and transforms using data as that catalyst for better customer experience, improved operations, and more profitable options. But to do that, we have to come up with a way of improving the underlying infrastructure that makes all of this possible. We can't have a situation where we introduce more complex applications in response to richer business needs and have that translated into non-scalable underlying technology. CIOs and 2020 and beyond have to increasingly push their suppliers to make things simpler. And that's true in all domains, but perhaps especially storage where the explosion of data is driving so many of these changes. So what does it mean to say that storage can be made more simple? Well, to have that conversation, we're going to be speaking with Eric Herzog, CMO of VP of Global Channels at IBM Storage, about, quite frankly, an announcement that IBM is doing to specifically address that question, making storage simpler. Eric, thanks very much for coming back to theCUBE. Great, thank you. We'd love to be here. All right, I know you got an announcement to talk about, but give us the update. What's going on with IBM Storage? Well, I think the big thing is, clients have told us, storage is too complex. We have a multitude of different platforms, an entry product, a mid-range product, a high-end product, then we have to traverse to the cloud. Why can't we get a simple, easy to use, but very robust feature set? So at IBM Storage with this flash system announcement, we have a family that traverses entry, mid-range, enterprise, and automatically can go out to a hybrid multi-cloud environment, all driven across a common platform, common API, common software, our award-winning spectrum virtualized, and innovative technologies around whether it be cyber resiliency, performance, incredible performance, ease of use, easier and easier to use. For example, we can do AI-based automated tiering from one flash array to another or from storage class memory to flash. Innovation, at the same time, deriving better value out of the storage, but not charging a lot of extra money for these features. In fact, our flash system's announcement, the platforms, depending on the configuration, can be as much as 50% lower than our previous generation. And that's delivering value, but at the same time, we added enhanced features. For example, the capability of even better container support than we already had in our older platform, or our new flash core modules that can deliver performance in a cluster of up to 17.2 million IOPS up from our previous performance of 15. Yet, as I said before, delivering that enterprise value and those enterprise data services, in this case, like I said, depending on the config, up to as much as 50% less expensive than some of our previous generation products. So let me unpack that a little bit. So historically, when you look at, or even today, when you look at how storage product lines are set up, they're typically set up for one footprint for the low end, one or more footprints in the mid-range, and then one or more footprints at the high end. And those are differentiated by the characteristics of the technologies being employed, the function and services that are being offered, and the prices and financial arrangements that are part of it. Are you talking about essentially a common product line that is differentiated only by the configuration needs of the volume and workloads? Exactly. The flash system traverses entry, mid-range enterprise, and can automatically get you out to a hybrid multi-cloud environment. Same APIs, same software, same management infrastructure, our storage insights product, which is a cloud-based storage management predictive analytics works on the entry product at no charge, mid-range product at no charge, the enterprise product at no charge, and we've even added in that solution support for non-IBM platforms, again. So delivering more value across a standard platform with a common API, a common software. Remember, today's storage is growing exponentially. Are the enterprise customers getting exponentially more storage admins? No. In fact, many of the big enterprises after the downturn of 08, 09 had to cut back on storage resources. They haven't hired back to how many storage resources they had in 2007 or eight. They've gotten back to full IT, but a lot of those guys are DevOps, people, or other functions. So the storage admins and the IT infrastructure admins have to manage extra petabytes, extra exabytes depending on the type of company. So one platform that can do that and traverse after the cloud automatically gives that innovation and that value. In fact, two of our competitors, just as an example to do the same thing, have four platforms. Two other have three. We can do it with one. Simple platform, common API, common storage management, common interface, incredible performance, cyber resiliency, but all built in something that's a common data management infrastructure with common data software, yet continuing to innovate as we've done with this release of the Flash System family. Okay, so talk about the things that, so your common API, common software, also I presume common, that core module, that Flash core module that you have, common across the family as well? Almost all the family, at the very entry space, we still do use standard SSDs. But we can get as low as a street price for an all Flash config of $16,000 for an all Flash array. Two, three years ago, that would have been unheard of. And by the way, it has six nines of availability, same software interface and API as a system that could go up to millions of dollars at the way high end, right? And anything in between. So, common ease of use, common management, simple to manage, simple to deploy, simple to use, but not simple in the value proposition. Reduce the TCO, reduce the ROI, reduce the operational manpower. They're overtaxed as it is. So by making this across the portfolio with the Flash System and go out to the hybrid multicloud, but bringing in all this high technology such as our Flash core modules, and as I said, at a reduced price to the previous generation, what more could you ask for? Okay, so you've got some promises that you made in 2019 that you're also actually realizing. One of my favorite ones, something I think is pretty important in storage class memory, talk about how some of those 2019 promises are being realized in this announcement. So what we did is we announced our first Flash system family in 2018 using our new NVMe Flash core modules. We had an older Flash system family several years ago that used the standard SAS interface. But our first NVMe product was announced in the summer of 2018. At that time, we said all the way back then that in early 20, we would be start shipping storage class memory. Now by the way, those Flash systems NVMe products that we announced back then actually can still use storage class memory. So we're protecting the investment of our installed base. Again, innovation with value on the installed base. A very IBM thing to do. Yes, we want to take care of the installed base. We also want to have new modern technologies like storage class memory, like improved performance and capacity in our Flash core modules where we take off the shelf flash and create our own modules, seven year media warranty, up to 17.2 million IOPS, 17 mics of latency, which is 30% better than our next nearest competitor. By the way, we can create a 17 million IOPS config in only eight RACU. One of our competitors gets close, 15 million, but it takes them 40 RACU. Again, operational manpower, 40 RACU is harder to manage. Simplicity of deployment. It's harder to just deploy all that in 40 RACU. We can do it in eight. And pricing. Yes, and we've even brought out now a pre-configured RAC. So what we call the Flash System 9200 are built into the RAC with a switching infrastructure, with the storage you need. IBM services will deploy it for you. That's part of the deal. And you can create big solutions that can scale dramatically. Now our stands for hybrid? RAC. RAC. Well talk to me about some of the hybrid packaging that you're bringing out for hybrid cloud. Sure. So from a hybrid cloud perspective, our spectrum virtualized software, which sits on-prem, entry mid-range, and at the upper end, can traverse to a cloud instantiation called spectrum virtualized for cloud. Now, one of the key things of spectrum virtualized, both on-prem and our cloud version, is it supports not only IBM arrays, but through a storage virtualization technology, over 450 arrays from multi-vendors, and in short, our competition. So we could take our arrays and automatically go out to the cloud. We can do a lot of things. Cloud air gapping to help with malware and ransomware protection, DR, snapshots and replicas. Not only can the Flash System, new Flash System family do that to spectrum virtualized on-prem and then out, but spectrum virtualized, coming on our Flash System portfolio, can actually virtualize non-IBM arrays and give them the same enterprise functionality, and in this case, hybrid cloud technology, not only for us, but for our competitors' products as well. One user interface. Now, talk about simple. Our own products, again, one family, entry, mid-range, and enterprise, traversing the cloud. Oh, and by the way, for those of you that are heterogeneous, we can deliver those enterprise-class servers, including going out to a hybrid multi-cloud configuration. For our competitors' products as well, one user interface, one throat to choke, one support infrastructure with our storage insights platform. So it's a great way to make things easier, cut the cat-backs and op-ex, but not cut the innovation. We believe in value and innovation, but in an easy-deploy methodology so that you're not overly complex, and that is killing people the complexity of their solutions. All right, so there's a couple things about cloud as we move forward. They're going to be especially interesting. One of them is going to be containers. Everybody's talking about, and IBM's been taught, you've been talking about, we've talked about this a number of times about how containers and storage and data are going to come together. How do you see this announcement supporting those emerging and evolving needs for container-based applications in the enterprise? So first of all, it's often tied to hybrid multi-cloudness. Many of the hybrid cloud configurations are configured on a container-based environment. We support Red Hat OpenShift. We support Kubernetes environments. We can provide on these systems at no charge persistent storage for those configurations. We also, although it does require a backup package, Spectrum Protect, the capability of backing up that persistent storage in an OpenShift or a Kubernetes environment. So it's critical. Part of our era of simplicity is this flash system platform with this technology can support bare-metal workloads, virtualized workloads, VMware, Hyper-V, KVM, OVM, and now container workloads. And we do see for the next coming years, think about bare-metal. Bare-metal is as old as I am, right? That's pretty old. Well, we got tons of customers, still got bare-metal applications, but everyone's also gone virtualized. So it's not, are we going to have one? It's you're going to have all three. So with the flash systems family and what we have with Spectrum virtualized software, what we have with our container support, we can give you bare-metal support, incredible performance, whatever you need, VMware integration, Hyper-V integration, everything you need for a virtualized environment and for a container environment, we have everything too. And we do think the, especially the mid-to-big accounts are going to probably run all three at least for the next couple of years. This gives you a platform that can do that at the entry point, up to the high end, and then out to a hybrid, multi-cloud environment. With that common set of software and APIs across. Now, every year that you and I have talked, you've been especially passionate about the need for turning the crank and evolving and improving the nature of automation, which is another one of the absolute necessities as we start thinking about cloud. How is this announcement helping to take that next step, turn the crank and automation? So a couple of things. One is our support now for Ansible. So offering that Ansible port integrates into the container management frameworks. Second thing is we have a ton of AI-specific based technology built into the flash system platform. First is our cloud-based storage management predictive analytics package, Storage Insights. The base version comes for free across that whole portfolio, whether it be entry, mid-range, or high-end across the whole flash system's family. Gives you predictive analytics. If you really do have a support problem, it eases the support issues. For example, instead of me saying, Peter, send me those log files, guess what? We can see the log files and we can do it right there while you're on the phone. You've got a problem? Let's make it easier for you to get it solved. So Storage Insights across AI-based predictive analytics, performance, configuration issues, all predictively done, so AI-based. Secondly, we've integrated AI into our spectrum-virtualized product. So as an example, our easy-tier technology can allow you to tier data from storage class memory to flash as an example, and guess what it does? It automatically knows, based on usage patterns, where the data should go. Should be on the storage class memory, should be on flash core modules, and in fact, we could create a configuration. We have flash core modules and industry-standard SSDs, which are both flash, but our flash core modules are substantially faster, much better latency, like I said, 30% better than the next nearest competition. Up to 17.2 million apps, the next closest is 15. And in fact, it's interesting, one of our competitors has used storage class memory as a read cache. It dramatically helps them, but they go from 250 publicly stated mics of latency to 125. With this product, the flash system, anything that uses our flash core modules, our flash system 7200, our flash system 9200 product, and the 9200 R product, we can do 70 mics of latency. So almost twice as fast, without using storage class memory. So think what that storage class memory will offer. So we can create hybrid configurations with storage class and flash. You could have our flash core modules and industry-standard SSDs if you want, but it's all AI-based. So we have AI-based in our storage insights, predictive analytics management, and support infrastructure. And we have predictive analytics and things like our easy tier. So not only do we think storage is a critical foundation for the AI application workload in use case, which it is, but you need to imbue your storage, which we've done across flash systems, including what we've done with our cloud edition, because Spectrum Virtualize has a cloud edition and an on-prem edition seamless transparency, but AI in across that entire platform using Spectrum Virtualize. All right, so let me summarize. We've got an absolute requirement from Enterprise to make storage simpler, which requires simpler product families with more commonality, where that commonality delivers great value, and yet at the same time, the opportunity to innovate, where that innovation is going to create value. We have a lot simpler set of interfaces and technologies, as I said, that are common, but that are more focused on the hybrid cloud, the multi-cloud world that we're working in right now that brings more automation and more high quality storage services to bear wherever you are in the enterprise. So I got to ask you one more question. I'm a storage administrator or a person who is administering data inside the infrastructure. I used to think of doing things this way. What is the one or two things that I'm going to do differently as a consequence of this kind of an announcement? So I think the first one is going to reduce your operational expenses and your operational manpower. Because you have a common API, a common software platform, a common foundation for data management and data movement, it's not going to be as complex for you to pull your storage configurations. Second thing, you don't have to make as many choices between high-end workloads, mid-range workloads, and entry workloads. Six nines across the board, enterprise-class data services across the board. So when you think simple, don't think simple as simplistic low-end. This is a simple to use, simple to deploy, simple to manage product with extensive innovation and enterprise- Simple to secure. And simple to secure. Data at rest encryption across the portfolio. And in fact, those that use our flash core modules, no performance hit on encryption and no performance hit on data compression. So it can help you shrink the actual amount you need to buy from us, which sounds sort of crazy that a storage company would do that. But with our data reduction technologies, compression being one of them, there's no performance hits. You can compress compressible workloads. And now anything with a flash core module, which by the way happens to be FIPS 140-2 certified, there's no excuse not to encrypt, because encryption, as you know, has had a performance hit in the past. Now our 7200, our 5100 flash system and our flash system, 9200, 9100, there's no performance on encrypting. So it gives you that extra resiliency that you need in a storage world and you don't get it on compression, which helps you shrink how much you end up buying from IBM. So that's the type of innovation we deliver in a simple to use, easy to deploy, easy to manage, but incredible innovative value brought into a very innovative solution across the board, not just let's innovate at the high end or you know what I mean, trying to get that innovation spread, which by the way makes it easier for the storage guy. Well, look, in a world, even inside a single enterprise, you're going to have branch offices, you're going to have local this, the edge, you can't let the bad guys in on a lesser platform that then can hit data on a higher end platform. So the days of presuming that there's this great differentiation in the tier are slowly coming to an end as everything becomes increasingly integrated. Well, as you pointed out many times, data is the asset. That's right. Not the most valuable one. It is the asset of today's digital enterprise and it doesn't matter whether you're a global Fortune 500, you're Herzog's barn grill. Everybody's a digital enterprise these days, big, medium or small. So cyber resiliency is important. Cutting costs is important. Being able to modernize and optimize your infrastructure simply and easily. The small guys don't have a storage guy and a network guy and a server guy. They have the IT guy and even the big guys who used to have hundreds of storage admins in some cases don't have hundreds anymore. They got a lot of IT people, but they cut back. These storage admins and infrastructure admins in these global enterprise, they're managing 10, 20 times the amount of storage they managed even two or three years ago. So simple, cross the board and of course hybrid multi-cloud is critical to these configurations. Eric's great announcement. Congratulations to IBM to actually delivering on what your promises are. Once again, great to have you on theCUBE. Great. Thank you very much, Peter.  I'm Peter Burris. See you next time.


So, I'm going to go through the whole thing and show a few things that we've been working on over the last year, just to kind of give you a sense of what we've been up to. So, one of the first things that we've done is built a new interface for exploring annotations. So, you create on documents, on media, but then you want to be able to look at those annotations and explore them afterwards. So, we built that together with an interface that lets you create profiles and look at the profile of users and for groups.  It becomes the profile for the group and it becomes the place where annotations accumulate over documents. So, if I go back to a document, here's a preprint on archive, and I go to my group control, I've now got the club there, and I can begin to do some annotations. And I might want to, for instance, grab a little app in the middle, sort the app tag here, the document tag, just a snap, post that. Then I might go to another place, for instance, here's a document, another article, and this is an HTML abstract. I'll just annotate this here. This one's at Elsevier, so I've got my physics club here. So, I'm carrying this group around the internet with me. I'm going to add comments and some tags. This one is maybe Matt, but it's about the documents. Can you see now what's in it? Absolutely. So, now I've got two annotations on two different documents, and if I go back to my interface and I press refresh, then I can explore those documents and the annotations that are on them. I can add, this is a faceted search interface, so the facet that's driving this particular view is the group physics club. But if I click on the tag, then I can start to explore using tags, using facets, the different views of this set of annotations that I want to look at. Annotations also now have addresses and direct links, so if I grab a link to this address or to this annotation, I can tweet it out, for instance, somebody can click on that and then it takes them right to that annotation and lays it over the document, and it's kind of fun to tweet out links to annotations, but also to link annotations together. So, for instance, if I go back to this first document, I can say look over here and make that link. And so, then I start to be able to link to, have annotations that link deeply into other things on the web, and that works, and have that as a ubiquitous layer that can lay over the web wherever I am. So, those are two key things we've released. We're also, I won't steal Juan's thunder here, but we are super excited about our integration with the EPUB reader, RediumJS. So now the same functionality that can lay over PDFs or HTML in about, I don't know, maybe six more weeks, we'll be able to lay over EPUBs using both RediumJS but also EPUBJS. So, that same functionality, this is a demo site, which is actually partially working. You can select text annotated just the same way that you could there. So our goal was to continue to expand the format coverage of annotation to be able to lay over more and more different kinds of things. And then the last thing I'll show you is a set of improvements to the interface that will be coming over the next couple of months, which allow publishers and others that embed hypothesis on a document or at a site to be able to determine and dictate the layers that are authoritative over those documents. So here a journal has embedded annotation on their journal and when you open the interface and you click on the annotations button, it inserts their group as the default layer over that journal. And also in this case that journal or that publisher has already authenticated their users and so the user, the pre-authenticated user comes to the annotation interface already signed in that namespace as opposed to the hypothesis namespace. So the publishers who want people not to annotate in some other system but within their own kind of world can fully integrate this sidebar and this annotation capability controlling both the layers as well as the namespaces of their documents. Of course as users we might want to bring, that's great that that publisher has embedded this tool, but we had a group somewhere else on the web in a different namespace so we still want to be able to bring that context to this journal article. So even though the journal still has, is running their own layers over their journal, I still want to bring perhaps a hypothesis group or a group on another server somewhere perhaps behind my firewall if I'm an employee at a corporation to this document and have those different contexts work fluidly together. So this is the beginning of us teasing apart this framework so that it begins to implement layers from anywhere. It's a truly universal plan. Our goal is that the hypothesis interface is really just a gateway to any interoperable annotation server and that vice versa that the annotations that are stored on our server can be viewed through other annotation clients. This is a long journey. True interoperability starts with things like standards, but some of the more challenging problems are making sure that at the user interface level that features a functionality that needs to be able to work together to be compatible between systems can do that. So it's just a quick snapshot of where we're headed. Thank you very much.