Carnegie Mellon vaccination database talks are made possible by Autotune. Learn how to automatically optimize your MySeq Core and PostgreSQL configurations at autotune.com. And by the Steven Moy Foundation for Keeping It Real, find out how best to keep it real at stevenmoyfoundation.org. Welcome! So we're here for another vaccination database, some of our theories are a talk. Today we're excited to have Hussain and Alex from Google BigQuery team. Alex has been a staff engineer at Google, leading the overall BigQuery project. He's been a PhD from UIUC, but not in databases in IoT devices. And Alex has been a technical leader of Google BigQuery project. And he spent 13 years working on SQL server at Microsoft Friday joining Google. So with that, take away guys, I have to be happy for you to be here. And as always, for the audience, if you have any questions for Alex or Hussain as they're giving the talk, please unmute yourself, say who you are and where you're coming from, and feel free to interrupt at any time, let this be a conversation and not just them talking to the Blackboard that is there. Okay, for that guys, go for it. Thank you for being here. Thanks Andy for the introductions. So hi everyone, I'm Hussain and I'm joined by Alex to talk about query processing in Google BigQuery. We are very excited to share some of the details of how BigQuery does query execution and query processing. So to start, let me give you a little bit of overview of what Google BigQuery is. It is a serverless, highly scalable and cost effective data warehouse, cloud data warehouse. And, you know, we have several growing cloud data warehouses, things that are really distinct about BigQuery and really kind of powerful characteristics of BigQuery. One is it's a fully managed serverless and clusterless service. Users don't need to set up a provision servers or clusters, they could just come into this service. And if they want, they can reserve resources and we'll talk a little bit about that, but everything we see scales up and down depending on the work shown in storage demands. It's a 24-7 service with four nines of uptime availability. Users rely on a predictable performance from the system because the queries running in the system are used in really critical reporting or business intelligence applications. In terms of a scale, it's a petabyte of scale storage and query system. We often have customers that store data at that scale and query data at that scale. Everything is encrypted, everything is terribly stored. It provides real-time analytics and streaming data for a lot of real-time analytics use cases. The main interaction, obviously, this is a kind of SQL system, so it's going to be a true SQL, but we tried really hard to make sure this is an easy to use SQL interface and it doesn't require any hints, any fine tuning, so everything kind of is taken care of by BigQuery. And we have also embedded a lot of features into this SQL interface. For example, we have machine learning primitives within SQL that are kind of really powerful and really useful for analysts to do various jobs that are not traditionally as part of their data warehousing and data analytics SQL workflow. So how do we do that? This is kind of a high-level architecture of what BigQuery looks like. It's a decoupled storage and compute system. We have a distributed replicated storage that kind of spans across multiple machines and storage devices in a Google data center. And on the right-hand side, we have a highly available compute deployment. And basically, the compute nodes can talk to storage nodes through a Google petabit network infrastructure that's available in Google data centers. And also in between, we have a distributed memory shuffle tier that's highly used to communicate between the compute nodes. We'll talk a lot about that. That's the key part of basically the query execution in BigQuery. To interact with the system, users can run queries using SQL. It's a 2011 compliance SQL. There are REST APIs and then there's a web UI and command line interfaces. And then there are client libraries in various languages that are used to programmatically access the BigQuery service, ODBC, JDBC drivers, and various connectors as well. Also in order to get the data in and out, there are streaming ingestion and reading APIs to read and write data at a very high throughput and high scale. There's also a bulk load and export functionalities in the storage side. If I want to iterate over some of the high level key architecture design principles for BigQuery, there is a combination of three things that makes BigQuery really successful at delivering some of those functionalities I mentioned in the beginning. One of that is storage and compute separation. We know that from various industry leading systems that this helps a lot with getting to larger scale, petabyte scale of compute and storage. It makes systems highly available because fault can be tolerated at different layers independently. And also it helps a lot with serverless and multi-tenancy because resources can be isolated, scaled independently. So this is a key point to enable those functionalities. This is combined with a really good use of co-location and caching across various storage systems and network system that really gives us a high performance at low cost even though there is a storage and compute separation. And then the third thing here is really integrated hardware and software stack. BigQuery can take advantage of Google technology, the hardware technology and data center technology to get really great performance in terms of storage and network and data transfer. So with that high level overview, we're going to do a little bit of more of a deep dive into query execution. And with that, I'm going to hand it over to Alex to talk about query execution basics. Hi, everybody. So yes, I'm going to talk about how BigQuery optimizes queries and some details about how BigQuery runs them, how we distribute resources between running queries. First of all, a little bit of terminology which could be unusual. In BigQuery, project or reservation denotes a pool of resources available to workload. And in the cloud, of course, we have various ways to end up quickly to shrink or grow that resource pool. For example, BigQuery can do that with one minute granularity. For simplicity in discussions, we're going to assume that we have just fixed pool of resources available, which is kind of quite common configuration basically for many users. So now about BigQuery query processing goals. Of course, we have a goal to run queries quickly and efficiently. But besides that, we also have a goal of trying to achieve more predictable query performance whenever possible. As Hossain mentioned, BigQuery has a SLA of four nines to put that into perspective. Essentially, the goal is to have just one minute a week of downtime if averaged. Such high SLA is definitely suitable for critical workloads. But then if you're running critical workload, it's also important to finish queries on time by deadline. So again, we also need to invest in as much predictable performance as possible. And this affected some of the algorithms and designs. So let's go back to the topic of how queries run. Query is executed using multiple stages. Typical of example of stages, you know, some, for examples, we scan local table. We do some filtering on top of that. And then we do basically partial aggregation. Each stage consists of tasks. Task is unit of work to be done. Assigned to workers and workers then can process tasks. So stages done when all tasks are done. Then another stage could become unblocked and we can start another stage to execution. Multiple stages could be running at the same time. Again, typical example, if, for example, staging needs to filter aggregation results. So that stage would need to wait before aggregation is done. And then that stage could start to execute. Number of stages in query is not fixed and sometimes could change depending on data distribution. But overall role of thumb is that, you know, each aggregation probably requires one stage. And each join adds one or two stages and so on. So basically in this slide, we see example of basically of simple query which requires two stages. In the first stage we do table scan, filtering, then local aggregation, and then second stage computes final count. Next slide, please. Okay, here I'm going to mention some interesting facts about how BigQuery runs queries. So, BigQuery tries to keep data in memory during query processing if possible. That is, if you are running a medium-sized query in an idle project, meaning you have some idle set of resources, medium-sized is something, and we consider that query medium-sized is something that processes, for example, one terabyte of data, then quite likely intermediate data will never be stories on disk or not even on SSD until we finally persist query results. So basically, BigQuery tries to do memory processing whenever it's possible. We will go into more details about dynamic scheduling later, but as I mentioned, query is executed using multiple stages, which has multiple tasks. Number of workers that process those tasks is very flexible and can change in matter of seconds from anywhere from one worker to many thousands of workers. About failure recovery, so stage tasks could be restarted or executed speculatively. This helps to maintain high performance even if you have rare hardware problems in a highly distributed environment. So for example, if one of the computers becomes less responsive, we do not have to wait a long time till we decide we have real problem or not. We could instead simply speculatively start task execution in a never-location even after slide delay. This way, hardware problems won't affect query performance. So, any question? Yeah, so for the sake of working, can you say what percentage of the tasks in the entire BigQuery fleet are executed speculatively and were incorrect? Like there's a hiccup in the network or something and you think a node is down, but it's really okay. So you start speculatively executing the work, but it turns out you wasted that because the node came back right away. You know what I mean? I don't think it's very large. It's mostly extremely large queries that require basically, you know, some duplicate executions of some of the tasks. But in grand scheme of things, it's not very big fraction of things. Okay, any additional questions so far? Again, queries consist of stages, stage of tasks, tasks executed by workers. You're talking about the scheduling stuff later on, right? Yes, yes, yes. I'll hold my questions for that later. Okay. Next slide, please. This slide illustrates various components that are involved in query processing. It also shows how stages interact with each other. Each stage reads data from either a distributed file system or from a distributed component called shuffle, and we will talk about shuffle in more details. All work for given queries coordinated using, you know, coordinator. There is also scheduler component that helps to assign tasks to workers. So on this slide, you see stage one reading data from the distributed file system and processing it, and then it puts intermediate results into shuffle. The next stage two will read data from shuffle, process it again, and put again partial intermediate results into shuffle and so on. In summary, each stage usually reads data from distributed file system or from shuffle, and then usually puts results back into a shuffle. So looks like this shuffle component is very important. So what exactly it is? Next slide, please. So shuffle is essentially very fast and very highly available storage combined with data repartitioning engine. Shuffle not only stores data, but also, for example, can hash partition it. And as we know, hash partitioning is very common operation in distributed query processing. If you want to compute aggregation, we usually hash partition data on aggregation keys. If you want to perform join, you know, between two large tables, usually we need to hash partition inputs by join keys. So shuffle component does exactly that. So besides storing intermediate results between stages, it also partitions data. Shuffle generally tries to keep all data in memory, but of course, if intermediate data sizes becomes very large, then it could write, you know, those intermediate results into distributed file system. All these processes quite fast. And in many cases, it's actually hard to tell whether the data was in memory or, you know, some of the data ended up basically on disk. So any questions about shuffle? I believe there's a question on chat. Someone asked, what happens when a server dies? Do you need to restart or like Hadoop MapReduce query continues? And Alex could elaborate on that one, but basically we use this shuffle as an intermediate persistence layer, like MapReduce or Hadoop, basically to recover work that needs to be restarted. And it's at the task level, so not the whole stage needs to be restarted. Yeah, so for most part, you would never notice any, like, there could be significant, quite significant problems with hardware and you wouldn't even notice that query would not even slow down, let alone be canceled. Okay, next slide, please. So if people have questions, kind of type in the chat, feel free to unmute yourself or just ask them. Okay, I just sometimes do not know that. You're fine, I'm telling the audience. So Swar, are you able to unmute yourself? Yeah, hi. So is there any parameters for handling the shuffle partitions or BigQuery does that behind the scenes based on something, because we have traditionally seen in distributed things that based on the data load or too much of, you know, skewness of data or doing the broadcasting, we have to handle by use case by use case this partition parameter. It was BigQuery does that by intelligent looking at the data or? Yeah, so we have actually many slides, you know, discussing how exactly the partition data, so we'll go into those details. Okay. All right, Javi, you have a question? Yeah, can you hear me? Yes. So in your serverless environment, so you do parallelism across servers, but in a given server, do you have a special parallelism for multi-core like you got 32 cores and do you have anything special for that? Yes, so basically it depends of what kind of workloads you're running, BI or data warehousing, but we definitely use more than one thread basically to process tasks. Yeah, because in multi-core also you need to shuffle data across, so do you use the multi-node shuffle or this is something special? Again, it's slightly different depending basically where we're using essentially a business intelligence engine or a regular, but there are still other things that you can process in parallel depending basically on query shape and state shape. Okay, so let's go into next topic. Okay. So in BigQuery, one can have different size reservations or projects, so essentially again to remind this is set of resources. The slot represents some fixed set of resources available for query processing in the reservation on project. These resources could be CPU, worker, RAM, shuffle, RAM, networking and so on. If you compare 500 slot reservations, for example, versus 2,000 slot reservations, then a second one is going to have four times more resources dedicated to query processing. So essentially usually we do not measure resource pool size using number of computers. We measure it using a virtual description of resources committed to that reservation. The most slots we have, the faster queries will run. So typical configuration, like most common configuration for reservation is 2,000 slots, let's say, and it maps to some number of computers. We just don't kind of distinguish that number. So when queries run, sometimes some of the query stages do not need many workers simply because there is little data to process. The system is able to very quickly take away workers from queries that do not need them. The system is also able to very quickly redistribute workers between queries in flight to make sure each query gets fair share. So for example, if one query finished, then immediately the rest of the queries could start using additional workers and then they run faster. This could happen as soon as next second. Or if new query was submitted, then a small fraction of workers is going to be taken away from each already executing query and assigned to the new query. Again, every query gets own fair share of workers. This fluid resource distribution is important for more predictable query execution inside the busy system. So if you think in terms of how much resources every query gets, it is clear that the dedicated resources correlate with average reservation load and not with how busy system reservation was at the time query arrived. So imagine typical scenario where a few independent pipelines are performing calculations by running series of queries. Pipelines are independent, so there is no coordination between the moments we submit queries. So if we were, for example, to queue queries or to lock in resources allocated to query at the time query arrived, then system performance would be much less predictable. Because performance frequently would depend on luck. There were particular pipeline query arrived in front or behind another large query from another pipeline. And then it gets sometimes more dedicated resources, sometimes less dedicated resources. Because of this flexible resource distribution, that is much less important here. Essentially, you know, we get more predictable performance because each query gets a fraction of resources which is averaged over the time query was running. So any questions about this? Yeah, is there any mechanism for a user to override the default of fare distribution so that they can prioritize queries? Jose, do you want to answer that one? So there are some mechanisms in terms of reservation controls today that BigQuery provides in terms of how resources are allocated between multiple reservations. So there is some of that, but, you know, there are tons of workload management features that, you know, we've been working on to not exactly maybe do it as you mentioned, but have some kind of control over how things are prioritized over each other. So that's something that we've been looking into as well. Thanks. Yes, essentially, there are quite a few options to adjust this algorithm, which they described, but, you know, key idea is that you have quite a lot of flexibility to transfer resources from one query to another very quickly. Next slide, please. May I ask a question? Sorry. Yes. So I'm not sure whether I missed this earlier, but could you elaborate a little bit on what you mean by this fare? It's like you are going to estimate the required resource for each query and then distribute it, distribute your slots accordingly. What could you elaborate a little bit on this concept of fare and how that's decided? OK. So if you have five queries running simultaneously, we will try to make sure that every query gets the same CPU time per unit of time. For example, it depends on your reservation size, but we'll essentially try to make sure that approximately the same resources are used to execute each of the five queries. Regardless of how big those queries may be? Yes. Regardless how big those queries might be. OK. Sure. Thank you. If I can add to that, you asked estimating the number of slots of resource usage beforehand. We have algorithms there to monitor the query execution and figure out how resource usage has been evolving over time and adjusting based on that. So it's a lot of dynamic allocation of resources over time. So that's why it doesn't require to estimate that from the beginning. Thank you. Thanks for the explanation. Next slide, please. OK. So let's talk a little bit about BigQuery Optimizer. So BigQuery Optimizer was implemented basically from scratch as part of Dremel development at Google. It uses both cost and rule based optimizations to produce an optimal query plan. So do you have any special goals for BigQuery Optimizer besides performance efficiency and query predictability? So one of the additional goals was increased focus on handling nested data, like structures and arrays. It just basically quite convenient to have very native support for arrays and basic land structures. So what are examples of cost and rule based optimizations that BigQuery does? So a good example of cost based optimization is materialized view choice. In BigQuery, one can define materialized views that, for example, hold pregregated data. Materialized view could have clustering columns defined. So essentially data and materialized view will be sorted by these columns. So then query optimizer trying to find most optimal materialized view to use. It needs to take a look at data filtering involved and how data was clustered so it can find materialized view in estimated minimal number of rows to scan. And this is exactly what it does. What about rule based optimizations? So of course everybody knows now simplest rule based optimization usually is filter push down. We always try to execute filters as well as possible. Another good example of rule based optimization is partial aggregation. BigQuery always prefers a due partial aggregation before final aggregation because it helps to reduce possible data skew by the time you need to hash partition data by aggregation keys. So this way query processing time is less sensitive to data skew and more predictable. And again, data skew is frequently very difficult to predict up front. Next slide, please. So for the optimizer, can you say anything about like, is it a classic case cascade top bottom or a system or bottom top? No, it's not based on cascade ADS. So in data analytics workloads, we have some specific which sometimes now you can do essentially things a little bit more I guess in simpler way. And but we have many other interesting things happening with query optimization. So let's go maybe to the slide. Can I ask one question? Yes. Sure. This is Rebecca Taff from Cocker's Labs. And I was just wondering, you mentioned that you're doing some interesting things with support for arrays and other types of structures. Are you going to talk more about that? So I don't know if you saw some papers about Dremel originally, like, you know, quite frequently people like to do denormalize data for extremely large data sizes. If you think something like petabyte in size, frequently, you know, at that scale, it's even hard to normalize data or you may not have enough time to do that. So for this particular scenario, it's extremely large data scale. Sometimes it's useful to have good native support of kind of nested denormalized data just because, again, a UATL process could become extremely expensive if you go with those data scales. So when you say nested support, are these things like inverted indexes and things like that? No, I meant like you can have some columns that have repeated data inside like arrays and each array element in turn can have like structures inside with multiple fields and that field again can have some arrays. So basically some deeply nested data which is stored in single table. Got it, thanks. Okay, so let's talk beyond basically rule-based and cost-based optimizations. So BigQuery also has optimization that it does in the middle of query processing. This is because frequently queries are auto-generated by various tools and are quite complex and it's difficult to reliably estimate a data distribution, let's say, after 10 joins. Up-front cardinal estimates we have errors that grow exponentially with query complexity. In addition, those errors could be quite sensitive to parameter values or data size changes or various statistics that are collected. For example, BigQuery has two different algorithms to run analytical function row number with over close which contains order by close. One works well with small partitions and another works well with large partitions. If we were to use estimates to make algorithm choice upfront, not only choice could be suboptimal but because estimation error could keep changing, we may end up switching from one query plan to another back and forth resulting in less predictable query performance overall. So instead what we do is we observe and monitor data sizes in real-time as we process data and then we can change query plan as needed based on these observations. Let's see. So by now I mentioned a few times that we are trying to make query performance more predictable whenever we can. I already mentioned partial aggregation that helps make query processing less sensitive to data skew. Another example is broadcast joins. Those have less penalty in case of skew joins and BigQuery prefers to run broadcast joins wherever it can. And we will go into even more details in the next slide. Next slide, please. So what examples are of query plan adjustments that BigQuery can do in the middle of a query running? First of all, parallelism level. BigQuery can change query parallelism depending on data size. Say initially we estimate it, we need to aggregate one terabyte of data but then it appears that amount of data is going to be larger, more like 10 terabytes, for example. Intuitively, if you have additional workers available then we should use them because the amount of work went up and this is exactly what happens in BigQuery. Another example of changes to query plan that we do dynamically is we could switch between broadcast join and shuffle join. So BigQuery runs two kinds of joins. Shuffle joins is when the hash partition both joins sides and then each worker gets own partition to process. And broadcast join is when one of the join sides is small then we can broadcast it to every worker. So I'm pretty sure you're familiar with both join flavors. So broadcast joins have some nice properties that are less sensitive to join skew and usually faster, especially when broadcasted table is very small. So ideally we should use broadcast join whenever we can and this is exactly what happens in BigQuery. We measure observed data size in the middle of a query and basically in the middle of a query that we are processing and then we decide and we can actually change join flavor on the fly. And again, it doesn't depend at all on query complexity. It works with extremely complex queries. And yet another example of query plan change that we could do is basically row number analytical function as I mentioned. We have two different algorithms to process it. You know, when it requires sorting and we would measure basically data size on the fly and then we would decide which one algorithm to activate and for which partitions. So any questions related to kind of dynamic query plan adjustments and Hossein in next slides will go in many more details how exactly these adjustments are happening. Yeah, so this is Hamid. So do you also change during the runtime the join method? Like you know, if you use hash join, sometimes it runs out of memory. So for those blocks, you may want to change to the sort of join because you have much more control on the memory. This is something that actually pressed on us. So we don't use sort-based joins but we definitely pay attention to data sizes and again, you don't need to worry about which one join flavor we will run. You will never run out of memory or anything. Query will complete always successfully. And again, we will choose join based on looking on what kind of data sizes we get and data distribution. And so in many cases we try to use broadcast join if we can, but if we cannot, you know, we'll run shuffle join. Yeah, so that I understand. My point was that for some blocks you will be using hash join and for some blocks you will be using sort merge join. Yes. Sort merge, no, we're not using sort merge join. So how do you avoid the discuses due to hash overflow? So broadcast joins usually don't need any special handling. Shuffle joins do need usually special handling. I think you were saying, it's like the classic textbook recursive partitioning, right? If you overhaul the bucket, you just partition it again. Yes, so Hossein will go into some details how we basically deal with large hash buckets. We have dedicated slides about it. Okay. So now Hossein will continue from here. All right, thanks, Alex. So with that kind of high-level overview of dynamic query optimizations, I'm going to go through some examples and some of the kind of basic operations that we do here. I hope that I would be able to answer some of those questions as well. So let's start with an aggregation example here. We have a query that tries to do a group by operation. And the dynamic partitioning algorithm is really aimed at being able to, in a scalable way, execute this distributed aggregation without having any statistics and knowledge about kind of the input to the operation. So let's say we start with some workers running the first stage of this query and doing some partial group by locally from the input table. And then they start to actually do some shuffle. Basically, we want to do a partition by a dynamic partitioning based on the key, the group by key, and then do group by locally. So that's how we get the distributed group by running. But this shuffle needs to have some notion of dynamic partitioning in it because we want to optimize the level of parallelism and the number of nodes participating in the consuming side of this shuffle operation when we want to compute the group by output. And I will talk about that more in the next slide. But after that shuffling with dynamic partitioning is done, data is actually distributed based on the group by key across a set of workers that can now do the final aggregation for each of these key groups. And then they perform the sort and limit operations on top of that as well. And then finally, the results are combined in a worker with the final sort and limit to get the final, like, 100 top results out and back to the customer. So let me go through that shuffle with dynamic partitioning operation and then I'll pause if you have any questions about this particular algorithm. So dynamic partitioning is really a key kind of a core critical part of the BigQuery dynamic query execution. The goal of it is to dynamically load balance and adjust parallelism every time we have a shuffle operation. Assuming that there is nothing known about the input to these operations because it's hard to estimate, you know, filter effectiveness or cardinalities or things like that, we have this powerful primitive that can always adapt the execution. So let's go through an example and see how it works. We have two workers producing some data. They start to write data into two partitions. These are hash-based buckets basically. And, you know, going back to that skew example, a lot of times one of these partitions will get a lot more data than the other one. So in this case, the partition number two gets a lot of data. And while the query is running, while the stage is running, the query coordinator detects that there is this skew in terms of the partitions and on the fly, basically, directs workers to actually change the way they partition the data and use two new partitions, partition three and four to further divide the data into more partitions. And they basically stop writing to that partition two and start to write into partitions three and four. So as the query and the stage basically progresses, more data gets into those partitions. And finally, the data that we previously wrote into partition two gets repartitioned into the new partitions, partitions three and four. So at the end of the query execution, partitions one, three and four, sorry, I shouldn't say query execution. The stage execution, we have the data in those three partitions. Any questions so far? Yeah, so what do you do with aggregate distinct? Do you still do the same thing? Because you have to remember what you have counted. You shouldn't count anymore. Like some distinct. So any aggregate, hash-based aggregate operation that runs in a distributed way can be built on top of this operation. So what we want at this stage is create a partitioning that is kind of load balanced and kind of roughly equal across these partitions. What we do on top of that, which is an aggregate, whether it's a count distinct or something like that, can be done. It can be done basically independently because these partitions don't overlap. The key here is that that partition two, the data there gets repartitioned into three and four. So the same data doesn't appear when we consume it at the next stage or next level. There are only no duplicate keys. The keys get repartitioned into those two partitions. Hi, Hosin. I have one question. So if the clustering is enabled on this table, along with this partition, so how does this worker handle that? Let's say on your earlier query, let's say I wanted to look at based on state, like California or Georgia, if you put it on the VAT clause. So I think I hear that BigQuery recommends to put clustering when this kind of distinct values are pretty high, like the state or gender whatever, right? How does it balance this partitioning with clustering? So that's a great question. So when you have clustering and you have your, for example, aggregation or filter on the clustered column, what you see is that your input workers that are processing it, do the partial aggregation and produce a tiny amount of data that actually needs to be shuffled through the dynamic partitioning. And that's where this dynamic partitioning works pretty well, because if the data is already partitioned properly for that type of distributed operation, you produce, you know, the query produces a little bit of data and all of that sometimes just gets aggregated into one worker and then sends back. So a lot of work is saved through that. Okay. All right. So I'm going to move on to join example now. We talked about broadcast and shuffle join and one of the key parts of the dynamic join execution is to decide between the two join methods. So the broadcast join is a simple diagram, a simple query that shows that if one side of join is more smaller than the other one, the worker, in this case, a single worker reads that, uses the same shuffle primitive to actually send the data, transfer the data to the workers that are probing the probe side of the join and building a hash table based on the broadcast side and then aggregate the result finally. Similarly, a shuffle join basically uses a shuffle primitive on both sides when we want to actually co-partition the data on the same partitioning keys and then have workers reading from left side and right hand side partitions aligned together and then compute the join and this is in this case a hash join and then aggregate the results at the end and send it back. So these are the two different execution graphs for broadcast versus shuffle join but what happens in reality is that we don't know beforehand about the size of these two tables, so we need to dynamically go from one to the other. Now I'm going to, through an example to show how that works. Before getting into that, the dynamic join algorithm needs to make multiple decisions and those are very important performance of join. One of the advantages of broadcast join as Alex mentioned is that in terms of data skew, there is usually no performance penalty and also it transfers much less data in most of the cases compared to shuffle join. So it's very important to find the right join algorithm. Also we need to decide on the number of partitions and workers when we want to do the join and the dynamic partitioning algorithm that I mentioned earlier is very useful here. The dynamic join also needs to coordinate across multiple joins and sometimes swap join sides depending on the sizes and shapes of the join. And a pattern that's very common in data warehouses is the star schema joins where there's a fact table being joined with a lot of dimension tables. That's called a star schema and our optimizer is able to detect such joins and then do a constraint propagation. Basically compute constraints from the dimensions on one side and propagate them to the fact table, those predicates such that the fact table is heavily reduced before the join happens and that helps with whether it's a broadcast or a shuffle join reducing the amount of data participating in the join phase. So with that I'm going to go through an example of choosing the join algorithm through the dynamic join method. So in this case we have two tables that are being joined on a key and the dynamic query execution algorithm starts to run this stage basically that scans both sides of the join and shuffle them. So at this point the query coordinator doesn't know how big of a join it is or how big each side of this join are. At some point it detects that one side T1 is too small and this means that we don't shuffle both sides and run it at the shuffle join we can actually run this query as a broadcast join. So what it does it cancels the other side the T2 side so that it could scan it probe T2 basically and broadcast T1 into T2. So in this case that the same data that was shuffled is now read as a broadcast through workers that are doing a hash based join from the broadcast side to probe side here. They compute the result and then the result sends back the query. Now with the shuffle join case the same thing might happen but in this case T1 and T2 might be both too large. So the query coordinator detects that case and in the middle of the execution decides that this join cannot be executed as broadcast join. So it continues to run it partition join and then basically it schedules the next stage which does the shuffle hash join. Any questions? So I'm going to go through a more complex example of dynamic query execution to give you an idea of for more complex query plans how we can use these dynamic query execution primitives to predictably run queries without the statistics about the inputs. This example is about computing row number analytic function. So in this case we have a partition by clause potentially and then we are computing row number for those partitions. It's an expensive operation because we require to sort data for those partition and because of that it's important to get the right level of parallelism. However it's a little bit challenging because when we partition the data sometimes some of the partitions might get a lot more data than the other ones and we want to be able to scale to that and kind of handle that skew in the partition sizes efficiently. So what BigQuery does is starts to scan the data performs the hash based partition into initially created sort of buckets the partition by buckets that are used in the query execution. And then we have some of these partitions will be small and the execution will be kind of simple for those cases. The data is sorted locally within the workers reading that partition and then the row numbers are calculated. But for some of these partitions that are larger we have a secondary stage within the query execution. So as the query is running a new on top of the buckets that were created to do the hash partition basically one of those partitions is now split into multiple partitions based on the the ranges of the values that are sorted here. And then each of the range partition buckets are sorted separately. So it's kind of gives us a global order because these are range partition global in the sense of that particular partition and then the row numbers can be now calculated in parallel across those workers. And finally the results of both small partition and large partitions are kind of all combined and returned to the user. Any questions on this? So does it mean that one partition that you need to put the row number on like 012345 with the max is really split across multiple then how do you compute the row number? So this is kind of a distributed sort problem and computational row number. If you know for example how many rows you have in each of these buckets and these buckets are not overlapping in terms of ranges they are covering you can create a global row ID but you need to know how many rows. So the actual partition is even more complicated than this. It's not just you know you distribute the data and immediately start running the row number calculation on top of that you have to do a little bit of kind of communication and detection of how the rows are distributed across these buckets but at the end you have these ranges and each of them will have a known number of rows in them so that you can assign a row number to all of the data distributed there. So this is going to be our last slide so this is kind of a summary of what BigQuery does in terms of query optimization. It uses a mix of cost-based, rule-based and dynamic optimization and we talked about why we really need to have a mix of all of these to have a predictable performance for a very versatile workload. The key to use that dynamic optimization algorithm is that there are two things really. One is that BigQuery's ability to collect real-time statistics and updating all of these operations and flights mid-execution so that is a key to do that dynamic partitioning or choosing the join method or doing this row number calculation for example. And then the second key ability of BigQuery to enable dynamic query execution is the fact that we can re-execute the small parts of the query because Shuffle creates this snapshot or checkpoint basically that we can go back and re-execute parts even small tasks within the query so that that allows modification of the query plan really easy. The dynamic optimizations we just talked about when it's combined with that adaptive resource allocation that Alex talked about in terms of resource allocation gives us that performance predictability because now we can really, depending on the shape of the query and the shape of the data, provide kind of the same throughput and output in terms of performance. And also it is a much more robust way of handling arbitrarily complex queries or datasets without relying too much on things that might break easily in terms of the statistics. So this concludes our talk. Thanks for listening and if you have any questions, Alex and I would be happy to answer. Okay, awesome. Thank you. I will applaud on behalf of everyone else. So we have a few minutes for questions. So if you have any questions for Alex or the team, please unmute yourself and fire away. Okay, this is a media game. So one thing over here is that you talked about your caching a little bit, so how do you handle the brownout that is you lost the cache and then now you have to go to the object store to get it and it takes a while to do it. So is your cache really persistent within the over cluster? Let me try to answer that. So in this execution model we didn't cover a lot of caching most of the time the data can be read directly from distributed storage and this is relying on how Google built the distributed storage system and the network infrastructure. So most of the time, without any caching, the performance of the storage system is pretty good. So customers don't even see the difference. Caching is used in certain cases for example, BigQuery supports result caching for certain workloads some other in memory caching for data, but those are really specific cases not that in the general use case. Yeah, but the modern servers even Google publish numbers actually. If you use a local NVMe, the difference between that and even NVMe on the other side of the network is like order or magnitude. So it does make a big difference to go to your cache versus object store. You're right but it depends on the type of workload you're talking about. Especially when you're talking about large scale analytical workload, you have a lot of downstream operations such as shuffle, repartitioning and those are very expensive orders of magnitude more expensive than the cost of a network round trip to NVMe, remote NVMe for example or remote disk. So in most practical use cases what matters in query execution is the query plan, the amount of data that's transferred over the network, not necessarily the first round trip to storage. But you have a good point I mean there's obviously a lot of those and there are various techniques that are being used in certain cases within BigQuery tracks as well. So this is Lin so I have a little bit higher level question. I'm just wondering so I think it's great to have the flexibility and adjust things dynamically based on data sizes etc. So I'm just curious have you guys considered to remember some of the choices you have on a particular table for a particular query or sub query so the next time if you see a similar thing you don't need to do the adjustment dynamically again. I'm just curious have you guys considered it or what's your thoughts on that? Alex do you want to take that? It's definitely interesting idea but basically for the sake of this discussion we're probably not going to cover that one but I can definitely see where you're coming from it could be an interesting discussion in the future. Okay sure thank you Alright question from Ryan Hi I noticed that throughout this presentation you mostly focused on a fixed number of slots are you investigating allowing projects to use more slots dynamically? I don't know really what a slot means precisely so maybe that doesn't make sense as a question but I'll just let you run with that. Let me try to answer that so a slot when we talk about it is really you can think of it as a unit of CPU, a virtual CPU unit really and we are not I mean to clarify queries go can use dynamically however many slots they need up to depending on how the resource provisioning or resource commitment for a customer project for example if there's a reservation up to a certain limit so nothing is really fixed here things change very dynamically in terms of how many slots are being used by a query if your question is more about provisioning for a project or a reservation you know when for example a customer comes in and they purchase a certain number of slots and they pay for it they want to dynamically adjust that that's a different thing we have multiple ways to dynamically adjust the size of a reservation but again at the query level things are very dynamic and the number of slots that are being used very very rapidly and dynamically changed during the execution thanks okay anybody else I will ask my question well I had I posed one question earlier but you said it to me as a direct message you said it to everyone else oh I see go for it yeah so regarding your shuffle so you mentioned that you persist the shuffle so that if there is an old failure you will get it from there and continue like what my use does in this part but also that is very expensive so do you persist that in your local cash or you have sent it all the way to the object store so as Alex mentioned we have an in-memory shuffle layer that can persist it in memory but with certain durability guarantees and you know there is obviously always some risks that you lose data that's in memory even if you replicate it do things like that but the thing here is that if you reduce the chance of that then most of the time you don't have to redo the work but that's a good question I think this is really the key here being able to do in-memory shuffle otherwise the cost of core execution would be very very high okay so my first question is the Dremel paper was 2010 so it's 11 years ago are there any assumptions that were made in that paper that given that BigQuery it's an opportunity now as a product and it's exposed outside of people outside of Google are now able to use it is there any assumptions that were made in that paper in terms of the system design that are no longer correct or have been invalidated or you've changed the queries in time I think that's a very good question so recently if you know Dremel won the test of time award after 10 years in VLDP and there's a kind of a short paper that we published that exactly answers that question what are the things that we assumed at the time and things that actually no longer hold and I think it's very interesting to go through that yes there were certain assumptions that were made into how we handle you know the data for example how we handle execution and a lot of those changed but some of it is still very valid and I think Dremel and BigQuery pioneered that area and still continue to do that so I'm going to refer you to that paper and I can send your link sorry that was a softball that was killing you out but actually another question I have is as you said Dremel was very influential there's a bunch of open source clones Drill and I guess Impala either any part of those systems or any features or something that those open source systems have done and you guys you had a BigQuery and you saw that and that was a really good idea and you ended up implementing it as well but do you guys still see BigQuery as the state of the art and farther ahead than any of the open source guys I think I'll stop at this one Alex please feel free to add your thoughts as well and there's definitely a lot of very interesting developments in the open source committee you know BigQuery can claim that it always stays ahead in terms of being a state of the art I think some innovations in the storage data layout formats that happened for columnar processing was very interesting things evolve significantly again if we go back in time 10 years multiple iterations of how we store data we didn't BigQuery evolved so that a lot of it some of it were actually influenced by how things evolved in open source and then we obviously added things that we felt like are important and are important in terms of query processing and performance storage performance Alex do you have any kind of thoughts examples you wanted to add No, I think you captured it perfectly Yeah BigQuery essentially changed quite significantly in last 10 years always dynamic execution query plan adjustments most were not part of original Dremel but we definitely watched other progress being made and it looks like we still have quite a lot of opportunities in analytical data processing to make things better because that's what it takes a lot of money because it's always something else to do Alright guys Hussain, Alex we'll clap in half everyone else thank you so much for spending the afternoon with us


Tarzan of the Eighth brought to you from out the pages of Edgar Rice borrows enchanting books Filtering through the tangled mouth of leaf and vine the darting rays of the rising sun stabbed the jungle gloom in little jets of furnished gold From above sounds the screech of brilliantly gloomed palace the ceaseless chatter of apes and monkeys Below from the dark green recesses of the jungles torturous paths comes the soft padding of cat-like feet on the spongy moss A shadowy movement in the fall bamboo a glinting of black and gold as sheet of the leopard takes a cool retreat after his nightly kill Flash of green more vivid than the shuttering leaves Soft slithering sounds like lapping waves on a shingle beach and his father snake slides into the ranked grasses On the little platform in the trees Jane Porter and Tarzan what's the ever-changing jungle away? Oh When you've already gathered the fruit root Well, yes white skin One by one Jane holds up the fruits as she names them and Tarzan picks them up as she lays them down then Then Anna banana No Banana White skin go come back many fruit white skin go come back Give Give And Jane as she says give hands to Tarzan a wild plum Tarzan's eyes brighten as for the first time he recognizes a new word Not at the end of a sentence white skin give Jane from White skin give Jane Yes, now you're really talking talking talking. That's right Jane emphasizes the movements of her lips or she says the words talking eat Talking No, no, no, no, no, no, no, no, no, no, no white skin No, no white skin eat Jane moves her lips without sound and Tarzan watches intensely to catch this difference He cannot quite understand now. This is talking Jane hungry eat Why can't hungry go quickly come back each fruit? Yes. Yes white skin talk talk Talk and Tarzan nods his head several times to convince Jane that he at last understands the difference between eat and talk Oh Tarzan turns as the young ape gays and swings on to the platform Chica they do Gays and Tarzan tries to explain that the young ape is gays and Chica's baby He turns again to the ape back a lot fool up, but pando John White skin go Come back quick all right white skin, but come back quick. Yes. Yes. Yes Jane cannot understand what Tarzan is going to do But where before she would have been terrified of being left alone now She accepts it with Tarzan's assurance that he will come back quickly as for Tarzan He does not have sufficient command of his new speech to tell Jane that talk about the bull a business Treating one of the tribe and that he Tarzan as leader of the great apes must do as gave on asks go to the dumb dumb and Settle the difference as the ape man swings off into the jungle On the platform To satisfy his curiosity about this white hairless sheet with whom Tarzan spent so much of his time Often another part of the forest Professor Porter Philander Clayton and darno press on toward the cannibal crowd This almost impenetrable wall of trees and vines and everything seemed endless We don't need to be making any headway We are making some headway and in any case Philander don't forget that while we we are more or less bringing up the rear Cleaking down no and the sails of the head are really making it quite easy for us. I suppose so. I suppose so by the way, Professor I've been thinking yes, I didn't want to mention this as long as Clayton was within hearing But you know it strikes me as being more than a mere coincidence that our jungle friend rescued me from the blast Come to the point Philander come to the point. What are you trying to say? And why should you be a first to letting Clayton here because Clayton seems to have taken the most unreasonable Slight to this pagan person what I used to say was perhaps Clayton is after all correct And Jane has already been rescued by him and yet and yet yet what Unbelievable as there can be two people living in this jungle I mean Tarzan of the apes leaves a notice on the door of the hut warning us that he's watching Then this pagan who neither speaks or understands English is always at hand just as if he were watching and unless he is Tarzan of the apes Then why have we not seen Tarzan for days yet the night the same thing has been puzzling me Philander I am almost convinced that this that this night of the jungle is Tarzan, but professor, how do you And there you go again Philander one minute to try to convince me of something and then when I am about to offer Corroborate in evidence you begin arguing against your own Possibility but listen professor Can you explain how this man can read and write and still not understand the talk English yes But you asked me a question and I gave you a most emphatic answer Yes, then explain yourself professor explain with eight come pictures of John moves shop for your by name Translated the room better stone of course of course I know that I know that and George Smith transcribed and translated innumerable tablets of cuneiform Yes, yes, I agree But what then what then but can you or anyone else talk ancient Egyptian or calcium or a serious of course not But you and I as archaeologists can read and write both we cannot however speak the language Why cannot they afford Tarzan of the eight have the ability to read and write English Without being able to utter a word of it Professor you you almost convinced me Still I don't see how you could have learned to read and write There has not enough that my dear sir at the lander is a different problem But one which is in no way detached from the soundness of my previous theory Amanda water God just reported that they've cited the village Rather early yet to tell definitely The end of our search is insane so clean just say now, Mr We do not want to act as it in any haste or with any fear of being outnumbered But we want to be on our God may we suffer so dear that goes without saying but we must not have the appearance of an attacking party I'm afraid I don't understand Here we come all this way to affect change rescue if possible and now when we do get within hailing distance of the place We're not to do anything now click You're all together to impetuous I'm sure that you came and down. No does not mean to sit down and wait for the blacks to release. They must be Yeah, mr. The professor, mr. Clayton We have no means of us failing how many there are in the village therefore to attack is foolish They could shoot us down with comparative ease That's not what I meant on all that I'm trying to say is that I think the time has arrived For strong measures not me at all strong measures my friend are ideal when you have at your command a strong force Now we have only a small party But we do represent the authority of the French nation I propose to reason with these blacks convince them if I can that resistance and our Deaths will be avenged by the arrival of many ships and innumerable soldiers with all due respect to the avenging arm of the French Government will do us little good if we put our heads so to speak into the lion's mouth and the lion gets shot Out for his eatin at the same time Clayton if we ruthlessly attack these people the first thing they'll do probably is to kill Jane Yes, yes, sir, and I totally agree. That is the thing I have most fear though I've not brought myself to speak of it then measure with a little noise as possible with nowhere kept that concealment We were blind To Jane Porter lying on the platform in the tree the breeze carries faintly the distant sounds of the search party The girl scrambles to her knees leans out as far as she can listening intensely Again the voices can it possibly be does it mean that they did not leave on the steamer for a moment She's threatened. She hears pencil voice and then fell and there's deep voice answer She glances about if only white skin would come back Again she listens The only answering sounds now at the jungle noises the sounds die in the distance Faintly in the lull comes a sharp crackling of underbrush the confused murmur of what may be voices Once more Jane branches down the jungle trail still no signs of white skin's return in an instant She makes up her mind. She knows it is a search party quickly Jane grasps a branch He's on the young ape realizing the danger in which this hairless. She is plunging tries to stop her She lowers herself by swinging feet reach for a good hold She's enticed desperately to stop her Far as I know that's hard in his care But Jane is heedless of the danger that scrambles to the ground and ignoring the warning chatter of the end Such as headlong into the jungle I'm sorry, I'm sorry


Russian propaganda means disputing U.S. propaganda. The Associated Press has published yet another article based on unevidence assertions by anonymous government officials about the scary, horrifying, frightening Russian menace this time to accuse another media outlet of promoting propaganda without a trace of irony. U.S. intelligence officials on Tuesday accused a conservative financial news website with a significant American readership of amplifying Kremlin propaganda and alleged five media outlets targeting Ukrainians have taken direction from Russian spies, AP reports. The officials said zero hedge, which has 1.2 million Twitter followers, published articles created by Moscow-controlled media that were then shared by outlets and people unaware of their nexus to Russian intelligence. The officials did not say whether they thought zero hedge knew of any links to spy agencies and did not allege direct links between the website and Russia. The officials briefed the Associated Press on the condition of anonymity to discuss sensitive intelligence sources, AP says. Repeating the refrain we're all familiar with now, which is always used to justify a complete absence of evidence or accountability for incendiary claims about government the United States doesn't like. You have to scroll down to the 11th paragraph for some examples of the Kremlin propaganda that zero hedge is accused of amplifying. These turn out to be entirely innocuous objections to Western imperialist narratives that were quote, written by people affiliated with the Strategic Culture Foundation and quote, an outlet the Biden administration sanctioned last year for allegedly taking part in Russia's interference in the 2020 U.S. election, allegedly. Recent articles listed as authored by the foundation and published by zero hedge include those with the headlines NATO sliding towards war against Russia in Ukraine, Americans need a conspiracy theory they can all agree on. And, theater of absurd, Pentagon demands Russia explain troops on Russian soil, writes AP's nomad merchant. You can read these three articles AP lists by clicking the hyperlinks linked in the description. Decide for yourself if these are sinister foreign psyops which require the urgent attention of the most powerful intelligence agencies in the world acting in concert with one of the most influential news agencies on earth. Or if they are not in fact relatively reasonable opinions which only stand out because they fall outside the overton window of what's considered acceptable debate in the tightly controlled spectrum of mainstream political discourse. For its own part, zero hedge says that this website has never worked, collaborated or cooperated with Russia nor are there any links to spy or any other agencies. The Associated Press offers no evidence or arguments to the contrary. Zero hedge happens to be one of the numerous outlets who periodically avail themselves of my open offer for anyone to republish my work free of charge if they want to. And what's funny is I was just the other day noticing the number of views they get when they do so and marveling that the US narrative control machine hasn't done a better job of reducing their audience. Now a few days later, here's the US intelligence cartel using its media mouthpieces to branded a Kremlin propaganda operation in what Electronic Intifada director Ali Obunima calls a direct government attack on free speech. I've disagreed with plenty of things I've seen published in Zero Hedge. I've also seen a lot of very useful information which is highly inconvenient for the US centralized empire. It's the exact kind of information that the empire has been working very hard to marginalize and minimize amid widespread public access to the internet. The US government is not actually worried about Russian propaganda. The US government is worried about people disrupting US propaganda. Every day we're offered new reasons why it's important to regulate what online materials are seen by people. From fake news clickbait to Russian propaganda to COVID misinformation to foreign trolls to domestic extremism to election security. And the only thing they all have in common is that the solution is a more tightly regulated internet. At a certain point you can only conclude that it is internet regulation itself that they are after. In the same way you conclude that a stranger kept offering you different reasons why you should let him hold your wallet that he's really just after your wallet. Whoever controls the narrative controls the world. The ability to control the thoughts people think about what's going on in their world is the only thing standing between the oppressive exploitative status quo and revolutionary change. The powerful understand this while the general public do not. There is nothing, literally nothing, they wouldn't be willing to do in order to ensure their control over dominant narratives. When they warn about Russian propaganda causing people to think wrong thoughts in their heads they are so close to admitting the truth that they're all acutely aware of. That human minds are very hackable and that this can be used to advance the interests of power. Protecting this narrative control mechanism is the first and foremost priority of our ruling power structures because it's what all their other control mechanisms are built on. That's a major part of why the media have been acting so strange these last few years and it's a major part of why everything seems so weird and why so many people feel like they're going a bit crazy when they have to try to understand the world. Right now we're looking at a race between humanity trying to awaken from the psychological cages that have been constructed for it and the designers of those cages trying to finish tightening those bolts and lock down our minds forever.


Hey guys, how are you today? We are here for art joy of sharing and for February and this month We're talking about water-based mediums and I love my watercolor paints if you've been watching my channel for any length of time You know that this is my favorite palette, which is my Daniel Smith palette. I love my watercolors They're my favorite water-based medium along with a few markers and crayons of which I have way too many I'm gonna just put that out there. I have some acrylic paint pens that are drying up, but that's a different subject I have this whole tray of sprays that I didn't even know I had some of which probably don't even work anymore a Sprays are nice and all and I do keep some around but They frequently like get clogged up. They're like the bane of my existence and most artists Feel the same way, you know, they're nice to have around but I find myself off more often than not Shaking the pigment up when it's settled and then just taking the top off and Just dripping it on things and using it that way and at some point. I think this year I'm doing something called project pan art supply style, which is a different video We're not gonna really talk about here, but at some point this year. I think I'm gonna have to use these up One of the things I wanted to take a look at is this box, which is crazy this box These are water-soluble crayons, mostly neo colors, but you can see I have some Dino wiggly scribble sticks. I've got some actually watercolor crayons. These are actual watercolors in a solid stick pigment These are probably my favorite because again, I love my watercolors I'm actually not a huge fan of the neo color twos So I think we're gonna play with these and we are going to see If I'm right in my assumption that the only ones I really want to keep are these and that the rest of them can go Maybe the Dino weight leaves, but I think the rest have to go. I think they have to go So we see so I'm gonna do some playing. I've got some scrap paper here. I Always am needing to make backgrounds and things What I want to see is how easy there they are to move around Because when I want to do that There's really only a couple colors of the neo colors that I ever use and if I keep any I may just keep those colors Like this dark purple, which I absolutely love this color But I like have every single color of neo color to crayon, which is stupid Don't do don't be me so We're gonna just take and the end so if you're gonna Invest in some watercolor or water-based mediums for your art Don't buy a whole set of something back when I first started I thought if you need a neo color, too, you had to buy the whole like line You don't need to do that find your local mom and pop art supply store They're the ones that are gonna have the neo color twos for instance You can just go buy one crayon or three crayons, whatever your budget will allow You don't have to buy the whole entire set Do that the same thing with like watercolor paints You don't have to buy a whole giant box of watercolor paints for God knows how much money You can just go buy one tube or two tubes your favorite colors the colors You think you're gonna use don't buy the whole set again. Don't be me. Okay, so this is These are all the neo colors and I don't know what color that is because it's way too small And I don't have my reading glasses on but it's a teally green color and this is a dark purple I think this is a black gray, dark brown something like that my problem with the neo color twos is always that They don't completely dissolve which drives me crazy. Okay. Here's a dino wakely media crayons. This is a their dark purple Let's see. Here's a similar oops That can't be good when the papers come out It's probably because it's old similar teally green and then We have Maybe this one like a dark grayish brown Okay, and then over here. These are Windsor Newton and Daniel Smith Actual watercolor their watercolors they're except it's a watercolor stick. So this is a dark purple and then Here's a teal It's cobalt teal and then Let's get a dark burnt umber, but you know, it would be so much easier with my glasses on. This is I don't know what this is. Oh, Prussian blue Widery black Windsor Newton Windsor green blue shade French ultramarine Lamp black moon glow Let's do Burnt umber So this one's Windsor Newton. These other ones are Daniel Smith And I have burnt umber and Daniel Smith too, and it looks like this So look how different the pigments are between the two I may even decide just to keep the Daniel Smith ones and like the dino wakely in a few neo colors So we'll see what happens. Let's get them wet now. Shall we? So this is your neo color. So with the Daniel Smith if I keep at it. It's going to completely dissolve With the neo color too, you can see that you always see the scratch marks And with the dino wakely it it works okay, but it's just not as pigmented But look at this. So if I keep at this that's nearly completely dissolved Okay, so here's this purple color And if I just like I said earlier if I decide to keep any neo colors This doesn't mean I need to keep the whole entire line of neo colors. What color that is Quedachron violet. This is the dark one. We'll bring this one this way This is the Windsor Newton one Which you can see is more like the Others where it doesn't completely dissolve Which might be okay in certain applications See I really I really like the Daniel Smith ones So I definitely want to keep those I think the Windsor Newton maybe has to go I think the dino wakely has to go and I think I'm just going to keep a few of the um neo colors in certain colors so Going forward for me in my art and my art Art supplies, I'm going to be doing more of this where I'm really looking at what I have Whether I bought it or was sent it in pr Do I really need to keep it? Do I really like it? If I don't like it? Why am I hanging on to it? I have just a lot of categories of this kind of product this month. It's going to be the water Soluble media and I want you to do the same thing. I want you to take a second look at what you have I mean, I could show you a art tutorial and how to you know, make a pretty watercolor painting I have some of those already on my channel but I want you to find which water soluble media you already have or Is out there that you really love and use that one I don't want you to collect every don't be me don't collect every single one on the planet And then because you think you have to have them all you don't have to have them all All right Okay, I'm going to edit this into the prior video clips somehow some way don't know exactly how yet Let me turn off the overhead light. There we go. So I Sad and went through more than just a few neo colors when we were on camera I went through a bunch of water soluble media crayon type things that I have in my stash I pick I didn't get rid of all of everything So for instance, they I'm getting rid of all of these neo colors I kept about a dozen crayons that are my favorite colors that I do use I most of these I've never used They've got to go.  I need this many colors I have a bunch of these other water soluble crayons. These are also water soluble and they're different ones some are recollections some are Sparkle gel crayons some are color peeps some are Rose art They work great too, but I have I again, I kept about a dozen. I don't need that many and I certainly don't need like five yellows The dino wake the media crayons we decided we talked about them. They're not my favorite the Daniel Smith watercolor sticks are totally my favorite. I love them There's nothing wrong with these or just not my favorite. They're not for me So they've got to go and the winds are newton watercolor sticks again, not my favorite So they've got to go too. I am going to be selling these I'll let you know in the description below whether they've sold or not if they haven't Make me an offer my email address is in the description below. I can't afford to just like these to just give them away anybody Google in or go into amazon and search neo color to water soluble crayons You'll see how much how expensive they are. They're expensive But I am willing to sell them and You can message me if you're interested and let me know and we can bargain we can discuss it and You know, I don't expect full retail. But yeah Anyway, um, don't be afraid to go through your supplies and get rid of Things that just don't serve anymore. That includes something that may be your favorite like for me water soluble media I love it my favorite is my Daniel Smith watercolors and I've got a few others that I love and I'm keeping those and the rest of it If I don't love it, it's not going to stay. There's no point in it clogging up my space All right, that's it for today. Go out and do those swatches Do those experiments try those materials and see what you like and you can use going forward in your art And what just has got to go because it's taking up too much room I know you're like me. I know there's something And I want you to go forward in I want you to let it go All right, that's it for today I have a lot of swatching and playing to do All right, don't forget to check out the video description for any relevant links go out and Try those materials if you want to do samples Here's one thought So if you don't have any watercolor paints and you're not sure if you're going to like them I know quarr which is by golden and daniel smith both have dot sample cards I know a lot of mom and pop art supply stores will have in the spring and summer They'll have demonstrations from the different art supply manufacturers and in those Demonstrations a lot of times they'll give free samples. So see if you can get yourself some samples to try some things Get together with some art friends.  Maybe you can do a trade Something like that also keep your eyes out. Of course at your Thrift stores and your art salvage stores, which seem to be popping up all over the country All right, I am going to go through these and see which ones i'm keeping and which ones are going away And I will catch you all in the next video go out and have a great day everybody Do something nice for yourself because you deserve it and I'll see you later. Bye guys


As part of an initiative held by the University of Kentucky's Healing Community Study, Narcan training will be taught to willing participants in hopes of combating opiate overdoses. Wednesday, Mountain Top spoke with prevention specialist Nathan Bartram about Narcan training and its importance. Just how to effectively use Narcan and pretty much, you know, make people understand this is a problem that don't just affect certain demographic that, you know, everybody knows somebody is struggling with substance use and we just try to get Narcan out there in the hands of people. You know, you never know when you're going to run across and overdose. Here at Pikeville Medical Center's Heart and Vascular Institute, we have assembled a comprehensive team of cardiac specialists bringing expertise from all regions of the nation and the world. We have coupled that with cutting-edge technology, providing them the best equipment and operating rooms available. The result is comprehensive cardiac care for the people of our region that is second to none. The Heart and Vascular Institute at Pikeville Medical Center. Bartram continues to say that Kentucky ranks in the top three for overdoses in the country and that cases have risen dramatically throughout the COVID-19 pandemic. Additionally, Bartram adds the dates of when they'll be holding their next event. On Friday, we'll be at the Bull Creek Trade Center down here in Prestonburg and then on Monday at the Martin Fire Department, we got an event that is open to the public. Anybody who wants to come in is interested in what we're doing. We can, you know, it's a real brief training. Training's around 10 to 15 minutes and we also got information on treatment options, you know, for people that want to be able to get into substance abuse treatment. And we got some locations for, we got seven locations around the county where you can safely dispose of unwanted prescriptions that way they don't fall in the hands of kids. Again, Narca and Training will take place at the Bull Creek Trade Center from 10 a.m. to 4 p.m. Friday and 10 p.m. at the Martin Fire Department Monday, November 22. For Mountain Top News, I'm Drill Horgel.


Hi, my name is Duncan McShig. I'm a PhD student in Trinity College in Dublin. I started as a law student, got dragged into the internet direction and now I'm hanging around for this conference for a couple of days. I mean, I'd like to say that, you know, this sort of stuff is the future of the internet actually, you know, thinking about it, talking about it, bringing people together from different backgrounds, not just the lawyers, not just the geeks, not just the geek lawyers or the lawyer geeks, not just the Americans, but you know, it's a, I think the future of the internet is really more connected to the future of what we're doing as nations and as societies and as activists more generally. I think it's a case of taking the internet out of its little corner and saying that the decisions we make over something like net neutrality, media regulation, privacy, relations with the developing world, relations with China, that sort of thing. There are questions that we're asking at this conference, but there are also questions that are of interest to people who've never gone to an internet studies conference or never got involved in internet or anything because these are fundamental questions and answering them I think is what we're here to do.


Hello and welcome to another edition of Frightfully Forgotten Horror Movies. Today we're going to be covering 1977's Orca the Killer Whale. Yeah. But before we get started, what are we drinking today? Just a mosaic IPA. Orca the Killer Whale was directed by Michael Anderson. He also did Logan's run. No! Don't go in there! You don't have to die! This movie stars Richard Harris, the legendary Hellraiser and Boozer. He's in so many damn movies. It's hard to pick a couple to mention but he was in Cromwell. He was in The Unforgiven. The last few movies he did in his lifetime were the first two Harry Potter movies where he played Dumbledore. Charlotte Rampling is also in this. She was in another horror movie, an anthology style movie from 1972 called Asylum. She was also in an episode of The Avengers. Yeah, The Superlative 7. That's a great episode. It's one of the best ones. And she was also in another great movie called Angelheart. So the movie starts out introduced to Captain Nolan and his crew and they're hunting great white sharks. There's this dingy with a couple of people on it. They're doing research too and they kind of get in the way. Just as they're ready to catch a great white shark. There's Orca Whale comes up and just headbutts the shark and throws it out in the water and everything. The sound is getting me.  Puts these two people in the dingy at risk. The woman that he rescued was Rachel and she's a marine biologist. And we see her doing a lecture at the university and she's talking about Orca Whales and how they're one of the smartest creatures on earth. They also have an extreme pension for revenge and family. So Nolan realizes that these Orca Whales are more money. So Nolan naturally wants to start hunting Orcas. He actually sets the sights on a whale. They harpoon the whale and they end up drawing the whale in onto the deck of the boat. The male that's in the water. The whale on the deck is a female and that it's pregnant. The fetus drops out of the whale onto the deck. Kevin Nolan is just horrified by the whole thing. Good God. Good God. You can't believe what's going on. I was horrified too when you were watching the movie and you were like whoa they went there.  Spray the fetus off the deck. Get it off the deck. Get it off the boat.   The deck hand goes out onto one of those boom arms and the whale all comes up and all eats him. They short work in this guy. God and Nolan is completely horrified. Again he's like my God. The male has had some funeral for the female and pushes it to shore and everyone is gathered around to see this poor Orca on the shore killed and Nolan's like oh my God what did I do. The male starts taking his revenge. He goes to the harbor and starts like headbutting and sinking all these ships. Except Nolan's ship. One of the other ship captains approaches Nolan and says hey yeah well we know you killed that Orca. There's the male over here. He's pissed. He's wrecking everything in the dock. You better do something about it. And Nolan at this point he kind of doesn't want to because he feels bad that he killed the female. He doesn't want to have to kill the male. Then the male Orca goes to like the refineries and starts headbutting all these pipes. Fats fire on this whole oil refineries. Explodes the Orcas like jumping up and down. So Nolan doesn't know what to do. He goes and sets up some sick scaricrow out. He starts to confide in Rachel. I had a wife and she was pregnant. She died in a car crash and I know exactly how this whale is feeling.  Then chief from one foo of the cuckoo's nest shows up. Juicy fruit. You have to do something here.  As sacred as the whale is, it can't destroy this community. He's got like his workers and his daughter staying with him. And the Orca starts headbutting all the stilts through like the house. The house starts collapsing and everything. His daughter is like falling down out as the Orca is like. Orca takes a chunk out of her leg. So his house is destroyed. All of his friends are being tormented here.   Yeah, you don't know. We don't know. If you want to find out what happens at the end of Orca, finish watching the movie. The first thing we have to address is the fact that you'd think that this is a Jaws ripoff. This movie was kind of coined as sort of the answer to Jaws. Nowhere near like Jaws at all. No, not at all. I was expecting a huge Jaws ripoff. It's anything but. Yeah, I was expecting a revenge story but from Captain Nolan like Richard Harris' character. It's from the whales standpoint. The whale is the good guy-ness, the whale is the protagonist, like you sympathize with the whale. Yeah.  Start to kind of hate Captain Nolan and sympathize with the whales. As it progresses and you kind of realize what Captain Nolan's history is and he had a wife and child that died.     You feel sorry for the whale at first and then later on you feel sorry for Nolan because he's got to do something he doesn't want to do. Yeah. And the whale actually becomes more of a villain as the movie progresses. It starts killing more people and destroying more things. The movie is a big circle, like it's a big wheel and it keeps going around and around. Exactly. And the whale is smart. It knows what it's doing.  It's like, come get me! I want you one on one! Yep. You know? And it does everything in its power to get it's way and it does get it's way. Exactly. And it's a really smart movie in that regard, right? Jaws is actually a lot more up front in your face. This movie there's so much more that's happening behind the scenes. More subject matter, it has more to do with the human condition and remorse and doing something you don't want to do and like it's yeah, there's a lot of emotion in this movie. So much emotion. And the character arcs for both Nolan and the whale. Because the thing is you don't really think of the whale as a character, but the whale is like the biggest character in the whole movie. Yep, exactly. And there's so much deeper than any of the characters in Jaws. The effects are fantastic and look so real like when I was watching this movie the first thing I thought was, I hope they're not harming real whales. That's what I thought. Please, please. And no they didn't, they're so good where they almost got shut down because everyone believed that they were using real whales and they weren't, they're all rubber or plastic. Practical effects all the way. The cinematography for this movie is amazing. To all of the ocean shots and the shots of the town and the night time shots are just so panoramic. Like you feel the expanse of the ocean. Whales jumping. Yeah, and it's like oh man, you feel the breadth of this. The music in this is fantastic and you know I was watching it and thought oh, this sounds familiar. It's very kind of western Ineo Morricone. Well how can you even get better than that? Exactly, the man himself. Yeah. It's fantastic. There's a great showdown at the end of the movie between the Orca and Nolan and who better to score a showdown than Ineo Morricone.  The music like it evokes so much emotion in you too, right, and you feel even more for the whales because of the music. The kills in this movie are fantastic. They're so fun, like the movie is just as silly and fun as it is like emotional and the kills have a lot to do with it.  Go get them, you know. They're rooting for the Orca. When Chief dies it's great because the Orca just like you'd think the Orca's just always going to eat everybody but no, the Orca's smarter than that. He'll like use things around him, headbutt things and make the boat go into the ice and then the iceberg. Yeah, that iceberg. He gets all crushed up by all that ice. Can you see his hand? And the ending too of this movie is completely not what you'd expect. The Orca leaves the boat way out to like the Antarctic and it's like ice and they're freezing and they've run out of gas at this point. I think there's no turning back. It's just so dismal and like. And Nolan knows that too and he's like well just run it out. How are you going to defeat this Orca at this point? You got no gas. Yeah. You got no gas. You got barely any manpower left. It's just you and the Orca and that's what the Orca wanted. Richard Harris kind of looks a little forlorn in this movie too.  And there's a very good reason for that because at the time when he was filming he learned that his wife was cheating on him and he wanted to actually leave the set and go and kill them both. Yeah. He's crazy.  Yeah. Oh Richard Harris.  Yep. It fools you. Fooled you. You definitely have to go and watch it. It's fantastic. It's such a fun ride. It's an emotional ride. It's funny. It's heartbreaking. It's hardwarming. It's everything. Yeah. All of the emotions. Yeah. So go check that movie out and until next time as Richard Harris would probably say keep drinking.