The Carnegie Mellon Quarantine Database talks are made possible by the Stephen Moy Foundation for Keeping It Real and by contributions from viewers like you. Thank you. So we're happy today to have Avi Kitty, the CTO and co-founder of CeliaDB. Prior to starting Celia, he worked on the KBM at Red Hat for a while and then they started CeliaDB in I think in 2013, that correct? 2012? Yeah around 2013. Okay and so we appreciate him for being up with us. He's currently in Israel so it's late night for him so we appreciate him spending time with us. Again we're sponsored by the Stephen Moy Foundation for Keeping It Real and we thank them for the support for these quarantine database talks and we'll do the same thing we do every week. We want this to be interactive as possible so if you have a question for Avi, just unmute yourself, say who you are and where you're coming from and then ask your question. Okay Avi, the floor is yours. Thanks for doing this. Okay thanks Andy. It's indeed almost midnight here so don't be alarmed if I turn into a pumpkin at the top of the hour. It's completely normal. So I'll talk about CeliaDB, no compromised performance. This will be a rather low level talk compared to the other talks that I've seen. I've sampled a few talks so this will be rather low level. I hope you'll enjoy it. So let's start. What is CeliaDB? There are about 8 million databases around so let's try to map it among all of the other options. So it's a distributed no-SQL database. The main use case is the online transaction processing so it does support analytics. CeliaDB are things like Spark or Presto but the main use cases are online transaction processing. It's compatible with several other databases so first is Apache Cassandra which was the inspiration and this is more or less the model that we followed with the Cassandra query language and protocol and also the Thrift protocol which is the other protocol that is used by Cassandra. It's also compatible with DynamoDB so we implement the same API which is JSON over HTTP and Cassandra and DynamoDB are also related in a way because both are based on an older database called Dynamo also by Amazon and it also has support for some of the verbs supported by Redis so only a small number because this is an infancy but it does show that we support multiple models and we plan to increase our compatibility along the way and add more interesting protocols. We have a very good performance so about up to 10 times on the same hardware with the same characteristics. What most users opt to is to have not exercise the full 10x but instead spend some of the headroom on reducing latency so typically you achieve about 5x performance but with better latency. You get very good latency and most of this talk will focus on the performance and latency and it's implemented in C++ and it's open source so you can find it on GitHub.  So the company was founded in around 2013 but the database was actually started a bit later so we started with C++ 14 and then updated to C++ 17 and C++ 20 when they were released. We're really excited about C++ 20 because it brings us support for coroutines which are really helpful for the models that we're implementing. I don't think we'll be able to touch into that but it is very interesting. We are very interested in coroutines as well but keep going. Yeah all right now so you support multiple consistency models. The first is the eventual using concurrent replicated data types that's an acronym that I guess many people just remember the acronym and not the actual words behind it and eventual consistency gives you very good performance especially with multiple data centers but it's not suitable for all kinds for every kind of use case. Some use cases like Internet of Things can use eventual consistency but other use cases need stronger consistency so for that we have lightweight transactions based on Paxos and we're also working on RAF support in order to improve the performance of our lightweight transactions and you can also have a mixed consistency model where within a data center you have strong consistency but the replication for other data centers is asynchronous so you get kind of a mix between the two models. It's aimed at multi-terabyte or even petabyte workloads so it's big data and also high performance so even the smallest deployments have tens of thousands of ops per second and the largest grow to millions of ops per second and we have demos of simple workloads that do a million ops per second per node so more complex workloads usually have lower performance but it does show that the raw throughput is very high.  So a very typical node is the Amazon i3 or i3n and those have the i3.metals has half a terabyte of RAM 36 cores or 72 vCPUs and 16 terabytes of disk and that is actually part of this slide we also aim for having a high disk to RAM ratio so a lot of modern databases or most of the open source databases really rely on on high caching so you need most of the data to be in memory cached in memory otherwise the performance starts to tank and this doesn't work well when you have a high amount of data because if you look at the price of RAM you want to keep a high ratio of disk to RAM because RAM prices are very high so if you have something like 100 terabytes that computes to a million dollars just for the RAM in the cluster so keeping a high disk to RAM ratio keeps the cost slow and we're also self-tuning and we'll have an example of that later on and the problem is with those databases is that they're complex and also the workloads are not constant they change with time and even if you manage to tune your system to a workload over time it changes and you have to keep retuning it and of course everything is in production so this is quite an effort. Okay so we have a symmetric architecture and this is more or less the same as the Cassandra so in this example we have three nodes and each node is divided into two layers the coordination layer and the storage layer or the replica layer and there is a full connection mesh the fact that we don't have errors connecting every client to every coordinator and every coordinator to every storage layer is just because I was lazy so the two layers are in the same process so when we have a coordinator talking to its own storage it doesn't use RPC or TCP it just uses a local procedure call and in fact we exploit this symmetry even more by using a thread per core architecture so in fact each node is internally divided into identical shards each of which contains its own coordination and storage layer which act more or less independently but they do share all of the resources in the node which is the memory and the CPU and and the disk so we need to both both we both try to share the resources in order to maximize the utilization but also we have to isolate them so that one of them does not dominate over the other and we have some mechanisms for that so we use a log structured merge tree I assume that you've studied this and you know what it is I'll just give a small overview of it so instead of having a data structure that is always sorted on disk like a b3 we keep the data as it is written in memory and we keep it sorted in memory using a balanced tree structure but when memory fills up we write it to a disk in a sorted way and that's called an ss table a sorted spring table and as time passes we get more of these ss tables being deposited on on disk and these ss tables can contain overrides or duplicate keys and these duplications are a performance problem because you need to read and merge all of those ss tables so you need to and also they occupy duplicate space so you need some process that merges those ss tables using a sequential merge scan and that process is called compaction and while this compaction is happening we're depositing more ss tables from the foreground process which is the writes so this gives us a foreground job which is the regular writes and the background job which tries to compact those ss tables and there are actually multiple levels of compaction because when you have this some ss table written to disk well you have to start the compacting ss table four and five and six as well and you will get another sort of some ss table and then you will need to compact compact those and the number of levels is proportional to the logarithm of the disk to memory ratio more or less it depends on a few more factors but that's sort of the the guide so you usually have maybe four or five levels of or tiers and this process causes a problem because it is a background job and those background jobs compete with the foreground jobs and you need to balance them if the background jobs consume too much too many resources disk bandwidth and compute and CPU compute power then your foreground processes suffer and you get a reduction in throughput but on the other hand if they don't get enough bandwidth then the data on disk becomes fragmented and your reads have to and reads that miss the cache have to read many ss tables and then your read performance suffers so you have to close the loop and make sure that these background jobs get exactly the correct amount of resources the correct amount of bandwidths in order to do their job so what are our goals so first we want to be efficient we're given a number of CPU cycles from from the hardware so a number of cores and we want to make sure that the cycles go towards useful work and not coordination or locking or things like that so this is one reason to use the c++ and not the language like java it it is a little bit faster although that's not the main reason another goal is to have good utilization so it's common to see on large machines that only a small number of the cores are busy and the rest are waiting on locks and the larger the machine the harder it is to utilize all of it it's also true for ior resources so disks are now very fast you can get millions of i operations per second but it's rare to see a program that can actually squeeze all of those iops from the disk so we want to make sure that we are able to use all of the cpu and with sila it's common to see the system running at 100% or if you count each vcpu at 70-200% and even more on larger machine and that's what would be setting off alarms in other databases is actually indication that the system is working as design and the last thing that we want to achieve is to have control to spend the cycles on the things that we want to do because we have multiple competing processes running on the system i mentioned the compaction a slide ago so we want to make sure that we can direct the the system to spend the number of cycles we want on compaction and the number of cycles we want on on serving requests or doing other things like maintaining nodes and performing repairs so a database that does OLTP is characterized by having a large number of very small operations so many queries can be just around a few hundred bytes so we want to make a coordination cheap we don't want to spend a lot of effort taking and releasing locks or exercising the cache coherency of the cpu we want to make everything very cheap and there are also lots of communications and the communication can happen within the machine so local procedure calls or inter thread calls or with the disk you read and write data from disk because we have a high disk to RAM ratio then we have to work with a significant percentage of cache misses we can't assume that everything is cached so we have a large amount of disk traffic and of course being a distributed database there is a large amount of communication with other machines usually if you have an application factor of two then writes require three round trips and reads require two round trips for a quorum so the solution to having lots of communication is to make everything asynchronous and since you're in a class then let's have a quick quiz does anyone recognize the object on the slide this is a SATA-based SSD so you're very brave but this was a trick question this is of course it would be a trick question this is a network device so those gold-plated connectors are actually a network terminal and although we are used to treating a disk as a synchronous device because the APIs are synchronous so you issue a read and the system called blocks until the data is there what happens under the covers is that the kernel prepares a message to the disk the same way that it would send the message to another computer send it over a kind of point-to-point link goes and does something else and switches to a different process and then the disk fetches the data sends it over the link and the kernel wakes up the thread that was asleep and and we go back to doing what you like but this is not an efficient way we wouldn't dream of doing that with a remote node so having like a thread per connection model everyone does networking asynchronously and we also do disk IO asynchronously so we treat the disk exactly as we would a remote computer and when we started this was relatively rare at least among open source databases but now with IOU ring and with the Go programming language I think it's becoming more common so let's have another quiz so what is this this time it's not a trick question is it an Intel Xeon processor e5 b4 product family yes it is it's a very good you utilized the but what does HCC stand for yeah I think core count high core count high core count if you if you were if you had said that it was a network device then I would say no it's a it's an Intel Xeon processor e5 or whatever but it actually does have a lot a processor is really a network of cores and instead of treating it in them in the way that most programs do which is to ignore that fact and just have a shared shared data structures that are protected with locks instead we try to treat each core as a separate node and have explicit communication between those nodes so we send messages between the nodes and this minimizes the amount of traffic on those internal networks and increases the scalability that we have within the machine so this is just as important as scalability across the cluster because it allows us to have a smaller cluster which is easier to manage so how do we do that we have one thread per core and that means that we must never block because if a thread blocks then that core will not have anything to run so we must whenever we do something that does IO any network call or any or any disk operation or any operation between cores we must be prepared to continue doing something else so we have a queue of things that we need to do and we never block on any operation and everything is asynchronous networking of course everyone does asynchronous networking file IO is more rare and we do that so instead of having a read call we have an operation to initiate a read from a file and asynchronous read and then we reap a completion later on when it becomes ready and meanwhile we go and do other things and also asynchronous multicore so if we need to access data that is owned by another core then we send the message to the other core go and do something else and that other core picks up the message performs the operation places the results on a point to point queue and we will pick up their results and continue so this is very different from general s&p programming but it's actually very similar to normal network programming distributed programming except that you can't have a failure so you send always send messages between course in the same way that you send messages between nodes and the fact that the database as a whole is distributed actually makes it easier because we use the same partitioning strategy that we use for nodes we use the same strategy for a course within the node so let's look at how the the programming stack looks and today there are actually intermediate stacks so for go and Erlang you have like intermediate models where you have the runtime doing some of this work for you but let's look at the traditional stack so you have a number of threads usually a large number of threads so that you can utilize all of the cores and some of those threads are running and some of the threads are sleeping and the kernel multiplexes those threads but the problem is that if you have a very lightweight operations which is the premise of the whole thing you have queries that return a few hundred bytes then you end up having a large number of context switches and a large effort to coordinate all of the threads and you also end up with problems where either you have too few threads to run in which case you don't utilize the machine or too many threads to run in which case your latency increases because you have a lot of contention and the kernel might not pick the thread that is most important for for you to run it might pick some other thread so our stack is with a thread per core it has its own scheduler so each its thread gets its own internal scheduler and it does a one-to-completion of small tasks so those tasks typically take a few microseconds to run and we can run around a million tasks per second per core and those those cores communicate via point-to-point queue so every pair of cores have a queue for sending a message requesting that core to do something and the queue for results and there is no sharing so the the network that lies within the processor has a lot less work to do and so everything goes much faster so let's let's look a little bit about let's talk a little bit about concurrency so I realize this is a computer science and not a math class so but I hope you'll forgive me so there is the little's law which says that the concurrency is the product of throughput and latency and that makes sense so if you increase the throughput and keeping latency the same then you need to do more things in parallel in order to compensate for that increased throughput and also if you increase the latency while keeping the throughput the same again you will need to increase the concurrency because each thing takes more time now let's do some math and look at look at how the throughput changes with the concurrency so you can see this is the same equation just transformed a little bit so you can see that in order to get higher throughput if your latency is the same you need higher concurrency and let's transform the equation again and look at how latency is a function of concurrency and here we see that latency is concurrency divided by throughput so if you want low latency you need low concurrency so the dilemma here is that you need both high concurrency in order to achieve high throughput and you also need low concurrency in order to achieve a low latency was there a question no it's when people leave keep going for good another problem with concurrency is that there are some lower bounds so disks want a minimum IO depth for in order to get the full throughput so think about the rate of rotating disks so there is a number of heads and let's say you have a rate of five disks so if the concurrency is less than five you are going to have one head that is idle so you need that at least five requests running in parallel and maybe even more in order to avoid the if you have some collisions so you need some kind of minimum IO depth for disks the same for remote nodes so there is network latency which you want to hide so you want to do some operations concurrently and also those remote nodes have their own minimum concurrency that they want so you have to supply that and for compute you need at least one operation running concurrently for each core that you have or you will have some idle cores so let's summarize we want high concurrency we also want low concurrency and we also want to be able to supply in order to fully utilize the system some minimum amount of concurrency so how do we solve this this conflict and the answer is is scheduling but that's actually not the right slide so I'll wait with that so what are the sources of concurrency so the first source is users they are clicking on their web pages and generating traffic but you're not always in control so if you have if indeed it's like a web application or then the source of concurrency is something that is a given it's not something that we can change we can add nodes as the operators of the database and this reduces the per node concurrency so you divide the same overall concurrency on a larger number of nodes but this is something that you might want to avoid and we also have the case that we have multiple workloads so you have say web users that are clicking their way through an application and also you have an analytics workload that is running concurrently and scanning the database so you have multiple kinds of requests that are running in parallel throughout the system and there are also internal processes to generate the concurrency so I mentioned the compaction before so compaction if you have a compaction process running it generates reads and writes and if you do read the heads you can also have a single compaction process generating multiple requests concurrently so the trick here is to have the internal processes generate a lot of concurrency in order to be able to utilize the system but also schedule those requests so choose which requests get to run when in order to limit the impact of concurrency on latency and this is how we do it so let's say that we decided that our storage can handle eight requests concurrently and later I'll show how we figure that out but you can imagine that we have a radar array that has eight disks each of which has one one movable head so and we have a bunch of inputs we have users that are reading from the database with a concurrency of say 30 and writing with a concurrency of 12 and we have an internal sources of concurrency compaction and screaming screaming refers to the process of starting another node so you need to move data from the existing nodes to the new nodes so it also generates a request and the idea is to push only to the storage only as many requests as it can handle without starting to back up and having congestion and this allows the scheduler to pick the right request to send to the storage so as soon as the storage completes one request we pick another request that is waiting on one of the queues and send it off and this allows us to have on the one hand have low latency because the storage is only handling the number of requests that it can without trouble and on the other hand we avoid having idle time in the storage because as soon as we have a slot free we feed it a request from one of the internal processes how sophisticated is like is like the schedule in terms of like like the what metadata are you exposing about the things in the queue you know i'm reading this file this offset and i'm and i have this priority do you go any deeper than that or or is it just so every request is tagged by the originating like the internal process that originated it so every week we know about every request whether it's a user read or a user write or whether it's a part of a compaction process and when we have multiple workloads we know which workload is doing that so we know this came from spark or this came from the web application and we know that the the spark should get lower priority than than the web application and those queues have an an amount of shares that are assigned to them and by the way i'm talking about io but the same thing holds for a cpu so every task that we run and recall that the task can be something that runs us in in a microsecond is tagged with the originating process so so and and we account for all of those tasks so we know exactly for each queue we know exactly how much we know it's execution history and we know how much shares we assign to the queue and we can then select the next task so that the the restrictions or the constraints on on the shares are satisfied and i hope that answers the question yes um okay so why don't we use the linux ios scheduler linux has a very capable set of ios schedulers but they have limitations so the first is that you can only communicate the priority by by the originating thread so you can assign a priority to different threads but in our case we have one thread doing doing everything so we have the same thread multiplexing compaction and user reads and writes and the spark reads and and everything else so everything gets mixed it will also do a reordering and and merging so it likes to send the request in and the order that you think is optimal for getting throughput but that's not good if it takes your the request that is latency sensitive and it puts it in the back because it thinks that's a that's more optimal so what we do is we disable merging and reordering and this gives us the control that we need in order to have a good latency response um so previously i've said that each disk has a it's kind of favorite concurrency it's actually a little bit more complicated oh and by the way you asked about what we track we also track the the size of the request and whether it's in the direction if it's a read or a write because this have a different response to reads and writes um so this graph shows the the response of a disk to concurrency so on the x-axis we have an increasing concurrency and by the way you shouldn't take high concurrency as good better actually a better disk would be one that achieves a high throughput with the low concurrency so and in blue we have the throughput so you can see that as the concurrency increases the throughput rises more or less linearly and then it begins to plateau and then the throughput stops increasing and what happens in the disk is it it starts to queue the request instead of serving them in parallel and in red you have the latency and if you squint then you will see that the response is opposite so on when the concurrency is low the latency should be more or less constant because when you feed a request a new request then it starts executing in parallel but as soon as you saturate the disk at around the 100 concurrent request the latency starts to increase it's hard to see it because of the error bars and because it's a noisy graph but this is a theory says that concurrency should be more or less constant at the beginning of the graph and more or less linear after we enter the congestion area so what we do is we run a benchmark when we install the system and we figure out how this what this graph looks like and that informs the scheduler the scheduler figures out where this sweet spot is and it only lets the disk process the amount of requests and it depends on on the size and the type of request as well that it can it requires for for good throughput and everything else it will hold in user space and the fact that to we keep those requests in user space allows us to select the next request that we run so this gives us the control and allows us to respond rapidly to user reads which want low latency and delay the batch operations like scans for spark or compactions or streaming or any of the other things that the database needs to do to maintain itself okay I talked about self-tuning and this kind of fell out of all of the schedulers that we have so we have a huge amount of control about what we do and this is a natural way to utilize this is to use feedback in order to change the controls so the way it works is that we have a bunch of queues so internal processes that feed queues of operations and the scheduler decides which which queue to consume so it can say we have a request coming on the query queue so the scheduler will pick a request from the query queue and let the disk process it and then it will might pick the query from a request from the compaction queue and so forth and it will select from those queues on the basis of the amount of shares that we assign to them but here comes the trick so we have a feedback loop how will we decide how many shares to assign to compaction so we look at the amount of work that is remaining for us to do in compaction we have a backlog monitor and it keeps track of all of the work that they still have to do and it adjusts the priority in order to try to keep that backlog stable and what this means is that if your backlog starts to increase which means that you're not doing enough compaction the number of shares that are assigned for compaction will increase and it will consume more resources from the system in order to to become stable and on the other hand if the backlog the backlog compaction backlog becomes lower then the amount of shares that the compaction needs will lower in turn and we will free those resources for other parts of the system we have the same thing for other components i'll skip over that because it's more of the same so this is an example of this in action so in green we have a request serve and we started a workload the right workload and it starts at the high rate and immediately it starts slowing down this is because when you start the workload it only talks to memory but after a while it starts have it has to flush to disk so it becomes more more intensive uh and in yellow you have the compaction shares and you see that in the beginning no share it's actually the CPU time so in the beginning there is very little backlog because there is no data on disk so the scheduler assigns very little CPU time for compaction but after a while it stabilizes on on some level um and it keeps us stable but in the middle we change the workload and now we're using a different size request which require different amount of processing and what happens is that the the schedule the backlog monitor notices that the backlog increases and it starts assigning more cpu for yeah i think that we we uh we're out of time if you if you want to continue then i'm happy i don't know if people have time and if not we can just move to questions although that was the really interesting bits yeah i mean yeah it is it is 12 30 so it's super late for you so i don't want to keep you up all night um so what is if you email if you email me my slides i'll be able to post them on on the website yeah but i guess i think i think i think some people have some basic questions there are some questions we want to go through first okay so go ahead with questions and i'll try to email the slides in uh in the background yes i have a question i i don't know if there's an ordering or no it's chaos go for it all right yeah i'm nick i'm calling in from the netherlands so here it's also uh it's actually 11 30 here in the evening but thank you avie for the talk um i have a question uh actually not what you talked about but i've been following uh cstar um and well it's clear that you also have your own tcp ip stack on top of dpdk and uh i i read some some papers that compare um a kernel bypass techniques and and and those papers also mentioned cstar and they met they mentioned then as a drawback like yeah this is a maintenance burden to to maintain such uh uh from scratch or written from scratch tps tcp ip stack uh if you compare it to just using say the linux stack or the or the host stack and my question is is that actually true is it a real burden like is tcp ip still changing and is it a lot of work to keep up with that or is it actually you write it once and once in a while you still have to maintain it but but it's it's okay okay so i suppose that if we invested effort in maintaining it then it would be a maintenance burden because even even though tcp doesn't change uh you still have to fix bugs but the truth is that we don't really use uh as a native stack uh in the end it turns out to be a deployment problem you have to detach the network card from linux and assign it to uh to to the database and and that is work that uh is not easy to do maybe today in cloud environments it's easier because everything is a lot more homogeneous you have just a few types of of systems but for like enterprise deployments in the end it turned out to be too difficult to use and we really did not invest much in it it's a pity because i really like it and uh you you can get the better performance and better distribution of compute across the course so i'm sad that we are not using it more extensively but the truth is that it's not a burden because we're not really using it as much as we should i see yeah i had a question so um i'm depend i'm an undergrad studying senior um i wanted to ask about uh how you use c plus was 20 code routines because uh we tried to implement that in our system but the problem we came across was that you can't suspend from a nested stack print right so um how do you guys go about making the execution model support stackless code routines uh what what i missed uh what you can do uh you can't suspend from a nested stack print so you can only like suspend from the top level of the function um so we know for stackless code routines you can you can suspend from any level so long as you return futures so our basic primitive is a future and all of our core routines return a future so every time you call a function that returns a future that is a suspension point uh and um i'm not sure exactly what where you saw the problem because for us it just fits naturally the the coroutine model works uh very naturally with their futures and promises so a coroutine is just a function that returns um a future and every time you have a core weight uh it's just uh every time you have you call another coroutine it's you usually call it with core weight uh which can be a suspension point if that function returns a future that is not ready okay i'm not sure that i understood the problem so maybe my answer wasn't very good so so the problem we saw was that you can only like return back to the thing that called it like you can't suspend from one function and go to like a completely like different coroutine right um so we have like multiple threads executing different things we execute one and then we want to switch over to something that's completely unrelated um were you able to achieve that with stackless code yes it's yes and it's completely natural so it's uh we had our system that used the continuations so future uh future dot then uh and we adapted it to use coroutines with very minimal changes so it's the same scheduler and it's the same underlying model um for us it worked very naturally maybe because the system already was based on continuations uh and uh every time you call a coroutine uh you you already have a suspension point so it's not like you're nested multiple levels in in the stack i would say to pine so lea db is open source so it might make sense to go look at their code and understand what they're saying okay yeah and feel free to ask questions on as a c-store mailing list how it works um we'll be happy to answer awesome thank you my name is jordi and i also have a question so this is more regarding stuff you might have done in the past i know you use i o u ring now for a lot of i o related asynchronous stuff but at any point did you experiment with using epol and our more synchronous apis and if so did you notice any impact on latency um so uh in the beginning uh we used the epol and for networking and linux aio for block i o uh and later we changed the linux aio to also support uh networking and this is what we do in preference we don't yet support i o u ring although it's like a an excellent match for the system uh we prototyped but in the end it didn't show any significant improvement uh in a way the linux aio support for epol which we implemented is uh really the forerunner of i o u ring and it's for epol the problem with epol is not that the interface is bad it's just that it requires a lot of system calls to maintain and it's not integrated so you have different apis to do different things and with i o u ring and or with linux aio you have one ring through which you send the request to listen on file descriptors or start the network request and one ring on which you get their responses so um you you save on a large number of system calls and it's all amortized so with one system call you a bunch up a number of file descriptors and a number of disk ios and did you see producing the system calls impacting latency or was that just like a throughput side effect but no the system calls affect throughput they don't affect latencies they're all non-blocking um of course uh if you exhaust the throughput capabilities of the system then you end up with the latency impact but the direct impact is the throughput okay thank you so as long as the system is not overloaded you get very similar latency as soon as it gets overloaded then of course q is built up and you get the high latency but the model of talking to the kernel doesn't directly impact latency thank you so much so my name is ling i'm a phd student here but thanks for the very interesting talk so my question is that i'm wondering how you decide the boundaries of those small tasks that you mentioned about is it just flows naturally or there's some like certain principles or guidelines you are following and also how do you ensure that the task is as small as you expected because i heard you you mentioned it could be just microseconds or some milliseconds right yeah so that's that's a good question and also a difficult one and we're struggling with it every day so some tasks really are very simple you launch an iO operation or you launch across those are the coordination tasks but the compute tasks are the ones that are difficult because they can take very long so for that we have a kind of user space tick and we use that by having another Linux AR ring that is running on a timer so whenever we have a completion for the timer whenever the timer fires the kernel queues the completion and when it queues the completion it modifies the indexes in the ring so and we compare the indexes to the ring to notice to note that we need to do preemption and all of our loops whenever we have a loop we also check for preemption and it's not every loop so if we loop on a small number of iterations then we don't really care but when you have a loop that is unbounded or can have a large number of iterations then we check for preemption and we break the loop and you will see that in a large large number of places in our code of course we don't directly check for for preemption we have primitives that do it for us but this is a source of latency whenever you have a computation that isn't that doesn't have this preemption check so indeed it's a it's a serious problem and we continuously find ways find places where we have this unbounded loops and and fix them we have a latency detector that sets a timer and when it expires it sends a signal and with a signal with a signal we get a backtrace of the loop that is consuming this CPU time without the latency check and our QA team sends us lots of nice bug reports with those backtraces and we figure out the source and then usually then it's an easy task to add the preemption check usually it's some data structure that we thought would would not grow too large but actually does and edge cases like if someone wants a database with thousands of tables whereas the usual number is a few dozen tables so in those edge cases we we get a high latency and because this is the cooperative system whenever you get that everyone feels it so the impact is pretty strong so this is one of the things to you have to worry about with this kind of cooperative architecture so on one hand you win in terms of performance because the coordination is cheap but on the other hand everyone has to be really cooperative and friendly or you get the latency spikes yeah thanks for the answer just quick follow up we'll do not make sure we'll say you do a preemption and then break up the loop do just to package up the remaining task of the loop and then req that into their whatever exactly so we repackage it so usually it means an allocation in order to save the state and package it in a task structure which gets queued in the task queue so this sounds like a huge amount of manual work but actually it's automatic if you're if you're working with core routines then or it's even the simpler and we also have user-level threads in which case the data is just on the stack so all of the automatic variables just stay on the stack and the thread gets resumed and we can have lots of those threads running of course we prefer not to have threads because they're expensive in terms of their memory footprint but we do use them when in places where it's simpler and where we don't have a large amount of concurrency okay that was awesome uh any last question for anybody else yeah i already went so it's a lot i go for nick go for another way um yeah uh so for i have a question about the internode communication for which in cstar you have this rpc class and you you do tcpip protocol tcp protocol uh did you ever consider like using udp with your own uh reliability uh on on top or do you never suffer from say the possible drawbacks of tcp in terms of latency especially if you have some packet loss we ever so i consider it almost once a week uh so i have like this routine where i think okay this is this sucks it really should be using udp and then i start thinking about all the mechanisms that we have to implement in order to have reliability and to package many small messages into one packet and also to fragment a large message into many many smaller packets because we have all all kinds of messages and to do the reliability and in the end it's just too much work for too little gain so tcp does have problems but it it it isn't worth all of the trouble of course the next week i forget all of that and i start thinking about it again but it's always the same the same result i'll alert you if i ever come up with a different uh but it is very tempting so uh and and i did implement in the past um rpc based on on udp and they do have advantages uh but right now it's not worth it for us maybe we should move to rdma because it's now becoming more um available on at least on on well in one of the advantage of the clouds is that they make um at the heart of more homogenous and there is a sort of minimal baseline that you can expect uh so maybe we will jump from udp to rdma okay awesome uh Len we'll give a quick question or yeah very quick one uh sorry a very quick question so we will mention that you are monitoring the cpu resource and performance as well i'm wondering how do you measure the cpu time or resource whatever like basically are you using the perf library or are you directly read out the performance counters from the cpu like how did you do that uh no so if you if you look at something like the rdtsc instruction you see it takes about the 20 odd cycles and for me that's too much uh so what we do is instead of measuring every task every individual task which would have way too much overhead uh we batch all of the tasks from the same q and uh uh we have a preemption timer every half millisecond and this way we we do the accounting of the granularity of half a millisecond so we let the q run and then when when it either completes or it preempts we measure it with uh with nanosecond granularity uh but we don't measure each individual task so we um the overall sharing is is fair between the qs or fair according to the number of shares but not every individual task so uh it's actually the same way that you do normal thread scheduling you you don't schedule every millisecond you schedule every some kind of time slice for us the time slice is half a millisecond and we just batch all of the tasks that the shares that have the same uh key is the same tag and it's simpler but we just keep all of those tasks in in uh in separate queues so we process the queue i see okay that's super jik all right i realized this one was 1 a.m i had to ask you this question because i'm asking everyone this uh how stupid are your are your users like how often are you surprised at them trying to use salia db in a way that like you know you never intended and it shouldn't be used or do you find that the users are coming to you having also maybe already been burned by your kassandra and looking for something better and therefore they're a bit more sophisticated uh so first our users are very smart for picking us so that's already a the after that they really can do no wrong but well you're right that many of our users do come from existing large deployments where they suffer latency problems and not just kassandra also mongo db so they they already they've already been burned they know where the problems are it takes some adjustments so even though we're completely compatible with kassandra there are still some things that are different like you need more connections and because you're talking to more shards we also have uh we can also connect directly to the shards so we have a specialized driver that knows to send the query directly to the shards that to the cpu core that will process it and it saves an internal hop and also improves the load balancing we we see mistakes so we we see people not using prepared statements or using data models that don't have enough partitions so they end up overloading a node and overloading a shard and actually our architecture is more vulnerable to this kind of mistakes because we although we will usually have many fewer nodes we will have many fewer uh a much larger number of cores doing processing and each core owns a subset of the data so you have to be even more careful about the data distribution um so people do mistakes we we also have some complete newcomers and they really have uh problems with basic data modeling which you would expect it's a specialized data model in order to go um be able to provide this kind of throughput you you need to make some trade-offs and without understanding those trade-offs it's hard to design the application so people do make mistakes uh and it's common we have a university course that is designed to ease them into it but of course uh people do sometimes keep on doing mistakes and we try to help them okay awesome i i immensely appreciate you saying up uh so late with us even given through other technical difficulties i will applaud on behalf of everyone else uh again in my opinion this is this is exactly like this this is extremely interesting and you're touching on a lot of things that as you saw a lot of my students had had questions for this so i appreciate you spending your time and letting us pick your brain on this


Okay and next we're heading into the reading room. It's whenever I really need to be focused and like in the zone I like to come here. So yeah we have all of these study desks and as you can see a lot of Ford students hard at work. So this is another part of the study room where there are study carols, there are armchairs and yeah and then now we are going to walk into the computer lab. We're getting to see a lot of Ford students at work today. So the computer lab right now there's not a class but there will be classes here. So like R for public policy, Stata for public policy, XL for public policy, data visualization and this is also like when there aren't classes students come in here to work on problem sets on especially for program evaluation or statistics.


Fantastic. Guys, love to welcome you back to the Independent Investor Channel. My name is Ryan. I'm going to rip through 10 life hacks that I feel like are holding a lot of people back. And this is part of the opportunity that I have through social media to speak with many, many people. And during my cross questioning and getting to know people, this is really what comes up the majority of the time. Any one of these 10 or all 10. And I think this is where a lot of people live. And I've shared with people here as of late the 16 things that I think people really need to focus on. This is kind of the dark side of those things that you should be focused on all the time. Unfortunately, people get consumed all the time and they get into this habitual focusing on things that don't matter. And these are the 10 things that are going to hold you back. Let's kick it off. Number one is addiction. Addiction can come in a lot of different forms. But I tell you what, a lot of people, man, they talk about their love for alcohol. They're for every Friday they're in the bar spending hundreds of dollars on it. Smoking cigarettes can be one of the most costly things you can do to your life. Set you back in way of years. It can be costly over the long term. Really, what are you doing while you're smoking? Well, you're not doing other things more productive. You're not exercising. You're not working. You're not looking to get ahead. You're taking that break out of your tough day, guys. I think a lot of people need to toughen up. A lot of people have chalked up smoking as a sign of weakness. I totally agree. And I don't mean to be judgmental out there, but God, guys, if you are involved in any type of addiction, break it. You can do it. It's taking away from all of the things that you need to be focused on. And by focusing in on those things, those addictions kind of dissolve away. You'll reflect back on the addiction if you do suffer from a little bit of that. And you'll wonder why you ever did that, because now, seemingly, you don't have enough time in the day to focus on those things that matter. And I think sometimes people resort to addiction to allow them to escape the very sheer reality that they are overwhelmed in their life. They are consumed by a lot of these things that don't matter and are failing to take a deliberate approach toward those things that do matter. Number two is overspending. Overspending in its broadest sense, man, when was the last time you shopped for insurance? I find this all the time in people's auto purchases. This is one of the fundamentals that I talk about in transportation you need, but how much do you need? Do you need a $75,000 truck? These are questions that are worth asking and challenging whether or not you can afford those things if you can. Great. If not, maybe you want to start at something a little bit more economical, something that can get you to and from work and then some, it can save you some gas a little bit. People's adherence to keeping up with the Joneses is sickening in this company. It's really holding a lot of people back and they feel like if they can show off to the neighbor or show off to family or show off to everybody else out there that somehow that speaks to a successful mindset. I'm here to tell you it doesn't. It's really unfortunate that people default to that type of mentality in this life and it's unfortunate with home purchases. Are you buying above your means? A lot of people think because they're approved for $675,000 that they need to go spend $675,000 on the dot. I'm here to tell you that you can find a home for less than $675,000. If you can't afford that half a million or even $250,000, look to live at least within your means. Dare I say below your means? I really wouldn't want to allow everybody else to have success while you're making sacrifices in your life. Here's the thing, guys. There's a flip side to why we avoid doing a lot of this. If we can avoid some of this overspending in our insurances, some of the memberships that we have. I'm not saying don't have a little bit of luxury, but what I find to be the most detrimental in people's lives is they look to consume all of their discretionary income with a lot of these expenditures until they're gone. And then you don't have any money left for things that matter. Wealth building, things that I talk about on the channel all the time, but the more we can avoid overspending for things, the more we can avoid. When was the last time you went to an outlet mall and spent your money on something that was 75% off? A lot of people, man, and I've made this example many times, I walked through Nordstrom's and I don't understand who's paying $300 for a sweatshirt. A lot of people out there would be like, Ryan, the image is everything, man. You really got to let people know that your image is everything. And it's like, I wear the same Nike shirt to prove a point, guys. I don't care. There's a lot of people who wouldn't be caught dead out in public showing that they wore the same garment twice over. It's faux pas. You just can't do that. People need to not give you the image.  It's all good.  And I say no a lot of the times. And if you can avoid overspending, it'd be a huge, huge plus for you. Number three on the list is one that you're going to love. This is one that gets people in a lot of trouble and a lot of people forgot 2009, 2010, the mortgage-backed security crisis, and that that home that they bought for $250,000 went up in value $100,000 or $200,000. And it's time to refinance. And it's time to take all that money back out of that, not knowing that the home may have a potential to actually maybe, I don't know, go down in value again.  I think refinancing with the fees nowadays, it's a bank's way amongst many of taking a lot of money from regular people who are unbeknownst to the power of investing and look at their house as a piggy bank or a savings account. It is not. It is not. What is wrong with just taking a 30-year or if you have the means of 15-year fixed mortgage and looking to pay that off as quickly as possible? And what people miss is they like the idea of refinancing and taking that $50,000 or $100,000 of equity. What they don't understand is that they've paid the house down for three or four or five, even in some cases, 10, 15 years. And that loan now resets back to 30 years. People miss that all the time. They're like, oh, this is great. I can just take the $100,000. Yeah, but you've just added you've doubled back to 30 years after paying the mortgage almost half off. Very few people end up not getting to the back half of the loan because they're stuck in this from 30 years to 15 years cycle of paying it down for a few years and then thinking all of a sudden that they've got to have that newest truck. They've got to have this newest home renovation project done. And the only way to go about doing that is to refinance the house and subject their refinancing opportunity to all of the fees, all of the expenditures, all of the expense that goes into writing those documents that get rolled right back into the loan. Guys, you end up paying for that in the long run anyway. And I think refinancing is one of those big ones that we fall victim to all the time. It's really unfortunate. If you can just stay the course with your mortgage, pay that one extra payment per year or more if you have the means, you're going to end up in such a better financial position than continually. And in some cases, I find habitual refinancing. Hey, it's time to go buy a new rig. Let's just go refinance the house. I don't understand that mentality. And it is absolutely one of those things that are holding you back. Number four, auto turnover. What was the last time you kept a vehicle five years? Ask yourself, I find it ironic how people think just because I don't know something I get a little bit of dirt on the grooves of my car and therefore I have to just fall on tilt and go buy a new one. I get, I'm getting tired of it, right? You know, it's dirty. So my solution is not to just go to the car wash and actually wash it and vacuum it out rather to take it in and let somebody else do that for me because I want a new vehicle. I've had the vehicle for two and a half years. It's time to get a new one. I never understood that mentality and this, this I will guarantee. I provide a lot of guarantees on the independent investor channel, which is ironic because it is a financial channel and I'm not supposed to talk about guarantees in the stock market, which I don't, but here's the thing with your vehicle. If you buy a vehicle, whether it be new or used, I can guarantee that you can get at least five years of use out of that with proper maintenance and with proper care. It's just that simple. It's going to free up a ton of income for you. And I think a lot of people, man, again with this habitual and a lot of these people, man, they duplicate this and their degree of lack of discipline because they deploy multiple things in this category that are going to hold you back. Man, it's planned failure right from the beginning. And then they enter into retirement and they wonder why they don't have two pennies to rub together.  It's amazing.  Over my career, I just turned in my one. I owned it over 15 years, 15, and I just treated myself to a new one. I really didn't want to, but there was a couple strategic reasons as to why I did that, but I didn't need to. But I have had two in my career, 17-year career. I've had two vehicles in my entire life. I've had four, my beginning car, my second car, five total, my apologies. I've had five total vehicles. I'm 43 years old. I've been driving ever since I was 16 years old. You guys do the math. Ask yourself, how many vehicles have you justified getting in and out of over the course of your life? But auto turnover is one of those that absolutely it competes.  It's my turn to go get the better car. And now it's my turn.  Now it's my turn to get the new car. Guys, it's a feudal activity. You're ended up running in quicksand and, you know, to each his own, great. You know, you get to roll around a depreciating asset every other year. And that's great for you. But I'd be willing to bet my portfolio is probably about, I don't know, 100 times bigger than yours. So put your loyalty where you want. Again, if you want to put your loyalty in a frame and a steel and a transmission and a depreciating asset that rolls down the road, then that's where your loyalty lies. Try to teach people a little bit of discipline. And if you can omit some of these things out of your life, you'd be that much more eligible to enter into a wealth building program. Number five is entertainment. Entertainment has no place in my hierarchy of needs that I've shared openly with the community. And this is a big one, man. It's crazy to me how many people, there was a long time where I didn't own an iPhone. And now with the business that I run, it's right next to me all the time. This is the Cadillac right here. Now I don't own a Cadillac nor do I own a Porsche, but this in and of itself is my luxury item that I can justify in a lot of different ways. It is a tax write-off for me because I do have a side hustle with the independent investor channel and I do make money with the independent investor channel. But for you guys that justify 200, $250 cell phone bills now, everybody has to have one. It's amazing to me. This is a real anchor around a lot of people's ankles. And for a lot of people who can justify it in the same manner that I do, it can add a lot of security to your life. It can add a lot of organization, even with the direction and mapkeeping, finding to get yourself to certain obligations here and there. It's a great thing. So if I'm going to yield one thing to you in the luxury category, it is going to be in the form of entertainment. But here's the thing. Do you need to go on vacation every month? Maybe you can come back a little bit. Do you need to go to the movies every single week and drop $50, $100? I see people at the movie theater. God forbid you just buy a ticket to go and watch the latest movie.  You know, my movie theater, they serve that stuff and I can guarantee they don't make the money off of the tickets that they sell. They make it off of thinking that you have to have, you know, the six foot long Laffy Taffy while you sit there and watch the movie. Entertainment, digital streaming services, man, how many channels do you need?  And I think a lot of people, man, they aspire to this. They have to have it. They have to always be doing something. I have to go. I have to do something rather than just pulling up a book and just reading for free rather than just doing what I do and go to the gym for free. It's one of the most enjoyable things that I do with my day. It costs me nothing. It's a benefit and a net positive to my health. It makes me feel good and it gives me spiritual positivity. It allows me to function throughout my day. Look at all those benefits. I spent nothing. I don't even, I don't pay for a gym membership, guys. Right? So if the entertainment category can be throttled back just a little bit, I think a lot of people give away the farm in this category. They've always got to be doing something. Again, it's show people that they're doing something. What did you do this weekend? I don't know. Same as every other weekend. I didn't do anything. I made some YouTube content and I went to the grocery store and I chilled out. I watched a couple of things on Netflix. Just chill out.  Well, we ran here and we ran there and there and here and there and we did this and we ran up to the here. We stayed at this hotel and this resort and this and that. It was like, it was only a two day weekends, man. Do you just kind of take a weekend off every now and then? I talk to people who they run weekends. It's not even a break. They just continually run, run, run. And it's like, man, chill out a little bit. See if you can make it through a weekend without spending money. Is that possible for some people? I would dare to say no. It is not. So challenge the status quo a little bit. Be different. And dare I say, be independent. Okay. All right. Number six on the list. This is an interesting one. And this is one that there is going to be consumers of this message that is absolutely guilty of this very thing. Restaurants. I don't mean to impugn the restaurant business, but I just took my son. We were on vacation in the Shenandoah Valley. As of late, it was a great vacation. It was free. I got a voucher for some poor service that I received at the last vacation that I got was which was very inexpensive too.  Just stay with me. I wanted to provide him the five guys experience.  Now, mind you, this was a double bacon cheeseburger.  He wants to get the big hamburger with fries and a milkshake. Have any idea how much that cost at five guys? Got any guests? It could cost under 20 or over 20. I'll give you a second. $28. Now, $28. And when he rang it up, I said, well, I'm just going to pay for you, buddy. I'm going to, I'm going to starve. I'm not going to eat here. It's bad for me. I had no intention, but $28 to feed a 10 year old. And we went to two other restaurants that same night. Couldn't even get in the door. It was an hour wait. And I was like, well, I'm not going to wait an hour to eat at Applebee's for crying out loud. Let's go. And the one the next morning, I decided to go down to Kroger and buy some groceries instead of wait for IHOP. I thought, gosh, if all restaurants we could get into and have a pancake or a, you know, an omelet or something, IHOP would be that place, right? Nope. Couldn't even get up there to check in anymore. Okay. But I think people's infatuation with restaurants is out of control. I do. And I think if you can curtail this a little bit, get your spending discipline in order a little bit, I think you guys would find that you would save a truckload of money. What's wrong with just looking at restaurants as an absolute luxury, which it is going to restaurants is fun. I enjoy it. I do. I love going to sushi, but I don't do it very often because it's very, very expensive. Look at it as a luxury. You know, if you want to kick out for a special occasion, a dinner out, an anniversary, a birthday, whatever it might be, great. And that can fine tune it down for you guys to a half a dozen trips to the restaurant every year. And a lot of people are like, what Ryan just said, I can only go to the restaurant six times in a year. A lot of you guys are like, I go six times in a week.   And the truth is you eat at restaurants way, way too often. Try to cut back a little bit. You'll be amazed at how much discretionary cash it'll open up. Number seven, clothing, personal care, haircuts. If you're a male, a lot easier to abide by this golden rule. I've had my wife cut my hair for the last 17 years. I cannot in all faith go in and provide $20, $21 plus tip. I'm in at $25 to do something that she can do for free. I invested in the cutters. They were probably, I don't know, $60, $80. Maybe a little more, maybe a hundred bucks. Let's say at Costco, they sell the kits. Boom, boom, boom. You rip it out. It's good to go. It's free of charge, guys. It's free. Okay. Some of these folks, and I get it, man, your personal care is your personal business. I get it. Okay. Now, if there's any room to improve upon those, do you need nails that are sticking out six feet from your hand? I don't know. I mean, I guess they're cool to each zone. To a lot of people, that's very, very important to them and gives them confidence. Right?  If you need that, great. Hair for women, I cannot believe that the cost, and this is why I say men, you got it easy. Women, much, much more difficult on the hair thing. I cannot believe what you guys have to pay for, for hair. It's amazing. $120 is just like the normal like, Hey, this is just the basic service.  I have colored my wife's hair for the last 17 years, me, and I'm good at it. I'm good. Want to know what I charge her? Zero. I charge her nothing. And I do a great job. It's consistent every single time I do it. It's however much it is for the color, to color the hair, I do a great job. How much have we saved on haircuts and hair color over the last 17 years? $48,000? I don't know. The idea is to deploy a little bit of discipline. And for you guys that say, look, I've got to have my nails that stick off my hand four feet. I've got to have my braids. I've got to have my weaves. Some of you ladies that have this done, it's oppressive. There's no doubt about it. I get it. And a lot of you guys not going to give that up. That's not the point. The point is to identify these things and say, you know what, this is a luxury that I don't want to live without. Maybe I can adjust a little bit in some of the other categories that Ryan talked about. I'm not looking to be judgmental. I think some of that stuff is great. Nice hair color, nice haircut looks great. I have to for my job. I get it. The intent here is not to judge the intent here is to identify that none of these things, none of them are essential for your survival. But I think a lot of people, they dwell in these categories and they manifest over these categories and they fail to identify the very things that do matter in life. And it's meant to draw a very specific distinction between the two categories, those that will hold you back in life and those that will accelerate your life. Okay, very, very important to separate those two and not combine the two. Identify when you're wasting money. It's okay, I do the same thing. Alright, I identify when okay, we're on vacation, we can splurge a little bit, I get it. But I don't stay in that habitual routine. I'm very, very disciplined in that I know when to retract and get back on my horse again on my discipline. Okay, very, very important. But clothing, personal care, haircuts, shoes, ladies, handbags. Do you need that much? I mean, I could pick on men and women just the same guys. It's those luxuries that maybe you can cut out one or two of the vices in your life. Okay, number eight. This is a good one.  If I did an eight minute video on the 10 things that'll hold you back, I'd probably do a lot better. But I like the open discussion because I'm looking to make you think hoarding hoarding is something that holds you back. What I can't understand in this country is storage facilities that make all kinds of money storing your crap. If you don't need it enough to keep it close to you and get some sort of passive or active benefit from said item that you have purchased at some point in your life, then why in the hell are you justify putting it into storage and especially paying for that storage?  And I think the habitual part about it is people are addicted to the need for things. They are. They just can't be happy with what they have. They're God given body, their ability to identify those things that are really, really important. And a lot of those things have really very little to do with things and a lot more to do with people, how you feel about yourself, how fit you are, how well you eat, how secure and how well organized your life is. Yeah, these are the things that matter. And I think hoarding, I never understood it. I used the example of driving just some time ago, it's driving through North Carolina, couldn't believe all the homes out there with six, eight cars. We got into a game of counting all the cars in the front of the driveway that were all broken down. They all have weeds growing off the tires. How many people out there need to justify that type of behavior? And again, I don't mean to be judgmental, but is it necessary? Maybe I'm missing something. Maybe I need eight, 10 cars in my driveway. And that's somehow going to be a signal of success to people. Maybe I could come onto YouTube and show my vehicles, none of which work. And my ability to grow grass out of the hubcaps of said vehicles is going to be a sign of success and people would come to the independent. They're like, yeah, right on, man, I want to be independent like this guy too. I want to have 10 broken down cars in my driveway hoarding, guys. Stop.  It's so overpriced. And we say yes to everything. Okay. There are hacks that you can enter into to pick up some of that stuff. The furniture that I'm looking at right now, I picked off a craigslist seven years ago. I love it.  And I like it a lot. So are the end tables that I bought at the very same time from somebody who was looking to sell that because they identified that they didn't need it in their lives. So they were looking to offload to somebody and I needed a dining room table. So I got it.  So go figure hoarding. If you can look to avoid that, it's the best thing. Number nine, this is an interesting one, learning proper sleep discipline. It can hold you back. When was the last time you got up at, I'll be fair, six o'clock, I got up at five this morning. This is a weekend for me. And I get up early every single day now. I don't sleep in. It makes me feel lethargic. It makes me feel terrible for the rest of the day. I cannot feel engaged. I cannot feel energized. I do not feel like myself. I do not feel like the person that I want to be without getting my butt out of bed. It forces me to go to bed a little bit early at night. When was the last time? What's your bedtime? You stay up till 12, one o'clock in the morning and expect yourself to get up at seven o'clock in the morning, six o'clock and perform like a Ferrari. Maybe this is holding you back. Maybe you're not achieving your maximum benefit by not learning proper sleep routine. Now for me, if I go to sleep by 10 o'clock and I can wake up at five every single morning, that's plenty of sleep for me. Okay, it's plenty. That's about seven hours of sleep. That's what I need. It's my optimal number. What's yours? What's your optimal number to perform at that optimal level? And a lot of people think that they need to sleep in or they need to catch up on sleep. That's a fallacy. You don't. When you get up in the morning, it should be a little bit painful. Okay? And that, yeah, you want to turn over and you want to fall back asleep again. But when you wake up, you should actually feel like there's a little bit of barrier to get over.  I love to sleep in until 11. I put the shades over my eyes, you know, hey, again, to each his own man, this is aimed at providing one small cornerstone to your ability in your holistic application to start to achieve a program that is going to give you the most eligibility towards success in your life and sleep discipline is a big one. Learn your number, learn what it is that you can take and challenge yourself a little bit. If sleeping until 11 o'clock is in your best interest on the weekend, great. Try my program. Try waking up early on the weekend and see how much you can get done by 11 o'clock in the morning. Pretty amazing. I don't have enough time in my day to waste time like that, sleeping in and wasting my time like that. And I feel better by getting an early start. That's just me. You got to find your, your niche in finding that proper sleep discipline. Now, number 10, finally, finally, number 10, what's holding people back? What's holding you back? I've just identified, this will be the 10th thing that's holding you back. People's inability to acknowledge in their wealth building program, the opportunity that's made possible through tax protection of your money. You can do away with some of these vices. You get a great job. You're making some surplus capital. If you fail to acknowledge how important it is to tax protect your money, you're earning that money. Now you want to shelter it as much as you can from the government. You do not want to work really, really hard to accumulate a lot of wealth, only to be stuck with a tax obligation to the government at some point. This is huge. A lot of people fall short of this and it will never be perfect in society because a lot of people think that because they just contribute to a 401k that everything is going to be hunky dory, not all 401ks are created equal guys. If you're on the hook for a tax obligation into your future, why don't you try buffering with a little bit of an individual Roth IRA account type of application to shelter some of that money for you as an individual or you and your spouse as a married couple, right? In your household, $12,000 per year, you can tax protect your money. The last bonus thing in the category here is wealth preservation. Wealth preservation is the ability to tax protect, yes, but to take your hard earned dollars and put it to work in a matter that liberates those dollars into the future, not to be subject but to fees. I'm speaking exclusively to the financial planning institutions that have over time identified a way through their gorilla math to affix a 3% fee over your money. It's an anchor that holds you back. If you identify the power of wealth preservation over time by the elimination of fees investing in those products that are low cost or no cost to entry, then you have found your way to a much better place by protecting your money and liberating your dollars in the proper account. Those two things, people give away the farm all the time. They go in, they subject their money to high fee products in accounts that do not provide any type of tax protection and you're really giving your way away hundreds of thousands of dollars. I saved the best for last. This right here is one of those things to where, yeah, if you invest deliberately over a 40 year time horizon, you're going to have a few hundred thousand dollars. There's no doubt. Some people would look at that and say, wow, you're rich. Well, it's the amount of money that you have surrendered to fees over the course of that four decades. That's really the jarring type of figure. That $400,000 looks minuscule to the comparison figure of the potential over that same amount of money over that same time period of over a million dollars in that portfolio. Guys, I hope you appreciated this. 10 things that'll hold you back. Really appreciate very, very important for you to draw that distinction between these 10 things that'll hold you back. And then you ask, where does the focus need to be, Ryan? The focus needs to be on those attributes that you identify in your hierarchy of needs. That is your real house. That's what you should be spending the majority of your time on and really revolving on and spending the majority of your time focused on and looking to reduce some of these things that'll hold you back. Guys, if you appreciate the message to make sure and subscribe to the channel, make sure and leave your comments at the bottom. If I've missed something in things that will hold you back, leave them in the comment section of the video, share the message with anybody out there, hit the notification bell, you'll be notified of future videos that I put on YouTube. Guys, thank you so much for tuning in to the message here and good luck in your investment future.


Igor Girkin. Russia would not survive another presidential term by Vladimir Putin. Igor Girkin, the former Russian commander, has said that Russia would not survive another presidential term by Vladimir Putin. In a post, Girkin changed tack from criticising Putin's military decisions to turning to the issue of his leadership of the country in general according to Newsweek. History has no subjunctive mood, his post read, suggesting that there is no uncertainty or doubt about the time that has passed since Putin first took power in 2000. For 23 years, a non-entity was at the head of the country who managed to pull the wool over the eyes of a significant part of the population. Girkin wrote, Now he is the last island of legitimacy and stability of the state. To shackle it illegally means to successfully complete the work of those who placed the unique advantage on the throne in due time. But the country will not survive another six years in power of this cowardly mediocrity. And the only thing that he could do at the end of useful things is to ensure the transfer of power to someone truly capable and responsible.  And if it does happen, then we have already seen many times before his ability to choose associates. He wrote, He expressed his conviction that a weak but intelligent king is inclined to find himself an intelligent and strong-willed chancellor to rule as long as he reigns. Historians full of examples of such successful tandems. He then warned of what might happen in the event that Putin stands again for president in 2024. The Russian constitution previously only allowed for two consecutive four-year presidential terms. However, following constitutional changes approved by Russia's parliament, the Duma, Putin can stand for two more six-year terms, potentially meaning that he could retain power until 2036.  The only useful thing he could do before the curtain falls is to ensure a transfer of power to someone truly capable and responsible.


Hello, I'm David Martin and in this series of short videos I've been explaining the principles of automated zone design. And in this video I'm going to look at some real world examples where automated zone design techniques have been applied in the creation of zones for which social science data are readily available. Since the census in 2001 in England and Wales, the automated zone design techniques that we've been talking about have been used to underpin the creation of quite a wide range of official statistical areas. And this commenced with 2001 census output areas and then the aggregation of those in 2004 into a set of units called lower layer super output areas, which I'll explain some more about in a minute. The whole system was revised for the 2011 census to produce output areas and another set, an updated set of lower layer super output areas, and then a completely new geography for 2011 known as workplace zones. And those are all available covering England and Wales. The same approach was used to create the Northern Ireland 2001 output areas and 2011 small areas. And at April 2016, the time of recording, we're shortly expecting the publication of a set of UK-wide workplace zones, including Scotland and Northern Ireland, which have been in production over the last few months. For users of the data, it's important to recognise that there are some common features to all of these geographical units. And in particular, the original output area creation has had quite a strong influence on all of the subsequent zone geographies. The original idea was to separate the output areas from the geography of enumeration in 2001. And the underlying zone design is based on a set of postcode or sometimes postcode polygons, which have been built up from tees and polygons around addresses. And they include some ancillary information such as major rivers, roads and railways, but they're not highly detailed in the sense that they don't follow property boundaries, they don't follow every minor road. And so those features will not be reflected in the placement of the boundaries of these statistical units on the map. And the nature of the tees and polygons and the fact that these polygons have to cover the whole land surface can mean that the boundaries are quite irregular and quite spiky looking on the map, particularly in areas where there's very little population, so in remote rural areas. And these are features which are common to the entire data set. The output areas provide the foundation for two layers, lower and middle layers of super output areas, and also the workplace zones. So the output area creation came first each time around, and therefore it influences the way in which all those small area census data have subsequently been aggregated and published. And everything in this hierarchy nests within the local authority district boundaries which were in place at the time of their creation. So if we start briefly by thinking about those 2001 census output areas, they were built from synthetic postcode polygons and they were constrained within walled and parish boundaries which were present at the time. They had a minimum population threshold of 100 people and 40 households and a target size of 125 households. And the shape was controlled using an accessibility statistic which was intended to keep settlements together such that in rural areas we wouldn't split one settlement into a large area of low density land and another part of the same settlement going off into another adjacent area, but rather to keep zones tightly packed where possible around those small settlements. And there's also a metric included which made it unattractive to combine urban with rural postcodes and again the goal there was to provide a clear demarcation of urban areas. The modularity in this original design was controlled using an inter-area correlation statistic and it was based on the tenure and the dwelling type classifications from the 2001 census. And the whole zone nation was implemented after the census had been collected and before the data were published. And it meant that there were no zones in the output which had to be suppressed because the populations were too small. Effectively every zone was designed to meet the census confidentiality criteria. What happened in 2011 with the boundaries which are currently available is that an updated set of areas was created and that included a review process of the previous boundaries and some manual adjustments if the 2001 boundaries were found to be unsatisfactory in very specific circumstances. And it was also necessary to incorporate some imposed local authority boundary changes where the boundaries had themselves broken the original output area geography. But only 2.6% of the output areas have actually changed between the two censuses so we have a very strong degree of continuity. Where change has occurred it's mostly been due to the need to subdivide zones where the populations had become too large and too large in this sense means over 625 mostly due to population growth in the decade between the censuses. And there would in some instances be aggregation of zones where the populations have become too small and that's much rarer. The same automated zone design procedures were reapplied in the situations where subdivision was required and that was done using the AZ tool software which has been described in a previous video. One of the 2001 constraints to the warden parish boundaries was dropped in 2011 in order to maintain the new statistical geography system and that results in 171,372 of these output areas in 2011. One further point of explanation is that the lower layer super output areas which were originally created for the government's indices of deprivation in 2004 are the result of taking the output areas and putting them back into the beginning of the zone design process and then running the whole process again. So the output areas become the building blocks in this second zone design problem. And here the thresholds are larger, there are 1000 persons or 400 households because the data which are to be made available are potentially much more detailed and sensitive and so the confidentiality threshold is higher. The 2001 output areas were the building blocks for the 2001 super output areas which actually were published in 2004 and then in 2011 the 2011 output areas were the building blocks for the new 2011 super output areas. In each case they're constrained within the local authority boundaries and typically the original populations had a mean of around 1500 so they would often comprise 5 output areas combined. There were 34,753 in England and Wales in 2001 and then in the New Zone Nation that's gone down slightly to 32,844 in 2011. And users of the statistics may note that there is also a zone called a middle layer super output area. That's an aggregation of these lower layers but it's not been produced as a result of a direct automated process. It was much more a process of consultation which led to the creation of those zones. So in the map that we have here we're seeing for an area of East London the super output areas from 2011. The statistic which is being mapped is actually a shaded set of deciles from the indices of deprivation in the 2015 version. And we can now superimpose on there the output area boundaries which allows you to see the difference in scale between the very detailed output areas and the less detailed super output areas lower layer. The same approach has been used in quite different ways to create workplace zones. So following the 2011 census the Office for National Statistics recognised a demand for detailed data on place of work and the workforce which are not really well suited by output areas based on residential geographies. And so using the information which comes from people's answers to the census questions about where they work the same data have been completely reconfigured to create a set of workplace zones. And these have a minimum of 200 workers and three workplace post codes as their basic population thresholds. In some cases they'll be the same as the 2011 output areas if there are neighbourhoods where similar numbers of people live and work. But it's very often the case that in residential areas the workplace zones are aggregations of output areas. And in city centres and business districts where many people are working they are subdivisions of the output areas. And so we're adjusting the trade-off between the number of statistical units that are present in this case the workers and the size of the zones. And in each case if subdivision has needed to be performed it's based on the same automated design design process using synthetic postcode polygons and these are the post codes which were described as workplaces in the census. The workplace zone geography is understandably a slightly larger set of zones than the output areas and there's 53,000 or so of them in England and Wales. And in this map from the Data Shine website we see the same area that we've used in the previous illustrations but this time for the workplace zones. What's notable here is that unlike the output areas the smallest zones are those over on the left hand side in the west of the map in the city of London. Because very large numbers of people are working, very high density small areas so we have a fine subdivision of the geographical space. And across to the right in the east which are mostly residential areas and some low density industry and the zones are much larger because there are smaller numbers of people working in those areas. And this particular map shows a classification of those workplace zones into different industry and worker types. So in summary we see that the automated design processes which we've been talking around in these videos have underpinned all of the census zones used in England and Wales in 2001 and 2011 and also appear in some of the census outputs in Northern Ireland and in Scotland particularly the workplace geographies for 2011. So all research users of the small area census data need to understand the implications of those zone design processes on the data which they are using. In particular we've seen that the design has a strong bearing on the relationship to other geographical units. These zones will all match perfectly to local government geography. They will match quite closely to the small area postal geography but there may be other units of interest for research purposes. For example, electoral geographies to which this will not have a very close association, at least not in detail. The exact placement of those boundaries and the shapes which we see within them are going to be an artefact of the road centre lines, the railways, the rivers, the features which have been included in the building block geographies and that explains some of the patterns which we see in small area census mapping. And it's the relationship between the zones and the underlying population geography which determines the size and shapes of the zones. They will be very much larger in rural areas because they are aimed to target the same population size so we've got to go much further to find the same number of people. And in really understanding what the implications would be for any statistical analysis of the census aggregate data and for investigation of ecological relationships in those data the researcher needs to be aware of the thresholds that have been used to preserve confidentiality, the size ranges which are allowable and the way in which social homogeneity, both on tenure and dwelling type, has been used to influence those own sizes and shapes because that may well have a bearing on any analyses which are going to use those same variables. And finally we might note that researchers who are wanting to use these data could consider running alternative zone designs at the same scales to understand to what extent the relationships which they see in their data would be preserved under different configurations, effectively different aggregations at the same scale.


What happens if this term is not defined under the constitution? It is left, it is an issue. I don't want to define this conception, But compelling reasons put it simply means a convincing argument. The work of the Africans is to put forth a convincing argument.


So look, I'm a bit nervous about this video, but I don't know why. Am I the only one? When it's the day before my wash day, I go to YouTube and I'm like, detangle natural hair. Quit detangle. So I've been seeing this one particular video from Natural 85 Whitney and it is 25 minutes detangler and co-wash and I'm like, girl, angle half angle half. So I said, you know what Vivian? Let's go ahead and try and see what happens. You know, if anything, we'll just try it. I can say I tried it.  So I'm going to be trying her method on extremely, I wouldn't say extremely, a little bit matted. Washingo. This washingo is seven days old. Yes. So I'm going to show you guys what I'm going to be using. I'm going to be starting off using warm water with a little bit of apple cider vinegar, probably one tablespoon. Okay. And then for my conditioners, let me back up. What she does is she applies coconut, coconut oil, excuse me, in a great condition with slip onto her hair. Allow that to marinate. Then she jumps into the shower with the same conditioner or another one and proceeds to finger detangle in the shower in 25 minutes or less. So we're going to see girl. For my conditioner, we're going to be using a combination of the VO5 strawberries and cream. I know that's focusing. And I have just a little bit of swabs coconut conditioner here. Of course I have coconut oil. All right. Now when she's in the shower, once she's done rinsing out the product, she uses aloe vera juice. I'm not going to do that. I'm going to show you what I'm going to use. For my co-wash, I'm going to be using swabs, almond and shea butter. This is bae. This is bae. This has the best slip ever. I probably will kill half of this, if not all of it. Once I'm done with that, I'm going to be applying apple juice, keratin and green tea reconstructor. And for my leave in, I'm going to be using the Taliyah Wajid, the curls detangler. That's going to be my leave in. Even though it's a detangler, it's a great leave in. Now I'm still debating whether or not I want to use my rice water. This is a jar of rice water that's been fermenting for by three days. This is orange filling here. This is jasmine organic rice. I'm not sure whether or not I want to use it today. All right, you guys, let me go ahead and get some warm water. Put in the apple cider vinegar and we're going to go ahead and get started. All right. So I have one more to here. I'm going to put in some apple cider vinegar. I've been not being a shower. What was that? I think there was a morning sign. Not to do it, girl. All right, you guys. So I'm just going to take my hair down. Like I said, this is a very old wash and go using the wet line extreme gel and the melanin haircare. I'm going to show you what the shrinkage is like right now. My hair is at broadstrap length. Yeah. So I'm just going to go ahead and spray my hair all over with this warm water apple cider vinegar mixture. And then I'm going to go ahead and section up my hair. Okay. Okay. So lightly misting my hair and the first thing I apply is the conditioner and then I lather on the coconut oil and I'm going to be very heavy handed you guys because I want my hair basically almost saturated with both products. Okay. Okay. So I'm just about done and just going to finish up this section right here. This took me eight minutes to do and I left the pre-poo on for two hours in the shower. Look at my face. Yeah. This felt heavenly rinsing out. However, I don't know you guys, my hair kind of felt funny with the gel and putting that coconut oil on it. I'm using a comb to lightly detangle my hair. It took quite a bit of time to rinse out the pre-poo longer than it should have in my opinion. Okay. So most of the pre-poo is out. I'm going to go ahead and dump in on that rice water mixture and then go right in with the almond shea butter by Swab. Okay. And I'm going to use almost all of it. Yeah. Almost all like 75% of the jar giving you a little hair point. You guys, my hair feels amazing. I absolutely love this particular product by Swab, the conditioner at least. I will say my camera stopped and my camera stops at the 16 minute mark. So I've already been in a shower for almost 20 minutes at this point, but I'm going to continue rinsing out the conditioner aka co-wash and we're going to finish up. Again, this is the Apogee Carrington Reconstructor. Next, I'm going to apply the leave-in. I'm going to use maybe, I have no idea where my camera left off at. I was adding my leave-in conditioner in my oil. My camera stops at 16 minutes and I do apologize. My battery went dead while I was recording. I did end up applying. What I was trying to say is that I normally do apply my leave-in conditioner and my oil in the shower. I don't use a lot because I style my hair two days later. Let me move on. So it was a little interesting, you guys. You saw my face. I'm animated. I know. I have not used coconut oil exclusively by itself in years. I typically use an oil mixture. So I felt like my hair was like, girl, what are you doing? We don't know if you like this. I love it. So it took a while to rinse out the coconut oil with the conditioner, basically the pre-poo, before I was able to actually co-wash my hair. Overall, I did I do it in 25 minutes or less. It was 35 minutes. So I don't know, you guys, considering that I left the pre-poo on for two hours, like I stated, those, that two hours, I could have been finger-detangling my hair, jumping a shower. I could have shampooed and conditioned my hair with the amount of time it took me to actually rinse out the pre-poo and co-wash my hair because I don't know. You have to keep in mind, too, I had gel in my hair. I use the extreme wet line drip or wet line extreme gel and so yeah. I would definitely recommend trying this if you don't have as much hair as I do. Now, I know Whitney has a lot of hair. She has a lot of hair and she has long hair. Those of us in the fourth category with wash and goes, we all know it could be a bit challenging when you detangle, which is why normally I do, regardless of finger-detangling, detangle, but especially with a wash and go, I take my time to finger-detangle. I would try this again on a looser hair, meaning an old twist out. I would not use coconut oil the next time. I would use my normal oil mixture, which consists of coconut oil, but mostly sweet almond oil. But I say give it a try, see if it works. That's it, you guys. Thank you so much for watching. Take care. Bye.


Well, when Ascension Health St. Joseph's Medical Center in Brainerd transformed it was transformed today into a shopping center for one of their biggest fundraisers of the year. Our Anthony Scott has more on the event. The hospital isn't a place you usually associate shopping with, but in Brainerd many community members were checking gifts off their list at Ascension Health's Holiday Bazaar. It's changed over the years, but right now it's different crafters coming in selling their wares. Some are like employees that sell things that, you know, they pay so much for a booth and then they get to sell their wares and everybody else gets to know what's kind of going on. The Holiday Bazaar has been going on for roughly 50 years and when it first started, members of Ascension Health's auxiliary would make their own crafts and sell them, but now vendors buy space to sell their items. I've seen this quite a few times and I came into the hospital and saw the posters. I thought I would really like to be a part of that and support that and I was very blessed that they let me come in. Men and women of the auxiliary were busy making cookies for the events annual cookie walk. We have a huge cookie walk. Our volunteers provide all of our cookies. We had 3,500 cookies at least. We were swamped at first. It's quieting down a little, but we were swamped at first, so. The event is one of the biggest fundraisers of the year for Ascension Health, with all the proceeds supporting the auxiliary which purchases equipment for the hospital. Reporting from Brainerd, Anthony Scott, Lakeland News. Last year, the Holiday Bazaar raised about $10,000 for the hospital. If you enjoyed this segment of Lakeland News, please consider making a tax-deductible contribution to Lakeland PBS.