So let's show this one, sketch the interactive network of water molecules in the liquid state. This is just wanting to show the hydrogen bonding between the water molecules. So you've got to know where the polarity of the water molecules lies. So when we draw a water molecule, just like on the test, it looks like this. So we know that in water, this is delta positive and this is delta negative. So delta negative is there, delta positive is there. And the delta negative goes with the next one, the delta positive. It's going to make a hydrogen bond. So all this is telling us to do is draw another water molecule. This little water looks like this. And the liquid state. We can just keep going if you want to, let's do one more. So hydrogen bonding is the cause of the inter-force line? That's the inter-molecular force and water. Yeah, so a hydrogen bond is the type of inter-


Okay, let's get started then. So first of all, thank you for coming over here. So it's the last session of the day. So let's try to keep it interesting. So let's talk a little bit about Ceph. Since you're all here, well, probably some of you have tried in the past to deploy Ceph in a highly-performance cluster and with a bit of luck, you might have succeeded. But most of you either have tried to deploy a highly-performance Ceph cluster and have failed or are only thinking about deploying a Ceph cluster and you probably don't know that yet, but you might have some problems with that. So during this talk, during this session, what I will try to do is I will try to go through some of the most typical pitfalls people run into when deploying their first or second Ceph cluster. Specifically, what you will do is we'll focus on the concept of hardware for the OSD nodes. So what are OSD nodes we'll cover in a little bit. But overall, we will focus on hardware, so networking, CPU, memory, and obviously storage for the OSD nodes for Ceph. My name is Piotr, I'm here with Bright Computing where I look after integrating our cluster management software with various cloud platforms, including OpenStack and my downtime. I also help out with managing our own Ceph cluster. So, Ceph, so as you probably have heard, nobody really wants to have a Ceph cluster. People want to have a storage solution which can be consumed by their application, so we are no different than that. So typically, what you would do when you combine Ceph with OpenStack, you would want to deploy it at least for a glance, probably also for Cinder and Nova to provide a full storage solution for all of the OpenStack services. And depending on whether you wanna go with Seven Routes Gateway, you might go with that, or you might go with Swift for object storage. But either way, during this session, we will focus on the Ceph in the context of OpenStack. So, when deploying a small-scale OpenStack cloud, many people simply go either with the reference drivers for Nova, Cinder, and Glance, so that's typically either NFS or just local storage. Not that it's a good practice, but it's simply a simple thing to configure just to get started. However, for larger-scale deployments and for any production deployments, you would typically want to have a distributed and redundant network-based storage. With that, you typically have two options.  So, if you have a look at the user survey conducted among our community, it's clear that when it comes to providing storage for Cinder and also for other services, Ceph is by far the most heavily used driver for providing that storage. So, why is it? I mean, is it because it's free? Is it because it's simple? Is it because it's flexible? Is it all of those things? So, in order to try to answer this question and the next slide, what you'll do is we will have a look at the very brief case study following a, well, a typical Ceph admin throughout their day of work. So, let's have a look at that. So, just as a reference, the color orange will refer to Ceph itself. Okay? Yeah. Yeah, so let's have a look one more time. So, yeah, I guess you could say that Ceph is a tough nut to crack. But, yeah, it's not so simple in other words. So, why is that? Well, Ceph is complex. I'm not saying it's a bad solution. It's actually pretty good and we've been using Ceph for a few years now. We have two, maybe three, depending how we counted Ceph clusters ourselves and you're very happy with it. But the fact is that it's not easy to configure, it's not easy to deploy and most of all, it's not easy to select the best type of hardware for your OSDs when you're just starting out with your Ceph cluster. So, the typical problem you will face is you'll have to answer how many OSDs or how many OSD demons, or in other words, how many disks per a single node you would want to have in a single Ceph OSD node, be it one, be it four, be it maybe more than that. So, with that, you typically talk about the concept of thin OSD nodes which have typically less than 10 disks in them or maybe fat OSD nodes which have, well, typically more than 20 or so, there's storage disks inside them. So, further on, we'll try to compare those and see what's the sweet spot for those. So, yeah, in other words, that's what you will cover today, how many OSD demos per node should you have. So, I should probably point out that, no, that's actually our agenda. So, maybe let's have a look at that first. So, I'll give you a little bit of background and just to give you an idea where we come from in terms of how we consume Ceph and why it's important for us to not spend too much time managing Ceph. We simply want to have a Ceph deployment work properly for us so that we can consume the storage but we don't want to have a dedicated team of people managing it throughout the day. We simply want it to run and behave reasonably fast. So, we'll go through a little bit of background for that. After that, we will have a brief introduction to Ceph for those of you who might be new to the topic and after that we will jump to the main parts of the session so we'll discuss all those individual pieces of hardware which you have to consider when creating your own Ceph cluster. After that, we will compare the FAT and the FINOSD nodes. Hopefully, there are also conclusions and at the end, hopefully, we'll have some time for some questions. So, a little bit of background. So, as I said, I'm here with Bright Computing. We basically are a software product company. We have a single product which allows administrators to easily turn a pile of hardware into a fully functional cluster. Now, this cluster can be any type of cluster. It can be a HPC cluster, a OpenStack Private Cloud, a Ceph cluster, Kubernetes cluster, Hadoop Spark, whatever you want. Well, that's all we do and we don't really want to spend time managing our Ceph deployment. Now, well, that's a typical cluster for those of you who might not be familiar with the concept. We have a head node in the middle which stores the configuration of the entire deployment and then we have slave nodes or compute nodes on the right-hand side. So, in the case of this diagram, they might be running some OpenStack services on them. But what's important here is that, well, unlike some other vendors, we try to provide our customers with more than one reference architecture. So, ideally, we want to provide our customers with full flexibility with regards to how they want to structure their cluster. Maybe they want to combine Ceph and OpenStack and Kubernetes and Hadoop on the same cluster. We want to make it easy. So, what that means in practice is that our developers, our basically entire engineering staff has a lot of different scenarios to consider when developing new features, when testing them, and so on and so forth. So, with that, we are in need of creating a large amount of clusters internally only to be able to develop our product. And that's actually where OpenStack and Ceph come in. So, we have our own OpenStack private cloud which is based on Ceph. We also have something which is called cluster on demand which is a small set of scripts built on top of OpenStack which effectively allow our engineers and also some of our customers to very easily deploy virtualized versions of our products or virtualized clusters within our OpenStack cloud. And what's cool about it is that it's possible to do in under two minutes. So, developers don't really want to wait two hours to deploy cluster only to test a feature or test a backfix, right? So, they want to be able to provision an entire cluster very rapidly. But I won't be talking about cluster on demand over here so if you want to know more about it just approach me after the talk and we can discuss it. But basically that's a bit of background in terms of why we don't want to care about Ceph all that much. Just a quick diagram, how the cluster on demand is structured. So, on the bottom part what you see is the physical layer of the cloud. That's our physical hardware and the top right corner you can see individual clusters provisioned by our engineers. So, we have one of the clusters over here is running HPC and Hadoop. One other is running some other resources and then yet another is running a virtualized version of OpenStack. So, all of those I guess that are pretty easy to deploy thanks to Ceph and copy and write. So, yeah, that's our private cloud. We call it Crusty the Cloud. It's not too big on this, about 17 hypervisor nodes. Give or take 10 Ceph OSD nodes. We are planning to extend it shortly to about 10 OSDs. But it's been working out pretty well for us. So, at any given time we have about 400 VMs running there which boils down to approximately 100 pretty small clusters which is enough for development work. That's our management interface. Incidentally that's something which both our cluster administrators see as well as the end users which provision cluster. So, since we use the same piece of software to provision the physical layer and to manage the physical cluster as well as the virtualized cluster. Clusters that's a typical, that's the management interface which is available for managing and monitoring both of those layers. I won't be going to Tomajite and this one. So, I only posted this over here in case somebody's interested in having a look at the how approximately our private cloud is structured in terms of networking and different control data planes. So, the slides are online. There will be a link at the very end at the last slide. So, if you do not have to take photos right now you can just download the slides later on and have a look if you're interested in that. Okay, so in a nutshell we create many VMs. We make heavy use of copy and write to create pre-installed head nodes and customize them towards users' requirements and the bottom line at the very end is that we don't really wanna care about managing SEV itself. Okay, so before we start with a brief introduction to SEV let's just do a quick show of hands. How many of you guys over here and girls are running their own SEV cluster? Okay, good. How many of you are not running your SEV cluster but are thinking about running one? Okay, great. Okay, so I guess I can skip that part since majority of you it seems are pretty familiar with SEV so obviously software defined storage solution, object block, file storage, yada, yada, yada. Popular with open stack rados. Probably most of you have seen this slide so might as well skip it as well. Rados were reliable, we have multiple replicas of the data so every piece of data is stored on multiple nodes ideally at least three of the nodes. It's so filling so if one of the nodes goes down SEV will try to restore that replica from the remaining replicas and to one of the other remaining nodes obviously is distributed so the failure domain can be spread across hosts, across tracks, data centers, even entire regions. Scalable so as you have probably most of you have heard it's one of the biggest SEV deployments over at CERN. Yeah, clients, one of the unique features of SEV is that clients can access the data directly rather than always have to go through a monitor node or through some kind of a point which will point them where the data is so that also makes SEV quite efficient. And yeah, dimension copy and write. So in our case we use copy and write to create head nodes from images stored in glance without having to actually copy all those gigabytes of data and simply do a reference copy inside of SEV. So it's been working pretty out well for us so far. So when it comes to types of SEV nodes as you know we have SEV OSD nodes, SEV monitor nodes and if you're running with SEV as you also typically talk about SEV metadata server nodes or metadata nodes during this talk we'll be focusing mostly purely on the OSD nodes so the nodes which actually store SEV data. So that's the same slide which we've seen earlier pretty much how many OSDs you wanna have. So the way you typically deploy SEV is you want to have a single OSD demon managing a single spinning or SSD, a single disk storing the data. So in this case you can see in the main part of the screen you can see a close up to a single node having a running a five OSD nodes each managing data which is being stored on a specific disk. So FAT nodes versus FIN nodes. So let's have a look at that. So whether it's better to have a bit more of a less dense nodes or maybe fewer nodes which are really dense as in they have many OSDs with many disks in them. So FAT nodes first. So typically when you talk about FAT nodes that depends but you typically talk about nodes which have more than 20 HEDs, probably a few SSD journals to speed up data rights. Well they are typically cheapest per petabyte because you save the money and not having to buy that many CPUs motherboards and so on but on the other hand they are a bit more difficult to set up but more difficult to maintain. Also because you have fewer nodes that means that one of them goes down save will have to take more time to restore the data to other nodes basically to recover from the failed state. So that's also something that you would have to consider. Also more HED more disks often means more CPU cores which often means more CPU sockets which means that you probably want to start thinking about NUMA and how that impacts the performance of your system. What does NUMA will cover in a bit? And yeah overall bottom line being that with dense nodes there's more services running within the operating system. There's much more data going through network, through NICs, through disks and simply there's much more potential for bottlenecks in various places. So much many more things which you have to worry about if you go with father nodes. Finals on the other hand, well they are typically a bit more expensive because you have to obviously purchase a bit more CPUs but it can depends on what hardware you purchase. So recovery is faster so if one of the nodes goes down that typically constitutes a smaller overall percentage of your cluster and that also in turn means that it will take less time for safe to recover to a consistent state. But on the other hand you will need more space in your racks and also more power needed to actually fuel that cluster. But if you're just starting out with SAF well from our experience at least going with thin nodes is a good place to start to experiment with SAF. Okay so what's the sweet spot over here so we will try to explore that in the next few slides and hopefully there are some conclusions over here at the end of the session. Okay so that would cover the introduction to SAF so the next few slides will go through networking and then disks, CPUs and memory recommended for your SAF OSD nodes. So let's start with networking. Again we're talking about seven the context of using SAF for OpenStack. And with that one of the first well you will typically need three networks to actually power such a solution. So the first network you will need is a some kind of a networking fabric to for the communication, for the traffic between your VMs essentially right. So VMs have to talk to each other one way or the other be it via flat network or via overlay networking or VLANs you have to have some kind of fabric to carry the traffic. Another fabric which you will need is fabric for OpenStack and for the VMs and services like Glance API to actually access SAF and manage it. So that's another network you need and then yet another ideally a network over which is used by SAF to replicate the data. So as I've mentioned, if one of the SAF OSD nodes goes down SAF will autonomously try to replicate the data to other nodes from the remaining copies and that data has to go through a network. So how to approach that? Well, there are several solutions. Well, the simplest one would be to go with a single networking fabric. So what do I mean by that? I can think of a single switch and maybe some VLANs on top of that which would fuel all of those three networks mentioned on the previous slide. That's by far the simplest solution but it has some problems. So first of all, if you're talking about Ethernet, you have a single broadcast domain which under high load might increase the overall latencies within the network over also bandwidth limitations. So there's only as much data as you can push through a single fabric. So for example, if your SAF is replicating data in the backend, that might impact the performance, the bandwidth available for the traffic used for the traffic between the VMs available for communication between the VMs and vice versa, of course. So that's something to consider. A different solution would be to have a dedicated networking fabric. So think of it as dedicated networking switch for each and dedicated NIC, of course, on each of the nodes for each of those three networks. So you'd have a dedicated network for traffic between the VMs, dedicated network for accessing SAF and then dedicated network for SAF replication. So with that, ideally, what you would also want to have is you would want to have all of those networks to be at least, well, equivalent of 10 gigabits per second. I mean, you could go with one gigi, but by modern standards, it's actually pretty slow. But then again, well, 10 gigi cards are expensive and with three NICs, it might be or three 10 gigi ports. Well, that also might be a bit tricky to come up with such a hardware. However, if you do not go for a converged solution, so if you decide to spread your hypervisors on a separate set of nodes and then you have your several SDs, or several SDDs on a separate set of nodes, then basically pretty much all you need is a single dual port 10 gigi NIC for each of those nodes. So you'd go with one 10 gigi NIC for your hypervisor nodes with one of those ports used for carrying, say, the overlay traffic. So either VXLANs or VLANs and the other NIC used by the hypervisor nodes to access SAF itself. So that's the so-called SAF public network. And then for several SDs, again, a single dual port 10 gigi NIC, one for data being pushed to SAF and then the other NIC used for the replication network.  So data comes in in this scenario, data would come in through the SAF public network and before the write would return to the client, SAF would use the SAF cluster network to replicate the data to additional several SD nodes which have to start the replicas for that particular object. So yeah, so that would be my recommendation if you have the flexibility to design your own networking solution, underpinning your SAF and OpenStack clusters. Yeah, many people will tell you go with 9K MTU rather than 1500. We did some benchmarks on that actually we haven't seen all that much difference but that probably points out that we are under utilizing SAF somewhere else. But yeah, the overall consensus that 9K MTU is definitely better. And like I said, 10 gigi as a minimum for those fabrics. So we've covered networking. Let's talk a little bit about what disks to choose for your SDs. So with a 10 gigi fabric, you typically wanna go with SSDs for storing journals for your SAF, for your several SD nodes.  So effectively IO is a bit faster when clients are writing data to SAF. One thing to keep in mind when picking your, when selecting a SSD for your journal, you do want it to be robust. So you probably don't wanna go with a consumer grade SSD because so those only have a very finite amount of data they can write, they can accept throughout their life cycle before actually burning out and dying. So you probably wanna go with a bit more solid, more robust SSD node over here. When it comes to how much, how fast your SSD should be, well, those are the two last bullet points on this slide. So if you happen to go with a one gigi networking, well, that effectively means that your clients can only push as much as almost 128 megabytes per second to your several SD nodes. Whereas regular SATA SSDs are capable of easily accepting up to 400 megabytes per second. So that's one, another reason why you wouldn't wanna go with one gigi networking, but instead should consider a 10 gigi networking. So with a 10 gigi network, you can probably fit in as much as three SSDs, three regular SSDs behind a single 10 gigi neck and or a single PCI Express, maybe NVMe-based SSD, which those are capable of much higher write speeds up to, well, 2,000 megabytes per second. So if you go with those, you might even consider a bit thicker pipe than only 10 gigi. Okay, so that's an example that's pretty close to what we use for our own OpenStack and SAP deployment. So in our case, we use Intel DC series S3700 SSDs, pretty small ones, 200 gigabytes, that's enough for an SSD journal. We did some research, a colleague of mine did quite a bit of research actually into which SSD is the best and well, that's basically the consensus from all the various sources he found. Many people recommend that, many people use those and many people are very happy with those and so are we in fact. Yeah, so there's a little bit of math over here. So assuming that you have a single journal SSD of this class, which is capable of approximately 375 megabytes per second, how many HDDs can you put behind it? So as you can see from the bullet over here, it's pretty much equivalent to about five regular data storing spinning drives which can sit behind a single SSD, which is actually in line with the Rural Thumb guideline which you might have heard already that typically you wanna have somewhere in between four to six HDDs behind a single SSD. But keep in mind that if your SSDs are really fast, like say you're using PCIe and VME drives, which are capable of much faster sequential write speeds, then you would be able to put many more individual HDDs behind them. So in our case, and assuming a 10 gigi networking, we can have as much as 15 HDDs in our SSDs. So with a 10 gigi networking, we can have easily three SSDs with five HDDs behind each one of those and in total, that's the pretty much the theoretical maximum of the amount of SSDs which we can have in our notes. Okay, so those are the disks. Let's have a look at how many CPU cores we should aim at over here. But first, how many sockets should we need? So as I mentioned earlier, if you go with multi socket architectures for your motherboards, you will probably have to start thinking about something called NUMA, which stands for Known Uniform Memory Access Architecture. And basically what it boils down to is, I won't be going into too much detail. So by the way, there's a excellent talk about NUMA over here. So if you wanna know more about it, be sure to check it out. It's from the previous OpenStack Summit, I think by guys from Comcast, if I recall correctly. Basically what it boils down to is if you have two CPU sockets, some of the, one of the sockets, one of the sockets processes interrupts from some of the devices in the box and the other socket process interrupts from the other socket, from the other devices. So one example is what we have over here in the third bullet. So say one of your CPU sockets processes the data coming in from your NIC, whereas the other CPU socket processes the data which goes to and from your SSD. So what happens here is, as the data comes in, it has to go into one of those CPUs. It has to cross the so-called QPI bus between the sockets before it can end up on the SSD. And it might not seem like a big deal, but this according to the guys which the talk I quoted over here is actually a pretty good deal when you actually start measuring that under high load for very thick nodes. Basically what it means is there's actually quite a bit of traffic going back and forth over the QPI bus and it might very negatively impact your overall self-performance. So with that, if you don't want to have to think about that and you don't have to think about things like pinning your several SD processes to specific CPU cores or CPU sockets, then a safe bet would be to go with a several SD node which only has a single socket. And so you don't have to worry about all of those things. Again, a more pragmatic approach. How many cores? Well, a rule of thumb is to have a one CPU or half a CPU core per a single SD demon. So half a CPU core under normal operation, maybe one CPU core when it starts under failure, when it starts data recovery, the data replication and then the CPU load goes a bit up. So with like 12, with 12 OSDs, you would probably need around 12 CPU cores. Ideally, you'd want to pin your several SD processes to specific several SD cores so that they don't jump back and forth between them. Hyper threading enabled disabled. So again, the overall consensus seems to be that when you talk about cores for these self-OSD processes, you want to think about hyper-threaded cores. So you definitely want to have hyper-threading enabled. Although I must say that I haven't done any actual benchmarks on that. So over here, I'm just conveying what I've heard from other members of the community. How much memory? Again, so a rule of thumb would be about one gigabyte of RAM per one terabytes of storage stored on your several SD nodes. Obviously, more is better since with more data, you have more, Linux has more capability of utilizing this virtual file system, the caching mechanism to speed up reads to obtain some of this data when it's being read by the clients. Let's see. Yeah, so here are some examples. Okay, so let's quickly summarize what you discussed so far. So what to consider when designing the hardware for your, or pick selecting the hardware for your self-OSD. So first of all, it's networking fabric because, well, it's pretty much determines the amount of the number of SSDs and the overall performance of the cluster which you will have and the amount of SSDs per a single node. Once you know the amount of SSDs per node, that will pretty much, once you know the type of SSDs you will want to have per node, this will also tell you how many regular spinning drives you would want to have per a single node. So in the example earlier which we've covered, we said that what a typical decent class SSD that would be about five disks per a single SSD. And then the number of disks itself, once you know that, it will basically tell you how many CPU cores you would wanna have. So typically one CPU core per disk. And yeah, the amount and the overall size of the disks will in turn tell you how much memory you will need for your seven SD nodes. Yeah, so we're pretty much almost done. So that's just a quick comparison of different types of nodes and different theoretical performance characteristics which one could expect of those. So from left to right, we go from a unreasonably thin node all the way to a very extremely fat, morbidly of fat node you could say. So in the unreasonably thin node, what we have is we have the unreasonably thin cluster. It consists of very thin nodes which basically have a single SSD and a single HDD in them. So a very unreasonable solution in practice, but just for comparison. So we have 96 of those nodes, 96 HDDs total. And actually you can see that all of those individual clusters, that's the first row, all of them have 96 HDDs only spread out across different number of nodes. So like I said, with the very thin nodes, so what we have is one HD per node and one SSD per node. That's something you wouldn't run in production. Obviously with only one HDD, you're heavily underutilizing your SSD. But yeah, like I said, you wouldn't want to run that in production. Then you have the thin nodes. So over here we have approximately 16 individual nodes, each node with six HDDs and a single SSD. So that's a more reasonable scenario. But again, over here, if you're talking about a 10 gigane networking, we're actually underutilizing the network because a single regular SSD will only be capable of accepting that much traffic. Then we have a more pragmatic approach. So over here we're talking about eight nodes with 12 or 10 HDDs, each approximately two SSDs per node. So that's a bit better utilization of the network overall. But yeah, with that many HDDs, you will probably need to go, depending on your hardware vendor, you will probably need to go with a bit higher chassis. So if you're talking about the three and a half inch drives, you will probably want to go with a two-year chassis for the nodes. Then you have a regular node. So over here we're talking about six SSD nodes with 16 HDDs each with three SSDs in front of those 16 HDDs. And over here we can see that with three SSDs, we are actually very close to perfectly utilizing our networking infrastructure, assuming that we are based on 10 GIGI. So with three SSDs, you can actually reach a theoretical sequential max speed of about almost 10 gigabits per second. So that's actually a pretty good solution for this particular networking fabric.  And then the last scenario. So over here we have with the same amount of HDDs. So also with 96 HDDs total, we are talking about only two nodes, but very thick ones, quite obviously, each one having 48 HDDs. So effectively 48 SSDs. And with that many OSDs, that also means that you will probably have to go into the NUMA territory. So probably that's not the place where you wanna go if you're not familiar with that. And well, those last two columns, well the obvious drawback is that if one of the nodes goes down, well you have problems with recovery. So in the case of the FAT nodes where you have only two nodes in your entire cluster, you will not have any recovery quite obviously, with the cluster with only six nodes.  Yeah, so that's pretty much it. So just to summarize some key takeaways. So we'll probably wanna go with a 2U chassis with when designing your self cluster, with a 2U chassis for your several OSDs with 10, maybe 12, three and a half inch HDDs and two SSDs. Or alternatively, if you can afford a bit bigger chassis, higher chassis then, you could go with 16 HDDs and three SSDs or one PCI Express NVMe instead of those three SSDs. That should also be sufficient. You should probably prefer similar nodes if you can afford the additional, causing the additional space in your rack. That simply makes it a bit simpler to manage to get started with. If you can handle it, if you have experience with self, then you might wanna go with more thicker nodes or definitely more better for optimizing for overall cost of your storage at the expense of complexity. You probably wanna avoid multi socket motherboards for several SDs. Start out ideally with at least 10 gigi networking. Unless a small POS, you probably don't wanna go with one gigi. Avoid small clusters. Obviously, like I mentioned two slides earlier, with only like six SD nodes, if one of them goes down, that means pretty long recovery. Yeah. Also some other random tips, just to keep in mind, be mindful and read up on deep scrubbing because it will kick in by default every week and every week you will be wondering why the performance of your cluster starts going down. It's deep scrubbing, obviously. Read up on object and data striping that also has a performance potential to increase the performance of your cluster. That's not something I'm gonna get into over here. One other thing is if you're using CIF for your Cinder nodes, you definitely want to enable QoS. So before we've enabled QoS on our cluster, every now and then you would have a runaway VM which would simply generate a lot of log outputs to the disk which would effectively saturate our entire CIF cluster. So by throttling that the amount of sequential writes each VM can generate and the amount of IOPS it can generate, we basically get rid of that problem. So again, definitely something you might wanna check out. Same applies for Nova, obviously, if you're using Nova for your VMs. Yeah, enable RBD cache on the client side and libvert so that it can aggregate small IO before sending it out to your CIF cluster. So it's called, a feature called RBD cache. There's tons of very interesting CIF videos out there on YouTube. So if you're just starting out with CIF, again, be sure to go through them. And one other here which I highly recommend is one from one of the previous summits from the guys from CIRN, CIF at CIRN. That's the last bullet point. So yeah, definitely check this one out. And that's pretty much it. So thank you. Any questions? The quick comment from the talk before about QEMA optimizations. They had some test hardware from Intel, so probably the fastest PCIe SSDs available. And they decided to run four OSDs per PCI device because other than that, they would not be able to saturate the queue. So we're already going in a direction that even the rule of thumb that everybody knows why OSD per device is with SSDs gonna be thrown out again. All right, that's good to know, thanks. Hey. Do you have any experience running OSDs on SSDs? And in that case, you have any recommendations on where to put the journal? Keep it on there or does it make sense to put it on the... We don't have any experience with that. If I were to guess, but again, I'll just take yes, you would probably wanna keep it unless you can afford the additional cost of maybe even faster SSD, right? So you could, for example, have two classes of SSDs, one very fast and ultra low latency only for the journal, and then simply several slower SSDs for regular data. So that would be all the solutions. Hi, one of the things that brought my attention on your presentation was the relationship between the bandwidth, the network bandwidth, and the, not only the type of the SSDs, but also the number. For every paper that you read, they always recommend that you should not stay with 10 gigs. But as you said, the regular size of the nodes, you only, you're gonna have up to three SSDs. And these guys, SATA SSDs, these guys cannot talk faster than 10 gigs, which brings me the impression that you say that don't waste money on going beyond 10 gigs on the SEF network. Is that, or? No, I'm not saying that. So definitely the faster your networking is the higher the overall throughput of your SEF cluster. So if you can afford 40 gigi or more than that, then yeah, by all means, go for it. But the SSDs will cap your throughput or no, because? That's my understanding. I'm not saying that will definitely happen. It might depend on type of SSDs, but that's the feeling I get from experimenting with SEF. And also, I mean, you cannot simply consider putting in faster SSDs, right? So indeed, you can, you will probably be able to reach 10 gigi with three data center classes as these. But if you go with several PCI express and VME drives, then yeah, I might not. All right, thank you. Yep. Hard drives are getting bigger, of course. We're looking at possibly using eight terabytes and 10 and 12s coming out by the end of the year. Any recommendations on how big to your hard drives to use in your SEF cluster? I didn't get the word, what's that? Hard disk drives, you know, at the moment they're, I think you showed four terabyte drives in your example. Yes, yes. Six, eight, 10. Yeah, that's just an example, of course, so yeah. So what's the risks of going to, obviously the regular time when hard drive fails, but any other risks or do you see going with bigger drives? Other risks or reliability, I would say, the bigger the drive, the smaller the size of the individual bid on it. So obviously the higher the chance that the higher chance of bid route, for example. But that's something you have to consider on your own, right? Depending on what type of HDZ you want to go with and how many of them you want to have, so. Well, you know, obviously if you're looking at the dollar per terabyte price, you know, and you use eight terabyte drives, you get a much cheaper solution. Some of the reference architectures are coming out now, so the Dell solution is a EXT with 16 eight terabyte drives and a couple of fast PCIe cards in France, so. Yeah, yeah. Hi, have you considered running CefOS this right with the Nova Compute on the same hardware? Yes, we have, and in fact we're doing it right now, although we will be moving away from that. So it's a bit hard to gauge how big of an impact a CefOS D process can have on the performance of VMs. So, no, in our case, it's not a big deal because our VMs are actually, most of the time, are fairly idle, so they do not generate that much load on our Cef cluster, but one of the problems which you might face is, well, the complexity of the management. I mean, say you want to restart your hypervisor node, but you end up restarting hypervisor any CefOS D nodes, which will generate, unless you bring up the SD out with your Cef cluster, it will generate quite a bit of traffic. So that complicates management a lot. But in other cases, if your VMs do generate a lot of CPU load, then you probably want to keep them separate from your several SDs. But again, there's something to be said about both approaches. OK, thanks. What's your experience comparing SAS devices with SSDs as journals and regular SATA, sorry, SATA devices with SSDs on journals versus regular SAS devices? Well, I didn't do any comparisons on that, so yeah. Because our experience has been SAS with collocated journals, usually compares almost the same in terms of performance. So if you do the maths, it's kind of one-on-one and you get better cost effectiveness with the SAS. For block storage? Yes. So for RBD, not for Rathus Gateway. So I was interested if anybody in the room has been comparing the two scenarios. OK, but wouldn't the latency still on SSDs be much better, right? The latency is the only thing that's a bit better.  OK. But yeah, I think there's one more question. I just wanted to make a comment to this gentleman over here about NVMe-based journals. If you have the money, it does make sense to put the journals on NVMe because your latencies for writes will go down. And also, you can mix endurance on your data-based SSDs. Yeah, thanks. OK, so one more thing. We have some t-shirts over here. So if somebody wants a t-shirt, yeah, just come by. Yeah, thanks.


Hello, everyone, and welcome. My name is Melissa Armell, and I am the Soxwush. And tonight, I'm going to talk to you about persistence. If you want to be successful as a day trader, if that's your dream, if that's your goal, you want to make a certain amount of money every day in the market, or every week or every month, you have to be persistent. Persistence means what? It means continuing forward, even when you're tired. It means you continue to keep going. You get up. You do what it takes. You work hard, even if you feel frustrated, even if you feel confused.  You overcome obstacles as they come upon you.  You keep learning. If you learn a trading system and it doesn't work, you don't quit. You learn another one. If you take a bad trade and it doesn't work out, you don't stop trading. You have one bad experience. You don't quit. You keep moving forward. You persist. You continue to move forward, despite obstacles. Even when you're sick, even when you're tired, even if you feel like you don't have enough money to trade, you still find a way to get the money to trade, to take a class like mine, to press onward. You don't give up no matter what. Someone tells you it's impossible. You can't do it. This many people fail. You say, I don't care. I don't care what you say. Poop to you. And you ignore that person and never talk to them again. You continue. And in your mind, you believe 100% that you can do it. That you can keep moving forward and it's called persistence. It's persistence.  Any of your dreams are their big dreams to come true in life.  So many people give up. First of all, people give up. People give up too easily and they totally lack any level of persistence at all, let alone a lot, okay? So persist, move forward. Do things that you think you can't do. You think you're tired. You think you don't have enough money to do it.  Ignore all those things, okay? Push forward, keep going. Even if you think you can't. There's very few people that can do that. So a lot of people say, well, I'm being rational, I'm being logical, and this is the facts. Yeah, well, the facts, the facts are exactly what your reality might be right now, which might be that you're losing in the market.  If you wanna change your circumstance and you wanna be a winner, then you have to change the way you're thinking about going after your dreams or going after the market and saying, well, I'm gonna look at reality.  It's what you're looking at. Do you think that this is gonna be hard or easy? Do you think you're gonna be able to overcome difficulties when they come forward? Do you have that mindset of a quitter, okay?  And I'm not saying that trading the market is necessarily a big dream, but I will tell you that a lot of people would love, love, love, love, love, love to be able to work a half an hour a day or an hour a day and make a couple thousand dollars. A lot of people would love to know how to do that. And not everybody even knows that day trading exists or that you can even do that, all right? Let alone that they have a way to do that like I do, which is the Golden Gate system. So when you think about it, all the possibilities in life and the fact that you even know that there is something out there that you can do, that you can do this day trading thing that you can make a couple thousand dollars a day that you can make a couple thousand dollars a week that you can do this in such a short time that it doesn't have to be something where you work 60, 70, 80 hours a week. When you get to the point where you find out that this is possible and then you find me who has this incredible system that can teach you then your mind starts to turn around and you realize the possibilities in life. You will persist when the obstacles come across your table. You will stop thinking negatively and you will start thinking positively and you will start to believe that the impossible is possible. And that nothing is gonna stop you from your dreams coming true and you will persist because knowing that every step you take forward gets you closer to your goal to the point of achieving it, which you will be able to do if you listen to my advice. So have a great night everyone. Listen to what I'm saying tonight. Persist and continue on to follow your dreams and make them happen and if it is to be a day trader then good luck to you. If you'd like more information on my Golden Gap course email me at melissa at thestockswush.com.


What's going on everyone? Sam here with LaytonSports, Gargerman23, Bowman Baseball, Jumbo8Box, case break number 13, random team style. There's a look again at your random results. As always, you can find those on the website under the results tab, as well as the link in chat. Good luck. What's up, Quack? How you doing, buddy? All right, let's see something big, man. Watching some hockey and breaks, oh yeah. Yeah, Hawks are playing the Celtics pretty, they're playing the Celtics better than I thought. I thought they were dead in the water with no Murray, but they look, I mean it took a buzzer beater to win, but still, once Murray's back could change it a little bit. Plus, it's back in Atlanta, so I'd love the Hawks to take them seven games. That'd be awesome. Gabriel Gonzalez on the base auto, nice one there, Mariners, to start us off. And Collier. Teal Shimmer, Ronnie Simon, 2-125 for the race. Or Aqua Shimmer, I should say. Gold paper coming up here, that is 2-125 on the Refrector, a Nolan Gorman, Brett Beatty Rookie as well. Gold paper, Austin Charles on the first for the Royals, the 50. Grissom, my dad tax me, of course. My dad's like the biggest fair weather Knicks fan ever. Like, he's a huge Yankees fan, he's not like the biggest fan of the other sports, so he just goes with the other New York teams. Every time the Knicks win, he tax me. He goes, Knicks win, Knicks win, Knicks moving on. Really? Get him pops. There's Green, James Outman, 2-3-99. Pretty much. What's up, Steve? Steve is Freilet Incarnacion on the Refrector first thought, 2-4-99. Yeah, I'm aware, Steve. My dad made that very aware, Gunner Henderson. Very apparent. Ooh, 2-75, Yellow, Crater is how you leap for the Philly, right, yeah, Philly. Give him a chat one time. Michael Vargas, I don't even know how to do that. Come on. Francisco Alvarez, 5-2, Drew Jones. There's Aria Almonte, 2, that's Lunar Glow, Shade Space. That is Matt Mervis on the Purple to 2-50. Nice one there, Cubbies. 2-49 of 2-50, Matt Mervis first, and base. All right, done on the first box. I don't have any good ears. There's base auto, or no, just auto, Shade Whitcomb, that is a base auto for the Ashropes. That is totally random. That is very random, Steve, very random. Base, probably green rookie. Mix in water, too. Vincent Perrezzo, or Perrozzo, 2-4-99. Yeah, he's good, Matt Mervis is good, dude. He's like one of the better guys in this product, too. He's like a tier two guy. Gabriel Moreno, Gordon Waller. Are you talking about for the Cubs, right? I'm assuming C.V. So I have to imagine it's probably like PCA, and what, Hernandez in that same conversation. Corbin Carroll, base. Yeah, gotcha, gotcha, yeah, he's a stud, man. He's a stud. There's Atomic Invicta, yes sir, Mercedes. Yes sir, Mercedes, the 1-50 for the twins. Refractor Javier Osorio, 2-4-99. Be down at some point for sure, Matt McLean. There's a downtown Ben Brown. Blue Crater, 2-1-50 for the Cubs, like that one. Corbin Carroll Rookie, that's his nickname, they call him downtown Ben Brown. Shay Langlears. I think you call him that. I think a lot of people call him that. No. Oh, for sure. They're missing opportunities, then. Well, so what, downtown, where is it? This is Ron, it's the name, downtown Ben Brown. Lazaro Montez. Think about these things before you say it. Oh, no, never. Lunar Glow is Christian Baccaro, glad I did. Glad I don't think things through. Oh, you really. Luis Sanjela Cunha on the Bowman Top 100 for the Rangers, 2-99. It's middle child, the Cunha right there. Son of Ronald and Brian, base. There's a Luis Sanjela Cunha on the back as well. Veen Elijah Green, Veen and Green right there. Veen and Green right there. Yeah, no, Veen and Green. It's Veen. Jayce Bowen, base auto for the Pirates. Nice one there, Jayce Bowen. I'll bring this cell. Nolan Gorman, rookie. Five tool potential is Drew Jones. Oh man, Red or Super, see it. Vargas. I just don't see him, man. Freylae Encarnacion to 2-99, future paper. The Red Sox. Blake to 4-99, let's go. Blake on the refractor. 4-99, I'll take a pic of that too. Two Blake numbered cards tonight. Spencer Trollkelson, purple pattern to 1-99. Tigers. Trisha Casas, L.A. De La Cruz, sites on September. If you got called up, it'd be awesome. The Electric. Base, Jordan Waller, who takes my shirts. And there's a little bit of detergent to the washroom. I didn't need a chance to. I work four days straight in that shirt. Some stained drawers. There's a Juan Burrito to 2-99, white speckle auto for the Rockies. These responses are so long, I don't know what to do. Kind of people are filming me. Yeah, times are tough. Really, really, really good. Another Juan Burrito this time. Purple, nice color match for the Rockies, the 250. Juan Burrito. There's a Peraza rookie. I think it's a bullpea, but it's not. There's Lunar Glow is Jonathan Mahill. It's up Rock. And Jordan Waller to 1-99 on the Equipment Creator. The next one. There's Junior Tillian, first refractor auto to 4-99 for the Metz. Oh, he's sitting in there. Zack Meade. Fire engine red. Yeah, $10. Oswald Peraza. Prospects of Jackson Holliday. Got a Teal Shimmer, Braylon Bishop to 1-25 for the Pirates. Braylon Bishop. Harry Ford, top 100. He has a chance to be a real problem. Yeah. Yeah, Harry Ford. Played really well in the World Baseball Classic. Adley Rushman rookie. Pink, paper Anthony Gutierrez to 1-75. Invicta Samuel Zavala for the Padres. Nolan Gorman. The Ellie Show. Bass, Ellie again. There's Pedro Ramirez for the Cubs on the true blue color match. Nice one there to 1-50. Grats on that Chicago. Francisco Alvarez rookie. Riley Green rookie. The real deal, Jonathan Mahill. Monta and Bass. Grizzly's making a bit of a push here. Riley Green rookie. Spotlay, Jordan Waller. There's Drew Jones on the Lunar Glow. Nice one there, congrats. Diamondbacks, Chuck S. There you go, buddy. Lunar Glow, Drew Jones. Play the best guy you can get in that trailer. We'll take it, we'll take it. There's Troy Johnson on the base first auto for the Marlins. Montec to 2-99 for the Reds. White's a speckle. That'll do it for box four. We're halfway through the case. On to box five, everyone.  What'd you say? No, just. Brandon Walter, White's speckle to 2-99. First auto for the Reds. What's up, Randy? Randy. Pretty solid. Bowman's been pretty solid tonight so far, I'd say. Definitely been a good one. I'm loving the design this year. I think it's a great design. They did a great job. I think the new parallels they added, I was very skeptical when I saw what was coming out, but they have impressed me, I have to say. They have impressed me. Corbin Carroll? Yeah, I agree. I would say that Prism is the bowman of all the other sports, though. Bowman's a little more like classic, I would say. Been around a bit longer. Brett Beatty? But yeah, I mean, that comparison's like that on, I would say. Jay Witt. Dead on. Blue Paper, Shohei to 1-50. Nice one there, Angels. Their niche Valdez, base auto. Their niche Valdez, it's got quite the auto. Take it over two letters, might not, looks kind of like me writing it, but I would take that over two letters for sure. All day long, yes, absolutely. Curtis Mead. Sedona and Rafael. Andy Pages. Pretty house, it's his house. Ray Wave, Jose, Perroza, 2-3-99. Cubs are very good in this. Cubs are a very good team for sure. Gabriel Morena, a rookie. There's a purple Daniel Susak, 2-2-50. Paper, the A's. Braylon Bishop, 2-100 on the Atomic. Nice one there for the Pirates, thank you. Think of it for a second. Atomics are clean, and it's like when the Atomics are out of 100. The Autos. Gunner, no, Ruchman, Jesus.  That's the hair, dude. Look for the blonde, and look for the brown hair. Yeah, no, I know, honestly. Hey, did you get them in there? Oh, Chris, that is Michael Harris, rookie. Lager Green. Lowe is Jose Perroza, and Ricky Teatman. Three boxes to go. Luciano, your Donny Day looks nice. Peace. Oh, is there no Donny Day? Not really, not really, unfortunately. Like the numbered paper has a little bit more value, I'd say, Phil, but now paper, paper, base, pretty much not at all. Jesus Baez, that is base. First auto for the Metz. Get that right now really quick, and Suss. No supers yet, no, no supers, no Reds either, surprisingly. Jumbos, 624.99. 624.99 for Jumbo, how much for Hobby? 280? 284.99. 284.99 for Hobby. Bloop, Marcelo Meyer, paper to 150. Oh, Mervis got pulled today. That Mervis Super got pulled. Yeah, we'll start seeing more and more supers start surfacing over the next few days. This is like the prime time when they start getting hit, it's like the first week or so, and then it'll slow down a bit, and it'll be like a few guys you're still going after. Like, I mean, Holiday lasted for a while in draft. There's, speaking of Holiday, Blue Shimmer coming up here, of Juan Brito, 2150. Silver strum. Purple, Junior Kimonero, nice to 250. On the purple base, congrats on that race. September Marcelo Meyer. I don't think so now. Between three now, Josh Young. Six I think, like Matt did eight, eight or nine. Like statistically we should have hit one, that's the thing, you know. Then there's going to be a time where we get back to back cases, have supers, like something like that's going to happen. That's Carlos Jorge. On the white speckled, two, two, 99. Nice one there for the reds, very nice. Carlos Jorge. Here at Mitchell. Like that happened with Tops Carone last year. I hit two supers the entire time we ripped Tops Carone, Jumbo and Hobby, and I hit the supers in back to back cases. Oh really? Yeah. I was like, man, this is easy. CJ Abrams to four, 99. That is Moises by Steros for the Cubs. Nice one there again, Chris. Solid one for sure on the base on it. Face and last pack of box six. That is Christian Veccharo. High rise, coming on our eyes. And Randy DeJesus on the Lunar Glip. Yeah, Cubs. Absolutely killing it right now for sure. Congrats Chris, box seven now. And guys, just two cases left after this. Everything has been sold out for hours now. So thank you. Before I even got in here, everything was sold out. So that's awesome. Appreciate that very much. Yeah, no problem, Chris. You got it, buddy. Is Nico Poyaca Grego, base auto for the Phillies. Dave Jackson, all day at 250, a lot of holiday color. Me too, Blake. Me too, I'd be so poetic, dude, if the super was pulled here. That's what I think everybody wants that. Ooh, there we go. Their first time. It is Refractor, Drew Jones, the 499. Finally, man, in six cases, our first Drew Jones. Congrats Chuck, 10 of 499. Drew Jones, there he is. Finally, man, congrats. Oswald Paraza, let's go. Finally, first Drew Jones. Now we just need the other Jones, we need Spencer. I think we'll have pretty much everybody on the checklist. Rochman, did you see the bounty for the Drew Jones super? Did you see the two scenarios though? 250 and then 750 when he hits the wall. No, a million. I mean, a million, yeah. 750. Drew Jones. It was like 250,000 up front. 250 up front, a million if he hits the whole thing. I thought it was the rest of it. That is Luis Garcia. Nice one there for the Blue Jays Lunar Glow, I think. I don't know, maybe. A million is total. Oh, okay, 750 or whatever it is, yeah. 750 if he hits it. Yeah. Simon Huang. It's supposed to be there, but they're never. Base. It's supposed to be 250 real quick. Right, Curtis Mead, base, painter, and base. Francisco Alvarez, rookie, Ellie. We pulled a refractor review Blake in this, I know Sean sent you a text. Oscar Colas at 250, the refractor to 499. Gosh, young. Lyme Green, Luis Lara to 399 for the brook room on the paper. And base. Jackson Cheerio, Marcelo Meyer. And base. There's a future paper, Cade Cavalli rookie to 299 for the Nats. We've got Dominic Kanzon on the gold shimmer, 250. Nice one there for the Diamondbacks. Rats on that Arizona did really well in that box. Or the last two boxes or so. And base. All right, last box of the break guys. Good look, y'all. Do a quick recap after as well. Tender's Optic tomorrow with Kenzie. There's Pedro Ramirez. Geez, man. Congrats refractor. There you go, Chris to 499. And everybody in the Cubs. Cubs Farm System. Riley Green. Deon Jorge. There's Emmanuel Bonilla. Base auto for the Blue Jays. Soft relic. Six autos. Let's go, Chris. Oh yeah, dude. Francisco Alvarez, rookie. There's Rookie Triton Casas, Joshua Baez. Lunar Glow, Brian Acunha for the Twins. Short fringe. Purple Adley-Ruchman, rookie. Nice, a 250. We'll take that. There you go, O's. Purple Raywave, Jays Young to 250. For the Tigers. T-Grease, Roger DeRais. Tyler Soderstrom. Jason Dominguez. And Jackson Chirio, Honey Nut Chirio. Oswald Paraza, Lakers. Pink Copy, I don't know if it's still going now. Ambiore Stavaris to 399. Base, three packs to go. Then we got a recap, and then we're on to number 14 after this, Josh Jung. Other Jung. Base, two to go. Gabriel Moreno. Brady House. Base, and last pack of the break. That is gonna be A, How You Lee. On the blue crater. Nice one there, two, 150. Very cool, congrats on that Philadelphia. Got his signature, so cool. That's so awesome. Do a quick recap here, guys. We've got Manuel Bonilla, Niko Poyoka Grego. Moises by Asterios, Jesus Baez, Derniche Valdez, Troy Johnson, Jace Bowen, Shea Whitcomb, Gabriel Gonzalez, Junior Tilian. That is Pedro Ramirez, Freyland Carnacion, Carlos Jorge, Brandon Walter, Juan Brito, Juan Brito, Matt Mervis, Freyland Bishop, Ben Brown, How You Lee, Pedro Ramirez, Luis on Alecunha, Dominic Hanzon, and a Refractor, Drew Jones. First Drew Jones of the night, 499. That'll do it for the break, guys. Thanks again for the fill. Next up, number 14.


Head of Wagner private military company Yevgeny Prigazin has released images from the cemetery where Wagner's were buried in Russia's Ekaterinburg city. Prigazin said that mostly people born in Ekaterinburg or conscripted from the city are buried in the cemetery. Prigazin, who inaugurated a monument in the cemetery, did not rule out the presence of fraternal graves in the cemetery. It should be noted that cemeteries have been built in different parts of Russia where hundreds of Wagner members have been buried. They are mainly soldiers killed for the city of Bakhmat. It should be noted that fierce battles for Bakhmat started last August.


Hello, this is Christian and welcome. In this video, I'll be talking about the asynchronous operations in JavaScript. So let's see what that's all about. And JavaScript is known as a single threaded language, right? What that means is that if you think about a train on a track, so the train is that thread, the main thread, we talk about the main thread in JavaScript. And the main thread or the thing, the main train is responsible for moving information from one place to another place. So if you are on this train and you want to move and go from one point A to point B, and that could be your first stop or your 10th stop, it doesn't matter, right? To get there, you are pretty much relied on this train here to get you there. So if there's anything that causes the train to stop along the way, then you out of luck. You may not reach there or you may reach at a later time. And when something blocks the train from moving forward, we say that is a blocking the thread, right? So it's quite common in JavaScript, by the way. And so you want to prevent that. Unfortunately, it doesn't have multi-threaded processes, like other languages like Java and C and C++ and so on. So these languages are multi-threaded. So again, the analogy here would be you have a lot of trains on multiple tracks and their jobs, of course, are to basically deliver those messages to what they're supposed to deliver. And because they're more trains, they can do it more efficiently. So to help JavaScript to kind of come back this or overcome this, JavaScript relies heavily on some very important web APIs that comes with the browsers. So remember that the engine in the browser is the JavaScript engine. But on the web browser itself, you have your own programs, right? It's a software itself that communicates directly with the server. So just put that into perspective. But anyways, these are some APIs that you've probably seen. Ajax is one of the very first APIs out there back in the mid-90s when it was created, I guess, to allow us to transfer or move data from one server to another server or from the server to the front end and back and forth. And there's some limitation to this. But it was very, very handy, very useful. And the reason why the X gets stuck in there is because of XML. And again, back at the time, XML was the de facto data format for use for transfer data. Nowadays, it's more on JSON, right? So that's why the name gets stuck in there. But anyways, you can see some of these different technologies you can use to kind of send some of this information out to a different track, if you will, so that the main track on your JavaScript program runs continuously. Okay, so we'll see some of these in action a little bit. And so Java runtime engine comes in every browser that you use. They have their own name, of course, like Google Chrome uses what's called a V8 engine, Firefox uses spider monkey, chakra for edge or IE and JavaScript core for Safari, opera uses caracan and so on. Okay, so each of these engines inside the JS engine, they have something kind of similar nature. We call a event loop. Okay, so this diagram is in no way is a complete specification for this event loop. I'm just showing you a very broad overview of what's really happening under the hood. So you can get a sense of understanding what's happening. And also how you understand how asynchronous works behind the scene. Right, so here we have, at the left side is the client, the client could be your browser, it could be another program that is communicating with your web application. And then this orange box here is the main thread, the JS runtime runs on this main thread. So every time you open a browser, you open the new tab, you have this thing going on continuously. And inside this JS runtime, we have a event loop. This is a loop that run continuously every cycle. It's also measured in one tick or sometimes called a one poll for polling. And so it receives information from the events, like these are like a click event, you know, mouse events, a keyboard events, or if you're on a mobile device would be like swipe and pinch and all those stuff, all those events, or even some other events from a data source to API, REST APIs. And when it receives those information, then depending on that type of event, at the event here has a single, we'll call a single stack or call stack. This call stack here tracks all the information is specifically function calls that you call. And it pushes those functions to your stack. And so which is why you see that when you run your code, if you have an error somewhere in your code, if you use the web browser tool to tell you, it will say Oh, it's here here. And so at this stack, you see the cost stack, it has a list of all your functions right in that order. And it tells you exactly where you started make the function call and it propagates to exactly what the error is. So it's through this call stack. So here if I call a function call f of x and inside f of x has some information that I would print to the console, it will add those to your call stack until these if it's a message that it needs to process right away, we'll process that, and then remove that from the stack. And then if there are no more information, and then you remove this function all the stack, and then goes with the next one, in that sequence in that order. So because whatever you pushes to the stack gets execute last, this is performing a this operational as a life or last in first out, right? The last in here at the top always get executed first. However, when you run into a function that will allow the event loop to move these data off to a mistake moving into a different track, these functions are very commonly used to the most important functions you'll be using are called set timeout and set interval, of course, and others as well called set media and so on. Or these are used to make we call asynchronous calls. Because now what this does is it will when it sees this function, it tells the browser or the engine that hey, move this information here, delay this until a later time. So if delay that information, and move it to another area or another truck, if you will, we call that the message queues. Okay, so over here is a queue. So a queue is different with stack, right? Queue means first and first out. So whichever receives the information first gets pumped out to the back to loop here. Okay, so it will delay that information. And while that's been delayed or being processed on a separate track, so this example here is like it's mimicking this multi-threading program, which JavaScript doesn't have. Again, this is a part of the browser, okay, it supports the system here. And so while this information is being pushed to the message queue, while it's doing its job over here, this event loop keeps pulling information keeps doing its task. Again, fetching data if there's any that's none and just keep waiting pulling until it receives this message back, and then send that off to the client. Okay, so this is kind of what's happening behind the scene here, all right? So let's go take another, see another diagram here. We'll set a code here first. For example, if I have a piece of JavaScript code on the left, this is a code you may, you know, write. On the right side is the result or the output of this code here. I'm just showing here whatever I put in the console law you will see on the right side. So the three statements I put in purple here are just a reference to tell you that these are, you know, put on a separate track, okay? So when you run this code from top to bottom at a very single tick, as you start a program, it starts a single tick, that's the first cycle. This is what happens, right? It runs out this entire little code on a single tick. Anything that's in purple here, it will push that off to a separate tick or a separate time and at least one tick behind, okay? In this case, I'm setting to a, this is a measured millisecond, so 2,000 milliseconds or two seconds, three and a half seconds, one more here for a one second, okay? So when you run down this code from sequence from top to bottom, this line here doesn't do anything. I just do an assignment. So we're not outputting that to the console. This line here is the gfx, it's a function call. We invoke a function down here, right, to print that gfx to the screen. So that runs right away. That's the first thing that happens. And then it runs to the next line, set timeout, it tells the engine, the browser, hey, send that off to the back, you know, do that at a later time. When do we do it? After this time runs out, okay? Remember that when you start this program, the time it starts right away, it goes at a constant time, it won't stop. So from time, from t0, all these things happens, okay? But this one here says, okay, run this after two seconds pass. Run this, then run this after three and a half seconds. And then this is nested inside the f of x, okay? So we'll get there. And then so this to a push to the next cycle, or beyond. And then it gets to this line. This one here runs right away. So you print that out. And then these are just functions. So they are doing nothing here. And then you're done here at the first tick. Then the second tick comes along, nothing happens, because now we're waiting for these timers here. So this is two seconds. This is sooner than this one. So this kind of run. And when that runs, when we reach a two second mark, it's going to call the f of x function. It goes down here. And we load this f of x to the console. So now when you start this function, the timer in here, it has its own internal timer. But again, this timer here is not a separate timer from the main timer. It still depends on the timer. So the overall timer here, even though I call the next time out here, it says 1000 second. This 1000 second here is still measured against the total time from the beginning here. So if you add this together, the 1000, 2000 plus the 1000 here is total 3000 or three seconds, because three seconds is still sooner than the three and a half seconds. So that means this is going to run first before this one does, even though this was run call first. Okay, so anyway, so the f of x will get invoked and it runs that out here. And then it runs the h of x because that is sooner than the three and a half. So you get that out to the f of x h of x here. And then finally, when we reach that three and a half seconds, then g of x runs, we get that out to the console. So you see the order here is complete auto order from your code here, because you're running on a separate track, separate time, I guess it's easier to think of it that way. Right. So this is a really important concept to understand asynchronous, when you make asynchronous calls. And because you're running code asynchronously, you prevent your JavaScript code from breaking or from blocking is the correct term when you use here. Okay, because they're not blocking, therefore your user will never see these information so they don't get distracted or get annoyed by your program being, you know, stalled. Right. So let's take a look at another diagram to see in visually. So here again, I have that same similar information here, one cycle is one tick or sometimes called a pole. So each of this block measures the constant time in milliseconds, I put here t zero and a thousand at 1000 millisecond, 2000 and 3000 and so on. So here is this time in seconds, you run continuously. In each second, you have a certain number of cycles or ticks that run through here. I put three here just to show you what it is, but really, there are more in that. Okay. So these are the functions I put them here on a separate track, they run on their own time. And then down here at the bottom is the actual code and the very first tick. So the first tick we run these because these are not, you know, put off on a delay, right. But when you make the set time off for f of x, this one here is put on a separate track is a wait until the two second time mark. So when these already ran, so these already ran and the existing memory. And then when we run this and this, we compare the two timers, well, f of x runs first because it reaches the two timer, 2000 time timer second first. So it runs f of x, it prints what it needs to be printed and inside that we have a nested timer to run each of x. So it will then call this function and this one runs, but it will wait until 1000 second, right from here on. So if you add this together, 2000 plus 1000, it will not execute until it reaches a 3000 second time mark. So at this point, this one runs already it executes. Well, you know, gfx is still waiting until the three and a half seconds. Okay, so this is can get really tricky. So if you were able to write asynchronous program is really complex to understand. And you can, you know, you can, you can really get a lot of headaches for doing that because you once you off the main thread, you have to find a way to come back to the main thread again. And that is one of the challenges that programers have to deal with when you write asynchronous code. But for us, we don't have to worry about that. We just basically use what we're given things like Ajax and fetch and, you know, the the other other methods that we can use their promises and things like that. So they already created for us to use. Okay, so I will stop here. I hope this is kind of give you a this kind of give you a overview of what is happening behind the scene when you run your code next time on using JavaScript. In the next video, I'll do a little demo to run this code so you can see what's really happening in live code. If you have any questions, please let me know.


Majority of the people just want to have a better quality of life. And a lot of the problems that we have today, especially now in our political environment, is talking about, let's say, automation. Or, oh, it's the rich people's fault or blah, blah, blah. Yeah, it's for votes. Everyone's blaming the fingers. For me, I view as... Obviously, there's always an educational problem. Education has to adapt. That's a place for everybody. But if we're talking about finances, if we're talking about the economy, if I'm talking about, for example, like real case studies, like my mom, right? Or my brother. Regardless of if my mom makes 60K or 80K a year, her buying power decreases every year and it decreases a lot. It's not like simply 2%. 2% is just inflation, right? You get all this quantitative easing and interest rates.  So it's like, how is a person supposed to live? So if you look at the buying power, it's lost 70% since the 80s. The same dollar you had in 80 is worth 70% less today. And so where does this go? You see the trend line goes to zero. This is why I'm not too big of a favor of all these government handouts.  I'm against the monetary system that we have. It doesn't help regular people. And so it's like, how do we increase buying power? How do we increase the ladies to buying power?  More people by real estate, more people will invest, more people will spend, but that's not the case. As we know, in North America, Canada, within North America, it's $400 savings. Average is a lot of people in dissavings also. Dissaving, yeah, average. Think about $400. That's a slim margin. You know, thank God we're in Canada. We have socialist healthcare system. Not the best, but I get a broken arm. I can go ER. Like ER is good. When it comes to social system, ER is good. Anything else? Not good. Not good. I give credit for ER. Like emergency related, like acute, like right now, I need to do it. Good.  What's the doctor call for like the liver and everything? Neurologist. Neurologist. Neurologist. Neurologist. Yeah. And obviously it's not life threatening. It's not an issue. It's like eight month wait.  You know, they got to meet out like resources to like the most urgent things. It's pretty, it's pretty like calculate. I have some buddies that are doctors and they're not a waste that's happening in the system. I mean, I'm amazed when they tell me, well, so who comes and sees you? And he's like, 80% of the people are just there because they're bored and it's free. Really? Yeah. And one of the communities near here, I was like. Bored. As an American, I've driven myself to the hospital. I think three times with like fractures, like bike wreck. I mean, the whole thing is like, and I appreciate healthcare so much that I don't ever use it unless I have to even, I still think like, well, I want to conserve it for the people who actually need it. And I talked to my friends who are physicians and the things they tell me, I'm just like appalled. And so I'm not, I don't, not taking a political position on it, but I just think in terms of like caring about, if we're going to really care about the. You have a broken healthcare system. We're ranked pretty low when it comes to healthcare. And what domain? I'm fairly new in Canada. So what, is it like the quality of it? Yeah. It's a quality. Quality and time, time waiting for something. Who's good in the, in the realm of social. Is this UK good or? France, actually. France is good. They have a two tier system. They're socialist and private. So that's one of the trends in Canada, trying to people to push is more private. There's, was it male? Isn't there a male clinic here now that you could do private? Yeah. Listen, there's private, but it's not how you think it is. It's not like. It's like pay for play. It is, but it isn't. So there's your, there is a spectrum of what's allowed in private, right?  Whatever you want to do. Basically. There's very strict laws of what is allowed privately in Canada. I mean, it really, really strict. Huh. And so if we want to make a much better and listen, dirty politics comes into play. Uh, blow to bureaucracy comes into play. Like there's a lot of factors for this. Let's sunk cost buys incentive models. People don't want to lose your position.   They don't want to lose their position. Yeah.  Uh, so ideally if you, I like to model everything in nature. Nature's the best sign as scientist. She's been here for what, six billion years. Quite some time. Yeah. Like on earth, but you know, Yeah, exactly. Right. Mother guy, let's say. Yeah. And nature's about competition and it's about conserving energy. All right. And if we look at systems today, uh, I'm a big believer in game theoretical models. And so if we only have one model, there's no competition. There's no incentive to improve anything whatsoever. And so for me, it's like, why don't we have a two tier system? Let's have private and obviously people, private's going to be expensive.   It's, uh, two parties are consenting to a transaction. Uh, you know, most, most Canadians that are looking for an FM, fMRI or nephrologist that I had to do or some form of a non-emergency. Let's say tests, they'll go to Buffalo. You pay it a pocket, you get your results in 24 hours walking, literally. It's like, how much? I don't know a couple hundred bucks or whatever it is, right? Um, but imagine you had that here. You would create a more streamlined competition. And I see now there's even fMRI. So somebody just created one for like new, new fMRI machine for $400,000. That's like less than half. Yeah, yeah. Just saw it. Yeah. That trend, the trend of lower imaging technology used to work in, uh, in healthcare. And the invention in dentistry was they had, um, 3D volumetric tomography machines you could get in office for like 250,000. Now they're less, but like replacing basically a high radiation cat scanner. And so a doctor could, could lease or buy this thing that would give you like one thousandth of the radiation of a cat scanner. And now it's available to the public. So technology, I mean, to kind of bring back in like technology is the great disruptor and the great equalizer. And so, and so when you talk about nature, like there are people kind of on the fringes thinking about things and those, whether it's the innovator thought, I'm going to create this X-ray machine, whether it's the fMRI machine that's cheaper or the X-ray machine that's much lower radiation in terms of microceiver. It's in cost. Like that innovation is that competition you're talking about in crypto. Is that competition I'm talking about? Like all the things that you're talking about kind of rope all back in DNA sequencing was like $10,000 when it first came out roughly.  Well, they don't do full, full gene. Yeah. They do just like simple stuff. So it's like 199. I don't know. Haven't done it. Haven't done it yet. Should we just never related? My dad's from Romania. Where are you from? For former Yugoslavia. Okay. Probably not. But yeah, I'm wary of doing it. Yeah. I'm a little uncomfortable about it too. I listened to one of my favorite podcasts lately is Hidden Forces. And let me pull up the last one right now. I'll send it to you afterwards. The latest episode was with Kyle Bass. And he was talking about the Chinese companies that do like, he mentioned 23 and me, they do majority of the gene sequencing for all these companies. And so it's, they, they offshore to China. And they have all the data. Oh, that's comforting. Yeah, exactly. He like, he mentioned that public. Yeah. Yeah. Crazy. Yeah. And so for me, I'm like, and I've still haven't done it. And I literally just listened to this episode. So if you guys are wondering, go check out Hidden Forces podcast. I'm feeling you're never going to do it now. You could go to like a private lab, but like, Yeah. There's one in BC.  Yeah. That's no problem.  It'll be John Doe. I'll pay cash. Like I don't want my name attached to that. You're related to outpatient services, they say. Yeah. I like privacy. Yeah. Like my goal in the future, like another way to make healthcare better is to have more interrupability of healthcare. Like even as an Ontario resident, I, it's very difficult for me to take almost impossible to take my medical medications from the Ontario government and to go to BC. It's very hard. Hmm. Very, very hard. And so this is why, in fact, a lot of times why there is malpractice in medicine, well, in Canada per se is there's a, there's a dissidence between communication between practitioners and different places.   It's kind of similar. Like they're not sharing data. So you don't have a cohesive unity of data. Uh-huh. So EMR has been around for ever, but it still is like archaic as it was when it was proposed. Yeah. Super archaic. So, you know, maybe there's a blockchain solution, maybe there's something that can like share information better. It just, it's really confusing to me. Ideally what I would like to see in the future is, there's also testing too much creates like double negatives. It's not always good to test. There's too many variables, right? Testing can cause X-rays themselves are damaging. Yeah, yeah, yeah. And also there's stuff like white coat syndrome, just a very act of going, seeing a doctor skews tests because you're just anxious and nervous. And so what I would like to see in the future, ideally is my dream would be a Star Trek tech where we have in-home units. So saliva, urine, the whole nine yards. Right? Let's say a unit like this. Yeah, yeah. Okay. That unit has my data. This is a node. It's not related to my name. Let's call me like, I'm a number, like number 12 or number 10. And it's encrypted, whatever encryption we want to use, like P2P encryption or whatever. And as a node, it's connected to an oracle. And the oracle is taking other nodes' information. And we can have some form of machine learning within these oracles that are doing pattern recognition. So let's say I'm using this machine on a quarterly basis, like a full thing. It can be like, let's say it does a Dutch test, it does all these metabolites of hormones. So like a panel. Like a panel, like beyond a full comprehend. I mean like full hormonal panel, cortisol panel, everything. Like we'll look at like top markers that matters. Maybe like you would come and do heart rate variability for that. Your devices communicate with that. Exactly. You could centralize healthcare. You know, I have like an aura ring that I wear. It shows my heart rate variability, my sleeping stuff. So we do this data aggregation. Yeah, yeah. And so basically let's say I do it every quarter for like the big panel, but on like a daily basis, taking like small markers, like heart rate variability, my oxygen, et cetera. And then we have this Oracle encrypted. I'm a number, but it's collecting a bigger reference sample pool of seeing patterns between other types of stuff. And let's say it sees something. It's preventative. I'm not looking for repair.  I want preventative medicine. I don't want repair medicine. I don't want it before it's too late. I want early, early detection. Yeah, yeah. And so see something. It's like, oh, number 10 node, you know, it comes back to me. Ding, ding, ding, ding. We see this all or whatever.  What it does is we've located in this neighborhood practitioners. So go pay attention. And then this can become like lead generation for the practitioners to them. Cause my wife's a doctor. I see it all the time, automatically just sitting there, seeing what she has to do.  Yeah. And they shouldn't. Yeah. Doctors as more should be on the human side of things. Ideally. Tests should be for robots at the end of the day. People are bad at analyzing data. And now there are a lot of people who are, they might be out of business like radiologists. Like you see that machine learning for analyzing. Yeah. I had this conversation. So my wife's cousin, he's an ER doctor. Yeah. And he, it's funny. We're just out a couple of days ago and he had pitched a similar idea to one you just mentioned, which I should probably not mention publicly, but it was like similar thought pattern. Yeah. Of like, hey, there's, he sees, just the amount of waste is really bothersome. And I think he's, you know, he's a smart guy and he was thinking technology and in a similar mechanism that you had described could solve some other problems that he sees on a regular basis that that is just bad for everybody that they're, that they're handled the way they're currently measuring. Yes. So I think that the fact that you're in parallel thinking with someone who, who is a doctor now sees people like in near death situations on a daily basis.  We're all pointing in the same direction. I think someone is going to crack it. I'm sure people are working on it. And so I could definitely set you up with him.  He's like, I see this every single day and this is me insane. And this would be a way to solve it. In the United States, they can do much better than we can over here. Unfortunately, in Canada, we have a high regulatory hurdle with our socialist, socialist health care system. It's like worse than like the FDA, like, or like the MAD, or like the medical system. A couple of reasons. One, it's a small country. Two, it's like the amount of capital and regulatory mental punishment you have to go through. It's not really worth it for Canada. Like it's easier. Not even as a pilot. Like it'd be like not somewhere with piling needs somewhere that's like a little more permissive. And a little bit more private because I have to somehow fit within our system. It can't be totally private. Like right now. Like you as like right now there's, there's very stringent laws of who's allowed to have access to your medical files. There should be. I mean, no, totally. But even you as an individual having access for them, there's for the longest time this change lately. It's really good that it changed. Like for example, LifeLabs is a popular lab over here. You would have to walk in to your doctor or walk back to LifeLabs to get your results. You can't like get a phone. Phone technically it's not in the scope. Security reasons phones aren't allowed.  They use fax because they consider more secure. I'm like, who the f...  I'm like, you know, maybe just a carrier pigeon. Yeah, the better way to do it. Or like smoke signal. I'm not kidding. Fax, man.  And so just recently, not too long ago, it was like, oh, thank you. Now I have email to receive my stuff. And so we're in the Stone Ages over here. And I think the only way that I see going back to using nature as an example, there's a great saying from, whether his name was Gall, that's called Gall's Law, where no complex organism, every valve from another complex organism, there is an evolutionary pathway where it's like simple organism or a single cellular, the multi cellular, then the cells get together, then it makes tissue and so forth and so forth. There's like a hierarchical level. It's not like, oh, boom, there we go. We got this thing. And this can be applied to anything. It's like, you can't just make a startup over nine to have a billion dollar company. You know what I mean? There's a process, right? Definitely. And so we can't just come in and be like, healthcare system changed. So ideally what would make sense where it's like, is there like a small town that we can run a pilot as you mentioned? You know, I don't want to give away the secret sauce of this person because he works very hard on these things, but I would mention that there are foreign countries that have databases that are much more welcoming and permissive for these kinds of things. For that crack, that would be like, so you have the software side and you have the hardware side. And then you have the regulatory side. So we were just talking about this and thinking, how would you launch something that would be, let's say stick with your example of preventive medicine? Yes. Like those are pretty difficult. I mean, just forget the regulatory for a second. Forget the hardware, just the software of like machine. There's also a lot of hype around AI. And I think you know this, but I feel like whenever you hear that word. Too much hype. It's not AI to begin with. I mean, it's funny. AI is what people have on slides when they're trying to pitch AI to somebody. Of course. Like even economists, we use regression all the time. That's in machine learning, but we don't like give it a sexy name. We just call it like, you know, ordinary least squares or panel date or time series, but like the public is like so enamored with this idea of AI. Like it's loops or it's, you know, other functions. I think when you sit down and tell people what, when people say AI today, so anyone listening whenever anybody's saying AI today probably 99.99% of time it's machine learning. Okay, but what that, what is that? The most algorithms that you see online or ML is literally just pattern recognition. That's all it is. It goes from filter one to filter two and down the line it goes. Recognizing logic. Recognizing patterns. Yeah. There you go. So yeah, it doesn't have wisdom, but it has the ability to move large amounts of data. Yes. That a human mind would not be able to look at like 20 million piece of data. Like you have, you know, put that on AWS and they can do that like split second. So I don't think AI is the penultimate solution. It is a tool. And going back to the radiologist thing, I asked him the same thing. I said, what about radiologists? Are they going to lose their jobs? He said, well, those that don't use technology are going to lose their jobs, which is true about anything. And so I think that we embrace technology versus trying to fight it. Like if it's better than human judgment, we should use it and be aware of its flaws, but we need to incorporate those things. I mean, but things, that's just what people are like that. Whatever it is, they're resisting forward movement and whatever it is because they're afraid or they're suspicious or whatever. And once those grounds are addressed, then I think that there's potential for things to evolve. The regulatory side, I think that if something's like political suicide, it's not going to happen regardless of how good it is for the public. I agree. I agree. And who's, you know, if all it takes is one screw up. I mean, look at what's happening with autonomous vehicles. Like how many people have gotten killed by autonomous vehicles? Like there was the one accident. Like I think it was in, it was in Texas. How do people get murdered every single day by people? Like your benchmark for perfection. A year is one million. I think it's like a million people die a year of car accidents. Our benchmark is inappropriate when we're looking at technology. People think zero is the appropriate number of mistakes, but like that's not the right number. The right number is what's the current best thing that the current system can do. And if we're doing a thousand times better than that with AI or autonomous,