Hello everybody. Welcome to my talk about container storage interface in Kubernetes. My name is Honza Szafranek. I've been working for Red Hat on Kubernetes for past four years, almost. And I was always working on storage in Kubernetes. Before I start, I would like to know who you are and what, so I can tailor maybe the presentation a bit. So if you could please raise your hand if you ever created persistent volume claim in Kubernetes or OpenShift. I expected more. And who is somebody who administrates Kubernetes cluster or OpenShift? Either local for local development or in production for testing, whatever. Not so many. Okay. So I will start with container storage interface. What it is? In a nutshell, it is interface between container orchestrator, meaning Kubernetes, meaning Docker swarm, Cloud Foundry, Mesos, and these guys and storage providers. Storage provider here is pretty generic term. It could be traditional storage like IceCasin, FS, or Samba. It could be software defined storage like Gluster, SAF, and many new startups. Or it could be cloud storage like OpenStack, Cinder, Amazon, EBS, Google, PD, and this stuff. The standard interface focuses on workload that runs as containers. That means that the guy who consumes the storage runs in a container. It is not limited to containers. You can use CSI without any containers at all, but some of the concepts come from container world. In Kubernetes, I have good news for the users here. There are no changes for you. You will still use persistent volume claims as you are used to. You will use storage classes. Everything will work. I have bad news for admins because the things that we traditionally run inside Kubernetes, inside Controller Manager, inside Cubelet, now we run them outside, and you will see new deployments, new pods in your cluster. And you need to maintain them, manage them, and you need to know what they mean. And the ultimate goal for Kubernetes is to remove all the storage from Kubernetes code base and run everything as CSI. We have just started. It will take a long time, perhaps several years, until we are there, but you should be prepared that all the storage will be CSI, and as cluster admins, you need to care about that. So, how CSI started? Once upon a time in Kubernetes, we started writing volume plugins. Volume plugins in Kubernetes is a piece of code that talks to storage. Throughout the time, we accumulated 22 persistent storage volume plugins, and every now and then somebody came and wanted his own storage plugin in Kubernetes. It quickly became obvious that we can't maintain all of them because we call them plugins, but they are not real plugins. You can't dynamically load a new storage plugin. All of them are hard-coded in Kubernetes code base. You need to follow Kubernetes coding standards. You need to send pull requests to Kubernetes, and you need to follow also the release schedule. If you want to fix a bug or change a new volume plugin, so we invented Flex, a volume plugin which allowed storage vendors to install drivers on the host on all nodes in the cluster, and Kubernetes would call this binary when it needed to talk to storage, and this binary would do the stuff. The problem with this approach is that the installation of the binaries on the host is not that easy. Now, people usually distribute software as containers, especially in Kubernetes. They don't want to install binaries on the host, and some hosts like CoroS or Atomic Host, they don't allow you to install anything on the host. We thought we were thinking how to improve this situation, and we started implementing Flex 2.0, which was gRPC-based. You could run the storage driver in a container, and it would be much better. Somebody got a smart idea, so why don't we talk to Docker Swarm? Why don't we talk to Crowdfundry, MazeLos, and other people who run containers? They must have the same problems we have, and that's how CSI was born. CSI, on the high level, it's a project on GitHub. You can join the project, you can download the specification, you can implement it. There are no fees, no patents, nothing. You can create issues on GitHub, you can send requests if you want to extend the specification. There are no barriers. There are weekly meetings, I don't join them because they became slightly political. Every storage vendor has its own cool feature that nobody else wants, and that's super important to him, and nobody else cares about the feature. They want to push it to CSI. Also, storage in Kubernetes and MazeLos and Crowdfundry and Docker Swarm, they look completely different. In December last year, we released version 1.0, which is declared stable, and we will maintain compatibility with this version. If you run older Kubernetes, we implemented 0.0 and that's not compatible with 1.0. The incompatibilities are not huge. We usually rename some fields here and there to make it more obvious what it means. But still, if you have older drivers, you need to adapt them to version 1.0. On technical level, it's GRPC API between container orchestrator, for purpose of this talk, it will be Kubernetes, and the server guy who does all the work is a storage plugin. In CSI, the storage plugin is used in the CSI specification, but since in Kubernetes, we have plugins, we use term plugin for something else, we call it CSI, we call them CSI drivers. The communication between them is usually in over-unit domain socket, and as the protocol says, implies GRPC, it's some remote procedure call, so Kubernetes just calls functions of a driver. Driver does something useful and returns some value. CSI standard doesn't define anything about how to distribute the plugin and what the plugin is, if it is installer or it is executable or it is RPM package. In Kubernetes, we expect that these are containers. There is a freeing community of storage vendors who implement drivers, who started implementing drivers. In Kubernetes CSI, we maintain a list of the drivers we know about. If you have any driver that's not listed here, send us a pull request, we have no, there is no entry. We don't check the quality, we don't check the support, and actually the quality of the drivers is very different. I would say most of them, like alpha or beta quality, I wouldn't run them in production, but if you are a storage vendor, now is the right time to start testing your driver. So back to Kubernetes. How we implemented CSI in Kubernetes? We have 22 voting plugins, now we have 23, one of them says CSI. We started with alpha in 1.8, I think. It is enabled by default in Kubernetes 1.10, and it was declared stable in Kubernetes 113, released last December. If you use your OpenShift, you need to wait because it won't be in OpenShift 4.0, you need to wait for 4.1, maybe 4.2. While the implementation is declared stable in Kubernetes, we don't have all the features, we don't have block volumes yet, we don't have resize volumes yet. If you used alpha implementation of snapshots in Kubernetes, we completely rewrote that, and we have another alpha implementation of snapshots in Kubernetes, which is much better, actually. And if you want to use CSI, you must use it through persistent volumes and persistent volume claims. You can't use them in line imports yet. For the users, I have a very short demo. I have a cluster running on Amazon. It's OpenShift 3.11, and this OpenShift doesn't know anything about Amazon. It doesn't have credentials to Amazon. But it runs a CSI driver, and the CSI driver is the guy who knows about Amazon. So I have a claim. It's the claim you are used to. It requires 500 megabytes of some storage. It doesn't say which driver, what kind of storage. It doesn't matter for the users. I create the claim, and I can see that Kubernetes, when it ends, and talk to the CSI driver, and new volume was created just for my claim. If I look at the persistent volume, it's pretty standard persistent volume, just as the volume source is CSI. The driver that's responsible for handling of this volume is something that calls itself EBS, CSI, AWS.com. And another important field is volume handle, which is ID of the volume. The ID is interpreted just by the driver. Kubernetes treats it just as string and pass it around. And if you just create a pod in the pod, this is some busy box that does nothing useful, but it uses the volume, the persistent volume claim. There is nothing special in the pod, and the pod is running.   Yeah, you're welcome. Why it's so complicated? Because the parts that were traditionally running on the master, we had our volume plugins running on master that did the dynamic provisioning, that did third-party attachment on Kubelet, we had the code in Kubernetes that found the block device, created file system, and mounted it. These parts are not outside of Kubernetes. So to install CSI driver, the best thing I can recommend is to go to your storage vendor and ask how to install it. Storage vendors knows or should know what daemon set and status set and service accounts and secrets and RBG rules you should have to install their driver. But since CSI is quite early stage, there are emerging patterns how to install a CSI driver and some vendors get it wrong in the initial phase. It will get better, don't worry, but some of them are pretty stubborn. I don't have a demo of installation, but I can show you how Amazon CSI driver installation looks like. There is a read me somewhere that tells you, fill this secret with your credential to the Amazon and then create these objects. And that's it, that's the whole installation. How does it look like? You create these objects and it creates bunch of service accounts, cluster loads, status sets, another service account, daemon set, and whole zoo of objects that are necessary to run a CSI driver. I will go through the components that are necessary to run a CSI driver that must be in every CSI driver installation that will run in your cluster so you know they exist and especially why they are there. So the first thing you need to have to run a CSI driver is external provision. Traditionally, dynamic provisioning of volumes was done by master which directly talked to the storage API, for example, Amazon API or to equity in cluster. Now master does not know anything about storage, so you must run piece of the controller outside of master and this piece does external provisioning. Each CSI driver needs its own instance of external provisioner because they can have different timeouts, different parameters. It's better to run them separately. What the provisioner does is that it watches for an API server for persistent volume claims. If it sees a persistent volume claim, it sees that there is a persistent volume claim that needs provisioning and it's me who should provision it and it calls the driver that's running in the same pot as the provisioner using the gRPC over unix domain socket. It calls the CSI driver does whatever is necessary to create a volume and in the end the provisioner creates a persistent volume. You don't need to know the details, you don't need to know the score flow but you should know why the provisioner is there and why it's necessary. If your dynamic provisioning doesn't work, you should know where to look. So if something doesn't work, which is pretty often in this early stage or if you are developing a CSI driver and something doesn't work, you should look. The first thing you should do is to describe, describe, come on, I can't write, your persistent volume claim because that's the place where external provisioner sends events. In this case, I can see that the external provisioner started provisioning and it successfully provisioned something. If the provisioning went wrong, you will see errors here in the PVC. If it doesn't show anything useful, then you go to locks of the provisioner and your driver and so on. This should not happen if the driver is mature enough. The second thing we need to run in the cluster is external attacher. This external attacher, again, before it was implemented in control manager on master that directly went to storage API and did third-party attach of volumes. Now master doesn't know anything about storage so there is external attacher that does piece of that logic. Only few volume types need third-party attach, usually in the cloud where you have API that you can say Amazon, hey, I have a note and I have a volume, please attach the volume to that note and the guy who asks is not the note and Amazon goes and attaches the volume to the note and block device with the volume appears magically inside the note. But if you run, I don't know, NFS or ISCSI, fiber channel, raster, safe, you don't have that API that attaches remotely volumes to notes, but you still need to run the attacher. That's the most important thing. At least half of the issues we have on GitHub or Slack are that my CSI driver is not working and the first question we ask is do you run attacher? No, my volume doesn't need attacher. I have simple NFS. You still need to run the attacher because, as I've said, master doesn't know anything about the storage. Master doesn't know that there is no need for attacher right now. We are working on a fix, but right now you need to run the attacher. Again, how it works, it's pretty complicated. You don't need to know the details, but if something goes wrong, you go to CSI attacher logs. If there is nothing, you go to control manager on your master. Attachers does the third party attach, sorry, repeating a question. If attacher is thing that does mount on each host and runs this node, attacher does this only third party attach only in the clouds. The component that runs that does mount, I will have a slide later. Back to the attacher. If something goes wrong, there is a new API object called volume attachment which says that the controller inside Kubernetes doesn't know anything about storage. CSI will plug in and the plug-in has no interface to CSI driver intentionally because on some Kubernetes deployments, you can't install the driver on the master. You must run it somewhere else. So the volume plug-in just creates volume attachment instance. The instance tells please somebody out there attach this volume to that node and the external attacher running somewhere in the cluster sees that there is a new volume attachment object. It checks, hey, I should do something, I should attach volume. It talks to the CSI driver over the domain socket and CSI driver does the attach. So I have a pod running and I have a volume attachment for that and the volume attachment in the spec part says who should attach something, the CSI driver, what should be attached, this volume and where to this node. And the attacher does that and fills the status. In this case, it was successful, volume is attached, no errors. If there were any errors, you would see some messages here. And the second thing that you can debug is pods. Here you can see attachment succeeded, but if it's not succeeded, you would see some events here, why and what went wrong. So again, even if your volume kind is not attachable, doesn't use for the party that you still need to run the attacher. And the last component we have is called node driver registrar. And that's the thing that runs on every node and it registers CSI driver to Kubler. I have a picture here, it's taken from OpenShell documentation because I'm bad at drawing pictures. So on the node, on each node, you have pod that runs the driver and here the month happens and also there is a driver registrar that registers the driver to OpenShell node service which is fancy name for Kubler. This is Kubler. But Kubler talks directly to the driver using the main socket and it's the driver that does the mount on the node. If you have Iskasi or NFS or basically everything needs to be mounted. So you run the demon set that installs the driver registrar and the driver on each node to have CSI and somewhere in the cluster you need to run a pod with external attacher, external provisioner which talk to the copy of the driver. Are there any questions? Because that's the complexity. You need to run external components, you need to know what they do, why they are there and if something happens where to look. So the question is if, in case of Amazon, if the volume is directly attached to the host or there is something in the driver that translates it somehow to the pod. And the answer is no. CSI does not stand in the data path. So the question is if, in case of Amazon, if the volume is directly attached to the host or there is something in the driver that translates it somehow to the pod. And the answer is no. CSI does not stand in the data path. So this is a single node cluster and what external attacher does, it attaches the volume directly to the node. So I can see xvdbx is directly available as a device on the node and what the driver on the node does then, it mounts it directly xvdbx. So the driver running on the node then mounts it wherever Kubernetes needs it. And it stands in the outside of the data path. It doesn't translate the data in any way. It just mounts the thing to the container and container sees it as a mom phone. Any other questions? There? The question is if there is some way how to limit IOPS because in Kubernetes there is no support for limiting IOPS. And the answer is no, not yet. Petchies are welcome. There is no global option. I know that in case of Amazon you can require volume with distinct IOPS and that's implementation of the driver. But there is no global option for all drivers. Anything like that. Any other questions? Question about node failures? Well, I'm not sure what you are asking about. If node fails then would you care about storage? Storage is gone. If a node fails then eventually all the pods from the nodes are evicted and since we can use third party attach and detach, we can detach the volumes from the node. And something in the cluster deployment, state-of-the-art set, whatever, starts new pods and the data will be mounted to these two, the pods will be scheduled to some other node that's running and the data will be mounted to the node. There is no difference between CSI and the Kubernetes today. It was the same. And there was another question. One module. What is module? Yeah. You can run, the question was, what if I have more CSI drivers like NSRS, Ice-Cuzzie, whatever? And the answer is you can run as many drivers as you want and each volume, each persistent volume in the cluster has a field that says which driver handles this volume so Kubernetes knows whom to call. And the dynamic provisioning happens based on storage class and each storage class has one CSI driver behind. Okay, I collected some mistakes that people usually do when they install CSI drivers. Maybe they are not mistakes per se, but I think there are better things how to make it. As I have said, you must run external attacher, external provisioner somewhere in the cluster in a pod and it should run just once. One provisioner should be running at the time and one attacher should be running at the time. They are running for the driver. One driver needs one attacher, one provisioner. People use Statefacet for that because that guarantees that only one attacher runs at the point. If you use deployment with one replica, it can happen in some corner cases during deployment, during updates that two replicas are running at the same time. So people use Statefacet because it guarantees that only one replica runs. That's true, but if your replica is on a note that goes down, it may take some time until the note is healthy. Until Kubernetes discovers that the note is unhealthy and moves the pod somewhere else. It can take up to five minutes. And during these five minutes, you are without dynamic provisioning, which is probably okay, but you are without attacher. You can't attach volumes. You can't start new pods. In this case, I recommend use deployment with multiple replicas and use leader election. All the components we have, the provisioner and attacher, they both have leader election. So they make sure that only one replica is running. And if one replica dies because it's on unavailable note, the new leader can be elected in a few seconds. Other things I noticed, especially in the Omplicev case, Kubernetes uses the CSI driver on the note. It needs to mount volumes. It needs to mount volumes in Warlib Kubernetes. That means that the driver pod needs to have mount propagation and everything for Warlib Kubernetes. But if you run OpenShift, we don't use Warlib Kubernetes, we use Warlib Origin. So if you use non-standard OpenShift Kubernetes distribution, like OpenShift, beware not all CSI driver vendors know about that. In OpenShift for a while, we will have Warlib QLED. Finally, these sidecar containers like extranet hr provisioner and driver registrar, they are tightly bound to Kubernetes version. So if you update your Kubernetes, you must update also these sidecar drivers. There is a table in our documentation that says it was here yesterday. Here it is. What version of Kubernetes needs what version of, no, it's a CSI spec. It's not the driver. Sorry. Somebody merged the, somebody merged a bit of documentation, which is good, actually. The CSI itself changed a bit, but unfortunately in combatable way. So if you have driver that implement the older version, you need Kubernetes 112. If you have newer version, you need Kubernetes 115. The most important, some people don't run Atecher and surprise, please run external Atecher, please. And since now you have pods in your cluster that care about storage, you must make sure that there are always running. So you must give proper priority of this pod. So if somebody schedules important workload, it could happen that your, that a pod without priority is killed, and you don't want that happen to your storage. So please run the pods with high enough priority so they are not killed accidentally. What's the current status in Kubernetes? The deployment is not trivial. We are working on that. Maybe it will become easier, but in the end it depends on the storage vendor because the storage vendor knows what it means to install a driver and creating some Kubernetes object in the end is the last step. Usually you need to prepare the host, open firewall ports, I don't know, kernel modules, whatever. Each storage is different. So you should go to your storage vendor. The community around is very active. It doesn't look like that, but we are moving forward quite quickly. You have seen beta in 110, stable in 113, that's nine months. It doesn't look like quick, but it is pretty quick in Kubernetes.  We are working on the documentation. There is a link to a pull request that improves the documentation. And if you want to join the community, contribute something either on the Kubernetes side or the driver side, this is an excellent starting point because you can review documentation, fix typos, fix the grammar. If you don't understand something, you can fix it. And while reviewing the documentation, you will learn a lot about CSI and how it works. In Kubernetes 113, we introduced alpha support for volume topology. That means that if your cluster is not uniform, for example, in Amazon or in cloud, you have availability zones and regions. And some volumes are available only in some regions or in some availability zones. Now we have a way how to show that in CSI. So each volume can be in different region and they can be attachable to different nodes. And now Kubernetes knows about that. So if you have a volume in zone, I don't know, US East 1, then Kubernetes will go and schedule a port in US East 1 where the volume is accessible. And we are trying to remove the requirement that you need to run extranetature for non-attachable volumes. It is alpha in 113, but still you need to, if you run without alpha features, you need to run extranetature. On the roadmap, we have a lot of cleanup and stabilization to do. We want to implement resize and block volumes. It should be pretty easy. With heart, we want to do inline volumes in ports. So far, CSI was consumable only through persistent volumes and persistent volume claims. But for some kind of volumes, for example, in Kubernetes, Secrets, Config Maps, then Word API, you don't create persistent volume for Secrets, right? So we want to have something like that in CSI where you directly in a port specification, you reference a CSI volume, and you can implement something like Secrets that downloads the actual credentials and Secrets content from, for example, HashiCarp vault, HashiCarp vault or somewhere else, then Kubernetes API object. Or you can inject, I don't know, Kerberos tickets or whatever into your ports. And this is quite difficult. And by the way, volunteers are needed. We want to do some performance testing for drivers. That means currently we run our Kubernetes N2N test only on Google Cloud and only in Amazon. That means we can test only GCE and AWS CSI drivers. And we want to create a binary or something with this N2N test so you can download the binary and run our test, our Kubernetes test, in your data center with your CSI driver with your storage. And the ultimate goal is to remove the storage from Kubernetes. You would like to do that. And how do we want to do that? The current plan is to do it seamlessly. So you have Kubernetes that's using internal volume plugins. You're happy with that? You update to new Kubernetes version and magic happens and instead of internal volume plugins, you will use CSI. That's the goal. We are very, very, very far from there. There are some obstacles because we want to upgrade seamless. But Kubernetes has also policy that downgrade must be seamless. So we upgrade to new CSI version, new CSI, everything is working. And you discover a bug or something that's blocking you. So you downgrade and the downgrade should be also seamless. So from CSI, you downgrade to something that's internal. And that's where we have issues, actually. We don't know how to do that. Maybe we will have something in alpha in 1.14. I'm quite skeptical, but you will see. And another thing that's blocking it is that Kubernetes API is stable. And we have quite strict policy about that. So if I have person volume that uses Amazon API, then this API object, this person volume, must be working also when we move the storage parts from Kubernetes to CSI. So we will need some translating library and this kind of stuff. It will be fun. If you want to contribute, this is pretty challenging. If you are bored with maintenance or something, this is challenging. This is new development. This is fun. Until we have the migration, the automatic migration, if we have it, I don't know, you can do it manually. If you have person volumes in your cluster, here's the way how to do that. Please back up. Maybe you will need it. It should work if it does not please report bugs. There are not so many people who try that. Basically, you have a cluster with your person volumes. You must stop everything that uses the person volumes. You delete the person volumes and replace them with the CSI counterparts. In the API, you can't edit the person volumes right now. This is prohibited by validation. Sorry, we can't do that. So you must delete it and create new one. And if you do that, it should work. I'm quickly approaching the end of my presentation. So CSI, it's GRPC interface. It's focused on containerized workloads, but you can use it outside if you want. For users, no changes. For administrators, some headaches. I think you will get used to it. It's just few posts running somewhere. Nothing special. And storage vendors don't need to go to us and ask for new volume plugin. They can implement it at their own pace, at their own release schedule, whatever. We don't care in Kubernetes. Here's a link to the specification. It's open. You can create issues and pull requests, add new features as you want. But if you add new features, please remember there's somebody who will need to implement it on Kubernetes site. And I don't want to implement crazy features. We have a Slack channel on six storage and we are looking for volunteers. We have on Slack workgroup CSI. It's pretty demanding. There is a steep learning curve, but it's fun. And please remember to run the external literature. That's literally, it's half of our issues. Every day, somebody asks on Slack, my CSI doesn't work. Please run the literature. Are there any questions? Yes. CSI to migrate to CSI from normal plugin. So the question is, what are the advantages of CSI? For cluster administrators, there are none. For me as Kubernetes maintainer, I need to maintain these storages as internal volume plugins. I have no clue. Sorry, these are CSI drivers. These are volume plugins. I have no clue. I can't test it in Kubernetes. So as a benefit for me, I don't need to maintain these beasts. I don't know what they do. And also, what works and storage OS, these are startups. You don't know. Maybe next year, I don't know. We have one plugin that doesn't do anything. Nobody knows if it's working. And the major benefit is for storage vendors. They can plug in their storage to Kubernetes without any code in Kubernetes. We don't accept any new volume plugins at all. That's the only way how to plug your storage. Any other questions? So the question is, if the development of Kubernetes volume plugins stops and we won't be adding new features, and the answer is sort of, we implemented, for example, block volumes recently and topology of our volume scheduling. And we implemented them for internal volume plugins first and CSI follows. But at some point, the CSI implementation will get the new features. And I think that CSI will start getting the new features. And the old volume plugins will be just, it will be there. It will be supported. We will fix bugs, but we won't add new features. For example, we have snapshots now. And snapshots are supported only in CSI. We don't implement snapshots for the old volume plugins. Please use CSI and don't use entry volume plugins. Any other questions? Yes? So is it compatible with the CSI version or Kubernetes version? The question is, if the CSI driver is compatible with CSI spec or Kubernetes. The goal is that the driver should implement some version of Kubernetes spec, either pre 1.0 or post 2.0. And it should be compatible with all Kubernetes version which supports the spec, ideally. So far, we haven't seen any troubles with that. I can imagine it will break up at some point, but drivers should follow the CSI spec and test on Kubernetes. Does it answer your question? Any other question? If not, please install CSI drivers. It's fun. It breaks your cluster in very strange ways. You are not accustomed to, but yeah, that's the future. Sorry.


How are you doing? This is your boy Rich from Rich TV Live. Join our website at RichTV.io, an ecosystem built by investors, for investors all over the world, and pretty mixed day in the markets today. US stocks did open higher after another encouraging inflation report. I'm not going to say it was encouraging. It just wasn't as bad as what we expected. Still pretty high inflation at 8.5%. Fed officials say more rate hikes are needed despite slowing inflation. We are not out of the woods, but we're going to be watching the markets very, very closely and very, very carefully. US retail gasoline price average falls below $4 a gallon for the first time since March. We are finally getting some relief with the pumps and that's exactly what we need. Dow closed off slightly green, only 27 points. So pretty choppy day. Nasdaq mostly green most of the day, but did finish slightly red 74 points. TSX did close off green. The Toronto Stock Exchange up 105 basis points. US PPI slows more than expected. That probably did affect the market a little bit. PPI inflation rises 9.9% in July. Stock futures did rally. The ones to watch today were Rivian, SIX, G-O-O-S, V-E-R-U, all earnings. We're going to be watching very, very closely. We're going to be watching Rivian, SIX, G-O-O-S, and V-E-R-U earnings. So stay tuned. Those will be out and you'll be seeing how that responds tomorrow. So calendar events for August 12th to keep your eye on. So you're going to be definitely watching all of this news for Friday, preliminary UOM consumer sentiment. This is very high impact news. So keep a close eye on this for Friday. Financial confidence is a leading indicator of consumer spending, which accounts for a majority of overall economic activity. So very big numbers coming out tomorrow. Mortgage delinquency numbers come out. This is also going to be very important to see how the economy is dealing with interest rates going up. And this is a low impact news, but good to keep an eye on for some insight. So definitely keep your eye on that on Friday. Cryptostats. We want to go through the crypto stats. We're going to go through some of the charts right now of Bitcoin, Ethereum, and Solana, which happened to be my top three cryptocurrency holdings. So let's go through that right now. All right. All right. So we are looking at Ethereum. And you can see Ethereum has hit a low in the month of a thousand hit a high just recently over the last 24 hours of $1942. So up almost a hundred percent in a month for Ethereum, obviously that is very bullish. When we look at Bitcoin, it's been as low as $18,900 in the last month, currently sitting right around $24,200 and hit a high of $24,900 in the last 24 hours. So very bullish for Bitcoin. Solana hit a low of $31, hit a high of let's say right here, $45.90, currently at $43, looking really, really bullish for Solana as well. If it can stay above that $43 mark, that's going to be very bullish for Solana. And I really believe Solana has a chance to be a multiple for investors. So we've seen Bitcoin go up to $24,000 and every time it gets to that $24,200, $300,500, $24,800 mark, even hit $24,900 seems to get some resistance. So we're going to be watching that very, very carefully. It has tested the $25,000 mark, but it got rejected very quickly. So $25,000 seems like a good sell zone for investors on Bitcoin. Ethereum did hit $1,997 trying to get back to that $2,000 level, but every time it gets above that $1,900 mark and it got to $1,942, it seems to sell off and goes back under $1,900. So we want to see it stay above that $1,900 level so that we get some juice and see it rip through $2,000. Solana trying to stay above the $43 level. We'll see if it does, if it does, then we think we may see it break 50. So what do you guys think about the news? What do you think about the analysis? Love to know what you guys think and love to know what you guys think about our daily news and analysis 24 hours a day, seven days a week. You can join our ecosystem at richtv.io and join our community and our chats where we're talking about stocks, cryptocurrencies, NFTs, and so much more. We'd love to hear from you guys. Comment down on the video. If you haven't subscribed, subscribe for future updates. We talk about stocks, cryptocurrencies, NFTs, real estate, metaverse, and so much more. If you're not winning, you're probably not watching.  If you're not winning, you're probably not watching. Thank you guys for watching Rich TV Live. I'm out.


So my name is Gianluco Mardente. I'm a principal engineer at Cisco Systems. I'll keep this very brief. It's lunchtime. I want to thank you all for staying with us for the whole time. We had amazing sessions. And personally, I enjoy all of them. That's kind of rare, but great sessions. And I did learn a lot. If you have any type of feedback on what was right, what type of content that you would like to see in future, please submit those feedbacks, because the multi-tenancy con wants to keep improving and being a great component. And as I mentioned, that's it, like it's lunchtime. So I want to again thank you all for staying with us. Thank you all our amazing presenters, the organizer and sponsor. So I enjoy Cubicon. I kept it like very short. So thank you so much.


So, there have been detailed theoretical calculations for NOE, we will not go into those detailed theories, but I just want to show you the results, NOE as I said this will depend upon the relaxation times and this will depend upon the molecular motion. How fast the molecule is tumbling inside the solution and that is characterized by a particular time constant known as the correlation time. For small molecules correlation times is of the order of 10 to the minus 10 to 10 to the minus 12, this is for nanoseconds, seconds are nanoseconds therefore 10 to the minus extremely small time, seconds so that means is the order of nanoseconds and for this is for small molecules and correlation time can be of the order of 10 to the minus 8 to 10 to the minus 9 for large molecules. So, this basically reflects how fast the molecule is tumbling inside your solution and the relaxation times will depend upon this. Now, according to that so the theoretical calculations on NOE have been done and it turns out that the NOE is equal to if we want to represent this as NXA and A is the one which is perturbed in the earlier case I wrote it as NiX, A is the one which is perturbed and NX is the X is the one which we are going to monitor and this is typically equal to half gamma A by gamma X where gamma is the gyromagnetic ratio and it all depends upon the individual spins also and this will be different for different situations, this is for this condition, this is for this condition. So, therefore what will be the kind of NOE, this is the steady-state NOE, steady-state NOE means I do an experiment in the as I indicated to you before, the experiment is done in this manner that I have a FID collected here and I do a pre saturation perturbation, I do a perturbation here and one particular line on a particular line let us I do it for the spin A and I monitor the spin X here. So, when the perturb here I monitor the intensity and without the perturbation I monitor the intensity and then I take the ratio get the NOE and this will now by calculations one can show that this is proportional to half gamma A by gamma X, what is the advantage here?  It is 4. So, therefore and this NOE will be a factor of 2, if gamma A is equal to gamma X then of course it is a factor half. So, this is 0.5, this is an enhancement, how much is the enhancement in the intensity of the lines then when you perturb one spin and observe the other spin and if you are perturbing proton and observing carbon 13 then means you are getting a factor of 2 enhancement in the intensity of the line. So, and that is the significant advantage, that is the significant advantage because the factor of 2 enhancement meaning it is like a factor of 4 in terms of the time with respect to the signal averaging. So, therefore the steady state NOE, this is the steady state NOE which we call steady state NOE and in the this limit is called as the extreme narrowing limit. This is the small molecules very rapid motions and this is also called as extreme narrowing limit. Here the tumbling is very rapid, the motions are very fast. So, the averaging happens much much better and the relaxation times are also very different. So, therefore without going into the theoretical detail this is to just to show you what kind of a thing is it will look like. Now what will be the situation when we are in this larger molecule range? The large molecule range it turns out that this is equal to minus gamma A by gamma X for large molecules NXA is equal to minus gamma A by gamma X. So, therefore you can see for proton if A is equal to X then this NXA is equal to minus 1. And if I want to plot here the NOE as a function of the correlation time here it will be it will look like this and this is my 0.5 and this is my minus 1. So, this is the case for homonuclear systems. So, you have minus 1 so the negative NOE you will get this is called as negative NOE. But nonetheless it is it is a kind of a significant change in the intensity and when you take the difference you will get a substantial enhancement and for large molecules you will get a minus 1 as a this one. But if it is not homonuclear then of course you are gamma A by gamma X that is proton and nitrogen or something then you get a factor of half also and then you will get minus 5. So, you get heteronuclear once of course you will have a different number for it will be minus 4 something like that minus 5 for proton or nitrogen because this is well nitrogen has another complications in the sense of the Garam ironic ratio for nitrogen is negative. So, therefore, there even for the small molecules there that is the gamma A by gamma X in the case of nitrogen it will be negative. So, this is what I showed here in the previous slide when I said half gamma A by gamma X you see here if you are talking about nitrogen proton nitrogen NOE and then let me also draw that here if let us say A is equal to proton then X is equal to a nitrogen 15 then this nitrogen 15 has gamma of nitrogen is negative. So, therefore, you will get a negative enhancement there and that will be a factor of 5 because gamma A by gamma X there is 10 and therefore, divided by multiplied by half then you will get a minus 5. So, even in the extreme narrowing limit when you are talking about heteronuclear NOE proton nitrogen it is minus 5 and proton carbon it is plus 2. So, these are the factors one has to remember one studying these experiments in the steady state NOE. Now there is another experiment so far I talked about what is called as the saturation pre saturation this is called as one saturation was done on the spin A and we are monitoring the spin X. There is another way one can do and that is called as transient NOE. Transient NOE what you do here is your perturbation will be like this you do the experiment like this let me draw this by a different color this one I will put like this. So, this is a selective perturbation selective inversion and this is a hard 90 degree pulse this is a 90 hard pulse what does that mean? So, then let us say I have a spectrum something like this. So, what I will do is I will selectively invert this line I will selectively invert this line after 180 degree pulse this is 180 degree. So, I will selectively invert this so then I will get here this one like this and this remains like that and whatever happens here and I give a time here called tau m and tau m is the is the called the so called mixing time what I have achieved here let me explain what we have achieved here. So, this is the experiment what happens? So, let us assume that I have the initial magnetization of all the spins all of these have different magnetization components right let me draw the X, Y, Z. So, draw the magnetization of this particular spin here and the magnetization of the other spins also will be here let us say. So, during the period when I apply selective 180 degree pulse what I am achieving is what I am achieving is I am inverting this magnetization I am selective applying a selective 180 degree pulse on the black line which means the magnetization which was here is now like this when you measure it of course I will see it as a negative signal now this is the perturbation this is the perturbation this perturbation has to recover back to equilibrium the system will have to recover back to equilibrium what it will do it will relay this perturbation to some other spins which are close by in space. So, then this will relay so as it recovers as it recovers like this it will relay this magnetization to some other spins.   So, therefore this will show up when I measure it it will show up in this some intensity of the red one if this were a red one that would have gone down or it would have gone up either way this can happen for both positive and negative NOEs you saw already that for small molecules it is positive NOE which means that is an enhancement for large molecules it is the negative NOE means that will decrease intensity will decrease. So, therefore there can be changes in this other portions of the spectrum as a result of the of the inversion here and I give certain amount of time here tau m because I am allowing it the magnetization to transfer this is this is dependent on the relaxation time depending upon the relaxation time t1 the efficiency of transfer will vary. Therefore, this can go to different ones as a result of the dipolar coupling between the two spins. So, therefore when I measure this I will get different intensities of these lines here not all of them but some some I will get some I do not get no something will be perturbed some will not be perturbed. So, now if I take the difference here if I take the difference between these two this fellow has not perturbed this has perturbed this has perturbed and maybe this is not perturbed let me let me take it up there this is not perturbed. So, then I take the difference difference is equal to what I will get I will get a thing here not much larger let us say this minus this then I will have a 0 there and this I will see minus there because this is increased this is small and this also I get a minus here and then I get 0 here again no perturbation. These are the perturbed ones these are the NOEs and this will depend upon the mixing time this will depend upon the mixing time how much time do I give why does it depend on the mixing time because it depends on the relaxation time of the perturbed spin and the relaxation times as other ones also. Now once I have done this will it always remain there will it remain at that point only question is let us say I have a molecule like this and I have a proton here a proton there a proton here a proton here a proton there so on so forth I perturbed one of these fellows I perturbed this fellow then it will relay from here to here because it is close directly during the T1 T1 but this is also now perturbed as a result of this this is also perturbed therefore it will also have to relax what it will do it will also transfer to something which it is close by so let us say I have I take that so from here it will go here one more or it can go from here to here or it can go from here to here and so on so forth. So what is this phenomenon called this phenomenon is called as spin diffusion. Spin diffusion means the magnetization which is perturbed at one point it will keep diffusing through your network of coupled spins and that will obviously depend upon the time tau m so this will depend upon proportional to tau m. Larger the tau m you give longer will be the spin diffusion and therefore if you restrict the tau m to very small value then it will go only to the directly neighboring spin so if I want to plot so if I want to plot the NOE as a function of tau m then it will go like this it will come down. So this is the linear this is called the linear regime during which time it is directly proportional to the neighboring spin only going to the neighboring spin only and it is in this region one can calculate inter proton distances from the NOE intensity okay the here the NOE intensity is proportional to what is called as the cross relaxation rate which is represented as sigma and this is proportional to 1 over r ij to the power 6. So if I say sigma ij so this is proportional to 1 over r ij to the power 6 in the in this approximation in this region in this region it is proportional if we go beyond that because the intensity NOE intensity is no longer just restricted to that it will start diffusing to other ones so the relaxation individual coupling between all the spins will come into the picture and you may not be able to do it. Therefore transient NOE is an experiment which is done with a controlled manner you can control the extent of diffusion that can happen from one spin to another spin and that is the extremely useful parameter for calculating the structures of molecules and this is the basis of structure calculations this is for short tau m okay and there it is restricted to the near neighbor interactions only for short tau m the transfer is is restricted near neighbor interactions therefore so polarization transfer has two objectives right so two objectives polarization transfer by NOE has two objectives so therefore NOE in summary of the time NOE has two gains one is enhance signal intensity less sensitive nuclei this is like carbon 13 nitrogen 15 and the second is derive structural parameters macromolecules so these are two important applications of NOE and we will not go into the theoretical details of this much more than this I think that is that is required for our purpose of structural biology which is a focus of this because otherwise it can get very complex okay so now let me take one more topic okay so we will take another topic here which is called as selective population inversion consider a two spin system AX A and X this can be heteronuclear mnemonuclear also and what I will have I will have here a simple 2x 2x spin system will have A will have two lines and X also will have two lines okay let me draw the energy level diagram for this I will have four states let's say the population of this is delta plus delta population of this is minus delta plus small delta and this one is delta minus delta and this one is minus delta minus delta okay so now I call this transition as A1 this transition as A2 call this as X1 and this as X2 okay now let me draw the intensities of the A transitions intensity of the A transition is population this population minus this so therefore that will be 2 delta and similarly the other one also A2 so if I call this as A1 this is A2 and both these have intensities of 2 delta and what about X1 and X2 this is this minus this so I will have smaller intensities and what is here delta is greater than delta okay and this is 2 delta and this is also 2 delta and this is my X1 and this is my X2 so now what I do is I invert this transition invert selective population inversion this is what I say selective population inversion so now what will be my intensities of the four levels now I will have to draw this levels again here this population becomes this so this will be minus delta plus delta this will go there delta plus delta okay I am inverting only this I am not changing anything else so this will be delta minus delta and this will be delta minus delta I am not doing anything here I am not perturbing these populations at all okay now I see what happens to the transitions what happens to the transitions the four transitions these were my A1 A2 transitions and the X1 X2 transitions are these X1 X2 what will be the intensities here let me draw the A1's first and this is now minus 2 delta okay the A1 will become minus 2 delta change the color here okay this became minus 2 delta and this one is this minus this this will be plus 2 delta okay so this is my A1 and this is my A2 what about the red ones red ones are these so delta plus delta minus of delta minus delta so minus delta plus delta minus delta minus delta so that is minus 2 delta plus 2 delta so this is minus 2 delta plus 2 delta and the X2 is 2 delta plus 2 delta 2 capital delta plus 2 small delta right so that is this plus this minus this so 2 capital 2 delta plus 2 delta see look at this what was small 2 delta here 2 delta here has become this large quite a substantial enhancement in the although the gain is not the same in both cases that is but the nonetheless each one of them has gained substantially in terms of the intensity of the lines you have at the cost of the magnetization of the A1 transition you have got the enhanced intensity for the X transitions enhanced intensity for the X transitions notice here these A1 and A2 transitions have become completely opposite to each other anti-phase this is called as the anti-phase they have become anti-phase the A transitions have become anti-phase because they inverted only one of them and therefore this kind intensity is changed so one is minus 2 delta other one is plus 2 delta in the earlier case both were 2 delta 2 delta and I have inverted one of them therefore that became one of them became minus 2 delta other one became remain the same as 2 delta and with the X1 and the X2 transitions which were initially both equal as 2 delta and 2 delta now one of them has become minus 2 capital delta plus 2 delta and the other one have become plus 2 capital delta plus 2 small delta and this is substantial gain in the intensity right. So therefore this obviously will depend upon how much is the individual difference between capital delta and small delta and that is dependent on the gamma of the gamma A gamma of A and gamma of X okay so that is proportional to the ratio of the gyromagnetic ratio. So the gain is proportional to this is this depends upon upon gamma A by gamma X okay so the gain will be proportional to the ratio of this gyromagnetic ratios of the two so substantial gain can be obtained if it is heteronuclear for single homonuclear systems you may not be that much of an advantage but heteronuclear it will be great advantage this principle is used in other techniques which will take up next time and that is particularly useful for all multidimensional multiples experiments so the important thing of course to notice here is that you may call this as a disadvantage while there is an advantage which you already discussed the disadvantage could be that you have positive negative signals you can say that disadvantages would be you have one positive and negative signals second you cannot decouple A and X because if you decouple the two will collapse when the collapse you will get back the same intensity okay only when the coupling is present the two lines are separately observable then you gain the advantage so therefore and you want to get rid of both of these kind of situations and that leads us brings us to other techniques which is which is called as insensitive net and of course in the third disadvantage also could be selective inversion requires long pulses requires long pulses that means it will not be 10 microsecond pulse it may be millisecond pulse okay if depends on the selectivity what you will want to observe so the better the selectivity you want the longer the pulse length will be therefore and this is a difficult thing to achieve and then the four selectivities selectivity may be difficult complex spectra so while the gains are there with regard to the intensities you also have certain disadvantages here to get over this we go into some other techniques which are called as inept and that will be taken up next and in fact that will be the most crucial thing for all multidimensional NMR experiments so I think we will stop here.