Hello, everyone. We're very happy to be in Vancouver with you today. And today we're going to talk about an interesting network Canada bug we found a while back. And so how it impacted our workloads, how we debugged it, and how we fixed and mitigated it. So before we start, let's introduce ourselves. I'm Laurent Bernay. I work at Datadog in the infrastructure teams. And I'm Eric Mountain. I also work in the infrastructure teams, sorry. More specifically, I work in the Kubernetes teams. We deliver the Kubernetes clusters that run all the services that Datadog runs on. And so this isn't about Datadog, but just I'll introduce Datadog very briefly. We're a cloud-based monitoring and security system. So we have lots and lots of integrations that allow monitoring different pieces of software. We're now over 5,000 strong. And we have millions of hosts reporting into Datadog, sending us trillions of points every day. So that makes for a lot of traffic. And quite a sizable infrastructure. So we have tens of thousands of nodes, split over dozens of Kubernetes clusters. So, yes, as I said, we're not here to talk about Datadog, but about a problem that we observed. And so it started with folks coming to us saying, hey, we're being paged at night. Our application is lagging. We're seeing issues with some applications consuming from Kafka. And the throughput we get does not align with the instance type. So that's a nice way of saying that the throughput is really bad, because no one would complain if it were too high, right? Looking a little bit more closely, one of the things that comes to light is that we have an 8Q network card. But all of the traffic is leaving on one single queue, Q0. Based on this, we start trying to do bandwidth tests, that kind of thing. So quickly cobbled together tests with IPerf. And we see that if we go host to host, we're at about 23 gigabits per second, which is close to the limit for this instance type. But if we go from a pod to another host, we're down to 16 gigabits per second for exactly the same test. Taking this a little bit further, we start using Flent. So Flent is an opinionated framework for network bandwidth and latency testing. The advantage with this, rather than our cobbled together IPerf test, is that it's, as I said, opinionated, so it has its own structure, it knows how to launch a certain number of things that will saturate links and things like that. So that's pretty good. And so here what we see is that we're at about 16 gigabits per second, which is the same as what we saw in the previous slide for the pod to host traffic. But we also got very high latency here. We're around 10 milliseconds. So at this point, what we've got is folks complaining that they're getting paged. It's so far relating to two applications running in AWS. And the symptoms are that we have packet drops and therefore retransmits. We've got, so this is a situation that changed, right? Initially people weren't being paged and then they started being paged. So we're now seeing lower throughput than the setup that works and we've got higher latency. The consequences here is that we're having to run more processing pods in order to compensate, right? We scale horizontally in order to compensate. And so it's costing us a lot more money. So if we backpedal a little bit to this single transmit queue business, why is that a bad or an impact or whatever? So for one thing it means that there's a single CPU that's handling all the completion IRQs. So once a packet's been sent by the network card, it's reporting back where an IRR queue and all these IRQs are coming to the same CPU because they're all from the same transmit queue and the transmit queues are mapped to CPUs for the IRQ handling. So it makes it harder to get, so this is something that we talked to AWS about as well and one thing that they told us, well a few things they told us are that it's harder to get to the maximum instance throughput if you're using a single transmit queue. And in particular, well you can get full bandwidth on certain instance types like the very large ones, but it can depend as well on the family of the instances and the generation, like very old generations, you will never manage to get the full bandwidth on a single transmit queue. So I mean, I don't know how it works exactly, but I can take a guess that it's a bit like a CPU, a generic CPU where you have pipelining and different stages of pipelines can run in parallel to process instructions while here the network cards likely are sending on the wire a packet from one transmit queue while they're dequeuing from another transmit queue, et cetera. So this is the parallelism that the card can have. I mean, at a guess. So how does this work? If we look at the stack on Linux. So we have the network card, the virtual network card. The card presents a certain number of transmit queues, so let's say eight because that's what we have in our examples. These are actually mapped to, well they have queuing disciplines ahead of them. So in our setup, the default setup for the operating system we were using in fact, it's FQ Codel. So FQ Codel is flow queue control delay. It's a certain algorithm for dealing with shunting packets around and deciding what is the next packet that should be handled, et cetera. So I'll talk about that a bit more in a minute. And ahead of this is the root queue discipline, which is multi queue, and it's a very simple one. It's pretty transparent. What it expects is to receive from the IP stack a packet that already has a transmit queue set, and it will then just simply send the packet onto the queuing discipline that matches that transmit queue. Okay, I didn't expect that that's like that. So if we look at some of the queue discipline statistics, we see that indeed, if we look at the queue discipline statistics, we see that all of the packets are being sent on the Q0, and not on the others. So we haven't shown all eight, but you get the drift. And so essentially what's happening here is that we've got all these queuing disciplines and all these transmit queues, and everything is just going where the red line is at the top here, okay? If we go back to the numbers, we see that one interesting thing is that we actually have drop packets on that queue discipline zero. And it's quite a significant number, 2%. That's an awful lot. So if we look at FQ Codel in a little bit more detail, how does it work? So there's a stream of packets that need to be sent from different flows. And so the first thing that's done is that a hash is calculated for the on the flow, on the, yeah, on the flow. And that allows to dispatch the packet onto one of a number of queues. Then from there, there's a round robin algorithm that will pick off packets from these different queues based on how recent they are and when they will, well, when they were last taken, when bytes were taken off them, et cetera. And this will be then the packets that are forwarded onto the transmit queues themselves. So one thing, Codel here, control delay. So this is active queue management. Basically, it is actively deciding to potentially drop packets if they are queued for too long. The reason for this is the Codel, the idea is that it's to prevent buffer bloat. Buffer bloat is, for instance, when you have, you can have a nominal stable state where you've got a rather high throughput, but your queues are fairly full, and so the latency is actually quite high, even though you're getting good throughput. And the idea of a queue call is to prevent that kind of situation from lasting. It tries to eliminate it by dropping packets and therefore forcing the TCP protocol into managing the congestion window, backing off of it, slowing down, that kind of thing. So this is, Codel here is likely why we've got dropped packets. So now let's try and see if we can hone in on what exactly is broken. So our network setup looks like this. The host has its host network namespace, it has a primary network interface card for which all the host traffic comes and goes. The pods, they have their own network namespaces, they have their own IP, and all their traffic goes through secondary nicks. So here there's only one, if we had many pods we might need more secondary nicks because you can only have so many E&I IPs per nick. What we know is that what we're seeing is the pod egress traffic being impacted. So that's the traffic here going through in S6. But what about receive? And actually, well, receive is fine. So it's balanced across all the receive queues. So we've got no problem there. Host traffic, well, we've done a benchmark. We can see that that works fine. Okay, what if we send the host traffic through NS6 instead? Well, that works fine too. So maybe it's our C&I plugin. So we use Cilium. So maybe let's just try and do a simple network namespace setup by hand. For one thing, this is a setup where we won't have the Cilium BPF programs loaded. So if we reproduce the issue, we know it's not the Cilium BPF programs. So we start off by creating a network namespace. In there, we create a vese, which is basically a wire with two ends. And we put one of those ends in the network namespace that we've created. We allocate an IP to the end that's inside the network namespace. And we arrange for packets going out of the pod to go to the host. I forgot one, no, that's right. Okay, and also we then add a root so that at the host level, so that traffic coming from the pod will be rooted out through NS6. And we have a rule to say that anything that is destined for the pod goes on the vese into the pod network namespace. So now we have a network namespace that's fully configured, what happens? So from our test, what we see is that we actually have the same issue here. So we've managed to prove at least that the BPF programs of Cilium are not the cause here. But we have something going on with our transmit queue setup all the same. So another thing we can do now that we know how to set up our own manual network namespaces, we can try putting more transmit queues on the vese inside our own network namespace, the pod network namespace as it were. And that's interesting because it actually fixes the issue if we have the same number of transmit queues as on the actual network card, the physical network card. But it's still not ideal because it doesn't fix it if the number isn't exactly the same then the behavior still reverts to everything going through the transmit queue zero. Another thing we tried was to use fair queuing instead of FQ coddle. So the difference here is, so although it might not be intuitive, this is something I discovered, fair queuing is actually more recent than FQ coddle. So the thing, the idea here is, for instance, within data centers you don't necessarily need or for servers, you don't necessarily need the coddle aspect, the active queue management aspect of FQ coddle. And so FQ is basically that part ripped out.  And so when the queue is full, what you expect is simply to have back pressure which will lead to all dropping. And so the server application will forcibly slow down because it can't queue the packets. So interestingly, there with FQ we do actually mitigate the issue. We have the native performance, the native throughput of the instance type, 25 gigabits per second, and we have around two millisecond latency which is five times less than the other test. Now if we start looking at the nodes, we've now got a situation where it's easy to check different nodes. And what we see is that AWS instances with Ubuntu 24.2 plus are impacted. 24.1 is not impacted and GCP instances are not impacted regardless of the version. So at this point, AWS Ubuntu 24.2 nodes have the network issues, the issues that traffic goes over a single transmit queue. We're using FQ Codel. If we switch to FQ, then it mitigates the issue. And GCP nodes are not impacted. So we could stop here and say, okay, we use FQ but that would be relatively unsatisfactory. So now we need to figure out how we can debug this and Lauren's going to talk about that. Yeah, so as Eric said, we had mitigations but they were not satisfactory. The first one is because we could use FQ but we're not convinced that it's the best approach. And also having multiple transmit queue on the VS device would be challenging because we use instance type of different sizes. Some of them are pretty big with eight transmit queue. Some of them are very small with single one. So we need logic to make sure that the number on the virtual center device would match the number on the physical device. But more importantly, it makes no sense. I mean, it works on all the kernels, doesn't work on new ones and we want to understand why. So the first question is we want to understand and want to answer is how do we pick the queue, right? And it's actually easy. You can just look at the kernel code. Well, I'm saying easy, to be honest, it took us quite some time. So this is a very simplified transmit sequence, right? When a packet is, when the routed decision has happened and a packet has to be sent, the first thing you do is you call defqxmit which is going to send the packet through the device. And two important things happen there, right? The first thing is we call the function called netdefcorepictx, right? And this function is supposed to pick the queue. And then you go to the transmit function itself which will first run qdisk and then invoke the transmission function from the driver itself. So let's dive into netdefcorepictx. I'm sorry, it's gonna be a mouthful because kernel function names tend to be long, so sorry. So the first, I mean, this function is actually pretty simple. It's like not even 20 lines. And you can see here that there's a simple test at the beginning which is does the device have more than one queue? Of course, this is a single queue. It makes little sense to pick a queue. And something interesting is happening next. You see the test there. Oops, NGO select queue, right? This is testing if the driver has provided a specific select queue function. And if it has, then we're gonna call the select queue function from the driver. And otherwise, we're gonna use netdefpictx. And remember this one, because we're gonna see this one quite a few times. So at this one, we're like, well, maybe we can expand the difference between AWS and GCP because the drivers are different. So we looked at the kernel code for these two drivers. And on AWS, we use the enedriver and on GCP, we use the virtair one. And as you can see here, on the GCP driver, you have the full struct and you can see that we're not implementing select queue. So we're using the default from the kernel. However, on the AWS side, the enedriver actually supplies a function. So maybe we're onto something there, right? So let's look at this function. So enedselect queue, the function provided by the AWS enedriver, defaults to using netdefpictx. So the function that is a default one used by GCP. So, but it looks like we're not doing that, right? Because it's working on GCP, but it's not working on AWS. So maybe there's something happening in this test here. So before we dive into this test, let's take a quick step back. You'll, I'm going to mention SKB quite a few times. So an SKB or SKB for long is the main networking structure in the kernel. So it has metadata about packets, right? So there's no actual data, only pointers to data, and but there's all the metadata. So the structure is very, very, very big. So I added only a few parts of it. So things that we're gonna use later, the device. So the device that is used to transmit the packet or the device on which the packet was received. And at the very end, all the pointers to different parts of the packet. So you have the network header, which is where you get the IP information. You have the transport header, which is where you have the TCP or UDP headers and so on. Something that will be very important for the rest of the presentation is this small field here, which is called Q-mapping. And it seems very interesting, right? It says Q-mapping for multi-Q devices, exactly what we want to look at. So something that's important is, depending on where you are in the kernel, in which function is using this field, it means a slightly different thing. When you're sending the packet, so we're actually sending it on the wire, this means which Q you're gonna use, right? However, if you're in the function, if you're picking a Q, if it's zero, it will mean that the Q has already been picked by another component in the kernel. And if it's zero, it means you have to go through the action of selecting Q. So let's get back to ENA Select Q. So the first check here, which is checking if Q-mapping is different from zero, is verifying if the Q has been recorded by another component, right? So if it's true, it means that we already have information, in which case we're going to restore it, right? So, and the way it works, I was saying before, is like to store Q information, you just add one and to restore it to subtract one, which means zero means the Q hasn't been selected, you can put pretty much whatever you want, but if it's greater than zero, it has already been picked. So what we wanna do now is see what the value is at different step in the kernel. And this used to be pretty difficult, right? But we're very lucky now because we have eBPF in the kernel, which means we can use BPF trace and had hook points at different functions in the kernel to see what's happening. And you can see here this very simple BPF trace code, which is about 10 lines, is going to allow us to actually look at what's happening at different steps in the kernel and look at the content of the Q-mapping field of the SKV struct. So you can see it's pretty simple. And this function is the FQX mythos, the initial function called when we're sending a packet in a device. And it takes an SKV as the first argument, which is why we're looking at arc zero. We have the field Q-mapping, of course, which is the one we're most interested in. We can also pick the network device. And the last few lines are just a very ugly way to make sure that we're filtering traffic for a specific IP, which makes testing easier, right? So what do we get? So we load this BPF trace program and then we run a simple ping from a network namespace on one of our nodes. And you can see here that we're calling this function twice, which makes sense if you remember our setup, right? We call it once in the podnetwork namespace and then one on the hostnetwork namespace. And the number after dev actually the device indexes of the different devices and I put the name because it's easier. So ECH zero is actually the name of the device inside the podnetwork namespace and in a six is the one on the hostnetwork namespace. So what's interesting here is the Q-mapping field has a different value in both places. It's zero in the podnetwork namespace and it's one in the hostnetwork namespace. And it's always the same, right? Because of course it could be one time, right? But it should vary, it never varies. So at this point, we have this program that works but it doesn't give us much. But what we can do because we have EPF trace is we can instrument many functions in the kernel and this is what we did. So we instrumented a few key functions to see what was happening. So here we instrumented all the function responsible for sending the packet inside the podnetwork namespace and you can see here that we start with devqxmit and we end up with vsexmit, which is the function of the vsedriver to send a packet. Once we reach vsemit, we actually leave the podnetwork namespace and we enter the host. At this point, if you remember what Eric said before, we have to route the packet, which is why we go through all the writing functions in the kernel and the kernel will decide at this point that we need to send the packet on the NS6. And then we get to the final part, which is transmission again, but this time in the hostnetwork namespace on the NS6. And now that we're here, let's look at the Q mapping fields, right? So in the podnetwork namespace it's zero, which kind of makes sense, right? We have a single transmit queue, so it makes no sense to pick it. At the very hand, the NSLX queue actually has Q mapping set to one and if you remember the code, it means the queue has been recorded, which means we restore it back to zero. And this means that every single time we're gonna pick zero because every single time we're gonna hit the NSLX queue function, the Q mapping will be set to one, so the queue will be considered recorded and we'll just subtract one and use zero, which is the first transmit queue. So now we know why we have the issue. What makes very little sense, though, is this here, this transition. Why do we go from zero to one when we cross the network namespace boundaries? So we looked at a few functions and after looking at VSXMits, we discovered this line there and you can see at the very end that we're actually recording the queue. I'm like, okay, and we look at this, we read it a few times. I'm like, well, we should be always recording the queue. So we should be impacted everywhere and should have been the case forever. So we're like, maybe something changed in the kernel. So we used Git and we found this very small patch, right? So there was actually an optimization in an idea that in some cases, when you were using XDP, you might want to record the receive queue to have the information when you transmit. So you could run the XDP code on the same CPU optimization purposes. And that in deeper, it turns out, this code has been introduced in kernel 5.11.11 and also back ported to a few Ubuntu kernels. Turns out the ones are impacted. So 20.04.2 and 20.04.3. So all these things are to make sense and if you remember, there's something very specific happening with AWS, but at this point, we wanted to be sure. And so if we get back to NSElectQ, if you remember this function here, NetDevPictics is the one we should be using. So before doing anything, we're like, well, is using this function okay for us? And because it's the one we should be using on GCP too. So it should be working. So we look at the code. Once again, the code is pretty simple and there's this very promising function here, which seems like it's computing a hash and allocate and associating a flow to a queue that seems perfect, right? Except if we look at this function itself, there's this very good logic at the end that reminds us of what NSElectQ was doing, which is if the queue was recorded, then restore it. And we're starting to be extremely confused here because of this code, we should have the issue on GCP too, right? Because the queue is recorded there too. So we get back to this function here and like, oh, there's this code here called get XPS queue, right? And I don't know if you're familiar with XPS, but it's a transmits package steering which allows for mapping flows to queues based on the transmitting CPU. So you have some form of data locality and some optimizations. It's a good optimization, but we don't use it anywhere. I mean, we haven't configured it anywhere. So that's kind of weird. Still, to be extra sure, we connected to a GCP host and well, turns out GCP has a magic demon that actually does some kind of configuration on nodes and it's setting XPS by default on all the nodes. Turns out if we disable XPS on GCP, we now have exactly the same issue, which is good news, right? So let's go to a quick summary of what we found. So we found that the code in VSXMIT is recording the queue and impact the kernels. At some point after routing, we're going through this function here, the net def core TX, which is the function responsible for picking the queue. Then we have two possibilities. If we're on AWS, we're always gonna use ENA select queue, which is gonna say that the queue is recorded and always use the first queue. If we're on GCP, we're gonna use the VATIO driver and then we have two different cases. If XPS is enabled, we will use the queue decided by XPS. If it's disabled, we'll use the XCP takes hash function, which will do exactly the same thing as ENA select queue and we'll use a single queue. All things start to make sense. That's good, but now we need to fix it, right? So we are kind of lucky because we work with the Cilium team pretty closely and when we started having the issue, we ping them and we expand to them what we found and Daniel Borkman immediately said, that sounds like a bad bug, let me fix it. And so he created this very simple commit here, which got accepted in a matter of hours and merging the kernel. If we get back to the sequence of events and what's happening as we go through different functions in the kernel, you can see here that because now we have a patch kernel, the VSXMIT code is not storing the queue anymore, right? So it remains zero. And now, once we reach ENA select queue, we're actually seeing that the queue is not recorded and so we're gonna now go into net dev pick takes, which we were not using before, and pick a random queue out between zero and seven in our case because we have eight queues. That's great news. However, while we also have existing nodes and as Eric was saying, we have tens of thousands and getting a kernel patch of a 10,000 hosts is pretty challenging. Turns out, once again, we're lucky because we use Selium. So we have a daemon set that is loaded on every single host we have and it's actually already instruments the visual scenario device with eBPF code to perform Kubernetes load balancing but also network policies. And because we have eBPF code there, we can actually modify the SKB. And so what Daniel suggested again is like, maybe we can just use the eBPF code we had to force the queue mapping to zero always. And this is what this code does here. You can see here with this very simple function called reset queue mapping that could be called on anything on every single packet and just set the field to zero always. So what does it look like now? So you remember, this is for a kernel that are impacted. So as we go through the VSXMIT code, we store the queue. So it's one, which is something we want to avoid. However, because we have this new patch in eBPF code with Celium, it's actually reset to zero as we go through the VS device. Which means we can now pick a queue and this time it was five. So we've been through all this now, what are the results? So the first one is, well, it looks very good when I was using all the queues, right? So very satisfying. However, it's not the only thing we wanted, right? We also wanted higher level metrics to look better because this is not really meaningful. So we started looking at throughput and throughput is now much better, right? You have host to host on the left-hand side and pod to host on the right-hand side and you can see that the numbers are pretty similar, right? We're maxing the instance, which is good news. Also, if we restart the test with plant, so this time we're using FQ code in both cases, we can see that it's much better once we've patched the node, right? We're, instead of having 16 kilobits per seconds, we're getting to 25, which is exactly what the instance provides. What about our applications? So key network metrics start to look much better, right? The first metric here is TCP retransmitted by flow and you can see that they're going down immediately after we deploy the patch. And the TCP latency, so this is the SRTT as measured by the sockets, was around eight milliseconds and after the patch now four milliseconds, which is, of course, much better. And what about business metrics? So this is the one that was, I mean, this is actually the reason we started to do this in the first place. I mean, the reason the teams were complaining was because they had issues with Kafka or in this example here, they had issues when they were transferring data with Nestry buckets. And you can see here that the P99 for the transfer latency used to be between like two and four seconds and after the patch we're now below 0.5 seconds, which is, of course, much better. And this gets us to our conclusion. So what we wanted to share is, well, achieving top network performance is still hard. It seems like it's a solved problem, except if cloud providers start to provide us with instances that can do 25 bits per second or more. And of course, if you want to do this efficiently and on node, it means you have to tune your operating system. And of course, interactions between the different components of the network stack are very complex. But you can see how, I mean, the commit that created the issue was extremely simple and understanding the exact impact was, took us a lot of time and was pretty challenging. We also wanted to share that, I mean, this totally sold us on BPF trace, BPF trace as a system team because it allows us to debug very complex kind of behavior in ways that are pretty easy. And also, in some cases, especially on the network side of things, we can actually do things on packet and mitigate issues without waiting for a new kernel release. And something we wanted to say also, it's, well, sometimes it's actually a kernel bug because of course, when we started, nobody would have guessed it would be a kernel bug. We're like, well, it's probably something we messed up with the configuration, right? Turns out this time it was a kernel bug. If you're interested in details, there's many more details in the issue on the Cilium repo. Thank you very much. If you're interested in debugging this kind of weird and funny issues, we're always hiring and we have a few minutes for questions if you have questions for us. No questions. It was all perfectly clear. We'll try our best. Okay, thank you very much. Thank you.


Okay, so we are discussing the angular momentum budget that can hold in the upper branches of circulations that are angular momentum conserving when the Rossby number approaches one, and where they are driven by eddy stresses when the Rossby number is much less than one. Again, let me re-emphasize this point again, it's important. When a cell which conserves angular momentum starts becoming cross equatorial, we have this upper level easterlies that develop over a broad region in the tropics. The eddies that do transport, effect transports of angular momentum are barotropic Rossby waves that propagate in upper level westerly flow. So there is already a hint here that cross equatorial hardly circulations might be more angular momentum conserving because the appearance of upper level easterlies shields them from eddies originating from the winter hemisphere higher latitudes. Okay, so then let's go back, questions? No, let's go back to the aqua planet simulations and here I am looking at a lot of fields and I know that this is a very packed slide, but here the important point that I want to emphasize, this is what the circulation looks like before monsoon onset and before monsoon onset here, I really mean that this very short lived time, this is of course the shallow, makes their depth simulation where things are more or less symmetric about the equator and after monsoon onset I'm picking a time where the monsoon is really well developed. So again, going back to what the circulation looks like, notice how here when things are symmetric about the equator, the circulation is characterized by two cells that again are almost symmetric about the equator. The southern cell is a little bit stronger than the northern cell. Notice how the gray contours represent angular momentum contours. We said that a circulation that conserves angular momentum is such that angular momentum contours and streamlines are aligned. We clearly see crossing of angular momentum contours by the circulation, so the circulation is not angular momentum conserving and in fact the color contours that show at the momentum flux divergence in red and convergence in blue clearly show that this cells are in fact very much in the limit of a small rosby numbers, eddy stresses really constrain the strength of the circulation, these are again the large scale eddies originating at higher latitudes, they propagate in region of upper level, questily flow, once they start approaching the critical line, the zero inline the break, and again they extract momentum from this flanks of the circulation and they converge to higher latitudes. So let's see what happens, just more or less 20 days after, I think it's 25 days after this snapshot and notice how in 25 days that the circulation really has changed quite rapidly from this more equinoxial pattern to this social pattern characterized again by just one single strong cross equatorial Hadley circulation. Points to notice is the fact that if you can see angular momentum contours now are not vertical, which is what you expect if angular momentum is dominated by the planetary component, but in fact they are, they take a horseshoe shape and they follow at least in the ascending an upper branch of the circulation, they primarily follow the stream lines of the flow. So this is a circulation that approaches way more strongly conservation of angular momentum. So why, why is that the case again, the fact that as the circulation becomes cross equatorial upper level easterly start appearing really plays a very important role because again the circulation is shielded by the zero inline from the influence of the eddies. So most of the, especially where the circulation reaches its maximum, it's really in a region where there is very little eddy momentum flux divergence. And so here the argument is that these monsoon transitions that we see in the absence of any lengthy contours that are really driven basically they represent transitions in the leading angular momentum budget in the upper branch of the cell from an eddy driven regime where again the circulation is really slaved to the eddies to a regime that is much more angular momentum conserving in much better agreement with the angular momentum conserving theories that Jeff discussed. So the question is why is this transition so rapid? Why does the circulation intensify so rapidly? In these simulations there is two feedbacks that operate on very short time scales. Again the first one is associated with this interaction between the mean flow and the eddies as the circulation becomes cross equatorial and the upper level estuaries develop. The estuaries shield the circulation from the eddies and so the circulation can become more angular momentum conserving. As it becomes angular momentum conserving you really shift to a much more non-linear dynamics abduction by the circulation really becomes dominant. Again remember that from angular momentum conserving theories as the circulation becomes cross equatorial you develop through thermal wind balance this reverse meridional temperature gradient. So temperature maxima are not found near the equator they're in fact the minimum in temperature near the equator and a maximum at the location of the poor branch of this circulation through convective quasi equilibrium arguments these upper tropospheric temperature structure is related to the lower moisturistic energy structure. We clearly see this the equatorial minimum is not. So I mean it's not really a minimum but you definitely see a much larger values in moisturistic energy in the boundary layer in the subtropics with the edge of the heavily circulation is. And so now as we start developing this reverse meridional temperature gradient in the lower branch the circulation itself through abduction of temperatures really allows very rapidly to push forward even further forward this maximum in the moisturistic energy. That is to say that the circulation can intensify and broaden very rapidly. And again here the key is not the fact that land warms up faster than an ocean because we don't have the difference but it's the fact that the lower boundary that here it says swamp really can allow for this rapid shifts in the distribution of the moisturistic energy and in fact can help push the maximum in moisturistic energy at subtropical latitudes and with it the upper branch of the circulation associated precipitation patterns. I had a question before asking to what extent this maximum coincides with the maximum in solar insulation it does not. Solar insulation maximizes at around 23. This maximum is in fact occurring at about 30. Later in the simulation it actually goes all the way to 30 degrees north. So here the interaction and the advection provided by the large scale circulation becomes dominant. And again this is also a caveat in this argument is that really emphasize the fact that we need to know the distribution, gradients, maximum, moisturistic energy to say something about the large scale circulations is that because this is a couple problem it's really provides that a mechanistic but not a prognostic view of monsoons. We have to run simulations or diagnose fields to be able to look at these interactions. It's very hard to make any prognostic arguments based on this. Okay, so very very short summary. The aqua-planet simulations suggest that rapid monsoon onset and end correspond to transition in leading angular momentum budget in the upper branch of the cell from an equinoct regime where the influence of the eddies is large and really the circulation is constrained by angular momentum budget to a monsoon circulation where the role of the eddies is minor and the circulation can approach more closely conservation of angular momentum. As a caveat this is based on an aqua-planet, right? And it totally neglects zonal asymmetry so I think that there's gonna be more discussion next week at the workshop as to how these mechanisms should be modified to include zonally asymmetric, sorry, asymmetric continents. Oh, sorry, zonal asymmetric continents. No, this is Katrina's talk, sorry, yes. So this is right. So more discussion about what happens when you start introducing some lengthy contrast and to keep things simple we just impose a continent that is completely zonal symmetric but then how these mechanisms are also modified by the presence of stationary eddies that are completely neglected here when you do have zonal asymmetries. And also I have a poster on what we discussed a little bit what is the observed angular momentum balance in the South Asian monsoon. Whether this transitions from a more linear to a nonlinear angular momentum budget is also in the observations. Okay, so then the other thing that I wanted to do because I'm sure there's gonna be a lot of discussion next week on the more static energy budget is, okay, we said that to the extent that a circulation is completely angular momentum conserving, the zonal momentum budget reduces to a trivial balance, zero equals zero, that is verified for any strength of the circulation. So what constrains the strength of the circulation? And so a lot of progress has been made in that by using the more static energy budget. But again, it's really the relevant budget for moist circulations. And in fact, in many ways, it's really the theoretical development to this quasi equilibrium views of the interaction between conduction and large scale circulations that I discussed earlier. So how do we obtain a more static energy budget? We talked before about the dry thermodynamic budget. So that is the, just the first law of thermodynamics. Jeff, I'm sorry, but I'm gonna use primitive equations here. Sorry, pressure coordinates, I should say. So the first law of thermodynamics can be written. So I'm just rewriting in pressure coordinates, Cp, the temperature, d time minus one over rho, d pressure, d time equal to any heating rate, okay? This is the first law of thermodynamics we should all be familiar with. So when we write it in pressure coordinate, then we have Cp. I'm gonna expand the material derivative. We're gonna have a horizontal temperature grid, and it turns out that you can combine the vertical abduction of the material derivative on temperature and this term in just one single term that is equal to omega, which is the vertical velocity in pressure coordinate, omega being dp, d time. And then here what appears is the dry static stability. That is the ds, dp, with S being the dry static energy. We discussed a lot about the moist static energy. It's just Cp, t plus gc. And that is, I should have written it. Sorry, let's do this. And this is equal to the heating rate, okay? Again, this is stability, it's basically d theta, dp, linearized form of stability related to the Brunweiseler frequency. There has been a lot of work that is being done by using the dry thermodynamic pattern. Once you know the distribution of the heating rate and the distribution of the precipitation pattern, so look at what large scale flow is consistent and generated and driven by the diabatic heating. But again, the problem here is that it's diagnostic, but it's not prognostic. Again, once you introduce a perturbation, because the distribution of the precipitation is strongly influenced by the large scale patterns, there is nothing that this budget can tell you, okay? So then how do we make progress? Then we combine this. We're gonna vertically integrate this. And I'm gonna adjust the node, vertical integral like this. And I'm sure that I'm gonna miss a few things, but that's how we write it. Again, I'm just d time, vertical integrated temperature plus the, again, this is just the horizontal wind, so this is just the horizontal direction, okay? Plus the omega, the SDP term. And then what are you left with when you integrate the heating rate is how radiative fluxes or surface fluxes can change your temperature. And so we're gonna have the vertically integrated diabatic heating due to condensation, plus sensible heat from the bottom. And then you have a net short with fluxes. And here what I mean the net is the difference between the net at the top and the net at the surface. And then you have the same for long wave fluxes. Again, net difference between net at the top and net at the bottom, okay? So then we're gonna take the moisture budget and we're gonna do something very similar. We're gonna use it, again, these of course is written in energy units. We're gonna do the same for the moisture budget multiplied by the latent heat. So we're gonna do LV dQ dT plus the horizontal advection plus the vertical advection omega dQ dP. And this will have to be equal to sources and sinks, right? Sources minus sinks.  And then what are the vertically integrated sources? And sinks, you're gonna have the condensational moistening. We call it QM. And then you'll have evaporation from the surface. So latent heat from the surface, okay? Then we're gonna sum these two. We're gonna sum the vertical integrated dry thermodynamic budget, the vertical integrated moisture budget and we obtain the moist static energy budget. Let me call this equation one. This is equation two. Taking one plus two. We obtain the moist static energy budget. So we have a term which is the storage of enthalpy in the atmospheric column which is CPT plus LVQ. Okay. Then we have the horizontal advection of the same quantity, vertical integrated CPT plus GZ. Sorry, the vertical integral is here. And then we're gonna have a term that represents the vertical advection or a favorite quantity which is moist static energy because it's the sum of the dry static energy coming from the dry thermodynamic equation and moist term. Say it again? No, I'm sorry, I'll do, sorry, yes, yes. There is no GZ there, yes. There are, there are different ways in which the moisture budget is derived. Some start from a statement that moist static energy is conserved. At the end you get to something similar but this is really the correct way to derive it and in fact in the advection and in the storage you don't have the GZ term. If you start from a statement that moist static energy is conserved materially then you have the extra GZ term. Doesn't make much difference that is a very smoothly varying term but that's the correct way to derive it. And then when we sum the sources and sinks of energy terms we're gonna use the fact that these terms that are the largest terms that appear in the dry and moisture budget actually cancel each other. So we're gonna use the fact that the condensational heating which is equal to LV multiplied by precipitation is equal to minus the condensational mustening. So the two are the biggest term in the individual equations but they cancel each other so they completely drop. So now we don't have any information that is related to the diabetic heating anymore and the sources and sinks are really our sunsets and sinks of a moist static energy coming from the top of the atmospheric column and the bottom. And so these are going to be equal to what we call the F net or net energy input into the atmospheric column and these are equal to basically the top of atmosphere radiative fluxes. Again, those contain the downwelling shortwave and the up going longwave minus the surface fluxes and the surface fluxes in addition to the radiative fluxes also contain the latent heat and the sunsets here. Is it clear? So we're really just saying that whatever is the net energy input that can enter an atmospheric column through radiative fluxes at the top and radiative and turbulent surface enthalpy fluxes at the bottom will have to be balanced by these other terms that are associated with how the dynamics respond. So it's really again bypassing this the large diabetic heating terms and it's really related large scale to the energy sources and sinks in a vertically. Integrated sunsets. Okay, so then approximations that are usually used in the most static energy budget are the following. Study state, so we neglect any storage. So we're gonna assume study state. We also are gonna make use of the weak temperature gradient approximation to at least to a first order drop the horizontal advection terms. Notice how the weak temperature gradient is of course justified for temperature is less justified for moisture. We're gonna say that we're gonna do it, okay? We're gonna drop that. And so then the balance that we have so study state, weak temperature gradients, then we are going to say that the leading balance and that is true in the deep tropics is between the vertical advection of more static energy and the net energy input. Again, if this is my atmospheric column, the net energy input is whatever it comes through the top and the bottom, okay? We also are going to assume that in the tropics, so far I haven't introduced any time averages. I've been a little bit sloppy about that, but we are gonna assume that in the deeper tropics, really the vertical advection is not dominated by eddy covariance is between vertical velocity and H. It's primarily dominated by mean vertical motion. So again, I'm gonna now put the bar and assuming that the HDP bar, this positive is stratified exactly as I showed you before, the fact that H tends to be on average, larger at upper levels than at lower levels, which means that in pressure coordinate, this will be negative. Then what this balance is really stating is that whenever the net energy input is strong as positive, what the atmospheric circulation will do, which means that you are actually gaining heat through the top and the bottom, the atmosphere will respond by diverging energy away through the development of vertical motions, such that omega will also be negative. So the way in which the circulation responds is to develop strong vertical motion in the atmospheric column. If you assume, and that is also a consequence of convective quasi-equilibrium, that really the dominant vertical dominant mode is a mode in which the vertical motion is characterized by one simple structure with maximum in the metroposphere. And then you have lower level winds and upper level winds confined to relatively thin layers close to the surface and the bottom and they basically are opposite sign to each other. Then the picture that you develop is that you have lower level convergent motion. These converging motions are converging moist static energy that is smaller than the moist static energy that is being diverged away by the upper level motions. And so in this way, you can reach a balance again, by which the net energy input coming through the top and the bottom is balanced by the circulation that is exporting moist static energy through the development of vertical motion. Okay? D H by D changes sign. Yeah, and again, this is also where you're using the assumption that the horizontal winds are primarily confined to thin layer close to the top and the bottom so that when you take a vertical average it's really basically the difference between the H and upper level, minus the H at lower levels. And the H at upper levels is larger than the H at lower levels. There is a lot of caveats that go into this and that's where we also are right now moving a little bit away from convective quasi-equilibrium. To what extent this is also true that always true that convective motions have a deep first baroclinic mode structure. There is evidence in the tropical Pacific have in fact circulations in which the expert is occurring at a much shallower level where in fact you are close to the minimum in moist static energy and you can still have convective rain bands such as the one over the Eastern Pacific that in fact is importing rather than exporting moist static energy because again, in this case when the return flow is at lower levels, let's say mid levels where the moist static energy reaches the minimum the lower level converging motion is importing more moist static energy than the one that is being exported by the mid level motion. But these are things that are being exactly next week there's gonna be a lot of discussion about this. There are a lot of different definitions of a quantity which is the gross moist ability. I'm sure that the gross moist ability will be used extensively and the gross moist ability can be defined in many different ways but it's really is a measure of the vertically integrated export of moist static energy by whatever circulation that you have normalized by the mass convergence at lower levels. Again, here all these arguments are based on positive gross moist ability also called GMS but again, there is a lot of work that is being done these days to understand how good of a representation of many tropical rain belts. The assumption that gross moist ability always positive is. Okay, so another point that I want to emphasize that goes back to how we should really think about the, and I should mention that really the first paper that introduced the gross moist ability is also a quantity that is really powerful to, for instance, model tropical precipitation is kneeling and held 1987. And honestly, we haven't done a whole lot of progress since that paper in understanding what really controls the gross moist ability. It has to do with the interaction, reading convection and large scale circulation. Probably I gave you the idea that if we have all figured it out, it's all about convective quasi-equilibrium, in fact, it's really not the case. There is a lot of active work that is really trying to understand where all of these assumptions based on convective quasi-equilibrium really break down and when we need to consider the variation from convective quasi-equilibrium. Okay, so the other thing that I wanted to say, again, because it really, oh, maybe I shouldn't. Okay, I think it really makes the case for how the moist static energy is really the budget that we need to consider, for instance, when thinking about Manzono circulations is the following. Again, the net energy input is the difference between top of atmosphere radiative fluxes minus the surface fluxes. Okay, so the surface fluxes are also the ones that control the surface energy budget. So now I'm thinking about the surface energy budget of a mixed layer, but then we'll talk about that how that gets modified, for instance, over land surfaces. Let me call just big C the heat capacity of whatever mixed layer I have. So the surface energy budget is just the statement that storage, let me call it TS, in the mixed layer depth. So how temperature can increase in the mixed layer depth will be a result of the surface fluxes. And again, notice that the surface fluxes are taken as positive when they warm the mixed layer, that is the surface, we have to switch side when we consider what gets into the atmospheric column. And then possibly if you have ocean heat transport, there will be a convergence of ocean heat transport here. Okay? Okay, so over land, there is of course no transport. And also because the heat capacity of land is small, then this term on any time average longer than a few days will drop out, okay? So land is really different than the ocean in terms of this moist static energy view in the fact that again on time scales of a few days, the surface energy fluxes are constrained to be zero. Okay? So the net energy input into the atmospheric column is really dominated by the top of atmosphere radiative fluxes. Okay? And so for instance, this is why over months in regions you have through season and insulation forcing a strong source of energy through variations in solar insulation. Okay? And so it's the atmospheric circulation because no heat can go into the surface that need to redistribute those, that energy input from regions where you get a lot of energy to regions where you are losing energy. Okay? So that's why we get much stronger in solar circulations over land. It's really not the lensy contrast per se, but it's the fact that surface energy fluxes are constrained to be zero over land. Over the ocean, now a lot of that that comes from the top can be stored into the ocean. Or can redistributed through the circulation. So the ocean can dampen out the atmospheric response which cannot happen over land. So this is again, beautiful series of papers by two annealing. I never know how I'm saying two usual annealing early 2000s. And again, de-emphasizes again the view of monsoons has really driven by near surface temperature contrast which should really be thinking about moist circulations that want to export moist static energy. And we should really be thinking about them more through this view that emphasizes, relates the circulation to the top and surface atmospheric energy fluxes. Okay, so I am left with 15 minutes. I'm trying to decide what I'm gonna do. I think that what I'll try to do is be a little bit more clear as to the consequences of thinking about monsoons as really cross equatorial hardly circulations and also trying to tie back this energetic arguments to for instance the main position of the ITCC which is north of the equator and possible shifts during the monsoon season. Let me do one more thing at the board, sorry. The moist static energy, one flavor, it's really, I'm restating the same things that will make it a little bit more clear the connection between the moist static energy budget that I've just described with this view that is trying to link shifts in the ITCC position to cross equatorial energy transports if now we take a zonal mean and I'm gonna indicate the zonal mean in this way and I'm gonna also neglect storage although the storage could be in fact effectively included in the net energy input. The moist static energy budget has been can be rewritten as the meridional divergence of the zonal average of the vertical integrated total meridional moist static energy transport and this needs to balance the net energy input is exactly the same concept I'm just averaging in the zonal direction and again I am relating any region of net energy input to divergence that is export of moist static energy through the development of a divergent circulation. Okay, so now we can integrate from the south pole you can do the same from the north pole in fact up to a latitude theta and again it's the fact that at the south pole and at the north pole there is no meridional energy transport and so we can express the energy transport again here it's really the total you can decompose into an amine and a net decontribution at any given latitude theta has one over the cosine of theta of the integral from the south pole to that latitude theta of the net vertically sorry this is also zonal average I'm also gonna take a zonal mean of a cosine of theta in d theta again you can do the same from the north pole and this is really just to say that if the net energy input is completely hemispherical symmetric then you cannot have a cross equatorial energy transport you can have a cross equatorial energy transport only if you have a symmetry in the net energy input between the north and the south pole and these then going back to the schematics that is really relevant to many things including the northward shifted position of the ATCC in the annual and zonal mean again going back to why is the ATCC north of the equator is really related to the fact that there must be an asymmetry in the net energy input with the north and hemisphere receiving more energy than the south and hemisphere and the question is exactly what is it is it through top of atmosphere fluxes is it through surface fluxes has John discussed on Monday it's really not through the top of atmosphere radiative fluxes in fact the short word term is remarkably symmetric the long wave term in fact is such that if anything just based on top of atmosphere considerations you would expect the ATCC to be south of the equator the north and hemisphere radiates very efficiently in desert regions so again if the ocean didn't do anything the ATCC would be south of the equator and this is confirmed in idealized simulations that we've done in our group who recovered the north and hemisphere with just a very simple land and without any ocean heat transport the ATCC is sitting south of the equator so the reason why the ATCC is north of the equator is because the ocean heat transport is transporting energy in the Atlantic marionnel of returning circulation from the south and hemisphere to the north and hemisphere in fact in excess to what would be required by the top of atmosphere radiative imbalance and so now that excess of energy that is transported into the north and hemisphere by the ocean needs to be transported back by the atmosphere and the way the atmosphere does it again is by shifting the ATCC in the north and hemisphere and again here these are just showing schematics that are showed before again in this framework north which shifted ATCC position is accompanied by cross equatorial energy transport in the opposite direction because again the Hadley cell on average is an energetically direct circulation and transports energy in the direction of the upper level flow so this framework, Sarah was really influential in the development of this framework really emphasizes how the ATCC position is anti-correlated with the cross equatorial energy transport if we include any perturbation to this already is only a symmetric present day climate configuration that for instance puts more energy into the north and hemisphere than a more northward shift shifted position of the ATCC will be accompanied by an increased southward cross equatorial energy transport I'm sure that another thing that will be discussed a lot next week is the energy flux equator which is a proxy for the ATCC that again has been developed based on these energetic arguments again to the extent that the circulation has this structure more static energy has this very simple stratification then the position of the ATCC that is the dividing boundary between the southern and the north and cell is where the mass flux but also the energy flux will go to zero so the ATCC in this framework has been argued if not necessarily collocated at least covary with the energy flux equator again where the energy goes to zero from being southward in the southern cell being northward in the north end cell so this anti-correlation between the ATCC shifts and changes in cross equatorial energy transport has been shown to be robust in GCM simulations forced through different forcings this anti-correlation is shown here this is the annual mean ATCC change as a function of the changes in annual mean atmospheric cross equatorial energy transport and with the sensitivity of so you need to obtain a three degree shift in the ATCC position you need one petawatt of change in cross equatorial energy transport okay so let me emphasize how while this energetic framework has really been used extensively to understand annual and zonal mean shifts of the ATCC there's still a lot of open questions the first question is to what extent is even this zonal mean framework useful in the sense that two different forcings is it true that in different monsoon or ATCC regions is the zonal mean framework indicative or representative of any different sector means the verdict is still out there I don't think there is any agreement that has emerged yet and there is a lot of work that is being done with different GCMs more or less idealized of course again here we have totally exactly as I did for the zonal momentum budget we don't have any consideration of zonal fluxes zonal shifts in the precipitation so how do we extend this theory to account for those and there has been recent work that has taken steps forward in this direction and then the other big question is here really we're assuming that a big assumption in all these arguments is that shifts in the ATCC are primarily due to changes in the mass flux of the Hadley cell without any change in the efficiency with which the Hadley circulation is transporting energy and it turns out that that is not always the case we cannot assume that the gross moistability and the efficiency of energy transport is always constant this is work that Sarah has done looking at the response of the ATCC to CO2 forcing we've also looked at changes in the gross moistability through seasonal cycle with aqua planet idealized GCMs so it's important to keep in mind that again sometimes the Hadley cell adjusts to radiative perturbations again not by shifting its ICDC but actually changing it's hard to change the moistatic energy stratification but you actually can change the Hadley cell vertical structure for instance through the development of return flows that don't go all the way to the detroposphere at level where the moistatic energy is large but can in fact occur at the levels of minimum gross moistability therefore changing the sign of the gross moistability okay so in this situations then cross equatorial energy transport and ITCC positions are not associated or related in any linear ways okay or anti-correlation okay so if I can have two more minutes the final thing that I wanna do is talk about the existence of tipping points in monsoons two minutes it's not even my work but I thought it would be interesting so how we define tipping points so these are really really very somewhat broadly defined again we'll or can we expect that monsoons will shift rapidly from very different states let's say wet and dry states for small changes in radiative forcing past a critical threshold and there has been some work that has argued that as the total top of atmosphere of veto gets above 0.5 monsoons will totally shut down there is evidence in paleo records of rapid changes in monsoon I will also say that probably there's gonna be discussion next week as to what extent these paleo records are really providing evidence to rapid changes at a specific location or more gradual changes of patterns that have rapid spatial patterns so again this is something that is to these days being discussed in the literature so why should we be thinking that monsoons have tipping points and I think that the argument that has been used is the fact that monsoon onset is a very nonlinear process can these nonlinearities that act on the season hours of seasonal timescales also act in response to a given changes in enforcing and of course here the concern is that through land use changes maybe areas or maybe increased greenhouse gas concentrations the monsoon will shut down okay so of course usually the tipping points are studied in very simple models I won't present work from previous work but I'll primarily focus on recent work by Bill Booze and he used a simple model that is basically just the moisture energy budget that we have described actually has two equations for temperature and moisture, vertical integrated those should be thought as fluctuations in temperature and moisture again we have the vertical terms that represent the antibody cooling in the low level moisture convergence meridiano winds are assumed to be proportional to the meridiano gradient the sign is the low level flow simple closure for precipitation you have an heaviside function well let me first of all there is observational evidence that precipitation is strongly related to how much moisture you have in the column and then so precipitation increases with moisture in the column but decreases with temperature and if the temperature becomes too warm the column becomes too dry and then you have a relaxation time skill and a heaviside that first of all guarantees that precipitation is non-negative and again that tells you that when you have too dry and too warm atmospheric column precipitation will not occur no rotation, no non-linear momentum of action no evaporation so in this simple model actually you see and the solutions for precipitation is the blue and the solution for the wind is in red again when you combine these two equations in one single equation for you actually find that the relevant parameter is again the net energy input into the atmospheric column is this Q parameter here and again you see that there is no evidence of any abrupt change in the monsoon there are some maybe non-linear increase of V and P as Q is positive once Q becomes negative again this is the net energy input into the atmospheric column no precipitation and then the circulation reverses so how do we explain the absence of tipping points in the simple models with previous work that have argued for the existence of tipping points so previous models especially this paper by Leatherman et al basically used some similar arguments as the one provided by the moist static energy argument but they neglected some key terms in the resulting equations they neglected the vertical term in the dry static energy budget that is the adiabatic cooling that is also the vertical motion that sustains precipitation they also neglect completely the lower level convergence the dress retain the abduction bi-rotational flow of moisture and they argued that that is the relevant non-linearity for the monsoon so to test that that is indeed the case that it's really not accounting for these terms that are so important and that really represents our current understanding of moist circulations in the tropics so then here what is being done you take the same model and by basically pulling the dry stability the DSDP term that has cast before it to zero then you obtain the following solutions basically as Q decreases plus this threshold you basically your monsoon completely disappears okay so again here what I would like to emphasize is that again our current understanding of monsoons is not in line with the existence of tipping points and I'll stop here


So, welcome to today's lecture. Today, we are going to discuss more details of high resolution NMR spectra of molecules which we are doing in the last class. So, in the last lecture, we looked at that at the same magnetic field different nuclei such as proton, carbon 13, nitrogen 15 has different resonance frequency and that gives us possibility to record in the same sample different spectrum for different nuclei. We also looked at liquid state samples are most favored for high resolution spectrum because liquid sample can tumble in the solution and because of the stumbling lots of emotional averaging happens and many of the isotropic interactions are averaged out and therefore we have a sharp line whereas solid generally gives broad line and gaseous sample has less number of spin to contribute towards the signal. Therefore, the intensity from the gaseous sample is less. So, for high resolution NMR spectrum liquid state samples are most favored one. Then we looked at electronic cloud around the nuclei cause difference in absorption frequency for same type of nuclei. We gave an example for like proton say proton in CH3 has a different electronic cloud around it then CH2 and then OH and therefore for alcohol we have 3 spectrum one corresponds to CH3 another corresponds to CH2 and one corresponds to OH and their intensity ratio was also 3 to 2 to 1. So this also we looked at and this differential electronic cloud around the nuclei is the reason behind the chemical shift. After that we were discussing about like how because of this differential electronic cloud around a particular nuclei give rise to different effective absorption frequency or the magnetic field. So H local is the magnetic field experienced by a particular nuclei is different than the H0 which is the main magnetic field and that depends upon what is the screening constant or shielding constant around that nuclei so sigma local. Now we also looked at that sigma local can be positive or negative and this sigma local depends upon two contribution one coming from the diamagnetic contribution another coming from the paramagnetic contribution. So generally diamagnetic contribution is positive and paramagnetic contributions are negative. So now we looked at the change in the field at the site of nucleus is called chemical shift and this sigma which is called screening constant or shielding constant. So this is the reason why actually same kind of nuclei experience different magnetic field local magnetic field and that gives two different absorption line. Next we were discussing that the chemical shift depends upon the applied magnetic field which is H0. So if higher the magnetic field higher will be absorption separation between the resonance frequency. So like if we record a spectrum on 14 tesla the separation is going to be like that if we express this resonance frequency in hertz and if we apply this is on say 600 megahertz and if we record that on 800 megahertz the separation between line will be far. So that is the case and therefore the higher magnetic field are better to resolve the spectrum. But this creates a problem if we express the resonance frequency in terms of hertz because at different magnetic field resonance frequency are different for same set of nuclei. So this create little complexity in the analysis. For getting rid of such complexity one has to express or explain the chemical shift in a field independent manner. So how one define this in a field independent manner by defining chemical shift as a ratio. So this ratio which is this is chemical shift say I am interested in getting the chemical shift of a particular nuclei what has to be done then you express that as a reference magnetic field and difference with that with a particular interest of nuclei and divide that the magnetic field of the reference compound and that is in parts that multiplied with 10 to the power 6 that is million. So now here sigma i is the chemical shift of a particular nuclei, Hr is the reference nuclei, Hi is the field experience by the ith nuclei which we are interested. So this is actually chemical shift for particular nuclei. So here if you look at now we are taking the ratio of two magnetic field. But if we wants to explain this in terms of frequency what one has to do that subtract the ith nuclei frequency from the reference nuclei and then divide that with a main magnetic field which is mu 0. This is because the difference between the reference nuclei and main magnetic field is not going to be too large and therefore we can take this as a mu 0. So if we explain this then sigma that delta which is chemical shift can be expressed in part per million because the difference in the chemical shift between the reference nuclei and the choice of nuclei is in hertz and main magnetic field is in megahertz. So hertz divided by megahertz is essentially part per million. If we express this chemical shift in terms of PPM we have some merit in that. So now defining this in PPM actually it removes the field dependence. So now chemical shift has become really, really field independent. Whether you record an spectrum on 600 megahertz or record an spectrum on 800 megahertz there is no field dependence. So at all field you can express your resonance frequency in part per million and it is going to be same. So here like if we are recording on 600 megahertz the PPM value of these two peaks is going to be same whether you record on 800 megahertz. The PPM value is going to be same which was not the case when we were expressing the chemical shift in terms of hertz. Another important point that since you are dividing the two units, so chemical shift in PPM is a dimensionless quantity. But for expressing this chemical shift in terms of PPM one requires a reference frequency and generally we put that reference frequency at zero. So what we mean? We take and compound which we can say that this is my reference compound and now all other chemical shift will be actually defined how far they are from that reference frequency. Therefore, we put our reference frequency at zero PPM. So now in broader term we can conceptualize that if there is a higher chemical shift that means low shielding or deshielding or if there is a lower chemical shift that is shielding. So what I mean to say if our reference frequency here is zero PPM reference, so if we are away from the zero that means the compound is deshielded or there is a lower shielding. And if we are, so this is deshielding and if it comes near zero so that is shielding means there is a more electronic cloud near this compound and therefore it is shielded and it comes near zero. So now, so we need a reference compound and reference compound should have ideally the maximum shielding so that we can have that as a reference and we express that chemical shift as a zero PPM. So generally reference compound should have the resonance frequency at higher field and for most of the case this tetramethylsilent TMS is used as a reference compound. So as a TMS, a silicon there are 4 methyl group attached to it. So if you look at these 4 methyl HHH there are like a electron, high electron density around these protons and that is why it is more shielded and its resonance frequency is defined as a zero PPM. So this is used as an internal reference. So what internal reference means this compound is added to the experimental sample and now we look for the peak which is coming from TMS, we put that as a reference frequency and with respect to that all other frequency as explained. So that means here zero PPM signal comes from TMS and then we have another absorption line. So what can it can be 1 PPM or 2 PPM or whatever it is. So here TMS signal gives at zero PPM. Now TMS is can be dissolving in many organic solvent however it is not very soluble in the water therefore for protein experiment TMS is not preferred internal reference. We use another compound called DSS it is a sodium salt of similar moiety. So if you look at here, here also we have CH3, CH3, CH3 and it is a sodium salt. So actual DSS is 4, 4 dimethyl, 4 silpentane, 1 sulphonic acid. So this is water soluble and it is used in protein sample as an internal reference. Now these are internal reference. So that means they should not interact with my compound of interest and they should not cause any harm to that or any perturbation to that so it has to be inert compound. However in many case if they are interacting with the experimental compound then we do something called external referencing means we record an experiment where only reference is recorded and then in another experiment in the same solvent same buffer at same pH we record our compound. So that signal from TMS or DSS is kept at 0 and rest explained in terms of difference from that. So that is a difference between internal reference and external difference. So there are many compounds which are used as a reference compound. So one of favorite is TMS. Now there are another like tetramethylsilopropionate DSP or DSS that we just looked at. So all these compounds their reference frequency is kept at 0 ppm and these are basically reference compound for proton. So if you look at their reference frequency is at 0 ppm with respect to that these can also be used as a reference like they do not change their chemical shift too much, acetonitrile or dioxane or T like tertiary butanol or tetramethyl ammonium chloride they have different resonance frequency than TMS and that can also be used as a reference compound. Now for 13C again TMS or TSP or DSS is used you put that their chemical shift at 0 ppm other than that for N15 ammonium chloride or ammonia can be put and you put that frequency as a 0 ppm or even for 31p phosphoric acid 85% phosphoric acid is used. So all these is calibrated to are put as a 0 ppm and then other frequency are measured with respect to these compounds. Now after getting the concept of the chemical shift let us move to another concept called anisotropic of the chemical shift. So we understood that in solid there is a there is no tumbling therefore spins were aligned in many other direction and many spins were closely packed. So because of no tumbling they all the spins had some orientation effect like here if you look at the spin here it has orientation for this orange spin along the magnetic field. However in the other spin here red spin is aligned along the magnetic field and in third case it is a this cyan colour spin is aligned along the magnetic field. So all these essentially contributes to the chemical shift and therefore when we record an spectrum for non spinning static sample all these spins oriented in different magnetic field contributes to the chemical shift therefore the average chemical shift or resultant chemical shift that appears gives a very broad line and this is called peck pattern. So now this broad line is because of non tumbling or insufficient tumbling of the molecule. So even in oriented liquid crystalline sample if where there is a incomplete tumbling you get some kind of anisotropy. Anisotropy means orientation dependent chemical shift and that is very much used for many of the experiment that we will discuss later. So insufficient tumbling causes the line broadening. The other chemical type of chemical shift that we looked at is isotropic chemical shift where there is a complete tumbling and it gives a sharp line that is called isotropic. Isotropic means no orientation dependence in the chemical shift. So what are the factors that influence this isotropic chemical shift? So anything that changes the electron density around a nucleus can affect the chemical shift. So some of them can be electronegativity of this of the substituent. Electronegativity means how it can pull the electron towards itself like say here is our wage. So if you are interested in knowing the chemical shift of this proton now O is an electronegative atom. So it will try to pull the electron towards itself. Therefore it will change the electronic cloud near this proton and thus it will be kind of a desildered means there will be less electron cloud around this. So that is the for proton. Now the like that will create electron density desildering. The other another one can be direct electrostatic effect of charge and dipoles. Suppose in sample there is some charge or some dipole. Now the charge or dipole can produce the electric field and that can polarize the electron distribution around these compounds. So that polarization of electron like if there was a spherical electron density but if you put a dipole it made it something like this and that will also change the resultant chemical shift around a particular nuclei and that will change the chemical shift. The third one can be inductive effect. What is inductive effect? It is a relay polarization to the neighboring nuclei. So suppose it is not directly connected but it is coming from like little far neighbor that can also induce the polarization in the magnetic field and that can be little far away and that causes also causes the change in the chemical shift. The third one is hybridization. You know that there is a mixing of orbital happens and that is called hybridization. There are like S, SP, SP2, SP3 or something like that. So if you look at this hybridization the contribution of different orbitals changes like in S there is completely a spherical shape. When it mixes to P and becomes SP or SP2 or SP3 contribution of S orbital changes in different hybridization. So for S characteristic there is a smaller electron density at the hydrogen and that is why it is deceded but when it makes SP3 the contribution of the S changes and therefore the electron density increases around the magnetic field around that particular nuclei and therefore different magnetic field will be experiences by this nuclei and that also changes the electron density and that is the classical case that we look for CH3 which is SP3 hybridized CH2 and OH. OH is a classical case of electronegativity therefore it is deceded. Here there is a more contribution coming because of SP3 hybridization and that is what it is shielded and therefore it is coming near 0 ppm. Next effect is Winderwald effect. So this is a direct steric interaction that affects the electron density. Winderwald like if two atom are approaching closer to each other then electron density will also be influenced by the like increasing density of the contributing nuclei. So if something suppose in a protein some moiety is buried inside and same moiety is exposed there will be different electron density around that particular nuclei and therefore that also changes the chemical shift. The next one is ring current effect or contact shift due to unpaired electron. So I will just go in detail what is ring current and what is contact shift due to unpaired electron. So what happens let us take a benzene ring. So in benzene ring there is a pi electron cloud around the magnetic field. Now this pi electron is circulating. Now because of the circulation if you look at this circulating this produce additional magnetic field and because of that the electron density around this all protons are all over the ring is not same. Here you have a less electron density. So you have essentially a partial positive charge at the edge of the ring and a partial negative charge at the center of the ring and that causes also different chemical shift for different protons or different moiety attached in a ring. So here you have a D shielding and at the center you have a shielding. This is called ring current effect and therefore even the aromatic protons which are essentially like if you look at they are CH. So they have a different chemical shift than usual CH because of this something called ring current effect. So next one is the contact shift. Contact shift actually arises because of the coupling between unpaired electron and nucleus. So what happens? Suppose in a solution there is some unpaired electron right here unpaired electron and we have a nuclei. Now there is a long range coupling between these two it is called formic contacts and all those and it is also an isotropic depends upon because this is a vectorial quantity. So it depends upon what is the angle between it where this electron is. This unpaired electron affects the chemical shift and actually this is quite long range. So chemical shift can shift all the way up to 50 to 100 ppm. So this is one of the major factors that influence the chemical shift of a particular nuclei. Next one is hydrogen bonding. Actually hydrogen bonding also affects the electron cloud around a particular proton or particular nuclei. So if it is hydrogen bonded it shifts. It shifts differently in a alpha helical NH and in beta sheet and that actually changes the chemical shift for a particular protein. Now other than these factors that we discuss the solvent can also affect the chemical shift therefore precise selection of a solvent is also important for interpreting the chemical shift of a different nuclei. So if you go little bit into details and we measure the chemical shift with respect to our reference compound which is TMS. So there are groups of these chemical shift and details can be found in any textbook of NMR spectroscopy but just for giving you an idea if you just looking at this CH3 proton. So you look at this CH3 proton which is essentially this is essentially kind of a shielded and therefore its chemical shift comes quite close to 0 which is 0.5 to 1.5 ppm. But if we attach here an N now here it was C and here N. N is electronegative therefore electron are pulled away from this CH3 and therefore it gets shielded and chemical shift comes at 2 to 4 ppm. Now we attach O to this. So what O is doing now it is pulling more electron towards itself because it is electronegative and therefore the chemical shift changes from 2 to 4 to 3 to 4. Now if you attach like see here if you attach double bonds like if you look at here there was only one carbon here there are two carbon but attached with a double bond. Double bond has a pi electron. So if you look at the chemical shift changes from 0.5 to 1.5 to 1 to 2.5 because of this pi electron present and it can also influence the chemical shift of this CH3. Instead of C if you attach O, O is electronegative it can pull more electron towards itself and therefore chemical shift will be 0.1.5 to 3. Now if you add aromatic compound that aromatic compound also as we said that has a pi electron and it can also pull 2 to 3. So therefore if you look at what group is attached and how the electron cloud is disturbed around a particular proton it influences the chemical shift. Similarly if you look at CH and if you attach a halogen, halogen are known to be quite electronegative so they can pull more electron towards themselves and therefore it will be more distributed and therefore chemical shift comes around 4 to 6 ppm. In CH here the another kind CH is kind of a only one proton is here if you attach oxygen here that will also change but halogens are more therefore it is slightly less than this. If you attach N it is less electronegative therefore it comes more towards shielded region and so and so forth. So here if you compare this with this now CH3 and CH2 this remain same. Now CH3 there are 3 protons so therefore in built here there is more shielding in CH there is less shielding therefore this goes towards higher ppm value or deshielded. Similarly if we compare this with a triple bond like triple bond is has more pi electron and therefore it shifts more towards higher ppm value or deshielded value. So these are broad concepts how you know that whether your proton of choice will be shielded or deshielded depending upon what is attached to it whether these compounds are giving electron or taking electron or withdrawing electron or pushing electron that dictates how their resultant chemical shift is going to change. Similarly for carbon 13 there is a range of chemical shift and those range are expressed in terms of like how they are far from TMS. TMS is also and the reference compound. So if you look at again CH3 it is a sp3 hybridized CH3 is shielded and therefore for carbon so generally carbon ppm range is not like proton it comes from 0 to all the way up to 200 or even more than 200 more than 200 so ppm it comes. So therefore this has a wide range and in that case CH3 should be shielded and therefore it comes around 20 to 0 to 30 ppm but if you attach instead of carbon if you attach nitrogen it will it will pull electron towards itself it will be more deshielded and therefore chemical shift comes around 10 to 50 ppm. Attach oxygen to same moiety more deshielding it comes 50 to 60 ppm and now instead of CH3 it becomes CH2. So if you compare this CH3 has a 3 proton attached here it is a 2 proton attached the chemical shift changes towards higher ppm value that is it is less shielded compared to CH3. Now similarly if you attach more electronegative like BR then it goes to more deshielding and it chemical shift comes at 60 to 75 ppm. Now if you attach chlorine right so it is even more it goes to 70 to 85 ppm but if you attach N or O so N is somewhere 55 to 75 O it is more electronegative. So if you compare these two this has more deshielding effect or with electron with effect and therefore it chemical shift comes around 70 to 90 ppm. So that is what we had discussed. Now more or less I gave you concept of chemical shift how a group attached around a particular nuclei causes shielding or deshielding and all those are measured with respect to a reference compound. Many cases it is TMS in protein cases since TMS is not water soluble so DSS is used and we looked at what is the range of different chemical shift. Details of these chemical shift range can be extracted from any NMR textbook. In the next class what we are going to do is looking at another important concept which is actually spin-spin coupling. So here what we had looked at how the electron around a particular nuclei is influencing its resonance frequency. Now in the spin-spin coupling we are going to say how one nuclei is going to affect its neighbor nuclei. So this is nuclear like a nuclei-nuclei talking rather than nuclei-electron talking. So our next class we are going to discuss about spin-spin coupling. I look forward for your active participation. If you have questions please write to us we will try to respond each and every question incoming lectures. Thank you very much. I am looking for the next lecture.


And then there's this chemical in the brain that's released from what Descartes referred to as the seat of the soul and the Egyptians referred to as the eye of Boris, the only unilateral structure in the brain. It seems like a very interesting structure and it's been implicated in, well, mystical states only because it has, there is some limited data like Horosio is alluding to that it's data is not very convincing, but it is the data that does exist that shows that it's a precursor to a chemical that's naturally occurring in the brain and is DMT. It's a psychedelic that's used for very short, intense psychedelic experiences. And so that's why I was thinking with Sadhguru, you've been referring to the moon and working with the moon, the cycle of the moon specifically for spiritual purposes. David, the thing is, see in the yogic system, there are simple processes with which particularly on full moon nights, where you can get totally stoned out just on moonlight, there are processes to do that and as you can see, I don't know what that DMT is, you can see I am on it all the time. No substance from outside, it's from within. It is in some way on one level, in the physiological level, the significance of yoga is to stay in tune with the sun and moon cycles in relation to the planet. That is the only way this life can be in balance and with least amount of friction within itself because then the life's experience, as I mean whatever the chemicals you're discussing serotonin, melatonin, all this stuff, essentially you're saying chemistry determines the nature of your experience of life itself. So to maintain that chemistry which is always blissful and ecstatic, moon and being aligned with the moon is a very important part of it. So there are various practices associated with that. One thing I would like to request Professor Horace is if they can invest a certain amount of attention towards what is happening in the gut with the full moon, new moon, I think it may be far more easily measurable than what is happening in the brain because what is happening in the brain, we see it only as a consequence. The real thing is happening in the gut. You may see certain effects in the brain. What is a consequence is one thing, but the process, the changes that are happening within the gut region is very big. What is happening is there more solid kind of stuff. It is more tangible that maybe it is more easily measurable than what's happening in the brain.