get the value out that I'm ready to print something Um, in fact, let's make this a little bit more interesting and make it be a No string is fine. Um So now the question becomes What goes in here and let me write something that's not cancellation safe to show you what can go wrong Uh, I have a buff here Uh, that is you know, 1024. Um, and I do something like, uh Let's say I have a tcp here Oh Tcp stream connect 1 2 7 0 0 1 4 80 80 await unwrap So here this takes a, uh New tcp stream And down here I do tcp dot read mute buff await unwrap Uh, and then What I actually want to do here is loop Uh Actually, let me just do s is string new Like this and then I do s dot Pruster or push Uh buff n And then if s dot len is greater than 100 Then I return, uh s 0 8 And bring me a tokyo tcp stream And bring me read async reduxed And That's fine. We'll do, uh We'll just assert that this is utf 8. Why not? All right So now I've written a read a sturdy json value that just returns a string It doesn't I guess sturdy json value here's a lie. This is really read a string Read a string of 100 cares That's really what I've written. So let's let's name it like that um So this all looks fine, right for sort of naively, right This asynchronous function creates a string. It has a buffer. It reads into the buffer. It keeps Pushing on to the string. I don't actually need the end here. No, I do need the end here It keeps reading bytes out of the tcp stream Parsing them as a string pushing them on to the string It's accumulating and once the string is greater than 100 characters, then it returns us looks great, right? The problem here is what happens if this connection is slow So imagine that we go through this loop So we enter the loop first we go into the select and then the select tries to execute this function This function runs it allocates a buffer. It does a tcp read And let's say it gets I don't know it gets 10 bytes 10 characters So it takes 10 characters. It pushes it on to the string The length is not greater than 100. So it doesn't return it loops It goes back here and now it tries to do a tcp read again And remembering now this now has 10 characters in it, right? But this await realizes there's no more data to read at the moment. So it goes to sleep and yields Okay, so that means that we end up back here up at the select the select tried to execute this But this future isn't ready yet. So it then goes to await the next future. Okay, so that's a sleep Great. So it now tries to await the sleep So now we have two futures that are and you know, it hasn't a second hasn't passed yet So now it's two futures. Both of those futures are not runnable And so now the select kind of just waits for one of them to become runnable So far so good Now let's imagine that One second passes so this future now becomes runnable like the io event queue or like the thing that drives timers rather Now has a full second expired. It realizes that this particular sleep future should now be awoken It calls wake on that on the waker for this future that moves that Future from non-runnable to runnable the select picks up on that and so it goes around again It might even try to pull this future again, but it still says I don't have any more bytes So it goes to this one. This one now resolves. So we enter this block Now that we've entered this block, we do this print and then we finish the select and when we finish the select what happens All the other futures that didn't finish get dropped. That means that this future gets dropped And then we go back around the loop and then we do it again And we construct a new future for this and a new future for that The problem is these 10 characters that we've already read Because they were in this future the first time around the loop And then when we dropped this future At the point of finishing this block then those 10 characters are dropped because they were contained within the future And so once that happened then we dropped those characters. They're gone forever So when we go back around the loop and then we create a new instance of this future That means we call this function a fresh It starts a new string with no characters in it and then it starts reading into this So we've lost we've dropped on the floor 10 characters that we're never getting back And so we might actually if this connection is slow enough We might never get into this branch because we keep reading a couple of characters But then we keep hitting the time out which then causes us to Exit the select which causes us to drop the future which causes us to drop all the characters we read and then we loop back around This is an example of The problem of cancellation the problem of cancellation when you're using selects is that you have a future that got cancelled It got aborted it got dropped and it had some state that was actually important that went away These 10 bytes are just as if they were never read and this can lead you to really problematic situations like Imagine that this was not reading just a string of characters. It would actually reason reading adjacent object Maybe you now dropped the opening curly bracket And so now as you keep going around you actually never end up with a valid jason object Because you never got the opening bracket or rather you got it, but then you dropped it This leads to enormous problems down the line where you just silently drop state of futures And this is the really big thing to be aware of whenever you use selects Is that either your futures need to be what's known as cancelation safe meaning that they can be resumed without having lost any state or You need to write your select so that it doesn't matter like these futures are never Attempted to be reused but cancellation safety is usually the thing that you want. There are two ways to achieve cancellation safety and the first of them is that you'll see on a bunch of tokyo types like If you look at mpc receiver for instance, you'll see that the receive method says under cancel safety This method is cancel save If receive is used as an event in tokyo select statement and some other branch completes first It is guaranteed that no messages were received on this channel The implication being that there are no Messages that might be received and then get lost forever as a result of that future of receive being dropped So receive for an mpc channel is safe to use in this kind of context This asynchronous future is not cancellation safe If you drop its future and then try to resume You will have lost state or you may have lost state, you know, if it if it never did any read, of course you haven't lost anything So In general you want to read through anything that you use in a select and make sure that they're all cancellation safe And the way to make something cancellation safe if it if it isn't is to construct the future outside of the loop like this And then in the select Use a mutable reference to the future instead Because that way you're never dropping the future when you get to the select because the future was never owned here in the first place And so that way when you go around you're actually resuming the same future This doesn't quite work because of pinning So the actual way that you need to do this and I think there's an example of this down here Oh, yeah, these are the things that are cancellation safe. These are not cancellation safe. You need to be careful But there's an example here of how you can avoid this problem Where you pin it outside of the of the loop and then you take a mutable reference This pinning is kind of annoying, right? You just but it won't compile without it. So at least there's that pin And I think actually we can use std future pin here Std task pin std pin pin. Okay, great std pin pin And then you can pass a mutable reference to it here instead But you do need to pin it in order to use this trick But once you do then now you you don't have to worry about the cancellation safety because you're not ever canceling that future So this is what we mean by cancellation safety I have more to say on this but I'll pause here for for questions first It's just only because selected is in a loop If select is not in a loop this tends to not really be a problem because it kind of it usually means That you don't care about the other futures being dropped Because they were constructed in here and you're never really resuming them because you're not running that line of code again So it only really tends to come up as a problem in a loop You could probably write some code where this problem arises even though you're not using a loop But that is much less common What does pinning do under the hood does it change the type of feud? Yeah So the future type here actually becomes a pin mutable reference to The future that comes in here. So it turns this into it's no longer just this future It is actually a pin a pinned mutable reference to that future Does this mean that ideally we need to make sure that all the processes and the http handler is cancel safe I'm not sure what you mean by http handler. This only applies to things that are awaited on in a select In the the arms the the arm definitions of the selects Yeah pain was a good freedom slip here. I agree um Yeah, so the behavior is different now Uh, which is good totally correctly pointed out here that here we will now only ever read one of these But that's kind of what we intended right because we had a break here So we intended for the loop not to continue anyway the moment this resolves Select is more like a future than a loop, right? We'll we'll select is A future that internally loops over all of its arms So kind of both You should think of select as being a future it isn't But that is sort of what it is Um, it is also internally kind of like a loop In this scenario the future is never canceled and it's just ignored Um Well in the in the pin case the future is never canceled in the old case uh When we say, um cancellation safety and when we say a future is canceled What we mean is that is dropped before it got to finish So it got canceled it got aborted right it started doing work It accumulated some state and then we dropped it rather than letting it complete in other words. It was canceled So in this case we did cancel it Uh in the pin case it does not get canceled because Dropping it here does not mean that it doesn't get to do any more work the next time around the loop Um If we pin the future once it resolves won't that make it a one shot future. I'm not sure I understand So one shots are different from this one shot is a mechanism for having two different futures communicate with each other So that one can send a value to the other Uh, it has nothing to do with a future only resolving ones all futures only ever resolve ones This versus calling timeout on fut in a loop Not sure I follow Uh, could we have made the future cancellation safe by passing in the string that we write to? Yeah, so that is another way that you could solve this problem. So instead of pinning here You could do this and then do At mute s And then have this like I guess not return and instead it'll print s here This will take an s which is a mutable reference to a string Uh, this no longer gives s here Uh, like so yeah, so this is an example of something that would also make it be cancellation safe Um, a but that assumes that that's only true if read is cancellation safe Uh, and we can check whether read is cancellation safe. So if we go to read, uh As in greed xed Cancelation safety this method is cancel safe if you use it as an event in tokyo select and some other branch completes first Then it's guaranteed that no data was read. So only because of this clause is this safe But given that clause, this is cancellation safe And usually I tend to tell people always document the cancellation safety of anything that you call from a select If I have some work after the loop that does not depend on the food Uh, we need to manually drop it to avoid unnecessary background processes. Um, okay, so this is in the This is in the pin case So if we have this then yes down here, you might want to drop feud Um, but it doesn't actually matter because remember this is not an actively running thing This is just the state of the future And so you're not really wasting system resources. You're wasting a little bit of stack memory You're not really like wasting compute resources because this future if it's not being awaited or run in a select It doesn't do anything. So you're not wasting compute Um Is whatever tokyo sync mpsc uses for cancellation safety the same or related to this mute technique? No, so the way that you can implement cancellation safety internally enough in In some kind of resource is just by having to do atomic things I don't necessarily mean cpu atomics, although that that can be the way to do it so that You basically don't have any wait points between when you consume a value or like consumer Resource like reading bytes from a file socket and when you return them So you know that if you consume the resource then you definitely also return it That is the way that you guarantee cancellation safety If you consume some resources like you read some bytes off of the wire And then you have some weight in there before you return that value then you're not cancellation safe Because you could be dropped the at any await point you could be dropped um It's funny how cancellation unsafe code is still safe rust. So safe rust is a very particular meaning right safe rust means that it cannot have Memory on safety things like one type being treated as another or have undefined behavior There's no undefined behavior here.  So it's wrong, but it's not memory on safety and that's what we mean by safe and rust Uh, so this is more of a definitional question of what the safe mean Um, and when we say that something is unsafe Do we mean the formal sense of rust unsafety or do we mean unsafe? And this is in general like this could lead to buggy behavior and those are two different meanings of the word Um What I mean by one shot is that it can't be reused in the select This is only useful in the case where you only want to use this kind of future once um Well, so sort of right so you could hear uh, if you end up in this branch you could then say um You could reassign to a few tier you couldn't actually do it this way because of the pin But you could imagine that you reconstruct the future in here. Um, in fact, I think you would use Is it uh Dot replace I can never quite remember what Um Set so you could do this in order to Uh Once you've read a thing you if you actually want to keep looping around then yeah, you would replace it with a new future So that but inside of the pin so that it itself Remains cancellation safe Um So if you want something to happen in one select arm on interval then also the interval should be to find outside the loop Right. Yes. So that's also true. So this thing will not happen every second It will happen a second after the start of every select So if you actually wanted something to happen on an interval, well, you would probably use the interval. So tokyo has a Inside of the time module um I haven't really talked about time because it it's fairly straightforward the only thing to know about here is that there's a Minimum limit on how long intervals can be in tokyo like it doesn't really support. I think intervals shorter than 100 milliseconds or something. Um But you can have an interval that you can construct And then what you would call inside of here is dot tick on that interval But you would construct the interval outside the loop precisely for this reason And in fact if you look at tick cancel safety this method is cancellation saved So make sure to always check that um I wonder if a different select API could exist that makes it less likely to make a mistake here You are not the first person to wonder that and we haven't really come up with one yet If you have a good idea then absolutely, but It it's actually surprisingly hard to figure out what should happen here. One of the questions is should futures Be cancelled on drop like what it should the drop behavior be for a future Currently it means cancelled, but it's not clear whether that should be the case. Um, there's this also ties into the question of Asynchronous drop so currently If you have a future and it implements drop then your implementation of drop is not async It like drop is a synchronous method as that means you can't do asynchronous things in your drop But trying to fix that is all sorts of complicated for other reasons. Um So this one's uh, this one's tricky um All right, so uh, that is Cancellation safety which mostly comes up in the context of select. Um Last thing then to talk about in terms of utilities is Is tokyo util so i've talked a little bit about this already so codec is the way that you connect between async read and async write and uh stream and sync Um, so that one we've already talked about i'm not going to dive into how it works Uh, I think if you need that you can go look at it This is after after all tokyo de-crossed not the tokyo ecosystem de-crossed um Compat is a way to have interoperability between the async read and async write traits from tokyo And the async read and async write traits from the futures i o crate which is used by some other asynchronous runtime in some other crates um There are a couple of useful helpers here inside i o and net for things like, um Um Things like bridging between synchronous and asynchronous i o uh things like copying things in and out of bytes Things like having um Uh, I don't even know what this thing does. Um, but under sync. I think there's some interesting stuff Uh, I'll talk about cancellation token in a second. Um delay queues for like basically things that are bigger Bigger and also a little bit more unstable constructions on top of the tokyo primitives And tokyo primitives are usually sort of lower level things building blocks that you can use Where's tokyo utils tries to provide higher level abstractions on top of that But there are a couple of things that I want to call out from here Um And they're both unknown one is under task and the other is under Where is the other one? Where is task tracker? Ha, it's even deeper. Okay So the first one of these is cancellation token. Um cancellation token is unrelated to cancellation in terms of select What cancellation token is for is specifically imagine that you have this like giant system You have a bunch of tokyo spawns like you have a bunch of different background tasks running And at some point like the user presses control c And you want to have graceful shutdown you want your entire program to like you want all of those different Uh tasks that you have to start exiting One of the ways you could do this is you could as we talked about on the runtime call Um shutdown and when you do that The next time every task yields it gets dropped and then eventually all the tasks are dropped. And so eventually you're your shutdown Returns because all the tasks are gone and now you can shut down But in reality, you didn't really let those futures finish up their work You just kind of dropped them when they were in some kind of pending state One way that you can improve on this situation is to use a cancellation token What this type does is that you can It's kind of like a notify and in fact, I think it's implemented using notify internally But you construct one you can clone them and give them out to all of your different tasks that you spawn and the idea with it is Every cancellation token has a canceled method that is a an asynchronous function That will only return the moment someone calls cancel on the cancellation token But all of the consulate all of the clones of a cancellation token are connected So if you call cancel on any clone of the cancellation token Then the canceled future on all the cancellation tokens will suddenly yield So the idea here is that for all of the tasks that you spawn in various places You have them do a select over the main work that they do and canceled And so that way whenever they get this canceled signal, then they can choose what to do in order to clean up and then they can choose to exit And so this is a way for you to have one place in your program where you say, okay time to shut down and then at that point All of the threads start hitting this clause of their select at that point they can choose what to do Maybe they exit their loop. Maybe they do some cleanup whatever But eventually when they get that signal, they should know that it's time to wrap up Stop reading from channels You know, stop reading from tcp sockets, whatever it might be and just sort of finish up and then eventually return And hopefully at some point after that your runtime should become idle There should no longer be any runnable tasks Because they should all have sort of reacted to either this cancellation signal or Cascading signals from that like, you know, they were reading from a channel and the sender of Of the sender end of the channel they were reading from has now been closed because that thing saw the cancellation signal So eventually it should sort of propagate all the way down until all of your tasks have gracefully exited and the cancellation token is just a handy means to have the trigger to start a shutdown and the Means in which to react to that shutdown signal There is also a Stream cancel there's an equivalent type specifically for streams um that has uh Sort of this this trip wire thing where you have you can take a stream and you can wrap it with A thing that will close the stream like have the stream start to yield none The moment someone calls cancel on this trigger and so it's the same kind of construction It just forcibly makes a stream start returning none with the same ultimate purpose Um So this can be a a really nice way to Give your application the ability to do graceful shutdown But it does require that you thread this trip wire or this cancellation token all the way through your program So that every task has a way to know that okay, it's time to shut down now. Let's see Can you keep reading from a channel after the sender has been dropped? So it depends If you look at The tokyo mpsc Channel right it is a multi producer single consumer And so the rule is that when all the sender handles have been dropped Then it's no longer possible to send values into the channel This is considered the termination of the stream. And so then poll will start to return none But if one recender is dropped, then that doesn't mean that the channel is closed and you you Will still block waiting for more values There is a way to specifically say um Where is it? No, there's not i am lying But yeah, so so the the answer to this is like the a channel If all the senders have been dropped then the receiver will still receive whatever may be in the channel But after there is no more it'll get a none and then at that point no to stop reading similar to an iterator really um What if you're writing a library like an actor library? Then it's hard to share of the cancellation tokens as the user might create their own one It's true. And and this is one of the ways in which it would actually be handy if Cancellation tokens was in the standard library so that everyone could agree on how to signal It's time to finish now Until that happens, and I don't know whether it will it would probably go inside the context maybe um Until we get something like that if you're a library author You could have sort of two apis right one that takes the cancellation token And one that does not and if you don't get one you just do whatever you normally do which is you block forever or you um Your awaits might take forever during shutdown. You might just get dropped But if the user provides you with a cancellation token, then you're willing to respect it um That of course would mean that you would take a public dependency on the tokyo util crate Which you might not be willing to do But you're sort of caught between a rock and a hard place here. There's no great solution for having libraries that support being Uh gracefully shut down sadly Unless you just write your own type that wraps this that you tell people to write in and then they need to figure out How to bridge between them It's unfortunate Because token is not mutable Can I use it as a static to avoid needing to pass it to every function that's going to depend on it? um You know that is an interesting question um I think you can actually because I think it is yeah, it's both send and sync So yeah, I mean I think in principle you could you could have a You uh, it couldn't actually be a normal static because the con unless constructor is const Yeah, the constructor is not const And so you would need to use something like a once lock in a static But if you did then yeah, you could use a static for this It might get really annoying for things like tests right because all of your tests will use the same static So if you go cancel in any test all of the tests will exit and this is the the perpetual problem with statics Right is that they truly are global But if you don't have tests then you could probably do this. I don't know that I would recommend it Well async drop replace cancellation tokens No, even if we had async drop That would be convenient But it wouldn't replace cancellation tokens because async drop is just to allow you to do Asynchronous operations when you are dropped But cancellation tokens are the way that you learn that you should exit and should start to drop yourself Um Yeah, often the way you would do um Cancellation and libraries you would have like Give me something that implements future and I will treat that as my cancellation So you just take anything that implements future with the output type as unit and Someone could pass in a cancellation token or whatever their own means are It doesn't really matter right so so they could just take Call cancelled and just pass this type in because as long as you're generic over what you take you're okay If you pass the cancel token every function does not behave just like go context. Yeah in some ways does um All right Uh This is a type called task tracker that can be useful to know about in in tokyo util It's sort of a combination with cancellation token. Uh, you can use it to get graceful shutdown I'm not actually going to talk about this in detail, but there's a decent Uh Blog post on this on the tokyo blog on graceful shutdown that I recommend you read talks a lot about exactly this And how to wire it all together correctly and how you can use task tracker to make the make your life a little bit simpler Um All right We're on the home stretch. I think those are all the utilities I wanted to talk about Now the only thing that remains is a couple of like Foot guns that are particularly common that people run into all the time Um, and I'll go through these pretty rapidly because they're they're not huge topics They're just things that you just need to be aware of so that you don't shoot yourself in the foot with them um Okay, so number one took your spawn When you spawn something Uh, and I think I mentioned this earlier briefly, but I want to reiterate it the join handle that you get back When you drop it, you're not waiting for that task that task continues to run in the background And there are two implications of this the first one is Um, when you drop the join handle, you're not guaranteed that that task has finished doing in the work But also it continues to run it continues to use resources And so because it spawned because it's running in the background Just dropping it is not sufficient if you want your system to not be wasting resources over time So if you truly want it to stop then you need to do something like call abort So that tokyo knows that the next time that future yields, it shouldn't resume it. It should drop it It should cancel it instead So that's the first aspect of this that's important. The second aspect of this is When you return from an asynchronous function that happened to run tokyo spawn You now have this thing that's running in the background that might be doing something important But the from the outside from the caller's perspective of the function that internally did in a um a tokyo spawn The caller doesn't know that there's now this background thing, which means crucially it doesn't know that it might have to wait for it So imagine that you have a background thread that's like, I don't know you tokyo spawned doing a uh A file copy Well, if your main your asynchronous main function calls the thing that calls the thing that calls the thing that ultimately spawns this thing that copies the files But nothing awaits the join handle Then what's going to happen is the thing that spawned the file copy returns And then we return return return return all the way back to main main also doesn't know about this background spawn That's going on so main finishes and at that point all outstanding tasks are Dropped because main exited and at that point your file copy stops in the middle And so it's really important to remember Whether you want to do something whenever you do a spawn think about the fact Or consider every time you do a spawn assign it to a variable and decide what to do with that variable If you truly want it to be dropped Then you can drop it. That's fine But you should make a conscious decision about whether dropping is the correct choice or whether it should actually be awaited Or whether it should actually be communicated further up or whether it should actually be aborted But make a conscious decision rather than just letting the the default behavior come back to bite you later um Yeah, it's not as though a background task is necessarily useless, right? It could be you actually wanted to run in the background. You don't want it to be aborted You don't want it to be awaited like it's a I don't know it's Your logger or it's um forwarding something from a channel or reading something from a channel So it could be the correct decision is to sort of forget the handle like drop the handle and just let the background task running It's more that that it should be a conscious choice. What do you want to happen? Uh, I think the default behavior is actually reasonable as a default It's more that you should really think about whether that default applies to you Um, okay Second thing to be aware of is the tasks are single threaded So when you do a tokyo spawn or whatever you do in your async fn main Is a single task, right? We've talked about what tasks are It's a single task which means that it's being executed by a single worker pool thread If you do a bunch of work in different futures, but they're all in a single top level task That means all of that work is only being executed by a single thread If you want work to be spread across multiple threads Then you need to spawn those futures so they become top level tasks so that they can be executed in parallel if you just use something like select or join or Well, not a join set because it spawns for you for exactly this reason, but if you use a join or a select Or a futures unordered for that matter Um, what you end up with is a bunch of futures are executing concurrently but not in parallel So they're all being executed by this one thread But only one of the futures being run at any given point in time That may or may not be what you want Sometimes you do actually want to multiplex them on one thread But you should be aware of the fact that futures embedded in futures are always single threaded You need to spawn them and turn them into tasks in order to get parallelism uh And then the final thing I want to talk about is uh a footgun that I see pretty often in an overreliance on D channels So we see those in particular with the actor model actually where you know, you spin up something that's an actor It owns let's say a tcp connection And what you do is you keep sending messages to the actor for things like write these bytes write these bytes write these bytes or Handle this request handle this request The actor is again a single top level task As a result it only has a single thread and as a result That single thread has to handle all of those requests one at a time Now that's not inherently a problem, right? It might be that one at a time is the fastest you can write out on this tcp stream anyway And if you had multiple Tasks all trying to write out to the same tcp stream they would need a mutex or something So that's not inherently bad The problem you run into is that when you have a Sort of fan in pattern into a single actor a single task that needs to do something about them It needs to pay a cost for every channel receive And again, if you think of these receives as being like write these bytes or handle these requests Sometimes that's reasonable, but very often the loop on the actor is probably going to look something like you know while While let some equals channel dot receive and then you know do a bunch of work with that message That means that by the time it comes around Depending on how much work there's in the loop by the time it comes back around to read from the channel again There might be five new things in that channel But it needs to read them one at a time even though it might be there might be more efficient ways to handle those incoming requests in batch How you choose to do that batching?  Um, you could use things like a Well watch is not really appropriate here But you could imagine for example that instead of having a channel to this actor The input to this actor is actually a mutex over a vacuate And so if you want to send something to the actor what you do is you take the mutex You extend the vector with the bytes you wanted to write and then you drop the mutex And then the loop on the actor's side no longer looks like Get you know one small set of bytes and then go again and get a small set of bytes Instead it takes the mutex It takes the entire vector out and leaves an empty vector behind and then it writes all of those bytes out at once That might be way more efficient And allow it to get to much higher bandwidth Because it allows it to to batch the cost across multiple calls into the actor This pattern doesn't always work. It doesn't always fit what the actor needs to do But but in general there's a tendency I see to over rely on mpsc channels, especially for actors Where it's often the right call for the simplicity of actors, especially at lower throughput, but when you get to Systems that strain the resources more You might want you might want to start to think about these kinds of optimizations to amortize the the cost of communication and synchronization here I think I think Those are all the things I want to say about tokyo But I'll I'll spend some time in chat and see if there are more things again Doesn't have to be about what about the sort of foot guns I just talked about but across the whole stream Let's do sort of quick. I don't want to say q&a, but a quick Everything I talked about at tokyo. Are there things that you still get confused by? You want more detail on Things that have bit you in the past to sort of round off the stream here uh Sounds a bit like a ring buffer. Yeah, it is kind of like a ring buffer In fact, you can use a ring buffer for this if you're willing to do the copy Um to copy out of it rather than a steal um What things in tokyo are actually parallel not just concurrent. It's only tokyo spawn um And of course runtime spawn, which is the same spawn blocking is parallel because it runs on a different thread Joint set is parallel because all of the all of when you Give a future to a joint set it spawns them. So basically the answer is only things that are spawned are parallel And the i o event loop is is parallel um When doing batching usually you want to paramerize a parameterize it with two inputs the max batch size and the max wait time for a batch to fill Um, is there something in the tokyo ecosystem that implements such a pattern? Not that I know of but we're talking like five lines of code. So I I I don't think so, but maybe Request says he wants a tokyo 1.x runtime when running with another runtime is tokyo doing something special their quest needs um, well, so this this uh This gets at the fact that In order for tokyo to know how to move when futures need to move from the non runnable queue to the runnable queue It needs to know which things those futures are waiting for Which means that when those futures use a for example a tokyo tcp stream Um, that's how tokyo knows is because tokyo implemented the tcp stream So it knows what to wait for so it knows when to move them and basically how to implement wake um If you try to use a non tokyo runtime, but you're trying to use the tokyo resources You run into a problem because the tokyo resources like like a tcp stream When you call poll on it and it realizes like let's say you call poll read on a tcp stream It knows that oh, I didn't have any bytes to read. I need to register myself somewhere so that the runtime So so that I can notify the runtime when I need to resume Basically, I need to register myself with the i o event loop So it looks for an i o event loop Specifically the tokyo i o event loop and it doesn't find one Then it doesn't know where to put its waker and its file descriptor for something to be picked up later Uh, and so that's why when you use the tokyo i o resources You need to also use the tokyo runtime because the i o resources are tied to the i o event loop Um, if you use let's say the the async stood runtime or something But you're just trying to use a tokyo resource Then when a poll read happens it doesn't have anywhere to to put its state that it it can rely on will call wake later And so Request I believe uses tokyo resources All the way down like it uses tokyo tcp streams for example And you can't run that in a non tokyo runtime Because that connection with the i o event loop would be lost Um Does tokyo not do mpmc? No tokyo does not have a multi producer multi consumer channel Uh, if this hasn't been asked already, why does the async write x have implementations for each individual type such as write u8 instead of a generic time parameter Um, I think that one is because it's not clear what it would be generic over Right, let's say it's you had a right Number right and it's generic over t Where t is what? Right, like what what is the bound that allows it to take a number and turn it into a sequence of bytes? And this is it gets complicated, right? Like if you get a u128 The order in which you write the individual bytes of a u128 out to out to the writer depends on the endianness for example and so you you There's no there's no generic bound to really express that and so that's why you end up with individual methods Uh Can you talk about broadcast and the overhead that comes with having a bunch of receivers and senders? um So there's there shouldn't be that much overhead when using broadcast with with multiple receivers I don't think broadcast allows multiple sender Right is it uh Sender oh, yeah send takes a reference to self. Okay, um So there's not I think a huge overhead to having multiple The main problem you run into with broadcast is if you have slow readers They hold up all of the readers or rather it's not quite true They don't hold up the readers, but they either hold up the sender or they cause rights to start to get dropped and they start losing messages Um, but that's just sort of an an inherent problem here. This is the the lagging problem that you see um Does tokyo spawn create an os thread or a green thread it creates a green thread Um, what about keeping a mutex guard between the weights? We already talked about that earlier Is there a way to access the blocking thread pool of the runtime? Can I get it to a handle to it and send it somewhere else? Um, sort of so if you look at runtime Uh, there is a method called handle that gives you a handle and on a handle Uh, which is clone there is a spawn blocking and spawn blocking lets you run something on the pool and get an an asynchronous Future back So so you can pass a handle around and use that to spawn things onto the blocking pool But that is your handle to the blocking pool Uh, can you talk about how you deal with handling what would normally require async drop like losing Third party async libraries you would like to clean up on drop, but those cleanups would be async There's not a great answer to this async drop. Just it sucks that we don't have it and it's really hard to get You will see that for a bunch of types not close as a bad example of this, but um What is a good example of this Um, I guess actually maybe file Um, no file is also a bad example here. Um I guess tcp stream maybe Or buff stream Maybe buff stream is the one I want to talk about. Um Although maybe that doesn't solve the problem either I guess maybe there isn't an example of this in tokyo So what I'm looking for is you'll see a bunch of asynchronous libraries have An asynchronous function called close or something like it Um, and the idea is that If you know that you want to drop this type then you should call This function in order to drop it So it's a close would be an asynchronous function that consumes self And so the idea is that if you have the ability to manually drop it Like you know that you're going to drop this thing rather than relying on the drop trait Like it just going out of scope call this method instead and then it will actually get to do its asynchronous cleanup The you also see this actually in in synchronous methods where, um You might have a close function just because the close thing might error And so if if you just let the thing be dropped that was the error from the result is just silence is forgotten So you are encouraged to call the close method if you can in order to recover that error or see that error And not have it just be silenced And you can the the same thing applies for async that you can provide an async function close That doesn't perfectly solve the problem right because there are cases where It's just going to be dropped and you don't really have control over logic that happens on drop except through the drop trait when that happens that the best thing that I know of here is um So there's a handle current or try current that lets you get a handle to the current Tokyo runtime and then When you get that handle you can then spawn code that would get to do your cleanup stuff It's not beautiful It means for example that if you get dropped off of the runtime Then now you don't get to execute anything to do that cleanup properly But it is sort of a best effort asynchronous cleanup Ultimately though this is a problem and we just don't have a great solution for it um Tokyo sync broadcast is not in multi producer multi consumer. No broadcast. So broadcast Is specifically not a multi producer multi consumer Q it is a broadcast Q what that means is and the distinction is an mpsc an mpmc channel is Any sender can send a thing and one of the consumers will get that thing A broadcast is any sender can send a thing All consumers will get that thing. So the semantics are different Uh, does the can do the cancel safe Futures or methods internally revert state? How do they know when to do that if they can become cancelled in the middle? Um, it depends actually So some of them are just written in such a way that they are cancellation safe So they don't actually have to revert anything. They're just written in such a way that there's no a wait point between the The sort of commit point like the point where they change externally visible state and when they return And so therefore they're just naturally cancellation safe and some of them will actually have a sort of drop guard So they'll construct a type Internally that they create an instance of before they do some stuff And so if the whole thing gets dropped the drop implementation of that instance will do some revert Stuff that is a much more complicated pattern, but we do see it sometimes Um You think tokyo will remain the most common asynchronous runtime in the future um At least for the foreseeable future. I don't see any particularly relevant general purpose runtimes at least um There are runtimes that are better suited for sort of specific domains, for example, like embedded But I don't know of any that are sort of compelling as a as a general purpose executor at the moment Um, there's a send method on channels called wake. Yes That is that is exactly the way in which they uh allow the receiver to Realize that oh this future has now moved from non-runnable to runnable Um Why doesn't tokyo implement unix sockets of the type sock sec packet? I don't know. I don't know what that Variant of unix sockets are there are a bunch of like utility crates around tokyo that That provide like alternate socket types, for example Um, I think also there is uh, so this is a thing called async fd Which is basically a wrapper type specifically on linux on anything that has a file descriptor Um that implements the standard Unix sort of read and write interface. Um, so you might be able to use that but tokyo is never going to support like Every single possible resource Directly in the library that would just not be feasible But the idea is that you should be able to write Additional resources as other crates that people can use and they will interoperate nicely through things like the waker Um Would changing an asynchronous function that is pub from cancellation safe to non cancellation safe be considered a breaking change for sember That's a very good question. Uh I There's no well-defined answer to this I would claim that the answer is yes because you have changed the correctness contract of the function Right like someone has been using your code correctly Uh, and now you have made their code incorrect By changing your code and they were relying on something that was At least assuming the cancellation safety was publicly documented before then it would be a breaking change If you never promised anything about cancellation safety, then no, I don't think it's a breaking change So this becomes a question of documentation really did you ever promise that it was cancellation safe in general? When you call third-party libraries for things you should be pessimistic about what they promise you So if they don't guarantee a thing, you're not allowed to assume that thing or if you assume it You should also specifically check it because it's allowed to change um Any plans on de-crossing the small ecosystem? No I don't think it's particularly relevant um Tokyo is not really good for real-time applications. I mean That's generally true for Anything that doesn't use a real-time operating system So but but but this sort of gets to what I was implying with embedded, right? Is that there are runtimes that are better for specific kinds of use cases? Um, so I was more referring to like a general purpose asynchronous runtime um Are there any workarounds for synchronous to asynchronous to synchronous to asynchronous call stacks? I mean the answer is basically don't do it. It will come back to bite you the slightly less um slightly more helpful answer is Channels are usually the way that you do this Whether one shot or mpsc. They are the best means we have for really bridging that gap um things like block in place are Crutches where if you use them a lot or similar like handle current if you use them a lot You're going to run into really sad times as a result Because those those nested patterns end up just performing really poorly having really hard to debug runtime failures You you don't generally want those like sync async sandwiches um Okay, I think I got to the end of chat Oh Okay, so I hate this so much YouTube has this feature now So I have the chat window up on the side and it defaults to top chat not live chat top chat Which means that I might have missed a bunch of messages. Let me change that to live chat and see what I've missed Uh Okay, it looks like I haven't really missed anything great. I am not an elixir user. No um What are good strategies for migrating from sync to async? I find that trying to write certain modules using a runtime and throwing handles around is difficult. Um There's not a General purpose solution here for rewriting things from sync to async Um, I think I've usually found that you kind of want to go outside in rather than inside out Like trying to write wrap asynchronous things with synchronous things um tends to be a bunch of pain because um You now end up spawning multiple run times in each place where there's an interface if you start from the top You can have one asynchronous runtime And initially you're going to have things that block the runtime and you can use block in place or spawn blocking and so that Direction of the port tends to be easier and then you just push async more and more down so that tends to work better um The problem with real-time async is I don't think rust is the right abstraction since real-time requires to some extent detail how to make a cpu Uh How much cpu a process is going to consume it's unclear whether in real-time operating systems You this is the right interface. It might be um, but it's much less clear um At the domain is without boats without dot boats Uh Are you a heavy user of tokyo? At helzing. I mean, yeah, tokyo is the asynchronous runtime like I don't for for general purpose Uh asynchronous code. I don't see a reason to use anything else um Okay I think that's it. I think we're done and hey Okay, I start when I started two three hours and 33 minutes and 33 seconds I think I guessed three to three and a half hours. So I'm pretty happy with that All right, great everyone. Thank you for coming out and I hope you found that interesting and I'll see you for the next stream


It's like they're all over the internet.  What's wrong, boy? What is it? What's wrong? What's that, boy? Traffic's backed up on I-25. Well, good thing we followed an apartment on Twitter. Now we get all the updates. This is really, really rare. Here we have a Denver Police Department Facebook follower in his native habitat. The hands are lightning fast, lightning fast. Officer Morales, did you watch DPD TV on YouTube? You want answers? I think I'm entitled! You want answers? I want the truth! Then watch us on YouTube! Well, he's dead. What do you want to do about it? We need to get this out on social media.


So our next speaker is Dr. Yuhei Takaya. Yuhei is a researcher in the Atmosphere, Ocean, and Earth System Modeling Research Department for the MRI, JMA at in Japan. Yuhei also gave a talk during the colloquium couple of weeks back. Thanks, Yuhei. Can you hear me? Yeah. Do you see my screen? Yeah. OK. OK. Thank you, Anish, for the introduction. And I would like to thank the organizers, the Anish, and Judith for a lot of work. And today I am going to talk about the enhanced Mayu Bayu rainfall in early summer 2020. I particularly focus on the influence of the previous IOD event, Indian Ocean Dipole event. And every season, we experience a lot of extremes, the sub-season time scale and season time scale. And the extreme events are not necessarily to be explained by the some dominant variability, including the ocean variability. But if we can identify the strong influence or strong driver for the sub-seasonal time scale variability, then we expect some predictability for that event. So in this talk, I focus on the ocean variability in the Indian Ocean. And we may be all of you familiar with the answer. The answer is the most dominant source of the predictability for the sub-seasonal time scale. But for the Asian monsoon, the Indian Ocean is also important. First, I'd like to acknowledge my collaborators, the Ichiro Ishikawa at Shiyaki Kobayashi and Hirokazu Endo and Tomaki Ose at the MRI. And this work was published in GRL last year. OK, maybe you know that in early summer 2020, we experienced mercury-enhanced Mayu bio-rainfall. And that event caused a lot of flooding in China as well as in Japan. And here, I'm showing the rainfall anomaly over the Asian region. Here, I'm showing the rainfall and moisture flux anomalies. And on the right, we see the Mayu bio-frontal zone rainfall, the rainfall amounts in the red box. And in 2020, it's the highest rainfall season. And the rainfall amount is about the 1.5 exceeds 1.5 times the crime torch. There are several reasons for this event. And here, I'm showing the figure from the JMA, Japan Meteorological Agency. This figure explains the possible factors for this extreme event. The first one is the southwestward extension of western north Pacific subtropical high. That brings the amount, a lot of moisture, to the Mayu bio-frontal zone. And there was an upper level top over the Hiroshi. And we observed the southwest shift of jet. And if we looked at the ocean condition, we observed the warm Indian Ocean. Actually, we know that the warm Indian Ocean causes the enhanced Mayu bio-frontal. Here, I'm showing the figure from the Kosaka at all, 2013. This figure explains if we have the warm Indian Ocean, then we often observe the anomalous anti-cycling cyclations over the tropical western north Pacific. And we also experienced the enhanced rainfall. Here, they show the Yantt-Riefer flow. And the Yantt-Riefer flows increased in the warm Indian Ocean condition. This variability is called the Indian Western Pacific Capacitor Mode. This condition often occurs after the El Niño. So here, they show the lag correlations of the Indian Ocean temperature, sea-sub-temperature, with respect to the previous Niño 3.4 SST. And we see the positive correlations between the previous winter SST, Niño 3.4, SST, and the tropical Indian Ocean temperature, sea-sub-temperature. So first, I checked the influence of the Indian Ocean conditions. So we conducted the sensitivity experiments. The Y experiment is the X control, the control experiment. This is the original season of predictions started from the end of April 2020. This figure shows the SST anomalies during the June-July period in the X control experiment. And the second experiment, IO-CLM, is the experiment with the prescribed impostery climatological SST condition is imposed in the Indian Ocean area. So here, I'm showing the SST anomalies in June-July period. And we see the anomalies over the Indian Ocean because we nuts the SST to the model plan torch. And here, we see the observed SST conditions. And we see a good resemblance between the control experiment and observations. So here, we see the SST precipitation anomalies in X control and X IO-CLM. So basically, if we remove the SST anomalies in the Indian Ocean regions, we don't see the suppressed convection over the Tokyo-Western-North And also, we don't see the enhanced precipitation over the May-U-Bio rainfall. And here, this figure is a zoom up of the precipitation anomalies, a difference of the precipitation between the X control and X IO-CLM experiments. And basically, the research shows that the warm Indian Ocean condition is one of the causes for the extreme May-U-Bio rainfall in 2020. And here, I'm showing the atmospheric circulation map. And again, here, we see the sea level pressure anomalies in observation and control experiment and sensitivity experiment. And the X control reproduces the anomalous anti-cyclic circulation and high pressure anomaly over the tropical-western-North Pacific as observed, but we don't see the anomalous sea level pressure in the sensitivity experiment. But the one question remains, as I said, the warm Indian Ocean in summer often occur after the El Nino events. But in 2019, we didn't observe the clear, typical El Nino event. Instead, actually, in autumn 2019, we observed the positive phase of the Indian Ocean diphmol and the high SST in the Western Central Equatorial Pacific as shown here. And this SST condition induced the history wind anomalies over the equatorial Indian Ocean. This wind, surface wind anomalies, enhanced, induced the record-strong, down-wading loss V-rape, as shown here. Here, I'm showing the subsurface temperature anomalies during the November to January. And we see very strong oceanic down-wading loss V-rape in the South Indian Ocean. And here, I'm showing the Hofmehrer diagram. The x-axis is the lat launch shield, and the y-axis is the time. Times goes to the downward. And we see westward propagating loss V-rape. Here. And this event, this event is actually very strong compared with the previous events. Here, I'm showing the Hofmehrer diagram of the subsurface temperature anomalies along the six-degree toss in the Indian Ocean. And here, we see the 2020 event. And the figure, bottom figure shows the maximum and minimum subsurface temperature anomalies in the Indian Ocean at the six-degrees south. And actually, the maximum temperature is on par with the previous strong event in 1997, 1998, a linear event. Then, as I said, the ocean loss V-rape propagated the westward, and the loss V-rape warmed the SST in the South Western Tokyo Indian Ocean. As I showed here, this is the SST anomalies during the February to April 2020. And then, the warmed SST in the spring weakened the monsoon flow in early summer 2020. And the weakened monsoon flow warmed the SST in the North Indian Ocean and the South China Sea. Here, we see the SST anomalies during May to July 2020, and we see high SST anomaly in the North Indian Ocean and the South China Sea. And in early summer 2020, this SST condition enhanced the IPOC mode in the Western Pacific Ocean Kebashite mode, and that caused the South West for the extension of subtropical high over the tropical Western North Pacific and the intense fire media value rainfall. So we can trace back the extreme media value event in 2020 to the previously Indian Ocean Typhoon event. But this is not the end of story. Actually, we know that the other influence of the isotropic circulations on the media value rainfall, here I'm showing the intermember correlation between media value rainfall and the atmospheric freeze, and set 500 and 200 hectopascal zone of wind and seedable pressure. As we expect, if we have the stronger Western North Pacific subtropical high or the southwest extension, then we have more rainfall over the media value rainfall from the zone. If we have stronger 12 over the EOC or over Japan, then we intend to have more rainfall. So there is an influence of the isotropic atmospheric freeze. So these results support the explanation shown earlier in this presentation. To summarize, the enhanced media value rainfall in early summer 2020 was associated with the Indian Ocean iPod mode with the future of North Indian Ocean and South China Sea warming. The Indian Ocean warming and the iPod mode was excited in an unusual way with the strong Indian Ocean iPod in 2019 instead of preceding the new. This case may exemplify another forcing process that can trigger the iPod mode, causing the enormous summer climate in Japan in Asia. So for the sub-season predictions or the extreme events in sub-season time scale, it's very important to understand the underlying processes to cause the extreme events. And if we can identify some predictive sources or the precursor or external forcing, then we may predict the extreme event in the sub-season and season time scale. Thank you for your kind attention. Great. Thanks very much, Yuhei, a very interesting case and also connection to historical cases. Any questions for Yuhei? I had one, Yuhei, in terms of the links to also the polar, like, arctic circulation. There's been some work showing the, I think it's more recent from a group in FSU, T.N. Krishnamurti and others that there is a link between the monsoon strength or monsoon oscillations and the arctic sea ice as well. Does interactions of the tropics and the polar circulation also influence this meubio rainfall? And, yeah, how does the polar region's variability influence this? Yes, I have a very good study. And it's possible to influence the polaration influence on the meubio rainfall. Because if we have some forcing in the polar region, and if we can trace the Rossby wave train that can cause the meubio rainfall, maybe we can attribute the meubio rainfall to the polar condition. OK, on the long-term chart that you showed, I didn't see specific links to the extreme low arctic sea ice in fall, like 2012, I think was one of the case. And then 2019, where the arctic sea ice was the lowest in the observed record. It was not obvious to me that those years had extreme anomalies in the meubio. Yes, if we look at the observation, we may see some wave train. So we can attribute the polar influence. But in the extra tropics, we have chaotic variability. And even we specify the sea ice conditions, it's very difficult to reproduce the observed conditions with only the polar region. So there is a difficulty for the attribution studies, in particular, the signal to noise ratio is low. So even in the tropical influence, the actually the signal to noise ratio is low. So the predicted anomaly is very weak compared with the observation. So I don't know how we can overcome this problem. But yeah, this is the limit for the attribution study, I think. Right. OK, yeah. Thanks again, Yuhain. Thank you for your really nice talk at 1 a.m. local time for you. Thank you for staying up late for us. Thank you.