watcher is receiving those events and they're correct because otherwise this assert would not work correctly Right So all of the things that have to do with the watches seem to be right Uh, the question is why does the reactor crash? Or crucially, why does the reactor exit? I guess we should um just for just so that we can convince ourselves that this actually does what we want we could do Uh While we're printing stuff anyway God watcher event Let's run this with no captures Don't think it should matter but Okay, so we connect Then we call exists And then we call create Then we get the watcher event saying node created synconnected foo. So that's right We call exist again Uh, we do the delete then we got a node deleted synconnected So all this seems to be right. It's just that when the At the very end the poll read Fails with reactor. No because it's not it's still not idle Let's see. So let's uh, try to instrument this a little Uh, dropping zk For idle Just to sort of see where Yeah, so notice here that um The uh, where is it waiting for idle happens up here Right and then it takes a little while before it's shut down specifically So this is where it fails So it does fail After we call shut down on idle But why is that a failure? Uh, try to set watch to false in the last exists Here No In theory all that should mean is that we'll get another response And I don't think shut down on idle should close any ongoing connections. That seems really odd Okay, so here's what I'm going to do for now Uh packet tizer Exiting if uh E dot kind is i o Wait, isn't it i o? Do we not have i o? Oh error kind uh Other rust i Let's see. Give me i o error here somewhere So i o error So kind gives me the error kind which just should be other The question is how do I print? How do I? Easily get at the Kind other error I want to get at the error Okay, so If it's this Then let's do e dot into inner That's really unhelpful. I don't really want to format this I guess I could but it's a little silly This is a It's just error send and sync Hmm So, I mean the the really silly way to do this is if format This Is equal to or I guess even worse It's equal to Right, so this is one way of doing it where If not that So that way it's basically a way to make it be quiet in theory Except it apparently didn't work that's so weird Well, that's what I get for trying to be clever So so it is interesting because the It's deeper for my guarantee to say say not at all. You should never do what I just tried to do ever. It's a really bad idea Uh, this was more I'm trying to find a way to just quiet that particular error about the reactor going away Although the reactor just really should never go away Until we Can oh What happened I don't think so Because like Why would the reactor oh it does enter that side though? So, uh, to be clear what we're doing here is we're matching over our poll reading on our poll right We're trying to figure out whether we should exit and in this case. This is saying that poll right has exited So we've been told to exit and the right socket has been closed down At that point there really is nothing else for us to do but exit Uh And what this is saying for whatever reason even though we return not ready Even though we're turning not ready the Tokyo decides to shut down the runtime which seems like it's wrong It is totally true though that in this case uh There's no reason for us to continue Because the right socket has been closed Actually, no, that's not true. It just means that we've written out everything that's in our buffers Nothing more to write And no more requests So the question here is what do we do in this case? Uh We certainly can't send that's this is not true Uh The observation is that there it could be that there's more More data that the sukeeper servers tried to send to us I guess the question here is um, are we shutting down the outgoing connection? Because this Yeah, so so um This does not actually mean that the connection has been shut down All it means is that we've flushed our outgoing buffer And we know that there are more no more incoming requests because the incoming channel has been close to us Um And so here we could totally be nice and send to the server like look i want to close my session But we don't technically need to it's fine for us to just shut down the Channel together, but in theory we should finish reading from the client too So I guess we Implement closed session Shouldn't be too hard. Um, where's my proto here? Uh close session Really not here, but here Close session, please Yeah, so this is a very similar thing to um What we had with uh, Delete where there's no response. So I think all we want to do here It's sort of like a ping almost actually Except we might want to do it in a slightly more rigorous way Yeah, let's just do it with our requests. There's going to be an empty request here And I guess this is just going to be close Session and trying to write out a closed session request Uh close session All that really does is it writes out the off code close. I think And nothing else Empty body. I don't know what the zero here is Uh FN request zero is the xid And a watch which is going to be none. So that's all fine. So if we send a closed session, all we really do is here close session It's just one of these So here I guess we will do Oh, I see that's why we don't actually need this Uh, the reason is that because the incoming channel has been closed We're not going to send a request in this way. Anyway, we're really just going to send this off code And so this will be very similar to sending our ping In that we do this Send closed session So we're going to do this I'm gonna send a closed session So Like So Basically what we're trying to do we're gonna do all of this and then we're going to shut down the outgoing channel Uh So that The read so that the server will eventually shut down the client to us so that we'll get ready from read I guess the alternative here is that we make, uh Once we get the response we're fine. Okay, so this will be xid zero the length will be the same No xid should be zero here Uh Minus one is for watch events minus two is for pings The reason I picked zero is just because that's what was used here And the request So if you see this is in rust zookeeper at least they use xid zero for closing the session. It might not matter Um, I think we already started one if I remember correctly Uh We do not Okay, fine. So I mean, I guess we could set this to one Well, we used ah, we use zero for the connect. So we might as well use zero for the close as well Right once you get a response. I think you're allowed to reuse the Uh The id I don't think they have to be unique Uh Right, so all we're really doing is doing that and then we'll send not ready In fact, then we will Is where I want tail recursion, but I think we have to call self dot poll And the reason we have to do this is um Because we need to go through this process of of rights again This is what we wanted to avoid with pings right is that um At this point We Who actually no this is not where we should do this We should do this the moment we get exiting This stuff Self exiting here So this is this is the thing that just uh Pulse the in queue until we get no more things and once we know that Then we know that we might as well send a closed session Right And this happens before poll right. So that means we don't have to do this like self poll trick And then pull right we'll make sure that we get notified again Uh right and now in um poll right We know that here If self exiting Exiting And we have written out our entire buffer and we've flushed it then we know that there's nothing more to send And so therefore we can do self stream shutdown Which is Uh We might actually not have a way to shut down I think we do I think for async, right There's a great dot shutdown like so I guess this is really a try ready probably And so this means that eventually we're gonna I guess this might end up shutting down multiple times Which is not ideal So we're gonna have something like we're gonna need something like a Uh shutdown So so the issue here is um Imagine the shutdown fails or like shutdown is not ready. It's not able to shut down the stream Uh Then We would end up with the packetizer being pulled again and when it gets pulled again It calls pull right again and then pull right Actually, I guess because the shutdown didn't succeed It's fine for us to try to call flush again. It just means that we do a bunch extra system calls. They're sort of unnecessary So let's leave this as it is. That should be fine Uh And now in theory at least up here Uh If xid equals zero and not self first Then this is shut down And I guess I don't actually know what the server response to a shutdown close Oh, it's just an empty response, right? So There is nothing really to do Uh, so the idea is that at least in theory Uh In theory Servers should now shut down receive end Right, that's the observation that we send a closed session And then we shut down our send of the channel are the right end of the channel And now in theory the server should send the final response and then close the Close the connection as well Uh, do you really need to self pull in the case where right equals ready plus exiting? Oh, sorry. No, no, no, this was supposed to My bad This is async not ready And the reason this is not ready is because we're not actually ready until We hear the response from the server. The reason this matters is there may still be outstanding watcher events Uh, that the server Wants to notify us about and we should be nice and wait for that Expected async. All right async Expected async found item. All right, uh ready is a thing that was a thing So this is going to be Shutting down this is shutting down writer. I'm just adding some debug statements for us here so that Be able to see what's going on where's the here God responds to Close session and then I guess also down here God responds to heartbeat Uh, can you combine the last two match arms? I could I sort of want to keep this one explicit just because uh I wanted I wanted to add a comment to it and did not um We have written out actually no, you're right Still failing Um, that's because the reactor is disappearing Why on earth is the reactor disappearing? so one sort of silly thing we could try here is thread sleep Sex Just to see whether it eventually tighties up. Ooh Connection closed with four bytes left in buffer Really and pull read what God responds to close session shutting down writer Pull read Read more bytes So somehow Everything works correctly except that we get We get more bytes so it's Telling me that I get four bytes after I get the response to the closed session I want to know what those bytes are Because I think that sounds really weird Uh self dot inbox In fact, I want that out in hex really really this way Fine I get four zeros at the end The length of closed session is That long right so an empty response If I haven't if I've been paying attention then empty response Just reads nothing which in theory you should mean that it reads Uh It should mean that it reads out the xid the zxid the error code the length of the beginning Which is 16 bytes Then it gets that's when it prints the response And then it's going back around and saying that it has another four zeros No, the acknowledgement of the session closed. We already got so see how If you look here, we get this got response to closed session And that is printed Here when we get a response that has xid zero And so that's already happening um I guess actually one thing that's worth pointing out here is Uh That this is currently going to drop error if there is an error I think this is gonna I don't really want it to search I think I want Uh if let If some error Then bail uh Failed to close session response to ping just to see if that's what's going on Uh That is true It doesn't look like we get an error Right it says got response to the closed session So that is the length of 16 Wait length is 16 Is 16 Why would the length be 16? There are only three fields of Oh, the zxid is 64. Okay, so that is 16 bytes Got the response read more bytes have zero Pull right Packeters are pulled pull read read more bytes have zero Wait a second. So we go through this we need four bytes The pull read we get zero It's saying inland is not equal to zero So what is the length here? That means that we got a a length field and that length field is zero So I guess maybe just like null terminates it in its entire session. I guess I could be That's so strange server Normally sends Four At the end So yes, that worked and then the packtizer exits without an error code. I guess this might as well Print out the uh Print out the bytes that are left Let me just double check that those are actually left in the buffer Yeah, okay So let's leave that at four And now this should work just fine the question is does it still shut down the reactor if I don't sleep here? What happens? Apparently it's so weird. I have no idea why they would do that uh Yeah, it still shuts down the reactor after that exists Which is quite odd really Because this happens Got through all futures All right, just uh Yeah, I should check that it's also zeros. You're right. Uh writer got through all future because it does do that So weird, uh, you're right if uh If it is equal to zero and fine left left is this if left dot len is four and left is equal to I think I have to do this Should be the same Yep But for whatever reason The reactor is gone But as far as I understand shut down on idle It should not shut down the reactor Until all the spawned futures have exited and they clearly have not I think this is a bug in tokyo I guess let's check exactly what the docs say For shut down on idle runtime runtime Uh protocol spec for zookeeper. No, it's awful. Uh, does it may have to do with the w dot weight at the end? Um, it never gets there Well, I mean it it does but I don't think that should matter I can try it. I don't think it matters. I think it crashes before that What? Oops, that was not the command I wanted to do but luckily that didn't delete anything What? Why is that even relevant? Also notice that it's not closing correctly here now Yeah But the futures haven't all returned async ready because the Because the packetizer is not returned yet And like the very first thing we do in the packetizer is we spawn the packetizer Right, so the packetizer is still running And should continue to be pulled Because it hasn't exited yet Right, it only returns async ready if it prints packetizer done and it never prints packetizer done Oh, no, no, okay. So the reason why this works is just because It's just because the main thread exits too quickly Yeah, so you see here the if I wait at the end then I I get to see it crash If I do this the program exits too quickly for us to observe the crash It does not actually fix the issue. Uh, so the observation is that the uh, yeah, great This should not be necessary Uh, so the question is what does shutdown and idle actually do?  So the question now is whether we add Like get data, which currently there's no way of reading data or whether we try to add Reconnection so reconnection will get into things like zx id's because you need to Negotiate with a new server that you're connecting to Sort of what version to start reading from and get data is more sort of for completeness But it will let this return data through these channels Or we could try to start tidying up the api I think get data is probably a useful thing to add Uh Yeah, so I guess it's really up to you whether you want to see get data or reconnection Reconnection is probably going to be harder, but get data is more useful Um That's right for x and inputted code return success. Nice. Uh, isn't it what is it step three profit? Um Yeah, I think we'll probably just have time for one more because the stream is already going pretty long Um Yeah, so what do you think should we add should we add data retrieval or should we add automatic reconnection? I think those are the most feasible and useful options Uh, one other thing I'd like to point out while you have you are here is that this create as you as you observe There or there are a lot of just like functionality missing. That's not as you as you can see from when we implemented Uh Delete for example, right and create for that matter. It's actually pretty easy to add methods like that to add Additional api operations. So if you want to try your hand at this Because I'll push all the code, right Um, you should try to submit a pull request where you add if you look at the the zookeeper api for the original crate, right? Those things like set data if you look at the Java api there's a bunch more operations, right? So there's uh Get acl get children get data Get various session state multi operations setting the acl and existing nodes set data um Ooh transaction I feel like this multi thing is something we'll want to take a look at Um, but so if you want to try adding some of these then you absolutely should the other thing we really want to add is um Add all of the documentation from the java code into our crate so that the crate is actually fully self documented um What are you actually working on like outside of this or what is this? auto reconnect get data Apparently apparently the two of you, uh, highly disagree Um, so zookeeper is a you can think of it sort of as a key value store Except its goal is not so much to store data as to store metadata Uh, and it it aims to keep that data very highly consistent. Um, so if you have many Many clients that are all doing many operations at the same time It guarantees an order of operations between them and lets them detect if there are conflicts And also you can have many servers that are running as And sort of presenting a single key value store even though they're Distributed across many servers and it does the right thing even when servers fail or clients fail and those kind of things um Get children into account of future. Yeah, that's nice Uh, yeah, I mean, I suggest that you watch Part one before you watch this Uh, because this is starting pretty deep into it. You could also watch some of the other streams I've done Um, and then you read is for metadata sort of although you should think of it as high consensus um Or sorry high consistency for the data Let's see Are you a russian hacker? Uh, no But then again, wouldn't I say that if I were a russian hacker? Um, but yeah, I would watch some of the previous streams, uh, and some of those might actually give you ideas for Uh small contributions you can try to make just to get yourself familiar with it With a codebase you already know something about Uh, let's see All right, I think that Oh, that's such a good question so Get data will be fairly straightforward. So let's start auto connect Because that way one of you can hopefully maybe add get data. Um, so let's do auto connect which is going to be A little bit interesting, but we'll see how that works out. So first, uh, So, okay So one thing that's really neat is because we have this, um This sort of single packetizer business Um, we can have all reconnects handled through that So to give you some idea, there's um, where is this over here somewhere? Pa pa pa pa Okay, so this text is sort of important. Let's uh make this bigger because I hate websites that do this Uh, don't mess with my line height Uh, don't mess with my font. Don't mess with my font size Don't do any of those things Thanks Much better See how much better that is right, so the This is sort of the crucial point port that we're doing now. Um Personally learning was because Oh, yeah, I've gotten a lot of questions about actix web. Um, we might try to do something on that That host name. Oh, yeah, I'm pretty happy with my host name Um Yeah, so the idea is that when you initially create a zuki procession you get a A 64 bit number that's sort of your session id that is given. So if you look at our response The connect response It has a session id and it also has this password and the idea is that um If you ever have to re-establish a connection with a new server You send along both the session id and the the password to prove that you are that same client And so we're going to need at the very least this and this there's also this read only flag that We're going to ignore for now But the basic idea is if you're talking to a zookeeper server that has lost its connection to other zookeeper servers It's not able to do writes because it has a guarantee consistency Uh, and this it places the client in a read only mode where If you try to do a write it's not going to succeed because I can't communicate in the further And so it will just return errors and then it places you in a read only session so that you're aware of this fact And here you can use things like watchers to realize that you are now in a read only mode We're we're just going to leave that aside for now and look at this session re-establishments We'll need session id and password from the connection response um And so the idea is that when you get disconnected Then you try to connect to one of the other zookeeper servers. So remember how Uh in our source lib Currently we just take a single socket outter But in reality what you should do with zookeeper is there'll be a deployment of many servers and If you lose a connection to a server you try to reconnect a some random other server and you keep retrying until you reach a server That's that you can't actually connect to And then once you connect to some server, then you either transition to the connected state So you reconnect send its your session id and your password Um And then it responds back. Okay. Your session is good again or the server will respond saying your session has expired um And at that point Right, so the idea is that the Uh The idea is that if our session expires We're just going to try to connect to a new server and establish a new session and we'll do that I think this is saying that this should happen internally in the library and maybe that is true They say they have some like heuristics to make it more efficient. We probably won't implement those We'll just say that if we uh, if we are told that our session has expired We'll just return an error to the client saying We tried and failed um, I think is basically the idea that if if it's if at that point like You we try to re-establish the session and we couldn't re-establish it quickly enough So as session expired then good luck. You got to connect again Uh We might end up sort of making it fit more with this text later. But for now, let's leave that Uh, the other note that's important here is that we don't deal with session expression It's not like we start a timer and after a certain amount of time it is considered expired it's we try to connect and then, uh, the Clusters or the zookeeper servers, they are going to decide that we have expired And then do all deletion of all ephemeral nodes and such Uh, and then once it re-establishes contact, that's when it's told that it's uh session has expired So let's see so connected Partition then we're disconnected After a while we connect and we're told that we have expired Yep These are pings we've already dealt with that Uh, perfect. All right Ooh Session moved. Yeah, I mean we don't really care about that for now Uh, so the first thing let's just for now stick with the case of there's only one zookeeper server So we're just gonna try to do this in terms of reconnecting to the same server And then we can expand it to having a list where we just like try them round robin or something later Um, just avoid Shaking up the api too much Um, so what this really means is We're gonna have to handle Um connection errors So the way we're gonna do this is Huh, that's a very good question We sort of want to be able to continue to use the question mark operator in here So I think what we're gonna do is Um We're gonna use the new do catch notation Actually, we could have inner pole instead. Maybe that's nicer. It's a little unclear actually No, let's put it here. That's fine. So we're gonna have, uh, no not that but a Uh Pole inner Which is basically just going to be our current pole An async Item is nothing and a failure error And that is going to be this entire function like so And then what we're going to have pole do is it's going to call self pole inner And it's going to match on that If we get any kind of okay value Then we just return okay v If we get an error Then we need to be a little bit more careful Because we basically want to see if the error we get is a disconnect Then, uh We sort of want to purge state and reconnect Might not even want to purge the state. It's a little unclear because we might want to In a sense what we want to do is Any outstanding, uh, operations We need to decide what we want to do with them with we could cancel them That might be the right thing to do Yeah, I think that's the thing to do Like if we if someone does a Someone does a create and it's asynchronous, right? So we send the create and then we're notified that the server went away Uh Before we even write that request to the wire.  We reconnect to another server. Do we then send it to the create that was sent? Sort of to the previous server just never got sent or do we just prune that give an error to the user and the user has to retry I'm not entirely sure Um, this is one of the things that gets a little bit tricky in the asynchronous case and I think they say this here too that um Of course received on a connection for a session which has been re-established And I don't think this is too much of an issue Well, let's just write it up in the most straightforward way first and send Errors to the client and then over time we can make the retry happen internally potentially Um, so the question is how do we learn that the error was a disconnect? Chris goes through and gets disconnected and the server responds to the request after reconnection Um So that is one of the things that zookeeper deals with You might not hear the response Actually, that's a good question. I think what happens is you You would get the response From the server you connect to so this is one of the reasons why we have this uh zxid value So if you look at the quest in connect This is last zxid seen so every time we get a response from the server it includes a zxid right When you reconnect you tell the server what the last zxid you got from the previous server was And then uh, it will send you anything that you have missed like anything that the anything that's been recorded of something that's happened that come that You haven't you clearly have not seen and so that's how you would hear the response nonetheless The response would be resent to you If you're an original request in fact actually went through Uh, what keyboard are you using? This is a filco majesto touch ninja It's pretty nice. It has the keycaps on the bottom School Right so so that is why that is why you're guaranteed to still see those responses later on The question is how are we going to even detect that there's been a disconnect Here's one of the problems of using failure right like We don't know where that error is So let's see A disconnect would only be if it's pole read or pole write And pole read and pole write All use so there's a bail here Uh, but that is If the server actually Even in this case we sort of want to reconnect right It could be that we just want to deal with any error like this Um, but that would mean that even parse error we would start over Which doesn't seem quite right Hmm I think we'll we'll start with the simplified version Which is if there is an error we're going to reconnect because it lets us start from a clean session regardless Um Yeah, so this is one of the things that um zookeeper is cool about is the zx id basically allows you to give clients a very Strongly consistent view of the world Um And there's some rule for like When zx id's expire. So this is the idea of session expiration, right like Uh, you know that if a client session has expired Then you can remove any state that you would have sent to that client because when they connect You're just going to tell them look your state has gone and try again like establish a new session Uh, so that's the way to deal with it So what it's you can think of it as it has to keep all zx id replies within the client session expiration time Uh, I'll assume all errors are Right, so the question then is what are we going to do like what is the What is the protocol for Doing a reconnection this makes me think that we should put Handshake inside of zk, but let's um Put it here for now So certainly we'll have to remember the address. So that's going to be one thing We're going to have to like let adder is equal to something we don't quite know what yet Uh, then we're going to connect to that Well, actually I think we're going to end up making packetizer have a State that's going to be an enum of either the thing It's going to call poll inner on which is the way it progresses or if it's disconnected It's just going to poll on itself connecting So we're what we're going to do here is basically construct a future that when it resolves will become the new inner state Think of this as Uh, where is it here a state is going to be Packetizer state and we're going to have an enum Packetizer state s Which is going to contain the stream all of this stuff in fact and this stuff I think it's mostly those current state And then we're going to basically implement uh s Packetizer state s and we're going to implement something that's sort of like poll So that's basically going to be this uh most of these methods actually are going to be on packetizer state instead So let's go ahead and do that so here This is going to be connected It's going to be this a stroke packetize active packetizer It's going to be one of these And then the alternative is that we're reconnecting Those are the two states we can be in Uh, impulse same as import. No. Impel is not the same as import. Impel is a way to say that these are methods on this type so For example here If I impel enq or enq that means that if you have an enq or you can do like Dot enq to call the method enq on x So this is going to have some kind of future for reconnecting That I don't quite know what's going to be yet Uh, and then we're going to implement future for packetizer state See roughly where this is going So the idea is that you're always going to be pulling whatever is inside and whatever is inside is going to depend a little bit by I think this needs to go further down 39 So packetizer here So if you have an active packetizer, then the way to drive that forward is just to keep doing all the things If you have a Reconnecting packetizer the future you're driving forward is to reconnect And so this is why a packetizer state implementing pull for that is going to be Basically matching on self And if you are actually let's make this a box future I haven't quite decided what this will be, but I think this will be an active packetizer s and the error Will be a Failure error So if you are connected, then you have a Active packetizer and you will just call ap dot pull and if you are reconnecting Then you will do this You will do try ready ap dot pull And then here we're going to do something like Set self to be connected Right, so the idea is that if you're in the connected state Then when you're when you call pull on the packetizer state You just drive forward like you read more from the stream you write more responses you send replies those kind of things Whereas if you're in the reconnecting state, you're just going to be pulling your reconnect And then when it finishes you're going to change your own state to be connected um And this of course means that this is not going to be on packetizer state, but on active packetizer It's going to have all these methods Pull in q is the one thing it will not have because that has to happen Further up that has to happen all the way up at the packetizer level Which is actually a little awkward and then what this thing is going to do Pull inner right, so All right, we'll read so we're now implementing future for packetizer state and the packetizer has one right So if you pull a packetizer now, what's going to happen? It's going to do self dot state dot whole If I it will first do self dot Uh Pull in q Right, so we'll first do that to try to get any additional connections Which I think currently is being called up here somewhere That's going to go away. So remember they're recall that what's now left in this right? So this is what we do when we pull When we pull an active packetizer what we do is we Just pull the reader pull the writer And pull for just completion of things So we're we're sort of building a hierarchy. Let me try to start this from Higher up so it's easier to follow What's going to happen is when you pull the packetizer It will first try to see if there are more requests to get and then we'll pull its interstate The interstate is sort of a second level future That is either that we are connected and we need to drive the requests that we have outstanding forward Or we're in the reconnecting state in which case we need to continue trying to reconnect And so this is sort of the top level so implementing future for packetizer What that will do is if we're not exiting it will try to get more More requests This is going to have to do something slightly funky, which I don't know how to do yet So this is going to be yet to do And then so first we're looking for more data and then And then we will pull the The current state and if that resolves then we just give whatever result it gave and if that errors out Then that is the case where the current connection has failed as we need to retry So let's see Let's deal with the retry case a little bit later. I want to clean up some of the more stuff Further up So the packetizer is going to have to have some way To enqueue stuff That's a good question. I think this has to be There's no reason for us to pull the enqueue thread If we're currently not connected. So I think what this is going to be is If let Packetizer state connected Is self.state Only then will we pull the enqueue And in particular because then we can add things to the outbox And pull enqueue will take an ap Pull enqueue will take an active packetizer s And that way We have a way to modify the outbox, which is what we want to do We still want to use unique xids Those are not those are not per connection. They are global state The reply is we're going to cancel all the replies if we fail to disconnect If we fail to If the connection goes away All of this just writes to the outbox. So that's all fine So if we pull the packetizer, we're not exiting and we're in the connected state then we're going to Try to see if there are more requests from us Otherwise we're just going to pull the underlying state and return whatever it returns If it returns an error, then we try to reconnect This is going to do things like self. I guess We can't actually do this yet, but we will So it's going to take all the replies channels And clear them and send errors on all of them When we pull the underlying future Then if what we hit where is packetizer state down here Then when we pull that that's either going to pull the current state, which is just what used to be there So it's going to do reads and writes Or we're in the process of trying to reconnect in which case if we succeed at reconnecting Then we do a mems This is going to Probably going to come back to bite me. I think No, that would be fine Like so Then we are now connected So then we swap itself with a Accutizer state Connected with that ap and then we call self.pol. I think we need to call self.pol because otherwise When we create There's nothing that will notify the ap that it needs to be The connection that we just created needs to start doing reads and writes. Otherwise it will never be pulled again So the question is This I guess we still need to deal with we haven't actually written what the reconnect stuff will be But connected is basically the old the old packetizer, right? So that's active packetizer And all that really does I guess active packetizer is going to have to implement future If we can find it actually it's a little bit worse than that And I'll you'll see why in a second The mem swap is pretty pretty great Actually, it's not a mem swap. It's a mem replace I'm going to give you credit for catching that So let's see these things Actually have to have access to exiting And they have to have access to The where is it down here somewhere to the default watcher And neither of those are inside an active packetizer Those are all in packetizer. So we need to have a way to communicate down that into the active packetizer Yeah, so I think we're going to call this thing And that is going to be given a exiting Which is a bool Because I'm pretty sure this just reads exiting and does not write it Be very surprised if it did And it will take a Default watcher or I guess watcher which is going to be a mute Mute one of these is we'll call it default watcher um So now Where is it? self.default watcher. This is just going to use that self.exiting this is just going to be using exiting Um like so and then similarly packetizer state is not actually going to implement future Instead it's just going to be an impulse Yes Because it needs to pass along this exiting and default watcher And saying here Yeah memswap and memreplace are pretty uh pretty magical All right, this is probably going to yell at me for also some reasons probably mostly because of this reconnect business So I'm going to just comment that out and see how much it yells at me a bunch 334 It's yelling at me because This needs one of these So I'm guessing it's also yelling at me here Because this needs one of these Where else is it yelling at me? All right, we need standard mem for our memreplace trick It's telling me that default watcher is not in scope. Okay, so that's been fixed Whole right is using exiting The lat needs to be communicated there It's going to be whole read Is going to be given the default watcher Whole right is going to be given exiting That's so many errors 339 This is going to be packetizer state this 40 that's just the same error more times 72 stream right so this is going to be state uh So here's an opportunity for us to fix that but i'm just going to Not deal with this for now And say that this is going to be packetizer state Connected an active packetizer With stream timer outbox these things Reply Think those are all the things Oh, I did not like me doing that. There we go That's much fewer errors better Uh 70 who apparently it doesn't like that at all Uh, yeah, all my configs are on github if you go to github.com slash john who slash configs Right there Why is it not sent? Oh, it's because of our um box future which we haven't send first Ah, it also needs to know whether it's first Uh first can also be I think that can be here So where is our first somewhere up here? packetizer First is also part of an active packetizer for the time being Basically, whether we we've heard the first response from this particular machine that we're connected to Wait, it's escaping your thing weirdly so you're saying this is turbo fish then this Yeah, I mean this is the That is the turbo fish. This is the I guess Fat turbo fish Seems seems legit Uh Async read is not implemented for s All right, I'm missing a where s Hidden somewhere far down here this guy 346 reconnecting 410 Pull enqueue expected active packetizer Uh Pull enqueue is given a mutable reference to that. It's not a given one 436 My guesses of the escapes are because of the Uh because of the bridge that's being set up, maybe Uh because restream is doing this like synchronization of chat across all the different rooms Uh takes two parameters. That is true. It takes self dot exiting and mute self default watcher Now what mismatch types? Oh Yes, indeed. This is supposed to return. What uh just make this return a Air e because I haven't All right, so in theory, I don't think I've broken anything although I probably have Uh, I move this into pull enqueue I guess while Let that is this then do that and I guess this Should basically never happen Oh, then sure async not really So this is uh, what should we return if We try to pull enqueue, but we're not connected. I think the answer is just it's just not ready. So that's fine Oh 428 AP not found well, that's a little unclear I think it's gonna have to be this because we we basically The issue with like dealing with three connections is there's so many cases where you could not be connected anymore like here um I think we're guaranteed that this match must work because pull enqueue can't change our state and it will always return Uh, not okay not ready if uh Pull enqueue Will never return error If not connected Uh, because it will only read from the incoming channel if it is connected And it will only return error if it reads from the channel and gets an error and so therefore this can never happen Uh, yep still working on the zookeeper library. It's true async stuff So we're building an asynchronous Uh zookeeper library in rust for 10. This will not longer do this and this doesn't take this anymore 379 What do we think? Hey, it works great. So we've now just like shifted a bunch of code around to the point where we can now, uh We can in theory now have the The packetizer be in a state where it is not connected Which is all we really wanted to begin with right. So now I guess we can do uh here or something like Place ourselves in And then I will do So this is Uh prepare for reconnecting state Uh, so what do we do if we try to pull and we get an error? Well We want to Send errors to all in-flight requests Right, so that's going to be not quite clear but something like For what is replay again? Uh I could do a packetizer It's a hash map of things it's going to be for tx in Reply values I guess drain So for all of those we're going to Do something to indicate to them that their thing got canceled. In fact, maybe what we want to do is just self-state Reply clear Now this won't work because state is an enum, but let's deal with that later. Um That sends cows into the future. That's right Um clearing this will cause all the All the receivers to get a channel disconnect, which means that they will know what in fact happened that their in-flight request got canceled Um, so let's see We're gonna Dis we're gonna start a new connection Once we get that stream, we're gonna do some stuff Uh, we're not going to do this map And We're going to Here we're going to basically make the stream that's all fine Uh And then we're gonna Mem replace our favorite function self.state With in fact This won't even be needed Overwriting or dropping the old state will also cancel in-flight requests So we're going to replace that with a Uh packetizer state Reconnecting Box new Retry So retry is going to be a future right and that future is eventually going to yield our new connection So we're just going to box that up and set that that's now our state of self And we know that we're going to keep pulling it because now whenever we call self dot poll It'll call self dot state dot poll which will pull this future. So we know that it'll continue to be resolved Uh, so now the question is just what goes in here And I think the answer is basically this Now this is going to be okay Async not and then this is going to be self dot poll Uh, I think you're thinking the other way around that es6 looks a lot like rust Uh, you boxed future instead of reply. Yes. I did indeed. Good catch Let's see. So here there are a couple of things we need to make sure so this is where the zx id is going to come in so remember how Uh, we somewhere here we like decode the zx id And this we're actually going to have to keep around somewhere The problem is that's hidden all the way inside the active packet iser So I guess last zx id scene is going to be here and i64 We're going to do self dot last zx id This is going to be i64 max of self Last zx id and this I think it has to be max It's actually unclear. I think the server guarantees it sends them in order So I think it's just this Let's just to make sure say let zx id is this assert that zx id is greater than self last zx id And then that is zx id So down here now When we're reconnecting Then we're going to have to I guess up here we're going to have to do something like if let Uh, actually here package Uh connected Active packet iser there we're going to have to extract the zx id from there Is self dot state Then we're going to do all these things Um And in fact if that is not the case if it's not we're not in the connected state Then we just got to return to e because this means that there's a An error that occurred while we're trying to connect error while connecting Don't recurse For now. All right, so now we have the I guess this is last Last zx id scene So now we know that we can pass that in here. We also need to extract out the password that we got from that server um So the packet iser down here is going to have Password Which I think we're guaranteed is Always a vec. Yeah Password vec ua So we're also going to do here Uh So down here This is also going to have Revenue password And we're going to do self dot password is equal to password Gee, we're just going to do mem replace No, we're going to do mem swap. Yeah self password Right, so what we're doing here is when we get a response to the connect Then we take the password out of that connect response. We're just going to get Send back. Why is it even sent back? What does it do with the response to that? All right, I think we basically ignored that response. Yeah Uh, so that response we're going to extract the password from it and stick it in itself So that uh down here We can also do Is here ideally we would do a member place, but we can just do uh Sure, uh a ref password Come back to bite me Password is password dot split off We're gonna need nll for this which is a little sad Because um that the issue here is that The I'm now borrowing into self state Uh, which means that I won't be allowed to mem replace self state down here NLL would realize that this is okay Uh, but it's not okay Uh Similarities. Oh, I see Don't worry about it. We all uh, we all say things that we afterwards believe were silly Okay, so password gets passed along here. I guess we also need the session id Which is also here Fields for re authentication or reconnection This is going to be session id It's also an i64 And so down here When we get the connect response We also get a session id and we say self Dot session id session id Work consistent reconnect So down here, we're also going to extract out the session id from the old connection that we had We're going to reuse it for the new one In theory, we could set some timeout I think for now we're just going to leave the timeout to zero so that it's auto negotiated with the server Um And then when this eventually comes back, I guess that's the question Uh, we want to make a new packetizer, but we want to make a new active packetizer Should I think we Yeah, we want to make one of these guys Which should arguably now be a new Good wrap this in a new probably uh Not do that for now just because I'm being lazy Right, so this will now be something that resolves into an active packetizer. Uh, I guess These are all fine They all start out empty I get fine. I guess we'll add an active packetizer Or is it active packetizer down here? We'll add a FN new In terms of self Um, these fields will all be set last zxid Scene is going to be zero Session ID is going to be zero Password is going to be vek This is going to be vek new The only thing that needs to be given is a stream of s and now active packetizer so This can be replaced with new stream And same with this guy. No this guy It's gonna be new street So in theory Um, this led you into writing replies that are retry. No you did it again Uh retry That's right Yeah, I'm not gonna I'm gonna try to avoid to write that Give no guarantees Okay, so what we've done now is we've constructed a Future that will eventually resolve into an active packetizer just like what we had When you initially construct a new packetizer This worse our other new this thing, right? So this thing just really constructed a new packetizer and we're reusing the same packetizer None of that changes. We're just swapping the connection that's inside is arguably we could Instead of calling it active packetizer we could call it like Connection because that is really what it is. It's a single connection with a single zookeeper server that is currently active But active packetizer seems fine. So this is gonna do a tcp connect. It is then going to Try to connect to the active pack or try to connect and when it does Create a new active packetizer and then I guess it will also have to Somehow Send in this connection request Don't entirely know how it's going to do that because it doesn't have an enqueuer And that's what makes it a little bit odd And it sort of needs to put it at the front too Ah This I think we need to do something like Enqueue Or it's it's basically we're doing whatever this is doing In reality though, I think all of this stuff We could really stick into Uh Stick into active packetizer. It's really where it belongs So this is going to be a fn and q Take some mute self And a Request I guess item which is a request And returns Probably nothing Returns nothing. So this is going to now be all this stuff is going to be self self Uh, it will however need to know the xid which is an i32 This is going to be self It's going to return a Uh Oh, it does have to return a one-shot Sender thingy So basically the what i'm trying to achieve is that this will just call ap dot and q so Xid item And I guess it does have to give the tx Because then all of this can go away from packetizer And the tx here of course is a Is a one-shot sender Oops Does not get to return one It's going to be given one which is going to be a one-shot sender of Whatever these are This thing and these are all going to be self now. Let's go away This becomes self Right, so now This thing we can just in q directly. We're going to have give it an xid and that's going to be zero because we're just Connecting I guess it can be self dot xid Actually, we could even do this now It's going to be self dot xid And then we're going to increment it by one and we're going to do the same thing here self xid plus equals one Uh And then I guess here The main question now is whether we should Wait to make the active packetizer Uh active Until it has fully done the handshake. I don't think that matters Yeah, I mean we're basically just implementing a fancier hash table So so what we're doing here is we're connecting Setting up the packetizer Giving it the first request which is going to be the The connection request And the question is are we going to Are we going to say that the connection is ready yet? I don't think so I think we uh Oh, we sort of have to because we have to pull it that's a little weird I'm not going to lie Um All right, let me explain why this is a little weird. Um My concern is here. We're constructing a connection to the To the server. I mean the first thing we're enqueuing is the connection request, which is the right thing to do um and then The connection request is asynchronous, right? So we don't necessarily know when the response to it comes back from the server Once we switch over to Uh, well once this future resolves we're going to switch over to being in the connected state Which means that we're going to start pulling requests from the client from the user again Once we start pulling those again and start sending into the server Uh The server I guess the server is guaranteed to receive them in order, but in theory might Respond to them in different orders I don't know whether that matters So i'm worried whether we need to like wait for the connection Request to be responded to before we make it the active packetizer or mark ourselves as connected I don't think we do we do however need to give a uh, A response channel here for enqueue Right, so the enqueue that we have also takes a tx for where it should send the response to the connected uh to the To the request we're making and in this case, I think the We can just make one that we never read from Pretty sure Because if we send the tx here then we're going to drop rx Uh Don't care. I guess we could respond rx Uh e-connection Because we don't really care what's in the response because most of that is handled internally, right? So I think all we really we just like set up the receiver. We could even just drop it if we wanted to Uh, but we set it up at least so that we get to see the response And that way it's I think this is fine. Like the response will just be dropped and ignored Let's see Oh, yeah these uh The streams end up being pretty long just because there's a lot to cover Like I was sort of hoping that we might uh finish tokyo zuki for today, but there's just too much stuff It's just it's weird how There are just so many things that you end up having to deal with in the hours to start to go by uh I think we will probably Like once this compiles, I think we'll probably end it off just because It's a lot of hours and I need to like eat and stuff. Um How do I become interested in async networking? So the interesting thing is I'm not I I think there are a lot of interesting programming challenges in in asynchronous programming But it's not like I really care about async networking specifically The reason I started doing these streams were more because I wanted to show more More advanced or intermediate rust code being written so that you could get experience with writing more advanced code without necessarily having that experience yourself And so I'm sort of trying to trying to help bootstrap people's experience in that regard Uh, and I think async networking is one of those things that a lot of people are interested in and don't really know how work And so it's a good way to get a very deep dive into this And it's totally true that this is a very deep dive like most of the times if you're building things in rust You will not have to deal with these things, but the goal here was to expose like Really meaty programming stuff where you're just Basically the idea is you get to watch me fail and then see how I respond to Having to debug my code because I think that's some of the the stuff That's the most useful if you're trying to learn a new language Or we're just trying to learn programming in general is to see that other people also get stuck and then Sort of observe how they get unstuck Uh It's true as well as uh stanislav says like You sort of need things to be async and you see this in the rust world too like tokyo is just everywhere now and it's because um, and I mentioned this somewhat in my much earlier streams on Building ec2 orchestration you run into this problem where Say that you're setting up like a cluster of a hundred machines You can set up all those hundred machines with a single thread because most of what you're doing is You're sending a request to the server somewhere It's going to do a lot of work and then get back to you and you don't want the thread that's doing that work To sit or just like hang around and do nothing. It might as well do other useful work in the meantime And that's when you get into async Uh, all right, let's see where we were Proto 482 Uh, this is going to be a request I think Yep Uh 15 mech does not have field password. That is true Active packet iser does wait what uh is it? password none of this Semi password business stop with that Oh, yeah, I don't think you generally want to read async rust code if you're tipsy Just a pro tip Uh 419 Where async read and async write? 42 all right. We need an address I guess packet iser is going to have to have Uh zookeeper Heather is going to be a socket adder And I guess we're going to have to include that So And then in theory down here This is going to be 6 Yeah, I think that's right. I think synchronous is not really how the world works at this point Adder is added all clone This Sorry, there's just we're just threading things through to make the compiler happy Um 510 What is it complaining about now expected i32 Oh 492 Future is not implemented. Oh, that's right. This is a map 516 expected mute That is true Did expect me Future is not implemented for that that is fine because we changed it type mismatch Uh Yeah That's awkward Hmm So I don't know if you you saw what happened here But the the problem here is we're establishing a new tcp connection But packet iser is generic over s Whereas tcp connections are not any yes. It is some specific s Oh, it's pretty awkward. I don't know if we have a good way around that actually Um, I don't really want to make this Not generic That's a good question Uh, it's sort of like I want I want to take any s where I can establish a new one But that's not trivial either Because I mean the the one way we could do this of course is uh Also, so I could do s anything that's from tcp stream Um, the the reason it's currently s is actually because it's for a couple of reasons Uh It's because it could be that people want to do this for testing Um, like you want to test it over Like you just want to use buffers instead of having a real zookeeper running Um, or if you want to connect over unix sockets, that should also be fine Um Yeah, it's not ideal I think we're going to need our own trait here It's a little awkward Like it would basically be a trait that has an associated type, which is the address uh And It extends async read and async write and it has a connect method But I think we're going to have to Uh, I do not want to do that But I guess we have to So we're going to have a pump trait, uh zookeeper transport It's going to extend async read and async write It's going to have a type parameter adder Um It's going to have an f and connect Which takes a self adder and returns a error connect error Which implements into failure error Um, and this is going to give an a result self And self connect error and then we're going to implement zookeeper transport for Tokyo net tcp stream Which is also really weird. So the adder is going to be a socket adder in fact, it's going to be A connect is going to be And this is going to be It's actually not going to be a result. It's going to be a I don't even know if i'm allowed to do this I don't think I am Food Which has to be Into future item is self error This self connect error and this has to return. Oh, this is awful Luckily the users will basically never have to see this But it's still pretty sad So a connect feud here Oh, I can't even name the connect future I think the connect future is going to be uh Tokyo What do you get back if you do a net? Okay, so it's one of these Uh, it's going to give you a self connect feud Uh, that's true. I could take a reconnect function instead. Oh, hey Uh That might so this is sort of the the proper way to do it But you're totally right that maybe it's better to just have a reconnect future All right, just a Reconnect function. It's a little awkward though because it uh, we want it to be asynchronous So it would have to be a Function the return something that is a future although we could totally do that Uh The generics turn out pretty awkward I mean, this isn't actually that bad when you use it. It's just Having this trait in the first place is awkward Because like if you if you look at it all this is going to be is a tcp stream Connect right like that's the entire implementation for tcp stream Uh, and then down here we would say that We implement here where s is what did we call it? Uh zookeeper transport and I guess struct packetizer Is going to have the same thing Uh, because this is going to have to be s adder And now down here All we're all we really have to do is uh s Connect self adder Right, so it's actually not that bad. It's just an annoying trait to have Um, and I think there's probably yeah, this has to Where s is zookeeper transport and this has to take an s In theory, this means that we could do the connect here directly Do connect directly here now that we can connect future returns a tokyo IO error 24 Oh Info. Wow. That's when you can tell that it's been a while Uh It's high We're not going to have trait objects of this type anyway 442 This also needs to be zookeeper transport. That is true Oh, that's so many errors Uh 50 What? 58 Uh Nope, we're going to make this not be generic at all It's going to take a tokyo net tcp stream This packet is yourself even out to be generic over s and not a few methods It does because it contains an s Right, it has to contain the underlying stream s here is the stream that we're sending over And so it actually does need to be generic over s for a mod 100 Send has to be send This probably also has to be sent That is because it is into future Maybe slightly off topic and better personal preference. Does linux make it easier to do this pure program in the windows? Uh, I have not used windows for programming for Uh At least 10 years so I can't really speak to that Uh, I have never found that I wanted a different operating system for programming I have several times observed people who Use linux and a vm for programming or for particular pieces of programming Which suggests to me that I'm not really missing out But I can't speak to it directly Uh map error We did say that the error has to implement oh into failure error Uh, okay. So we have to do it this way. I'm almost there 542 with a Basically what you're doing with this transport is distracting away the stream Not quite. I'm not actually abstracting the stream away Uh, the s is doing that what I'm what I'm doing with this with the trait is I'm saying I need a stream that has certain properties So you should think of traits as like behaviors that a type supports So in this case what I'm saying is that I can be generic or packetizer can be generic over any s Where s implements async read async write sized send and has a connect method that returns Something that is a future over itself, right? So that's still packetizer being generic so I'm not I am abstracting over it in the sense that I'm allowing it to be generic, but it's not like I don't need to know what the underlying stream is. It's really the implementation doesn't care about which stream it is But it needs to know Like the types the compiler needs to know which type the stream is otherwise It couldn't put it into the struct in the first place That is also true This needs to also implement send Oh, that's not what I meant. This needs to implement send Oh, I don't know if I can even say this Like I need Into future Okay, so this is a little awkward I need so so I'm saying that connect can return anything that can be turned into a future But I need to say that the future it is turned into is send Which I can do with ware clauses, but I don't think you can have a ware clause on type So I think I actually have to do this which makes it quite awkward to write Connect Probably still fine, but Uh, and it has to be static probably Oh, right. That's our reconnecting. This has to be static So the expression is assignable where 516 It is telling me that this thing It's telling me that this thing is not Static which is true because it's borrowing self this Oh, I still have the into future. Yeah Oh, because x Xid is going to be self xid Self xid is going to be plus equal one This is going to be xid and this is okay because we know that the Will not start accepting new requests until this is finished Therefore that xid is still valid Connect feud may not live long enough Yep, because connect feud also needs to be static That's better. I like those errors better. Yeah, so this is where I told you where Uh, this is where we need nll Because I'm I'm swapping. Wait, that's not where we need 542 We're swapping self dot state But we're also borrowing out the password down here Uh, and the compiler doesn't know that that's okay Um So what I will do is this Which is all so terrible this of course now it's unreachable because we're checking it above Um, so Now we have the password down there and that's how you work around in a low Uh, oh, sorry. Yeah, you're right that we could totally store a uh box zookeeper transport, um Inside packet iser. This has always been the case, right? We could also always have Stored a box async read plus async write plus send plus static if we wanted to But there are a couple of reasons to have it be Generic one of them is to produce a faster code although that probably doesn't matter too much for zookeeper Because you don't have dynamic dispatch within of it The other reason is because it means that you can look at the type and see what kind of connection you have And this can make it easier to do things like imagine that you wanted to connect to zookeeper over tls Right, then now a part of the type will be this is a zookeeper connection over tls Which would not be the case if you boxed it. Well, that's a bunch of errors Okay, so apparently zx id's are not increasing Uh, that's also kind of interesting. Uh, zx id self lasts That's odd Oh, I see it's it's greater than or equal Might as well keep it, but Minus one G Thanks, why would zx id be minus one? All right, what does russ zookeeper do here when it sees last? It's zx id scene because that's awfully strange I don't think russ zookeeper does reconnection Yeah And if it does reconnection it does not do it correctly Because this can get minus one So unless they're Yeah, I don't think that's right I don't think this can be just be blindly accepted because what we're seeing here is I'm getting a request where the zx id is minus one I think this is a bug in upstream In the russ zookeeper crate the synchronous one I think you need to ignore things that are negative one If zx id not equal to negative one Then do this Uh, if you get negative one, then it's from the watcher What makes you say that? Because this should be a response from oh, I see I see Yeah That still makes this wrong. This line is still wrong. I guess we can assert that x id is minus Minus one Only watch events should not have zx id It's also really weird. I don't know why zx id's would not be sent for watch events Panicked a whole right why? Let's just double check to this code Not yet Just to see that that code is not actually reached No, both ping and connect disconnect should have zx id's And in fact you see that they do right like here we're getting the response to To connect I guess we should print them out here. How many response to xid? zero with opcode create session And there you see we've got zx id's 349 The exists we get 349 So this is basically how many writes they've been so a read does not increment it But you see the connect does give us zx id zero plus minus two No, zero gives us a zx id to connections does a search and failed Well, that's interesting At lib 132 153 Oh, it's because it crashed halfway through last time So I need to delete slash great I think I agree with patrick too that Now is probably a much better time to get into rust than like in the past many years I think it's also there's still things that are shaking out like the whole async await things I think tokyo still needs a bunch more work But I think there are a lot of resources. There are a lot of like the rust book is really good And so we are we are getting pretty far and at this point You can actually use rust for a lot of things like there are crates for most of the things you want to do And so I think I agree like this is a good good place to start Uh, okay, well, that's awesome Notice how the So if you look at it the initial zx id when we connect here now Let's connect Is 372 And then we do an exist and we still get 372 Then we do a create and then we get 373 because a modification happened Then we do an exist and we get 373 again because nothing changed Then we do our delete and then we get 374 because it was a change And then 374 because nothing changed 375 5 is when we leave because a a node leaving Also triggers ephemeral nodes to be deleted. And so therefore 375 hours to go up we could add a bunch of tests for things like ephemeral nodes and such I think we'll leave that for some other time We now have all the infrastructure. We haven't really tested whether reconnections work Uh, so I guess To do test this But I think that's basically I think now we have all the stuff that's in theory. This should be all that's needed for the reconnects Um There will all the clients that are waiting for us replies will indeed get errors And any that gets sent after we detect the disconnection happened those requests will still be sent as normal So I think we're in a pretty good position now So let's commit that first round Our first attempt at automatic We'll still have to do things like if you If the user gives many addresses for zookeeper Nodes and we'll have to sort of iterate through all of them and keep trying because it could be the one server You connected to went down But there are others that where the connection will work successfully We'll want some tests with watchers But I think now we have most of the sort of core protocol and the higher level api At least functional. It's not necessarily well designed yet, but it is functional Um, I think we're going to leave it there because this room is getting pretty long Um, but hopefully there'll be another one in not too long Um, so I hope you enjoyed it. I'll I'll push this um, and as always Like if you have questions or whatever then reach out to me on So I'm on twitter at twitter slash john who so you can ping me here if you Apparently someone has already If you have something you want to say there's also the patreon page Where you can follow me and subscribe If you want to hear about upcoming streams, you can also Follow on twitch or youtube I did a test for deleting non-existing things No, I so I've I've not added any test beyond the one we have There are lots of tests that could be added now because we have all the functionality and adding those is a good idea I just haven't yet. I figured I would focus more on the The functionality and the async part of things than on adding tests for everything Although that is obviously something that should be done But yeah, you can follow me on twitter follow me on youtube follow me on twitch follow me on any of the others I do stop by the rest I see chat at least every now and again. I'm the same username there I'm basically john who pretty much everywhere except twitch and youtube and mixer Which are the three streaming platforms Because on those that username was already taken in one get one of the cases taken by an old user of my own And I can't change it. So that's annoying But yeah, thanks for stopping by. I hope you found it interesting I'll post a recording of this once we're done here, and then you can go back to the places where things moved quickly So thanks for joining Bye everyone Like this


So this is day one of filming when I say day one I mean as you can see by my hair and my costume and where I'm at I've not even started filming I'm getting ready. I'm waiting for my head to dry. I am setting up the camera and I thought I'd use this opportunity mirror to Really take you through how I film one of these sketch and especially this one as it's going to be quite Well the start of something big That's the plan anyway So let me show you sort of roughly of a proper behind-the-scenes look of how I film The script I will always have this nearby. I've mentioned before here is sort of a little look at page one And you know page two and three if you wanted to pause and have a look at how it can change from Scripts to screen next is the camera. This is my camera. It is a lovely camera Canon I got this back in 2018. Actually, no 2017 Because it was a Christmas present that came with the mic. It's very good 500 pound if I'm not mistaken and it's lovely Tripod, I don't know the brand actually. This is not the OG tripod. I've had two tribals over the years Yeah, it's a newer there you go As I trip over the chip costume is one of the most important parts This is me in the McGann outfit that I've sort of cobbled together waistcoat from hello cosplay this from Amazon It's a woman's coat hence why the sleeves are a bit short just a blue tie for the sort of scarf thing He's got white shirt some rubbish costume trousers this belt Which rarely sees the light of day and usually I'll be wearing boots, but you won't see them So then the rest of my costumes what all of my costumes except for some which are in there Go in here. This is mostly doctor coats You know, you've got the tenor one of Capaldi anything that can't really be folded away or I'd feel bad about folding We'll go in here and all the smaller bits Go in Eccleston's little cupboard, which here's a proper good look at this Right now. It's got my friends birthday presents in there Jack from the future here I want to make it know that I have actually fixed this because for Christmas I've got not one but two sets of drawers. There we go Lovely nice and clean these stay there box of tat a bit more tidied away as is this There we go. It's not always I'm gonna keep it cleaner. That's what I'm saying Also, that's got sand in it. Oh my god. Mrs. Moore spare I guess cool Something else I've been doing more of is lighting usually I will just use you know the normal lighting I do have that on but recently I've been using these more often especially in the stone model as such you would have seen It just gives you more of a more definite light as you can see if I stand here You can see more of my disgusting face, but if I stand over here, you know, it's a bit darker. It's a bit grainier I bet when you stand over here Even on my phone You can see so much more Fun little thing there. So using more lighting is always preferable That's sort of a rough sort of look of what I can see and what I set up on a on an average sort of scene The next scene after this first one, which I haven't again filmed yet is a green screen Heavy one basically what I did in the Super Smash Brothers I'm going to do again as you know I'm explaining it as if you haven't seen it.  What do they call it? Is it just tenants vlog or was it something else video diary? That's what it was video diary. I keep looking at myself as opposed to the camera I was gonna say I'm doing the green screen stuff. I am a bit I'm doing the Smith bit at the moment about to do Smith the shot of Smith and we're going looking at the crack But I need the green screen for that obviously can't quite be a split screen because life sucks. So Fun little life thing. I saw I'll stop the fun little life thing on Saturdays. Usually we do washing And I you know shove it all in the washing machine because we stink Including my work trousers, which is what these are They are still wet But I have to use them Suffer for my art So this is a few days after I've started filming and this is the setup. This is for the flying through the vortex scene I've got the two lights Set up here It looks like this side. This is what I'm looking at two lights and the camera and Yeah, the plan is just to flail around on here like I did in Super Smash doctors And hope it works. I've got a green coat on so I'm gonna have to do a test first See how it goes We are spreading the sand To make Gallifrey help me Look at that it's actual sand. I'm not faking it He's very glittery, but I think that looks quite nice I mean, I'm glad it's soft. So that's good We do that couple times just to make sure I've got it. Oh God, I'm gonna regret this Kick something Right out of the camera. Fuck. I knew this was gonna be the hardest shot That's good That's a good one. Here is the set For the Ainsley or Ainsley. I keep doing that. Ainsley's is someone very different Ainsley part of the anti-matter universe There's Jim So we got a dog cage which I nicely borrowed from my parents That's a fun conversation Just some random tat. The way I do set design is very basic. It's set up a cool bit of light Just throw shit at a background and see what sticks. This, however, is cool We, well, Jim, I've got this for a nursery production But I insisted I keep it because I knew I was gonna get more use and here we are getting use out of it I'm not excited to get in the cage. I'll be honest Need's must and I am Ainsley Cribbs Coming at you live from the cage I mean, it's no Pandorica, but it's it's a prison. Hello. Welcome to Cage Cribbs with me, Anthony Ainsley As you can see, I've got a pipe for a talking into like that and if so food and water comes out This is where I piss in. It goes right round and pisses just straight back onto me.  That's all I got Right, where's my next one? Since when does Cybermen have a concept of beauty? My roleplay scenarios Since when does Cybermen have emotions and concepts of beauty? You know, in my roleplay scenarios And Type 1 was born. This still doesn't explain what you're making. You know, this still doesn't explain what I'm You do understand that it still doesn't explain what you're making You do realize that it still doesn't explain what you're making Was my first original question, if you can remember that far Oh my God, Janet, you're covered in sand Oh God, no, is it? Oh, I'm sorry, I'm sorry, I'm sorry, I'm sorry That's okay I'm not the... Just tell me Oh, I'm not the... What you're making? You are Doctor Who confidential Oh, that's a good shot.  I like that Hashtag set up Hashtags It really hurts to get in and out Oh yeah, catch this on camera The glamorous part of Doctor Who Road Me, dicking about in a dog cage Can you turn the screen around so I can see it So I know where my fackey is Welcome, this is how they film the Caves of Anjouzani Boo, oh I'm very cute There I am Hello Wait, hold on Did you create the cracking time? This is what, this is what it's all about No, the hokey-cokey's what it's all about Oh, sorry, can we just... Can we just...  Literally all of the Anthony English Just so I don't have to do it again I'm going to be filming in the Omega set This is... I don't know Again, I just sort of threw things together This is going to be the un-tempered schism So you won't see it much Just a loose background, laptop there Again, it's just a 180 degree set You don't need much else, at least not for this But with the lighting, I've connected up to an old power outlet here So with the press of one button The whole thing lights up like that I've got my Time Lord panel there Just a lot of things to hide the background So from certain angles You wouldn't really be able to tell I mean, you can always tell But you can't win, can you? And typically on a day my hair looks quite nice I've got to wear two masks Okay, life's cruel like that And meticulous broadcast Has sent over the Omega and Mrs. Moorline So I can mime to them That's the plan anyway Don't know if it's going to work I hope it's going to work Now to show you my favourite thing I'm in my Omega get up And my mask Is attached to a welder's mask So to help with filming I can just winch it up and down The only issue is, because of how big the mask is It is very heavy And especially when I have it up like that So, I mean, we'll see, won't we? This is the first time I've filmed as Omega I'm going to take a couple of pictures first I'm quite happy with the get up I quite like how one of his big villains is Pertwee and he's wearing the Pertwee jacket I think that's just very funny Anyway, back to it So I've just finished filming the first Omega scene It's short, there's not much to it but Yeah, filming with the mask on And all of this on I should have guessed it was going to be harder Than I thought I can't really see the focus The mask is so big it's hard to get in frame It's going to be a challenge And I'm very worried it's going to look like absolute Dogs, bollocks And not in a good way But unfortunately I just have to keep going And hope And that is for now Mrs. Moore and Omega wrapped This is probably the closest Do you like this by the way? Look at that Got that for Christmas This is probably the most going to be cobbled together in the edit Sketch I've ever made Because I really don't know what it's going to look like There's only so much a small set And costuming can do You know, meticulous broadcasts Lines are hilarious But will they work? Will the tone fit? It's at this point A part of the way through a sketch I always get the jitters The doubts The will it even work Thoughts spreading into my head It usually works out But we'll have to see But now the fun bit Having the tidy all my Flubbing sweet way And Gemma's out at this point So it's all me Yay My life's fun Where do I even start? So it's Jack Back in the began Guess I'm sitting at my desk So it is January 16th As I'm sure it says on the bottom There And I've got this Which is exciting Got this a few days ago I took some tenant pictures with it There's a couple of my favourites Facts on the discord I don't know what he wants to be going by Not in use Facts, whatever Made this amazing Edit Which is brilliant Anyway, so I bought this Which is a background for Well, let me show you from the back It's the poles that will hold it up And that is to be used for background And I was about to film the next big bit With Romana Now, originally I was waiting For something for Romana Something for on Etsy It looked like it was selling A time-long headdress But there was some confusion So now I'm working on getting a refund So I'm now scrapping that idea And I'm just going to move on With filming without it At least I was going to Until I found out that this plug down here It's broken Part of the plug of my camera charger Is jammed in the wall And it's broken off into the socket Which means it's not usable I can't charge my camera So I've now had to buy a new one Which will arrive on Monday in two days So the only thing I've managed to film Is just that bit of McGann Appearing in the capital That's it, that's all I can film So that's annoying Oh well Also our boiler broke for like two weeks But that's not a conversation I'm going to talk about Sometimes it's just not fair The boy's going on a snow walk And why wouldn't he wear something like this? So I'm trying to film the Davros scene This is the set, I've got the black sheet Got the light giving me hopefully a nice silhouette I've got the Eccleston jacket on backwards And the head will be this But backwards Only issue, well one of many Is that I can't breathe When I wear it like that But there's no other real ways Of doing it so Choking So here I am sat on the floor It didn't work As you would have guessed as you've seen it So I've got the voice recordings lined up here As you can see this is where I'm at in the edit I might as well show you At this point in time We need all hands on the high council tip This is where I'm at So I've got the line I've recorded As of time recording I don't know If I've got someone new But so time lords and Gallifrey So I've got it set up and this Is what I'm looking at A tea towel, a cardboard box Two shortbreads Boxes on either side Classic who DVDs Surrounded by this thing And there he is Daddy Davros And looking at it on the screen Look at that Look at that That's so cool No I'm very chuffed with that As a plan B Very nice Okay so there was a new issue I realised I needed to do the hologram shots Where he needed to be green screened So I've balanced the green screen on the sofa I've got Davros there And there I've got the camera As you can see there's the There's the shot But Davros is supposed to be in shadow So what am I supposed to do? Well very clearly Just that Keeps the green screen light and him dark Works a treat That's that kind of quick thing Can you get it Doctor Who Guy ink or corp Or dick head with a camera And now I'm filming the TARDIS stuff Here it is Originally it was going to be green screen Here's some footage of what that would have looked like Terrible Agreed So I decided to make it a physical set That was always going to be tricky But this is what I've come up with These things I've used a few times now Throughout the sketch I've got random little bits and bobs I can interact with And then the TV on the background will have the scanner Tom Baker scarf as well McCoy umbrella and Hartnell walking stick Just some fun little Easter eggs Two big lights There it is And yes This in the background which has become a necessity When filming I love this thing now It just helps It's beautiful Here's something I haven't tried yet A time-lapse I've filmed most of them again So you'll mostly see me change into the Ramana Get up Time-lapse time I'm going to go for a while I've been mostly putting off the last couple of sequences Because it involves a lot of work Here is my current setup This is the Scarow Calan set All of the things printed off from Action Figure Theatre Got that off my friend Mondes Productions And I've got to say I'm quite proud of it You know, if you take your camera in It looks quite impressive I'm quite happy with that turned out I don't know how the green screen stuff will look yet Because I haven't made it But we'll see I've also got my storyboard here My very rough and incredibly drawn Storyboard Just so I can sort of tick off As I go Also loving this fucking coaster Hell yeah Am I getting a figure adventures again? Because look at some of these shots I'm getting The worst part is I'm replacing that with an Imperial Dalek So Like But it looks so pretty So Here we are I'm about to film The final scene of Return of Omega And hopefully Some of the bits for the cracking time Or just to get that done As I have it all set up But speaking of setup I thought I'd show you Just how much of a pain in the arse return of Omega Is to make From a setup level Here is pretty much the stuff I always have to get out For filming any scene So here is the Frame for this Which is the big black sheet This will cover basically An entire half of a wall So I need that for the background Just to make it so it's not just Gallifrey but the flat Which there is a bit of But you know it can all be helped These I've got my two lights in them And those are my light stands Of course I've got the cords in there too Obviously I've got the tripod and the camera This is how I store the Stuff just in case I need them again But yes This is pretty much all I need for Any given scene of Return of Omega So I don't film with Omega a lot Actually for this sketch I've only done it the one time But look at that Look how gross that is All the hot glue keeping it to the welder's mask And also if you want to know what I'm seeing That Just that Omega can't see Can you see the reflection? Sort of What am I doing? And with Mrs. Moran Omega filmed The washing is allowed to be back on And even behind the curtain This whole time I've been Transferring the footage of Super Smash Docks Onto memory sticks And putting it onto the new computer Blooper reels I can say Return of Omega Is filmed Well at least all the people Shot anyway I want to do a couple more Green screeny bits Specifically Of the Tom Baker TARDIS Flying through the Thing What's it called? Vortex But other than that I'm glad to call that A wrap On Return of Omega Now to do the crack Into time war Hello to be fair I have already filmed quite A few bits for the crack As I've gone For example the master Killing war master That's filmed already I did that ages ago And I also filmed the first Romana Omega scene Literally as I was filming this Same background, same characters Probably take these off the doors So yeah, that's already two scenes done I've also filmed a bit in the TARDIS too Because I didn't want to rebuild that set Might have to though But yes Very happy Very tired It's hot in here I'm going to reopen that window Hope you enjoyed this confidential Planning on doing the same For the crack in time war When that comes about But for now You don't see a lot of sunlight in this sketch