Hello, everyone. Thank you for coming. Ni hao, si siu. And that about exhausts my vocabulary here, unfortunately. So I'm Mark Barel. I work for VMware and I've been involved in the Etsy OSM project for a little while and partially helping out with the VNF onboarding task force. So I'm going to be taking you through a little bit of a journey on what it means to onboard a VNF and what some of what Etsy and the VNF onboarding task force is working on. The example that I'm actually going to be using is a pretty telco specific one. It is an EPC open source EPC software called Next EPC. And because I might not, you know, not everybody may be familiar with what cellular data control plane and data plane looks like, I will just give a little bit of a brief explanation of, you know, sort of what we're doing. I'm going to take a little journey in walking through the entire deployment, you know, from I've got this piece of software to how do I get that up and running and how do I model it. So first of all, and I'm going to go through this fairly quickly because the last thing I want to do is read this all for you. But OSM was founded, it's part of under Etsy, basically, and they've defined a whole bunch of specifications and everything around NFV, VIM, NFVI and more acronyms than you could possibly imagine. And the the entire idea is to, you know, to form those standards and to use OSM to provide an implementation of those standards. So the main benefits of OSM is that their information model is based upon standard specs, the Etsy Sol 001 and 006, which I'll give the URLs for those at the end of the slide deck as well. So so it's a full life cycle, automating the network functions, whether they're virtual or physical. And yes, they're still working on defining containerized network functions. So there is no spack out on that yet. Network services and network slices. So for this, we're primarily concentrating on network services as part of a VNF. So it does go through the entire life cycle. Initial deployment, which you can call day zero. And that is about, you know, just laying the software down, getting the base operating system up and running. Day one configuration, which would be kind of last mile configuration, getting things to talk to each other. And then what they call day two operations. And that is, OK, once everything's up and running, typically there's some care and feeding that needs to be done. Things like monitoring and making sure that it's OK. Healing or in the case of the EPC, adding subscribers or adding routes to the P gateway so it knows where to go. And the good thing about this is the information model is completely infrastructure agnostic. Right now, OSM does have plugins for OpenStack and VMware VCD. So I can take the exact same service and deploy it on different virtual infrastructure managers. And OSM takes care of abstracting that for me. In fact, as long as I've got some sort of network route between them or an SDN controller that I can talk to, OSM can actually deploy VNFs or even virtual data units, aka VMs on OpenStack and VMware and have them talk with each other. So what is VNF onboarding? So one of the challenges as associated with actually getting things up and running is figuring out, OK, how do I get through all these specifications and know what I'm talking about? So the leadership group decided that an onboarding task force should be formed. And they're basically tasked with, OK, we got to, you know, come up with documentation, working procedures, examples and models of how you can actually deploy this and get it running. It, like, participation is open to all OSM members, but the contents and whatnot are all freely available. You don't need to sign up with Etsy in order to be able to read any of this documentation only if you want to contribute. They're working towards publication of white papers, documents, et cetera. There is a full VNF onboarding document as well as a walkthrough that is already available on GitHub. And they're working on actually coming up with a repository of VNF so people can use that as a sample. So onboarding requirements. I talked about day zero, day one, day two. So breaking that down a little bit more, day zero is the type of thing where, OK, I need to, you know, know what my images are, get them uploaded to OpenStack, figure out, you know, sort of the basic operations around there. And in as part of, you know, the OpenStack images, you'll also sort of need to make that decision of, do I use just a base cloud image like a Sentos base or an Ubuntu base and then install the software at deployment time? Or do I want to pre-install all of my software, save that as an image and send it out? Day one is service initialization. And typically, you're, you know, really closely tied with day zero because you don't have an operational service until day one is complete. And that's kind of the last mile configuration of gluing the pieces together. Making sure that, you know, they have IP addresses that were assigned, that each part knows their corresponding peers IP address to get set. Everybody can talk with each other and it's operational. And day two, runtime operations is all about modification to accommodate any form of changes in the environment. You can also do things like invoke healing routines or restart VMs or basically any script that you could think of can fall in under a day two operation. Monitoring of KPIs is actually taken care of by OSM. So you can put into the service descriptor directly. These are some of the KPIs I want you to gather. So the overall workflow looks like this. You take your VNF package, which is what we're going to be talking about today. We're uploading that into the orchestrator and it's going to go ahead and instantiate that with whatever parameters you need to specify the instantiation time and stick that into OpenStack or VMware vCloud or AWS. Sorry, forgot to mention that earlier that there is a plugin for OSM to manage AWS as well. And that will actually create the network services that get instantiated. OSM will then log into the various instances, perform any IP address, like configuration files or whatever is needed to get things talking with each other. And then three, monitoring the system and performing any additional operations that may be needed at runtime. So talking about the next EPC. Is everybody familiar with kind of what an EPC is or yes, no, maybe? Okay, well, we'll take a little bit of a walkthrough then. An EPC core is composed of a bunch of components that were already defined by the three GPP consortium. So they've come up with these various functions in the network that need to be present in order for cellular data traffic to be able to pass through it.  So all of these functions are required. So the first one is the mobility management entity, which is what ultimately your cell phone is going to talk to and register itself on the network. It, you know, it's the gateway into your cellular data control and cellular data user plane, but it's the control function that makes sure that you're authorized. It figures out the serving gateway, which is the module for signaling between the mobility management entity and the packet data network gateway. And its responsibility is specifically once you start distributing parts of your EPC network, it actually is responsible for which packet data gateway you're going to go out of. So in a multi-edge scenario, you could actually have multiple P-gateways out there. So the P-gateway is your kind of last mile hop, which takes you out of all of the encrypted GTP sessions, the bearer sessions inside of the network, inside of the cellular data network, breaks out your packets and then routes them to the appropriate endpoints. So those could be things that are actually inside of the carrier network or go out onto the internet at large. And lastly is the home subscriber service, which is where all of your authentication keys get stored and as well sort of profiles of what sort of access you can get inside of the carrier's network is stored inside of the HSS. So the NME gets the keys from the HSS, challenges the end user equipment, and then sets up a path out to the internet for you. What does that look like in a picture? So we've got our user equipment, UE in standard terminology, is sitting out here and it connects to what's called an E-node B or the radio. That then uses the standard interface published by the 3GPP consortium as the S1 interface to talk to the NME. NME uses an S6A interface to talk to the HSS, goes out through the S gateway and P gateway and uses the SGI interface to go out to whatever network you have out the back end. So ultimately you're ending up with a secure channel between here and here and then the breakout occurs to go to the internet or as needed. So the next DPC software that we're looking at today combines these four network functions as one piece of software. Now each one can be run independently or they can be run all inside of one VM. For the purpose of today's sort of little demo, I'm creating one VM that is going to be a combined S gateway, P gateway and NME and a stand another VM for the HSS. So the reason why I didn't specify what the protocols were in here, because they're not relevant right now, they're going to be running as processes inside the VM so there's no external connectivity going on. So we don't need to define a network for them. So now we're going back to our journey of day zero, day one, day two. The objectives for day zero is to make sure that we understand all the necessary elements so that we can actually create the VMs at the end of the day. We need to know if there's any sort of network that networks that already exist or networks that need to be created at instantiation time. Requirements for that? Well, you're going to need your images in OpenStack. We're going to need to find out all about your networks, whether they exist or not. IP addressing schemes may come into play there because it may be a network that already exists in the carrier and you have to adhere to their subnet. And host names and IP addresses as needed. So what I have here now is I have taken those two VMs and I'm now modeling it in terms of, well, I know that coming in one side, I've got the S1 interface and coming out another side, I've got an SGI interface. I know I'm going to need management to be able to get into these and control them. And I know that there's going to be another network in here. So for the VM itself, we're going to get down and, you know, a little bit into some of the details and go, okay, well, I have to define network interfaces here. As otherwise, how am I going to connect to the networks? So I just randomly picked, okay, well, management is always going to be zero because that's kind of the most important one for me to make sure that I can even get into it and talk to it. And then just kind of added IP address or, sorry, interfaces as we go along. So in my demo, the S1 gateway, or sorry, network, an SGI network already exist. So I'm pretending I'm going into an existing carrier and that they've already got a cellular data network set up. So these already exist. What doesn't exist is this little interface between the SP gateway MME and the HSS. And that doesn't need to exist outside of the VNF that I created. It's a private network that exists between the two only. So I'm going to let OSM create that. And it can be any arbitrary subnet as long as it doesn't collide with something else. And we can go ahead and, you know, assign IP addresses as we need to inside of there. So breaking things down a little further. I've got my S and P gateway and MME, single virtual data unit or deployment unit, basically, in other words, a VM that contains these three pieces of software, giving it a host name, defining what my external IP addresses are. I've got my HSS, give it a host name. I don't need to necessarily worry about the IP addresses because it's got no external connectivity. And something important here. This is going to be handling user traffic. So it's going to be passing a lot of data through the P gateway. So I want to enable some CPU pinning huge pages and SRIOV in this particular VM. So these are all things that can be specified in the network descriptor under OSM as well. And when it goes to hit open stack or whatever technology you have underneath, it's going to, you know, require that of the Nova schedule or saying, hey, schedule me on something where these requirements can be met. So next I get these cloud images, which are freely available for download. And I, you know, upload them to Glance, make sure they're there. And I take a look, I go, okay, what do I need? Well, these are cloud based images so I can use cloud in it to define my host names and have an SSH key pass through from an external source. So I could generate my SSH key locally and just pass that through OSM and OSM won't store the SSH information. That's for me to know, not for OSM to know. So that's day zero. Now we come to day one. And that is, we got to figure out how to make sure that all of the services are initialized and any dependencies between them such as, well, the HSS needs to know the IP address of the MME because they're going to be exchanging information back and forth over the 3GPP diameter protocol. So that's, that's the sort of stuff that we need to look at here. If there's any configuration that you need to do to your VM before it's operational, this is the time to do it. So taking a look at the next DPC side of things, here's my same diagram and I look at this and I go, okay, well I know I'm going to need four network interfaces and I'm going to need four addresses for it. I'm going to need to generate SSH keys and get them in through there. I know that the MME configuration needs to be updated so it knows where to find the HSS. The HSS only needs two interfaces. It needs SSH keys as well. And I'm going to want to update the HSS configuration so it knows how to get back to the S gateway and MME. At this point, our service should be operational. Things should be able to go out there and we should be able to say, okay, it's up and running. So what are the things that I might need to do after this has been running for a while and my users have been using it? Might need to reconfigure the VNF. We might need to do things like scale out or add additional capabilities to it. We also want to make sure that we're monitoring the KPIs and have the ability to raise alarms if anything happens to it. So with this, again, I want to see if there's dependencies between components. In the case of some sort of scale out, I would need to know, well, is there a load balancer that's capable of making sure that I'm scaling out and there's nothing that needs to be reconfigured? Can I just add additional IP addresses to the load balancer and it'll take care of it all for me? In the case of the EPC, that's not going to work because diameter doesn't work with load balancers. You need different software for that. Figure out what different configurations for the runtime operations you might have and as well figure out what metrics you want to collect from your VNF. As well, you're going to want to define if you've got any closed loop operations in terms of scaling or healing, this is the time to think about them. What does it mean at production time, at runtime, when this is already ready going and executing? If I want to scale out, tying back to the dependencies, what are the things that need to be affected in order for me to scale out? Next EPC, I decide to keep things fairly simple. I'm going to be able to say, well, I want to define different routes out this interface here so that the next hop is internal to my carrier network or external to the internet or perhaps pointing it at cash or something like that. I also want to be able to add and remove subscribers from my cellular data network. And I want to make sure that I'm capable of collecting CPU and memory metrics. Actually, I forgot one point there. I also want to capture the network throughput because I want to see if I'm saturating my network or not. So a little bit of a step aside again here for a moment before we get into a little bit of a demo of this and that is this is the published VNF onboarding guidelines. And it actually goes through an entire walkthrough, similar things to what you saw here, breaking down the individual units, how to build this all together. And if you want to actually get into it a bit deeper and see the Tosca templates for the network descriptors, this is where you're going to find it. Tosca templates aren't all that interesting, so I'm not going to pop them up here right now. But another thing, the code for today's demo. So this is basically sort of a screenshot and a couple little video type of demo. So if you wanted to be able to replicate this or do this yourself, first of all, you're going to need OSM and you're going to need an open stack or AWS or vCloud or something to be able to run it on. And then you can grab source code from here. You can also take a look at it and you know, I call it code, but it's really the the actual network service templates that were used for this. So it's not so much code as the full descriptors and how it plugs together. So what does this look like once you've loaded it into OSM? So here was the picture that I drew for myself originally saying, okay, here are the interfaces and here are the external connection points that go outwards. And so once I model this inside of OSM and there's actually a visual modeler inside of there so you can build some of your network service and virtual network function descriptors in OSM using the web UI, but here these green points are external connection points. So these are things that are going to be exposed to the outside world. The pink ones are internal connection points with an internal network.  So that's what it actually looks like once it's been visually modeled inside of OSM. So day zero, the OSM is going to automate the whole life cycle, the virtual network life cycle. So creating networks and removing them once the service is being deleted. SSH key injection and it does this through two key elements. The network service descriptor, which defines the entire environment around the VNF and the VNF descriptor itself, which describes the data units or VMs or physical network functions that may be used in your environment. So here's a little short video basically of me deploying, building and deploying the package. So inside of OSM right now we can see there is nothing there. I flipped over to my shell and I'm just running the build script right out of GitHub. And so it's basically pulling everything together and I'm going to pause for just one moment there and go. I'll get into it a little bit more in a few minutes, but the main automation tool that's being used under the hood in OSM is juju. So we're actually building juju charms to be able to do some of the work for us. Now these are not hugely sophisticated right now. It's more along the lines of log in, set this file, or log in, execute the sensible playbook. This is not hugely complicated stuff that I'm doing in this particular example, but it can become as complicated as you need it to be. So here I'm showing, oh, this foe LTE is more open source software and it's a radio and end user equipment simulator. So a little bit later on you will see I'm actually going to be logging into this here, starting up the radio software, starting up the cell phone emulator. And we'll be able to see that it gets an IP address from the P gateway. So there's no VMs currently deployed and the network topology is pretty similar to what I said before. There's just an S1 and an SGI network. So there you can see there's just the two of them. And as far as the instances go, just there. So here it's actually going ahead, uploading it now to OSM. So there is what these packages look like. We've got an NSD and a VNFD and now it's actually deploying the network service descriptor. And pause right here for just a moment. Here we can see it's now saying, oops, sorry, waiting for VIM to deploy. And what that is, it's now turned around and said to open stack.  So we can see here that it's expecting up to two VMs to be created and it's going to validate up to four networks. One for management, one for SGI, one for S1 and the internal S6A1. So going back to open stack now, so continue, we can see that it is now created this internal network that I wanted. And as far as the network topology goes, it's created the VMs and it's hooked all the VMs up to the appropriate interfaces. Inside of the VM, my ETH0, ETH1, ETH2, they're all plumbed in the way I wanted them to be. And here we can see open stack says, okay, well, here are the IP addresses that you asked for and the service is now up and running. Well, with one exception. And that is, and I didn't automate this as part of the demo. I actually manually logged in through the web UI because I kind of ran out of time to show this being automated. But this is certainly something that can be automated as a day two operation. I'm actually logging into the HSS right now, and I'm adding a subscriber. So if you actually look on your cell phone, at some point you'll find that you have an IMZ, an IMSI, which is a unique identifier that identifies who you are as a subscriber. And in as part of this, we need to add the encryption keys that are going to be used to challenge the end user device to make sure it's authentic. So those keys are actually burned into your SIM card. And that's about it. Oh, the other things that we can do is you can actually configure access point names, but that's getting a little bit far down into the EPC core that we don't need to worry about right now. So day one. So now that I had everything, you know, sort of deployed, what we didn't see or I guess what we did see was it deployed it in OpenStack and then turned around. And here's what I was talking about the VNF configuration and abstraction module or VCA for short, which is another term, you know, Etsy loves their acronyms. They love defining these things. So this is a part of the man of specification, which is the thing that interacts with your VMs for you and keeps them running or logs in and does the work for you. So as I mentioned in OSM, this is implemented by a juju controller and it logs into the VM and performs actions. And in this demo, really all it did is it was modified configuration files to add the IP addresses for the current network and restarted the services so that they could find each other. So this one is not quite as interesting because I didn't use a real cell phone and it's more command lines. But basically what we've got here right now is, so this is my faux LTE VM again. I've logged into it a couple times. You'll see a couple terminals in a moment. So the first thing we do here is I show the IP configuration for this particular VM. This is the software radio starting up and then over here you can see with the IMZ, well, scrolled off and the keys that were acquired. So this is actually the cell phone starting up and now we can see we've got ourselves an IP address that wasn't there before. So this is actually attached to the EPC Core now and I can ping the P gateway and the P gateway can ping back to the cell phone. And this actually takes five to ten minutes altogether end to end and the entire thing is up and running. Oh, sorry. And here when I detach the cell phone, you can see the IP address went away again. So day two, monitoring. So as part of my network service descriptor, I said I want to collect CPU usage, memory usage, network packet sent and received. So that's already running now and OSM comes by default with a Prometheus database to use as its time series database for all of its metrics. And the alarming module also uses that to raise alarms. You can also have your own elk stack or ask OSM to provide you with an elk stack. So this Grafana dashboard is something that, you know, if you ask for the full stack to be installed, you'll be able to get this information right off from your Grafana instance that OSM has installed. And here what I have is brand new instantiation. There's nothing really running. My CPU is less than 0.8%. Memory is pretty low usage here, 490 megabytes. Yeah, 490 meg. Network packet sent, less than 0.7 packets per second, packets received, less than 0.65. So it's a pretty quiet system. The next thing I did was from the UE side, I just simply started up IPERF headed towards the P gateway. And on the P gateway side, I have the IPERF server running. So now I'm generating traffic running between the two of them and I can see in OSM, all of a sudden my packets sent are starting to go up. We're not at point something anymore. We're 250 packets received, 800. All of a sudden the memory has gone up because, well, IPERF is running and it's doing things. Same thing with CPU. CPU usage goes up. So all of this information is collected automatically and I can do things with it as needed. As part of the overall OSM architecture and the things that it provides, it actually does provide an alarming module. So you can set simple rules like if the memory goes above 64 gigabytes, raise an alarm. Or if the CPU drops below a certain amount, maybe I want to scale in at that point because maybe I've got too many instances running and if they're running really cold, why do I need them? Why don't I just scale on in? So OSM does have scaling operations that can go with it and the VDUs can be monitored and scaled independently. Yeah, I alluded to this earlier as well. They too, operations are actually capable of executing. You can write them to be arbitrary scripts and functions. So you could have something like an external monitoring system that either is taking a look at the alarms that are being raised by OSM, looking at the raw data, or maybe it's something like VMware vROPs where you realize operations that has been monitoring your VMware infrastructure and it's raising alarms on its own, which we can then turn around and tell OSM, do something for me. So there's built-in actions or there's the availability of external triggers as well. So the last thing is, what are the Etsy specs? These are the ones that I was referring to here earlier. So you can review them, Sol1 and Sol6, which are almost mirrors of each other. And just a quick little wrap up in any time for questions or I think we might be out of time because we were just asked to finish. All good?


We mean business on this. Meshima, just aside from today's session, we want to discuss about what is happening to Rift Valley. Earlier last week we lost a world record holder, Kevin Kiptoom. I know you are very saddened about the loss of the legend. I don't know if you have something to comment on that because there has been speculation going around on the course of the day. I know you are in the right position to comment something that will maybe make people see it in a different angle. Very sad and I still condol with a very heavy heart on the loss of that very young man. At the age of 24 he had achieved some of the epic fits that you know very few if any had achieved that level and obviously it tells you he had a greater future. My concern about this is we have had several other notable stars that I have there was one just about a month or two ago who was mugged and he was killed at night. There are those that have also died in mysterious cases. Sometimes you know maybe you know issues to do with social matters. My key call going forward is to understand that when we have talents to the level of Kevin Kiptoom when we have identified such talents as those people become must become state properties to this sense that they must be given protection 24 hour you know protection secondly they must be given drivers people who surround and protect this valuable you know talents because they hold this country on the international stage to lose somebody in the name of Kelvin Kiptoom is the worst tragedy ever and therefore going forward all the stars that have gone through a particular threshold must be given state security full round security and because you realize probably was driving at night tired they must be given state drivers. My name is Honderbo Barnard Kituor Member of Parliament for the great people of Nandi Hills constituency in Nandi County. Today has been a great evening bringing in various organizations and actors who have come probably at the right time to engage with the young parliamentarians in the legislative house especially now when we are entering through the beginning of the budget process Every 15th of February the PFM Act or the Public Finance Management Act demands that budget policy statement must be tabled in parliament which then initiates the process of the budgeting and as well we have to table the the medium term debt management strategy which details more about the debts that the government has been able to get and how what plans do they have in repayment of such debts and I must say that overly we have been sharpened as you've seen several members of parliament are here today why we are young parliamentarians that are determined to make an impact for our country we are elected at such time as this in the recent past we've been hearing people having challenges of issues of understanding matters debt matters repayment the issue of the foreign exchange going so high up sometimes that may be affecting the repayment of our country and you know as young people we will be paying debts of sometimes that which we did not we did not participate in acquisition of such debts therefore our being in parliament and today being being shared for to understand will inform greatly our interventions in parliament even as we begin now the budget making process in the last finance act it was the first finance bill that had the greatest um interactions by kenyans where kenyans was scrutinizing but where kenyans have not been able to access more is the budget making process that were the now that is the beginning of the budget policy statement and therefore we are calling upon kenyans of course now as as representatives of the people and as young parliamentarians that such sessions as this has given us a great hindsight we will not let kenyans down yes we are in the executive of this country but we will want to participate and assist kenyans in making sure that everything that is being budgeted for is being done right of course of course the good thing that we've seen is that the every indicators that they have shown shows that our economy is doing well but there are gaps that now even ourselves you are going to look more like the absorption of the budgets our devising several times revising the budget we then affects the implementation of various programs of government these are the things that we are living here today with that as we go back to our various committees we will be very vigilant to make sure that we do not have several times that budgets are being done over and again and then they are not being implemented or secondly sometimes the over projection of things for instance you can project and say that you are going to get down a funding that sometimes are farfetched such kind of things must now come to an end we will be very careful to make sure that anything that is put in terms of projection must be very realistic implementation of government programs must be done with the two you know as government being the doing the over citing or the parliament doing the over citing and also us as young parliamentarians i can tell kenyans that are watching us today we will be more involved in making sure that we have a better posterity the young people that are out there we will be living for long and therefore the the effects of the decisions or the policy intervention of government now will affect us for a long time we want to assure you as young parliamentarians we will be your uh uh watchdogs we will not let anything that is being done that will negatively affect many of us that are still living for long we will give the right representation in parliament therefore today we have been greatly sharpened just to ask you this because so many kenyans are asking who is accountable for this because we've seen previous members of parliament now leaving the positions uh they're not accountable for the project they've done with the budget what do you propose to be done are there may be a strict close now should they be now be maybe a judge um regarding to what they failed to deliver for the people what i would say is um i think as we go along we must be able to review several laws because for instance we do over citing as parliament and sometimes we give recommendations having when we in in our committees we have both lawyers we have the office of the auditor general who check all the books of government so i think we still need to strengthen the ability of parliament to give tough sanctions which we don't have at the moment so as young people as we now go forth we will be adjusting several laws particularly in the pfm act so that anything any monitor has been appropriated to be done for something and it is not done we must be able to take very tough measures and this is going to be progressive we've started it well you'll be seeing it as we go along and we mean business on this all right mishima just aside from uh well for today's session we want to discuss about but what is happening to risk value you know early earlier last week we lost a world record holder uh kevin kiptoom i know you are very sadden about the loss of uh the legend i don't know if you have something to comment on that because there have been speculation going around on the course of the day i know you are in the right position maybe to comment something that will maybe make people see it in a different very sad and um i still condol with a very heavy heart on the loss of that very young man at the age of 24 he had achieved some of the epic um fits that you know very few if any had achieved that level and obviously it tells you he had a greater future my concern about this is we have had several other notable stars that have there was one just about a month or two ago who was mugged and he was killed at night there are those that have also died in mysterious cases sometimes you know maybe you know issues to do with social matters my key call going forward is to understand that when we have talents to the level of kevin kiptoom we when we have identified such talents as those people become must become state properties to the sense that they must be given protection 24 hour you know protection secondly they must be given drivers people who surround and protect this valuable you know talents because they hold this country on the international stage to lose somebody in the name of kevin kiptoom is the worst tragedy ever and therefore going forward all the stars that have gone through a particular threshold must be given state security full round security and because you realize probably was driving at night tired they must be given state drivers to make sure that we protect such persons as this and this is not a matter that will just say and pass on it must be taken seriously and we'll be calling upon the minister we'll be calling upon the the PS of the same department and even if that they make some regulation they bring it to parliament we'll pass it because it doesn't cost much to be able if you have all these officers who are around here some of them you know you find one one state officer having almost ten officers protecting them why not give a few to protect such kind of talents like those so for me i am of you know equivocal thought process that those those uh at least with talents or any young people with exceptional talent that can place our country in a position that they are seen globally must be protected by government it is no more even a negotiation it is a demand


so now let us do the maximum term method as i saying here that this is one can show that the maximum term method is bounded and a very rigorous bound the bound is that means zero there is a quantity pn which is very small one can show that maximum term method and the remainder you know new and that goes to zero so now i want to take and take the replace the partition function by logarithmic term so what i am going to do now i am going to take the that ml star so i am not going to do this sum now i am going to replace that by one ml star that ml star that particular distribution which maximizes this thing so i don't have this sum anymore so this is the quantity i have to maximize right clear so now we do that so this is the maximum term method so i take this is the mistake i said i don't know how these mistakes come this should not be there anymore because we are just maximizing ml star so now i take the log of that when i take the log of that now i have to have the with the constraint that lml equal to n so log of that you can have all these uh term ml star minus lml ml ml ml sum over ml is n so this quantity is here and this is then vbl to the power m all these things you can do check it and you'll find that apply now maximum term method with the constraint that's why the next time n will become and then you get this quantity it used to be z with the back edge lml minus n n does go out lml remain and then i put it as a ln z the my my the grandian multiplier uh take the rotation ln z knowing that what will come later and then i take with respect to ml and that would this ml lml remember that is a constraint that you have to be n that's why it is there and then you get this quantity from here ln ml ln z then you as i said i put it by hand as a lagrangian multiplier ln z ml star vbl to the power z these come from there everywhere it should have been ml star actually like it should be star here it should be star here uh it should be star here i can try to do that with certain luck and kbt should be lower case these all should be star ml star so this is right here but everywhere it will be ml star the ml star is that so now you understand the details and you also understand the grand picture so n so once i get that i multiply it i know that mass satisfies n i do that hit by l and do it then i get n i get sum over v lbl z to the power l then i get density equal to lbl z to the power l this is a very important very important equation correct right so this is the this is this all right okay now i a partition function like that i take log of that then i can now calculate the free energy free energy in terms of my ml star i know my ml star my ml star is this quantity here right i know my ml star now partition function does not have the sum this is correct then i can calculate the free energy by ml star coming out the ln of that vbl and this term believe me this is all this term is here and then i can get the pressure by taking derivative of that are you following the thread or you are losing okay then right now if you go into the very detail of the equations you will get lost this you can check it up and it will take me one probably half an hour more to do that so i don't want to do that and it's probably since these are details of a keeping at that level of probably these are not details you can do it because it will take you 10 minutes to do it but you can do it so that goes inside and i when i do that there are these two terms here lnz nlnz you will go to 0 ddv but i have this term left vbl term left so i take ddv there vbl so then i get pressure doing that is kbtbl0 so pressure case we will work is p by kbt is the second quantity so these two together are called the cluster integrals the cluster expansion sorry so when you call a mere cluster expansion this is the thing is really a matter of a choice and these are exact one can do probably somewhat simpler way through the done canonical ensemble but there are some results what you won't get from there so these now as i was saying the game plan now is to eliminate if it's cause a fugacity from these expansions both are exact believe me that elimination of fugacity from here gives you another infinite series that infinite series is the following and this is a virial series that is one of the things you taught in your on high school also the pressure is expanded the density first term is a second virial coefficient which will show is exactly given by f2 term of mere cluster expansion f2 term is an integral over interaction in term of potential intermolecular potential so now from the hysterically from b2 the interaction potential the first force field was derived from b2 because it was done after mere did one finds that second virial i can get the second virial coefficient how i plot pressure versus density and this is the ideal gas but if as interactions become important i see this starts deviating so i know experimentally this quantity and i in the first part of deviating is very temperature dependent like high temperature it's like this low temperature is like that so i from there i fitting i get the temperature dependence of second virial coefficient and i'll now show you the second virial coefficient is the integral over the first irreducible cluster integral or b2 second virial okay so now so i'm going a little bit back and forth to give you the understanding of what is going on okay this will not need immediately but very important so what may i did then these bs that we have done are called reducible cluster integrals these are with the definition given here these are integrals however as we know that this cannot be reduced any further but this can be reduced so now we think is it possible for me to express bl following the same logic that i can now say okay i have a cluster we wait i have given as bl that wait now that particles now i can now define this quantity as one it is constructed of two bonds if of them are same bond but this is a different thing so i can i further decompose these group of clusters into these quantity and these quantity given that this is the irreducible and that was the next thing that may have did reduction in terms of irreducible cluster integral now is it okay let me define these are the fundamental quantities these as these i cannot reduce any further this and these are the quantities this actually should have one more because it should have the one without a bond okay that somehow it's not given here so this one now i define this is not kappa this is k k where beta k this volume this quantity exactly define the similar way as we did before we can do the exactly same thing that i now want to distribute in my ml or nl how many of them are these kind and how many are of that kind so we do the exactly same analysis we did before and you get slightly different expression but you can show that it goes over the further decomposition and this however has an approximation in the process of going this approximation that you have to assume that these are volume independent only then these reduction can be done and that turned out to be very important limitation in this theory which made the theory more questionable at high density okay so so i will not spend too much time on this if i want to do it i will do it later but right now i am going to do something else that so this is the irreducible cluster integral so now my aim is to show these quantities the second virial coefficient these quantities the third virial coefficient and some of all these things are the fourth virial coefficient and as i told you these are important historically and also for force field development even today so with time priorities have changed there is that huge amount of effort at one time went to get these integrals really huge amount of work went and that succeeded to some extent but not to a great extent so basic idea is that this is the thing and then there is a reduction you can there is a distinction you should know beta one is two particle so beta one is first of the irreducible integral that is good gives on to b2 so b3 is the three three particle clusters so this has two of these but there is one stands out which is beta 2 so when i think of b3 i b3 is the that's how it was done the way i always do or you can always do you can always i never do it this way i always write down two particle then i do the three particle then i go to four particle then i see the pattern that's how may i did it bs that doesn't come from there that come from these definitions so there is the three part of a will come three of them i think three factorial three will they are all the same value three three factorial cancels the three i think you are left with two that that's work it out but as far i remember one or three factorial into three and then you left at this two and here however nothing is there so it the three comes in because the three three of them are not there that's why you get a three five three here but let me say something very interesting so these part was done by mayor now mayor had a very very smart wife you might be knowing called gopat mayor maria gopat mayor so they were in gottingale and maria gopat mayor was daughter of a professor she was very beautiful and very smart so they are all those people are there george gammo and all the all the people mayor they are they are going to gottingale to study so finally joseph mayor managed to convince her and maria and they brought her to him they are in first in job he called me those days women's faculty position was very difficult so maria gopat mayor didn't get a faculty position she was just i don't think she was probably not even paid but she is but she was incredibly smart so she is so while mayor did the first part this part was done by maria gopat mayor and that is acknowledged in so there was a book of stratmac which is mayor and mayor very famous statistical mechanics books much of the things are taken from that book also the patria r k patria has a good job and my book also i hope the equations are all correct that was done by maria gopat mayor then another interesting thing happened sometime later there's a psst student of mayor called stop mayor who was very famous stop mayor did the theory of soljel transition when the polymers get together and from a gel there is a difficult combinatorics of this kind and that was also done by gopat mayor and uh stock mayor wrote the paper and acknowledged the contribution of gopat mayor in that theory and uh in chicago then they came even then uh gopat mayor didn't have a didn't have any uh office so in the in the uh first floor or ground floor american first floor or ground floor in james frank institute i think the she produced had in the endico farming path which is demolished all these things now except the why the farmer did the nuclear chain reaction that part is only kept and the capsule is there there's a long hallway and in the hallway one side they put a desk and a chair and maria gopat mayor used to sit on the hallway and when it would be so for me so people know his mayor's wife so once for me was going there and maria gopat mayor was doing some calculations so for me stopped and asked what is she doing so i'm trying to develop a theory of nuclear shell theory for me said that's very interesting i was thinking about it in you know it should be something like both theory you know the same kind of thing and that gave maria gopat mayor presumably the basic idea which went on to do in the the famous theory of nuclear shell which they got the Nobel prize uh ultimately so there are all these stories which are uh hangs around in in famous places like this james frank institute and the co-farming university of chicago everywhere else for example like you when you go to harvard you hear the chemistry department lot of lot of beautiful stories about bob woodward who is one of the father of organic chemistry that whatever so now uh let me go back and forth little bit now uh so these are the integrals the same as with the factor of two same as the second of reducible claustrical say is the first reducible particle there is this difference of one between them but this is the integral you have to remember so is f12 and remember f12 is mayor a function so f12 is this quantity e to the power minus beta u12 r minus 1 so then beta 1 dr right and then that is dr 4 pi r square beta ur minus 1 okay so there is a factor of volume in front so this is a strongly temperature dependence that comes from beta 4 pi comes out 0 to infinity 1 pi 4 pi v right yeah 4 pi v so this is the expression of my first reducible claustrical which is also v2 by 2 now i'll show you how that happens and we'll stop after that uh before that one tiny little bit that one can go and calculate the average number of these clusters by using this there is a what turned out it's very powerful that this exact partition function is nothing but a class of polynomials called bell polynomials a very powerful polynomials used the famous mathematician bell that has been used in many ways particularly in study of zeros or polynomials that polynomial allows you to write a recursionization it's just a bit of good luck now if i can write this then if you can give me the z1 and then give me next one i can get the higher level partition function so i can construct this called recursion relation the input of course are the bl plus one so if i have that i can construct this partition function if i can construct the partition function i can calculate what is the average number of clusters in the polynomial by using this as the weight uh this as the weight so this is the weight of your one uh of your one particular distribution right if i have the partition function they can i calculate that weight i'm not going to get the details of that but then you can now say okay lml is the number of clusters lml is the number of particles in size of cluster lml is the number of clusters of size l each of them have l so lml is the number of particles in cluster of size l now you plot it against l by n n is the number of particle l is the size of the cluster then you find when you have low high temperature then this goes to be zero but when you lower the temperature you find a second peak appears that second peaks is the is the is the liquid so a second big cluster appears with l by n close to one rightly less than one may be at point eight so as you hit the coexistence these peak signals the appearance of a macroscopic cluster in the system so liquid is envisaged in mayath's theory as a macroscopic cluster okay so when you cool down like nitrogen we have i cool it down and make the density large then the particles come close to each other they all start interacting i call it system a liquid when the all the particles are interacting each other or one of them that many others but if a one is interacting with say 10 which is a good number actually this correct almost correct number then the neighbors of that my name 10 neighbors they are interacting with another 10 then another 10 so liquid means when the particles are all talking with each other or molecules are talking that means in this description they are part of the same cluster they are talking with each other means they are connected by a bond this is not a chemical bond the bonds are breaking and forming but they are interacting so liquid is that not each particle does need not interact with everybody okay so this is the again i repeat ml is number of particles in cluster of size l ml is the then ml is number of clusters of size l ml is number of particles in that cluster okay and then this is l equal to small one or so one by n equal to very small that are the gas the small clusters when they are large this goes close to a when l is close to n that means a gigantic macroscopic clusters this close to one maybe 0.8 or so so low temperature this peak appears high temperature there is no peak this is a low temperature high temperature i should have just like this so that should have been there and it is not there so this is a nice physical picture that comes as i told you ideal gas doesn't form liquid liquid forms because of interactions and that interaction lowest level is the second or third visual coefficients that is not perfect theory by any means this theory to take you all essentially a picture and how interactions are taken into account but that is the universal scheme all other theories flew from that so if i take now l l ml and then l by n then as she asked in the high temperature gas phase this is just like this i lower the temperature increase the density it becomes like that then finally there appears this is one okay so this is this is the pure gas no liquid large clusters are appearing then at the coexistence you have lots of gas and then also one gigantic cluster so if i plot this thing as pressure versus density then this is like that which is here then it becomes more like that which is this one and then it becomes so this one is this one gas and liquid are together this is the coexistence so you have one gigantic cluster which is the liquid is amazing when your liquid is in a equilibrium with a vapor phase just like in glass of water then in one with the liquid you have this avocado number of particles interact with each other but your liquid your gas is just cluster of two or three this is a huge separation and one is stabilized by entropy which is your gas one is largely stabilized by your enthalpy which is this phase and this really took huge amount of time for people to describe this phase transition huge amount of money i always consider phase transition is where almost all our concepts came including high energy physics because that's where came the order parameter that's where came the interactions around atoms and molecules so phase transition is the one everybody should study the free energy diagram and all these things even our chemical kinetics is deeply more deeply even though independently it was developed in the beginning now one little bit because we i don't know when you will meet first one little bit the virial expansion mayors theory so then i go to the partition function and i say okay i expand this this expansion now i start putting my v is together i take only the this term then we know n into n minus one two of them i will do rest of the integrals i give v to the power n minus two correct there are only two i cannot do which two are kept here they are vector integrals then i change the center of mass to what one volume comes out this is my may or a function actually it should be u however now i take this v out it become v n minus one large n limit i can neglect this i have n square by two and then i bring it here take v n out one by v comes here it becomes one n square beta one by v beta one as a volume term okay and in general so this you can this is the partition function now i take this tiny little bit of partition function only the this part tiny little bit i just looking at a correction that correction so i am i am interested in this part i am interested in this part or this part i am i am not doing this one but i am doing a tiny little part to make it simple a full derivation can be done that will be little complex so i am not doing it so now i am going to do that put that back in my partition function v z n i put it back in my free energy this definition then i log this is a large volume and n v and then i take this derivative and then this becomes one over v that becomes one here and that derivative gives it do this this is suddenly developed but at the end of the day this is n by v by n so p v by n k b t you get exactly from here you get exactly this term so this is the first of an expansion so p v by n k b t is this the virial series so now i identify v two second virial coefficient is minus beta one by two this quantity from minus beta one by two and this is then my microscopic expression of the second virial coefficient except this is in not v and u writing in from of his room streets notes and screwed up everything very nicely but this is what it stands for this is a beautiful expression second virial coefficient which describes this bending here is in terms of mayor cluster integral and this is a general expression one can derive and well little bit i can go to van der Waals equation i can find out my virial coefficients this is a homework problem we always do you expand it pressure in terms of density and equate this thing you get the second virial coefficient third virial coefficient go back find out betas then find out betas this is betas find out your mayor cluster integral which can be done by use a very similar recursion relation so you can construct whole statistical mechanics that lies under van der Waals and that that has been done so we stop here today there are some limitations one of the major limitation is that when one goes to that level this is only temperature dependent you in doing these integrals the way you find the only when volume dependence of those integrals are neglected and that is the so this is the it it does not describe because it diverges there and it does not describe the liquid state so mayor see it does not work in the liquids but it describes how liquid forms it describes how singularity develops and it describes how which singularities are the of it so what we will do now after this we will do land out theory is the theory of phase tensor i am hurrying a little bit any questions you quickly ask yes that's a very serious quantitative change this part is quantitatively exact correct no say for example when you go to liquids then something fundamentally different appears here for that you need infinite number of them and that part where it breaks down but it as i said here it reproduces the complete flattening it reproduces this part but then it does not because for that you know people have done little better and corrected this theory but people are quite happy with this part that it starting from an interaction potential to see how a temperature dependent behavior emerging was really quite satisfying and the concept of clusters we don't want to put percolation this is the way after that percolation theory was done soljel transition done so this played as a catalyst of development of a lot of things that's why historically in statistical mechanics these things should not be ignored mayors theory people don't teach it because it's too complex as you have seen and it so next we'll do don't press transition i'll explain little bit okay i think we'll stop here today thank you


Thank you. Thank you. Nice to see you all here second day morning ready to continue Yeah, so let's talk Gutenberg and collaboration these two things Do we have a presentation already? somewhere Okay, we have to wait But you can watch me at the you know in the meantime. I will maybe start since we have no slides yet Do we know? So Why am I actually talking about this there are two main reasons first of all? I'm a huge fan of Gutenberg the project from the day one that it was created and second of all I was working as a JavaScript technical lead at CK source where I was heavily exposed to Collaboration and collaborative editing in the web browser in the JavaScript. Oh Yeah, we have this So it's me. I'm David the rice came a full stock web developer What was developer for many many many years tech lead? I'm huge fan of Gutenberg react TypeScript JavaScript everything web development related And as I already told you I love Gutenberg and I worked as a JavaScript technical lead To a collaborative editing experience in the web also long-term roadmap for Gutenberg states for phases First is the easier editing, which is the Gutenberg editor itself. Second one is the customization it includes For example block patterns block themes block directory and full-side editing. I had to check this out Second third phase is the collaboration. We will be just touching this a bit today And phase four is my personal personal favorite is a native support for multilingual in WordPress and Gutenberg Phase two is still in progress if you want to check what still needs to be done You can check this issue on github in the Gutenberg repository and kind of unrelated But one of them most favorite features of mine in this phase to Gutenberg is this command center feature Which is basically like a spotlight for Gutenberg and the WordPress so you can add stuff quickly and do stuff Or even in the WordPress admin eventually just by hitting a shortcut and doing what you want So let's first start with basics. So collaboration. What is actually collaboration collaboration is when multiple people are working? together to achieve one goal and What the goal is in context of Gutenberg the goal is to create or produce content So it could look like this in like bigger companies Producing content could look like this We have editing part where multiple multiple people are editing the content and they are Creating it editors then eventually they optionally may pass this to another stage like correction And then maybe compliance to the legal team if we are working on for example terms of service page And then maybe we have someone who can approve the changes and once all of this is done We have ready to publish page so we can tell that all of these people have collaborated to create this publishable version of the page Real-time collaboration on the other hand is just an extension of collaboration It is the same thing, but allows you to do this simultaneously In our case a great Place where we could put real-time collaboration is the editing part So instead of people passing the document to each other so they can apply more changes They can just do it simultaneously, which would be great like Google Docs wouldn't it? Basically, the idea is to just get rid of this of this post-lock model Instead we want to allow everyone in and enjoy editing the content at the same time Still there is no real-time collaboration without collaboration So it would be great to have these flows of collaboration in Gutenberg before going real-time collaboration full Now let's get into some specifics These solutions for collaboration. They usually are network layer agnostic But when you try to implement this in a real product, which Gutenberg is You probably just stick to one of the connection methods So we could use like one of these short or long polling with traditional HTTP requests Similar to what heartbeat API is doing in WordPress We can open a web socket to the server and we can use webRTC to just connect people in a peer-to-peer network Speaking of peer-to-peer should we use server or peer-to-peer network? This illustrates the difference. So in peer-to-peer, obviously we connect everyone Directly to each other. This has an advantage because there is no server in the middle But you might already tell that adding such a Synchronization in such a decentralized network is probably a bit more difficult than normal On the other hand, we have a central server in the middle Which really reduces the problem of synchronization because we have one central server Which is a single source of true, but we now have a server So now we also have to ask other questions like where this server should live Should this live alongside with the website? If yes, it's probably have to be written in PHP Which is trust me not an easy task to create a web webhook server with PHP If not alongside with the web website then when where maybe an external server or Network of geolocalized servers hosted by someone like maybe automatic or WordPress org But then we enter into a data privacy issues because all of the data is flowing through those external servers So we need to probably have some encryption some advantages again some disadvantages If we are working in a system where multiple Machines are connected across the network. It's unavoidable to stumble across concurrency. So let's see this very simple diagram We have two timelines and two nodes node a sends some information to node b and node b sends some information to node a then they receive one and the another and When we look at this from an external actor perspective Us we know the absolute order of all of these events the total order because we see both of the timelines We know that node a I mean the a one happened first then b one happened second a one cream happened third And the last one is be one cream But if we look only at the one node perspective, let's let's look at the node b And let's think only based on the information he has can we figure out the whole order? I just told you if you know probably you know since I'm asking the answer is no and it is no because if we look at This and from the node b perspective, what is the difference between this from for him and this? No node b timeline didn't even moved It's the same for him and this is all the information he has so going this and this Changes nothing to him. So what we are saying that a one and b one these changes are incompatible. So therefore they are concurrent So at this moment, you might say well just use timestamps just send it when this happened and I I don't have time to enter into these timestamps and clocks in general but If if you want to like and get into this stuff ask me about this on the in the QA session But trust me if I'm saying yeah, if you have multiple machines You cannot rely on the physical clock. They are operating with Also, there is this misconception that you can just avoid concurrency altogether by just using a server And this is simply not true Let's see the same exact example we had before and what we just do let's just change the labels Let's say the top one is Jane the editor and the bottom one is the server So as you can see we have still the same situation. We have still have concurrency. The only difference is that it's between the client and the server and If you think about it for a second, it's great because we still again reduced the Concurrency issue to only two actors and You might also think how is it even possible that the server has some concurrent changes to James? Well, it could receive it from another node that is also connected to the server and as a byproduct We also have a single source of truth on the server about the total order of events Because for the server the order is basically whenever I received the event is the order of the operations And this will be important later For the comparison, this is the decentralized network example. We are sending all the information to all the nodes We have only clients. There's no server now But and the sending part by the way, it looks complicated But it's it can be simply handled by WebRTC for example So it's not an issue but the problem is the ordering of these events How do you can ensure that all of the clients all of the nodes have the same content in the end? It's a difficult problem to solve but we'll get to this So handing concurrent editing ideally should have two main properties, which is strong eventual consistency That basically means that at some point in time in the future all of the nodes that are connected They end up with the same content. That's as simple as this The second one is a bit more tricky to implement which is intent preservation So what we are saying is that we when someone edits the content and someone else edits the content When we merge those changes, we don't want to just merge them Whatever we want to have some intent preservation We have two words. We don't want this these letters to interleave for example just creating gibberish We want to have two words one by one Even if they are in the wrong order that's still fine because it's easily fixable But if the letters interleave, there's no intent preservation and it's obviously bad So are there any solutions already? Created by the by the people and they are Two types of solutions are OT and CRDT if you don't know what it is bear with me I will explain this in a moment So operate OT stands for operational transformation and it's like super super super old piece of algorithms ideas Software and all the stuff around it and most notably it's being used by Google Docs for example and CK editor 5 so how does this actually work? Let's see this diagram and let's imagine both of these nodes. They have some content in the editor ABC and D now Node 1 wants to insert X between C and D and node B wants to remove B Basically, so now they end up with a content ABC X and D and on the right a CD now they exchange information what I have done and The first note says insert X at index 3 and the second note says delete B at index 1 now when we received this operation we pass it to a Transformation function this and this function is clever enough to know Oh, the B is still on the index one nothing had changed it So just do nothing with this transformation do not do those operations do not transform it and we end up with the con applying delete B a latex one as is and Result is AC X and D on the other hand. We have the same exact Transformation function, but now it's clever enough again to know. Oh something had changed and just Basically, we have to account for this change in our Operation so we transform it and we say instead of index Insert X on index 3 we now we have to insert X on index 2 which ends up again with AC X and D there are multiple Multiple algorithms created for these OT solutions But that through to be told as you can see the OT is very old because it's even have some stuff from 80s But most of these have been proven to be just straight wrong for some situations and the only two I know are Correct are the highlighted the one who are once highlighted on the slide And the they both use server in the middle and the Jupiter one is used by Google Docs So I already told you that order matters, but what if all operations are? Commutative so what does it mean in mathematics is where it's very simple to explain commutative means No matter what which order you apply the operations That they the result is the same basically so we can shift operations and we have the same result So let's let's see if our operational transformation is commutative imagine We are creating a title for this whole conference and we start with the word come word in the editor I want to insert. Hello at the beginning and you want to insert rules at the end. We actually if we Represent these changes as operations and then exchange them Now let's apply the the first one. Hello insert. Hello at index zero So index zero is here at the beginning we inserted hello word come rules all good And now let's do the opposite one and you probably see where this is going the index 8 is now between o and r Which we and we end up with hello war rules early come which is basically wrong So both of these content diverged and they this is basically like a critical error of the system and they will never get back They will never converge So what we are saying that these two operations are not commutative We cannot apply them in any order because the result will be the Different and that's the main reason why we need these transformation functions in the first place And just to say also about these transformation functions This is the main criticism of OT because those transformation functions are very complex They are full of magic in direction because they they what they do is they transform operations against against another operation So you have often like a chains transform operation then transform operation then transform this operation then transform this operation And we have like very long chains that nobody is able to debug or even understand the code that is Creating these transformations and also to foresee the result of this code, which is pretty bad If they would be commutative it would be more like this we start with the same state an empty document We have three operations. We shuffle them doesn't murder the order and then we end up with the same result If they also are idempotent, which is a big word, but it's pretty simple It means that no matter how many times we apply the operation we end up with the same Result so multiplying by one is idempotent operation because we can multiply by 100,000 times and we have the same result In our operational transformation world or CRDT's If we have operations like this and we somehow Receive the same operation twice exact the same We only apply it once and we have one paragraph with text. Hello in this example So if we would have a system where we have operations that are commutative both commutative and idempotent We have a very simple Way to Synchronization because think of this if I can send anything in any order to anyone and they receive it And we don't even care if they receive that multiple times It would be pretty simple. Yeah, I mean Here we enter CRDT's which stands for conflict free replicated data types also known as commutative Replicated data types or convergent replicated data types. This one is the most popular and CRDT's are the set of data structures and also algorithms and again stuff around this to Basically enable this commutative idempotent everything is going you know merging itself hence conflict free Some advantages of this are for example, you can create a software that is ready for offline editing AK local first apps, so you can have two different editors that are completely offline We either changes you the changes and then once we are online somehow it magically merges It supers both operation based and state-based things So we can both use the normal operations as I showed you in the operational transformation Or we can just send the whole state of the application and they are be will be getting merged This is obviously more expensive the payload wise But it's also an option and there is no need for the server because we don't care about the order of operation So we can send it to anyone at will some of the Solutions and algorithms for CRDT's are listed here This is these are the ones that I know and the I'm not going to go into details about this But the wood the first one is pretty funny because it stands there This abbreviation stands for without operational transformation So you can see that this has been created just to kill the operational transformation And it's like a new and more fancy way of doing collaboration these days So how actually these CRDT's work because you might think it's kind of difficult like I sent any order any Duplications and it's still somehow work. So how actually does this work? I? Will give you an example of one of the algorithms There are multiple ways of doing this But this one is the simplest one to explain so I will get this one in CRDT's if you have again the same content ABC and D We instead of working and operating on indexes, which makes shift we operate on IDs on unique IDs So in this case, I made these IDs one two three and four So now well if I want to insert X between C and D well, there's no ID there because it's three and four What do you do? You just create a new ID in the middle 3.5 and now you might think okay But if two people want to insert between C and D they both Have ID 3.5 which is bad. We don't want this and yes You you would be true because it's not that simple. I will explain this on this example. Let's say we have a Collaborative session between John and Jane and John have created some content first ABC and D and now you can see instead of just Adding ideas and number what we are doing is we have an array of Topal tuple tuple is basically to value our eye. Let's say to value Data structure so we now have ID of like index or ID of this element and also an ID of who actually created this So in this case, I'm using emojis because it's shorter, but we know that one Index one created. I mean ID one created by John ID to created by John three created by John and for created by John And now if Jane wants to insert something between C and D She's not longer need to shift and start do stuff with indexes. What she does is she takes whatever was before Three by John and now she appends it and she says this is three by John followed by one of me by me And now let's go even deeper like like now we're removing be the same situation We had before removing be and now John was at the same time insert X Y and Z between X and D So what we do? We just right remove the two by John and the John inserts After X so it takes whatever was there and it adds Y and Z adding one and two by him at the end So this way we have like conflict free merging of the letters There's also one and by the way these These fraction numbers I just showed like 3.5. They are actually Created by taking all of these numbers in the ID So it's 3.1 point 1 for the white white letter and therefore we can order all of this based on these values There's also one edge case for this. Let's imagine. I Removed C as a John and I added D also as a John So we would have again the same situation But different letter and this would be bad so what we are doing we are just using something called logical clock or or just counter basically and Every time we do any operation we bump the counter Which is kind of like a timestamp for us and the whole thing for the whole row is Actually the ID the whole row is the ID of the letter The two main known Libraries or software that is actively using in real prod projects CRDT is out on birch and yjs and Now if you can just Google this and find this on github now, let's go. We talked basics. We talked how Collaboration actually works now. Let's talk Gutenberg if we look at this issue. It's actually from 2017 so it's six years ago already and people were discussing collaboration and if you Look closely here. We also have a mention of operational transforms It's also sometimes called like this and CRDT's and they also say P for peer-to-peer by the way It's six years ago already And it's by Chris blower From automatic he's still working there legend then in Around the same time a Guy called Abyshek Gallot back then working on automatic He created this poor proof of concept using web RTC and then it was taken also by Gregor Zhukovsky from automatic and put into the WordPress Repository to to work this issues and it was at some point it was closed, but it was nice proof of concept Then the guy who created yjs, which his name is Kevin yuns He actually created took yjs, which is exactly what they do They take yjs and put it into all of the editors with the weak editors and he took took yjs and said okay If I can add this to any editor I can also add this to Gutenberg and he created this poor proof of concept and if you are Paying attention. There's a link here And it's life it's still life it's 2019 but it's still life you can go there and you can check collaboration in Gutenberg right now if you want in 2020 this is the latest take on Collaboration as you can see it's something is going on although we don't have the phase 3 yet This is the latest take it's created by Enrique Piqueiras who back then worked at automatic now He works at Google, but I Took this gift by the way, excuse me the Quality because it was just downloaded from the comment on the pull request, but you can see how does this work on? This pull request and it's you know kind of impressing impressive you type something other person sees this you can comment on stuff Etc. And this is using also yjs So it's using CRDT is kind of superseded the previous pull request. I told you from the yjs creator There's also this as blocks calm created by Riyad Benguela who also works at automatic and this is not embedded in wordpress But you can he took just just the Gutenberg and created this a nice piece of software when you can create a session You can share it you can visit the link in another browser and suddenly you have collaborative Editing enabled in both of these Gutenberg editors Until now I was talking only about Like text editing mostly because it was simple enough to explain But it's nice to know that CRDT's also can handle like this all of the different types of data For example, this is showing the offline capabilities on a to-do app Which is probably is just a JSON under the hood. It's just sharing the state and merging it somehow auto magically After the days, you know change the toggle to online from offline, which is super cool so Exciting times to get ahead of us amazing times ahead of us. I truly believe that adding collaboration Real-time collaboration to Gutenberg is going to be the biggest milestone that WordPress Had to face and will actually achieve since the creation of Gutenberg itself. Oh You got a problem But the next slide is thank you. Yeah, something popped up here, but the next slide is thank you. So that's it for me Thank you, David. That was an awesome talk. Thank you If anybody has questions again, we have the mics in the aisles if you want to ask a question Just please proceed to either of the two sides This was very exciting to watch especially the last gif Showing how it could work. I can't wait Yeah, I can't wait to I mean it would be awesome. Yes. Yes. I think we have a question over there Hello. Thank you for a talk. I Have a question. How will this influence? Database and also revisions. Oh, yeah. Yeah. Yes. So it's a good point. It's a good point So it has to all I probably know what you're talking about Yeah, it obviously has to be integrated somehow with revision history For and what was the first one? Can you can you can you? Oh, yeah? Yeah? I mean, I don't I don't know answer to this to be honest. I'm not working at automatic I I did my due diligence and I spoke to people working at automatic. Thank you. By the way, thank you Adam Chinese key But I don't know and I don't have answers to like all of this because I'm not like Very close to this development cycle of Gutenberg But I can check this one and I know there's like a revision history feature and all of the other features like suggestions like Content locking so it should also work with like locking the templates and everything so you Some people can edit some stuff some people can't so this also should be aware of all of this Which is not an easy task. That's why we only have proof of concepts We don't have the real real thing yet But you know, hopefully at some point in the future that we will get this also I know from the inside that to be absolutely honest It's I'm not representing automatic But I know that the real-time collaboration is not like super main priority They they off icing flows for content teams is kind of like more important thing for automatic for the Collaboration part because they don't specify real-time collaboration. They are working on it looking at it but the async collaboration is the Is the the main priority because you now the the bigger bigger content teams They are using external tools they create content there and they just move it to the WordPress in the end Which is not ideal So yeah, any more questions from the audience? Looks like okay. Thank you very much David. That was amazing. Please everybody give it up for David