Residents in Brainerd expressed concern over the safety after the recent appearance of a sinkhole. The hole was located next to and underneath the railroad tracks east of 11th Street between Norwood and Laurel Streets. Burlington Northern Santa Fe owns that area of track where the sinkhole appeared. BNSF tells us that the track has been decommissioned and the area of the sinkhole was filled and repaired yesterday. If you enjoyed this segment of Lakeland News, please consider making a tax-deductible contribution to Lakeland PBS.


Can you hear me in the back? Okay, all right, so let's get started. We'll have more chance getting the USB and then download the file you need later on. But let's start with the actual session. So welcome to the OpenStack tacker hands-on session. My name is Kai Yukien. I'm a senior cloud deployment lead with Ericsson cloud platforms. Hi, everyone, my name is Anthony Lin. I work in the same department as Kai Yuk. I'm also a senior cloud deployment lead at Ericsson cloud platforms. Okay, so for today's session, we're going to begin with a high-level overview of the workflow of VNF creation using tacker. We have some objectives that we need to achieve by the end of the session. And also, we will have a little bit more time spent on VirtualBox and DevStack tacker installation, which is the USB drive that we're passing around right now. If you don't have it, we'll get a chance later. We will spend time on that as well. And once we have our DevStack with the tacker environment running, we can begin our hands-on activities. And we'll have a list of tasks that we can go through. Here's a high level of workflow for tacker when we want to create a VNF. So first, we have to create the VNFD catalogs. These catalogs are based on Tosca templates. And once they are created, they reside in the tacker database. And when we initiate the VNF create, either from Verizon or CLI, we will access the VNFDs, which depends on which VNF you want to create. For example, we have some Serous, some Open Twin, also some Open WRT VNFDs in our example. And then tacker will convert the VNFD catalogs into heat template. And then heat template will utilize the underlying OpenStack components to create the VNF VMs in the virtual infrastructure. And that's basically how we will create our VNFs. And for today's the objectives, we will learn how to create the VNF catalogs. We will go through basic lifecycle management. We will create, we will delete VNFs. We will also see the health monitoring and auto healing capabilities based on the monitoring policies we define in our VNF catalogs. And finally, we will do the VNF configuration update as well. OK, as I mentioned earlier, we have a USB stick that we're passing around. It contains a DevStack environment on virtual box. Depending on what PC you have, we have a Windows version. We have a Mac version. And we also have a virtual box executables about binaries if you don't have a virtual box installed already. And we also have a virtual box and DevStack VN installation guide. It's a step-by-step procedure on how to bring up your DevStack environment. And we also have our own Wi-Fi access point to download the lab manuals. In case you don't get the USB, you can go to this link to download the hands-on lab manuals where we have all the steps to complete all the tasks for this session. And of course, this will be very useful. Later on, if you want to try it, you can spend some time on this as well. And we have put the TOSCAL template that we will be using today on this HTTP server as well if you guys need it. We did not put the image there because it's pretty big. If everyone used the Wi-Fi access point at the same time, it's most probably going to jam it. So anyone else that doesn't have the USB stick have not copied? There seems to be all the guys behind. So who has finished? If you finish with your USB, you can pass it probably to the back. We'll give you some more time. If you want to write this down whether you have the USB or not, this is the SSID. And then, I don't know, in the back, if you cannot see it well, it's SSID is tacker-lab. And then the security key is tacker123. My colleague, we did not enroll. And then the IP address. Do you have a slot for us? For the manuals and the templates are <IP_ADDRESS>. Column 8000. So whoever has the USB stick, it depends on your OS. You just copy either Mac or Windows. For those with Ubuntu OS, we will probably open up our private cloud. And you can try and connect to it. But you can only use Horizon for that cloud. We are not going to give you CLI access. So that's the only limitation. OK. So we'll have five more minutes. Five more minutes. Yeah, let's just make sure you have as many people have their own DevStack environment up and running before we begin. So for those who cannot use the Mac and Windows, you can actually connect to our SSID, which is here, and try and connect to it and see whether it works. So this is the one. And the password is tacker123. So we will join the network. Yes. Oh, password? OK. Tacker123. All small letters. You want to type it and maybe so for people to see the actual password. Actually having just put on like an iPad or something.  So this is the password. If you're not able to connect, give it a try maybe a couple more times. Yeah, it might be because a lot of people are assessing it at the same time. That's why we prepare for it. Yeah, which is why we prepare the USB stick. Yes. It's inside the USB stick that you managed to copy it earlier on. It's inside the virtual box folder. Yes, you're back. It's not connecting. It's not connecting. Anybody? I'll pull the plug and try again. Yeah. Tacker123. Because when we tested earlier, it was working for us. It works. Yeah, it should work. I mean, we were just hoping that. OK, again, if you have finished copying the USB, you can pass it around so that other people can use it. OK, so I power-cycled. So once you can connect to the Wi-Fi, you should be able to see the 192.161.21, which is our own cloud for those who cannot access using the, I mean, let's say if you cannot install on time. If you want to try, you can use this. But I mean, I hope that most people can use the virtual box itself, because if too many people use the access point, it might be a problem. The IP address is <IP_ADDRESS>. Let me see. How do I show this bigger? So <IP_ADDRESS>, port 80. Is it better? HTTP. Yeah, it's HTTP. So it's port 80. So this is the IP. Are you guys able to connect to the access point? I mean, yes? The username password, it's enter. Yeah, so this is the dashboard IP. The password is the username is nfv underscore user. USB for Windows, yes? Yes, there is. So the USB drive contains Windows and Mac version. Anyone else that's holding onto a USB stick? I think you should pull up the guide. Let's pull up the guide. Oh, yes, the guide. At the end, we have all the access information. So it doesn't, but anyway, this is the login and password. Yeah, this one with the local host. If you have your own running open stack, DevStack environment, then that's how you access your horizon. Yes, what's the question? I'm sorry? Zoom in? So this is going to be <IP_ADDRESS>. The login is nfv underscore user, the password is DevStack. So this is only for people who cannot set up a virtual box on their own laptop. I hope most people are able to go in. OK, how many of you have your DevStack running? Most people have it running. I mean, we can go through the instructions again just to show you how to bring it up. So I think we are half an hour past the starting time. Maybe we'll start now. OK, so basically, for those with virtual box, you should restore your image. And after that, do not select Create Snapshot. You should start running after that. Once you start running, you're going to see, so let me just exit out of this. And we'll go in using port 2220. So we'll go into the local host using port 2220. So the password is DevStack. It's same as the username. So you should be able to log into your VM by SSH from here. So we did include the instructions as well in the lab manual that we provided. So let me just minimize this. Yes? Yes, please restore from the snapshot. Don't just power on because then the OpenStack service will have some problems. So just restore the snapshot, and I'm sorry? Right, after you restore the snapshot, you can start it. The password is DevStack, same as the username. So password username is the same. We are using port 2220 for SSH port forwarding. Everyone managed to get in. OK, so once you're inside the VM, you're going to see a few folders. So the first folder is, of course, the one that we used to build the DevStack. And our RC file is here. So you will source the RC file. And then you go here into the tackle folder, which has all the torscar template inside. So we will start with, I mean, if you look at your lab manual, it's going to be the first thing that we're going to do is we'll do some sanity tests like things like novel list, it should work. And of course, you can do your Glance Image List. So in this lab, we have two images that we're going to use. The OpenWRT is an open source firewall router that we're going to demonstrate in this session. And then we have the Ubuntu image, which is a Apache HTTP server that we created for showing monitoring as well. So this is our basic sanity checks. We know that it's working. So the next thing that we want to do, according to the lab, is that we're going to start with the first task, which is to create a VNF with multiple zeros image. So we do have a couple of things, I mean, all the different template here. So we can do this based on what we have here. We're going to use the VNFD, our descriptor create command to create our VNFD. So we'll have to give it a name. We'll have to tell it which file we are actually using. And then we'll hit enter. So it will create a VNFD after you issue this command. So I can show you guys on the horizon dashboard how it's going to look like. So on the horizon dashboard for this particular VM, we are going to use port 2221 on your local host. And it's the same login as earlier, NFV underscore user followed by DevStack. You should go into your virtual box horizon grid. So over here, you have different tabs. So you do have a NFV tab over in the dashboard. So after running the VNFD command, which we execute from the CLI, you will see this being created. If you click on this, you can actually see the content of the VNFD that was created earlier on. So you have things like description. Basically, you give it a description on what kind of template you're creating, the name of the template. And over here, we are saying that we will use the zeros image. We have M1 tiny flavor created. The availability zone is, can you guys see it? It might be a little bit too small, but it's okay. Okay, so you have your management driver. So we do have three of them. As in three kinds of things that we expect, we have management, we have infra, and we have monitoring and tackle. So then you have your network interfaces over here. Your virtual link, one, two, and three. And this is your VDU virtualization deployment unit. And this is your VDU tree. So within this template, it's going to create a VNF which will spin up three zeros VM. So of course, this is a demo. You can definitely use it for some other VMs that you have. And the next thing that we can do, either we can use the command line, which is pretty straightforward. So we'll do like multi zeros VNF, D, VNF. And then we'll select this one. And we'll just click on deploy. You can either use the horizon or you can use whatever that we have inside the lab manual, which is basically a command line. So when you hit deploy VNF, it's going to use that VNF D template to create the VNF. You see that it's pending create under the VNF manager tab. So we just have to wait for a while for the VNF to come up. It depends on the computer speed, of course. So it will show up under the instances. So basically using heat, it will trigger the creation of the virtual machines. So in this case, we'll get three zeros using that template. It all becomes active. Let's look at the console just to make sure that everything is working. So we will, in this case, it's going to be port forward on localhost 6080, which is the default that we have. So you can see that the zeros VNF did come up. Just show that we can, you know, that's the same thing we can log in using the default password, which is here. This is a default password. And just do a EVE config. You can see that it gets a DHCP IP on the management interface. And of course, from here, if you do tackle VNF list, you will see your VNF being created. And you can do things like just ping it. The one that we were looking at was dot three. It pings. Then of course, I think inside the lab manual, I was also, we were also trying to show that you can do like SSH into this guy. Right, so it goes in as well. So from the Dev Stack, you have full accessibility to the VM via the management interface. So, and this is like the first use case that the first lab exercise that we're gonna show. So it's a straightforward one. It basically shows that you can create multiple VMs with the VNF kind of functionalities. So after this, we're gonna create the second one. So just make sure that you clean up the system because we are running on very limited resource. So we will just use the horizon GUI to terminate the VNF. Yeah, so make sure you terminate the VMs using the VNF manager so that it will be automatically deleting all the three instances that you have just created. Right, so it will go and clean up everything. So when you go back to instances, no more. Everything is gone. You can check from CLI, it's gonna be the same thing. It's gonna return an empty set. So I'll show you guys how to do the VNF catalog from the horizon GUI. You can do onboard VNF, select the file, which we did put on the HTTP server. So you can download from there. So the next thing that we want to do, we'll do a simple use case of monitoring the CROSS VM. So just select the YAML and I'm just gonna give it a name called CROSS Monitor and onboard. So you will look at the Tosca template and now we have all this different content. So this is a bit different from earlier on. We include the monitoring policy. Uses ping ICMP to monitor the VM. If it is not able to reach it within the timeout limit, it's gonna respawn the VM to recreate it when it dies basically. So the rest of it is pretty similar. We will again use this to deploy a VNF. We'll just call it CROSS Monitor. We'll select the monitor template and then we'll just deploy. So it will come up. Okay. So now that it's active, we will go again, go back to here, click here, go to the console. So it's up now, right? So it is assigned a DHCP IP of dot six. So just from here, just make sure that we can ping this. Basically it's monitoring via the ping. So the next thing that we want to do is from here shut down the interface to simulate a loss of ping. So after the monitoring times out, can I see it? So it gets, we're just waiting for, you see the background server is running. You can see that the hot template was actually triggered again after it lost the ping and this thing goes down. So next time when you check the instance, you will see something called a respawn one dash one, which is the first time when it respawn. It's gonna show that it comes up. So again, we'll check that indeed it comes up. So you can see that the host name has changed. It has actually come up. So this is a simple example of how tackle can be used to monitor VNF, VM. So we'll clean up the system again. So the next thing that we want to show is that you can actually pass a configuration file into the VM before you spin it. So we'll use the data. So I'm just going to call it user data and then I'm going to select the file. It's this one, user data config monitor. That's just a file that we use. So a lot of all these files are found inside the open cycle website. So we took the samples and we kind of changed it and just adapt for our usage. So just take a look at what's inside here. It's passing the user data that we want to set up the ETC host file of the CROS VM with this particular use case. So in a lot of cases, sometimes when you create different virtual machines, you kind of want to set up things like if config, your network interfaces file, your ETC host, your DNS, for instance. So in this case, what it does is it just captured the if zero IP address, check what's the host name, and then after that, basically we just use the SED command to pass it into the ETC host. So you can actually see it when it comes up. So we will show that it does that. So it becomes active. So this VM comes up. If you log in and you do a cat on the ETC host, you will see that the if zero IP, sorry, it's been passed in the host name. It's set up according to the expected results. So of course you can do different things. I mean, you can think of something else for your VMs. You could actually do other stuff. This is just a batch script that we are actually passing information inside. So this thing can be done depending on the needs. So anyone has any question? Yeah, go ahead. So if you are, yes, it's actually, one dot. Yeah, so the HTTP server is 20. Is it connected? No, but. It should be inside. Yeah, it's under the attacker folder. Correct. You have your virtual box VM DevStack, right? It's in the home directory and then inside the attacker folder. One dot 20, it's not responding. This is one dot 21. So yeah, let's say if you cannot see it, it's actually here. It's inside the VM itself as well, 246. Yeah, it's all here. I think I might need to power cycle this guy again. Sorry for that. Yes, you have to use our local host. 127.001 using port 2220. After that, did you restore the snapshot or did you actually start it just like this? You kind of need to restore the snapshot to make sure that the VDAT, okay, let me show you. So you need to do something like this. Right, okay. So this is your Mac. You kind of need to, for instance, like this, you need to do this. You need to click on this, restore, and do not select this and then do restore. Okay, I'll show it again. So suppose this is your VM because mine is running. I don't want to shut it down. So you will see the base image and you will see something with the current state. Don't actually start with the current state because with the current state, all the services will go down. So the way that we actually do this is that we use snapshot of a working environment so that when you restore the snapshot, you have all the open stack services running properly. So then the things that you need to do is you need to use click on this button, right? Do a, you select the first one, click on restore, uncheck this box, and then you hit restore. And then after that, it will show you as a safe state. Yeah, safe. And basically you just have to click here. And you will start to restore your VM. After that, all the open stack services should be running, right? So are you guys, is it okay? Yeah. I don't see any snapshots. You don't see any snapshots, so, okay. Let me go. Start with the Windows VM. And there are no snapshots when I go to. What's going on with this? Oh, okay. Did you restore? Yeah, so someone had a question earlier on how to get the YAML file. It's over here. So it's 1.20 if you're connected on the Wi-Fi. So I think the server hung. Maybe we can try again. Because I'm able to access the open stack in that cloud as well, so should be fine. So these are all the PDF and the guide and all the YAML that we have. You guys having another problem? Yeah. Can I have one more? Yeah. Sorry, guys. Yeah, maybe I'll just carry on with the last three task cases that we're going to show. So the next thing that we're going to do is we will show that you can do HTTP Apache Server Monitoring. So basically it's using command line. We will do a VNF create of the... So the important thing is to take note of the ID over here. We basically use the ID of the VNFD file to do the creation. So then we will go back to the dashboard and take a look at what we have over here. It might have time out. So I am creating Ubuntu Server that it's a HTTP server as well. So it takes a while to spawn because this is much bigger than the zeros. And depending on your machine, it might take like five minutes just for it to come up. It's actually caching the image onto clients. So it takes a while. The first one, the first time. So we just wait for the VM to come up. So in the interest of time, I'll just do another one for... We're just waiting for now. So this VM is taking a while. I'll just show you guys the one for updating the firewall configurations on the OpenWRC. So if you look at the firewall config, we are taking away the last two blocks and updating some of the configurations here in this virtual machine. So we will look at what we have here. We will use the vnf update command and pass in the configuration file. So one of the things that I want to... You guys notice that we have this here. So after we update, this block will be gone. So tackle can also be used for monitoring the vnf. It's called open firewall. Maybe I did it wrongly. Sorry. Let me just try again. Or it's been another one. I think because I passed the wrong thing. I probably forgot to wait for the SSH to come up. That's why. One of the things that I have to wait for is for the SSH to be ready before I can run the config. And that's why it's not working. We have to wait for a while for the SSH to come up. SSH is up. So we'll go to the config file and then we'll look at what's inside the firewall. We'll remove these two blocks with the command. Ah, okay. Yeah, here it is. I'm too sure why. Oh, okay, okay, I know. I gave a wrong command, sorry. So I know I'm jumping around, sorry. So while we wait for that to come up, I'll show you about how we actually can monitor our purchase service from here. So we will shut off the service and it will actually respond from here. So it's just to show that you can monitor port 8,000 on the UPon2VM, which is our purchase server. And the last thing that we want to try is, again, we'll still go and try on this guy. So the correct command is with the file. And this time around, I'm calling it one. Okay, it updated the VNF this time around. So when you come back to the VNF, you will notice that the last two blocks is actually removed now. So the idea of this is that other than monitoring, you can also do things like management of the VNF, provided you have the correct management driver. In this case, it can manage a firewall VM. So that's what it is doing. So I think we have one and a half hours, so it is about the end. Do you guys have any questions? I mean, we have one last task that can be followed from the PDF. Yes. Yeah. Yeah, we can take questions. Sorry. Yeah. Yeah. Yes. How do I implement the firewall? Oh, the driver. So it's here. Right? So if you go into your DevStack, OPT stack, tacker, tacker, VNF, I mean VM, management driver, you will actually see the management driver that's available. In this case, you have the NOOP and OpenWRT. So it comes before with the DevStack. Of course, if you want to manage other VNF, then what you have to do is to write a management driver for that VNF. So that's how it's able to manage the VNF itself, the VM and the firewall, for instance. Yeah. So it's inside here. Right? Right? It's basically a Python file that's inside here that can be done. Yes?  Right, right, right. Okay. Yeah, sure. After everything. I'll give it to you.


Welcome. I welcome you all to this lecture in the course, Samasa in Panini and Grammar 2. As is our practice, we begin our lecture with the recitation of the Mangala Charana. Vishpesham Satchidanandam, Vandeham Yokhilan Jagat, Charikarthi Bari Bharti, Sanjari Harthi Leelaya. Vishpesham Satchidanandam, Vandeham Yokhilan Jagat, Charikarthi Bari Bharti, Sanjari Harthi Leelaya. In this course, we are focused on three types of important samasas in Sanskrit, namely, the Avyai Bhava Samasa, Bhavrihi Samasa and the Dvandva Samasa. We have already studied Avyai Bhava Samasa in detail. Together with the Samasa Vidhyayaka Sutras, we have also studied the Samasanta Pratyaya Vidhyayaka Sutras. And then we proceeded to study the Bhavrihi Samasa. We have in the previous lecture studied the general features of the Bhavrihi Samasa, a very important type of samasa in Sanskrit. The features of the Bhavrihi Samasa can be explained briefly in the form of an equation presented on this particular slide where you have X and Y, two independent units in terms of the word form as well as the meaning as well as the accent. There is also a plus sign in between the two which indicates that X and Y are semantically related. The speaker of Sanskrit decides to merge X and Y together and form an output in the form of one unit, namely, XY. So, input is X and Y, two units and output is XY, just one unit, one unit in terms of the word form as well as the meaning as well as the accent. So, the three features of the compound process in general are As far as the Bhavrihi Samasa is concerned, both X and Y, they do not assume the role of the head in the newly generated output in the form of XY. So, none of them is marked with the bold characters as we did so in the Tathpurusha Samasa as well as in the Avyai Bhava Samasa. In the Tathpurusha Samasa, out of XY, Y was always shown with the bold characters because Y acts as the head in the Tathpurusha Samasa. In the Avyai Bhava Samasa, amongst XY, X was marked with the bold characters to show that X acts as the head of XY in the Avyai Bhava Samasa. But in Bhavrihi Samasa, neither X nor Y acts as the head, rather something independent of them, namely the Z which is outside of the compound acts as the head. This is how the tradition sums it up. And this is why this Bhavrihi compound attains a supreme status, a unique status as far as the compounds are concerned because both the constituents do not become heads, they become subordinate and the other meaning which is out of the compound that becomes the head. In general, we have also said that the Bhavrihi compound becomes a qualification, a Viseshana and then its forms are determined in accordance with the forms of the Qualificand that it qualifies. Now the Bhavrihi Samasa is dealt with in the Ashtadhyayi at various places. For example, in 2.1 and 2.2 in the Ashtadhyayi, the Samasa Vidhyayaka Sutras are stated in general. For Bhavvrihi to be specific, the Samasa Vidhyayaka Sutras begin with 2.2.23, Shesho Bhavvrihi and it goes up to 2.2.28, Tena Saheti Tulya Yogi. This is a small section of sutras in which rules are laid down for the processing of the Bhavvrihi Samasa. Incidentally, 2.2.29 is the sutra prescribing the Dvandva Samasa, Charthe Dvandvaha. Now the Samasanta Pratyayavidhyayaka Sutras in the Ashtadhyayi are stated in 5.4. For Bhavvrihi Samasa, the section that begins with 5.4.113 and goes up to the end of 5.4, that is 5.4.160 is a section in which we find sutras which prescribes the suffix to be added at the end of this Samasa. Samasanta Pratyayavidhyayaka Sutra. We also noted that, in this particular section, a subsection is such that there is no Pratyayya prescribed, rather there is a substitution prescribed. Then Swaravidhyayaka Sutras, they are primarily stated in 6.2, 6.2.1 says Bhavvrihauprakritya Purvapada, the Purvapada retains its own accent. And then from 6.2.106 up to 120, as well as 6.2.162 up to 177, these are also the sections dealing with the accent on the Bhavvrihi Samasa. This is how Panini deals with the Bhavvrihi Samasa in his Ashtadhyay at different places. Let us now study the Samasa-Vidhyayaka Sutras with respect to the Bhavvrihi Samasa. The first one is 2.2.23 Shesho Bhavvrihi. There are two padas in this Samasa. There are two padas in this Sutra, Shesah, which is Prathama Ekabachana 1.1, 1 slash 1, which means remaining, and Bhavvrihi, which is Prathama Ekabachana also, 1 slash 1. What this simply means is that remaining compound is called Bhavvrihi. Remaining compound is called Bhavvrihi. And these sutras stating the Bhavvrihi compound are from 2 to 24 onwards up to 2 to 28. So let us revisit the question, what is Shesha? Shesha we saw means remaining or what remains. So the next question is what remains? What does remain from what is already stated? Uktadanyaha, Shesaha, something that is previously stated and something then remains. So in order that something remains, you need to have some previous statement. What is the previous statement with Bhavvrihi's Samasa? What is the meaning of Shesaha in this context? So before this particular Sutra, in 2.1 and 2.2, the Samasa Vidhayaka Sutras are stated. The compound presrarakipa describing sutras are stated. Remaining sutras, the other sutras, they are called Sheshas. Now in 2.1, we have sutras prescribing the Avyai Bhava Samasa starting from 2.1.5, Avyai Bhava up to 2.1.21, Anya Badar Thecha, Saudhna Yama. And after that the Tattpurusha Samasa is also prescribed from 2.1.22 onwards up to 2.1.72, the last sutra of the 2.1. In this section is stated the Vibhakti Tattpurusha Samasa, the Dvitiya, Tritiya, Chaturthi, Panchami, Saptami Tattpurusha Samasa. And also the Samanadi Karana Tattpurusha Samasa or also known as the Karma Dharaya Samasa, Tattpurusha Samanadi Karana, Karma Dharaya. This is stated in this section from 2.1.22 up to 2.1.72. When we go to 2.2, we have sutras prescribing the Tattpurusha Samasa from 2.2.1 up to 2.2.22, including the Ekadeshi Tattpurusha Samasa at the beginning of 2.2, as well as Gatshasti Tattpurusha Samasa, stated by the Sutra Shasti, as well as Gati Tattpurusha Samasa, stated by the Sutra Kugatip Pradayaha. Then we have Praditattpurusha Samasa, also stated by Kugatip Pradayaha and Upapada Tattpurusha Samasa, stated by the Sutra Upapadam Ating. So, what is not stated so far is what is Shesha that comes out as the immediate reaction of what is Shesha in the context of what we saw just now. Secondly, what is Shesha and we said what remains? What remains is Prathamantham Ittyarthaha. What it means is that two interrelated subanthas ending in Prathama, they are not yet stated to be compounded, so that is what is Shesha. In that sense, the Bahubrihi Samasa takes place. After having studied Shesha-Bahubrihi and the meaning of the word Shesha therein, let us now proceed further to study the next compound prescribing Sutra, Samasa Vidhayaka Sutra, Anekam Anyapadarthay, which is 2.2.24. Now there are two padas in the sutra, Anekam which is Prathama Ekavachana 1-1, which means more than one. Now this word Anekam is mentioned in the Prathama Vibhakti, therefore it becomes Upasarjanam, Prathama Nerjitam Samasa Upasarjanam. And since there are more than one padas which are mentioned in Prathama, both of them will fight for occupying the initial position of the Samasa. Now the next pada in the sutra is Anyapadarthay, which is Saptami Ekavachana 7-1, which means in the sense of the Adarpadha, Adarpadha out of the compound. That is different than the constituents of the Samasa. The words continued are Sup from Subha Mantriti Parangavatsvare 2.1.2, Sahasupa from 2.1.4, Sanasa from 2.1.3, Samarthapada Vidhi from 2.1.1. And after having collected all these together, we get the following meaning. More than one interrelated Subhantas ending in the first triplet namely Prathamaantam, in the sense of the meaning of the other or outer word, get compounded and the resultant compound is called Bahuprihi. I repeat, more than one interrelated Subhantas ending in the first triplet Prathamaantam in the sense of the meaning of the outer word, get compounded and the resultant compound is called Bahuprihi. I repeat, more than one interrelated Subhantas ending in the first triplet or Prathamaantam in the sense of the meaning of the other or outer word, gets compounded and the resultant compound is called Bahuprihi. This is the meaning of 2.2.24. The interrelation between the constituent Subhantas in a Bahuprihi compound is called coreferentiality or Samanadhi Karanya, coreferentiality. The interrelation between the constituent Subhantas and the outer head word is that of the meaning of the Vibhaktis except the meaning of the Prathama Vibhakti. Prathama Arthetu Navhavati, that is the traditional statement. So, here is a list of Vibhaktis which mean the following. Dhritiya Vibhakti denotes Karman when not expressed by the verbal suffix. Dhritiya Vibhakti means Karthu or Karana when not already expressed by the suffix that follows the Dhatu verbal root namely Ting. Chaturthi Vibhakti denotes Sampradana, Panchami Vibhakti denotes Apadana, Shastri Vibhakti denotes Swasvami Bhavadi Sambandha and Saptami Vibhakti denotes the meaning namely Adhikarana. Let us take the example. When the Dhritiya Vibhakti is used and Karman is denoted, then this is what happens. So, when the meaning intended is the village where water has reached, we have Praptam Udakam Yam Sah. This as the Laukika Vigraha in which Praptam and Udakam these are the two padas prominently. And of course Prapta is a krdantavad formed by adding the suffix T to verbal root Aapha with the proverb Pr. So, we have Pr plus Aapha plus T and the suffix T here means Kartha. So, Praptam means one who has reached. So, who has reached? Udakam. Udaka has reached. So, water level has reached or water has reached. Yam. So, Yam indicates the Karman, Sanyoga Shraya and therefore it is in also the Dhritiya Vibhakti. So, Praptam Udakam Yam Sah. The word Yam in this case is written in red characters as well as bold ones as well as with an underline to indicate that this Yam is the linkage of the constituents with the Anyapadaartha. Now, Praptam Udakam Yam Sah is the Laukika Vigraha since the word Anekam appears in Prathama Vibhakti in the sutra Anekam Anyapadaartha. Both these words are eligible to get the term Upasarjanam by the sutra Prathama Nirdistam Samase Upasarjanam and both of them would get the Purva Nipata but the sutra Nistha says that a verbal form which ends in the Nistha suffix always falls in the first position in the Bhavvrihi Samasa and therefore we have to place Prapta as the first member of the Samasa. So, we have Prapta plus Su plus Udaka plus Su. Both of them are in Prathama Vibhakti. Now, the Samasa Saudhna takes place and then the Prathivedika Saudhna takes place and then suppose Prathivedika Yoha applies and so we delete both the Sups. So, we have Prapta plus Zero plus Udaka plus Zero and so we have Prapta Udaka and when we join them together, we get the form Prapto Daka. This is the finally derived compound output of the Bhavvrihi compound. Now, Prapto Daka is a qualification, is a Viseshana. When we decide to use it in the sentence, we add the Supratyaya to it and then we get the form Prapto Daka. So, Prapto Daka Gramaha, the village such that which has got the water levels reached. So, water has reached the village Prapto Daka Gramaha. What this indicates is there is an action denoted by the word Prapta which is reaching and Udaka is the agent of this action of reaching, Udaka means water and village which is the object. What is being reached? Prapto Daka Gramaha. So, this relation of grammar with Prapta is indicated by the word Yam which is marked in bold as well as italics, bold as well as underline in the Laukika Vigraha and the process therein. Let us now look at Tritya Vibhakti which expresses Kartha as the meaning, relation between the constituent's meaning and the overall meaning of the word. So, the Laukika Vigraha is Udaha Rathaha Yena Saha. This is used to denote the sense one who has carried the chariot.  And then the Samasa Saudhna takes place and the Pratyavadika Saudhna takes place. Therefore, we apply Supodhatu Pratyavadika Yoho and delete both the Sups. So, we have Udha plus Zero plus Rathha plus Zero and then we join them together and we get the form Udha Rathaha. Udha Rathaha whatever is Anyapadartha. That Anyapadartha acts as the Kartha of the action of carrying denoted by the verbal root Vaha in Udha. When we decide to use the word Udha Rathha in the sentence, we add the suffix Su to it. So, we have Udha Rathha plus Su and then finally we get the form Udha Rathaha. Udha Rathaha Anadvana, the bull who has carried the chariot. So, here the action is that of carrying denoted by the verbal root Vaha. The agent is the bull denoted by the word Anadvana and Rathha is the object. The chariot is the object. Let us now have an example where the Tritiya Vibhakti is expressing the sense of Karana Karaka, the most effective instrument. The meaning to be conveyed is an instrument which destroyed the enemy. Nastaha Ripuhu Yenasaha. This is the Lauki Kavigraha. The word Nastah has got the verbal root Nasha to destroy. Nastaha has got the suffix T which means Karma. Ripu is the Karma. Yena is in the instrumental singular. Yena is highlighted as well as put in the bold characters to highlight the fact that the relationship of Nastah and Ripu with the Anyapadharatha is that of the Karana. Nastaha Ripuhu Yenasaha. So, we have the Lauki Kavigraha. Nastaha Ripuhu Yenasaha. Then we get the Allauki Kavigraha.  So, Samasa Saudhnya takes place. So, Pratipati Ka Saudhnya takes place. So, Supodhatu Pratipati Ka Yoho applies and deletes both the Sups.  We get the form Nastah Ripu. Nastah Ripu. Now, when we use it in the sentence, we have Nastah Ripu plus Su and then we get the form Nastah Ripuhu. Nastah Ripuhu Khadgaha. This word which destroyed the enemy. So, we are making Khadga the Qualificant and Nastah Ripu as its qualification. Nastah Ripuhu Khadgaha. And there is this Pahuvrihi Samasa. And the relationship of Karana denoted by the Tiritya Vibhakti is what is the relation between the Anyapadhartha and the constituents of the Pahuvrihi Samasa. Now, let us move ahead and look at the example of Chaturthi Vibhakti which denotes some Pradhana to be the relationship between the Anyapadhartha and the Padhartha of the constituents. So, the meaning is one for whom the oblation is brought. Uparhatam havihi yasmai sahah. One for whom the oblation is brought. Now, in this case, yasmai is marked with the bold characters as well as an underline only to highlight the fact that it is this word that shows the interrelation of the constituents of the Samasa with the Anyapadhartha, namely the Sampradhana. So, we have uparhatam plus su, plus havis plus su as the alaukika vigraha, whereas the alaukika vigraha is uparhatam havihi yasmai sahah. Now, once we get the alaukika vigraha, we get the Samasa saudhnya. We also therefore get the prathipadi ka saudhnya. And so we have supodhatup prathipadi ka yoho applying and we get uparhutahavis and then we add the samasanta suffix. And then we add the sup suffix su and then by the sutra halgya bhyo dirgahat, so this separate tamhal, su gets deleted. So, we have uparhutahavis plus zero and then we get the form uparhutahavihi as the prathama ekavachana uparhutahavihi. That is the prathama ekavachana uparhutahavihi devah. It is for God that the oblation was brought. So, God is the recipient or the Sampradhana of the action of bringing as denoted by the purple root ru with the proverb uparhutahavihi devah. To summarize, interrelated subantas in prathama are a necessary condition as input for a bahubrihi samasa. Prathamaantam ittyarthaha. The interrelation of the constituent with the other or outer meaning is however never that of the prathama. Prathamaarthetu nabhavati. Prathamaantam samaschate prathamaarthetu nabhavati. The interrelation of the constituents with the other or outer meaning is that of the karaka denoted by respective vibhakti and the other relations expressed by the shashti vibhakti. This goes to show once again that the samartha theory is based on the karaka theory. This also shows the dependencies the outer meaning has with the action denoted by one of the substituents of the bahubrihi samasa. We continue studying this particular sutra anekam anepadarthe for some time now. We will continue studying it even in the next lecture. These are our references. These are the texts referred to. Thank you very much.


What's up everybody Jason here from jazby's case breaks comm this fills the break guys the last two flawless random hit spots are being Given away here in 1920 panini Donner's basketball hangar one box break random teams number five So first we'll do the break randomized customer names and teams to distribute the hits out of the hangar Then at the end of the break or randomized all 30 customer names again in a separate dice will a randomizer and Customers in the top two one and two get a spot each in the break and then flawless in a separate it'll be coming up So let's do the break guys. Let's dice roll it and We got ourselves a three and a five for eight times. Good luck one two three four five six seven down a Bradley eight times one two three four five six Seven Charlotte Horn is Denver Knights. I remember no trades guys. Just think of what you got quick hangar box Skimmed through the list right now. So Adam covering got the pelicans You got Bradley the Mavericks there Bradley at the Grizzlies Nick at the Lakers And I'm sure everybody's in there your team Alpha ties it you just actually All right, let's quickly do the hangar box Kyrie Irving relic Brooklyn Nets edition Marcus Mark Andre Roberson and a great expectations of Jackson Hayes You got a RJ Barrett and Kelly gonna Walker Cody Marden PJ Washington, Jr. Nicholas Clarkson Kobe white Calvin Johnson Terence man Dylan Winnler Somalajic Brian Clark Bruno Fernando and Schuma o Kiki Just the base cards here Now let's see who's gonna win those flawless thoughts guys. Good luck. Here you go. Everybody's in good luck Roll it again, and it's a four and a one for five times top two get in One two three four and Fifth and final time Oh wait, yeah four four and one five times.


The Royal Central Police Force, which is to inform the public of the temporary route closure of John Compton Highway, the closure starts at the intersection of Daisy Avenue and John Compton Highway to the VG roundabout. This closure is to allow for the hosting of the Uber Soaker Cruise Experience, which is scheduled for Wednesday, the 12th of April, 2023, between the hours of 10am and 4pm. Alternate route diversions have been put in place for vehicular traffic during this period. Persons coming from the city of Castries, heading northbound, are alternatively directed to Daisy Avenue via San Susi, through La Claree, then on to VG roundabouts. Persons headed south to the city are to exit left at the VG roundabout through La Claree, on to Darling Road and Calvary Road respectively. Articulated vehicles such as trailer trucks, heavy equipment and container trucks are to conduct transfers before the stipulated time as no allowances will be made. The necessary signage will be put in place to ensure that all route users are properly directed. All traffic regulations are to be adhered to during this period. Please be advised that police personnel will be on the ground to facilitate proper guidance and direction.


All right, this is the last session for the Data Facility Lectures. And we're super excited to have Lauren Fuchs from Oracle. Lauren did her undergrad and PhD at RPI at Rensselaer in New York. And I'm told skip the master's part, but she's not even bothered. She only had a bachelor's and a PhD. That's crazy. Right. Yeah, they give it to you for free. You just sign the form, or maybe RPI is different. You don't care. I don't need it. She's been at Oracle since 2007, working on the core development system. And so she's going to go retro today and not have any PowerPoints. Everything's good on the slide. So let's thank Lauren for being here. Thanks. Morning, everyone. Thanks for that introduction, Andy. Before I get started, let me get a feel for the audience here. Who here is an undergraduate student? None? So who's here as a graduate or a postdoc? OK, and I assume everyone else, you are professors? OK, so I assume I don't need to go over, like describe to you people what a B plus tree is or what acid properties are and all that. OK, all right. So before I get into the design of Berkeley DB, I'm going to start with its history. It all began long time ago in 1991 at the University of California, Berkeley, hence where its name came from. It began with some graduate students, Margo, Keith, and a few others. They were finding that the current database library for UNIX did not meet their needs. So being the intrepid students they were, they wrote their own. Created the database, put it online, made it open source, and went on to other things. As time went by, as often happens with open source products, they'd get requests for bug fixes. And being the busy students they were, they had to go, oh yeah, sure, we'll try to fix it, but we don't know when we'll get to it. This went on for several years until 1996. An up and coming web browser company named Netscape came to them with a list of bug fixes. Again, we'll get to it when we can, but we're busy. So Netscape came back and said, OK, if you do this now, we'll give you a big stack of money. And that was the birth of Berkeley DB. Since then, Berkeley DB's been deployed millions of times. It's become the back end of many different products ranging from Amazon, MySQL, Subversion. It's pretty much you log on to Linux or any Unix system like that, and DBD is already installed. We are everywhere. Now, before I get started, also another thing. The Keith in the story I just told you, he being the slacker that he is, started another database company and created another database after he sold Berkeley DB off to Oracle in 2006. That new company, Tiger Wire, he's the same Keith who talked to you in the beginning of the seminar. Wire Tiger. Why? Wire Tiger, sorry. So now on to the meat of the presentation, the design of Berkeley DB. BBB can be summed up in two words. Simple and flexible. We are a key value store. There's no schema, no query engine, no optimizer, none of that. We take your data. Doesn't matter what it is. We'll take anything, pictures, video, text. It's all the same to us. We'll take that. We'll store it for you. And some later time, we'll give it back to you. Because we're so flexible, as I pointed out before, we're the back end to a lot of other databases that are more complex. MongoDB used us as a back end, as a proof of concept that they could get rid of their old one. Berkeley DB offers a SQL API, which we created just by taking SQLite, ripping out the pager and sticking ourselves in. So that's a description of how flexible we are. Now, as you all probably know, storing and getting data, even though it sounds like a simple operation, giving that its asset protection is not. Berkeley DB provides asset protection using five major subsystems, the cache, the data store, locking, logging, and recovery. And I'll start today with the cache. Let's see. All Berkeley DB databases are just a set of pages. You start with page zero, and zero being the page number, which will be very important later. Page one and all pages are the same size. They never change once the database is created. On and on, page 3,000, and on and on and on, up to terabytes of data. That's possible. So let's say we have an example database. MyDB. And we have a record in MyDB where the key is CMU, and the data is awesome. And we'll say this record is stored on page 3,000 at offset 1,000. So if this, we'll say MyDB is on disk, and if we want to access that record, all we do is open the file, start at the beginning, seek to 3,000 times the page size, read the page into the cache, and then offset at 1,000 and read the key and the data. Now, the cache, of course, wouldn't be very efficient if it had to read the entire database into memory or access the disk every time it read something. So what we do is we share the cache by assigning each database an in-memory, well, a shared memory, and yes, we use shared memory, hash table. There, each page is hashed by its page number, 013000. So again, like most caches, when you go to read a page, it'll first look through the hash table, go look in the correct bucket for the page. If it's there, yay, we can read our data. If not, pull from disk, then we can read our data. So going back to the example of looking up our key, there's probably, there's an obvious question. Well, how do you know CMU is awesome? Is at page 3,000, offset 1,000? Well, that brings us to the data store. Berkeley DB databases offer several different methods for organizing the data. There's a B plus tree, a hash, heap, queue. You can decide when creating the database whatever type is used. We'll say, for example, my DB is a B plus tree. So again, we'll say we're trying to locate key CMU and pull up its data. How we do this as follows. First, we go to the root of the B plus tree, which is always stored at database on page 0. And then it does, as you all pretty much know, how a B plus tree works. This, the page will contain offsets, will contain, I mean, the page number to other pages. So to do a binary search for the key, we'll say we found it at, we found that the next part to search is page 13. There, binary search again, and located at page 3,000. And since this is a leaf page, it'll contain the data of CMU. So it's pretty much a basic B plus tree. However, there is one wrinkle here. Berkeley DB supports keys of any length. I mean, you can have keys that are multi-gigabytes long will support it. I don't know why you'd want to do that, but it's possible. Have you seen people do that? We had once a bug where they ended up with multi-megabyte keys, but that was a product where we were generating the keys for them. And they managed to hit this one awful spot where it just kept growing. As you can imagine, it was ridiculously slow. But yeah, generally, the keys are pretty small. But yeah, since we support keys of any size that makes building a binary search structure kind of a problem, this is how Berkeley DB solves that. I'll draw you an example page. This is, say, random page in Berkeley DB. It starts with metadata at the top, various metadata used by the pages. Next, it has an index of an index that contains offsets into the page where keys are located. And finally, it has the keys. And there was the keys and the associated data. Now, the keys are actually not sorted on the page that we pretend to stick them wherever they'll fit. However, the offsets to the keys are sorted with the first offset pointing to the least key and the last pointing to the greatest. So whenever you do a binary search, again, start in the middle of the index, read the offset to the key, and then compare that with your search key and move on, as you would. And then just continue from there. That gets into off-page duplicates. Well, we have a special structure where, once it goes to the offset, it gets another page number instead of holding the key. It holds a page number that points to another page that holds the key, which in turn could include part of the key, then include more metadata that points to another page, and so on and so forth, till you've got the entire key. Yes? Repeat the question for the video. So your question is, let me see if I get this straight. Basically, what do you do if you, say, insert a key, since this has to remain sorted, how do you deal with that? You know, I'm sorry, I actually cannot answer that question right at the moment, because I'm not 100% sure how we deal with that. I know it's something clever. If you could give me your email address at the end of the talk, I'll try to get it to you later. OK, moving on. So we have our data. We know how to get to it. This leaves the important question of, well, how do we protect it? How do we prevent collisions, overriding, make it durable, inconsistent, and all that? Well, that gets into the locking, the logging, and the recovery systems. I'll begin with how locally DB handles locking. A locking system is page-level read-write locks. So to go back to our example, let's say, instead of reading CMU is awesome, we want to insert it. So to go back to our example, we would start, our process would start by requesting page 0. Before it could get that, it would need a write lock on it. It would request a write lock, I mean a read lock, sorry, a read lock from the locking system. If the page 0 has no write locks on it, then the read lock is granted and we can continue. If there is a write lock, then the request is put in a queue and the threat of control is forced to block until the read lock can be granted. It would do the same for page 13 and the same for page 3,000. Once we're here and we found where we want to stick the record, then the process has to ask the locking system to upgrade its read into a write. If there are no other read locks, then it's granted we can write our data to the page. And once the transaction is resolved, all locks are released in one fell swoop. If there are all other read locks on the page, then we have, again, the upgrade request is put in a queue and it's forced to wait until it can get a lock. Now, page-level locking is great. A lot more concurrent than database-level locking that a lot of systems use. But it has a major downside. I guess everyone here, you know what it is? Well, deadlock. That's the big problem with it. Berkeley DB handles that by implementing its own deadlock detection system. Whenever certain locks are requested or at certain interviews, Berkeley DB will go through all the current granted locks and lock requests and build a matrix and use that matrix to determine if there's any deadlock. If there is, then it will select one of the transactions involved in the deadlock to be the sacrifice. It'll release all its locks and force the transaction to roll back. So any of the other transactions in the deadlock can now move forward. We actually allow you to configure your policy. I think our default is its random. But you can do the longest transaction, the shortest, who has the most locks, who has the least, anything like that. How often do your customers tune that? That seems like a hard thing to do, right? They don't really tune it much. They're pretty, I mean, we don't really have that much deadlock, because how smaller pages are, it's just. And also, people tend to use Berkeley DB more for just to insert a single record at a time or just a few, not really long, intricate transactions. So it usually doesn't become a problem. But the deadlock transaction is not a separate thread. Listen, stop everything. They're just a demo. Yeah. OK, so. How do you conduct this printing of notes in the B plus 3 where there is a conflict where there is no space? I'm sorry, I don't know. So when you're inserting your record in B plus 3, there is a possibility of a lock to overflow. There is a conflict. How do you conduct that printing? When do you go and read each of the locks and you do? I came to require you to acquire a red lock. You can choose not because you felt differently. You are reading the read, you're getting the read. Locks until they leave a lot of different from the right, right? Oh, the read locks we keep. So if we have to start splitting and going up the tree, then we just go back and try to upgrade the reads. Yeah. So now I shall get into, well, the transaction that's sacrificed, how do we roll back its operations? That, of course, goes into the logging system. Now, before I go into detail on the logging system, I want to explain how Berkeley DB is different from a lot of databases. I know many of them use a journal file. They'll use write ahead logging, which Berkeley DB does too, to just write the logs to the journal until it's time to resolve the transaction. Then it'll flush the journal to disk. Well, it'll flush the journal logs to disk. It'll flush the database pages to disk. And then it'll just delete the journal, because now everything's done. It's consistent. We're good. Berkeley DB doesn't do that. Our logs, they're not a journal. Instead of just a single journal, we use an ever-increasing number of fixed-length log files. And these tend to start at, say, log 0, 0, 0, bunch of zeros. 1. Then once that fills up, we go to log 0, 0, 0, 2, and so on and so forth. And these log files, they can stick around forever unless you do a checkpoint and specifically decide to delete the old ones. Is that missing what the distinction is between the other systems? It's just that these files are fixed-length. They're fixed-length. They never get deleted after a transaction commit. We don't delete them. They stick around. They theoretically stick around forever. And some people do keep them all. And once I get more into the logging, you'll see why we do that. Again, back to our CMU example. Say we found our page. We've gotten our logs. We're ready to write it onto page 3,000. But first, before that can be done, we have to write a log entry for the update. Now, every log in BDB begins with a log number. You usually call them LNS. This consists of the number of the log file and an offset into the log file where this log entry will be stored. Next is the transaction ID. That is exactly what it sounds like. It's a unique ID used to identify the transaction. So in case we have to roll back, we know what logs apply to that transaction. Next, the page number. All operations are logged and performed at the page level. If you have, say, a data record that has to be inserted on multiple pages, each insert is broken up into a different operation. And each operation gets its own log. Next is the previous offset. This just points to the previous log of this transaction. In that way, when undoing a transaction, it's easy to just read one, then skip to the next log. That's part of this transaction. Because since we've got a bunch of transactions going all the time, the log after this one could belong to a different one. So we don't want to have to read all through them to decide just start at the beginning and jump, jump, jump, undo, undo, undo. Finally, we have the operation data. This is another thing that Berkeley DB does that's different from many systems. Not only do we log enough information to undo the transaction on an abort, we also log enough to redo it. In fact, any Berkeley DB database, if you've got your log records back to the beginning of time, we can recreate the entire database from scratch, just reading through the logs. We've got automated systems that do that. And some of our users actually do do that. They'll create in-memory databases that never on disk. So if there's ever a crash, they just go back to the logs we created and continue. So we have our log for our entry. That's already, now we stick it in the log buffer. Then we write our data onto the page. And now this is actually really important. We write the LSN of the log entry onto the page. In this way, we know what, since the log numbers are always increasing, this way we know what was the last log entry to update this page. Now that this is all done, we flush the log to disk, and we're done. Consistency is guaranteed because, as I said, well, now we're actually sorry. We resolve the transaction first, of course. If it's a commit, then we create another log record that specifically says this transaction's been committed. Put that in the log buffer and flush all that to disk. And then, as I was saying before, we leave all the dirty pages in memory. We don't have to flush them to disk to guarantee consistency, because if there was a crash right now, right after the commit, we could still just read back through the logs and have enough information to just write everything back onto the page. So how do we do that? Well, that comes to the recovery system. As same as it's equally as reliable or some better than others, or that means mostly the disk controller has issues or less issues. Charles would be a lot more better to handle that. I think we have had some issues with f-sync, and we've had to use other, on certain systems, use other methods of getting it, yeah, other systems called other methods to get into the disk. But yeah, and I do know it's actually been a specific problem where it didn't actually sink immediately. But I'd have to talk to Charles because he's run into those problems. Let's say, oh, well, actually before I get into the recovery, there's one more thing I have to describe, and that's checkpointing. As I said, we don't flush our dirty pages on commits, so they just keep all collecting in memory. And eventually, you'll probably want to just sync them to disk. Well, we use checkpointing to do that, and how that's done in Berkeley DB is first it flushes all the pages to disk, then it creates a checkpoint log entry where the LSN of that log is guaranteed to be larger than every single log entry of an operation that's part of a committed transaction. We put that checkpoint log in the buffer, flush the buffer, and we're good. Now, the database is to a point where if we crash, we don't have to read all the way back to the beginning of the logs. We can just go to the checkpoint, and we're guaranteed that everything beyond that is on disk, so we don't have to keep reading back. I believe it's blocking. So this brings us to the final part of core BDB, which is the recovery system. Say we've inserted our CMU example, committed the transaction, but we haven't bothered to do a checkpoint yet, and then our application crashes. BDB recovers from that automatically when you go to open the database after the crash. What it does is, before you can perform any other operations, it looks into the log files and finds the last log. It reads that log, finds the page number of the page it operated on last, and reads that page into memory. It then checks the LSN on that page, the log number, and compares it to the log. If the numbers match, or this is greater than that, it knows that page was flushed before the crash, so it proceeds to actually undo the operations, believe it or not. If this log number is less, we know it was never flushed after it was altered, so we just ignore the log. After undoing or ignoring the operation, we read the next log entry in the log file and proceed with the same operations, just keep undoing or ignoring and also keeping a record of all checkpoints, of all transactions we find that have been committed, and we keep doing that until we get to the first checkpoint or the end of the logs. At that point, we know everything on disk now has been committed and been flushed, so we proceed to walk back forward in the logs, but this time only redoing operations that were part of transactions that were committed. Walk back forward through the logs until we get to the end, and once that's over, we're guaranteed that all uncommitted transactions that were still going on when the crash happened have been undone, while all committed transactions are now redone and the database is in a consistent state. That is the core of BDB. As you can tell, it's a very simple acid-protecting paging system, as I alluded to before, because it's so simple, it's easy to build things on top of it. In the past two decades, it's been out. We've added a bunch of features on top of it. The biggest, most interesting one is replication. Berkeley DB supports a single ReadWrite Master multiple replica replication system, and it's all built upon the fact that all operations can be redone just by reading through the logs. What happens is whenever the master node commits a transaction, it flushes the logs then before reporting that the transaction has been committed, it sends all the logs in that transaction to each of the replicas. They take the logs, redo the operation, and then respond back, we're done. And when the master gets a quorum of responses from the replicas, it can declare the transaction committed. We do actually support partial replication. Each database, Berkeley DB works by having an environment and any number of databases in that environment, which all share the same locking, logging, and cache. So while every database, you have to replicate the entirety of it, you can choose to skip some databases. A recovery system is actually written so that if a database is missing, when it does recovery, it just ignores it. It assumes, OK, it's not supposed to be there. Another feature we're adding, and this will come out next release, which should be in a few months, release 6.2, we're calling it slices. Remember how I just said Berkeley DB works by having multiple databases in a single environment? Well, slicing will allow you to split your database among multiple environments. So when you've got hardware that has lots of CPUs, you can take advantage of that by not having to share the locking or the logging system. The early version of this feature unfortunately doesn't support transactions across slices because it's a very early version that we had to knock out in five months. But future examples, future versions will include cross-slice transaction protection using XA transactions. Now I'm going to finish this talk with a bit of a sales pitch. As I've been saying over and over, Berkeley DB, it's flexible, it's simple. We are written not only so that we can take any data, but actually so we have our APIs written so that if you want to replace the locking system with your own locking system, it's made so you can do that very easily. Want to do your own logs? We'll let you do that. Through the API, you can write your own logs. How will people see you doing either of those things? I said we have page-level locking for our B-tree. We had one customer who really wanted record-level locking for that. He said, screw it, I'm throwing out your locking system and writing my own. And actually the logs get used sometimes to for companies that just want to do a special audit trail. So it actually does happen sometimes, not very often, but it does. But yeah, it's very flexible. It's open source, free to download anytime. The code, it's written in C, very easy to read, very well-commented. So basically it's the perfect database for experimenting and research I've found. And I'm not just saying that, I implemented my PhD thesis using Berkeley DB. So it's very easy to just take and if you want to implement your own locking system but don't want to build everything else in a database, you can do that. Your own logs, all that, or just build something on top of it, it's wonderful for that. If you want to download Berkeley DB, I'd give you the link, but it's long and weird looking, just Google Berkeley DB download. We're right at the top. Download us. Yeah, no, it is, I admit, but he doesn't take my phone calls. But yeah, now download it, open it up, and stop playing, and stop playing with it, and even ask questions on our forum. Again, the guys that, when I was doing my PhD, I couldn't have done it if they weren't so helpful on our public forums. Mike Feingold, he was working at XML at the time, the Berkeley DB XML database, and he answered all my questions really timely. It was great. So that concludes the talk. Do I have any additional questions? So what's the history of Oracle's business interest in Berkeley DB since it's open source, and there are many of you database companies, why did they invest in it? Well, one thing it seems, since they bought us, it seems like they really wanted a back end for a lot of their other stuff, and didn't want to write it themselves or depend on anyone else, because we're starting to be the back end to a lot of their products. As for how Sleepycat made money before its purchase, it was, well, it worked in two ways. People would buy support. That's pretty common. But also, our license works so that if you're open source, you can have us for free. If you use us internally, we're also free, although I think they may have closed that loophole.  So that's been a source of revenue. That's been a big revenue stream for us, just many people wanting to use us, again, as a back end to their application for their commercial product, so we get money that way. I've heard rumors they brought it to squelps that might see a back end, so that might see open new transactions. Oh yeah, I know we were the only transactional back end for a while. That happened right before the inner TV cafe. You guys bought it and killed it. Really? I thought that happened earlier than 2006, because that's when we were bought, but it could have happened. Everybody thought, yeah, yeah, yeah, yeah. Any other questions? So could you talk about how you handle multiple day-laces, multiple processes accessing the same day-laces? Is there anything special you're doing to share memory locks, or is it just they're writing to the same memory space in use, but an IPC control QM, or is it all holding for the locks? What's the architecture for that, for the multi-processing case? Yeah, we do make use of shared memory. Yeah, we do use locks. I know recently we moved to, sorry, this is more someone else's field, so I can't get into too much detail, but I know recently we used to a feature called latches. And also just shared memory locks and latches is how we handle that. The architecture is here. Big thanks for DJ2B's locking, for keeping me straight on this whole thing, and thanks for all our guests who've come to give talks. So maybe we'll try this again next year. So thanks for coming here, everybody.


Good morning. I'd like to welcome you out to my talk, Capacity Analysis, Mapping How Interventions to the Spread of COVID-19 Improve Healthcare Providers' Ability to Meet Patient Demand. Sorry that was a bit of a mouthful, but by way of introduction, my name is Christian Echinus. I'm a product engineer at Esri and I work on the ArcJS API for JavaScript. Let's go ahead and dive right into this. If there's any lesson that the pandemic has taught us, it's that geography matters and that spatial thinking matters more now than ever before, and that will be an overriding theme throughout this talk. Ultimately, this came down to one goal. We wanted to build an application, a web app that properly communicates how interventions to the spread of COVID-19, such as physical distancing, reduce hospital stress for a particular area of interest. But before diving in, I want to take a quick minute to describe the work that Esri does and the work that I do to put it all into context. Esri builds GIS software. That's short for geographic information systems and that's just a fancy way of saying that we build mapping software and we build a variety of spatial analysis tools for desktop, mobile, and web clients. We also provide dozens of APIs for developers to extend our software and to build their own customized applications. So this chart here on this particular slide is a little overly simplistic. We do build all of these apps, but we expose a variety of APIs for developer community. But I specifically work on ArcGIS API for JavaScript. This is a powerful API that allows us to build web applications, specifically for mapping. And I'm particularly, particularly involved in many of the projects that you see here on the slide and they're related to data visualization. We do everything from 3D mapping to multivariate mapping, predominance, and particularly what I'm most interested in, which is exploratory mapping. And that's going to play a heavy role in my talk today. But I also want to mention this fast performance with WebGL. We have a WebGL rendering engine, which allows us to build highly interactive applications in conjunction with this client-side query engine that we have within the JavaScript API So I'll touch on that a little bit later, but I just want to note that this is all about exploring data. Some apps that are built with the JavaScript API include configurable applications that Esri openly provides to customers. And these include dashboards such as this one. This is one that you're probably familiar with or may have seen in the news. This is Johns Hopkins University's Coronavirus Resource Center. It's the most up-to-date tracking app of the virus out there today. And so they basically built this using a configurable template that's built on top of the JavaScript API. But before going into the hospital capacity question, I want to give a brief overview of just a small portion of our APIs to set up the work that we did there. The ArcGIS API for JavaScript isn't about just showing where things are located, but it's more about what is there, how much of it exists, and doing that in a meaningful way. And there's many ways to do that, and I'm just going to touch on a couple of techniques, and those are size and color. So if we look at this particular map on this slide, we're looking at the total number of positive test results by a particular date. So this, I believe, was displayed for June 1st of this year. And the way that we created this visualization is using the renderer property of our layer. The layer is U.S. counties, and we represent them as icons or circles, those pink circles in the centroids of the county. Where the data-driven aspect comes into play is with visual variables. The visual variables just simply allow us to override a property of the symbol. It could be size, it could be color, it could be rotation or even opacity, and we override it based on a data value. So in this case, we specify a field, so the number of infections on June 1st, 2020, and then we'll indicate what size the icon should be based on the value of that data. So in New York, we have a value of over 200,000, which would give the size 20 pixels, whereas other places in the Midwest and West would be much smaller. So we think that this is a very simple, intuitive API that allows you to create dynamic visualizations that allow you to understand what's happening. You can also look at color, such as here, we're using a fill symbol instead of an icon, and we're changing the fill of the color based on a data value, and you can see those color steps down in that blue highlighted region below. What's different here is we're actually using what we call arcade expressions or value expressions. Arcade is an expression language developed internally at Esri that allows us to calculate new values client-side. So when we don't have access to an attribute or a column in our table or database, then we can calculate that on the fly client-side. So in this case, we want to look at the rate of infections per 100,000 people, and so we use an expression to do that, and you can see that there. Now, when we talk about data exploration, there's a couple of ways you can look at it. There's a way of exploring data using filtering, that's just simply removing the features that don't conform to a particular expression. It's a very common way of looking at where things are high or low, or you can actually modify the style or the renderer itself, so the color stops. I want to change the color stops to see where the most infections are occurring, perhaps it's the highest rate. But when we update the color stops, it's simply updating those stop values or even the colors themselves, not the data itself. And that's important because when we get into the multivariate or multidimensional data exploration, it becomes more about modifying the data value itself. So in that previous example, we're just changing the colors, seeing, okay, these are areas of higher or low, but that's only based off one field. It's that number of infections up to May 31st. And what if we want to see the number of infections or the rate of infection over time? That would require a constant set of stops here, and then updating this value up here. So that's something we can do using a technique called multidimensional data exploration. And that's simply taking, you know, a few dimensions, such as the location, and then the variable and this GIF we're looking at temperature in the ocean. And then we can explore it with another dimension. It could be time, it could be an agent cohort or income brackets for demographic data, or perhaps Z values, like depth and elevation. That's actually the third dimension that's being used in this particular application here. But when it comes to mapping COVID-19, the most obvious case is or dimension to use is time. So we want to look at not just what is happening, but when is it happening? Let's build an app that allows us to explore COVID-19 data through time. And so that comes from a number of different questions you might ask, like how many people are currently sick? How many new test reports or results were reported today? Or how many people have been infected or who have died to date? Those have been the predominant questions that have been asked, and you've seen that with hundreds, even thousands, of mapping applications that show that, such as this dashboard, where you can see how many cases have been reported per day by geography. Okay, here I've opened up my browser to show, conceptually, another way you could look at the past data as a time series. I've just taken the Johns Hopkins data. I've added a time slider, which is out-of-the-box widget in the ArcGIS API for JavaScript. And when you hit play, you can see how the number of infections grows per county in the U.S. So it's no longer just about a static display of what's happening today. It's about what's happening over time, so you can interact with that. This is not a time filter. I'm not filtering out different counties. I'm actually just updating that visualization, and I'll show you that in just a bit. You can also use Arcade to create more complex visualizations, such as doubling time. So I see the total number of infections, but then I can use color to indicate how fast those infections are growing. So I can see that in the Midwest on this particular date, May 7th, that they were still doubling at seven days or less. And you see that in Chicago, too, right there. So you can create some pretty cool applications using this multi-dimensional approach. So if you look at this CSV data up to the right, this is the Johns Hopkins CSV that I was talking about just a second ago. Basically, they add a new column every day with the total account for the infections up to that date. So it's up to date, and you can work with it seamlessly. So if I want to map that, I just create the render like I did before, and I reference one of those fields. So I'm referencing April 1st, 2020, and then I can map it in my view there. And if I want to change my visualization, rather than copying all my features and then rendering it based on a different field, I can simply update that field each time the slider position changes, and that will allow me to see how it changes over time. We're able to do that because we bring all that data down to the GPU. And so when the renderer updates, we actually have access to that data already. It's in a lookup table, so we can just point to that field. It's already part of the geometry, so we can update the style of it. We don't have to discard the geometries and re-render them like we used to. This allows us for highly interactive applications. All right, now let's talk about the future.  And that all depends on modeling, authoritative modeling, and forecasting that because we can't obviously predict the future. And this is a chart that from the CDC that we're probably all familiar with, and that compares two different curves. It shows the number of cases over time if no intervention is taken during a pandemic outbreak. That's the purple one. If you look at the smaller curve, that indicates the number of cases over time if intervention takes place. And that's important for two reasons. One is that it reduces the overall number of cases, so we save more people. And two, it helps our hospitals out because they only have a certain number of beds, a certain number of ventilators to help treat people for a disease like this. And so by intervening and flattening the curve, we can help them out so they can treat an appropriate number of people. So but looking at this chart, there's no numbers on it. It's all conceptual. So we need models with real numbers to help us truly understand which interventions will work and what needs to happen in order to help us in this scenario. And so various things are proposed such as social or physical distancing, wearing of masks, shelter in place guidelines. But that also prompts people to ask, well, how much does that actually help? And when can I get back to normal? Does the phased approach to reopening actually work? And how much does that improve versus reopening everything now? And so what we do is we can run various outputs of models as what-if scenarios. This allows us to understand which policies will work. That'll allow us to properly allocate hospital resources to areas that are in more need and to know where test facilities should be located. Ultimately, an application like this that I'll show in just a second will allow us to transparently communicate results to the public and to the people who are making these decisions. Now when we talk about forecasting using models, there's two authoritative models that we at Esri have looked at as helping us in this effort. There's the CHI model, which was developed by Penn Medicine and the CDC's COVID-19 surge model. They both look at similar parameters, such as the number of cases in an area, the number of hospitalizations, the total number of hospital resources, and susceptible populations. The differences are that the CHI model looks at social distancing as a parameter, so you can indicate the percentage of the population that complies with it and how that affects the number of hospitalizations as well. Then the COVID-19 surge model looks at a stage intervention approach and how well that actually will help reduce the number of hospitalizations. In our particular applications, we want to look at three variables. How does variations in these parameters affect the number of required hospital beds versus ICU beds versus the number of ventilators needed? Our spatial stats team at Esri developed these two two geo-processing tools based on these models. That allows us to hand the tool to the experts, the epidemiologists, the health experts, so that they can put in their own authoritative data and run hypothetical scenarios, such as what happens if 30% of the population complies with social distancing, and what happens if we reopen businesses in three phases versus no phases. I was not involved in that part, but where I became involved was in developing this web application called Capacity Analysis, and that allows us to understand how impactful a particular policy can be, and that the fundamental idea behind it is comparing two what-ifs and running them side-by-side, so you can see how well the curve flattens for the overall data set and also for a particular geography. So I'm going to open up this application here in my web browser. Here I've opened the kind of a default view of of this Capacity Analysis app, and now I'm comparing the two two different outputs of the CHI model. The view on the left shows what happens if 29% of the population social distances, the view on the right indicates what happens when 50% of the population social distances. This is a really simple app when it comes down to it. There's three attributes that the user has control over. They can look at the number of hospitalizations, number of expected ICU patients or ventilators used, and then they have this time slider down here at the bottom, which allows them to animate through time to see what they can expect will happen based on these different parameters. So if we look at May 31st, we can actually see that 29% social distancing, people are much worse off than if 50% social distance. Our hospitals will be only up to 27% capacity. Now you'll note that that's not necessarily true across the board. That's the benefit of using maps. If we actually look at our map legend, this will help describe our methodology for the visualization. We actually use two visual variables. There's the color variable that indicates the capacity, so the hospital stress. If you're under capacity, or if a county's hospitals are under capacity, they'll be rendered with a small icon and they'll be gray, or maybe an orangish-peachish color. Once you hit capacity, that color will change to orange and then get a deeper red as you get worse above that. And then that's when the size kicks in. The size does not indicate the total number of patients. It only indicates the number of patients above the hospital's capacity. So the idea is to show not only are you at capacity because that's the critical variable, but how worse off are you? So you can see these metro areas such as Orlando and Miami and Tampa being much worse off in this case of lower social distancing numbers than when there is. So having the side-by-side views is, it's an easy way to see, wow, this actually will help our hospitals out a whole lot more if people, more people comply to this order or to this recommendation. We looked at other ways of visualizing it, like displaying both layers in the same view, or maybe calculating a difference between the two, but they don't effectively communicate it like this does, and this allows us to also associate them with this chart below. So if I click on feature such as here you can see the chart update specifically for that county. Now what went into building this application? So the outputs of that model that or both of those models that I described do not follow the same format that I showed with the Johns Hopkins data. The Johns Hopkins data was outputting a new counts with a new field every single day. That was not practical for this particular case because we have three attributes that we're looking at, the hospital beds, ICU beds, ventilators, and if we created three new fields per day we'd have a huge table that would be hard to manage and we actually run into technical limitations with our feature services once you hit a certain number. So we decided to basically generate these numbers and then provide them in a single field as pipe separated values. So when the user selects an attribute such as hospitalizations they can then move the slider to a particular position and then using the same technique that you saw before with parsing of the arcade attribute we can actually use arcade to parse that value out and then map it in both views. So this is actually happening two times in the same app with each slider change. As the slider changes this is what's happening. We take the position of the slider and then the variable selected by the user and pass it to a function called update renderers. What that function does is it creates new visual variables, a color variable, and a size variable. The color is what's indicating that percent capacity that you saw in the app itself and then the size indicates the number of patients over capacity that a hospital will be at. That has to be calculated with an arcade expression because we're just looking at raw counts in the actual data set. And so those functions for generating the arcade basically will create these expressions that you see below and the important bits are highlighted there in yellow. So we look at the total number of hospitalized people. That's how we parse it out in arcade. It looks similar to what you might do in JavaScript. Obviously these are custom functions that we've developed but we get the number of hospitalizations based on this field and then we just grab the the count for a particular day indicated by the slider and then we do a simple normalization and percentage calculation. And then with the overages exact same line of code here and then we just calculate the difference between hospitalized versus capacity. If you're under capacity you're great. If you're over capacity then that's when things become a problem and we need to visualize it appropriately. In conclusion the goal was to make an application to put into the hands of the experts. So we converted this into a configurable app template. That allows experts and their development team to download the application and then run the models themselves and then they can deploy it on their own terms. So it's not us at Esri saying this is what's going to happen or this is what we predict will happen. We built the software, we deploy it, and we allow the experts to take over and communicate their own results. We wanted this to be interactive. We wanted it to be clear, understandable, and personable and easy to use. As you know there's only two controls and it's side by side. It's easy for people to understand that one method will may substantially reduce the number of hospitalizations versus another. So with that I'd like to thank you for joining me for this talk. Again I'm Christian Ekinness. If you would like to continue this discussion or if you have any questions or comments or feedback I encourage you to reach out. There's my email address. You can also reach out on Twitter and perhaps you could even visit my GitHub and see some of the projects that I've worked on there. And once again thank you for listening.