Alright, thank you very much. Hi everyone and good morning from my end. Thank you for joining my session. So today I'm going to be talking about Wikitons and how it's been preserving language diversity and also empowering communities through language documentation. Okay, so as you all know, or maybe you might not know, but there are over 7,000 languages spoken across the world today. But then every an average of one language dies or that's one language every two weeks dies. And when a language dies, it's not just the words that are lost, which maybe some people might think it's also the history of those people, their culture and the knowledge of the people who spoke it. We lose important information about the world and we also lose a way of seeing the world. And this is one of the things that Wikitons is working to prevent from happening. They're using technology as well as Wikimedia projects to document endangered languages so that they can be preserved for future generations. They're also working to empower communities to use these languages so they can thrive. Some languages, when you check them today, they only have 20 to 30 living speakers. The Donsa language, for example, in Papua New Guinea has an estimated number of 20 to 30 living speakers. There is also the Kambap language in Cameroon and so many more languages. Now you might be wondering what Wikitons is because I know a lot of people are not familiar with Wikitons. Wikitons is also a Wikimedia user group and a non-profit organization whose mission is to preserve endangered languages and empower communities through languages. Now you'll be wondering, language documentation, why is it important? I mean, it's necessary. It's actually very important to document languages and some of the reasons why it could be that it preserves endangered languages. When a language is documented, it's preserved for the future. And this is important because endangered languages are very valuable to humans and it's also a very valuable aspect of human knowledge and culture. And if we have to, should I say, if as the Wikimedia movement, we are promoting free knowledge, free and open knowledge, then language is also a very vital part of that promotion and that knowledge we're talking about. And another reason for documenting languages is that it empowers communities. Language documentation can empower people by giving them a voice. It helps you to know where you're coming from and where you're going to. If you do not know your language, sometimes you feel like you're being left out, especially from the part of the world where I come from. It also helps to prevent language loss. So when there is documentation, the future generation can have access to them. I used to know of a language that the last living speaker died in 1992. And ever since then, it's been hard getting other living speakers of the language. Now, you'll be wondering how Wikitons actually documents these languages. So the first one is that it uses technology to make documentation more accessible and efficient. There's a crowd sourcing platform and what we call a seed bank where linguists actually a group of or a network of linguists get to document languages and upload them for use. There are over 700 languages in those seed banks. It can be there in form of videos and audios and so much more. Then there's also the community engagement part of it. When Wikitons works with communities, they ensure that language documentation is done in a proper way. You can just go into a community and then you decide to start documenting. Sometimes it causes problems. So they try to teach you how to be respectful and how it should be beneficial to the community. And then they also train community members to document their own languages. Now we have a language accelerator program as well. Wikitons organizes a language accelerator program. Since 2022, it's been organizing this program and we started with 10. This year, we have 21 people in the program and these linguists or these activists, they come in to do a whole lot of things for their communities to preserve their languages. Some of the languages I noticed, they had like 2,000, 3,000 living speakers, some had 7,000 and they came from different parts of the world. Some came from Africa, some came from Asia, Europe, Australia and so many places. And for this year, the selection was made across 20 different countries. And for the seed bank of language diversity, which is another way that Wikitons documents their languages, this is way that in this aspect, you can contribute your videos, your audio recordings, your dictionaries and other resources related to your mother tongue. So for example, the participants of the language accelerator program, some of them develop keyboards. I know of someone who is developing a keyboard. There are people who are building up dictionaries and so many things to use and preserve their languages. Now these resources can be uploaded to the seed bank of language diversity. Some of them can be in form of audios and those keyboards that I mentioned about. And then there's the global network of linguists. We can't do this alone without linguists, our language enthusiasts or language activists. So that is the reason why we have a global network of them. A lot of them get to donate videos, a lot of them get to donate audios that have to do with those languages that they are working to preserve or to revitalize. And there is also the language sustainability tool kit. So for example, as a Wikimedia, you come from a community which the language is endangered. We have the language sustainability tool kit that you can use to document your language. So this tool kit helps you to develop your, should I say like your plan and your goals for starting a revitalization project for your community. It guides you, for now it's still in English, but we're working towards getting as many translations as possible. But then Wikiton's projects are not restricted to English. We have speakers of other languages and during the language revitalization project, for example, we have speakers of other languages and we have access to translators and translation tools to make it easier. Then there is the Jewish Languages Project which is intended for the documentation of endangered languages. There are lots of endangered Jewish languages which the Wikiton projects are working to revitalize and to also document.  There's a role for you. You can either visit Wikiton's website to contribute documentation efforts and also spread the word about the importance of language diversity. And another thing you can also do is to join the Wikiton-Suzar group. So the Wikiton-Suzar group has been down for a while, but now we're back to the community and we're going to be starting our first program in September next month. So you're welcome to join the Wikiton-Suzar group if you're enthusiastic about languages or preserving a language. And the good thing is that the language accelerator program is open to everyone. All language documentation and revitalization projects are also open to everyone. So it's not something that is just for outsiders or linguists or you as a Wikimedia can join. We have 10 Wikimedians in the language revitalization accelerator and they're also working to preserve their projects through various areas and aspects. Some of them are building Wikimedia projects, they're building Wikipedias, some are building legs, some are working on developing keyboards that can help in contribution to their languages. So you're welcome to join us in any way you think that you might want to help your community grow or revitalize their languages. In conclusion, let's remember that language diversity is a treasure that enriches our world. And Wikiton's dedication to preserving languages and empowering communities is an inspiring example of what we can achieve when we work together. So I'm inviting you if you feel that your language has less speakers, less living speakers, or it has some that actually have one living speaker. And once the one living speaker dies, that simply means the language dies if it is not preserved. So you're welcome today to join us in Wikiton's to preserve your language. Thank you for joining me in this exploration. You can email me at stochie at wikitons.org or you can email the whole Wikiton's team at hello at wikitons.org. I'm also on LinkedIn at Stochie Precious. Once again, thank you for joining the session. If you have any question regarding language revitalization or documentation, I'm open to that. Thank you, everyone. And hello, Tochi. This is Daria from Wikimedia UK. I have a question for you. Hello. I'm curious if you feel what Wikitonx is doing is completely unique in the Wikimedia movement, or maybe are there other initiatives that you think are, I don't know, similar or connected? I guess I'm thinking, are you like an island that is doing this thing, or are there like connections, maybe overlaps or other projects that are supporting you? I was thinking, for example, about, I don't know if you know, like the manuscript preservation, it's more Wikisource. But it reminded me a little bit of the idea of preservation. I think I lost you, but I don't know if you have any questions. So, simple question. I don't know. I don't know how to help. Are you working alone or with others? I can write in chat, maybe. I don't know. Okay. But it's, it's red. So, yes, yes, yes. I think so too. Oh, it's a shame. I can try again later. Sorry, Daria. I lost you. Okay. Did you hear anything? I heard when you were asking if there were other communities in the, if there are other user groups or communities being something or are you collaborating? Yes. Are you collaborating? Are there other people or is your project, would you say unique?


Welcome to our hands-up date. Here's what's happening around your Navy. Preparedness and military readiness go hand-in-hand in the Navy's role to protect national security. Navy installations across the U.S. as part of a DOD-wide effort will be partnering with FEMA as part of America's Preparathon April 30th. Research vessel Neil Armstrong arrived at its new home port at the Woods Hole Oceanographic Institution in Massachusetts during a pier side ceremony featuring remarks from his wife Carol Armstrong. Armstrong is a deep ocean general-purpose research vessel used for scientific studies including ocean chemistry and geology. Armstrong features the latest in navigation and ship positioning systems while running on clean diesel engines. Time is running out if you have not filed your 2015 taxes. The deadline for filing this year's taxes is Monday, April 18th. You can read more about what's happening around your Navy at Navy.mil.


Crude glycerol, the main co-product of biodiesel production, has various value-added utilization opportunities such as being used as feedstock for producing other chemicals and animal feeds, which contributes to the viability of biodiesel.


Well, it's been a challenging start to the year for the Bemidji High Boys hockey team. The Jacks have high expectations this season after making their second straight trip to state last year. But they hit the Christmas break with a 5-4 record after playing a pretty difficult early season schedule and today probably their most difficult game yet as they travel to Hermantown to take on the defending state Class A champs. Bemidji beat the Hawks last year and handed them one of their only two losses on the season. Different story in this one. Well, Hermantown in the first period as they fired shot after shot on Bemidji's net. But goal tender Matt Fitzgerald would thwart their attack. Fitzgerald was big in the first and it was scoreless after one. Second period Hawks finally solved the Fitzgerald puzzle there. Matt Velour on the back end to make it one-zip Hermantown. That was all they would need. Hawks goalie Cade McEwen with the shout-out turns away Alex Pollock here. One of the few good scoring chances for the Jacks. Bemidji with just 12 shots on net, Hermantown pulls away to win it 4-0. If you've enjoyed this segment of Lakeland News, please consider making a tax-deductible contribution to Lakeland Public Television.


How we made this shot from a recent short film. To get our top-down shot, we started by rigging the camera above the actress using a C-stamp. On to lighting, we brought in this lamp and set it up to the side. For the streetlight outside, we used an Amran 200D. Close the blinds to add some texture, and then used a gel to make the light warmer, like a streetlight. When she gets the text, we need a vibrating phone sound. I'm using Artlist to find this. Here's one that sounds pretty good, but we can look at similar sounds here. This one sounds even more like it's on a bed. This is a decent scene, but we wanted to imply a text from a crush without showing the text. Something like hearts would be too cheesy, so we instead added some pink curtains to motivate pink light into the scene. Pink is often associated with love, so this will help sell it, along with the most important part, music. I found this song when browsing Artlist, which will make our scene perfect for Valentine's Day. Head over to artlist.io to get the best creative assets for your videos.


How are we finally feeling? We are feeling a little better. What does a little better mean? I just got to be selective to what I eat. That's all. Yeah, I can eat just. I just got to be selective. Like, I'm on a real bland diet, like mashed potatoes, rice, bread. How about the fluids, you know, getting the fluids? I know the hamstring and everything. Yeah, everything's good. Like, I do whatever to prepare, like IVs and everything. I'd be fine. Yeah, I felt fine. I felt fine. Yeah. What's the challenge this week against the Chiefs? To outscore them. I mean, I'm going to make it as simple as possible. They got a good offense. We're not here to talk about the offense, but they score a lot of points. You can't just forget that. But they got a really good defense. They got some guys on the back end, and we got to do what we do. It's really not about them. It's about what we do. We got to score. We're getting the red zone. We got to finish drives. We got to start fast, so. You had mentioned that you lost, what was it, 8 pounds? Yeah. How are you, as far as, like, getting some of that weight back? Eating gradually, I guess, I mean. Maybe they help me out. Maybe they help me out Sunday, so. You're still getting the IVs on a regular basis, but still, yeah. What's the total weight loss at this point? Uh, I don't want to share. You and Ryan got into a rhythm in the second half on Monday. Is that something that can translate and sort of keep that up? Um, of course, I would love to. You know, we got going, and we just kept going. I'm definitely going to try to use that momentum and head into this game. And today's practice was really well, so I'm definitely going to try to use that momentum. It's been a little bit of a steady patience for you this year and for Julio, too, in terms of I know you guys wanted to play together so much. Most definitely, most definitely.  And it's a personal game, you know? And you just got to enjoy the process where you at. So, and that's exactly what I'm doing. I definitely have changed my mindset 100%. You know, I was a little frustrated, but I'm fine. I know I'll be fine, so, you know? I guess maybe the good thing is you can talk to one another about the whole process of coming back from there. Oh, for sure. For sure. We're going to do whatever is necessary, you know? If we're not catching passes, we're going to do whatever is necessary to impact the game, like blocking, doing anything. So, you know? When you guys are hurt, they probably get a little sympathy when you got food poisoning. What's it been like? I think even there is a kind of cracking joke. Yeah. What have you heard this week? A lot of Chipotle jokes, you know? A lot of jokes here and there, but, you know, but guys reaching out just making sure I'm okay. So, yeah. Has Chipotle reached out to you, though? Yeah, Chipotle has reached out. They want to know what location and, you know, possibly do something with me. So, you know, I don't know. I'm not eating it, though. No, sir. Jay, when it comes to just this offense and especially when you're going up against Kansas City offense, they can put points up on the board. Often, how important is it for this offense to really start fast in this one? It's definitely important, you know? Coming out, getting the first, first down, moving the chains, isn't every time, could burn no third down. You know, that's a really good offense, you know? And to be honest, you kind of got to control the game, you know? They score at will sometimes, you know what I'm saying? So, like I said, we got to not just control the game, but like play our game and just do what we do. And we'll be fine. How tough is it? Some guys were talking yesterday about Nick and learning all three positions real well. He was? Yeah. Scarler. He's a scholar. Yeah. Tough to do, to turn all three that well? Most definitely, most definitely. For him to just hop in and just learn. It says a lot about him, you know? That's definitely not easy, you know? You wouldn't get attached to a rookie.   Were you serious about that swearing off all fast food? I'm so serious, I promise you. Because the way I was feeling guys, I'm telling you, no fast food for me. I'm not eating at a five star restaurant or something like that, I'm eating at home. Like, I got a, I cook.  I feel you, I feel you, I feel you, too. You've been playing more slot than maybe you have in years past. I'm curious, is that somewhere where you're comfortable? I mean, you know, I know you did a lot of that in college, what's sort of the difference there? You gotta recognize, you gotta recognize faster, especially in the league. You know, you see something as soon as the snap of the ball, you see something else. So, just picking up, picking up the little judgments quicker. You know, in college, you may see one thing and you just see something else. I mean, it's kind of simple.  How do you focus on the other defense, you just said, but you gotta like how we feel when the fellow receiver will watch him play. What kind of jumps out? His speed, you know, I think he's, in my opinion, I think he's probably like top five receiver in the league, you know. He's a well-rounded receiver. He can do, he does everything, run great routes, you know, he's fast and he gets away from people. So, you know, I hope he have a not so good night against us.


No, yeah, okay, so I guess my slides aren't showing up. I don't know if that's on my end or what's going on there, but hold on, see if we can get this sorted out. Hello? Okay, sorry for that. So I guess I'll start. I'm Andrew Lasky. I'm a core developer on Nova. I'm one of the people leading the effort for NovaCells V2. I'm one of probably five or six people who are kind of key to this effort. There's a whole lot more who are involved in the effort, but I'm going to get up and talk about it. So NovaCells V2, what's going on? I'm glad you all asked. So I'll start off from the very beginning. Let's just talk about what are cells real quick. Cells are, one, a strategy for scaling. There's two things we may want to scale here. There's the database and the message queue. So if you think about Nova today without cells, you might have a single deployment. You might have 1,000 hosts in it that may overwhelm the message queue and may overwhelm the database depending on what you're trying to do with it. So what you can do is set up cells. You end up with a little bit more complexity in your deployment. You have this API cell at the top. You have a database and a message queue up there. If you have cells be one, you have a message queue up there. Then you end up with two cells underneath that may have 500 hosts each. That means that the database there, the message queue there, only need to deal with those 500 hosts. So you've reduced the amount of traffic and load that they each have to deal with. So that's one part of what cells is. It also provides failure isolation. Let's say you were to lose a database without cells. You lose the database.  You can't query them with cells. If you were to lose a database or a message queue within a cell, you can still access everything in any remaining cells. So you do, you get some failure isolation there. It is also an optional grouping mechanism. Some of the cells be one deployers. What they like to do is put all of the same type of hardware in each cells. You might have one cell that's purely bare metal nodes, one cell that has SSD computes in it, one cell that has spinning disks. That's something you can do. It's not required to do that, but it might make capacity planning really nice. It might make failure isolation really nice. There's some benefits to doing that. Another thing you can do is scale out deployments by adding a new cell. So you can take, you know, a whole new cabinet of computes, stand them all up, kind of deploy everything except the API on top, test them out, run them through QA, send builds there, make sure everything works, and then you plug it up into your production deployment. At that point, it becomes accessible to all of your users, but you also know that everything works. You've tested it beforehand. So that sounds great. There's already a cells v1. So why are we doing v2? Talk a little bit about the v1 architecture. It works sort of by capturing messages and replaying them. It tries to do this in a very transparent manner. It's a really great scheme, but it does lead to race conditions. It was kind of implemented in a really cool way, but there's just inherently race conditions and how it operates because it's not built into the code base in a core way. So that's been a problem. There are two levels of scheduling with cells. You get this top-level scheduler that has to pick a cell, and then once you're within a cell, you schedule as normal with NOVA. The problem is the top-level scheduler has very limited data and has course data. It only knows that cell A over here has these flavors available, or this number of slots for this flavor available. So you can't do anything like affinity with it. If you have a host in cell A and you want to build a new host and say, I want it close to an instance, and you want it close to that instance, there's no mechanism for doing that. The cells, top-level scheduler, doesn't have any information that would allow you to do that. So there's certain things that aren't there. It's also lacking some of the other basic features that NOVA provides. Security groups is a big one. People like host aggregates and availability zones. Again, that's kind of tied into the information that the top-level scheduler has. And just in general, there are other scheduling options that aren't available. It's also bolted on. And this is probably one of the biggest issues with it. Few deployments use it. Mostly larger deployers use it. Those who have resources, they can devote to kind of holding internal patches to make it work. A lot of deployers are not, or a lot of developers are not familiar with it. That makes it really hard. One for people who are pushing patches into cells code to get reviewers to get that code merged. But also people working on NOVA in general, they'll add a new feature. They'll forget to add it for cells. That's part of why we have all these features that don't work for cells. There are also a lot of bugs. There's a lot of testing things around it. And we just, there aren't a lot of developers who can really work on it. And that's a problem. And it's because it's kind of shunted out of the way in the code base. It's for the most part very transparent, but also very hidden. It's also difficult to upgrade from NOVA without cells to NOVA with cells. So if you start your NOVA deployment, and at some point you're like, you know what, we're really starting to scale. Things are going great. I would like to now run with cells because I need the scaling ability that it has. There really is no path for doing that. If you want to run cells, you really should have started with it from the beginning. But it's also very complex, so we don't recommend that anybody starts with it from the beginning. So you kind of get into this weird situation. So the lessons learned, probably the biggest one is don't duplicate data. There are a number of race conditions that cause real problems for people. It's really hard to keep just instance data in particular. There's a copy of it in the API cell. There's a copy of it within a cell when there's task state changes, VM state changes. You get messages coming down and messages going up at the same time and things just get weird. Everyone needs to use it. It should be the main code path. That way developers are familiar with it. Deployers are familiar with it. You know, it stays well tested. That would be really great. The parent cell for cells v1 still doesn't scale well because there is a copy of every instance at that top-level API cell. There's no path forward for that. If you got to a million instances in your deployment, you'd still probably hit your message queue issues for the API cell, your database issues for the API cell. So the v2 architecture, we're starting with some simple principles based on the lessons learned there. Probably the biggest one is data only in one place. We don't ever want to have to duplicate data and therefore we don't ever want to have to keep the data in sync. If we keep it in one place, it's always what it should be. There should only be one way to deploy Nova. So v2 is going to be the way to do Nova. And we want as little global data as possible in order to keep that top-level API database very small. Let's keep as much as possible within the cell. So a little more about each of that data only in one place. One of the things we're doing to accomplish this is we've stood up this new Nova API database. Any of you who have deployed Metaka have run into this. You're well aware of this by now. So that lives in the API cell. That will store data that we consider to be global, like flavors, right? If you go to boot an instance, you have a flavor. That flavor is not specific to any one cell. You know, theoretically that flavor could go into any cell. So we keep that data at the top level. There's going to be other data that's up there. We don't have that all sorted out now. We're working on that in Newton. We're actually going to discuss some of that on Wednesday in the design summit. But anything that we can think of as global is going to live in that top level database. We also store mappings to cells, instances, and computes. I'll get into the architecture a little bit later, but basically because instance data is not replicated between the API database and the cell database, when you make a request for an instance, we need to know where that instance data is. And so we have a mapping that says, this instance lives in this cell, go query that database, get it, and return it. Only one way to play out. So it's basically Nova now powered by cells. No more decisions. Should we use cells? Should we not? We actually probably should stop even saying cells at some point and it's just Nova. So and all features will work. That's going to be a big one for people using V1 right now who really want that. There's a little caveat, which is NovaNet. We'll talk about that in a little bit. In as little global data as possible, basically we just have these small pointers to heavy data in each cell. And as much as possible, we want to avoid putting things in the API database that scale with the number of instances or hosts. So putting it all together, here's what cells V2 is looking like right now. This is the path that we're on. You have an API cell at the top. It contains your Nova API services, your conductors, and they have a very specific purpose. And you know the API database and your schedulers. That's all going to live at the top. You have this kind of special cell zero that you see that's not really at the same level as the other cells. It contains a Nova database, but nothing else. No conductor, no message queue, no hosts, nothing. Essentially because we don't write an instance in the API database, it has to live within a cell. If you encounter a scheduler failure, if somebody tries to boot an instance and all your cells are full, and it says, I can't put this in any cell. We need a place to put that, and that's cell zero. It doesn't actually need to be a separate host. If you don't want it to, you could co-locate that database with your Nova API database if that makes it easy. But we just have this concept of a cell zero. And then you have your cell foo and cell bar. Basically each cell is conductors, computes a database, a message queue, some of the other services, the console, and some of the other things will have to work out exactly how those live, but you won't have a scheduler within each of the cells. It's mostly the database message queue and computes. So what's been done? Within Mataka, we created a Nova API database. So that's now a second database to be stood up, to be managed. It has its own series of database migrations to get run. And more and more things are going to be put into that database and migrated into that database, but not a ton. We have this concept of database connection switching. You know, if you run Nova now and you go to query where an instance is, it's in the one database that exists. What we are going to need is this ability to say, I have three databases for each of my three cells. I need to be able to connect to this one in particular and pull the data. So we had to write some code to do that. That's merged. And we have a few upgrade tools in place. They're experimental, but they kind of let you play around with it, test it out. They allow us upstream to write some testing jobs that test the upgrades and make sure it works. So that's what was done in Mataka. In Newton, there's a whole lot going on. So what has happened already is we've done a flavor migration to move them out of the cell database and into the Nova API database. That was a fair amount of work, but now flavors are up there. That's pretty cool. Currently, I'm working on the boot process and scheduling. What is going to happen now is when you go to boot an instance, the request comes into Nova API. We're going to store some data, enough data that you could list or show that instance, but we don't actually write the instance to the database yet. We have to go and query the scheduler. That's the job of the Nova conductor that lives in the API cell. It's actually going to hold onto that data. It's going to talk to the scheduler, say, where should I put this instance? When it gets a response, it's going to go and write an instance into that cell and then send the request to do the rest of the boot. Cell zero is... That's another work in progress. It's mostly standing up a database. This database looks like the Nova cell database. It's an exact copy of any other Nova cell database. It's essentially just standing it up and creating a mapping that says this is cell zero, and we're going to have a very simple command for doing that. There are more and simpler upgrade tools. Actually, a bunch of them are up for review right now, so they haven't quite merged yet. They are much easier to use. For the most part, it's just one command you need to run and it will set up everything for you. But those are in progress and we're trying to simplify those as much as possible. Message queue connection switching is another one. We have it for the database. We don't yet have it for the message queue, but we need the ability, like with the database, to say, I would like to reboot this instance over here, throw that reboot message on the message queue for that particular cell that the instance is in. So that's another patch that's up and in progress right now. There are going to be more migrations to the Nova API database. We don't know how many yet. Like I said, we're going to talk about that on Wednesday. Aggregates are one, we're pretty sure is going to move up there. Quotas are another one. We're pretty sure is going to move up there. There's some open questions about other ones. Like key pairs is another likely candidate, but we haven't quite decided yet. So we'll keep everybody up to date as that happens. And then we're beginning multiple cell support. Kind of the goal for Newton, I want to say neutron. The goal for Newton is that by the end of it, everybody's running cells V2. It looks and acts like a cell. You have separation between the API and this cell. It's treated as a cell, but we don't yet have all the pieces in place to use the database connection switching, use the message queue connection switching. So if you stood up two cells, some operations might work and some others might not. We might have reboot figured out, but not resize, maybe. So we're going to start the work on multiple cell support, but we'll see where we get and beyond. So afternoon, we'll see where we're at, but some of the things we would like to do is more scheduling control for cells. People who run cells V1 right now have maybe not great control for that cell's top level scheduler, but they do have some control. They can say, you know, I have flavor A only in this cell and flavor B only in that cell. And so when a request comes in, the first thing the scheduler does is rule out half the hosts because it's not even going to look at, you know, whatever cell doesn't have that flavor. Whereas the scheduler that we have for cells V2 currently still just looks at all of the hosts in aggregate. It doesn't yet have the ability to say, I have some properties on this cell that I would like to take advantage of for scheduling. So that's something we're going to be working on. We're going to finish multiple cell support. You know, that's very important. That's kind of the reason we're doing cells is to be able to have this support. So until that's done, that's where we're going to be focusing on to a large extent. And we're going to work on cells V1 to V2 upgrade tools. You have not been forgotten. I know that's a huge concern for V1 people. We're trying to get to it, but we have to have multiple cell support working before we can do that. So that's the only hold up there. The upgrade process. For people who just want a simple upgrade, they're simply running Nova right now. They don't necessarily need the scaling aspects yet, but they would eventually like to have them with this very simple two-step upgrade. But it's kind of actually one step in a sense. But in Mitaka, you already would have had to run this new NovaManage API database sync command. And everybody has to do that simple or not. Then what you're going to run after Newton is this cell V2 simple cell setup and pass in a transport URL. That's actually part of... Well, so that will set everything up and it sets it up with the constraints listed at the bottom, which is basically cell zero. The database is going to live on the same, you know, my SQL Postgres server is the Nova API database. It basically pulls the config option for that Nova API database and sets up a new database. All active hosts in your normal Nova API database, or sorry, normal Nova database will be migrated. I should say mapped. Basically, the API cell becomes aware of them. And after that happens, all of the instances that live in that database are also mapped. So the API at that point, you know, it treats it as cells V2. Everything's set up. We would like to make this a little bit simpler. There's this transport URL that has to be passed in. That's because of a limitation in the Oslo messaging library. It reads the configuration option and does a whole bunch of magic on it to make things work. So we can't actually, at this point, kind of reverse engineer that to get the transport URL that you can figure out in a way that cells needs to use it. So it's kind of required to be passed in. We're going to be doing some work around that. So hopefully the command, ultimately, will just be Nova managed cell V2, simple cell setup, or whatever we end up calling it. But right now, this will work if a few patches merge. If you need a little bit more of a custom setup, you know that you want to start scaling a little bit more now and you don't want to co-locate your databases.  And then what you actually do is you figure out where you want cell zero to be and you pass in the database connection string yourself. And that will do the job of setting up and mapping the cell zero to the database. Then you run this map cell and host command where, again, you have to pass in the transport URL and there's a name option which got cut off. But you can actually name your cell and kind of manually make sure the hosts get mapped in there. And then you can run this map instances command and pass in the cell UUID, which you get as a return from the map cell and host command. But you can run it in a number of stages if you'd like, you pass in this max count parameter. It's an optional parameter. But you can just say, like, I want to map 100 instances at a time if you don't want to overwhelm your database as you're doing this migration. So as a result of this, all of your active hosts in the Nova database are mapped and all of your instances are mapped. There are a few caveats. One of the big ones is it looks like a, maybe a bad thing, but it isn't. You look at it a little bit more. Is if you live in a multi-cell world and let's say your database for one of those goes down and you want to list all of the instances in your deployment, we can't return that data because the database is down. However, we can give you partial data.  In the future, you only can't list the instances that are in that database that's down. So compared to V1, we don't have this, you know, at the top level, we don't have all the information to pass back. But again, we avoid all the race conditions by not doing that. So partial listings is going to be a thing that you'll start to see for like the list command. Novelist instances in particular. Networking. So I say in progress for Neutron. Actually, networking with Neutron is going to work with V2 out of the box. As long as what you want to do is just have a flat network across all of your cells. That will work just fine. What's in progress for Neutron is going to be the ability to, they're doing something called segmented networks. It's going to be the ability to kind of tie your segments to your cells if you'd like. So you can kind of, you know, split your networking up and say that this cell has this networking and this other cell has that networking because most people who've deployed V1 find that their networking is a scale bottleneck as well. And so they end up kind of splitting Neutron up behind the scenes and there's a lot of hacks in place to make that work for V1. It's not something that works upstream. So we're trying to work with Neutron to get that to be just a recognized thing and something that works for everybody. Novenet is not really planned to be supported. It will work with one cell. There's a little bit of work we might do that makes it work for the most part with multiple cells, assuming that you're again okay with this global flat network across all of your cells. And it's probably also going to tie Novenet to whatever your first cell is in particular so that you can never really remove that cell or you lose Novenet in the process. However, most people are running Neutron now. We're hoping this isn't really a thing but it's still under discussion. We haven't made the decision one way or another yet. So if we do need to work on Novenet, you know, we'll go ahead and do that. There's always the unknown unknowns. There may be obstacles. Even with cells V1, it's only scaled up to a certain point. I think the most number of cells people have is maybe in the dozens within a certain region. You know, somebody wants to stand up a thousand cells. We don't quite know what's going to happen there. But we're going to learn. You know, as it scales up, we'll continue to run it. We'll continue to test. We'll continue to fix the issues that come up. So cells V2, expected fall 2016. That's it for me. I can talk about anything in more depth if you want. I just want to kind of do a quick overview. But are there any questions? And if there are questions, would you please go to the microphone so that everybody can hear you? Yeah. It also gets recorded. Talk a little bit about how the scheduler is going to work in reservations with cells V2 or is that out of scope? What do you mean by reservations? So right now, if you make what cells V1, if you want to build an instance, it'll go from the global cell into the child cell. And then if there's no capacity there, then it's dead. It's lost. And so there's this concept of host reservations that's being talked about upstream. And I'm kind of curious how that's going to play with cells V2. So it sounds to me like the need for host reservations comes from the fact that there's a time lag between when the scheduler within a cell knows that you're out of capacity and when the cell scheduler knows that you're out of capacity. And in cells V2, where you don't have that two layers of scheduling, it seems like the problem just kind of goes away, right? Like if the first scheduler you go to knows that you're out of capacity, it's not going to say, yes, you can get past here and then fail later. Do you really need that concept of reservations? Okay, so that in that case then it's not really something that's required or in scope for cells V2. Right, as far as my understanding. I'm sure I didn't explain it all that well that everybody knows exactly what's going to happen, so please, if there are questions. Can you talk a little bit about partial listings that you mentioned briefly, but how's that going to work exactly?  And you have 10 instances in one cell, 10 instances in another. And the database for one of those cells goes down. What's going to happen if you say, you know, list your instances is at the API database we know that you have 20 instances connected to your tenant because we've mapped those instances to each cell. We're going to go talk to the database at the set or the database that's up and return all the information for those 10 instances. The database that's down, all we know is that you have instances with these UUIDs in there. So all we can really return is a list of those UUIDs and say, this is incomplete. So we can give you all of your UUIDs, but we may not be able to give you things like this is the flavor that it was booted with or, you know, this is the RAM that it has, this is the metadata that it has. And that would just be if the database was down, it wouldn't be in your normal condition. Right, this is only if the database has gone down, you know, which hopefully, you know, an operator is going to see that, try to get it back up quickly. But yes, this is just in the abnormal. Something has gone wrong case, not normal. I got a question. Okay. You said you eliminated the two layers of scheduling, which is a great thing. When a request comes in, though, how will the API cell make a determination of the conductor to pass the scheduling request to any particular cell? The scheduling request does not get passed to any particular cell. The scheduler lives at the API level, right? Yeah, the boot request. The boot request. And so then how does it pick a cell to then get scheduled? So you're saying after the scheduling, how does the boot request continue on to the cell? Yeah, I do a boot request to the API. It's got to go to a cell. Well, the boot request goes to a nova conductor that lives at the API level. Right. So it's going to talk to the scheduler. The scheduler knows about all the hosts in the deployment. It's not picking a cell, it's still picking a host for you. Okay. And then once it has a host, we say this host is in cell A. We write the instance to that database and then through RPC send it on. Okay, thanks. If you lose cell zero, do you lose everything underneath it? No, all you lose are the instances that were not able to be scheduled into any other cell. And the cell zero is really just a holding ground until those instances can be deleted, right? Like, because they're going to be in an error state, they weren't schedulable. All you can really do with them is show them in the API and delete them. So hopefully they won't even stick around that long there for most users. So that's a good question. And what he asked is, can you run cells with different nova versions? I should have put that in the and beyond slide. The long answer or the short answer is yes. The longer answer is there's some work that needs to go into being able to make that happen because you have to have the nova API database understand how to talk to databases that might have schemas at two different versions. And you need to be able to communicate kind of lowest common denominator over RPC. For all of that, we have, I've talked with a few people about ways we can go about doing that. We do plan to make that happen. It's, you know, a huge part of the upgrade story, right? Somebody's going to want to just take one cell, upgrade it, make sure everything works, and then roll that out across all of their cells. So it is planned, yes. Going back to the scheduler, and what about the scheduling based on affinity? Will you be able to do affinity based scheduling with this approach? At the moment, yes. Because again, the essentially what I'm saying right now is that the scheduler is going to have no concept of cells built into it. It's still going to know about all of the hosts in the deployment. It's going to work exactly the same way that it works today. Essentially nothing changes for it. So affinity will still work at this point, yes. Of a VM that lives on a different host than another VM. So anti affinity rule for every per VM. But you don't have the VM information on the API level, if I understand. Only the host information, and the mapping of the VM to the cells. The API, yes. Nova API only has access to the database at Nova API, which only knows about that information. But the Nova scheduler is still going to be privy to all of that information. It's still, right now what happens is there's a mechanism in place that Nova computes, say these are all the instances that I have, and they pass that information up to the scheduler. And that's not going to change at this point. So all of your hosts across all of your deployment are still telling the scheduler everything. Now, real quick I will say, you know, there are people in cells V1 that really like this two level architecture or may want it for scaling or because they don't want to change the filters and weights that they already have. So there's talk about changing the scheduler to kind of work that way. But what would happen, I think, is that the scheduler API would not change. And so for most people, if they want to do global scheduling across all of their hosts, they could continue to do that. The people who want this two level scheduling would actually plug it in in the same place. And instead of calling the scheduler twice, you would have Nova API, the conductor called to the scheduler, and then it would make a decision and call to a subscheduler. But from the Nova perspective, it looks exactly the same. Yes? So you mentioned the assumption on Neutron, assuming a flat network. So from a scalability point of view, the cells sort of solve the scalability of computer Nova. Is there a co-design with Neutron to do similar kind of segregation and scaling? Can you elaborate a little bit more on how that will happen? There's not, as far as I know, a plan for Neutron to scale its database or any messaging system in the same way that Nova has done. I think Neutron right now is more focused on scaling, I guess, what you'd call the data plane for that side and making it so you can have your actual physical networks be segregated in such a way and scale those. Your broadcast domain is a little bit smaller than being totally global. However, I'm not at this point aware of any effort they've gone to to do the same type of message cure database sharding that we are. There's been initial talks of conversation, but I don't think they're in a place to really push on that yet. So we can assume that in the future, cell may somehow map to the network topology that is created by Neutron. So Neutron sees all the cells the same as the one sort of giant pool of Nova nodes. Yes, their concept right now is what they're calling routed networks or segmented networks. And it's actually flexible enough that you don't have to map it to a cell in particular, but I suspect most people are because their physical deployment infrastructure is going to look very much the same on the compute side and the networking side. But again, it's just about kind of scaling the broadcast domains there, not making the Neutron deployment similar to the Nova cells deployment. And we are actually in any of the discussions we've had with Neutron, we're trying very hard to make it so that Neutron doesn't have to follow what Nova does in terms of cells architecture. We want them to be very free to do what they want to do. Thanks. Hi, two questions. You mentioned the aggregates. So host aggregates, server groups, all that is moving into the API database? Host aggregates are... I don't know about server groups to think on that. I'm not sure yet is what I'd say. If it makes sense to do it, it will. There was a very specific reason we moved host aggregates and had something to do with... I'm trying to remember now. Basically, we're using host aggregates as a way to map into this new concept of resource pools. Which is getting into some very low-level details of Nova. But that's actually what's going to be used to tie into Neutron to be able to make the segment to networks and Nova cells work together and also do a whole lot of other nice things for the Nova scheduler. So aggregates are a way to map that. If server groups don't need to be used for anything like that, they may end up living within the cell's database. Used for affinity and anti-affinity. So I think if the global scheduler is going to allow affinity and anti-affinity, you're going to have to have those up there. Okay. Yeah, it's one I haven't actually looked at specifically. But again, if the data is available to the scheduler from within that child cell, from within that database, or if it's pushed up to the scheduler, it's kind of one of those things where we want to balance not putting too much information within the API database. So if we don't need to push it up, we won't. However, if it ends up that we need it for some reason, we'll do it. I just don't know for that one specifically. Okay, and my other question, you just mentioned a little bit ago that compute nodes we're going to report up to the global scheduler about capacity, about what instances are on each host, just like in the normal, in the current architecture. And my question there is about scaling. Is each compute node then going to have a connection up into the global? Or you said there is no global message queue. So I guess the question is how do those compute nodes communicate that information up to the scheduler? Yeah, that's a good question. I don't know that I haven't answered for that yet. So getting into more of a long story, I will say the direction ultimately is that the Nova scheduler is going to kind of get split off to the side where it's going to have its own database and its own message queue. And it's really going to kind of be like separated from Nova. Even if it's not pulled out as a separate project on its own, we're trying to separate it as much as possible. So we may end up with a message queue that has to run just for the scheduler in order to continue passing that information up. That's maybe one of those unknown unknowns that now is a known unknown. Okay, thanks. I forget when this goes to, but we might be at time. Okay, cool. Thank you, everybody.


A warm welcome to the third lecture on the subject of digital signal processing and its applications. Let us take a couple of minutes to review what we did in the previous lecture. In the previous lecture, we had initiated our discussion on sampling. We had tried to answer the question, when can we retain information even after sampling? To expand a little bit, when can we consider values of a signal only at sampling instance and yet not create any confusion or not lose information? We had not completed our answer to that question. We have only got thus far and we shall now summarize that. So let us put down the theme of our lecture today, reconstruction from sampling or from samples to be more appropriate, aliasing and discrete systems. This is the theme of our lecture today. Well, what we had seen the last time is that, if you have an original sinusoid of frequency 0.1 kHz and if you have a sampling rate of 1 kHz, then you lined up with a confusion of sine waves and the confusion is as follows. On the kHz scale, the samples could have come either from a 0.1 kHz frequency or from a 1.1 or a 0.9 kHz frequency and of course, if the amplitude of this is a0 and the phase phi0, the amplitude of this is a0, phase phi0, the amplitude of this a0, phase minus phi0, the amplitude of this at 2.1 is a0, phase phi0 and we can keep doing this, amplitude a0, phase minus phi0 and we can continue. So at every multiple of 2 kHz, we have these confusing sine waves if you want to call them that which contribute to the same samples. Now I want to say a little more about this observation. You see, what we had shown in the previous lecture was that all these sine waves generated the same samples at those instance. However, we need to show something more. We need to show that if I indeed add up these sine waves as I have shown here with amplitudes a0 and phase either phi0 or minus phi0 depending on whether it is after the sampling frequency or before, if I add up all these sine waves, I generate what I really call a train of samples and what do you mean by a train of samples. In fact, this is a challenge which I put before you. It is a practice that I follow in courses that I teach to challenge your imagination right from the beginning to put before you exercises which will make you think and work certain things out on your own so that you get a much better grasp of the ideas that are being taught. And we are beginning with the first challenge, take these sine waves and add them one by one. What I mean by that is take the sum a0 cos omega nut nt plus phi0 or in fact let us put sample you know at nt of course you will get infinity there is nothing much to be seen but let us instead add up the continuous so you know we should say not these sinusoid at the samples but sinusoids the continuous sinusoids so 2 pi by t as we would have it minus omega0 times t minus phi0 plus a0 cos 2 pi by t plus omega0 times t plus phi0 plus now what will come next let me do it for a few terms for you so what I mean is you see you have taken the original sine wave as a continuous sine wave here you have taken the next one the next one is the sampling frequency in radians per second 2 pi by t minus omega0 and then the phase would be minus phi0 sampling frequency in radians plus omega0 plus phi0 now the next term would be 2 times the sampling frequency in radians minus phi0 and the next term after that would be 2 pi by t times 2 plus phi0 let me write down 2 more terms to make the concept clear let me write down completely you can keep on writing more and more terms anyway we got the basic idea now the challenge before you is to prove that as you add more and more terms you have a constructive interference at the point of sampling and a destructive interference at all points other than that of sample in other words what I want you to prove in the challenge and let me write that down prove that this sum is constructive at the points of sampling that means t equal to nt and destructive at others so what is going to happen as you add more and more terms essentially this sum is going to become localized near the points of sample it is going to rise higher and higher in fact it is very easy to see that at the point of sampling the sum is divergent it would go towards infinity we knew right in the beginning from the previous lecture that the samples are the same when they come from all of these sine waves so at the points of sampling of course constructive interference is obvious because all the samples are the same all the values all the sine waves take the same values the key thing is to prove that at other points the sum is destructive that means as you add more and more and more and more terms they all compete and cancel one another out so I leave this as a challenge to you and I give you a hint you know you will have to use trigonometric identities you are summing up sine waves of the same amplitude of different frequencies use trigonometric identities and show and try and see what they are converging to what that sum converges to if you like you could decompose a sine wave into a sum of 2 oppositely rotating complex numbers as you often do so that is also acceptable we will be dwelling on that idea in more detail later they are called phasers anyway for the moment I am putting this before you as a challenge why I am saying this is because it tells you what happens when you sample when you sample what you are doing is essentially to create small pulses at the instance of sampling of value or strength equal to the sample value of the waveform at that point at all other points you essentially have a 0 signal so sampling means to replace a continuous signal by a train of pulses.