Hello. For those who don't know me, I'm Enrico from Italy. And I'm going to talk about a little kind of, sorry, well, I'm going to talk about a sort of prototype system I made to publish Debian information called Debian Data Export. So the point of this is Debian is a bit of a data hell. We have a lot, a lot, a lot of different kinds of data in Debian package files for binary packages, which are in RFC A22-like format split by distribution then by architecture or for source packages. Well, it's the same. The mapping between maintainer and source packages is available in a few places and in different formats. Depth tags information has its own format as well. The depth tags, whether we have the depth tags vocabulary of tags, extra sources of depth tags information, popcorn rankings that can be useful to have so that packages can be sorted by popularity. Bug information, you can access the bug tracking system in several ways, all with different formats. Package change log desktop files for packages not installed is another thing that people may need to use for features like other new application in this menu. Information about what's in the new queue, package screenshots, APT file information, statistics about the archive, license information, localization information, status of use can't build the logs, size and lines of code of packages, Debian weather reports. Yes, you know the Debian weather reports. They tell you how installable is unstable today. So if you can upgrade or if you are going to end up into a dependency mess, Debian information specific to Debian Pure Blend, what flavor of Debian this package is used in, UDD, which has all of those and no, not all of those, but is yet another data source of access in a different way. I have no idea because I'm not on the topic. You see the weather. OK, I have no idea because also you're not asking. You're saying that it's used there. I need to look for it. It's a data help. You need to dig for all of these things and find out where they are, what's their format, what's the protocol to download them. So you can probably you can think of more like Joy has been showing graphs of how the helper is used. And yet, you know, yet more information that we have that most of us generate that statistics about something in Debian and put them somewhere and then we are happy with it. And then no one else uses them except sometimes people mention them somewhere and people are like, I have been looking for that for years. So it's a bit of a shame because some of these things could potentially be used like by package managers to tell these packages bugs open as RC bugs open since more than a month that package has less RC bugs. But the other package is not as popular. You could make use of all this information in an interface. So and then besides, you know, the data itself, there's formats. So we got RFC A22 like files used for several things which has a sub format for package descriptions which, you know, empty line starts with a dot and to make a bullet list, it's kind of wiki like tags have a sub format as well in the packages file dependencies. They also have a sub format. You got soap interfaces, LDAP interfaces, SQL interfaces. Plenty ad hoc formats. There's HTML scraping going on. I think VAC made an HTML scraping interface to the BTS or something or to something else. And then where do you find the data? Sometimes it's on the mirror. Sometimes it's in a directory on people. That would be an org or sometimes you need to log into the right machine that will have it in the file system. Sometimes it's elsewhere outside of the debut network. Sometimes you can only access it from specific machines like if I'm setting up a package search interface on my own server. I cannot access UDD, for example. You can only download it from the server. Something you can actually compute it on the user system to relieve servers of computational things. So obviously, if you want to know what other, if you want to know popularity information, you need to get it from the server. But if you want to know if a package is installable, you need to compute it on the user's machine to be really precise because they may have a different sources list. Something can be computed only by cooperation between the user machine and the server, such as package recommendation. Because in order to compute that, you need to index all the popularity contest submission and then query that using information about what packages are installed in the user system. So we've got a bit of all of that. And so I want to start writing a replacement website for the Deptec submission, the Deptec stack submission page. And I usually, or even in the existing one, I usually have the problem that I need to collect a lot of data, such as information about all packages in every architecture on specific releases. And so, well, I had the problem of, how do I get all the data that I need kind of got bored to go and track all the places where I should take it out from? And then I realized that the information that I was collecting myself was used by other people. So, and I had a piece of code ready from other projects. And I thought it all fits together. So I made this prototype. My goal is that producing data should be easy. The major task should be computing the data. And all the rest should be kind of easy to do. Finding data should be easy as well. And downloading the data should also be easy in terms of protocols and in terms of format. That was the idea that I'm attacking here. Both to try and use it in Deptecs.debian.net, which I would like to allow people to tag all packages in Debian, Ubuntu, Pure Berlin, possibly other derivatives on all architectures. Or I would like to make also completion in web form fields. It's, you know, it's a few, well, it's some lines of JavaScript, but the problem is getting the JSON information. And I would like also to have to get to build some machine readable interface to all the data that I produce. I produce, like, Deptecs information. And, well, I wanted, at the moment, you need to know the URL and get it. I would like to make, like, a convenient interface. So the prototype, which I'm going to demo. And I will impossibly increase the font size and hope that it works. OK, I'm gone. OK, like this. Well, OK, the solution is DDE, well, the solution. The prototype I made is DDE, Debian Data Export, which blah, which is kind of like a file system. Well, it's like a tree. Well, OK. It's like a tree of where, well, you got plugins. Generate or access data. And I publish it in a file system kind of way as branches of a tree. And you can query every bit of the tree, and it will give you data. So for example, I can list the, well, I made a plugin to export the polygen grammars installed in the system. And then you just, oh my, I just say get a grammar, and it will automatically run polygen and give you the output. That's a kind of simple thing. Or you can ask it, give me the same output in JSON format, and it will do that. Or give me the same output in YAML format, and it will do that. So the plugins kind of produce, it's written in Python. The plugins produce a Python structure with the data to export, and the front end will encode it in whatever the user requires. And that kind of solves a bit the format issue, because when you query it, you just ask it, give me the data in this format, and it will do that. So you can list what's available, and you can get a value, and you can also ask for documentation about, well, not here, but the plugins allow to get documentation about, even here, allow you to get documentation about every single piece of the tree. So it's all self-documenting. You can browse the tree and know what's around and know what you're getting. Now, this is kind of command line, and it's a bit awkward to use. Well, you just run it dash-desk-server, and it blows up because I'm already running it with dash-desk-server somewhere else. When I'm OK, can I have a browser? You can run it as a server, and you get an interface that you can browse to see what data is around. It tells you about the available output types. So you can get out. Is it? At the moment, there's output formats for CSV, JSON, self-documenting HTML pages, Python, Pico, Yamla, and text-based presentation just to see what goes on. So anything that every piece of data that you see in this tree, you can query it in all these kind of formats. And when it works as a web server, it's restful. So the tree is directly mapped into URLs. So the way it works here is you go around the tree, and you're like, OK. Oh, there's nice Polygen grammars. What are these grammars? OK. The description of each grammar is generated using Polygen. So everything the plugins do is completely dynamic. And then you get the information, and you add the question mark type equal JSON, and you get the JSON to download. So basically, you go around the tree, and you say, OK, this is exactly the data I'm looking for. Go away. This is exactly the data I'm looking for. Then you take that URL, add the data format you want, and use it in whatever software you have. Now, it exports JSON, which means you can use it from a web page, from JavaScript. You can make mashups and whatnot just by getting the data. In this case, another example is app package information. So at this level, you get information about all packages, and you can go down and get information about a single package. So depending how high you are querying the tree, you get more data. The more you go down, you get specific data. You need to download the whole package list then you query at a higher level. You want information about only one package, you query at a lower level. And when you browse around, you get an example of what you get. So when you are trying to create your JavaScript thing, then you can look around. Or you have, well, completion for binary package names. I didn't put the documentation yet over here, I just add it to the plugin and it works. And I can put a prefix and it will give me all the depths that have that prefix. There are several plugins written. For example, I made a plugin for the apt-exapian index. I don't know if you heard of it. I blogged about it heavily some time ago. It's a big index of package information, tags, and things like that. That is extremely, extremely versatile. You can do a lot with it. And you can see you go here and it tells you what's around. You can put a query with keywords, say an image editor in the URL. And then you get a data structure very fast with a tag cloud related to those features, possible completions, corrections, list of packages that match, suggestion for extra keywords to add, and lots of things like that. All nicely in JSON. If you want to implement an HTML, if you want to implement a dynamic page using it, it's easy to do. There you have it. So it starts with a tag cloud. As I type, everything changes. I have possible completions and suggestions and packages. And you can see it's pretty fast because the back end is very fast. And this is a static HTML page. Or for example, you can use it to complete package names in pretty much every web form. So there you go. Pretty much on the fly. The source of this is a static HTML page. It can be put in the Debian website. The source of this is basically an HTML input. You give it a class, and you tell it from what DDE server to get information. And it works. It also includes the little bit of JavaScript. But that can already be put in every Debian web page, and you automatically get completion with proper full-back for browsers that don't have JavaScript. And whereas if a package manager wants to download popcorn information, one can make a popcorn export. And it's very easy to query. The data space that is exported does not try to export every single thing. It does not try to be a super generic, big relational database of everything. We have UDD for that. It just tries to export views corresponding to common use cases. The idea is that, well, you have a special need. Then you craft a SQL query for UDD, or a nailed up query, or whatever. And if it becomes a general need, that query can be turned into a plugin for UDD. And then it's readily accessible in whatever format to pretty much everyone in a self-documenting way. Current users of UDD, I've put online the Aptx Appian index version in Aptx.debian.net. Screenshots.debian.net uses DDE to get a package information for its database. It can be used to make completion in web forms. And it can be used to emulate apt-file without having a local file information. For example, this is a little script that just queries DDE for any package that contains bash and gives you the result right away without downloading all the package information. It's not a drop-in replacement for apt-file because it doesn't support regular expressions because the way I index it to make it super fast, no, but the apt-file author is here at Debconf and I need to catch up. And the idea was to not include it into apt-file, but if there's no local database, apt-file can tell you there's no local database.  So obviously, this can be used to add extra features into package managers. apt can make a fetcher to get data out of it, although it may not be a good idea because its data is dynamically generated if we start operating it with lots of apps. I don't know if it copes with the load. It can be used to feed more external websites that show Debian data. For example, if someone makes a corporate distribution and has a website showing the list of packages, even in the internet, and they want to feed that package database with information coming from Debian, they can pull it out of DDE without problems with parsing it and so on. Or different flavors of Debian can put online a DDE of their own, which only shows a personalized view of the package database so people can set up DDE instances to provide different views. And if they keep the same tree layout, it will be just a matter of switching the software that uses the data to use a different URL. In terms of deployment, it's a Python WSGI application, which, in theory, you can deploy in any way you like because WSGI is an extremely thin, small and simple interface that you can put into a CGI, you can put it into a web server, you can put it into pretty much everything. In practice, if you deploy it as a CGI, it does not scale. If you deploy it as a standalone web server with Python modules that turn a WSGI application into a web server, you'll have all sort of problems because none of those really have a clue about the streaming data. There is one that has a clue about streaming data, which is cherry pie three, but it doesn't have a clue about dependencies because it conflicts with cherry pie two, which is needed by pretty much anything else because they made a complete incompatible rewrite without changing the module name. If any of them are listening today, that may explain why things like turbogears are switching away from it because it's a hell to live with. Python paste, if you ask it to reload the web server, it will kill all the current streaming, which is not nice. Fast CGI needs careful tuning because it tends to say this CGI process has been running for more than five minutes, I'll kill it. And maybe you are streaming the whole Debian package information to a slow machine. So, you know, a DD query may take 20 minutes, as far as I can tell, because it doesn't generate the data set locally, it can stream information. And in the other hand, you can have something that processes the data as it streams. And mod WSDI runs in the apache process space, which does not sound to do. So ideas about deploying anything like that, which is basically a restful dynamic interface to all sorts of data would be appreciated. In terms of scalability, it's already done only, and there is no state. So it's a wonderland for cache, which is excellent. You can put varnish in front of it and be happy. You can use aggressive cache adders like everything I give you is valid for six hours, because most of the information is generated, it changes when you have an archive run. So it can be replicated, it can use DNS round robins because there's no state. But obviously, if all web forms make a lot of small queries, and there's lots of people using that feature, it may not scale. So it all needs to be seen if app starts getting data out of DD for some nice use case, maybe all package managers will kind of kill the thing. Since it's dynamic, you can actually use the mirror network because it's generated on the fly every time. Nice things that could be done is JavaScript workshops that get data from DDE is running in multiple places or they run on a server and get data from a DD running on a different server. This is currently an issue because the security model of JavaScript does not allow you to query information from a different server than where you come from. And that's very annoying. Firefox 3.1 should have support for a new webby protocol thingy that allows you to do that. But, well, that's kind of new and it's only Firefox. DD supports JSONP, which is a standardized attack against the HTML security model to allow you to get information from elsewhere. And it's absolutely painful. They also, I can even add JSONPP, which is an even worse attack against the JavaScript security model, but that road makes me sick. And in terms of future development, it's mostly on a works for me at the moment. I will say it's a couple of inches of mine, but I won't do all the work because I don't have the time. I thought it was cool to provide completion for package naming web forms and I made it. And so it could be nice if later I can talk to someone and it kind of, oh, sorry. No, no, I'm just stupid, don't worry. It's all right. But if you want to upload, if you want to share extra stuff to it, please write a plugin. I'll be glad to show you how. Existing DD is DDdebiannet slash DD. Slash DD, this runs on Debian machines. Oh, I guess if you want it, I can paste it on, I can paste the link on IRC. DD.debian.net slash DD, as soon as they find IRC. Okay. And it currently exports apt-file information that index I showed you before to feed data to our apt-file, it exports data out of UDD. There's different views on the big, big package list and you can get package information, say all the packages on ARM for every distribution and part of the archive or you can, again, go down. And so at this level, you get basically every package in Debian or Ubuntu, in any version of them, in every architecture of them, anything. And as you go down, you can limit things to one architecture or one distribution or only main. Ah, that's a full-screen terminal that shows up. That's DD, which is low in responding, unfortunately. But you could do that. And I made a sandbox, well, not quite a sandbox. I made an experiment with publishing information, publishing static information. If you, I need to find a blog post of mine. Basically, if in your own home, in people.debian.org or on merkel.debian.org, create a .DDE directory and put stuff into it, it will be published in all possible data for months and so on. So for example, I put some in my, okay, this explains how to do it, so it's not long ago, it don't remember anymore. But again, I don't remember it anymore and it's kind of self-documenting, except the page does not exist. Why? It's now called static data. Okay. So you just put, sorry, in this directory on merkel, you put a file.jamler or .json and it will show up. So a very simple and really nice thing to do now, if you are generating any sort of daily statistics about the archive, make it in Python pickle or .jamler or .json, put it in that directory and it will automatically be exported as part of the .de.dev.net today and that works and that sorts out the problem of how to publish data and that this is a call for everyone who's generating any statistics to just do that. I'll put this on IRC as well. Yeah, it supports .jamler, .json and pickle and you just put different file name extensions and it will do the right thing for you. You can build your own hierarchies. There is a way to add documentation to whatever you put online. You just put a file .doc.jamler or whatever that describes the data and again, it will appear when you browse those three it will tell what it is and it's all documented here. There is a sandbox directory if you want to play with it and everything you put is mounted inside the DD tree and if two people put files with the same name since they are mounted in the same namespace, the first person gets it and the second person is ignored so if you have something to publish with a fantastically common name, hurry up and then we have another DD at .debtakes.dev.net which has the apt-xapian index of all Debian packages and also Wuntu packages. So if you want to do something like this, you feed it with the .debtakes.dev.net slash DD and you can already use it, generate TAC clouds and whatnot and of course the .debtakes.dev.net got the Polygen plugin which is extremely important. Again, the interface is self-documenting, it tells you how to use it by a JavaScript so I don't need to explain it, you just go and you say, okay, I will use JSON and it tells you, you know, this way. So okay, those are the two instances, the two public instances that can already be used nowadays and I think I mostly, I think I've shown all and I can take questions. I guess my question is, yeah, it's on, I'm just not talking very loud. I wonder how many things in DD actually require dynamically query data at runtime versus how many things, I mean, all the data that you're having people upload to the sandbox is just static data. So I think it's from an interesting design choice that you chose to let it pull in runtime data, do you need it for the UDD and things like that or could you do it entirely statically for most things? I'd welcome ideas on how to do it entirely statically because since if you have like a five level hierarchy you are basically generating, I mean, if you render that statically, you have a lot of stuff because, you know, different views on things, it can be kind of fixed by limiting the amount of views we provide to only those that are useful, which is a perfectly nice approach. And then you need to render them once for JSON, once for YAML, once for Pico. So if you look at the data space that it's dynamically generated, it's pretty big. Unfortunately, package name completions cannot be rendered statically, they can just be cached in memory with trees and so on. So yeah, it's a bit tricky. Also the static information that you can publish, it's read into memory because you can query, you can slice it when you query. So you can publish a hierarchy of directory or well, hierarchy of directory would be static but you can publish a YAML dictionary and then you can query only one entry of the YAML dictionary or if you publish a hierarchical JSON structure then you can query it at any point. So I don't know about one interesting thing that could be looking into new stuff like MongoDB. Maybe something can be done because they have these ways of indexing JSON documents and then querying them very, very fast. I don't know if that will help, I haven't looked too much into it yet. But yeah, the data space is huge. I'd like it to be more statical but then it's probably less useful. So I'm not in two minds over there. But certainly for example, for those very slow UDD queries, those can be terming to, at least the query can be made once and then hashed somehow. That can definitely, that can definitely be done. Or actually instead of using UDD, it could be an interesting idea to generate a MongoDB of the end package information. And then query it with their own fast stuff. Microphones, would it be possible to feed images into this? Say I'm generating some data that I've collected and I would just like to put it in too. So I don't have to store it somewhere else. You could if you can and call JSON or whatever. Well, I would like to include it as a thing. I guess the easiest way is you generate a static place online with the images and then you spit URLs to it, which is what can be done for Debian org. You have a view where you can ask where's the screen of this package and any guy URL, which you can use then from JavaScript and package managers, whatever. I have a question. In Edinburgh, we had to talk about data mining popcorn. You remember and are these data anywhere restored and can I have access with DDE onto this data? No, you can't because DDE has no access control whatsoever and the data mining popcorn thing gives you a sort of limited view over the popcorn information that we can't really package. Sorry, we can't really share because it may lead you to know something about what someone has installed. No, I mean, it's results we get. I just posted the link because not everybody here. I just want to, because this data mining was done and you got the results. Users who installed package full have also installed package bar. That can definitely be public. There can definitely be a plugin for it. Yes, absolutely. Is anybody here who knows where these data are? I just hear the talk and then nothing since two years. I know, I didn't do that. Okay. It was somewhere else. It was Alan Schröder. Yeah. I just posted the link to this talk but I haven't heard anything about this for me, quite interesting data. It's interesting but I didn't chase it and if we managed to chase him and get the data out and if the data is still computed, it's definitely something that can be posted on the IDE because you put it in some index somewhere and then you just offer a restful interface to it and done. I think it would be very interesting for if you even have a plugin in Synaptics also. And then you put it there and then you know where to find it. Yeah, okay. Okay, thank you. Anybody else? Okay, I think if there's no more questions, there's a few links posted on IRC. These slides will be available and and if you couldn't see over there, you can see it back in streaming. Fast forward only where you see this slide. Okay, thank you very much.


OK, welcome. My name is Greg Waynes. And this talk is basically to share Starling X teams' experiences in evaluating using OS Tree to do parallel atomic software upgrades on our Starling X cloud servers. So the abstract to paraphrase, OS Tree is basically an upgrade system for Linux-based operating systems or Linux-based deployments. I believe it's typically been more used in embedded operating systems. But in STX-8 of Starling X, we introduced OS Tree, and we're using it for patches in STX-8. And now we're starting to evaluate it for full software release upgrades of Starling X. So in these slides, I'm not an OS Tree expert, but I'm going to give an overview of OS Tree, what it is, how it works, how upgrades are done with OS Tree. And then I'm going to take a look at how we're proposing and looking at it to use it in Starling X, and then what's the impact, the positive impact, on upgrades, the elapsed time and outage times for upgrades in Starling X. OK, so myself, Greg Waynes, I'm a principal architect at Wind River for the Wind River Cloud Platform. That's the commercial version of Starling X that Wind River has. I've got 25 years of kind of platform experience kind of dating back to kind of nortel days. So mostly telecom space. I'm a founding member of Starling X. Five years ago, I was kind of here in 2018 when we announced Starling X. And I've worked in various areas of Starling X, some security, some open stack, high availability, software management stuff now, even docs. Last couple of years, I've been on the technical steering committee for Starling X. That's been fun. And then previous to Starling X, I've done very, very minor contributions into Masakari and Horizon. So yeah, so the agenda, I'm basically going to give a brief overview of Starling X, very brief, a single slider. And then, like I said, introduce OS Tree and then talk about how we're actually going to use it in Starling X upgrades. So Starling X, I'm sure you've heard a lot about this and this week. So as you probably know, Starling X is an open source project in the Open Infrastructure Foundation. It basically provides a complete, ready to deploy, fully integrated, private cloud infrastructure management. And as far as infrastructure management, it manages bare metal servers, the OS resources, device configs, manages all the infrastructure software that's running on the servers, so the biggest of which is Kubernetes, manages containerized application services that we have in Starling X. Most of them are other open source projects or services that we've really just packaged and reintegrated into the Starling X. As far as deployments, Starling X has a wide range of deployments. It can be deployed as a standalone cloud and really on a single server standalone cloud. And then it can be multiple nodes in a standalone cloud, the classic controller nodes, worker nodes, storage nodes. But by far, our most popular deployment option is distributed cloud, where you've got geographically distributed sub-clouds. And those in the telcospace, those sub-clouds, are typically single server clouds. And those are autonomous clouds. And like I said, they can be single server. They can be multiple nodes. But they're autonomous clouds on their own just for reliability. And then the central cloud and the distributed cloud environment is really there for automation and orchestration of managing the infrastructure across all the remote sub-clouds. So that's Starling X, OS Tree. So as I mentioned, OS Tree is an upgrade system for Linux place deployments. It basically does two things. It manages versions of bootable Linux file systems, so RootFS, in a very Git-like fashion. And then the other thing it does is it manages bootloader configurations and defines a file system layout in Linux such that the Linux system will boot and run a RootFS that is effectively a checkout of a RootFS commit from an OS Tree repo. So with those two things, with OS Tree, you can manage software and manage software upgrades such that the bulk of the steps actually happen while you're still providing service and still running the active RootFS, basically. And so resulting in primarily the outage time being reduced for an upgrade on the server. When you first look at OS Tree, you kind of think it's a package manager. It's not a package manager. And actually, kind of in embedded systems, it's basically used instead of a package manager. But there's been developments where there's really hybrid approaches to using OS Tree with a package manager. And I'll talk a little bit more about that later. So like I said, the first thing that OS Tree does is does version management of RootFS just like a Git. So I thought the easiest way to show you that was just show you some commands and you'll quickly realize, oh, it's like Git. So you can just see there's an OS Tree init command initially that initially creates an OS Tree repo and I just dumped out the subdirectories under there. It looks very much like a Git repo. If I wanted to create a first commit for my OS Tree repo, I could make a temporary RootFS. I'd populate it with a RootFS for a Linux deployment such as Starling X. And then I'd basically commit it into OS Tree. And so I'm doing a commit there. With specifying a branch, SCX-9. If the branch doesn't exist, it'll create it. And then I'm putting a subject tag on it saying it's a GA release SCX-9 and specifying the RootFS content that I just created. So I'll create the commit in OS Tree. I can do an OS Tree refs to list out the branches. You can see there's an SCX-9 branch now. And then I can do an OS Tree log to list the commits and the commit is there, the GA commit. And then I can obviously remove the temporary RootFS because I've checked it into OS Tree. And just like Git, I can check it out if I need it again. You can see the RootFS is back. I can make a change, I can fix a bug. And then I can commit the whole RootFS back into the OS Tree. And you notice that just in this example, I'm gonna commit it back into the same branch SCX-9. I'm gonna tag it as, okay, it's patch one for SCX-9. And then you can see that I just still have the same one release branch, but now I've got basically two commits in my OS Tree branch, just SCX-9 GA in the patch. And then also, like Git, you can actually specify in a local OS Tree, you can actually specify remote OS Trees, the repos that you can pull from. So in my little example here, I said, oh, I've got an SDX build server. So I'll create a remote for this OS Tree. I'll call it my SDX build server, specify the URLs, specify the branch that I wanna pull from. And then once I can do that, then on my local OS Tree repo, I can pull that. It'll pull all the commits in that branch down into my local OS Tree repo. And then now you can see that when I do the OS Tree refs for the branches, I've got SCX-9 that I just locally created. And then now I've got a remote branch from my SCX build, SCX-10. And then I can list the commits there and then I got a SCX-10 GA commit. So you get the idea, it's exactly like Git. Okay, so that was the easy part, the hard part. So like I say, there's an OS Tree deployment layer that's defined that effectively defines a file system layout. So, and a Linux deployment can basically boot and run from effectively an OS Tree checkout of a root FS. So I'll go through the key concepts there. So first key concept is that the most OS Tree-based Linux deployments have a sysroute partition. And in that sysroute partition under kind of sysroute OS Tree repo is my local OS Tree repo. So that's the main OS Tree repo for this OS Tree-based Linux deployment. And it's basically got one or more commits in it, basically all the releases that I've kind of loaded onto this system at one time or another. And then under sysroute OS Tree deploy, basically I have one or more deployments which are actually basically checkouts of a particular commit in OS Tree. And though those commits, one of them is the, one of them is the actively running root FS and other ones could be ones that I'm in the process of upgrading on. And yeah, I forgot to mention that just when you do OS Tree checkouts, it's very similar to Git in the sense that it uses hard links to, you know, to conserve disk space. Okay, the next, okay, so the next thing is there's three top level directories in Linux that you need to understand in order to understand how OS Tree manages the software. There's slash user, slash ETC and slash var. So slash user, slash user in an OS Tree-based system contains the entire root FS. So it's exactly what you committed in OS Tree as your root FS. It has all the software that's being managed by root FS by OS Tree, OS Tree completely manages it. And you can see that it's actually read only, that the actual slash user top level directories actually read only bind mounted to the checked out root FS commit in that under the deploy directory. So actually kind of side benefit of OS Tree is that the software managed by OS Tree is immutable in the sense that even a user with root permissions couldn't accidentally change the software. All the software is all changed through OS Tree operations. And then obviously in a normal Linux deployment, all the managed root FS software is not under slash user. There's other top level directories like slash bin and slash lib that have managed root FS software. Those all end up being symbolic links to the analogous directories under slash user slash ETC. So slash ETC is partially managed by OS Tree. So slash ETC contains configuration of the services in the OS that are being managed whose software is being managed in slash user. So obviously it's configuration data that the user will configure for a particular deployment. So OS Tree can only partially manage that. It's got to preserve the user configuration. So what OS Tree does is that when it's actually doing a deployment from one root FS commit to another root FS commit, it will actually do kind of a funky three way merge of the old default ETC config in the root FS that I'm migrating from the active systems ETC that's got the user's configuration in it and the new default configuration of the root FS that I'm upgrading to, basically. And then finally slash var. So slash var is in an OS Tree based system. It's basically used for storing all the runtime persistent data for the OS and for all the platform applications in your Linux deployment, like in the case of Starling X it would be the Starling X infrastructure applications. So this is not managed by OS Tree at all. So OS Tree does not touch this at all. And again, obviously all the runtime persistent data in a typical Linux deployment isn't necessarily under slash var. Again, there can be root level directories with persistent data. And again, it's just symbolic links to under slash var. Oh, and then finally, finally, there's the slash boot partition. So this is like a typical slash boot partition for Linux. It basically has the boot loader configuration to indicate what kernel to run. And in the case of an OS Tree system it also refers to what deployment under assist root OS Tree deploy that I'm actually wanting to boot into the next time my system boots up. And so when an OS Tree deploy happens to do an upgrade it actually knows how to update the boot configuration in order to get it set up so it will boot into the right root of fs. Okay, so how does a vanilla OS Tree upgrade actually work? So I've just started out with an initial condition here where I'm running an OS Tree based Linux deployment. I've got one commit say commit zero five in under my assist root OS Tree deploy. And so that's the starting point. So first thing I do is I do an OS Tree pull to pull down my new software release. So I'm doing an OS Tree pull from a remote to external OS Tree repo that pulls down into my local OS Tree repo. Obviously there's no impact to the root of fs that I'm running at the time and the services I'm supporting. And then the next step is a multi step step is basically doing the OS Tree deploy. So what happens at this point is that a new deployment directory happen gets created under deploy. It basically OS Tree checkout is done to check out the new root of fs commit which is the new release of software. And into that commit it does the OS Tree will do the three way merge to manage the ETC directory and then OS Tree deploy will also update the boot logator configuration to indicate that the next time you reboot you will boot into that deployment under sysroot OS Tree deploy. So again, all that can be done without impacting root of fs. And then finally you reboot the server to reboot it back, reboot it into the new root of fs. The updated boot configuration will run the updated kernel and the updates to the boot loader configuration scripts will actually change the rebright mount that are the bind mounts in order to point to the right root of fs that you wanna come up in. So that's a general idea and the last thing on OS Tree is, I mentioned it's not a package manager it's a really typically used instead of a package manager. I think in a classic package manager like with RPM packages or Dev packages those deliver partial files to some trees that contain the software for a particular software. It's got metadata, it's got install scripts and typically the package manager takes those packages and installs them directly on the running root of fs. So in the case of OS Tree, as you've seen, OS Tree deals with complete bootable file systems the complete root of fs. So it has no idea about how the root of fs got built or anything it just knows how to save them in the OS Tree repo and it knows how to deploy them basically. So I mentioned that there is a hybrid approach now there's a number of tools that use it. RPM OS Tree is a good example of a hybrid tool that can be used on the server that you're managing the software. And it makes it a combination of the two software still gets delivered as packages but then you use OS Tree to actually deploy it. But what that means is that the hybrid package manager has to effectively check out the current version of root of fs on the target. It's got to install the packages in a shrewd environment and then it's got to commit the updated root of fs back into OS Tree so that it's basically got the staged new root fs ready for an OS Tree deploy. But so why would I do kind of hybrid? It's basically kind of get the best of both worlds. You get the package delivery which certainly in the case of patching you get small kind of much more readable software updates in an air gap scenario. I don't have to do OS Tree pulls I can deliver through files. And I can also deliver different package sets to different deployment sites if that's a requirement. So I can do all that with the flexibility of the packaging delivery but then I can still leverage OS Tree to kind of be able to build and stage my root fs in parallel with running my active root fs system as well as do the atomic upgrade on reboot. So how does this translate into Starling X? So the proposal that we're looking at is that we're gonna use a hybrid package manager. The diagram that I show here shows a multi-node Starling X cluster. Obviously each of the nodes will have a local SysRoot OS Tree repo. That'll be the main OS Tree repo that it boots from. But then we're also gonna have a central remote OS Tree repo on the controllers. So it'll be in that repo that our hybrid package manager will basically stage and install the packages into a root fs and store the commit back in OS Tree that basically stage the new release upgrade in that remote, in that central OS Tree repo. And then when Starling X software upgrade orchestration kicks off to do basically its rolling upgrade of the different nodes in the cluster, basically whenever it gets to the certain point for a particular node to say install the software, instead of wiping the disk and using package manager to install to the root partition, it's gonna just do an OS Tree pull from the central OS Tree repo to bring in the new software in a root fs commit and then do an OS Tree and deploy and reboot like we saw before. So how does this help Starling X software upgrades? And what I wanted to show in the slide was basically the major steps of a Starling X software upgrade. And specifically this is for a single server Starling X deployment, just cause that's the one where the impact is the greatest because I can't do a rolling upgrade on a single server. So basically what I've got here is the major steps for an upgrade of a single server. I've got some time estimates here for each of the steps just to kind of get a ballpark view of what the times and what the improvements might be. So to just walk through this, so when you start an upgrade on a single server system we actually have to back up the Starling X platform system data that's on the root partition because today I'm gonna wipe out the root partition. This is like today's model without OS Tree. Then I administrative locked the host to get it into an upgrade mode. I wiped the disk, I wiped the root fs because I'm gonna install the new load on it. I obviously don't wipe the partition that has a backup that I just did and I don't wipe the set disk because it's holding the persistent data of my guest applications. And then I basically run the installer ISO for my new release of upgrade to basically do the package installations on the wiped root partition. And then I reboot into the M plus one software. But then I basically got a new M plus one installation which I really have to bootstrap using the backup data that I have to basically restore the system data associated with the Starling X deployment. So I have to do that. And then more than likely you have to do some migration of Starling X data due to data model changes. And then you finally unlock the host and upversion the containerized system apps. So there's a lot of steps in our current upgrade and most of them are actually in the outage window when I've basically disabled my hosted guest services. So how does it change in with OS tree? So two things change. One is there's less steps. I don't have to back up data because I'm not gonna wipe the root partition. And then I, because I don't back up the data and I don't wipe, I don't have to do the bootstrap and restore the system data from the backup. So those steps are gone. And then other steps basically move outside of the outage window. For example, the whole installation of packages actually happens even potentially outside of the maintenance window because my hybrid package manager is basically going to create the root of S and the shrewd environment, install the packages in that shrewd environment. So that can all happen well before even the maintenance window. And then I can even migrate the data in that shrewd environment such that that's done outside of the outage window as well. So kind of with a combination of reduced steps overall and some of the steps moving out of the outage window, you can see that kind of the elapsed time and outage times could be significantly improved with the OS tree solution. So this is a summary, kind of talked about, gave a brief overview of the OS tree as an upgrade system for Linux deployments and just talked about how basically it manages versions of root FS just like Git. And more importantly, it has this deployment layer with a file system layout so that you can boot and run effectively checked out versions of the root FS and that allows you to basically do a lot of the staging of the upgrade and upgraded root FS in parallel with continuing to run your active root FS and then the atomic upgrade to switch over to it. And I also talked about just fact that OS tree can be used as a hybrid package manager and that's the proposal that we're looking at for Starling X and then finally just kind of walked through the impact, positive impact of the elapsed time and outage times for upgrades just because of reducing the steps, not having to wipe the disk and actually moving some of the steps outside of the outage window, especially in that single server scenario. And I didn't mention before, but I mentioned at the beginning that like to say in a lot of the telco kind of 5G deployments that we have, a lot of those remote sub clouds are single server solutions just because that's got enough horsepower to run kind of the remote 5G apps. And that's really it. I just wanted to say that if this sounds like something cool that you'd wanna work on, you can get in touch with the Starling X community either through our mailing list or you can check our website for when our community meetings are you can come to the main community meeting and then if you're interested in looking at that, we can kind of connect you into the different sub projects within Starling X in order to kind of collaborate with whoever is working on this. And that's it. I don't know if there's any questions. Yes. That's a good question. I suspect it just fails and the three way merge that happens is not when you haven't committed to doing that deploy yet. So I suspect it just fails and you have to kind of figure out what happened and likely supply a new commit in order to do it. Yeah, like I say it's Git like, but it doesn't have all the Git capabilities. You just get an error on that merge and you're gonna have to fix it manually. Yeah, so I found this last three is actually applied like would that mean that the actual repo with the S3 would actually explode in size? I mean like assuming that you would see it through the package manager upgrade for the S3, so there is a delta which is not really like... Yeah, there are some, we haven't got into details of that, but yeah, there are some cleanup requirements in two areas under the SysRoot AusTree deploy. There's cleanup that's required under the different deployments that you have there. And then there's also cleanup of the AusTree repo itself. AusTree does support pruning of the tree and definitely, yeah, after in a lot of product deployments, you would have frequent patches and you would get a lot of dis-space used and cleanup required, so that's definitely something that has to be looked at. What about fail back in case of some errors during the upgrade? Is it possible to restore the previous version? You can, I mean you can effectively do an AusTree deploy to any commit, it doesn't have to be the next commit, so you can really go back to a particular commit. And certainly, I mean there's gonna be details with respect to migration of data. I know in the first iteration of doing this, we really are supporting going back only in kind of a rollback scenario where we can roll back the data easily to what we were just previously running as far as data migration. And for some things it's actually better than that. You can just, in your bootloader, select your previous tree, just like you used to select your previous kernel. And that works, you know, through common line parameters. We're better if you make sure that you version some of your data and if you're doing data changes Yeah, yeah, like for the migration that we're doing, I didn't go into the details here, but yeah, for the migration that we're doing for the Starling X data, we're managing that data in slash var and basically versioning the data in there so that when we do the migration data, so we could kind of get that data when we go back. But the reason we're only doing it for rollback is because we still have the data and we can go backwards and get it and we know on a rollback, nothing's changed as opposed to if I wait three months and now try to do a downgrade. You know, typically in Starling X, we don't write downgrade data model scripts. We've been asked to do that, but yeah, typically it's only upgrade scripts that we're doing, so that's why we're kind of trying to limit the scope to rollback. How do you generate like the first commit, you know? It's a base image. Yeah, so for the base image, we'll deliver, like today, we'll have an installer ISO that basically has all the Dev packages in it and it'll really be, again, I don't know the specific details about this, but it'll really be like a reboot strap type setup of the initial RudaFest and then using Shroot to install all the other packages on top of that. All right, thank you.


My experience of child development in the classroom My recent long journey through university had my interests caught on many subjects and topics. One which I finally decided to major in was that of educational psychology. The human brain, physical body, our intrinsic and extrinsic environments that everyone is conditioned by strikes my interest on this specific subject. Keeping with topic chosen I would like to share my interest, views and maybe slightly biased opinions on child development. The basics of child development There are some basic concepts of child development that I have learned to date. Child development, both psychologically and physically is for me personally an extremely interesting area to learn from especially as a teacher, student of life and parent alike. One of the main reasons is that the information that you are exposed to never ceases to help you learn more about your own children's development. In the process you also acquire a better understanding of other children within your own social and working environment. From what I have benefited in this learning process are some of the main goals in learning more about all round child development. Some of the main points have been to learn how to describe the changes that typically occur in the developmental process within a child through the various ages. To be able to explain what causes these developmental changes and to learn how to predict certain behavior based on past and present characteristics. Finally, a point of utmost importance, to help improve the well-being of the child's development and to contribute positive change if or when necessary. I have been fortunate enough to have been involved in teaching for 10 years in various schools and I have witnessed a gradual development of students from year 1 through to year 7. I have been fortunate enough to have met up with some of these students in their more senior years and have enjoyed a more advanced conversation than when our company departed 5 years previously. The Six Stages of Development Although the development of my students and my own children has been a continuous process, I have found it useful to be able to divide this witness development into stages. Using their unique characteristics and skills combined with learn research and knowledge gained has shown that most individuals develop significantly during each specific stage. I have been extremely privileged and honored to bear witness to six stages of development from prenatal, to neonatal, to infancy and to early childhood with my own children, and middle childhood to adolescence with my students. Having been a physical education specialist for the first three years at my first teaching posts, I was able to bear witness to three years of physical development from boys and girls of the ages of six years to 13 years. This facet of development that I saw and tell changes in the child's body such as weight and height. One other important aspect of the physical development of the children is, was motor development. I have also been able to bear witness to my own children's gradual physical development concerning body movements such as sitting, walking, and grasping. Just the tip of the iceberg. There are definitely many detailed factors that contribute to child development and as a whole I can confidently say that I have barely scraped the tip of the iceberg with what I have shared in this post. From expert research involving scholarly articles and case studies for both past and present, including multiple theories and ways of applying unique scenarios of child personality, and as mentioned earlier, personal intrinsic, extrinsic environment, will all ultimately influence the journey of a child to adolescent development. If you are in the position to contribute positively without passing on any negative conditioning to a child's overall development, it is in my opinion an obligation as an educator, parent or anyone who is around children to help build on this honorable piece of living art. Are you ready to teach English abroad? Speak with an ITTT advisor today to put together your personal plan for teaching English abroad. Send us an email or call us toll free at 1-800-490-0531 to speak with an ITTT advisor today.


I told him like right away like, hey, I feel like myself and then from there I was able to do everything. I think if people, if people struggle the first day, then the second day is kind of, it's a ramp up, so. I think the media do it too, like, you know, they see people go on the COVID list and they start thinking, okay, who's going to be able to play and who's going to be able to line up and things like that. But sometimes, maybe people forget that you guys really suffer, you know, when there's COVID. Yeah. Yeah, no, um, yeah, I mean, we're such a weird business, like, there's been a couple of times throughout the year where I'm like, wow, like, I'm an NFL player, like, pretty much, like, we're playing in a stadium and there's however many thousand people cheering us on, like, we're gladiators. Like, that in itself is a strange profession, so I think it is what it is.  Elijah, when you prepare for just on the offensive side, what would you like to see weapon-wise? Obviously, that guy in 2-11 is coming along pretty well. Yeah. And a weapon he likes in 7-8? Yeah, I think, you know, they're feeding 17 the rock a lot. He's had a heck of a year as a rookie. The office is creative in getting him the ball. And then they're also looking to set up 2-0 with some throw, getting the ball out of his hands quickly and then building confidence off of that. So they're, you know, obviously they're on a roll. They're on, you know, they're a good team. They've got a great defense on their side too, so it's going to be a good challenge.


This is your current ops support. USS Ross returned to Naval Station Rota, Spain, July 20th, after completing its second forward-deployed patrol into the US six-fleet area of operations. During its four-month patrol, Ross conducted maritime security operations in the Mediterranean and Black Seas. Ross participated in numerous bilateral exercises with NATO allies and partners as part of Operation Atlantic Resolve. I'm Peddielson Daniel Schumacher, and this has been your current ops support.