Changing focus to football now on the Sportsmax zone. Match week 7 gets underway this weekend and these are the fixtures. So we have Lime Hall playing Bear United, Donby Holden up against Malines United, Montego Bear United play multi-pleasant waterhouse face humble lion, Portmore will be seeing Tivoli Gardens. Well, with us this afternoon to give a rundown of the season so far and preview the fixtures is our in-house football analyst Ligea Williams. Good afternoon Ligea, welcome to the set. Good afternoon, lovely to be here as per usual. Alright, so I'll start by questioning you so you'll have an easy start. Let's talk about the weekend and which of the matches are we to look forward to. I just had on the screen the exciting match-ups, which one you think will be the most exciting of them all? Well for me I think, well at least I think the match-up that I'm looking forward to the most would be on Monday, the Portmore United versus Tivoli. I think both of these teams are looking likely or one of the likely bunch to try and get to the top of the mountain in terms of the bunch that are chasing Mount Pleasant. So I think Portmore, they won the link-up last year, they had flashes, they didn't make the players but they had flashes, they've retained a few of their players, yes the last players like Lamar Walker but I think that they've recruited well, replaced them well with Martin Davis and I do think that they have the makings and the coaching teams have a really good team so they're a team I'm really looking forward to this season and Tivoli guidance, I can't heap enough praise on them and Jerome White, I think the job that he has done so far in terms of the difference in stylistic play that they have changed in addition to the quality, they're basically the same squad from last season, I think it's really brilliant what he's doing over there. So that's how much I'm definitely looking forward to this weekend. Yeah, I want you to talk to us a little bit about Martin Davis because I remember him as a youth player with so much promise and so much expected of him, had a stint in Spain as well during his youth days, saw him on Wednesday I think it was with a beautiful play getting the ball to the byline and getting the cross inside which resulted in a goal. Your own thoughts on his overall development and what he's brought to this Portmore United setup? Well I mean his development you know unfortunately he didn't or so far he hasn't reached the heights that his talent I think would meet, like his talent you'd expect him to get so much further than he's gone so far but with that being said I still think that he's played in some good places to develop his career, you mentioned Spain when he was in Europe otherwise apart from that and he's a really talented footballer that's something that's never going to fade in my opinion and I think he brings a lot especially I mentioned the fact that the last Lamar Walker, Lamar Walker if we cast our minds back to the link up final he was the man of the match at that game, he was really good dictating things from deep but also being really penetrative in an attacking sense, Martin Davis I don't think he's going to give them that deep build-up threat but in terms of being penetrative attacking wise there are very few players I think that Portmore could have brought in to approximate what Lamar Walker could do and I think that this Portmore setup with how they play with three narrow forwards he's going to be really close to Chevron Walsh really close to Alex Marshall players that have a similar level of talent to him I think that's going to do his form in the Jamaican Premier League a world of good. Yeah well Tivoli lost to Mont Pleasant yesterday they would not feel too hard done by that result because Mont Pleasant has been on fire this season what would they need to do differently from that loss in order to ensure that they pick up all points on the weekend? I think it's tough to to criticize them over the last that they would lose against Mont Pleasant because as you mentioned Mont Pleasant have been flying they're the best team in the country so far by a really a landslide so Tivoli they are an emerging team I think that by time if health permitting no injuries permitting I think if Tivoli were to continue on this trajectory there is no doubt in my mind there are semi-finals team at minimum with just the squad that they have and I'm hearing that they have players to come in changes to be made but just development of the coaching the familiarity of what they're trying to do what their coaches trying to do as well I think that Tivoli just need to continue going rolling with the punches and I think generally they'll get better and better and then over time their project will go boom as as my guffar said Miquel Ariteta but I do think that Tivoli have the squad they have the capabilities and I do think that they'll get better but in relation to this Portmore game Tivoli versus Portmore I'm sure Ricardo especially would know when I was growing up in probably mid-2000s that when you're still growing up by the way go ahead what I was growing up but I was around six seven eight because I know Ricardo is an old boy but as a young man myself when I was growing up I was going to prison over I was going to national stadium I was going to Ferdinita Park I was going to Edward Seattle Sports Campus because this was the game that so often the end around finals the JP the at the time that well it's right now I guess I can say it during every Premier League finals this game was so important in the grand scheme of things so this is one those games where it's like a L classical I know that there's a actual a classical Arnett and Waterhouse but to me this is a significance that it would play Arnett Waterhouse is a L classical this is the Milan Derby how big of a game it is this is not a game that will lose its prestige so I'm looking forward to a really tight encounter no matter what the talent levels made the form of the teams this game is always going to be an exciting one and I'm definitely looking forward to that's why it's my matchup of the weekend yeah I want your assessment of Tivoli Gardens under Jerome Waite in comparison to the Tivoli teams the Tivoli setups that we have seen in more recent seasons well I think it's just there the profile of players that were within the squad just fit Jerome Waite perfectly and it's not something that I saw what as soon as I saw Jerome Waite get a point I'm like yes this project is going to be really good but now when I watch them play I realize that what he is trying to accomplish and the position that he has the players in it's perfect for them so for instance Penny Cook he has him playing yes he's playing in the back three also but in terms of what he he try he pushes him up into midfield similar to what we see with John Stones and Manchester City so and his engine allows him to do that Howard Morris playing him just off of the strike at the tip of a midfield three diamond that works out brilliantly and then you talk about Justin Dunn six goals in the Premier League this season he was playing right wing back at times last year and now he's playing in a pseudo right wing sometimes a lot of rotation in their front three so he will be pick up striking positions you see making penetrative runs from deep that's what he's really good at and his finishing ability so I think what Jermu it identified immediately is a type of profile that he has in the squad and he has built a system around them perfectly and this team is really flying and that's why I said I have a lot of confidence in them not only I don't think it's just a flash in the pan I think that when the season is over this team if injuries injuries with standing this team should be minimum in the semifinals just based off their talent level the level of their coaching and just their application in games as well so this is a real this is a team that I'm really excited about going forward yeah has Jerome found a statistician yet no not as yet but is he still doing the stats from the stands no he's gone back maybe maybe he was just putting it out there for anyone who wants to join I personally can't do it why I know that you wouldn't want me to leave sports max personally let you go on alone alone alone deal a season long deal you couldn't do you couldn't build out me for salaguer card could it do it I think you all to have your bromance on that side and I'm just in my little corner but I don't want to ask you though about my lines United yeah they have not got a win this entire season they played down beholden on the weekend are we expecting the results to of course change these are two teams that I think are really similar actually I'm two teams at one to play attacking football have good attacking players yes but defensively they just haven't been showing out this season especially more lines from last season actually they're one of the worst defenses in the Premier League and Alex Thomas is a coach that I really respect I think that he's a really good manager he has quality players at his disposal as well but I'm just not defensively just has not been clicking they were always liable to collect as my friends would say collect a bargain at certain points of last season so I might give them five they collect a lot of big scores as well I just think defensively their application isn't good enough most of the time and I think they need to work on that they haven't worked on that coming into this season with terms of number holding although I do think that number holding are struggling right now I think they're going to get better they have one of the best coaches in the island for goodness sake in Lenny Hyde and they have quality players as well so I think they'll get better but more lines United I think they have to button down hatches and try see if they can get a result this weekend because if they're not going to get it this weekend pressure will just continue to build and build and build and the last thing I would want is someone like Alex Thomas a young good coach like that to get sucked and then more lines has to reset and then who knows even fall into a relegation scrap like they're in right now so I think I need to start getting some results going dumbo holding I think around a less pressure let's see how they perform this weekend. Yeah we spoke about it at the top of the show Montego Bay United Mount Pleasant that game originally scheduled for 7 30 p.m. on Monday at the Montego Bay Sports Complex at Catherine all however renovations taking place at that venue and the match time had to be changed now Montego Bay United wanted the match to be played on Sunday at 3 p.m. at West Pole Park understandably so because Sunday at 3 p.m. would be a better time for fans to come out and support however the match remains on Monday it will be played at 3 p.m. at West Pole Park and a large part of the reason for that there are no lights at West Pole Park so it has to be a day game the management of Montego Bay United have come out to say they are disappointed that the request to have the game played on Sunday was not granted and as I said at the top of the show it seems like a no-brainer from the Montego Bay United standpoint but there is also another standpoint to be considered and that is the Mon Pleasant standpoint because Mon Pleasant would have planned and prepared for a game on Monday and I think the rules are pretty clear that to have a change in a situation like this then you need both teams to submit the request and you need both teams to agree in this case I suspect that Mon Pleasant would not have agreed understandably so from their standpoint because if you are preparing for a Monday game and all of a sudden a few days before you hear that it's going to be on Sunday that's not something you would necessarily agree with I don't know Leje if that makes this contest or adds some amount of intensity to this contest on Monday the developments leading in yeah I think it may especially there's going to be no fans less people to to view so maybe the mobile people are saying okay let's just go out and be physical as we made the crowd isn't going to get on top of us so be the case but Montego Bay 90 they've been struggling again this season they've brought in some players of Brazilian heritage to go along with their Brazilian coach but it hasn't quite gone too well for them then they had a really good game over the weekend home ball and again against home ball and it was a close game but this Mon Pleasant unit they're playing some liquid stuff right now they have easily the best squad and paper I'm talking about now on their defense we see how their attack is going with Kimoni Bailey Devante Campbell across a chunk but I'm a midfield man yeah I'm an engine room man so when I see a midfield three coming to against a team and that midfield three is got three Demario Phillips and then on top of that they're going to add Ramon Howell one of the another one of the best midfielders in the league that midfield alone is a title winning midfield so good luck to the rest of the JPL but you're going to have a warm time de-throning Mon Pleasant this season yeah very much the case and quickly I want you to talk about Kimoni Bailey who has been in fantastic form four goals in five matches this campaign what has led to this success that we didn't see for example when he was at Dunbeholden you said quickly so I'm not going to get too much into it but tactically there is a shift in terms of when you think about Trivante's stewards it's similar to what you see take Man City for example early in Holland everything is funneled into him as a target man but when they're playing with a false nine you see the wingers getting more involved you had Sterling scoring 20 at goals when it's a more nimble striker like Acosta Chong is there's going to be more help on the wing more there's a lot more facets to the attack and I think that's what causing the wingers to get more involved it's not only Kimoni Bailey who has been fantastic this season but the Vante Camel and the left I think has been just as good just hasn't scored as many goals he has his assists however but yeah I think because of the the shift in type of striker that they're using that's what causing the wingers to really explode this season so far yeah and quickly again even quicker than the last time we show that graphic at the top of the segment with only five matches scheduled for this round there are a few teams who will not be involved yeah Cavalier Cavalier will be taking part of course in the Cavalier and Harbourview will be taking part in the Caribbean Cup Harbourview of their third place matches Cavalier of the final coming up is a two-legged affair so good luck to both of those teams I would love to see Cavalier bring it home because they have been absolutely fantastic in that tournament Shanieel Thomas the top scorer so hopefully they can bring on the trophy for Jamaica yeah DJ Williams thank you very much for joining us on the sportsman zone with your expert analysis and we'll discuss that one-year loan deal let's take a break


In this video, we present the solution to question number eight from practice exam number three for math 1050. We're given a rational function f of x in factored form.  We're asked to find the domain of this function. Well, when it comes to the domain of a rational function, the only thing that we have to worry about is dividing by 0. So whatever happens in the numerator doesn't matter whatsoever.  Even if there was a common factor in the numerator that cancels with one of the denominator, the domain is always computed from the original formula, not some simplified version of that. So what makes the denominator go to 0? Well, if you set the denominator equal to 0, this is a factored quadratic equation. We set each factor equal to 0 and solve for it. And so from the first factor, we see that x should not equal negative 2. From the second factor, we see that x should not be negative 4 over 5, which actually I'm writing over the right answer. This would then tell us that the correct answer is then b, all real numbers except for negative 2 and negative 4 fifths.


Okay. We will proceed. Mark Andre Lemberg will talk about advanced database programming. He's a seasoned Python developer, being around since 1993. Also founder and CEO of Egenics.com, one of the founding members of the Python software foundation and a board member of the real Python society which brought this lovely conference to you. Give him a warm welcome, please. Thank you very much for coming. I'm going to give a little talk about advanced database programming because we've in the past heard a lot about the easy stuff. So I thought it might be a good idea to talk a bit about the more advanced things. A bit about myself, I'm Mark Lemberg. I've been using Python for a very long time. I've studied mathematics. I have a company doing Python projects. I'm a core developer, Python software foundation member of the Europe Python Society and I'm based in Düsseldorf. So this is the agenda for the talk. I don't know whether I can actually do everything that I have on this agenda because of the time constraints. But I'll try. First I'm going to start with a short introduction of the Python database API 2.0. How many of you know the Python database API? Interesting, not that many. So the Python database API was designed in the, the design started in the mid-90s. That was the 1.0 version which is now deprecated and we're now at 2.0. So it's a very old kind of standard that was developed. It's the development is ongoing on the Python DB sync. So if you want to join the discussion there, you just have to subscribe to that mailing list and you can add your thoughts to the standard. It's supposed to be a very simple kind of standard. It's supposed to be easy to implement so that we get as many database modules as possible. And I think that has worked out really well. The two main concepts in the Python database API, one is the connection object and the other is the cursor object. So you use the connection objects to actually connect to the database and also to manage your transactions. And then if you want to run queries, you open a cursor object and you run your queries on the cursor object and the cursor actually works like a cursor in a text processing system.  So this is how a typical application looks like that uses the DB API. So first you import your module, you get the connect API from that module, you open a connection, you pass in the database name, the user name and the password. Then you create a cursor object on the connection object and you run your queries on the cursor object. And finally you free the resources by closing everything again. So that was a short, very, very short introduction to the DB API. The next part is going to be about transactions. Transactions are a very, very useful thing in databases. You can do stuff on your database and if you find that you've made a mistake, you can just roll back your changes, which is very nice to have. You need it in production systems to work around bugs or input errors from users so that your database doesn't become corrupt. So it's very useful to use these transactions. However, there are a few mistakes that people often make and this sometimes causes people to not like transactions. One common mistake is they forget to commit their changes. So they apply a lot of changes on their curators and connections and then close the application and see that the database hasn't really changed. Because the database API defaults to transactional, it doesn't actually store the data if you don't do an explicit commit. Now a workaround for this is to just disable transactions, which of course is possible in databases as well, but it's not a really good workaround.  Another common mistake that people make is they keep the transactions running for too long time. And I'm coming to that later in the talk, the transactions are basically your units of locking things in the database. So you want to keep transactions short to not lock other processes from accessing the database. So what you have to do is you have to try to make transactions short. Now the best practice is of course, like I said, always use transactions. Even if they are sometimes annoying, don't use auto commit. Always try to make use of them. Keep your transactions short. If they get too long, you can run them in batches. For example, if you're loading data into your database, it's much more convenient to do that in batches, say a thousand rows at a time and then you do commit. That also keeps the transaction lock of the database short and the performance will stay just fine. So you won't really see the overhead that is caused by the transaction mechanism. And if you know that you're not actually writing to the database, it's a very good practice to set the read only flag on the database. You can do that in the connection. You can usually do that in the connection options. And then the database will know that it has a read only connection, so it'll basically not work on this, not use the transaction lock and make it, make the whole query mechanism run much faster. So that again was the simple kind of level of transactions. Then we have a more advanced level of transactions. If you want to connect multiple databases and you want to have transactions span the different databases, then you have to think about what to do when you read data from one database and then put it into some other database. Of course, you only want that to succeed if all the databases have actually received the data. And that's what's called distributed transactions. Typical use cases are in accounting, for example, you debit from one account on one database and you credit the amount to some other database. You only want that to succeed if both databases have actually made the change. And you have similar things in Q-processing or if you want to integrate different applications. Now the typical buzzword that you'll hear when you're talking about distributed transactions is two-phase commit, which is kind of the standard method of approaching this problem. So it works like this. You have a first phase and the first phase, the commit is prepared. So all the different databases are asked whether the commit would succeed with a high probability.  There's a tiny probability there that the database, some database in that process may fail in that second phase. And then I can say I'm sorry then your data is corrupt. You have to work around that in some way because there's no easy way of undoing the second phase. But most databases just handle this fine. To make it easier to deal with these transactions across multiple databases, there's something called a transaction manager. This is not in Python. This is usually done in some other system. For example, there's MQ series from IBM or J2EE or Microsoft has this DTC mechanism, some database systems, DB2 and Oracle offer these transaction managers. You can sometimes hook into them from Python. There are Python APIs for some of these. And you can then use them or you can use a database specific one like for example, the one that's integrated into Postgres. In the DBA API, we have addresses with a new set of APIs, the TPC API. Those are modeled after the XOpen standard for these transaction managers. But unfortunately, not many databases support this and not many database modules actually provide these APIs. So you have to check your database whether it supports this or not. Okay, next point. Concurrent database access. Am I going too fast? Too slow? Concurrent database access is very important if you have multiple processes accessing your database. For example, if you have a web application, because of the GIL, you would normally want to have multiple processes set up to talk to your database. And so it's important to think about how the databases deal with the problem of concurrent access. So you have typical setups, what I've written down here. The most typical one is of course again the many writers, many readers. So there's no real special case that fits in the common case. You definitely need to make compromises in these setups. So when writing an application and thinking about this, you have to ask yourself some questions. For example, should readers immediately see changes made by other transactions or made by other processes? And should they even see things that went into the database even though the transactions in those processes have not yet been committed? Or should the reader just see everything as it was when it started the transaction? So it doesn't see anything that came into the database after it started its transaction. And the database, the databases can handle all these different situations. They provide different what's called transaction isolation levels. But they have to implement this using LOX. And LOX is something that usually tries to avoid in your application because they keep your whole application from running in parallel and using most of the resources that you have.  The first one is the easiest one to implement. It's read uncommitted, which basically says you don't have any LOX. With read uncommitted isolation level, all the processes that you have that talk to your database will immediately see everything that changes in the database, even uncommitted changes. So strange things can happen. You can have dirty reads, which means that you read data from another process that hasn't been committed yet and you're not sure whether it's actually going to be committed later on. You can have phantom reads, which is something that basically says you add something to the database in some other process and then you remove it again. And your process might have read this row that was added by another process and later on it's removed again. So it's basically a phantom that you're working with. And there are some other things that you have to watch out for. If you want to read up on these things, there's this URL down there. It's going to be in the talk slides. You can click on it. It's a very good explanation of these things. Then the next level is the read committed. This is the default level in most database applications. When you open your connection, you will usually get this isolation level. This isolation level basically says you're only going to see changes that were committed to the database. Now, you can still see changes that were made while you're running your current transaction. So if some other process commits while you're running your transaction, you will still see those. But you will not see any uncommitted things from other processes. So the way that it works is you have this cursor. I drew this table up there with the yellow bar in it. It will lock the current row. It will put a read lock on it. The database will, if there is a write lock on a row, it will wait for that write lock to be removed by the database. So if some other transaction has written to that row, that other transaction will have put a write lock on the row. And the write lock is only removed if the transaction is committed. So if the other transaction has committed to change, then you can actually go ahead and actually read this row. So this is basically you just looking at one row. Then the next level is repeatable read. This basically says that your data won't change in the transaction. So everything that was returned to your application by the database is guaranteed to stay the same throughout the whole transaction. And this, of course, requires more locks on the database.  And then the highest level is serializable, which basically means whatever you do on your database, the database will stay like that, will stay exactly like it was when you started the transaction. And nothing will change. And this requires lots and lots of locks. The locks will not only address the things that you've read from the database, or you've written to the database, but everything that you've ever touched in the database. So even whole tables, if necessary, all of these levels are necessary for some applications. For example, if you want to run a report, you may have, you may want to avoid inconsistencies in the report. So you may, for example, want to use the serializable isolation level. And there are these other levels can be used if you have situations that are not as strict about data processing. So how do you do this in Python? There are two ways to do this. Well, actually, there are three, you can usually have an option in the connection settings that you can set to to set a default isolation level. But you can also do it dynamically in your application, and you can actually do it on a per connection basis. So you can have multiple connections to your database and different isolation levels. You can run a statement set transaction isolation level, for example, or you sometimes some database modules have special ways of directly setting the option on the connection. What's important to know is if you want to change the setting while having a connection open, you need to make sure that no transaction is currently running on that connection. So the easiest way to do that is just commit your connection or rollback, right? Optimizations. So you have a database application, of course, you want to run it as fast as possible.  So you're interested in putting lots and lots of data in. You're not so much interested in making complex queries on the database. Or you want the other thing, you want to have data analysis. So you already have all the data and usually huge amounts of data in your database. And you're interested in doing complex queries, multi-dimensional, faceted, search, drill down all these things. And then you use an OLAP system. Now just like with transactions, the situation is often that you actually want a mix of both. So you want to have both run fast. So there's one way you can do this, depending on the size of database that you're talking about. One way is to put an OLAP system in front of the system that's actually taking in the data. And then every now and then you copy over that data into your OLAP system to analyze it. On a more lower level, you can, in Python, you have a certain number of problems that you can address directly.  And the queries are simple. So you're just doing, for example, the select on a few tables, a few columns. The usual way to address that is just you add more indexes. Because adding indexes is very easy in database. Some people, they add indexes to everything they have in the database. This slows down things because every time right into the database, the database has to update all these indexes. And so you should really only put indexes on columns or on combinations of columns that you actually need in the database. And the best way to find out which tables and which columns you have to index is best to use a query analyzer. The database is usually an awful way to get the information about how a query is running in the database. And you have a look at that, you analyze the data, and then you check which indexes you should put on the database, and it will increase the performance enormously. If you're using Python, you can in some situations also add caching at the Python level. So basically you read your data from the database and you store it in memory for subsequent use. You can even use SQL Lite for that if you have smaller data sets and they do in-memory processing. Now the next point is complex queries run too slow. For example, you have a report that's running on millions and millions of rows. Those will usually take a few minutes to run, depending on how complex they are. Of course users don't want to wait a few minutes for this. So a common strategy for doing this is to pre-process some parts of those queries.  And again, if your queries themselves are too complex, you can address that in Python as well. What you do is you simply split up your queries, make them easier to handle for the database, and then you combine the results from those queries in Python. A typical example of that is if you want to run a report that has aggregates right in the result set. So doing that in SQL is really hard, and you can do it, but it's really complex, and it's much easier to do in Python.  Tips and tricks. This is just a collection of random stuff that I just thought might be interesting for you. A typical problem you have is record ID creation, so you want to put a new row into your database. You need an ID for that, for that record. And you have this kind of chicken and egg problem, because a typical way of doing that is to use an auto increment column in your database and just have the database deal with that, or you have a sequence and get your IDs from that sequence in the database. The problem there with the auto commit, for example, is race conditions, because with the auto increment, the database will take care of adding the increment value. And then you have to, of course, fetch that value again, because you want to continue working on that row. And the usual way is to ask the database for the last used ID, and depending on how you do it, you run into race conditions, or you run into context problems, because it's not really clear what the last ID is. There could have been some other transaction, for example, that also just inserted something. And so it's not clear where to get that last ID from. Another way of doing that is you just let the auto increment field insert your ID for you, and then you just query back the row simply by knowing what's in that row, and you query the database for that row, but that introduces the performance overhead. So it's not really ideal. Something that we always use in our applications is a very simple approach. It's kind of a randomized approach. We simply have a big integer field for the ID, and then we use a good random number generator to generate the ID for us. And we just bet on not having collisions. So we use that ID in the row, we put it into the database, and it usually succeeds. In the very, very rare cases where it doesn't succeed, you just create a new random number, and then you try again. How does this work in Python? Well, first you have to set some constants, so you have to have a row range ID. What we often do is we set the highest bit, so the IDs look nice. Then you have an API get random row ID. This needs to be thread local, because every thread should generate its own IDs, so you don't get any overlaps. And then you have to deal with setting up the random number generator. And the best way to do that is to use system random to get a good seed for the random number generator. Then you put the seed, you put it into hex, and then you feed the random number generator with that, and then you use that in your thread local. Right. Next point is referential constraints. People are usually very happy about using them. Does everyone know what a referential constraint is? Oh, very few. So it basically means this. Instead of, for example, you have a table up there with a, you want to reference a product. Now instead of referencing the product name directly in your table, what you do is you reference the ID into the table that has all the product names. And then you just put the ID into your table instead of the name. And then in your report, you combine all those things into a nice looking output. And this process of referencing from one table to another is called a referential constraint. Usually you use foreign keys for that. You can implement one to n mappings. You can implement n to m mappings. The constraints are enforced by the database, and that can sometimes lead to problems. Because if you, for example, you have lots of references in your database schema and you want to load data into your database, then it'll often fail because the way that the data is loaded does not actually match those referential constraints. And with some database, you can switch off the checking for those constraints during the load phase. But it's not really ideal. Another thing that can happen is if you delete something, then you can get cascaded deletes, which is not necessarily what you want. So what we do for these things, we just completely leave out the referential constraints and put everything into the Python database abstraction layer, which has much more knowledge about these things. It's also a good idea to put those things into the business logic. And then you avoid all these things. So if you wanted to emerge into an emergency situation where you have to quickly load data from a backup again, you don't have to think about how to turn off the referential constraints. You just load everything and it works. High availability is another one of those things that you have to think about. So you have multiple databases and one database server breaks. You want to switch over to the other one. There are very systems for doing that. They tend to not always work perfectly. So what, something that can happen is you can have automatic failback, which means that you have a failover situation where the system switches to a different database and then it automatically comes back. But you're not necessarily sure that all the database servers have actually synchronized by then. So you can have the split, what's called the split brain situation. So you have, the data is spread across different servers, but you're not necessarily sure whether all the servers have the same data. And of course some clients may miss the failover event. So some clients may continue to write into a different database server than the one that is actually currently being used. So again, the best thing that you can do is you move back to Python and manage everything in Python. And then you can also handle the communication between the clients and make sure that all the clients know about this failover event. You can do the failback in an optimal way. And you can also use this for doing read optimizations. You can have, for example, the database, the application part, writing to the database, write to one database and then have the synchronization between the databases, take care of moving the data to the other server. And at the same time, while writing here, you can read from the other database and you also avoid some of these locking issues. So I actually made it, 42 slides in half an hour. That's it. Thank you for listening. Any questions? Yes? Thanks. So thank you for talking. You said transactions shouldn't be too long. What's a good measure for the size of transaction? If you're writing to the database, I'd say just maybe like 10 to 100 rows that you write in a single transaction. Okay. Thanks. You may not want to use random numbers for your IDs since you may break the internal awareness of location and the internal balance tree. So it's maybe a good idea to use auto increment for the awareness of location. Otherwise, you may have some performance issues during writing to the database. Okay. Yeah. Well, we've measured it and it worked out fine. So, but we're in some databases, we're using sequences, for example, from the database to generate those. It depends on the implementation of the database. So I think it depends on the implementation in the database. So my CQ and NUDB would may break them. Okay. We're mostly using Postgres, so maybe. I have a question. Maybe you can help me with that. Maybe I'm missing something because when you are talking about problem with slow, complicated queries, you suggested that you may split it into multiple simple queries and move the weight of the computation a little bit to the Python. And my question is why would it even speed up the process? Because I imagine that the database is constructing such a way that such queries should be done in the maximum performance. So why would Python do the same logical thing faster? It's not so much about speeding up the operation. It's about making it possible in the first place because there are some things that you want to do in the reports that are not possible in SQL because SQL is too limited for that. And even though if you sometimes you can do things in SQL, but you get really huge SQL statements to do your processing, or you have to resort to procedures and everything, which makes things a lot more complicated. So we found it's usually better instead of just wasting time thinking about how to structure your SQL properly and making it very complex, it's easier to just have a few simple SQL queries and then just do the combination of those in Python. Okay. Thanks. The slow forward may be confused a little bit. Thank you. Hi. Thanks for the talk. It was very interesting. Just a suggestion based on experience because I've been a DBA in a former life. If you people do plan to do applications, talk with your database people beforehand. And just they're usually nice. They don't bite you. Just talk with the database people. They can make your life much easier because there is so much implementation detail to do. So maybe that's a good question just to consider. By the lunch, get out, talk to hours of them, it saves you about eight hours of programming time. Yes, thank you. That's a very good suggestion. By the way, many of these things are database specific.  Some databases are better, some are worse. We found that Postgres is a great foundation. Thank you. Thank you.


How often does this happen to you? You're exploring a lost city when suddenly, assassins, you and your buddies try and run, but find yourself blocked in by a dead end. Those bright buildings stare back. Now, cold and unfeeling, trapping you in and drowning your hope. That's just the worst, right? If only you could jump two stories straight up into the air, be on the rooftops and just leave them in the dust. On fifth edition, there's rules for that. In D&D fifth edition, you can high jump a number of feet equal to three plus your strength modifier. That means that all those average low-level barbarians out there are jumping six feet into the air, which is, it's impressive, but it's not roof scaling. So you can also reach half your height upwards, and that gets us to nine feet, but a standard one-story building is 10 feet. So we're not gonna get there on our own. We're better to look for help than magic. And what magic could possibly help us more than the aptly named jump.  Doesn't even need concentration. Suddenly, that plus three strength barbarian is jumping 18 feet up. Then we add in those two or three extra feet for reach and we can get right on top of a two-story building. If you're a sorcerer, use MetaMagic to twin this spell and make two different party members look like they've never skipped leg day.  Now, you could be a good teammate and give your friends this jumping prowess and have them pull you behind them on a rope, but that's not why we're here. So let's build a strength-based character who can utilize this spell to its fullest potential. So we're gonna start up a very human barbarian. Taking magic initiative is our free feat. That gives us two cantrips and one first-level spell from a single spell list. Let's pick wizard. The cantrips we choose aren't important because they scale with intelligence and we don't have any intelligence. But just pick whatever seems fun. Once prolonged rest, we can cast jump and surprise every monologuing bad guy who thinks they're safe just because they're flying around in midair. Did I mention on top of all this that your long jump is just equal to your strength score flat? With a 16th strength tripled, you're jumping 48 feet. And in a 10-foot running start, you can cover 58 feet in a single move action and still attack.  But let's get down to business and really bend these rules to their limits. Let's test the maximum altitude we can get. As a level nine monk, we can run up sheer vertical surfaces That's pretty cool, but what happens if we use our 10-foot running start required to long jump and ran it up a vertical surface? We could argue that we are long jumping upwards. That way we can jump triple our strength score flat up into the air. So if we had the aforementioned 16th strength, we could run 40 feet straight up a vertical wall, launch ourselves 50 feet also straight up and be a total of 90 feet in midair. And as a monk, we don't take fall damage. We can use this to get our bearings in the woods or scale a castle wall in a single round of combat. But where are we gonna find sheer surfaces just out in the wild? Our paladin's tower shield. That's it, that's the only one. A quick alley-oop can get us under a ledge 60 feet up in the air or we can deliver a truly dramatic night kick onto somebody who definitely deserves it. Every time I take the spell jump, it leads to one of those D&D moments that just gets talked about the entire campaign long. Not always in a good way, but it always does something and it always gets me somewhere even if it's in over my head. Jumping mechanics as a whole are so often overlooked in D&D and that's a shame. A simple obstacle of crossing a hole in the ground or climbing a mountain is the perfect puzzle for a game where there are no right answers and so very, very many wrong ones that will still work somehow. Jump is a tool that amplifies the creativity and resourcefulness of the players and that's what makes it so rewarding. Hey guys, thank you so much for watching and thank you so much for watching. Last week's There's Rules For That which had a great response. I just wanted to pop in on the end of this one to bring up a few ideas that I saw in the comments of the last video. There's some really great ideas and I think it'd be cool to talk about them. The number one thing I saw the most was people talking about the Titan Fighter archetype which is a fighter archetype instead of the Titan-Moller-Barbarian archetype which is what I used and there's a couple of reasons I went with Titan-Moller instead. First, I really liked the thematic consistency between the Titan-Moller going into Bloodrager sort of a barbarian all the way thing. The other more important reason is that I had been trying to make this character for a very long time. In my original build, predates Titan-Fighter. Y'all are right about that one. The penalties are less, it's pretty cool again. The second thing I saw a lot of was people talking about getting the oversized lemon straights from the characteristic table for tieflings. I think this is very fun. Tieflings are always been the king of choice in Pathfinder. Any sort of option that's possible to have a tiefling can get some variation on it and this is a really interesting one for tiefling specifically because I think they tend to run kind of too close to just red shaded human and this doesn't let you do that.  So, good luck hiding in your edgy tavern then. And the third thing I saw a lot of was people talking about vital strike and this is actually a really good point to bring up. Vital strike has been kind of a long, maligned feet chain in Pathfinder because one, it makes you use an attack action to use it. That means you can't use it with a full attack. You can't use it with cleave. You just have to use whatever your character can do. You can make one attack and the idea is that it doubles your weapon damage or your damage dice and normally in Pathfinder, mathematically speaking, you want to be multiplying your damage modifier. That's just the better flat rate to go after. But when we are using a Warhammer that is doing 8d6 of damage, we really do not need to be full attacking and using greater vital strike is a super great option for us. And so I think those were all super cool ideas. I'm excited to hear what you have to say about this video and on top of that, if you have any ideas for convoluted rules topics you would like me to tackle in a whole new video, just put them in the comments. I'll look at them and see if I'm brave enough to try them. Thanks.


All right, our next speaker is yet another Brendan. His name is Brendan Irvine-Broke. He is the product manager of Cloudflare Workers at Cloudflare, where he leads the workers' runtime team. Prior to this, he was part of the engineering leadership at Misco and Intercom, and is a former Y Combinator startup founder. Today, he'll be sharing on deploying Wasm to the edge. Please welcome Brendan. Hi, everybody. So my name is Brendan. I work on Cloudflare Workers at Cloudflare, and I want to just give you a little overview of some of the things that we're working on right now around WebAssembly. But sorry to disappoint you to start. I'm here to talk a little bit about some very boring technologies decisions that we've made and actually how those help us give you a little bit more leverage and connect back to some of the exciting standards that I know we're all talking about today. And so those two that I'll be talking about later are Witt and Witt Bynjen, which we're really excited about, and I'd love to talk to folks after. But before I do that, I have a saying that I go back to really frequently. It's a mantra called run less software, and I picked this up from an engineering director who I used to work with. An idea is like trying to remove undifferentiated heavy lifting, like kind of grunt work, and to really fight to manage fewer pieces. And that's, I think a lot of why we're all here, we're trying to make things more portable to remove a lot of the craft. Brendan, big shoes to fill, was talking a little bit about APIs from the 1970s and how do we get away from them. And so for those who aren't familiar, Cloudflare has a full stack serverless developer platform. We have kind of workers for serverless compute, a number of databases, et cetera. And this is all kind of globally distributed and serves many, many, many millions of requests per second right now. So all over the world. And so workers is fundamentally a web interoperable runtime. And that's really important to us. I know I'm in a room of people who are working on interesting new run times and ecosystem things that are kind of breaking out of where WebAssembly got its roots in the browser. But we're built on top of V8. And we really think that gives you the power of the web platform and web standards, but in a serverless context. And so the way that we do this right now is we have a project called Worker's RS. Rust bindings make it possible to write a worker 100% in Rust. So we make really heavy use of WasmBindGen, the web sys module within that to make that happen. And so run through a little basics looks kind of like this. This is a kind of worker that lets you fetch something from our KV store and then return a response. Pretty simple. There's a lot of people building similar things like this. But WasmBindGen really allows us to shift really fast and make APIs available in JavaScript and then get them available in WebAssembly on a really quick timeline. Another example, this is talking to D1, which is our SQL database. And so this all works today. There's people using this in production, relying on it at scale all around the world. So that's the boring stuff that works out of the box. But we have this kind of idea of being web interoperable. If you can do it in a web browser, why not in a serverless function? And so some of what we're working on right now is taking some of the exciting APIs that are growing in web browsers that I think really have a strong feature and making them available in a serverless context. So one example of that is the WebGPU API. So this launched really a couple of months ago. I know people have been working on it for years. But we're working on adding this to the worker's runtime itself so that you can make use of a GPU. And this works today. Check out the open source worker's runtime cloudflare slash workerd. And so you might be asking like, well what is this JavaScript API have to do with WebAssembly, like why are you talking about it at a WASM conference? But the thinking here is like the more of these that we can support, the faster we can make really interesting capabilities available in a serverless function from WebAssembly via bindings. And so what are the other things that are part of the web platform today that if you could support them in a serverless function, things like web transport, things like the WebRTC APIs from browsers, like what are the things you could build for a globally distributed application all over the world? And so I was talking about WASM bind gen before. There's also publishing wit specifications, wit definitions and kind of using wit bind gen to move a layer beyond that so that we're not kind of having to write bindings on a kind of language specific basis. And so, you know, weboperability really gives us like a lot of leverage, like a lot of portability across client and server. And that's where we think the opportunity really is. So I mentioned WASM bind gen, like writing bindings for each language, obviously an unsustainable thing for everybody in order to support the broad set of languages that we know that developers want to bring to WebAssembly. And so, you know, what we're increasingly thinking about is like what are the things that we can publish to make it easier for people to build their own bindings on top of Cloudflare? What are the interesting proposals that are happening around kind of on top of WASI? What are the standards of like, how do you talk to a key value store in a way that's portable across platforms? And so we're excited about all of these new standards. We love your feedback on workers. A lot of the reason why we're here at this conference is to talk to all of you and for you to tell us like what do you want out of Cloudflare's platform? So love to meet as many of you in the hallway, just free to say hi. And thanks for your time.


people's question is like, Ty, how quickly can I run through a dangerous jungle and get the treasure at the other side? And I'm saying for a lot of people, see Joel never went in that jungle of trying to make as much money as Elon Musk. And I'm like, I think that was maybe smart, you know, because like, I don't think now I've gone into that jungle. And sometimes I'm like, maybe I shouldn't have, you know, I think it's okay for some people to go into the jungle. Some people have the gift of knowing how to do business. Some of you have that gift, you know, about how to make money. And you can use that gift. But a lot of people who don't have the gift of making money, ask me, how can I be the king of the jungle? And I'm like, well, maybe you ain't making men for the jungle. Maybe you're meant to be Joel Saludson.


Surprisingly decent show on Discovery back in the day was a Amish mafia. Yeah, I heard about that It was the crickets and the antiques or something wasn't it? Yeah Weird show, but it was surprisingly good. I always wanted to make an old-school fighting game with Amish people But like the only difference in the character models would be the color of their suspenders other than that They would look exactly on the weapons like one would use a pitchfork and one would use like a scythe You


This 2019 GMC Yukon SLT in White Frost Metallic is stock number 11561Z. We are here at Summit Automotive in Fond du Lac, Wisconsin. You're new and used SUV headquarters. If you want to subscribe to our YouTube channel in the upper right is a link to go to our page and subscribe there. Remember to click the bell notifications. This one has the 5.3 liter V8 motor. It has been fully safety then inspected by our service shop has a fresh oil and filter change. All the fluids have been checked and topped off and this Yukon is 100% ready to go. We shoot all of our videos in 1080p 60 frames per second. So if you have HD capabilities on your computer tablet or smartphone device, turn them on right now. Because it is definitely your best way to check out the quality and condition of the vehicle before seeing it in person. And if you'd like to check out all the photos on this vehicle in the upper right hand produce screen is a link to our website. Click that and check us out there. This one comes with the 22 inch painted alloy rims. This rim is in excellent shape. It has Bridgestone Duler HL 28545R 222 tires. And these tires have right around half the tread left, maybe just a hair under. Front fender is in excellent condition. I didn't see any dents or dings on that. Comes with the projector lamp headlamps, the LED running lights, factory fog lights, front bumper parking sensors, front bumper is in excellent shape. I didn't see any dents or dings or cracks on that. And you do get the chrome trimmed grille. The hood is in very nice shape as well. Didn't see any dents on that and the passenger side front fender, excellent shape too. Passengers side rim, no scuffs or scrapes on there. And as you go down the passenger side of the car, take note of how clean the body is, how reflective and mirror like that paint is. We take these HD videos so if you are far away or even if you're close by and just cannot make the trip down, but you're still interested in purchasing the vehicle, you can see the vehicle, hear the vehicle, and have confidence in what you're looking at before you even get here. So that when you do get here, there's absolutely no surprises. And that's why we offer the Summit Auto Fastpass option in the upper right hand part of your screen. A link right to our website where you can apply for financing, get pre-approved, see a payment quote, even appraise a trade off from the comfort of your own home. So when you do get here, all you got to do is drive the vehicle, love it, and you'll be all set to go. Back tires have just as much tread as the fronts and you can see this back rim, no scuffs or scrapes on there. Coming around to the back of the vehicle, you get the LED tail lamps. The rear bumper is in excellent shape. It does have the towing package with the receiver hitch and seven pin wiring. That rear gate is in excellent shape as well. It is a power lift gate.  That's in pretty nice shape as well. These seats do fold down there, manual fold down so they go down like so. And you can see just how nice and flat those are. When the second row goes down as well, it's even, it's a bigger area for you to set stuff. A little bit of storage underneath there. And to get this gate down, there's a button right here that'll come on down. Now you do get a button for your flipper glass as well. And that goes like so, those shocks. Doing a nice job. And that's nice to have if you have a boulder camper on the back and you can't open up that gate. You still got to get back there. You can just get in on that flipper glass. Down the driver's side, just as clean as that passenger's side. No dents or dings on the rear quarter. And it does have the capitalist feel. Looks really good up there. And for full disclosure, there's back rim. Excellent shape as well. Four for four, no scuffs or scrapes on any of those rims. Really nice condition. And when you got 22 inch rims, you hope that you're going to get that. Has the power mirrors with the blind spot mirror inside. The SLT package gives you the tan leather interior. There are no rips or tears on those seats. They are in excellent shape. Both of these seats in the front are heated. You get the all weather GMC floor mats. Auto headlamps turned out four wheel drive. Tilt and telescopic steering wheel. Power pedals, stability control has lane departure, parking sensors. Power windows, power locks, power mirrors, memory driver seat. You get that really nice wood grain trim on the doors there. We'll hop inside, turn the ignition on so you can check out miles and everything. You can see that this one has 30,517 miles. Instrument cluster is very nice and clean. Comes with the leather wrap steering wheel, forward collision warnings, Bluetooth and information center controls on the right. Cruise and forward collision warnings on the left. Comes with the six-speed automatic transmission with the optional tap shift. This one has the Chevy MyLink radio with the factory navigation system. It also has AM, FM and Sirius XM radio capabilities. You get the hideaway storage bin behind there. And then you also have projection manager where you can project your cell phone to the screen via Android Auto or Apple CarPlay. So if you've got a different navigation system that you like better, you can project it to the screen and use that. Or if you just want to play songs and media and project it to the screen and use it through there, you can do that too.  You do get two USBs and a 12-volt power point. Pass in your side format and seat are in excellent condition as well. No rips or tears on that. Smells very clean inside this vehicle. I don't think it's ever been smoked in. The headliner is in nice shape and it does come with the power sunroof. Up here are your map lights, home link buttons for your garage door security systems and lighting systems, and on-star capabilities in that rear view mirror as well as SOS buttons. We'll take a quick look at the back seats. Start it up and then check out under the hood. Second row bench seating. No rips or tears back here. You do get the latch child safety system for any child car seats you may want to put back here. And you do get three spots. So up to three car seats in that second row. Here are your rear air controls. You also get a 110-volt, 150-watt plug-in. There's your controller for the screen and it does come with two headphones. One there and one there. And that's for the Blu-ray DVD player up there. Third row seats are in excellent shape as well. And you get those all-weather floor mats throughout on this vehicle. No rips or tears on that third row. Very nice and clean. And the bottoms of the doors all look really good on this Yukon. We'll start it up. Take a quick look under the hood. Starts right up. This is also where your backup camera shows up right there. I would personally like to thank you for checking out the video today. And hopefully from this HD video you'll have been able to tell just how clean this Yukon is all the way around inside and out. Under the hood, we have the 5.3-liter V8 motor. Engine bay is very clean, runs very smooth. Once again, this Yukon has been fully safe and inspected by our service shop. As a fresh oil and filter change, all the fluids have been checked and topped off. And this vehicle is 100% ready to go. I would highly recommend this Yukon from a quality and condition standpoint. And to see more pictures of it or one of our other 450 new and used cars, trucks, SUVs, minivans, wranglers, you name it, we got to go to that website right there, www.summitauto.com. Full pictures and descriptions of every single vehicle from two locations all at summitauto.com. And if you'd like to check out more HD videos, you can go to youtube.com. Also at summitauto, remember to like, subscribe, and share on this video and all the videos that you see there. In fact, in a second, you will see a link to subscribe to our YouTube channel in the upper left. A link to more Yukon videos like this one in the upper right. A link to this vehicle on our website in the lower left and a link to our latest YouTube video in the lower right. Click those, check us out, and we're super excited to help you with this Ultra Clean 2019 GMC Yukon SLT in White Frost Metallic. Thanks again for checking out the video.


All right, can everybody hear me all right? We got sound this time. We got sound in the stream. Very good. All right, I guess we're good to get started here. Welcome, thank you all for coming out. I know your time is valuable, and I appreciate you coming to hear what I have to say. I'm here to talk about implementing PowerSafe Atomic over-the-air updates, as the slide says. I want to thank my company, Toradex, for sending me here because they're covering the tab. It's my first time in Japan, and I'm happy to be here actually seeing people in person. It's nice to pick things back up after being on lockdown for so long. And I definitely want to thank the organizing committee for putting this together and getting us all out here to hopefully learn something. All right, so just real briefly, because I'm kind of obligated to mention it, we do have a booth here. I work for Toradex. We sell hardware. A lot of this talk is based on some of the architecture that we use in our Verizon operating system and platform. And that's a full end-to-end system that's intended to be used in an embedded IoT industrial controls, that kind of thing. And the architecture that we use for our over-the-air updates is what this talk is based on. So it's a third-party open-source package called OS Tree. And just to kind of give you the background, that's where the meat of this topic has come from. It's our work with the Verizon operating system and using the open-source OS Tree infrastructure to develop the secure atomic over-the-air update system that we use in our platform. Real briefly, since everybody seems we have to kind of introduce ourselves. So you guys know a little bit about me and maybe why you should listen to what I have to say. I've been in the embedded space, and embedded Linux specifically for about 15 years, embedded in general, much longer than that. Like I say, I work for Toradex. I'm currently a solutions architect helping customers implement our solution in their design. So we sell them the hardware. We also have software. And my role is to figure out what portions of our system are appropriate for their environment and help them make best use of all the features that are in our system. Briefly, what we're going to talk about today, obviously, is a little bit of background on what OS Tree is and how it's used in the over-the-air update space. We're going to define what it means to be power safe and how OS Tree implements that. And then we'll talk a little bit about the architecture in OS Tree and how specifically it will allow you to get these over-the-air updates in a safe manner. And then I've got a short demo, which is really just a video that I had to take. The timing on most of these things is pretty tricky. The system will finish and reboot before I can actually demo it live. So I've got some video that can show a few of the components of this system to help you understand how it's useful for this environment. So just some background to motivate this. Simple fact is the updates in software are no good. We all know this. We've all stared at these screens for many, many hours. And while you might be able to get away with this on your cell phone or your desktop laptop system, it really doesn't scale well when you're talking about an industrial IoT system where you might be deploying hundreds, thousands, or if you're lucky, tens of thousands of devices. You need something that is much more robust, much more reproducible, guaranteed to be atomic, and that kind of thing. We do need a better way to handle updates. There's been a lot of talk, a lot of different software packages implementing updates over the last five, six years. It's a hot topic, certainly, in the industrial and IoT spaces. And the simple fact is we need to do better. So why do we care about updates? I think we all understand the basic motivation for needing updates in the field. There's patches, not just to the kernel, but really to all software that's on your system. We know there's lots of critical vulnerabilities that are found all the time. We need to add additional drivers. Sometimes we need to add new hardware support. Our users might have a USB that they add, and they want to add a device driver for a specific device that may have not been tested at the time you produce the system. Custom firmware binaries are constantly getting updated. So for your Wi-Fi chip, there's your custom firmware binary that the manufacturer might push an update for. You want to make sure that your end users get access to that. And just other vendor updates, think Spectrum, Meltdown, all those kind of things that resulted in lots of changes at the firmware level and in the kernel. And then, obviously, user space, stuff, heart bleed. There's been quite a few documented cases in the last few years that make it all that much more obvious. One thing that a lot of people don't think about when they're thinking about updates, though, is you can actually use updates to deploy new features and therefore generate more revenue. Tesla has been a good example of this with some of their add-ons that you can add after the fact. You can buy them when you buy the car, but if you decide you don't want it then and you come back later, they'll sell it to you for even more than it would have cost you to buy it in the first place. And they're happy to do that. And because they've got a full-fledged OTA system, they're able to actually deploy that. So it's very desirable from a business perspective to be able to deploy these updates because you can make more money. And just in general, all bug fixes, obviously. That's the main impetus for adding OTA. The first step to securing a system is to being able to update it because the simple fact is all software has bugs. And I think we all know that. So what is OS tree? So OS tree is an infrastructure piece of code that we use in Verizon. And it's used in a number of other implementations of similar systems. This quote you see on the screen is directly from their documentation. There's a lot of words here. It's both the shared library, Swedish command line tools, yada, yada, yada. But there's a few important key points that I want to point out. It uses a Git-like model. So if you're familiar with how Git stores the source control information for your source code using a content addressable file store, OS tree does a very similar thing. So each file is essentially associated with a name that corresponds to its hash so that if I then try to upload that same exact file again, the system is able to very easily detect that it's the same exact content as another file. It doesn't need to deploy that object. It's able to easily locate it by that hash of the file. So if you're familiar with Git, you'll be well on your way to understanding how OS tree does things. Where it differs from Git is that OS tree is really meant to manage bootable file system trees. So Git is managing just an arbitrary collection of files, whereas OS tree is really designed about bootable file system trees. And one of the things that is kind of a result of that is you don't really have the concept of branching and merging in OS tree. You're not gonna go work on another branch and then come back and merge it together at the OS tree level. You might do that back at the source code level that generates the OS tree, but for actual managing of the bootable file system trees in the device, it's pretty much going to be a straight, linear history of the device. Sometimes you might have different versions that you put in there that may not be completely linear, but you're never gonna be merging back and forth within OS tree. And it also manages the bootloader configuration. That's just kind of a big, because of the fact that it is managing bootable file system trees, all of the configuration needed to select the appropriate bootable file system is handled through the bootloader configuration as part of the bootloader spec that you see here that's part of the free desktop foundation's definition. So a little bit more detail here. I mentioned the content addressable file systems and so all the files or as OS tree calls them objects are stored indexed by their checksum. Like I say, the system will take the checksum of the file and it uses that to generate a file name with some level of directories so that you don't have all the files in one directory. And then they are checked out via hard links. So that's the key here. So that when you have a bootable file system tree, each of the files in that tree are hard links to the appropriate objects in the repository. So you have the repository, which contains all of these checksum content addressable files just in a directory. And then what they call a hard link farm is generated to reference one specific release and each of the files in that hard link farm are a hard link to the object in the repository itself. By necessity, one of the characteristics of an OS tree bootable file system tree is that it is immutable. It cannot, it has to be read only. It has to be managed by OS tree. So you're not gonna go in and just modify files directly in one of these file system trees. OS tree does have mechanisms for handling things like configuration files in the Etsy directory and there's other things that are ignored as part of the OS tree managed files. But for any file that is managed directly by OS tree, the only way you will update it is to deploy a new version of your bootable file system. It does provide a mechanism to commit and check out branches as I said. Occasionally it'll get branches but you're never gonna get a merge back. The boot loader specification that's available on that free desktop link, it basically tells the system how to automatically discover what versions are available on a system, where the kernels are located, where the device trees are located for a specific version and that kind of thing. It does support both grub and u-boot for all of the systems that we support. We use u-boot on all our arm-based systems but it can also support grub and there are a number of desktop distros that are using it in that fashion. And it's important to know that it is a set of user space tools and that run at the command line. There is not an active runtime that is associated with OS tree. It's all based on the hard links and standard file system features so you don't have an active runtime that's slowing down your system or anything. It can run on any file system that supports hard links and because of the way that hard link form is set up, your performance is just the same as if you weren't using OS tree. There's no performance hit to using it except maybe a little bit in initialization time at boot time when it's processing and doing the auto detection of which versions are available and selecting the appropriate bootable file system tree. So this kind of repeats a lot of what I just said. Just a couple more points though. Normally the root of your file system is mounted as slash or the root. In an OS tree-based system, you actually bind mount slash to somewhere else in the actual file system. It's kind of an inception-based indirection that's a little bit awkward to get your mind around but normally once you've started playing with it, it's really not that complicated. To actually be able to do all this processing, it does require an init ram FS. So the system will boot, launch the init ram FS. The logic in the init ram FS is what does all the manipulation of the bootable file system trees and sets up that bind mount for the root file system. So I think this picture kind of helps explain it. So the diagram on the left is the actual raw files that are in the system. So you've got the files under boot and this is where all the bootloader configuration is stored and then you have the slash OS tree and that's what I referred to earlier as the repository. So underneath this OS tree repository under repo, you have a number of, that's where all the objects are stored. So if you look down inside OS tree repo, you'll see directories that are just, the names are just hexadecimal numbers and then the files are just longer hexadecimal strings that refer to the checksum of each file. And then under the deploy directory, that's where we start getting into these bootable file system trees. So deploy and then we've got a specific OS identifier and then another deploy directory and then underneath there you will have multiple commits and each of those commits is essentially the entire bootable file system tree. So the runtime view on the right is essentially looking, this is where that bind mount comes in. So you've got the files on the left but the runtime view is that slash is actually looking down at one of the specific commits down inside that OS tree repository. And then just so you're aware, there's also the var directory. Note that in the repository view, var is actually outside of the commit object. So in this case, var is actually persistent data that's essentially unmanaged by OS tree so that anything you put in that portion of the directory tree will show up and persist across versions and across updates. So for instance, this is where we store our container volumes and configuration and that kind of thing. Anything that we, you know, any user data that's part of the system goes under var. And then I will mention, I don't think there's actually a slide on it but there's also some special handling for Etsy. So you see here, we've got Etsy that's in the managed portion of the repository but actually what happens is the Etsy is there's a read-only version and then there's the runtime version which is read-write and on any update, the system will actually do a three-way merge between the previous version, the new version that you're installing and what's currently on the device. So any changes that your system makes in the field will be maintained via that three-way merge of the slash ETC directory. And just briefly, some users of this. I mentioned that there's a number of desktop distros that are using it, Fedora Silverblue and Fortora IoT, notably. I've not personally used them. I know some people who have and like them a lot and then there's a number of others that I have not played with. I come from the embedded Linux space so we're focused mostly on Yachto-based distributions. You've got the meta-updater layer that can be included in a Yachto configuration to provide all the classes and configuration needed to add OS Tree support to your configuration. And then some other interesting uses where they actually go up to the package management level where things like RPM OS Tree and Flatpak, they actually use OS Tree under the hood to essentially reimplement the standard RPM view that we have of the world today. Again, I've never played with them. I'm not sure how they work but the intent there is to give you much more atomic updates and rollback which has always been an issue when you're dealing with package-based updates. And some alternatives to something like OS Tree, at least as far as implementing over-the-air updates. The one that most people have heard of is the dual-AB, dual-bank redundant partition setup that a lot of the OTA systems out there use today. Obviously, the downside of that is it takes up additional disk space. Package managers, there are systems, you can just go in and do an APT update and APT disk upgrade. Doesn't scale real well because it is hard to manage that atomically. It's hard to guarantee that the exact same set of packages you tested in your lab is exactly what is on all your devices in the field. And then there are, you can do some containers. There's SystemD has some container infrastructure that's part of SystemD itself. Again, I'm not terribly familiar with it so I can't give you the details. But to give you some of the advantages we think of OS Tree is the first one is space-saving. The fact that it's got automatic deduplication based on the check sums makes it very space-efficient both in terms of the space on the disk as well as the amount of bandwidth you need to download any new update. OS Tree is very capable of figuring out exactly which objects you need and transferring only those files that you don't already have or those files that have changed. Whereas when you're doing a dual AB you might be able to do a binary delta to transfer less data but you still have to write the entire partition. So OS Tree is gonna save you significant time both in download and updating of the flash. And another very important thing is that integrity can be verified. So the check sums and cryptographic validation of all these objects is integrated into the system. You know that if your system has downloaded completely in the check sums, check out. You have exactly what you're expecting to have. You don't have to worry about man-in-the-middle attacks and things like that. That's all integrated in part of the system. And yes, I'm sorry? You are assigning the list of changes and verifying that something has been altered that way. As far as I know, yes, I couldn't, I can't tell you all the details of it because it's a little bit outside of my scope. I haven't dealt into the details of that. But I know for instance, in the get side of things, let me repeat the question. I forgot to, I forgot that. So the question was when you're dealing with the cryptographic check sums and that kind of things, what kind of guarantees does the system have that they can't be modified? And it's very similar to the way get handles things so that if the file is changed, the check sum is gonna change. And then there's signatures based on those check sums and that kind of thing. So they all feed in to be able to guarantee that the files are unmodified. I know I didn't really give a real thorough answer, but that's essentially it. And another thing that we find is very valuable. The last thing on this slide is the fact that it is immutable. The system is read only. We know that our devices are running the exact version of all the files that we have tested in our lab. We don't have to worry about package drift where somebody modified something or they install the package. They either install everything for a new update or they install nothing from a new update. There's no way for them to just do a half update and get two packages out of six or whatever. So that gives us kind of a rudimentary revision control of the bootable file system tree on the device which helps, especially when you're dealing with large numbers of devices. So what does power safety mean? The issue with a lot of these devices is that they are generally stored in environments that you as a system developer don't have control over. Our customers range from people putting things in public spaces all the way to very tightly controlled lab environments and industrial controls environments. And so the range of customers we have spans the gamut but in general the idea, you wanna think of these devices as they're pretty much out of your control. And so they could be attacked at any time by network attackers. Somebody could reach over and grab the plug and yank it out of the wall at the wrong time. And it might even be very expensive for your team, for your operation staff to access them. They're generally not on site with your team so it could be very expensive. And the idea is you wanna be able to make sure that when you are deploying updates, any of these kind of situations that can interrupt the update do not leave you with a brick system. The simple fact is the cost is just too high. So you wanna make sure that the system is either completely installed or not installed at all. No component outside of the update system even really needs to be aware that an update is being installed until the update has downloaded and been properly propagated through the file system. So a couple of things that the OTA system has to do, it has to detect these failures and know when they happen and it has to be able to clean up after itself. It has to be able to handle OS changes to that bootable file system tree as a completely atomic operation and it has to be able to handle automatic rollback. That's also very important, right? I deploy a bum update with a kernel that oops is on boot. I don't wanna leave the device sitting there completely waiting, so you wanna use Watch Dogs and things like that and have logic in that init ram FS that's going to detect that and be able to roll back to that previously known good configuration. So looking at the OS tree update states, so these are the main states greatly simplified of an OS tree update. So on the left in blue is you have anything that happens in the context of the old deployment and then on the right in green is anything that happens in the context of the new deployment, okay? So when you're getting ready to do an update, obviously the first thing you have to do is fetch, then you assemble that deployment directory that we looked at inside of the repository and then you do that three way merge with Etsy that I mentioned and you'll switch over to the new deployment and run from there. So let's talk about each state individually just so we can kind of understand how OS tree is able to detect failures at any of these states. So fetching pretty straightforward. All new objects are fetched over HTTP or HTTPS. There's compression built in. Any existing objects don't need to be redownloaded. Fairly straightforward stuff, everything you would expect. Check some is verified and then the objects at this point are stored raw inside your repository under that OS tree repo objects directory. At this point they're just objects getting added to the repository. You haven't done any configuration. You haven't actually built up the bootable file system tree yet. And there is an extra feature you can actually enable a binary delta per file. So depending on what files are in your root file system you might even be able to get a little bit more savings of download time by using per file deltas. Out of the box OS tree generally does not enable that. Since it enables downloading only needed objects you already get quite a bit of savings there. And so the additional savings for implementing per file binary deltas is minimal. So if power failure occurs during this stage there's a couple things that happen. First thing is any of the objects that are downloaded they are fetched to a staging location first. That is based on the current boot ID. So each time the system boots there's a unique boot ID that is generated and the objects that are downloaded are gonna be indexed based on that. After all the objects has successfully been fetched if the power fails during that fetch to a staging location then this is one of the things we'll see an example of then it's able to clean that up. Once all the objects have been fetched then they can be moved from staging and clean up the staging area. So and then when the system boots it can inspect the staging area and if there is something associated with the boot ID that is not the current boot ID it's able to say that there must have been some kind of power interruption and is able to clean up. And then any active deployments that are ongoing from the server at that point will be redownloaded and the objects retransmitted. So the next step is assembling that deployment directory. So this is where we create the hard links that make up that bootable file system tree. So we're going to create a new deployment directory based on a checksum that is created cryptographically tying all the components together. This is where all the hard links are created for each file pointing to the specific object in the repository and the ability to detect failures in this case it basically all comes down to a single symbolic link that is created at the very end. And if that symbolic link is created then the update has completed and we're able to switch to it if that symbolic link does not exist but some of this deployment directory stuff exists and the system is able to detect that a power failure has occurred. So it's all tied to that atomic creation of that symbolic link at the very end of this phase. And then after we've created a complete new deployment directory then we have that three way merge of the Etsy directory. So the system is able to look at the unmodified Etsy configuration files from the old deployment the unmodified Etsy configuration files from the new deployment and any files that have been modified in the active running system and is able to do that proper three way merge to make sure that any new changes get pulled in any changes from your devices that have been made in the field get pulled in and that all those changes persist properly. Similar to the previous phase a data origin SIM link is used to determine when this phase has completed. So once the three way merge starts that file doesn't exist once the three way merge completes the system is able to create that SIM link and be able to detect on which side of the power failure you were sitting at this point. And so now once that has completed we're ready to move into the new system. So at this point we're gonna update the boot configuration now we're in the context of the new system even though we haven't booted yet we've got everything we need downloaded to the system we've got everything staged properly so that we can boot into the new system and so now we're just going to modify the boot environment so that on the next boot we'll select the new version. So a new boot directory in this case will be created with an extension for the index so in the case of Verizon we only ever have two we have the current and a new. OS tree supports an arbitrary number but that index is essentially which version is at zero or one and it's just gonna ping pong back and forth between them and then similar to the other phases there is a SIM link created once all of the configuration is updated and that SIM link is just OS tree slash boot and that will point at which index we're going to be booting the next time around. So until that SIM link has created at the very end of the process the system will stay in the old version if it gets past the point where it is able to create that SIM link then when the system reboots it will be in the new version and then the next step is the reboot that's pretty straightforward and then on any boot and of course the reboot could actually happen anywhere in here this is only showing the successful reboot but if the reboot happens at any of the other phases there are the system has ways with those SIM links and with the boot IDs of detecting where in this process it was and being able to clean up and retransmit as appropriate. So with that I've got just a few minutes left for demos let me just see if I can pull this up here. Yeah so this is the first one I hope you can see it in the back it's a little bit small. So what I'm running here is I'm actually running a horizon system with OS tree in and all I've got displayed here is the staging directory so this is just periodically viewing the staging directory that's happening during a download of a new version. So I wanna say it takes a while before anything happens thank you. So at some point here we come in here and we will start to see things showing up in the staging directory and what I've done in this particular instance is once the staging has started I just yank the power on the device and so we see now that in the staging directory our files are based on the boot ID which is that big long string I know it's kind of small you can't probably read it in the back but you could probably tell there's a long hexadecimal string there and that's the current boot ID. So if we let this run a little bit more at some point I reboot so we start to see the reboot here and then when the system finally gets booted I log in and we look at the staging and so in a moment we'll see that the staging directory has gotten cleaned up so it knows that the staging was from our previous boot ID or rather from not the current boot ID is all it knows and so it just deletes it and we'll start again so we'll see another staging directory show up here in a minute if I didn't cut off the video before that happened so it cleans everything up but the server side component in this case is still deploying and so I guess I killed it before it actually got done. So that's the basics for power safety during a download. Now the next one is the slightly more complicated one is the power cycle that happens during a deploy and activate phase so at this point, let me just pause right there so you can kind of see a number of different things here you see the sim links that I was talking about that point to load or ending in the dot one and then you see we have a couple different boot ID based repositories and we start to see new things starting to get created here, if I come forward a little bit. Okay, now we're starting to see the system is actually has downloaded all the objects and now it's creating that new bootable file system tree and then we also see now we see that we've got the loader dot zero but the loader sim link still points to loader dot one so that means that the system is actually actively setting up the boot configuration and then I think at some point, yeah, so now we see loader points to loader dot zero and the system reboots so the system is then able to clean up and retry the downloads as needed. So with that, I think we have just a few minutes for questions, let me get to the last slide here because there's a number of resources and links, the slides are up on the conference website, this last slide, if I can get through all of this has a number of documentation links and things that are potentially very useful. So a lot of this talk came out of this next to the bottom link the lot more details about this on that blog post. So with that, we've got I guess just five more minutes for questions, anybody have any questions? Yes. On to the edge device and then rebooting into the new deployments. Do the old part of the main on the device for any couple of questions? Yeah, so the question is when you're installing a new update what is done to clean up files that are no longer needed? So the way that repository works is it's just a bag of bits, you've got a bunch of files in there and for one release, you've got one set of files for another release, you've got another set of files, some of them may be the same and in general OS tree won't clean up at that point. Any more than get would clean up an old version unless you explicitly tell it but there is a mechanism within OS tree to prune old releases and so with our system, once we have detected that the new update is installed all the user installed post install checks have completed then we will prune out any unneeded versions to help save that disk space. But we obviously keep it around because if we have to roll back we've got to make sure that we've got it. So that prune only happens once we have completed all the post install and sanity checks. Yes. Yeah, sometimes the system requires file system upgrades for example, like from, for example, like FAT32 to NTFS, such kind of stuff. So how do OS tree deal with those things? Simple answer is it doesn't. We are, in our system, anything that requires manipulating the partition table is pretty much unsupported. We do have, we are getting ready to roll up a bootloader update feature but that uses EMMC boot blocks and a few other things that are part of the hardware. Once these devices are deployed in the field changing anything at the partition level is essentially impossible to do completely safely so we don't allow that. But I will say that that is one of the benefits of using OS tree versus the dual AB mechanism is with the dual partition mechanism you have to define the sizes of those partitions at the very beginning of the system's lifetime. Whereas with OS tree you have a single partition so you don't have to worry about the sizes changing but yes, if you had to do something like move from the EXT forward to ZFS for whatever reason you're gonna have to find an external method to do that because you just can't do it robustly. All right, and I think we've got just another minute or two for questions. Is there a plan for something like DM integrity? I'm sorry, a sign for what? Is there a plan for something like DM integrity where you have a per runtime check on your data that there's nothing either malicious or unintentionally damaged? Yeah, so that's a great question and it's something I didn't point out. All these safety mechanisms built into OS tree and the OTA system only protect at update time. DM verity is intended to protect at normal boot time, right? It uses essentially check sums of the kernel blocks and all the blocks of the root file system. So they're really orthogonal to OS tree. Specifically, are there plans to include it in Verizon? Yes, there are. It's not included today. I don't know all the details because ARM has trust zone and all these other things and there's a lot of components that will play into that but that's completely orthogonal to the OS tree mechanism. Yes, so for me as a developer, I'm just interested, for example, to really make a quick update to the file system. For example, deploy a new binary of my own application, right? So I understand the system is read-only but how does it work in reality for me as a developer? What have to do? Yeah, so I mean, as a developer, there are ways you can kind of temporarily remove that read-only and just force a binary in. It will break the ability to then do updates with OS tree later. So that's really just a do it at development time and know that you're gonna have to wipe the system later. The alternative would be to store those binaries in a portion of the file system not managed by OS tree. So under that var directory, you could just scp a file in or even just put it under slash etsy. We do have IDE integrations that actually do that. They copy when you're developing your code. They just copy a whole bunch of files into the user's root directory, which is not actively managed by OS tree so that you don't have to deal with doing a full OS update every time you want to deploy a new version of your application for testing. Okay, thank you. Yep, anybody else? Very good, well thank you so much. And if you have any questions, I'm around all week.