still morning. It's 10, 10.47, right? It's Sunday, November 24th. 24th, see? Without Fred, what would I know? Nothing. Anyway, we're headed for something. You are good for something. So we're headed over to the grocery store this morning and of course we created Lily and she started crying before we even got out the door. But you know, what are you gonna do? We can't bring her to the store with us. So I spent the morning in a women's group chat which was really great and productive this morning. It's with the my sister's fairy feng shui group and so if you want details on that and her Facebook group and where you can ask her how to join the chat and all that stuff, more of what the chat's about. I'll link her Facebook group in the description below. You can go check her out. It's not about art, it's about improving oneself. Anyway, we are, I also spent the morning deleting all haul and giveaway videos here off of YouTube. My channel is not about bribing you all to watch the channel or bragging about what art supplies I may or may not have. You don't have to have specific supplies to create great art or to work through art for therapy. I also don't want to bribe people to watch my channel. Either want to watch it or you don't. So all those videos are gone. Of course I saved them. I do have them in my backup but they're all gone here off of YouTube. I think I might be delete happy but you know there we go. Anyway, we're gonna go get our shopping done then we've got some house things to do and I'll be back. Sunday evening Fred's cooking. Fred's cooking. Yeah, we're gonna just put this as part of the vlog. So he's complaining that he's almost out of rum. You're not completely out of rum. There's some in there. There's not for one more. Yeah, maybe. Anyway, so we're making, we got another meal kit from Fred Meyer which is part of the Kroger chain. Not sponsored but if you want to sponsor and you're watching, let me know. We do like their meal kits because they're easy and it's perfect for just two people. This one is the crispy orange chicken which I don't think we've had yet. Sometimes, yeah, I mean, sometimes it makes a lot of food. Yeah, this one's calorie heavy. Holy cow, this one's very calorie heavy. I won't be eating a full portion. It's 1400 calories for one half of the kit. That's more than my daily allowance on Luzet and I've already had breakfast and lunch and snack. Wow, all right. Yeah, it's crazy. Anyway, so Fred's cooking. There's not a lot for me to help with this time because I think everything's done already. He just has to assemble. Yeah, so he's gonna do it in a pot the old-fashioned way rather than use the rice cooker. I know. That's what I told you to do, so you're doing it the way I said. Anyway, so he's doing that. We already also today spent some time replacing all the smoke detectors in the house because we discovered that the old ones, the date on them was 2005. For those that don't know, smoke detectors only have a 10-year lifespan that's recommended. So yeah, that's an issue. So we got that done and we cleaned out some closets upstairs and got the Thanksgiving grocery shopping done. So now we're gonna just cook some dinner and we still have crazy pants. Where is she? See the black... I see the black face right there, the black spot on the couch. Right there. That's crazy pants. Huh.  She does not like that. We went out yesterday to a concert. She was really pissed. Yeah, we went last night. If you watched last week's vlog, then you know we want to see the Trans-Siberian orchestra, which was great.  And when we got back, we couldn't get her settled down to go to bed until after midnight. After midnight. And then I had to be up early this morning for a women's group Zoom conference call. And so, yeah, I'm all tired. And we have to be up early again tomorrow because we have me and my bright ideas. I made morning dental appointments. I don't know what I was thinking. Yeah, you weren't. I evidently wasn't, yeah.  Okay, so anyway, we're gonna do this and we'll let you know how it turns out and we'll be back. Fred's all done and there you have it. Looks pretty good. We split it into three servings rather than two because yeah, holy cow, it's a lot of calories, but it looks delicious. Good job, friend. See, lots of calories. Yeah, a lot of calories. All right, we're gonna have some dinner and I'll be back tomorrow. Bye guys. It's Monday morning, November 25th. It's about a quarter to 8 a.m. or so. We're on our way to a dental appointment at our new dentist. It's our first visit. I'm a little nervous. Not too much, but a little bit. I've never had a different dentist other than my old dentist, which is my uncle, my dad's younger brother and his sons. I've always gone to Bronzini Dental in Millbray, California always since I got my first tooth. So this will be my first trip to another dentist. I'm sure it'll be fine, but yeah, a little bit nervous. All right, I'll be back. Okay, so the good news is I like the new dentist. The bad news is I need some work done. It's not anything I didn't know about. My uncle for quite a while now has been watching a few spots in my mouth. I do have a tendency to clench and grind my teeth, which is how I cracked one of my upper molars a couple of years ago, or more than that now I think. And I had to have a root canal and have it drilled out because it got infected. And I knew there were a couple other molars that were cracked he was watching. We're going to just go ahead and rip the bandaid off and Wednesday, yeah, two days from now, I'm getting both of them fixed. And then I also have a little spot near the front that they were also watching at my uncle's office that they thought was maybe a cavity. And we're going to go ahead, it is a cavity. We're going to go ahead and have that filled.  I said, can we just do it all at once? He said, that's like a two or three hour office visit. I said, if you're okay with it, I'm okay with it. I'd rather just get it done all at once. It's the day before Thanksgiving, but you know, my mouth is going to be sore. It means I won't overeat. Okay, there's always leftovers. 15 minutes before 10 o'clock on Tuesday, November 26th. The ring keeps going off this morning. It's mostly just been the cats, but right now it's Bambi. You see that? She's looking right at me like, what are you doing? Why are you filming me? This feeling. It's pretty cool that she's just standing there. I'm sure she'll run as soon as I turn the car on or make a loud noise like shutting the car door. But I got to go. I got to get to the hairdresser. All right, I'll be back. So as you heard from me earlier, I went to the dentist. I had some dental work done. Two crowns and a cavity to be exact. Yeah, a lot of poking.  It's not too bad. I did take a Tylenol, but I'm not taking and it was just an overly counter Tylenol. It wasn't anything special. I'm not going to do more than that. But we want to get some steps in.  So we're, but it's too cold out to be honest. It's only, well, the car dashboard says, well, it said 50. Now it's going down. It went down to 49, 45 and icy outside. So we are going to actually go to the mall and go walk around the mall. I know it might be very peepily, but that's okay. Bob does need a couple of things. So we're going to go do that. If anything interesting happens, you know, I'll bring you with me. So we'll be back. So that trip to the mall was interesting. That was interesting. And then of course there was the little boy who threw up all over the floor. Yeah, and his parents were trying to be very discreet about it, but nobody was helping them. So I went and got a girl from a local shop and said, can you call security or something? They're having a problem. And then there was the couple that was walking around the mall. He was carrying a camera, taking pictures of her. How do we describe her? He didn't have much on. She did have a long sweater on, I think, to kind of cover up and be discreet about what she had underneath it, which was a nude colored dress, a flesh-colored dress. Pretty sure she had no underwear on. On top or bottom. So yeah, that was a thing. So yeah, that was interesting. We're gonna head over to the Sporting Goods store now to buy Dear Corn. It's Thanksgiving morning November 20th. What is it? November 28th. Thanksgiving Day here in the United States. And so we have the turkey in the oven. And it's just gonna be my husband and I and our daughter and her boyfriend, fiance. I have to start calling him son-in-law soon anyway. So they're all coming over with Lily of course. And it's gonna be a good day.  And I'm all good. I do have temporary crowns in. So I need to be careful what I eat, how I eat it for the next couple of weeks. But it's all good. I'm not in any pain. I took a Tylenol yesterday for some of the aching. But otherwise I didn't need anything. It is weird not being able to wear my night guard. So I am having dental work done because I do clench and grind my teeth. And I am, I have some cracked molars. And I had one repaired years ago and I knew another one was coming or two. I knew I knew it was gonna happen. But my new dentist, who I totally agree with him, we decided not to wait for them to be infected or have other issues. They already were pretty cracked, these particular two. So I'm like, let's just fix them. Let's just fix them. So we did. So it's all good. I can still eat turkey and mashed potatoes. So it's all good. My husband is downstairs. If you can hear his voice in the background a little bit talking to his mom. And it's gonna be a good day, I think. So anyway, I hope you're all having a good day, whether you celebrate Thanksgiving or not. And you're spending it with your family. And I'll get in 22 a.m. Friday morning the day after Thanksgiving. I didn't get a lot of any footage of the dinner yesterday. There's just too much going on. As you can hear the crunching in the background, Lily is here eating breakfast. I'm gonna spend the day today putting up the Christmas tree and decorating the house for Christmas. So we'll see how that goes. Alright, I'll be back. At the Slay Bell's Christmas tree farm. We're gonna cut down a real tree. That's not something we've done since Rebecca was little. But we're gonna give it a shot. So let's see how this goes. The kids are gonna cut down a tree. Becca keeps saying she's gonna do it. But yeah, this ought to be funny. We'll be back. It's not gonna be cheap. We have an angel. We always put an angel. Where are you from? Are you a pair of an angel? Yeah, you want to make sure it's far a little to the ground. Oh, look at the view. The view. You can see Mt. Hood. Oh yeah, the clock says 12.15. As you can see, we're getting ready for Christmas. The inside's pretty much done. We just need to do the outside, which we may do today. We may do tomorrow, but we're gonna try to do before Monday. I think I vlogged yesterday. We went and cut down the Christmas tree. It's the first time in a long time we've had a Christmas tree. We either haven't done this since Rebecca was a baby, or more likely than not. We've not done it since before we got married. That was a long time ago. So anyway, we're just having a quiet day at home. We may go out and do some running around. We don't have any particular plans, but we'll see what happens. And if we do anything interesting, I'll bring you with me. Of course I will. Alright, I'll be back. We got the outdoor decorations and lights up. One of the timers I think is set wrong because those lights above the garage are not on. Where are we? There we are. So he's working on it, and I'll get you all a picture of what the house looks like. Anyway, it's been a fun day. A cold day. We don't have any snow, but it feels like snow weather. Anyway. Oh, there we go. There are maybe just a loose connection. Anyway, he's working on it. They're going on and off. Anyway, it's been a fun week. I have some projects that I'm gonna work on this coming week. YouTube apocalypse or not. I'm gonna... I do have some things art-wise I'm gonna do. I will be sharing them on YouTube if we're still all on YouTube. I'll be sharing them on Instagram, Twitter, and the appropriate Facebook groups and all of that. So follow me on one of those to keep in touch. I guess in case I get in Facebook jail again or something happens on YouTube other than what I expect, you might want to pick a couple of different social media platforms to follow me on. In the meantime, ornaments on that little tree. We're gonna try to light it but we just can't figure out there's no outlet out here and it's not safe to run it across the pathway. But there's the hedges and the LED lights in the window. Garage lights. That's the most that we're gonna do but it looks good I think. We all hope you've had a great week. Follow us on social media. Pick one or two of them in case one of them locks me out and I get in jail again. Anyway, there's some fun projects coming up for the new year for the month of December. I'm gonna be trying to finish up some watercolor journals. My friend Vicki Brown has been inspiring me to pick them up again so I'm gonna do that and it is cold out here so we're gonna go inside. We hope you've had a good one. Don't forget to like share and subscribe and above all go out and have a great day and do something nice for yourself because you deserve it. I'll see you later. Bye guys.


Now, to actually create a table, we first have to get a connection to our database using DAO, and then once we have our database connection, here we call it DB, we then invoke on it the method define underscore table. And with the first argument we specify the name of the table we're defining, and then after that we have any number of field objects representing the columns of this table. And for each field we first specify the name and then the type. So here we're defining two tables, the first called cat, the second called cat underscore owner, and in the cat table we're giving it four columns, a column name of type string, a column age of type integer, and a column weight of type double, and then also a column cat owner, which is a reference to the cat owner table. So this is a foreign key pointing to the primary key of the cat owner table. So in the cat owner table we don't define any primary key because as I said the DAO will automatically give every table a primary key column called ID, which is an integer. So both the cat table and the cat owner table here have primary keys, we just don't explicitly declare them. Having now defined these tables, the connection object DB will have an attribute for each table we defined. So there's now db.cats, which returns the table object representing that table. And db.cat underscore owner returns the table representing the cat owner table. And within each table object you have an attribute representing each column. So within the cat owner table there's dot name, which returns the field object representing that column. So do note there's a bit of a strange asymmetry here because whereas we create actual field objects, which we pass to the defined table method, we don't create any table object. It almost would make more sense if we were to create a table object first and then pass that to the defined table method, but that's not how it works. I imagine simply for the sake of conciseness. Now be clear that the end effect of invoking defined table is to, one, actually create these tables in your database and also to create these table objects which you can then interact with in your Python code. The strange part about this is that you will invoke the defined tables method to define your tables in every request, which sounds odd because wait, you don't want to recreate the tables in the database every single request, that doesn't make any sense. Once we create the tables the first time we didn't just want to use the existing tables. So what actually happens is in your application there's a subdirectly called database in which are kept files describing the tables of your database, such that the Dal knows when you invoke defined table whether or not that table already exists and whether therefore needs to actually then create them in the database or not. Only when we use defined table to create a new table with a new name does the database actually then create a new table. Likewise, we can modify one of our existing tables by simply say here just removing one of the fields from the cat table or adding in another field, the Dal will then see that the table as you just find it differs from how it's defined in the database directory and so it will then actually alter the table in the database, which in SQL is done with the alter table statement. Now in the terminology of databases when we take an existing schema and we modify or remove one of the existing tables or one of its columns, that's what's called a migration. It's an altering of the schema. I believe it's called a migration because usually it's used in the context of taking data from one database and putting it into another so there's like migrating from one database to the next, but the term also came to be used in reference to just altering an existing database itself, not just moving your data from one database to another. Basically, a migration means we're changing the structure of the data and that can be problematic because sometimes you can't just take say an existing column and convert all of its values from one data type to another. So while in some circumstances we can successfully and without error performance called a auto migration, basically a migration that's done for us automatically, but in other circumstances we can't get away with an auto migration because the database is not smart enough to handle the changes for us or we're going to end up losing data if we try and do an auto migration. So by default in Web2Py auto migrations are enabled but there's a configuration setting where you can actually disable them or you can actually disable them on a table by table basis. The defined table takes an optional named argument migrate. If you specify an argument of migrate equals false when you define a table that will stop the dial from even trying to create or alter the table as it exists in the database. It will just assume that the table is as you defined it with the define table method. So the problem there though is that there might be a discrepancy and you might get an error later on when you try and actually use that table if it doesn't conform to the definition as you laid out with the define table but at least it doesn't sure that you won't harm your data so that's why you would set migrate equals false. So to restate what defined table does it creates a table object and assigns it to an attribute of the connection object so like db.cats here but then also it will actually create that table if the table doesn't already exist or it will modify the existing table of that name if it doesn't conform to the definition we just specified with defined table. Now once we have our table we want to put data in it and we do so with the insert method of the table object. So here we're invoking db.catowner.insert and the insert method expects a named argument for each column so our cat owner table has two columns name and address both strings so we provide a name argument and address argument and this inserts a new row in the cat owner table and remember that every table implicitly has a primary key column named id of type integer but that's automatically set so we don't specify an id because the database is picking an id for us. The other notable method of the table object is drop which when invoked will actually remove the whole table from the database. Of course removing tables isn't something you commonly do in the normal course of an application but it's there if you need it. Now to select update or delete rows from your database you need to first create what are called query objects and the primary means we have for creating queries is not to invoke the query constructor itself but to actually use these overloaded operators of the field type. The field class has operator methods for the quality operator the not equality operator less than greater than greater than equal to less than or equal to so when you use those operators and the left operand as a field object what you get back is a query object. So we have three examples here the first expression db.cat.id greater than zero well first db.cat gets you the table object and in that table object we have a column id which is the primary key of that table so db.cat.id is the field representing that column and as i just said the field type overloads the greater than operator so given the way overloaded python operators work this is actually invoking the method double underscore gt double underscore of the field class and it's passing to it first the field itself and then this the second argument the value zero and what the method returns what this expression evaluates into is an as yet unperformed select delete or update statement in which we are selecting the rows from cat where cat.id is greater than zero so be clear this just creates an object representing the as yet unperformed select update or delete likewise in the second line first db.cat.id that's a field object representing the id column of the cat table and when we use the quality operator on a field object when field is a left operand that's actually invoking the method double underscore eq double underscore of the field class and to that is being passed first the field object itself and then the argument four so what we get back is a query object representing all the rows in cat where the id value is four which of course should be just one row because multiple rows aren't supposed to have the same id id is supposed to be unique in the last example here we're again using the quality operator except now both arguments are fields not just the left operand db.cat.catowner that is the foreign key column cat owner in the table cat and db.catowner.id that's the id column in the cat owner table so the query object that results represents a selection of rows from the joining of these two tables because we're involving columns from two several tables so of course they have to be joined first so it doesn't enter join between cat and cat owner and then we're filtering from that join the rows in which cat.catowner equals cat owner.id so note that the thing a bit peculiar about how we express queries in the dowel is that the joining of tables is sort of left implicit it's like we're just specifying predicates to filter and then whether tables get joined together that's that's just implied from whether there's more than one table involved if you just have one table no tables need to be joined but because the expression here involves two tables both cat and cat owner well they have to be joined first so in the query the join is implicitly there now to actually use these queries to do an actual select update or delete we first have to wrap the query object in a set object because it's the set class which actually has the methods select update and delete which as i discussed earlier i believe is actually just a mistake in the api the query object itself should have just had these methods itself but for whatever reason this unnecessary complication made it in in the early days of web2py and that's stuck around since then so here in this example code the first line we're creating a query object with the expression db.cat.id equals two remember db.cat.id that's a field object the equality operator then is overloaded for field objects and that method actually returns a query object which we're assigning to the variable query the parentheses around the expression here are not necessary just included them for clarity and then in the next line the way you get a set object wrapping the query is you pass query to an invocation of the connection itself it's kind of strange you don't pass it to a set constructor rather our object representing our database connection db is an instance of DAO and the DAO class includes a method called double underscore call double underscore and that's effectively overloading the function call operator itself which is a pair of parentheses so when we invoke the connection object db as if it were like a function what we're actually doing is invoking that call method and so query is being passed to that call method and what it does is it returns a set object wrapping the query so not only is this a step which should be unnecessary it's done in a very strange way the justification for this oddity I assume is that at the time it seemed like the syntactically most compact way of expressing this common operation rather than having to create some other method to which we pass both the db object and the query object you could just invoke the connection object itself and therefore not have to pass it in also as an argument but again for whatever reason that was mistake because the query object already contains information about the connection because these field objects belong to tables and tables belong to connections so the query object should have encoded within it knowledge of what the connection object is so we don't need to introduce it this in this very convoluted way either all the information that's needed is there in the query object and I guess for whatever reason when Massimo first created Wattupai he just didn't see this and so he missed this opportunity and introduced this unnecessary complication which is just stuck around but anyway so we have our set object and now we want to actually perform a select update or delete well say we invoke the delete method that then performs a delete of all the rows in the cat table where cat id equals two if instead we were to invoke update we would specify named argument for each column we want to update and so we'd write lives equal eight to set the lives column to the value eight name equals fluffy to set the name column to the value fluffy so this invocation of update will modify all the rows in the cat table where id equals two which again is presumably just one row and so the values of lives and name in that column get updated do the values eight and fluffy respectively now with the same query object if we were to invoke the select method with no arguments that would perform a select and return the results as a rows object and because this would be a select over the cat table in which the id column equals two that should again be just one row and to get the individual rows from a rows object we use the subscript operator just like a list so rows subscript zero here will return the first row the first and only in this case and then with the row object we can then access the columns by an attribute of the same name so row dot lives here is an expression that should evaluate into whatever value lives has in the column where the id equals two now if you don't want to select to return all of the columns you can specify which columns you want by passing field objects to the select method so here we're just selecting for the columns name and lives of the cat table and so if we then try and access some other column in one of the raw objects well that will throw an exception because there is no such attribute because we didn't request it in a select so here row dot wait should throw an exception because the row has no such a tribute because we didn't select for that column lastly an important thing to keep in mind about row objects is that if the query involved multiple tables well then you might get name collisions between columns of the respective tables so when your query involves multiple rows the column values get stored as attributes in storage objects named after the tables so in our row here to get at lives of the cat table we we write row dot cat dot lives row dot cat here is a storage object representing all of the columns of our query from the cat table for the sake of more complex queries the query class overrides the ampersand and pipe operators which effectively then are the and and or operators respectively so if you want to and two queries together you connect them with the ampersand operator if you want to or them together you connect them with the pipe operator and then query also overloads the tilde operator to negate a query but this is a unary operator not a binary operator so it always has just one query argument so we're taking the query object representing cat dot cat owner equals cat owner dot id and then we're negating that so we're filtering for the rows in which cat dot cat owner is not equal to cat owner dot id so expressed in terms of SQL that would look like cat inner join cat owner there's an inner join because there's two tables here right cat and cat owner and then the filtering predicate is cat dot cat owner not equals to cat owner dot id and that less than greater than sign that's just the not equals operator in SQL syntax so just be clear what's going on here we're creating a query object with the equality operator from these two field objects and then we're using the tilde operator on the query object because the query class overloads the tilde operator to act as a negation operator and that negation operator the tilde operator returns a new query object representing the negation of the query that we created with the equality operator in the second example here first we're creating two separate query objects with the equality operators first cat dot cat owner equals cat owner dot id and second cat dot name equals mittens so we have these two separate query objects and then we combine them into one query using the ampersand operator which produces a new query a third query object that represents the inner join of cat and cat owner with the filtering predicate of cat dot cat owner equals cat owner dot id and cat dot name equals mittens so it's a composite predicate here with two conditions two equality tests and both must be true for the whole predicate to be true so it's selecting for only those rows where cat dot cat owner equals cat owner dot id and cat dot name equals mittens any row in the joint table where that's not the case gets excluded from the result set of rows understand that in this example here the parentheses actually aren't necessary because the equality operator has a higher precedence than the ampersand operator i just included the parentheses for clarity now our third example here is exactly the same as our second just we're using the pipe operator instead of the ampersand and so in the effective query instead of the and operator in the sequel it's using the or operator so it's those rows in which cat dot cat owner equals cat owner dot id or cat dot name equals mittens so as long as either one of those is true then the row is selected only when both are false does the row get excluded from the result set and finally in our last example we're taking that same query but we're just negating the whole thing and note that does require placing the whole thing in parentheses and then putting the tilt in front otherwise we're just being negating that first query the cat dot cat owner equals cat owner dot id we want to negate the whole thing so note the effect of this in the sequel is to take that whole predicate expression and then just apply the not operator to the whole thing so the end effect of this is to filter for precisely the opposite set of rows from those selected by the third query so the logical operators of the query class ampersand pipe and the tilde the negation operator be clear they take in query operands and return a new query object that's why in the first example here actually those parentheses are required because otherwise the tilde would be operating upon not a query object but the first field object there db dot cat dot cat owner which is not valid because the field operator doesn't have a tilde operation what's happening there is the equality operator of field is producing a query object and then the negation is operating upon that query to produce a new query now when combining queries using ampersand pipe it then becomes possible to get a query involving more than just two tables and in that case what happens simply is that all the tables are joined together and then all the predicates are applied so here for example first it's again cat dot cat owner equals cat owner dot id but then the second query we're adding to that is dog dot name equals phyto so what happens is first the tables cat cat owner and dog are all joined together and then we filter with the predicate cat dot cat owner equals cat own id and dog name equals phyto so the simple rule is that however many tables are involved in the query they just get joined together first then all the predicates are applied now in the usual case of a request in our model we open a database connection and if in the course of a request we need to do any sort of database business we use that connection to perform selects updates deletes inserts and so forth and then at the end of a request when we return a successful response web2py will then automatically commit the transaction if for whatever reason we need to explicitly commit our transaction we can do so by invoking db.commit assuming of course db is our connection object which is by convention and web2py what we normally call our connection object we normally sign it to a variable db and then likewise if we want to explicitly rollback our transaction to undo any changes we've made we can invoke db.rollback and as i mentioned earlier there are circumstances where a rollback will happen implicitly if an exception other than the http exception is thrown and not caught in the course of processing a request then an automatic rollback is performed on any open database connection so it's not common that we invoke these methods explicitly but occasionally there are cases when we would call them directly


Hello, everyone. Hi, my boss is still taking a picture. I want to make I want to make a selfie with all of you. Come on, let's say. Raise your hands applause. Okay. So I'm Mark. I'm the maintainer of the Linux Khan framework and the drivers. And this is my colleague, Oleg say, and we will talk about socket can in the beginning and then Oleg say will focus on J 1939. Not that you have a little bit of information about me. I want to know something about you. Please raise your hands. Who of you has used can before? Oh, nice. Let's keep your hands up. Who has used that on the Linux mainline kernel? Oh, okay. It's not getting less. Who has heard of J 9 to 39 before? Oh, quite so many people who has used it on Linux. Some and who of use of you have used the mainline Linux kernel stack. Me and my colleague and one of my colleagues is cheating. Okay, I want to talk about first about can that you all that we all on the same level. What's what's can about can is networking hardware and then stack. It's you need just two wires for it. It's something different than either net. It's much more simpler. This is why we have different different constraints on either net. The speeds are just up to one megabit per second. Yes, per second, just one megabit. And you only have eight bytes of payload per frame. So and a frame has these eight bytes of payload and an addressing field that might be 11 or 29 bits. And based on that can ID. This is a prioritization how to access the bus. It's like Black Friday shopping. Only the strongest can ID wins the can ID was the lowest number gets a bus and all others if they should send at the same time they will automatically get away from the bus so that would be no collision. And this means all the can frames they are broadcasted so that everyone can receive them. How was the can infrastructure before we introduced it to the Linux kernel? You have the kernel. And because everyone thought they are can is so simple. We don't need it in the current we just have some library or whatever. And this is usually per vendor. Some vendor or 530 site are we bundle some library with my with my either with my can networking card and we ship it to the user. So you have the protocol and everything in your application. And some other vendor sells a different networking card and a different library. And for example, tell your IT guys when you switch your Intel Nick for AMD Nick you need to recompile your web server. Everyone says what are you crazy. And because of this we decided you're let's let's get the can stuff into the kernel that you have a unified interface for all applications. Just the usual socket open close read write that you all all know. So before it was in the kernel you have no compatibility between all those different can implementations. No tooling because everyone invented their own library and there was. Resulting of this test coverage. And then we move the protocols take into the kernel. We can make use of the in kernel infrastructure for drivers for networking. And you open a socket and this is your one and only interface between the car and your application.  They wrote their own drivers and brought the main line and for others we did and other people did. And so now you can use the same application for testing on your laptop with some USB adapter or different or the same application on your sock with the internal can adapt. So this is how can looks now. You have different applications. The socket layer. The different protocols then the networking core pack it scheduling routing and the can interfaces. This is how the flow of packages in this. It's might look complicated in the end but as a application developer you just interface with the socket layer and with the driver developer. We just at the bottom. So it is all quite nice and shiny. And with all this we have a lot of infrastructure now. Cannot yields for basic interacting with the canvas and good examples to start with. We have can tests. We have wire shark plugins and so on and so on. Yeah but coming back to the limitation of the canvas packets are only the can frames are only 0 to 8 bytes and yeah this is just single tiny spoon of data and if you want to have make a firmware update in your device or whatever where the firmware is several megabytes big several tens of megabytes it's quite complicated. And for the can the can ID is not in the lower levels not standardized. What to do with it is just an ID and what you do with it with the sender or receiver or that defines what the content of the packet is it's up to you. So this is where J1939 comes into play.  And this is where my colleague will take over because he has cleaned up and implemented a lot of ideas and brought it into mainline. So we already get some idea about challenges of can it is slow. It provides we can transfer really tiny bytes of data and usually we have lots of choice what we can do but well it is it provides us huge power to do really different designs but how we work with this power depends on us and sometimes we provide interfaces which are not compatible with each other. So here is the place where J1939 comes into play is it's providing or well it's providing a number of recommendations what you can do with all of this and well everyone is free to apply these recommendations but if you do your device will just work with another device. For example we get recommendations for physical layer and the main part of this specification is a number of different defined predefined PGNs. PGNs are parameter group numbers and with this number you can already know how you can process all of data which are in the payload independent and the way how you get it. This PGN is located in the can ID, the can ID is 29 bit, the data is 64 bit and you get in can ID source address and you can even get a destination address inside of PGN. It depends on the PGN level so until some PGN number you are able to use a destination address and after this it's some kind of generic groups like this ID for example for engine, status, rotation temperature and so on it's everything encoded and predefined inside of PGN and well we have three bits of preo and you can use it as you wish. As well we get here transport protocol and based on this we get even extended transport protocol defined by isobus. With transport protocol we can more or less reliably send and receive data. It is not common and common because usually you just fire and forget and in this case we have a unshaking request to send, request to send, clear to send and end of message and then we have a acknowledgement which is done after complete transfer was made so between different MTUs you are able to see if this data was actually transferred completely or not and if not then you are still able to somehow recover it. With transport protocol you can send message of 1792 bytes and with extended transport protocol you can send a message of 112 megabyte and this was a main challenging part of implementation in the kernel because we need to provide abstraction there for MTU maximum transfer unit of 112 megabyte. Just a comparison of why we actually do not use some existing protocols theoretically we will use IP with TCP or UDP headers but well let's see IP version 4 we have 20 bytes of header it is even not fit to one can package it is a lot and with TCP we have extra 20 bytes of header with UDP we are close enough we can put complete UDP header in one can package but what can we do else with this? So we need it somehow and here's where comes to use of this protocol and how can this be used in the Linux. Currently this protocol is spread somehow and there are different implementation in user space and different operation systems so it closely the same situation like it was with socket scan back in 2013 or so yeah so yeah 2008. What kind of challenges we have with this protocol? Well usually on the Caspan we work with like probably about 2,000 packages per second it's not really huge amount of data for normal high power system. These packages are tiny really small so with SKBS we have a huge overhead for each package. The specification provides relatively relaxed timing requirements so usually we will not go to time out if you are running from user space with this implementation but for example on some low power system with I don't know old version of ARM with 400 smear hairs for example iMix 28 it will be problematic. There are user space implementations and you can spot a different variants in the wildness. The kind of J90, 39 demon running in user space and routing all the data to different applications. There is one library used by different applications or all in one application which is just doing everything or even you're probably familiar with this. Some product is made by different developers or different companies at different time or status of development so you get just different stacks and same product. In first case well it's possible to do this but you'll get really long round trip times so your can message will go to process by kernel then it will be copied to user space and processed by J90, 39 stack for demon then it will be somehow on some way I don't know pipes, unix, domain sockets, TCP or whatever transfer to the application. There's a huge amount of time and not needless overhead in this case. If you use a library it will be easier for update or page something so your applications will be updated at the same time but you'll get copy all the same data for each application so for example if you run 10 application in user space you'll get all this 2000 packets per second for each application. It is needless overhead as well. All in one well it's possible to do this but you'll get other kind of challenges, security, availability and so on. Is it really good to manage a huge project all in one? I don't think so and I think this part is we don't really need to explain this if you get different stacks different kind of implementation on same system. It's just nightmare to manage this kind of project. So we have kernel stack for J9039 and at least theoretically it should cover three or four variants of this protocol, original SIA J9039. Isobus for this we was needed extended transport protocol. Probably it should work on NMA 2000. It is for marine applications and mil-can is for military applications.  We were testing mostly only first variant but I don't think it will be a big challenge to extend or fix existing protocol to run with all other variants. So we have now the same well-known socket implementation from user space point of view. If you already worked with different kind of socket you will be able to apply your knowledge with this protocol as well. Kernel will give you all power you need to work with this stack and take some tasks for example for others claiming and then for transport protocols. So if you are starting to use transport protocol with J9039 kernel will gather all needed packages and assemble one huge data package and transfer it in one time to user space. For others claiming you should do your own others claiming in user space but kernel will cache these claims and so you don't need to resent new others claims for each application and this will save you some time on the bus as well. If you want to try to play with a kernel stack you can already use some examples in can-you-chills for example JACD is for the demon for J9039 others claiming. It is really straight forward simple demon but it's ready for use and JCAD was used only for transport and extended transport protocol testing. It is similar kind like net cat so you just get JCAD on both sides and both units and cut data from one unit send it over can with transport protocol and gather it on other side so you can just put it like a file. For kernel in kernel we have a documentation network J9039 files so you can read as much as possible from this. There are some examples, some motivation explanation if you are missing some information in this file please contact us or just send pages. Here is one example how you can implement a simple transfer in user space. You will need to provide a structure, SOC, other with all needed information. You can use a name if you are already familiar with others claiming on J9039 then you know what name is if you are not. The name is with can ID we have only one byte for others and in some cases it is not enough to identify what device it is. And name used as 64 bit ID so you can really pretty exact identify what device it is, what manufacturer and so on. So at first stage we claim this address, this 8 bit address by sending a 64 bit name to the bus and other applications then able to reuse this name to communicate with it so you can use address directly or name. It is up to you. And you can use Pagain as well. In this case in kernel implementation we use source Pagain differently as destination Pagain, destination Pagain is used to compile a message which will be sent to the bus and the source Pagain is used as an input filter so you can set it or not if you will set this Pagain on your source address then you will get messages only for this pgn. As soon as you prepare your addresses for source and destination then it is just enough to open socket bind to it and wish you can connect. If you do not use connect then you won't be able to use send or receive. In these cases you will need to use send message or receive message. In this example I provided here connect so we are able to use send. Here is an example just proof of concept that it is working. On the right side is a riot board with iMix 6 solo and here I am starting jcat receiver with address 90 0x90 and parameter minus r means that it is in receiver mode and all that I will be redirected to TMP RX file. So I started here each top to show CPU load and the left side similar system it is riot board as well with same CPU and here I am starting similar jcat application but in send minus e means this input file which we use to transfer data can zero it is the interface which should be used to send data so you can use multiple interfaces on your system 0x80 is a source address and 0x90 is a destination address. Here you can see status update it was sent by the kernel over error queue so you are able to use this you should not use it. In some cases it is really useful because with this huge M2 you just give the kernel huge amount of data and it starts on processing and well if you need to know somehow in user space what is going on you can request it and prove that all the data was correctly transferred I provided here checksum on both sides so it looks okay now. If you are already familiar with J9339 maybe you will spot here bug which is this code this Friday. Can anyone see it? Okay. It should be different. This size this is a end of message acknowledgment message and it should say I received so much bytes or so much packages and well in this case it was wrongly implemented and thanks to open source community this bug was spotted and reported. It is just one example how it usually works. Main challenge with huge M2 was solved. One challenge is at least now is how to explore the others claiming cache should use some kind of similar procedure like with ARP cache currently it is not really decided. I prefer to not mainline not really not really not really implementations so you can have enough time to decide what is good and then provide proper UIP for this. And well we need some kind of test automation. It is I was playing with small com so maybe you will go this way. I don't know if you already have some experience with this tech or with testing for J9339 you are welcome and I really would like to hear some suggestions and ideas. And well it is already accepted for mainline. With kernel 5.4 you will be able to use it. Thank you. Do you have any questions? So when you have these 112 megabyte huge packets do you preallocate buffers on the kernel level when you receive them? So do you have this massive chunk of memory on your little embedded SOC which is just wasted and ready for receiving these packets? Yeah. The implementation end protocol was designed so that not all systems are able to receive this amount of data. So we kindly ask first that we are trying to send a huge amount of data. Can you accept this? And other side can say I can accept only this amount of data or I can say I don't want this. And for the sending part we don't allocate this 112 megabytes inside the kernel again and don't copy it on block to the kernel but we just use a socket memory and allocate from there like how much is it 128k or so. Then we copy data to it, push it into the stack and then the next allocation either sleeps because you have a blocking socket and then it starts again if the driver has sent some data and frees some socket buffer or if you are in a non-blocking mode the sender process will return to the user space and say I just have sent like one megabyte and it's up to you to send the rest. And you have to provide the same or the remaining bytes of data into the kernel with another send or send to or whatever system call and then the kernel picks up from there and completes or tries to complete until it blocks again or goes back into user space or a signal comes. And for the receiving side it's yeah as Olex explained if so much memory is available it will be allocated. We have no we see if in chunks and provide to user space in chunks we currently receive the complete 112 megabytes at a time and then provide it to the user space. For the sending side it was not for the receiving side it wasn't that easy to do it in chunks so we skip that any more questions? Yeah. Have you got any drive how many drivers are currently in the kernel and are people starting to look at actually implementing device drivers for these against your stack now? This J1939 is based on the normal socket can as May 9 since 2008 right yes thank you. Since May 9 since 2008 and there are many drivers inside the kernel now and the normal socket can in raw mode without this J1939 is available and it's used everywhere I think and this stack is used on top of these existing drivers. One side note this stack is designed in a way that the send packages and the receive packages are needed to be in the right order. If the send and receive packets get mixed up the stack gets confused so maybe your driver that's already in the kernel needs some tweaking. Another question? Yes hello yeah first I would like to thank both Mark and Olexi to put some more things into the can network layer because we really have two things one is can network layer which is in Linux net can and the other thing is really which is under Linux drivers net can so the drivers are really for SOCs and USB whatever you have so at least this increases the pressure for me to also bring my ISOTP mainline which is just hanging around on my GitHub but one question from my side what about J1939 and can fd is there also something something which is going on to bring this also to can fd because it has not only 8 bytes it has 64 bytes in the data of the can frame. I think the problem with the SAE that is the organization who does the standardization all their standards are closed or you have to pay money or something like this but I have seen now that the next next can protocol like the can excel is being implemented now the people from SAE are looking for can fd and J1939 I think it's in the making and I think there is a draft but you probably can only download it if you're a member of the SAE you cannot even buy it as far as I know. Marek, so how do you handle moving this big buffers between user space and the kernel? I know that the graphic stacks uses DMA buffs to efficiently shuffle big buffers between user space and the kernel does the network stack have something similar or do you just like copy to user copy from user? For the sending path we use just the normal stuff from the networking stack I think you can use zero copy this is all abstracted by the networking stack it makes a handling from user space a little bit more different more difficult because you say send and then send returns but you are not allowed to discard the buffer because it's still in use and if it's the kernel is done with it then you get some message from the kernel that the buffer is done and you can discard it the question was is there a library that does this it's inside the kernel it's called zero copy look into documentation net and then look for zero copy use regex and then you will find something there are examples in the kernel send file was the question if you think about it do we have to implement a callback for this I think we have to implement a callback for this we should put it on our to-do list yes the first stage it was not really important we had to provide at the first stage some basic UAPI which has no questions to say the well-established read write and receive stuff and all the optimization is on top another question I think no one then thank you again thank you


So, variation in fuel costs, right, those variations were generally allowed to be passed on directly to the consumers. Okay, so if the price of all the fuel the utility had to buy doubled, then the generation portion of your electricity bill would double, right. So the utilities were generally allowed to do that. Most regulators, most escape regulators broke up the utilities costs into investments, right. So capital investments in wires and substations and power plants and things like that, and operational costs, which were things like labor costs and fuel. And the basically the deal that the utilities had with the public utility commission was that they could pass through all of their operational costs to consumers. Okay, so fuel prices increased, electricity prices would go up, okay, but that they weren't allowed to earn any profit on fuel, that they weren't allowed to earn profit on stuff that they could build. This was called the rate base. So every time the utility built a new power plant or built a new transmission line, then it would get to earn a profit off of that investment, okay. And so anyway, so this regulation created a very stable climate for a long time, but you sort of see where we might start to have some problems, right. One, if the utilities profits went up every time it built something, right, then the utilities sort of got in this mindset where their business model was building stuff, right. Second, you know, when you're spending other people's money, then you're maybe not necessarily as careful with it as you might be in other lives, okay. And so, but you know, for these sort of what economists on incentive problems existed for a long time, but they really didn't manifest themselves until the 1970s when we basically had two things happen at the same time. One, we had the sort of energy prices of the 1970s and fuel prices increased dramatically. At the time, 20% of U.S. electricity was generated with oil, right, and so when oil prices went up dramatically, that impacted the cost of generating electricity. The second thing that happened was for environmental and other reasons, the utilities, well, utilities in many states kind of halfway were forced and halfway decided to make large investments in nuclear power plants. And as it turned out, the utilities did not know how to build or operate nuclear power plants terribly well, right, and so there were all sorts of delays and cost overruns and cost escalations with nuclear power plants, and you know, because of this deal that the utilities have with their regulators, all of, I mean, all or most of those costs ultimately had to be paid by utilities' customers, okay.