My name is, my name is Tom Zanussi and I'm a software engineer at Intel working on the Yachto project, mainly in the area of the kernel and tracing and profiling and things like that. And today I'm going to be talking about working with the kernels in the Yachto project. Really? With the mic I need to talk louder? Okay, is that better? Okay, so I'm going to start off with an overview of what we have available in Yachto as far as kernels go. And then I'll go into a little detail about each of the different recipes we have available. And then following that, sort of a little bit of guidance about when to use which one of those. And then I'll go into a couple practical topics such as working in recipe space versus working in tree in the kernel repo and using local clones. I just wanted to point out that everything that I talk about here is covered in a lot more detail in the kernel manual and if you have any questions at all about working with the kernel you should be able to find it in the kernel manual and if not we'll add it if there's anything in there that's not clear. I also like to point out that this, everything I talk about that's related to the current version of Yachto which is 1.3 or Danny is covered by a set of hands on labs that I've just recently updated and they're available at this link and past Dev Days what we did was we had, we brought in a bunch of machines and we had people sit at those machines and work through the kernel lab and to get sort of hands on experience with working with all of this kind of thing. And what we did this time was we decided instead to just have to cut it to half day and make the kernel labs installable on your own machine and much better instructions to allow you to do that on your own and as a side effect of that is that we actually, I was able to expand those labs quite a bit so now there are five labs instead of three and I think they should cover most of what everyone, most of what people want to do in a really hands on practical way so if you want to look at those labs and even start working through them now and ask questions later on in the day or in the week that would be great too. So basically the first thing I want to say is there's really no such thing as a Yachto kernel and we don't have a team of a hundred developers working on the Yachto kernel like Red Hat does and essentially we have a small team that sort of manages a set of Yachto kernel repos and one of our recipes called Linux Yachto points to that, those Yachto kernel repos. These repos are essentially based on upstream kernel, the kernel.org kernel and then we merge in Linux the stable kernel and even LTSI into those kernels as they become available. As such what's in there is the patches that we put in there are essentially sort of there to bridge the gap or things that are expected to eventually end up going upstream. And so those are the and there are a bunch of branches in each repo that are sort of machine specific branches and they're all based on the base branch and what I'm talking about is the base branches we don't put in typically put in things that are that are experimental or what not. But the other branches that we have in the repo the BSP square branches are there specifically for that purpose though. So those whatever is in those BSP specific branches it's not expected to go upstream and anybody who's done a BSP knows that in order to get certain things working on a machine you need to basically add hacks all over the place and obviously those are not meant to go upstream. I'll get into a little bit later what these repos actually look like. So we have the Yachto kernel repos that we maintain and then we have the Linux Yachto recipe that adds machinery and a set of metadata on top of that that's meant to be useful for building up your own configurations and so on, your own kernels. So in a sense I guess it would make more sense to say that what we have in Yachto is a meta kernel not necessarily a kernel and that kind of map that kind of goes along with the idea of what Yachto is with Yachto itself in that you can look at Yachto is not a distribution it's more of a meta distribution. So in that sense we have a meta kernel instead of a kernel. So those Yachto kernel repos there are three of them at any one time one of them is right now in Danny 1.3 version of Yachto we have kernels that are based on 3.2, 3.4 and current Linux master which is what Linux Yachto does that recipe points to. In 1.4 which is coming up pretty soon we'll have a 3.4 kernel we'll get rid of 3.2 and we'll move to 3.4, 3.8 and again Linux Yachto dev which is continue to track Linux master. LTSI kernels actually we work to schedule our kernels on the same sort of cadence as LTSI now and so when LTSI becomes available we actually merge LTSI kernels into the Linux Yachto kernel and we've done that the last LTSI release was 3.4 and we've merged the LTSI 3.4 release into the Linux Yachto 3.4 repo so if you use any of the branches in there you automatically get LTSI by doing so. So those are the repos that are managed by Linux the Yachto kernel team but you don't have to use those you can use essentially you can use any kernel that you want to and we provide a set of recipes for doing that. Linux Yachto custom recipe with Linux Yachto custom recipe you can point that at any get repo that you want to along with the def config of your own and get that kernel into your image. We also have traditional tarball based recipes that handle only tarballs and I'll talk about that in a minute and starting with Yachto 1.4 you can actually point Linux Yachto custom at tarballs as well and so you really don't need to know anything about traditional recipes if you don't want to after 1.3. So I'll start with the traditional recipe the simplest recipe this kind of recipe is familiar to you if you've used OECore before it follows a typical pattern of an OECore recipe the source URI points to the static kernel tarball and you change that you add changes to that tarball via source URI. The actual build process of building the kernel is helped along by the kernel that BB class which you inherit as part of the recipe and the configuration that gets used in this with the traditional kernel is exactly what you give it as a def config. So here's an actual example of a traditional kernel recipe you can see that it's pointing a kernel.org mirror and it's pointing at a static tarball. The def config is the config that is going to be used to build this kernel and below that you can see the source URI we're adding a patch to this kernel using the source URI which is just exactly what you do with most OECore recipes. And this is a sort of a graph of picture of that it's kind of unnecessary for this particular recipe for this particular type of recipe but I may be building on this picture for the other recipes to make it more clear what's going on. So here's on the right you have your kernel source code in that case it's coming from a static tarball and you combine that with a def config which is coming from your def config monolithic def config that you put into the recipe source URI. You build that and you get your kernel image and again that's really there's really no point to having a diagram to explain that but yeah it's about as simple as you can get. This kind of recipe a traditional kernel recipe to add configuration items on to that you essentially what you do is you replace the def config with your full with another config and in Yachto essentially the easiest way to do that is to invoke the menu config task and it brings up the kernel menu config that you're probably familiar with you turn your settings on that you want turn the settings on that you want save it and it gets saved into the build directory as a .config take that .config just copy it over to your def config and that's your new configuration for that kernel and if you want to see what changed you obviously the easiest way to do that is to keep a before version of the .config and do a def of the new config and that's how you get a listing of the options that were changed. So this traditional kernel method recipe is essentially it's a fallback if you're using 1.3 or earlier and you have to use a static tar ball this is the way you do it as I said before Linux Yachto custom can actually deal can actually point to a tar ball so you can if you're 1.4 using the next version of Yachto or later that would probably be a better way to do it because not only do you get to just to use your kernel tar ball but you can add the other features that the kernel tooling gives you like config fragments and if you if you want to work through an actual example of using a step by step example of working with the traditional kernel recipe in lab one kernel lab what lab one takes you steps you through doing that. Okay so that's the traditional kernel recipe the next the next progression in complexity I guess or capabilities of the the recipes that we have available is Linux Yachto custom it's actually pretty similar to the traditional recipe except it allows you instead of pointing to a tar ball to point to any any arbitrary git kernel that you may have laying around or that you use and def config just like the traditional recipe that also takes def config. The one thing it does give you on top of that is the ability to to do things like config fragments and here's here's a here's an example of a config fragment this is these here are two config options and these are exactly the same options that you would stick directly into your dot config except in this case they're held in a separate file smp.cfg so that's our config fragment and to add this into to add this on top of the def config or the configuration that we we already have you just simply add this to your source URI similar to how you would do the same thing with the patch except now now it's just a config item instead so that's that's basically it for Linux Yachto custom it to use a Linux yet to if you want to use try Linux Yachto custom yourself go into this directory this is an OECore meta-skeleton recipes kernel Linux and copy out the file linuxyachtocustom.bb that's the Linux Yachto custom recipe and then just modify there are actually some pretty good and not extensive instruction but pretty good instructions about how to use it inside that bb file itself I mean you can also look at lab 3 in the hands-on kernel lab if you want to go through a detailed step-by-step example of using a custom recipe so this is what this is an example of what a Linux Yachto custom recipe actually looks like here you can see the source URI points to right here we have it pointing to the stable kernel and the stable kernel has a whole bunch of branches and if we don't specify a particular branch it's going to take it's going to use the master branch in this case we don't want to use master branch we want to use this specific Linux 3.4.y branch in that repo and so that's basically what's specified here and then you can also see the def config just as similar to the traditional recipe we have the def config which is your configuration but in this case we can also add configure config fragments on top of it which we couldn't do with the traditional recipe and here is another picture of it a picture of this sort of building on the previous one again we have the source on the right and in this case the source is coming from the stable Linux stableget and the 3.4 branch in particular that's what makes up the source we combine it with the def config and create our image in this case if you look to the left you see that some of this configuration is most of it's coming from the def config but some of it like this is actually coming from a config fragment and that's mainly the main difference between the traditional recipe is that not only you're pointing to a get repo and you also have the ability to config fragments now okay so to recap we have the traditional recipe that points at a static tar ball we have the injective stable that points at yeah now this yeah this is a recipe this is essentially an OECore recipe for the kernel and these essentially these are assignment variables in the in the in the build system and the build system goes through and processes the recipes and and at build time substitutes the values that it finds here in into the build so let's see in this case everything we have here is just a simple assignment well actually we have a we have a a an include statement this is actually saying to like you do a pound include and see it's including another file and this is that this little bit contains the machinery that that actually is behind building the kernel yeah this is these are a bit variables a bit big interprets and uses in the build so it's not it's not a make file at all it's just a standard bit big yeah a recipe okay so that was yeah a little ahead but so yeah that was the linux-yachto custom and yeah so the linux-yachto custom that you point to a get repo give you the config frame and so on and the next sort of progression in recipe space for kernels is the linux-yachto recipe and as I mentioned before what this is actually what the linux-yachto recipes actually point to is the yachto kernel repos that the yachto kernel team maintains that I described a little bit before so those the the linux-yachto repos contain us in addition to the machine branches they contain a set of ready to use objects config fragments one I've already I've already described then you have kernel features which are kind of config fragments on steroids there are config fragments plus source chains patches and described in a .scc file .scc stands for series and configuration compiler but that's sort of a historical name and no reason to know that but I guess anytime you see sec just substitute feature this is essentially a kernel feature and then we have kernel types which are sort of reusable collections of features and fragments that you can sort of subroutines of objects that you can use reuse in your kernels and and you use them to essentially provide a policy or capabilities that a number of different types of kernels would use so they're they're they're actually yeah just sort of I'll talk about those a little bit later but so the linux-yachto recipe points to points to get cloned again just like the linux-yachto custom recipe did but in this case it points to two branches instead of the one like the in late linux-yachto custom did one is the machine branch and essentially that's just the the source for the that's used to build the kernel and in that map that's essentially a machine branch and I mean a machine specific branch source code that has changes for that particular machine and another branch which is one a single branch in the repo that's shared amongst all the uh between between all the the other branches and that's called the meta branch and it's an orphan branch that contains shared objects and by shared objects I just mean a pool of config fragments and kernel features that is sort of centralized in this meta branch for anyone to use it also the meta branch also contains bsp definitions these are sort of the starting points for a particular machine to be built and these are actually these are kernel features as well but they're top-level kernel features and the build system essentially looks for the the top-level kernel feature for a particular machine when it's building the kernel and finds it and that's it starts the process of everything that's included within that feature config fragments and so on and and the end result of all that is the for instance a def a dot config that's used to build the kernel so in this case to add a config item for a linux-yachto recipe because because this is all sort of this is in the kernel repo the most natural thing to do would be instead of putting it in as a source uri which you can still do instead of adding the doc cfg to the source uri the most natural way of working with a repo is to actually add the the thing directly to the branch so in the in the case of adding a config item instead of putting in the source uri you would just commit it to the meta branch and and it would be available for anyone else for you for your bsp to use i mean for your kernel to use and anyone else to use similar case with patches i mean the linux-yachto if you're working with the repo directly the most logical thing to do is to add the just like you would if you were developing locally for your developing a kernel on your own machine is you would modify the code and check it into the the kernel repo directly as a as a commit to the to the branch you're working on same thing here the most logical thing to do would be to add it directly to the machine branch but you don't have to again if you're more comfortable using a source uri as as you do in other open embedded recipes that's fine that that still works and you can also actually add it add a add a patch to the machine branch from the the meta branch using a patch command in the sec file but i won't talk anymore about that right now if you want to use if you actually want to walk through of working with the the linux-yachto kernel you can look at lab 2 and and and that'll show you how to do that as well as other things like uh work with modules and so on so this is here's a here's a here's another another example of another recipe this one i think i already went over this one thinking i was going over the previous one but linux-yachto custom but this is um linux-yachto recipe this is actually the recipe that's in oikor meta branch um and this is sort of the base recipe that gives you default branches to work it's naming a branch that's going to be built by default and the source uri as you can see doesn't point to it points to a get repo but this repo is actually one of the yachto maintained kernels um and it's uh again i mentioned that it uses two branches one is the machine branch and then the meta branch is the other branch it uses and by default that's called meta um so that's that's the that's what you get um without any modification typically when you are creating your own kernel you want to add modifications um and you would normally you do that via bb append and this is a bb append for linux-yachto 3.4 an example of a bsp that um that exec is this in metaintel and you can see what it's doing is it's uh it's overriding the the default branch and using uh this branch which is dedicated to that particular machine sugar bay and the other thing it's doing is it's um it's using one of the kernel features that i mentioned from recipe space you this is one way to do it you could also directly include this feature um in the repo itself but this is just to show that you can do this from recipe space if you want to um so it's a pending feature called i9-15 which is graphically a graphic feature graphics feature required for this particular machine and then here's here's uh another picture of here's a picture of the linux-yachto 3.4 way of doing things um again the familiar on the right the here's the branch that's actually built in this case it's a machine branch common pc inside the linux-yachto repository and we have a config that's built with it to produce the image but you can see here that as with linux-yachto custom it's pulling config fragments from from outside of the configuration the big the difference here is that those config fragments are not any longer just uh that's config fragments on the source uri they're actually um in this library of objects in the in the meta branch that anyone can use so in this case um you can see common pc this is actually defining this is actually the the top level feature that starts to build off and it's adding some basic uh feature uh options for for that architecture um and below you can see um it's also making use of the i9-15 feature and um it's adding these config options into the config end up in the the final config but this it's also touching the code itself through a patch so it this sort of uh demonstrates the uh utility of features like graphically I guess because it's it's touching not only configuration but source and that's meant that's sort of meant to depict that um and this is like this is the same thing a little more realistically I just mainly mainly wanted to show that um the the repo the linux-yachto 3.4 repository has a bunch of branches and you know when we do a build we're just choosing one of those so in this case our common pc branch um and one one of the BSP features is being is the thing that actually is building that particular branch if we were to switch over switch the build system over to using to the QEU-PPC32 machine this branch would be built and it would use it would start using this top-level kernel feature um so that's those are the basic set of the the uh the base set of recipes um in 1.4 there's another a neat feature which allows you to um if you saw here we only have one meta branch that's the only that contains all the features um that are available to this uh to the to build these branches um there's nothing else besides those or you can you can use this obviously a source URI append but in 1.4 you can actually add more uh sources of kernel features so this allows people to reuse and share kernel features from anywhere essentially um and just quickly uh how that's enabled is how you do that is add you just add add add the new meta branch to the source URI here we're adding two new sources of kernel features and we're naming them um feet one and feet two and then down below in in our recipe we're actually um using the kernel features append to pick and choose features from each of those branches in addition to the one that's available with the repo itself so we haven't used them this much but um yeah but it should be pretty useful in the future um and here's what that looks like graphically again here's our single meta branch and uh the the source being built and the the configuration that comes from that meta branch but then we also have these meta branches that are being added on top they contribute config fragments as well to the final image the configuration on the image um so uh so just to recap the the three different kinds of three different types of recipes traditional recipe um essentially meant to use uh with tar ball uh static tar balls and then we have linux yachto custom which is very similar except you can point that at any arbitrary git repo on both of those use def configs and um then we have the linux yachto which points at the yachto maintain kernels um and it gives you a whole bunch of things beyond that including the ability to do config fragments and and features um um and um and it also has a pool of config fragments features that you can okay yes yeah that's well the way yeah that's a good point um you know you can turn on one config item and then when you look at the dot config you can have a whole number of other things that get pulled in and that's the way we do it I mean there's no magic to it I have described earlier what we typically do is um use the menu config do a diff between the two so you do the use you you actually um when you're uh setting up these config items uh config fragments um essentially you're you go into the working directory and you build uh you actually uh do make menu config from there it brings up the the menu uh the the config menu and then you obviously you've saved off the dot config from before and you have your new dot config and um do a diff of those and that's what you put in the fragment oh you so you mean you're you're talking about taking a config fragment from like the 3.2 kernel and moving it to 3.4 yeah I don't have a good answer for that I mean you know we have run into cases like this where um you know we have a a fragment that works fine in one place but then we use it in a in a slightly different situation and you see that well it it requires this um instead so those solutions so far has just been create a another config fragment for that under under those conditions but yeah I yeah um well I mean yeah I'm I mean there's a there's a uh a config check um a config check test that gets run that that is basically essentially match it that checks what you specified versus what was actually produced in the dot config and if they don't match it'll spit out warnings and and if it's severe enough um like the the uh the config option no longer exists for instance and it'll actually stop the build saying okay well this this config option no longer exists um I assume you want to know that and and stop the build but yeah I we can talk about it more I mean Bruce is I don't know if Bruce is here Bruce Ashfield um but he'll be here later on I guess and um he's kind of the guy who uh who who actually wrote a lot of the underlying um config merging and checking and all that stuff so he might have a better answer than I do but well there'll be plenty of time afterwards if you want to talk about that okay so um I guess when to use which is uh okay um yeah so the tar if you have a tar ball essentially you're going to you're going to want to use a traditional recipe unless you're using 1.4 but you can point to the linux yato custom get based kernel that you just want to get up and running quickly the def config point it point linux yato custom at that and and you should immediately get your uh kernel up and running and that also gives you config fragments etc um and if you can possibly do so consider moving your kernel to linux yato um you get certain benefits from doing that um one of them is that and I'll show you in a minute concretely what that looks like is the your your uh code will automatically get updated with fixes security fixes and um the latest stable kernel uh continuously merged into that as well and again you you also get access to this uh instead of config and feature pool and meta um and one you you don't have to uh you don't have to commit to uh doing it right what you you can actually do most of what you need pretty much everything you need to do in recipe space first if you're more comfortable doing that and then move it over into the repo and I'll show you an example of that in a minute um I but I just wanted to uh to to show you exactly what features were available um currently in the the yachto kernel this is the uh the meta branch uh contains like config features bsp directories and so on um this is just the features and I'm not going to go into uh details about what all these things are I mean should be self-explanatory what a lot of these things do um just wanted to give you an idea of of sort of what's available in in the in the meta branch of the features specifically um I'm just diving into one of those features as an example um i9 15 feature it's uh there it's a set of two files essentially dot cfg and scc file the dot cfg is just a config fragment like I described a set of uh config options in a file the scc file is actually the feature file and that's what you see down here and it contains a bit of metadata describing the feature um and at the end it this kconf essentially is an include saying to include the config um and that's to give you some idea about what a feature actually looks like doesn't this one doesn't have any source but most of them do um so yeah that's another point is that the config fragments features are actually a super set of config fragments so they don't a degenerate case of a feature would be a config fragment um so the other thing I wanted to point to show was just how the the the branches in linux yachto do get continuously updated here on the left are all our branches that are in the 3.4 repository um these are machine branches here and you can see in the middle that there has been activity the most recent activity has been uh to essentially merge the stable kernel into all these branches and that's what 3.4 at 28 is is where the stable kernel was at this point in time 11 days ago and boosat ashfield is a member of the yachto team that actually does the actually does the uh maintain the repo so he does all this work and um and you can also see here there's ltsi branch um so that's available in the repo and what it does what this doesn't show is that you know because it's showing only the latest commit messages is that the ltsi branch has actually also been merged into all these branches so if you use one of these branches you get the benefit of uh saying that you're using ltsi kernel five minutes okay um so this uh yeah I'll skip this one and for the time being I kind of talked about it already um the difference between recipe of space and repo space that you can do things in the repo or the recipe space and sort of transfer those back and forth that will just giving you an idea that you can you know you can do whichever you want you don't have to commit to any one way um the final thing I like to cover I guess would be using local clones and this is sort of talking about development workflow you know if you're if you're used to building your own working with your own kernels what you typically do is you you know go grab a clone linus's kernel or the tipped kernel ingo monos kernel whatever and check that out and then you check a branch out and modify the code um and commit it and then rebuild your kernel make make vz image make modules and reboot your kernel and that's sort of the normal workflow um in the yachto environment if you want to do the same thing it's it's it's not much different really except that you're using bit bake wrappers around things um but one thing you do need to do is to um use a local what what you normally would want to do is use a local repo um to actually do your work in and the way you do that is it's there's really not much to it and and these recipes like linux yachto customer linux yachto you um here's here's your normal source uri that's it's pointing to a get repo out there somewhere the stable kernel um and if you want it to uh work locally you'd clone that kernel and then um i've cloned it to this location here on well this is actually a different one but it's a local kernel um that it's on my local file system and and i uh i also changed the protocol from get to file and um then the build system should be able to find the uh the kernel repo on my local system um and typically what we do when we're we're working with the kernel in this way is you create um a local clone create a bare local clone and this is essentially um create a bare local clone and then you create a working clone a local clone of that and then um you actually point the recipe at the bare clone and then you do the work uh in the local clone push it to the bare clone and then rebuild so um that's the basic workflow here's um here's how you actually go about creating those clones use bare um to clone whichever kernel you want locally and then you clone that one this is where you work this is where the recipe points and the workflow is some pretty much this cd into the work repo edit your edit your um kernel code commit the code into new commit push from the work clone into the bare clone and then just build and build the kernel and you're good to go we'd up run kmyu or whatever to test it out so that's a bare clone you don't certainly need a bare clone but um that's um you can actually just do a direct clone and then the same kind of thing commit edit commit and then build the kernel but what i found when i was working by the way all this is in the lab if it's one of the labs uh lab four i think that you can actually work with the local clone but when i was working through that i found that this wouldn't really work unless i did a clean all and i'm not sure if that's a bug or a feature but um but it doesn't really matter because you can always do this and you know this is what we normally have used uh normally use them use for a while and this works just fine so and it's only one more step at the setup stage instead of one more step at every build so i guess i'm out of time