I'm very happy to give this talk. I know you must be tired because it's the last one but I hope you will enjoy. I realized preparing the slides with all the colleagues that our title is not very nice and we have to change it so we have a new title which is much longer and I will try to explain what do we mean and what we would like to do in this project. This project started in January officially and we are still in the starting of the project. So I will just introduce the idea that we have the things we want to do and some progress we just have. Our motivation is about music. I mean there is a lot of information about music around the web. For instance only Spotify or only platforms you find billions of songs and they are usually commercial recordings so they have been recorded in the studio or maybe mastered later so they are good quality, good nice recording. But in addition to that you find a lot of user generated videos or recordings that one has taken with a mobile or with a camera and they are of different quality and different characteristics. In addition music is not only recordings or audio or video but we also find textual description about music. We find images for instance album covers which represent a little bit the music and the style and we also find musical score which is a symbolic representation of the music which is also used by musicians and by composers to create and interpret music. So our motivation is to deal with this large collection of music material. In particular I will put an example of one of our projects which is an European project that finished this year where we have a deal with symphonic music. As you may know symphonic pieces are very long, one hour duration and they are usually recording from different perspectives. You can see this figure here where you find for instance in this case in this recording of symphony number three of Beethoven we had like 17 audio tracks, we had also six or seven or eight video cameras from different perspectives. We have Kinect recording also, we have scores, we have also some descriptors we automatically compute from audio or from video. So we have a huge amount of information in this case it was 15 gigabytes. But depending on the quality you have you can have up to one terabyte of information for one piece performed in a particular concert. So we created a multi-model database to store all this information and to try to access and visualize it and have some meaningful description of this information. In particular we have been working in our department on methods that try to describe separately the audio part, like the auditory information that we can get from music and in another way the visual part of information. In particular myself I've been working on trying to analyze music material mostly scores or audio signals and my colleague Gloria Arau is seated there, she has been working on extracting information from images and from video recordings. So then we try to join forces in this project too because we as humans we don't use only our ears only our eyes but we just perceive music especially when looking at videos we perceive it in a complementary or in a combined auditory and visual domain. So this is what we wanted to explore in our project. So our team is composed of now four people mostly myself or the newest one is Olga Slithovskaya, she's seated there and she's the one that her PhD work on this project. And then Gloria and myself we are trying to contribute with our algorithms and expertise. This is for instance the algorithms that Gloria has co-developed also in the image processing group of ticker flow estimation. And I am trying to contribute with the descriptors I have developed or co-developed for extracting melody, spectra content and harmony from music signals. And we have also one poster which is Julia Norvano who is specialized in evaluation and evaluation of information retrieval methods to check because we found out that it's needed to find which databases should we use, how to measure the reliability of those datasets if they are represented real world problems or not. And we are also having some advice from external people from here from the department. So now I will explain you the title of a project which is very long so first we want to work with music but in particular we decided to focus on a very challenging material which are user generated music performance videos. And I will illustrate it with an example. I don't know if who here has a child or a friend or a couple that play music and in this time of the year you have all these final concerts, you know like the choir and then people go to those final concert and record their videos. Similar way to this. Sorry.  You want to focus on your child and then I will play this. So then people try to record but then they are nervous because it's their child. They are so excited. They try to remove the camera and then they talk and they say wow it's so wonderful. And then there is also the grandmother and their aunts which are filming together. And if there is an orchestra there is the whole parents filming and then they have a WhatsApp group. They share all the videos and then you have like 10 or 11 videos of the same performance. So this is the material we are dealing with. Then we want to develop things which are content based. So of course you can tag, you can say it's a cello. The piece is a Suzuki book number, I don't know which one. And you can tag it but you don't tag it in a very exact way. You always tag global tags. You don't know if this video is very long, when is your child playing and where in the scene is your child. So we want to get some information about the content, the video itself. And of course we use, this is an illustration of different aspects of music percentage. We also use the context information that we can obtain for instance from YouTube tags and so on. What do we mean by semantic descriptors? Of course we don't want to provide descriptors which are very low level. Like for instance this is spectral centroid of this frequency range or the optical flow gloss from here to here. But we want to have textual descriptors which are richer than tags. So we want to work on semantically meaningful descriptors and address the so-called semantic gap. And finally we want to rely on methods that allow trustworthy content based semantic description because we realize we cannot trust our performance estimates, for instance precision recall of measure. If we don't know if our data set is reliable, if we don't know if our data set is good enough and representative of the problem. So we want to investigate also how to build better data sets, more representative and if maybe not a lot of data is needed but we need data which is well sampled and well distributed over our problem. So this is our goal and we want to go beyond the state of the art. For instance state of the art in music information retrieval mostly focused on either the audio file or the image of the album. And we want to work on commanding those. State of the art only works on studio recording with professional performance, very good quality material. We want to work with these user generated videos. We would like also to address the current limitation of systems which work with low level and then classifiers and to try to understand what are the limitations and the challenges of these classification methods. And we want to build data sets which are multimodal, not only in single modality and finally we want to go from validity, reliability and efficiency to understanding data set reliability while you are building a data set and after you have built a data set. So this is the things we want to do in this project. And then I will just illustrate with some of the first tasks we addressed which because we of course there are many problems related to this scenario. For instance quality assessment, how can we measure quality in the visual domain, in the audio domain but also musical quality of the performance. How can we synchronize different videos of the same performance? How can we for instance segment long videos and provide meaningful descriptions of one hour videos or two hours videos. And finally how to build mashups by combining videos which have better quality among the different sets of mix. But for the moment we have been working on music instrument recognition. Of course this is a very well known task in the literature. There is a MIRX competition running for four years only in audio. So we wanted to address this task and how does it perform in this particular material because some people may say you can recognize an instrument in an audio signal with a very good accuracy if the instrument is playing alone. It's what we call monophonic. So it's like people think it's a well known problem and more or less solved in the literature. So this is an example. Can someone identify the instrument here? It's a bass but some people say maybe it's a cello. I don't know, I mean depending on the size. And sometimes also depending on the bow but here you don't see a bow. And also if the instrument is in this position people may think it's a guitar because all the guitars are played like that. And of course in this video we don't have any tagging of this instrument but of this other instrument. So this is what we want to identify. We want to identify its play in a violin then there is a bass which is not played and then when the instrument starts playing and where in the scene. So of course there are many datasets for that but all of them are in the audio domain or in the image domain. For instance we selected a set from the ImageNet dataset which have two different instruments and this number of images and you have here an illustration of the kind of images we have. And then we got datasets for audio that are well known in the literature where you have samples of instruments labeled and you see the heterogeneity of number of classes and number of instance per classes. This is something that would be nice to take from publications because you always have to build this table of how many classes, how many instruments and then which are the features and then it's very difficult to compare the results because they are all running in different material. So the first thing we didn't find was a video dataset. We didn't have to use a YouTube API to crawl videos or user generated videos and also this was our first contribution to in this project. Of course videos are tagged but they are not tagged in terms of they are with global tags like this is an instrument but you don't have information where the instrument is playing or when we try to develop an online tool also based on the experience of the department on building annotation tools which are collaborative and then we try to for instance connect with the YouTube or you can upload your video and then you can annotate for instance the region and where the instrument is playing and then we integrate algorithms for optical flow tracking and to track this object in the scene and then we integrate also algorithms for audio processing to detect when the instrument is playing or not you may see there, here, there two instruments and then we can focus on one of them or we can focus on both so this is the kind of annotations we are now collecting. So in addition to that we try to train our models on only images and or on only audio and then apply them to video recordings and we were very unsuccessful for the moment. If you train of course with images and you evaluate or test with images you have a very good accuracy we could even improve state-of-the-art methods. If you train with audio material and then you evaluate with audio material you have a very good accuracy measure, you have even a very good accuracy in different data sets, cross-validation and everything. But of course if you apply on this material this is an illustration of four different videos and on the top left you have the estimation of the image recognition engine and on the bottom right you have the audio so you see there is a lot of errors and confusions between instruments because of course this material you haven't used for training and then we might think okay so let's use it for training so let's build more data, more data and to build it so this is one option but another one is to understand how much data should we need and if not the highest number is the biggest accuracy of course. So we then research also in the same way on reliability of data sets of course there are methods or statistical methods that try to model and simulate data sets that allows you to check for instance if you reduce your data set how representative are the performance measures you get or if you for instance sample it how do you want to sample and you have some statistical measure but also some measures related to the content if you have only piano music or if you have only a particular player of course you will have overfitting over your mother so we try to develop our package to try to measure the estimated reliability of data sets this is mainly the word by Julián Orbano and we have already this package where we try to apply this to existing data sets in the music information retrieval area and also in our own research. Of course there are many people also doing similar work to us and in this Maria de Maeth there is also the opportunity to collaborate with other people for instance with the University of Technology we have been collaborated in the Phoenix project because they have a system that try to identify the faces and then try to identify if someone is playing or not playing because it shows up it was easier to just look at the face of the player and not at the instrument so because they are occlusion they are a lot of challenges so we are trying to collaborate with them in order to have complementary approaches and also tools for video annotation and integrate some of the things they have we also have some collaboration with other researchers that do specific work on a specific instrument this is an example of candombe which is a percussion band and they try to detect when each instrument is playing and they have a specific database we are using for training also our models and then we are trying also to collaborate with people for NYU and Spotify and build more meaningful music information retrieval evaluation methods this is more or less ongoing work of course we have started with instrument identification because we want also to contribute to the initiative evaluation initiative in the Izmir area and of course we would like to address other tasks which are exploiting more the temporal aspects of videos which are not at the moment very complex our approach is not very complex to our coson synchronization and we would like us to provide textual descriptions maybe using some natural language synthesis or textual synthesis or for having our descriptors in a user friendly way and this is mostly everything I wanted to mention I think it took 20 minutes so if you have any questions or comments Hello, thank you for the talk I have the questions and the first question is related to if you have the cocktail boys party which I think is similar to identifying different instruments in a scene can you lie accurately identifying different instruments? Yeah, as I said there are state of the art models we have done for instance that can have training you train isolated instruments and then you can recognize in mixtures at least the most predominant instrument for instance in the image processing I know there's also object segmentation it's not my error but there's no single approach which combines both things in the same kind of algorithm of course it depends on the data you have if you have a very clear recording and you know more or less the style people can at least predict which instruments you will have and it's much easier if you use contextual information to have something like this so it depends but there's some approaches always they are based on training so it's also a bias to the data you have for training Is it a supervised learning? Yes and the ground truth? Is it provided? There are several datasets, the one I mentioned and for instance at the music technology group we publish one on polyphonic so they are soloist instruments and then you can have ground truth from instruments there's another which is called RWC which also even denotes the transcription of the score My second question is regarding the semantics can you identify different type of players? Can you say players of this instrument have a similar style of playing? Yeah, that's a ultimate goal to compare quality and compare similarity on the way they perform not only auditory but also on the way they behave in the scenario or in the visual scene but yes we have some methods for comparing performances for instance you can compare the tempo you can compare the dynamics of the piece you can compare articulations several there are many methods in the literature or different approaches for building comparison between performances Thank you Emilia for the talk I wonder if the extra information that comes in YouTube like commentaries or maybe the title of the piece that has been played all these are you going to exploit it? Yes, we wish but for the moment we only get the instrument tab from the title and to build our first dataset and then of course for training we would like to have more control of the dataset because it's very noisy sometimes this information you have on YouTube and of course these are maybe very long videos and you want to train with the parts that are really containing the video or the instrument because sometimes you have different instruments but yes we use it for a first dataset generation Any other question? Well I guess thank you again


Follow me. I thought I heard your name. What is it? What? S-E-B-S E-B-S There we go, top one. I'm following reason vision, bro. I'm running for you!


So now I'm going to go back on up and say invoice and Say let's say this is customer number two again And so this is actually a fairly complex transaction that this records now And so what's this going to do when we when we record the sales tax? It's gonna there's inventory involved. So there's a lot going on. It's an invoice. It's going to increase the Accounts receivable it's going to increase the accounts receivable by the full amount including the sales tax Let's make it the generic five generic five Including the the sales tax to 130 125 revenue is going to go up by the 125 The difference is going to go into the to the sales tax payable of 625 It's not going to hit the income statement. That's kind of weird because you can see you can imagine it Why don't I just record income as 131 25 and then when I pay the sales tax I record a sales tax expense because the sales tax is not in theory subject to you your your business It's it's you're just a collection agency. It's not part of your revenue And it's not going to be part of your expenses. That's the idea It's going to go on the it's going to go on the on on up a balance sheet account and then also Inventory is going to go down by I think we said $50 the amount that was driven by the item here for the cost and We're going to have the cost of goods sold go up the net impact on net income will be 125 minus the cost of goods sold at Which I believe was the 50 and we also have a sub ledger for for the customer to Which will track the accounts receivable that will need to collect in the future and a sub ledger for the inventory item Now because we're tracking it on a perpetual inventory system tracking it by unit So let's save it close it and check all that out and see if that is indeed the case going to the to the balance sheet Running it and then scroll down. So now we've got the accounts receivable. So if I go into the AR There it is and that's for the full amount of Hold on a second. That doesn't look like the right one. That's not the right one at all Let me change the range up here. Did I put it in for 2023? 2023 Boom There it is. Let's bring that back to 2022. I'm gonna change the date going into that one Let's bring this back. You're too far in the future. You're too far in the future. We're working in the past man So there let's do that. Let's save it All right, so there it is 131 25 going back the other sides on the income statement Running it. So now we've got the sale of product going into it There is the 125 scrolling back up the other side is in The sales tax and this is where our focus is of course where they put this long account name California Department, that's because you might have multiple sales tax You would think you would just call it sales tax payable a liability because you owe it in the future But they put this long name because you might have multiple people that you might be paying the sales tax to you there's the 625 and Inventory is going down So inventory went up when when I added like the beginning balance of inventory and then it went back down by I believe this 50 this one right here that one right there So scrolling back up and then the cost of good sold over here on the income statement is Affected and we're also tracking the inventory and the sub ledger for the customer So the point is the main point We're looking at of course is the fact that I had to enter an invoice or sales receipt and the item in order for me to get this This payable to track the payable that I'm in the future going to then pay With the sales tax widget so we can imagine after a month or after a quarter or after a year We're gonna have to take the money that we collected from the sales tax Which is going to accumulate in the liability account. We'll use the sales tax widget over here, which is going to be in the sales tax area to then pay the sales tax using using the pay the sales tax which is a little bit difficult to kind of view in real-time because I Mean it's difficult when you're trying to do it like in the past in a practice problem because a sales tax Runs kind of real-time, but it'll generate a check when you have an amount that you owe here You can generate a sales tax check based on you know, what you owe and that will decrease the sales tax Here when you pay it so the point is if I if I look at my flow chart over here I have to then Enter my sales tax if I'm gonna use that widget method the whole cool little widget thing within quickbooks I have to enter my sales using items using Invoices and sales receipts because those are the things that help me to track the sales tax So I can't really just wait till something clears the bank if I want to use that cool little tool


أعوذ بالله من الشيطان الرجيم بسم الله الرحمن الرحيم والحمد لله رب العالمين والصلاة والسلام على محمد وأهل بيته الطيبين الطاهرين اللهم صلى الله عليه وسلم محمد وآله محمد my dear brothers and sisters my dear viewers wherever you are I seize the opportunity of the start of the holy month of Ramadan to congratulate all of you on this wonderful occasion and I wish you the best and the fruitful, spiritual and joyous Ramadan The Almighty Allah سبحانه وتعالى has said in the holy Quran بسم الله الرحمن الرحيم شهر رمضان الذي أنزل فيه القرآن هدا للناس وبينات من الهدى والفرقان We are all have been in a situation where we are waiting for or are anticipating a very important person to arrive whether that person is a loved one or a long time friend or a very important person an executive or high ranking official We have been in a situation that we are waiting we were waiting and anticipating for that person We may differ in some aspect of a preparation but the common theme is the same We take our time in a preparing for that event We take our time in a planning for that event We make sure that at the moment of the arrival of that person we are ready we select our words carefully even our statements are carefully chosen We make sure that all the amenities and conditions are available We make sure that the scene is suitable for that event and for that meeting Eventually, we take stride in making sure that every aspect of this meeting comes fruitful and suitable so we can utilize it at its maximum Why? Because it's a great opportunity because it's a great moment and maybe it's not so great after all However, we do our best to make sure that we take advantage of every single minute of that event Why? Because it cannot be repeated That great moment is intolerance of being repeated again It's only once The opportunity can be easily missed Therefore, we plan ahead of time We spend hours and hours for planning and reviewing and editing our statements to make sure that this meeting turns to be successful because we cannot tolerate losing that opportunity We want to make sure that we benefit the maximum amount of benefit of that event When we look at the holy month of Ramadan we see that it has similar situation Ramadan, the holy month of Ramadan is an extraordinary opportunity that we cannot tolerate missing If we miss this year this is a great opportunity We never know whether we will have the chance next year to repeat this opportunity We don't know if next year I and you are alive and healthy and can perform our functions again Therefore, we should consider that this could be our last opportunity our last month of Ramadan in this life When we met with such kind of opportunity what should we do The Almighty Allah SWT has said that the most favorable month has arrived Therefore, you need to make sure that we take as much opportunity and maximize our opportunities as much as possible The Prophet says that you are faced with two important opportunities Number one, the valuable guest that is arriving upon you and second, that you have become a valuable guest arriving on the court of Allah SWT and other opportunities at one occasion First, a valuable guest in the name of month of Ramadan has arrived upon us Second, we have become as a valuable guest to the Almighty The Prophet SWT says سبحان الله ماذا يستقبلكم وماذا تستقبلون Do you know that Allah is awaiting for you and what you are awaiting for these are great opportunities that hardly anybody can recognize Then the Prophet SWT elaborate further At the advent of Ramadan he had this eloquent and a beautiful sermon where he says أقبل إليكم شهر الله بالبركة والرحمة والمخفرة Oh people The month, the most favorable month of Allah has arrived upon you with blessings, mercy and forgiveness شهر هو عند الله أفضل الشهور A month that in the view of the Almighty الله سبحانه وتعالى is considered to be the most favorable وأيامه أفضل الأيام and its daylights are the best daylights ولياليه أفضل الليالي and its nights are the best among all nights and days وساعاته أفضل الساعات even its hours its minutes are considered to be the best among all hours and moments Therefore, you need to take the opportunity Then the Prophet mentions this great effect of the Holy Ramadan where he says is in this month that you are considered to be the noble guests of the Almighty الله سبحانه وتعالى So we should treat ourselves as the noble guests arriving upon the court of the Almighty الله سبحانه وتعالى Now the months or calendars are too kind one is the solar calendar the solar year and the other one is the lunar calendar or the lunar year The importance of the solar calendar is very well known we know it by evidence and by experience we know at what time of the year winter arrives and at what time spring arrives and what time of the year summer and fall arrive these we experience through years the evidence has shown us what importance a solar calendar can bring therefore you see that the farmers, the peasants get ready for those moments they get ready to cultivate the land to prepare the land and plant the seeds well ahead of spring for example when the spring comes they plant the seeds and then they pollinate the trees they harvest the crops by time because experience have showed us at what particular date what season will start and what season will end the same you can say about the migrants who travel at certain season of the year at what part of the the planet to another part of the planet even the animals the birds understand the significance of the solar year the migrant birds they travel for long long distances from one part of the planet to another part of the planet due to changes in the climate all of these have been seen and sensed by experience we have noticed the importance of the solar year by its evidence and by our experience but what about the lunar year what about the lunar months and lunar calendar how can we recognize recognize its significance unfortunately physically we cannot tell the significance of the lunar year unless we are told by the divine revelation either it is narrated in the Holy Quran or by the narrations of the infallibles those two sources that tell us the importance for example of particular eaves and nights the importance of particular months for example we would have no clue what is the importance of the night of destiny is unless we read and recite the Holy Quran the Holy Quran would tell us the importance of the night of destiny the night of destiny the almighty Allah says the night of destiny is more valuable than 1,000 months or 30,000 nights we had no clue recognizing this if the almighty God has not revealed the secret when we look at the Islamic traditions and Islamic literature we see ample ample and abundance of these narrations that speak about the importance of those calendar those lunar calendar moments for example the night of destiny in the holy month or the eve of Mishaaban or the 27th of Rajab or the day of Arafa these narrations tell us the significance of those important days now how important they are we do not know except that we take it by faith from the infallibles they are the ones who tell us the exact amount of importance that we are experiencing therefore we should make sure that we do not lose those important night's we should not lose those opportunities because if we lose those opportunities we don't know how much and how many good things we will be losing for example a man come to Imam Al-Sadiq and tells him it is the month of Rajab and I want to compensate for that month what should I do to compensate for those days that have already passed what you have missed is so much that no one would know except the Almighty therefore our duty brothers and sisters in these holy days is to recognize the importance of the holy month sometimes we are totally oblivious we don't know whatsoever the importance of the month of Ramadan sometimes we have only a vague and general understanding like someone who has his own iPhone the very advanced iPhone and he only knows certain capability of this phone he just use it for calling friends and some texting but he doesn't use any of those applications he is not aware of those tremendous applications that are in the iPhone this is a great loss as well therefore we need to understand the value of the holy month of Ramadan the prophet peace be upon him says if the person the believer knows for sure what is the value of Ramadan he would wish that the value of Ramadan would be the duration of Ramadan would be for one year one whole year why because of so much importance and values that Ramadan has going back again to the first statement that we said we have to be prepared for this holy month sometimes our preparedness can be physical we should be ready physically for those great moments we should make sure that our bodies are fit for those days that we observe fasting we should make sure that we don't jeopardize our health ahead of Ramadan and make sure that we end up the 30 or 29 days of the entire month of the holy month of Ramadan and we should be spiritually ready for this great month in the same manner that when we get ready by cleansing ourselves and wearing the best attire when we meet very important person we also should make sure that our soul is cleansed from any odors from any pugnations odors of insincerity hypocrisy jealousy indecency these are the contaminations and stains that sit on our souls that we should make sure we get rid of them before the start الله سبحانه وتعالى has said if you want to see the full benefit of this great month then you should complete it and it's complete course just like the antibiotic course of medication that we take you should take the entire cycle of the antibiotic tablets in order for them to be beneficial the almighty Allah says والتكبر الله على ما هداكم in order to see the full potential of the holy month of Ramadan we should complete it's entire days I pray to the almighty that give us the ability and the guidance that we observe the fasting of this great month الله مني يسألك بسمك يا الله يا رحمان يا رحيم يا كريم يا مقيم يا عظيم يا غديم يا عليم يا حليم يا حكيم يا حكيم سبحانك سبحانك يا لا إلى ألغاث ألغاث خلصنا من النقار يا سيد السادات يا مجيب الدعوة يا رافع الدرجة يا ولي يا الحسنا يا غافر الخطيئة يا معطيئة المسألات يا غابلات توبات يا سامع الأصوات يا عالم الخفيات يا دافع البلياء سبحانك يا لا خلصنا من النقار يا خيرال وافرين يا خيرال فاتحين يا خيرال ناصرين يا خيرال حاكمين يا خيرال وازقين فارثين يا خيرال حامدين يا خيرال الذاكرين يا خيرال منزلين يا خيرال المحسنين سبحانك يا لا ألغاث خلصنا من النقار أنصرين يضعون حقا لأنواقي وإنراضة وإنحاء الأفطار وإنحاء مباشرة ويواطف ويواطف ويواطف ويواطف ويواطف ونجل جبرايل قد قد تخبرهم يا محمد الله قد ترسله لك ونسألك لك أن تنسى المرسة التي تجدك المرسة وحرية في تحديدها تنسى المرسة ونعطيك شيء أكثر يمكنك وكوك وكوك وكوك تستخدم عندما تستخدم هذا you will seek a protection from the Almighty And he gave him this beautiful du'a called du'a al-Jawsh and al-Kabir That consists of 100 segments In each segment there are 10 names of Gods 10 attributes of the Almighty The segment number 55 include 11 names So the total of the beautiful names and attributes of God are 1100 And it starts like this بسم الله الرحمن الرحيم اللهم إني أسألك بسمك يا الله يا رحمن يا رحيم يا كريم يا مقيم يا عظيم يا قديم يا عليم يا حليم يا حكيم سبحانك يا لا إله إلا أنت الغوث الغوث خلصنا من النار يا رب We will try our best to translate some of its beautiful vocabulary In the first segment it starts with اللهم اللهم is similar to يا الله oh God That Arabs use at the beginning of each statement Then the word says إني أسألك بسمك يا الله I ask you, I ask thy, I ask thee by thy name When you look carefully You see that the supplications And the words of prayers of أهل البيت عليهم السلام Always use an intermediary A vehicle, a means between us and the Lord We never say that we beg you or your divine existence Rather we beg the name of God Because he himself has told us how to start the prayers When we say بسم الله الرحمن الرحيم In the name of God We use the name of God as an intercession As a vehicle, as a means that take us closer to the Almighty When you look at the tradition of أهل البيت عليهم السلام You see always the inputs They bring an intermediary between us and the Lord Sometimes it is the name of the Lord اللهم إني أسألك بسمك by your name Sometimes the attributes of the Lord For example When he starts the eloquent dua of كميل He says I beg you, I call you upon the mercy that has engulfed everything I beg you by your power that has overpowered everything So we always bring something as an intermediary To seek that intercession That gets us closer to the Allah Sometimes we use other valuable entities Like the infallibles Like the Prophet peace be upon him The Almighty Allah says Those people who have committed sins And incurred injustice against themselves When they seek repentance They go to the Prophet peace be upon him And they ask repentance from the Prophet When the Prophet peace be upon him Repent and ask forgiveness for them Then God will forgive them As the ayah says So this is the first segment The theme of the second segment Is that we seek help and assistance from the Almighty Where it says يا غافر الخطيات يا معطي المسألات يا قابل التوبات يا سامع الأصوات يا عالم الخفيات يا دافع البليات We stop at this beautiful world يا غافر الخطيات The one who forgives the sins When we incur injustice And wrongdoing against some people Maybe you do it once They forgive you Maybe you do it twice They forgive you Maybe you do it three, four, five times Until a point that they will abandon you Even our parents Even those that are very close to us Our loved ones If we do wrongdoings to them Multiple times at one point They abandon us They will become unforgiven The fact that the only entity That can be forgiven And no matter how many sins we do It's the Almighty Allah سبحانه وتعالى No matter how piles of sins we commit Still the Almighty has this beautiful attribute يا غافر الخطيات He will wash away all our sins Not only that He will even change those sins If we seek repentance And forgiveness from God He will change those sins Into good things Into reward As Allah سبحانه وتعالى Says إلا من تابا وآمن وعمل صالحا فأولئك يبدلوا الله سيئاتهم حسنات God will change those wrongdoings Into good things Into rewards Provided that we seek Genuine repentance The third segment brothers and sisters About seeking superiority Attaining honor and dignity Where it says يا خير الغافرين يا خير الفاتحين يا خير الناصرين يا خير الحاكمين The best out of the forgivers The best out of the victorious The best out of the judges The best out of the bread winners Here it tells us how you need to attain To the highest level of things You do not leave yourself only Subsided to simple things If you enroll in a class You better be the best in the class If you are a judge You want to be the fairest judge If you are a doctor You want to be the best doctor This is how the Almighty has told us This is how we need to be As he says in one ayah In the words of متقين He says واجعلنا للمتقين إماما Make us the best out of متقين The leader of the متقين Meaning that you become The best of your class Or الإمام علي ابن أبي طالب عليه السلام Says واجعلني من أحسن عبيدك نصيبا عندك وأقربهم منزلة الميك Make me among the best The closest of people Oh people Surely loyalty And truthfulness are twins I do not know a better shield Against the assaults of sin Than it One who realizes the reality Of return to the next world Never betrays We are in a period When most of the people regard betrayal as wisdom In these days The ignorance call it excellence of cunning What is the matter with them Allah may destroy them One who has been through thickened thin Of life Finds the excuses to be preventing him From orders and prohibitions of Allah But he regards them despite capability To succumb to them And follows the commands of Allah One who has no restraints of religion Seizes the opportunity And accepts the excuses For not following the commands of Allah


Hi, I'm Leonard Pottering and I work in system E as you might know I'm speaking today about something called the bootloader spec, which is actually like the specification for this We actually came up with a long time ago, but I kind of want to Like spend more time again to make this more Popular and established in the in the community because it's actually awesome concepts This is actually same talk I did at all systems go last year. So if you have been there, there's nothing you for you in this For everybody else. I hope this is gonna be interesting So, yeah, the bootloader spec. What is this? Yeah, long story short if you go to that URL you can actually read the specification What is SD boot the other half of this talk it's yeah, if you want to know the details go to that URL and I guess I can go now right like now, you know everything SD boot is something that I was used to be called gummy boots in an earlier life and then we Moved it into under the system the umbrella and now it's called System e-boot or short SD boot So let's go a little bit into detail what the bootloader spec is there's a specification for any platform any firmware for Bootloader entries the boot entries are drop-in files right this takes Inspiration from how we generally do things in rpm based and devian based systems Where we nowadays like have for example desktop files, which are dropped into one directory and simply the act of dropping it there We're just as the the application with GNOME and KDE and whatnot right and we do this all over the place in rpm Right, we always have like for example if you want to extend the stuff that I don't know started like it's run When you log into your bash we have these directories in Etsy profile D or something like this Or you can just drop stuff and get run when you log in right so this mechanism of having drop-in files It's something we have been using on on our PMA systems on Linux for a long time It's something we have been using on system D for for a since the day one like for example If you know temp files dot D it's exactly same thing you have one directory you drop it in there The incident's dropped in it's kind of registered with the system and everybody reads it right So the idea of the bootloader spec is let's take this very basic idea And just use it for bootloaders so that every bootloader entry that you see in the bootloader menu is just one of these drop-in files These drop-in files are located below below something. I generally just abbreviate dollar boot here Looks like environment variable, but it's supposed to be just a placeholder that dictates where to find these drop-in entries They're actually what dollar boot actually means that to option that's option a Where dollar boot is a petition And if you use mbr petition table like the good old classic dust petition tables that are posted popular in the cloud Then you use that petition type there which is EA in hex or on GPT you use this long Type u80 that is where these entries are supposed to be found Option number B the dollar boot X equals the EFI system partition This is actually on UFI system. That's much nicer because you don't even need anything Like additional the only reason why it option a exists is basically because yeah, not all systems are UFI yet and People are a little bit allergic about Growing or shrinking or changing the ESP hence It's all sometimes a little bit nasty to actually drop additional files in there simply because there's no space So that kind of makes sense to use a separate petition But the data we put in there has kind of the similar life cycles as the bootloader itself that we put in there Like it's seldom updated, but it is updated and the update is relatively simple There are no Like as particular attributes file system features anything required. So yeah, this can be two separate Petitions can be the one petition on classic MBR and it can be the other one the SPO and Moren system can be both in parallel, but yeah Because this these options exist. We just refer to either of them depending on what actually applies to your system as dollar boot the file system of these file systems supposed to be a VFAT like UFI mandates that kind of for UFI for the ESP, but we also say yeah for the for the The other put the petition the one of option a we say yeah, probably should be the VFAT too It's not enforced by sophistication. It's just a suggestion The reason why it's a suggestion is is because it's actually the file like that the file system former that really everybody understands all the the Operating systems understand it and the firmware is due to like UFI generally does at least and a couple of other firmware types That also support this so it's kind of It's not a good file system Nobody thinks that it is but the the basic functionality supports is absolutely enough to manage a couple of drop-it files so System he actually knows that this is how dollar boot is defined and actually helps you with things So in case of option a after you booted the system Yeah, this file system if it is found it will automatically Mount to slash boot only if the user has an FS step entry that mount something else to slash boot though Right like so we don't try to fuck with your FS type configuration It's always takes precedence, but if you have no such configuration and we do find this petition on disk Because it's in the petition tail. We just use it mounted for you We mounted for you actually in a really nice way because it's like auto mounts and stuff like that So it takes credit like like we accept the fact that VFAT is not the best file system in the world so it's actually set up so that When you actually access slash boot We mount it then you can you make your changes and a couple of seconds after you stopped accessing it gets unmounted So it gives you this wonderful guarantee that in almost doing the entire runtime of the system This directory is actually unmounted right again if you do the have something that's fs f stuff All of that doesn't apply you don't get this nice beautiful auto discovery just Yeah, you get the classic stuff and VFAT is fucking broken In case of option B Systemally currently does the same thing but monster to slash EFI because then it's the ESP and that's where it should be Yeah, so this just just like this has nothing to do with the bootloader spec It just is an indication like if you adopt the bootloader spec how you get these wonderful things where? The these VFAT partitions are managed as nicely as we can oh by the way also when you first access These directories we do an implicit of s check in the background so you can actually yeah You have this lazy behavior where there's the best guarantee that when your access either got fixed up And then when you stop using it, it's definitely in a clean state because it's unmounted Okay, let's go back to the actual bootloader spec stuff Now we know where to look for these droplets So now let's look at the kind of droplets. We're actually interested in that as type one Type one is basically inside of the partition that are called dollar boot Well, we have this VFAT file system or not for some other file system We have a directory called loader and under that and what directory called entries and then we have All kinds of confiles, right? every confile Just synthesizes one boot menu entry These confiles are simple text files. They are intended to be extremely simple. There is no no Pattern replacement. There is no, I don't know multi line Stancers. There's nothing right. These are files that are supposed to be generated by user space and read by user space But primarily are supposed to be read by bootloaders bootloaders should not have programming languages built in is my firm belief They should be simple, right? So these droplets are dead simple They describe the kernel they describe an inner d they describe a name a version come over a sample of other metadata This metadata is kind of enough to boot the thing But also kind of is useful for actually ordering it on screen, right? So that the newest version is always on top and things like that There's a second type. The second type is something that I think is actually even more interesting But it's less flexible then the second type is in the same partition dollar boot And they are subdirect to EFI Linux. We have binaries called something EFI these binaries are You if I executables, you know, I'm not sure how well, you know, you if I but basically the ufi firmware it has a concept of Basically you have like Windows X of binaries or Linux ELF binaries You can just program your code and then you can put it then it can execute it if you want so type 2 stuff Type 2 droplets are just binary files And there's nothing else done. There is no further metadata stored outside These binaries are unified EFI kernels, right like so they combine My kernel and in already some metadata and possibly other things into one binary This is beautiful. I think it's beautiful for a couple of things and we'll see later why but yeah So the gist to take away is Taiwan descriptive configuration file type 2 One binary and that's all from the menu item So let's go back to type one. There one is generic flexible. You can use a text editor because it's a very simple text file Yeah, it's a it's a very powerful thing But it also means that because it is a text file It will change from every system to every system because every admin can add of them and make changes type 2 is Much simpler because you only drop in one file instead of having just a description file along with the kernel was And early and stuff like that so it's very simple that makes it very robust because the only change We actually need to do to the VFile system if we install something is drop in one file and hopefully that's we're good enough to Doing that rather truly safely It also means we can't sign these entries as one thing right like we can because We live in a world nowadays where if you sack your boot in these things and it's a second world You basically have these EFI binaries that you sign with some key and this key is like the public key for it is known by A new firmware and things like that So if you have everything in one single file and one single PE file even you can sign it as one And that basically means that everything inside of it is sign it once so it's a kernel that is signed It's an already signed which is something with by the way currently never do Yeah, it's on the other hand specific to you if I just inform right because it assumes that you have something like an executable And yeah, I already mentioned that it's one file updates Right like because if you install a new kernel you get one file drop them if you remove a currently get one file removed It's that easy. There's nothing else To give you an example how this actually looks this is example for type one I hope the font is not too small, but as you can see it's extremely simple It's just one word some white space some value one word some white space some value and so on so in this case That's the point actually work So in this case you have the title and it's fedora 30 you have a version this version is supposed to be used for sorting the entries in the boot menu so that if you never Did anything specific to it just the new stuff can be ordered first the machine ID is basically You know, it's the same thing as at the machine ID. It just tells the bootloader That these things belong together for one installation of the West It's primarily interesting if you install multiple operating system in the same hot as give that's what you want to do Like for example install. I don't know Ubuntu and fedora on the same thing and it should do something useful Options you guess it is a kernel command line Linux is the kernel In and already is the inner elite combined that's simple The parser You can write for this in any programming language you like probably in five minutes and there's nothing fancier to it Right there are no line breaks that allow you multi line fields. There is no pattern replacement Nothing dad simple simple enough for any bootloader to implement Key take away here is both types can be used together Right like so if you have one if you actually have fedora, but that does the snippet thing We call the ones powerful and and things like that and then you have Ubuntu or some weird other thing And they use the unified one totally fine combine them together Boodlers are supposed to look for both on platforms that supposed like support each right like they again the config thing like the config drop-ins works kind of for every like conceptually works for every kind of form or every kind of architecture the the PE file drop-ins only works of course for you if I Another key takeaway is really these bootloader entries right like these directories are supposed to be standardized So you can share them between OS's We always had this problem with how the way the way how we install grab that if it's all one distribution So the other distribution they start fighting for who owns the the the the bootloader basically Who gets to be the one that is starting when the system boots up and then had all these magic code in there I tried to find the other operating system in particularly windows And then generate bootloader entries for that and then so they can still boot that but then if you update windows And it override the bootloader again. You just everything explodes right so this is supposed to resolve this issue right like because There is nothing in the specification where It's not clear who owns it right like because everybody can create the directory there are no options in creating a directory But the files in them they have specific owners right like every file is supposed to be owned by one for operating system installation And if I'm Fedora, and I see the Ubuntu entry there Yeah, I'm not supposed to ever touch that right like I'm not gonna remove that I'm not a changer or anything like that So there's clear ownership of each files exactly how we do it for IPM right like if I if I want to Remove the bin LS binary for my system. I know it's owned by the quarry utils app I mean that's the only owner and I don't get to change it because it's open for you to so it's defines Clear ownership and that allows us that he actually can share these drop-in directories between OS is for the first time safely Of course, you still need one bootloader because the firm was just gonna load what right?  I implemented myself a grab couple of years ago Now we're gonna merge because crap is in this weird statement state back then I think probably still now But I know that a couple of other bullets implement this. We have one in particular We support actively it's called SD boot. It's not used by Fedora I think Fedora should use it, but it's a different question and this talk I don't actually want to push people so much to use SD boot. I mean you should of course, but The key takeaway is really like I want to push people to acknowledge that we should really go to something like the bootloader Specification which defines clear interface so that every peer whoever has to do something with bootloader entries Actually can cooperate in the same way and it reads the same stuff Yeah, most importantly from my system the perspective these are the two things that implant as far as like our bootloader SD boot Which is a ufi bootloader awesome thing a second part of the talks about that, but also If you do system these kxsac reboot, you know in system D you can do system control Reboot and reboots a machine, but there's also system control kxsac and reboots a machine with kxsac So that the kernel is automatically loaded you never go to the thermo, right? For this stuff, of course, we need to figure out where the fucks the kernel whereas the inner dirty How do we find that to load it right the way we implemented this is just like yeah We another user of the bootloader spec we look at these directories and then we see it there We can parse these file because I travel to parse and then we know What to tell the kx a kxsac system call to actually load into them and like what to reboot it So yeah, so in a way system itself is actually bootloader because you can boot up a system You system then you can use system control kxsac to use all the same bootloader and bootloader entries to start something else So much about the bootloader spec. I hope you hope you got what this is about You know just to compare this how we traditionally do this with grub, right? Like we have in grub we have a couple of scripts that Generate scripts that generate scripts that eventually drop out a grub configuration file, right? So it's all about yeah, and then everything has to be owned by this one operating system that install the bootloader And they fight for each other and you never know what how this actually ends up and the the Discovery of all the bootloader entries always happens on the operating system and never In the actual bootloader. So it's extremely fragile messy and Like from a computer science perspective having shell scripts and on the grub scripts and stacking all these together is this just I don't I'm not gonna use ejector for this. You figured out what I mean Anyway, any questions so far about this part by the way, I really like if people interrupt me ask questions instead of us doing doing that all in the end Yep So So I think I'm supposed to repeat the question the question was regarding the version field like if you look at that you actually see the RPM field filled in there directly from the kernel package and Different distributions use slightly different versioning schemes for the packages and how you're supposed to compare them So I think we actually use the Damian one here and so that I don't know the details you're like good question Yeah, I mean it's like yeah, you can probably mangle it anyway, right like this is But yeah, it's a valid question So so the question was regarding the metadata for the if I so if when I talked to you about the if I I said kernel in early metadata all linked into one PE binary So yeah, this is your answer the metadata contains that stuff So the the when we create unified kernels we take the kernel image itself We put a stop in front of it if we want we can put the inner D Then we can put the metadata the metadata is actually the way I think it's what people should do is it's just Etsy OS release right and then Etsy OS release You have the metadata the information about the operating system and that includes the OS version and we parse that In the bootloader and there you go, right? So we built on the on the Specification that is Etsy OS release basically because it contains this information for us It actually can contain a boot splash and all these kinds as well, but yeah different story. So very good question the question was regarding changing During boots the kernel command lines and how that would look like with if you use the specification like this very good question like I like later in the SD boot example, I can show you how that works, but it's basically There's the menu that like the data that you should populate the menu with right like a most bootloader is nowadays have a Have an editor for current command line stuff, right?  Like so this is not supposed, you know, the the drop-in stuff is supposed to be editable So you're supposed to be able to edit and and if you use sake boot and use unified kernels All becomes more complex if you actually sign them right like because I need to be careful what the user During boot is allowed to do with the current command line, right? Like because you cannot allow it to I don't know Do privileged stuff this way? So it's a bit different there, but in general Bootloader is supposed to use this as a default but gives the user the ability to edit it if they like So the the question was regarding if there's a central place where the kernel command line options can be configured That applies then to all of these things. So Yes, but it's independent of this stuff. So and in in system you since a longer time we're shipping the script It's called kernel install and it's some distributions use it not all Fedora does it's if you install a new kernel package that stuff It's called and it has a couple of call outs to a couple of things and Some of the call outs are like great good to build an already But for us it actually generates these snippets and the generates these snippets reading the meta information from the kernel Blah blah blah and and they're taking it already, but then it also A Returned one configuration files called Etsy kernel CMD line and that the one is that is used by this generator But I mean the specification doesn't really care about this like where this comes from right like Yeah, this is this is also semi supposed to be something that we semi-standard eyes, but it's Different place to standardize this and I don't know people can totally write tools that they generate these drop-ins From some other data at some other time if they like with some kernel command line coming from somewhere else Right, but the key takeaway really is that these files should be generated by the West and then never be modified during booted anymore Automatically like but the user always fine, but never automatically so that we don't have this that the bootloader executes a Turing complete script again, right? so So the question was regarding the relationship between the BLS and Fedora and the bootloader spec like we define it So the major difference is I would never call it BLS right like I let the Fedora people call it BLS I call it bootloader spec, right? I don't use the acronym So if you see the BLS of you see the Fedora one, so yeah, the Fedora people like it took us a long time Like we implemented the original bootloader spec like I don't know five years ago Eventually Fedora picked up and then I fucked it up completely It's like they really didn't get this idea that these snippets are supposed to be Stateless not a programming language Drop-its right like this idea that we inherited for an RPM like if you install an RPM The files that RPM installs for you They are exactly the same version for everybody who installs at RPM right like that's the idea of the RPM This is the idea that we implemented here, too Right you drop in these files and then they stay the same and the interpreter of these files doesn't do anything with them It just uses them as they are Sorry sure. I live in this world where we actually change things for the better Anyway, so the Fedora people they then thought okay Let's turn the bootloader into a interpreter for for pattern matching and doing Templating and all these kinds again So I mean this is kind of the problem this thing that always criticized with grub that has a touring complete programming languages There's a variable expansion and these kind of things I think a bootloader shouldn't be doing that right like so I think the use case why they want that is absolutely valid Right like they want to be able to change the stuff the way they went for it by making it something that that the bootloader interprets and that there has to be a Pattern language variable expansion in the bootloader. That's the wrong approach, right by all means Do it generate them if you need to change something from the West Just rerun the stuff to regenerate it's all fine But I'm pretty sure that these configuration files should be static Unmodified and not require everybody who implements a spec to basically like built M4 again Whatever so So So if you use grub, then they use this run my original petro cross to the real one So if you use a grub, they have the upstream grub thing for that as they use the floral one What but my original petro to the real thing the distribution like arc linux That use SD boot so they've used the real thing right system the itself is the real thing It doesn't use like it doesn't implement parent Expansion I mean one of the reasons why I'm doing this talk is kind of Trying to get the point across why having these static stings and having multiple Components deal with this in a well-defined way By not requiring much of it. It's kind of that's a message. I want to get across and hopefully in some people's heads here Why does the good thing? There's another question like how much time do I have now for Okay, then let's do more questions about it because I have a still the second part We actually talk about SD boot. So, okay, I understood half of this. Can you repeat it loud and the people are getting up? Just can you be quiet? So the question is regarding if you install to this distribution, which one is actually going to be the first one It's up to the for the bootloader to decide right like in SD boot we implemented like if the user never specified that never made one the default We just use alphabetically because for lack of a better algorithm. So sorry Fedora Debian wins Okay, I'll just jump to the other thing. So talk let's talk about SD boot this which is one implementation I personally think it's what Fedora should be using on the ufi systems because I mean one of the key takes it will take away I also want to kind of get into your head serious It shouldn't really matter which bootloader we use on on the various Architectures and various firmwares because we do use different one anyway right now like we use weird shit on weird architectures And I think that's completely fine right like a because I think the people who maintain these architectures should have all rights to maintain the bootloader because these things that tend to be highly specific to these things What I kind of want to the message get across is that don't standardize on a bootloader necessary I mean great if you've managed to pull it off But standardize on the bootloader back instead right so that it doesn't matter how it's implemented Just Agree on the data that they process as to do this one ufi focused Actually, it's not really bootloader as boot menu because it doesn't do anything right like a bootloader in my opinions suck Kind of something that loads stuff into memory and does some magic Low-level assembly stuff and then actually jumps into it and executes. We actually don't do this in sd boot Everything sd boot does it executes EFI binaries right so the actual execution of that stuff It's actually done by the ufi firmware together with the if I stopped that all our kernels that we built in fedora by default have So it's if I only hemp because it only execute if our binaries It's shift with system D It enumerates type one and type two of course It also has this wonderful feature that it automatically discovers windows and macOS during boot time right because the windows Installations and the macOS installations They have like they're totally recognizable if you just look at the partition table and a couple of other things so instead of finding them ahead when we install fedora and trying to then generate a Grub configuration file that then eventually gets out of date and everything is sad This thing actually in every boot. It just quickly looks miss their windows installed and offers the windows thing automatically So macOS installs it just offers that robust simple. You never have to think about this It also like if there's if I shall Installed or if the firmware supports boot into firmware. It just adds the menu entries for that as well because why wouldn't it? Underlining again John's runs EFI executable. That's why I call it a boot menu not a boot loader so much It's extremely simple to install you just to type boot controls boot control install and just drops files in the ESP There you go there's a boot control update The control update is a tool for updating the boot loader everything. It does it looks at the boot loader that is installed looks if it's actually something that Has been installed originally by our OS checks the version if the West has a newer version of the boot loader installed somewhere And use a lip just copies it over again. Just simple copy operation into the ESP. There you go another cool tool called boot control status just shows you which boot load is currently installed and What are the the boot menu options that are there very dead simple? Also has a couple of other features command-line editor the stuff that you asked for earlier has a really nice one that does emacs and non emacs like Control e control a and all these kind of stuff is really cool What also is pretty nice is that you can actually When you interact with it you can press a couple of keys and change what's the default and it's just instantly applied Like it's not that if you want to change the default entry that is being booted into that You actually have to do this from the West and write out a new configuration file that will fuck everything up and break again But instead it just use EFI variables, which is like concept that if I introduce choose store Which one is supposed to be the default so if you're unhappy with the alphabetic Ordering between Damian and for Laura you just go to the fedora entry and press D And then that's the default run for all future boots There's a question So the question was regarding boot control update if it fiddles with a firmware like I'm going to change the if I variables I figure it can do that, but it you don't like there's a switch that says no if I variables and just doesn't do that but I mean you're supposed to change the firmware, but it's That only really makes sense if you actually do that stuff on the system that you eventually want to run this stuff on So it supports both Yeah, there's there's also keys to change like you can press plus and minus and they can change the Boot menu timeout and as you press this is instantly applied and applies for all future boots So it's really nice If you want to change these timeouts you don't have to actually do that from them like and rebuild stuff and be risky and stuff like that It also passes timing data to the West super useful right like so that you actually know that how much time is being spent in the Bootloader so that you can do pretty graphs Afterwards that you know how much time the firmware took to initialize how much time our boot lower He took to initialize how much the kernel how much the inner Dean how much later boot checks and then you have performance data It's kind of beautiful Yeah, it's a there's also an if I variable referencing the ESP that's kind of nice and then the use by system D because and System you wanted this ability so that instead of having having an at CFS stuff We can automatically discover the petitions that there are on the on the on your heart isks and mother Automatically simply because they are there because the information the GPT is kind of similar to do to the information and At CFS tab it just lists file systems, right? So we figure it let's tag these file systems and what to do that was them explicitly and nicely in the GPT partition table But that really requires us to know where which petition table to actually look like look at if you have multiple heart Isks, right? So what SD boot does it actually encodes the? Petition it itself runs from an if I variable passes on to the operating system So that the operating system then can use this data from this variable to find a heart Isk that the bootler was booted from and then looks for the GPT partition table on that and automatically Can use it to mark the root file system the home file system and whatever else you might have What else very nice is when is this thing initializes? It finds all these bootloader entries right like type one type two and it finds windows and macOS and if I show stuff like that So after you boot it up you might want to know all these things right like you want to know what what was actually in the boot menu So it actually passes that to his operating system as well So that all the discovery all the automatic discovery is totally like you can figure out a posteriori after the west is up What did the bootloader actually find that? There's an if I variable decreeing a feature set because we kept adding a little couple of features so that Yeah, the operating system can actually figure out what the bootloader supports and then can interface with that All these stuff that you see here is actually something that we implemented an SD boot and that System the consumes at various places right like for example if you type system the analyze it actually use the Good menu performance Like the timing stuff to show you the breakdown how much time was in the bootloader and a couple of other System tools use the other stuff as well We want to keep this open that other bootloader can do the same thing right like because SD boot is the one way like we like best But also I think yeah system. You should not be tied to that one bootloader Other people want to do the same thing. It's fine. So we wrote this down. It's extremely simple so that other bootloader for example drop could do the same thing and then would integrate as nicely with system D as Yeah, the boot the SD boot And it's also support for automatic boot assessment automatic boot assessment means like in and if you use a system that is installed somewhere and it should be robust and Yeah, you don't necessarily have local access to it You really care that if you do an upgrade of the West or update of the kernel and it doesn't boot and automatically Reverts back to the previous version without you doing anything so they can automatically recover Like emirate devices have had that for a long time We didn't have that for in Fedora for the longest time So as the boot implements automatic boot assessment since quite a while in a relatively simple and robust way It's documented here how this all works it's Yeah, the the if you do automatic boot assignment you what you actually do is like you have a counter every time you Boot into one boot menu you increment that counter Before you actually boot into it and then if on the next reboot you realize that this counter is beyond some threshold Then you say okay apparently I booted into this too often Unsuccessfully and you don't boot into it anymore This of course requires that you somehow have after the OS has booted up some code there That figures out is the system correctly booted up and if it is resets the counter again, right? So you basically yeah, you start from from zero it comes up comes up comes up come up And even every time it files it comes up, but every time it succeeds. It just sets it back to zero so The question is how to store this data It basically means that the boot loader needs actually write stuff during boot right like because it heads to maintain this counter before it boots into something We looked in the for a while and figured out we'll do this kind of information file Rename because you always have this problem that the only place you can actually persistently Stored data in at every boot like I mean you could you could use the if I variables for this But if I variables are not the best choice for us because and VRAMs like the the backing store of the if I variables It's generally not a particularly high quality and that means it's okay to change it That's probably not okay to change it in every boot, right? So keeping a boot counter in the if I variables not a good idea So we don't do this other place where we can do persistency is for example, ESP itself. So now the question is um How do you store the data the most obvious way that most people probably think about is Storing this information like in a file, but if you do file then you need to block allocation is kind of sink So we said maybe you know firmware Files of some drivers firmware probably not the best quality. So let's avoid all the complexities of block allocation Let's do something that is highly likely to just require one block update without any block allocation of the allocation replacement algorithm So anything like this you figured that file renames are actually the best choice Right. So because that is a single operation that the firmware has to include To define there's no allocation of any new blocks involved. So it's extremely simple. So how does this work? We have the same droppings as we always had before and now we slightly extended it So that you have a suffix that's called plus three So the counter that was talking about is actually Reverse of what I told you because we actually set it to a high value once and then we count down and when we it hits zero We never booted again So on failure this counter is decremented by one and we also have you counter how often we did this then it fails again Decremented again Fail decrement again now we would never boot this again because we give up on it because we tried it three times all times failed Anyway, that's a question So the question was what makes me think that trying with the exact same setup is gonna work We never know why it failed right like one of the reasons why it might fail is power failure Right like somebody pulled the power blocks and you cannot distinguish that between software problem and and this kind of Hardware problem does not even a hardware problem, right? So I think the logic needs to be we try a couple of times in the system when we're reasonably sure. Yeah If you set it to three or to ten or the one is completely up to you Right like if your own decision is to just try once then go ahead. Yeah, that's exactly what we do so The boot assessment that SD boot implements just works by these three names, right and If you have multiple boot loader entries like multiple of these droppings We put the ones that have the counter at zero to the very end implicitly, right? So yeah, so the newest version that has a count above zero gets to the top So if that one constantly constantly fails the counter reaches zero and then ends at the bottom and then automatically The next newest one is at the top. We put that one and everything's good So extremely simple thing the boot loader is automatic boot assessment is an extension to the boot loader spec It's implemented by SD boot. We documented it. So actually other boot loaders are Totally okay to employ the same thing if they want but yeah, that's entirely up to to them The boot loader spec is something I first want to push towards making it more accepted across distributions and boot loaders Without this stuff, but if they want to go for the stuff as well, that's totally welcome. Sorry So the question was regarding can we define different criterias of what the failed boot actually means? Yes, but it is kind of out of like I mean the boot loader doesn't care anymore about that at all, right? Like this is something we added to system D as a general concept as a general concept a concept where we define a couple of targets like like multi-user target, but it's called Boot check dot target. I think I don't actually know what it's called the reason specification explains the thoughts to you but so the idea basically is that you can plug any program you want in front of this target and If that thing fails then we consider the boot failed as a whole for example You could do something like like a short small script that checks if you've got a DHCP leaves And then suddenly when the system fails to get a DHCP list you say yeah, it's failed boot, right? And then you automatically revert to the previous one, which maybe got it successful DHCP thing once Okay, very good question. So what happens if we cycles for all entries and all of them are in zero, right? So that none is interesting anymore. We use the counter for sorting only, right? So if you have a zero counter you sort it to the end, right? But it's a it's a primary key we sort after the secondary key is the version, right? So if all of them are zero then it's the newest one of the all the ones that fail It's gonna be good next.  Well, I mean it comes it depends on what you actually expect from bootlur I think this one is way simpler and everything it does because it does not implement a turing complete language does not implement it Scripting language it just takes this concept that the ufi firmware provides us with anyway because it provides it with so much You're like file system access in these kind of things Tries to figure out what we actually need from a bootloader, right? Like because in grub you live in this world where they have a scripting language that is turing complete that does Pattern expansion that does yeah, but it also does all this kind of file system in complex storage and things like that I genuinely believe there's way too complex for bootloader like this is stuff We should should do on the West right and it breaks our backs doing this properly on the West because it breaks all the time It's complex storage. I'm pretty sure this kind of stuff should not be Done by the bootloader. So it's a mostly a question of robustness and simplicity, right?  It should be boot menu not a boot doing crazy stuff. So Yeah faster I don't know. I don't care about faster, but probably it is. I don't know. It's like I mean grub is that slow Like I mean it's definitely is slow like if you rerun the grub stuff It's probably takes a couple of seconds, but who cares about a couple like if you if you install it I don't know. Yeah, it's probably faster, but it's it's not the not the I You know, I can criticize many things about grub. I probably wouldn't criticize them for performance. It's yeah So the question was regarding file system choice, right? Like because we fat is not a great file system I mentioned a couple of times in the beginning. So Yeah, I mean it's not a great file system, but there's no way you can go around it, right? Like because you if I firmware says we fat is the thing that you use, right? So you have to upload we live in a world nowadays We have where you have to upload bootloaders is there's no there's no option that right like bootloaders need to be updated Like any software we have like firmware needs to be updated like everything, right? So that we fat is in the mix is something you just have to accept I'm sorry like it's not the choice that I made it's a choice that the firmware developers of you if I made for you So this stuff just built on that and it tries everything that we can do to make this as safe as possible Is it fully safe? No way because it's not a transactional file system But so isn't the choices that we actually make in grub right now because I mean if you want to have a Save file system you need a journaling kind of thing, right? And they generally don't implement this it particularly not if you want this stuff rideable And we do want to have this rideable because I believe inherently that the automatic boot assessment where you count stuff We need a rideable file system for the bootloader so that we actually can do the counting stuff So my answer to this is no the fat is not safe But also we try our very best on every level to make it safe, right? Like for the automatic it boost assessment we boil it down to Fucking file rename right to to actually do the counting We don't we avoid everything that the fall says whether the former can do wrong and hence I think it's as safe as it could possibly be but not safer, right? And yeah again if you update grub it needs to write the to the VFAT as well, right? And it's an illusion to believe that grub never needs to be updated if I run like you know grub is this core code base That implements file systems that implements network protocols that implements ice cutting I think or something really like this HDP has certificate in the kind of thing It's an illusion to believe that a bootloader never needs to be updated and you have I makes us to update it in VFAT so we have to so yeah, that's my answer and how much time we have five more minutes Yeah, back to the automatic assessment Yeah, the user space assessment logic that was asked about earlier all part of system It's independent of the actual implementation of SD boot. So if you have a different bootloader like for example Grub you you and you don't want to implement the bootloader specs and okay. Sorry, but sure It's fine. You can still use the user space assessment logic that figures out of some things, okay in generic way and yeah Yeah, the bonus a system called Kexic already mentioned that it's kind of cool because you now can do a system control Oh, so actually there's a typo that's supposed to read Kexic Oh, no actually not had some different intention with this. Sorry. Actually, this is the correct line But because the bootloader communicates with the OS about so many things like for example all the bootloader entries it found It also works the other way around so that the West can tell the bootloader what to boot on the next thing Like on the next reboot and then system control we have system going to reboot dash dash bootloader entry if your bootloader Implements all the stuff that I was talking about like as the boot does that's for example You can use this to boot into a different operating system the next time And the bootloader will basically Like enforce the choice you made on the reboot How can you use this you can like for example, this could be exposed in gdm That automatically when windows is discovered to be also installed a system you get a reboot into windows options somewhere And then when you click on it It just uses it stores any if I variable the request that the next reboot should actually be into windows Then you reboot the bootloader sees that if I variable says, oh, I'm supposed to boot in the windows Let's see if I windows installed. I do. Yes, then this is the Bootloader entry that I'm going to boot into I'm also going to delete this if I variable right now so that only applies for this one and everything's good so This implemented as the boot again every bootloader can implement this actually It's long as they implement the bootloader spec and the bootloader interface and these kind of things. So, yeah, again It's all gonna be so much better if people implement the bootloader spec and bootloader interface one easy way to do this is actually use the sdboot bootloader there's also similar to the system with a reboot to bootloader and you can just boot into the bootloader menu and it just tells the bootloader menu to just turn off the timeout, right so that the bootloader menu is shown and You're not time out or to automatically boot something or I mentioned that system The Analyze uses the stuff and yeah, as you would as faction fedora you mostly can do sdboot and boot control install just works Most other distribution uses it as well like have it as well our clinics user by default This is my last slide that's unified on the bootloader spec Supports the bootloader interface support automatic boot assessment Use sdboot would be the best thing it gives you all of the buff for free and That's all I have and I don't think I have much time, but I maybe a couple of minutes. I don't know I still have five minutes so more questions, please. Sorry. Sorry. I didn't get it My question like so the question was regarding was so why is it not the default of fedora? Yeah, good question like I mean, it's kind of the thing is like I'm not a fan of grub as you can guess And I think grub's fine like if we probably always have to have it as long as we have Architects to support that don't use ufi which we do because that's aws for you that forces that on us But the way I think this all should look like is that we can all standardize on the bootloader spec Maybe a couple of the other things I just presented then an ideal world We then would use sdboot and on x86 ufi and use something else and something else But they all would just agree on the bootloader spec and implementing the bootloader spec properly in Grub isn't that hard, but yeah, it's a political thing let's say the That grub has the mind share inside of redhead that I kind of want to try to break a little bit with this talk I hope that's the answer So I think I understood the question correctly and you're asking how to easy uses and distributions like fedora It's package for fedora as mentioned and you can just install the package Consistently boot rpm and then you can just type boot control install and it mostly works. I hope I'm not promising too much because As mentioned there are two implementations of the bootloader spec the BLS one and the real one and If you and a witty snippet that the the current grub stuff does then it it's a script that was written there That does require variable expansion as the boot doesn't read that Same neutral reboot doesn't read that nobody does but so Not sure if I'm promising too much, but I think What is sufficient if you install says any boot and remove all the grub stuff and then now probably just works Yeah, so the question was regarding Whether stboot works with us if I know the answer is absolutely clearly and no it's not and never gonna do this But the bootloader spec doesn't require you any of this So my hope was that let's just all agree on the bootloader spec super simple everybody can implement this We already do on ufi then you have the option to use stboot and everything else use something else But yeah, I see who's not gonna deliver you booting from nbr. I'm sorry But I think in general one of the core ideas that we should follow is actually that whenever we do think about booting We always focus on what the gold standard of booting is these days and the gold standard of booting is like it or not I don't care. It's ufi. It's not nbr, right? Like so it should never be a race to the bottom where we make everything work like the old stuff from 1981 or something But let's do it the other way around where ufi is kind of what we want to go for what we want to support And then if we have systems that are not ufi then let's try our best to make them Enough like ufi. It's add a little bit of components that make it feel like ufi that add the Basic concept that we need so that yeah, we are not a race to the bottom But a race to the top we agree on the on the best functionality and not on the least functionality So the question was regarding and whether If you have installed windows and this stuff at the same time and then windows is updated and overwrite the bootloader This is mitigating anyway. No, unfortunately not so I mean it's it's Yeah, I mean the the primary bootloader that gets invoked by the firmware. It needs to support us for these options to show up, right? But yeah, so at least I mean if you have 25 Linux distributions and they would all implement this they won't Fuck it up anymore But yeah, we don't control windows and unfortunately when this doesn't implement something like this They have no interest in making our options show up in that boot menu if they had that would be great Maybe the new Microsoft is actually open to Be friendly to the sky that would be awesome, of course, but I think it's illusionary, but yeah Any other questions, but I think my time is one more who signs that It's up to you. I don't care Like it's a it's like I mean Fedora has an infrastructure For a signing Yeah, kind of the idea. Yeah, I mean second boot Thank you boot is about Making sure that code runs in exactly that version that the vendor of the code wanted it to run in right so when if Fedora builds a unified kernel that Locks all these things down. It would be ridiculous if we would break it up and not Actually make sure that yeah, everything's probably method the idea of second boot is to lock this down So second boot should lock it down and that's the answer to this, right? I Think my time's over. Thank you very much all for the good questions