So this is an astonishing equation, this is describing to you the process of photosynthesis at least in terms of the reaction running from left to right. So photosynthesis would go from water plus carbon dioxide to a sugar starch plus oxygen.  We could not have respiring, oxygen breathing using animal life on this planet without the presence of the other kind of life, photosynthesizing. This only developed after the development of the Earth at 4.567 billion years ago. The first, oldest evidence of life we have photosynthetic life is on the scale of the Earth's existence, not that long after. But let me just put the numbers up. We've got pretty good evidence that the Earth was first formed 4.567 billion giga years ago. The oldest rocks we have are in the order of 4 billion years. We've got rocks from our neighbour, the moon, which are older than that, which are very close to this. The oldest evidence of life we have is of the order of this 3.8 billion years in places like the Pilbara of Northwest Australia. And it's algal. The kind of life evidence we have is in the form of stromatolites, the sort of things you can see growing right now in Shark Bay. But that's the oldest life we have. Now this was photosynthetic life, so early on the Earth was doing this, or at least the life on Earth was making oxygen and these sugars and starches. Initially, there wasn't a lot of oxygen in the Earth's atmosphere. The life had to be going for a long time and one of the things that happens to oxygen, if you're not using it, is compounds like iron just love oxygen and making an iron oxide. And if you add water to that, that's a compound that you don't want around really, quite often, certainly not in your car, it's rust. But there's so much iron in the basalts that make the surface of the Earth, at least on the ocean crust, that for a long time they will soak up all the oxygen that photosynthetic life has to offer. You just rust the Earth. Eventually, if you slow down the rate of the manufacture of ocean crust, if you do that, so that there's less amount of iron being produced per year, slow down the rusting rate. In other words, make a veneer of rust over the surface of the planet. You've now got the opportunity to start accumulating oxygen in the atmosphere. And many people recall this, the Great Oxidation Event. When did you first start seeing free oxygen appear in the Earth? And let's say it's to the order of three billion years, more or less. So the G-O-E, Great Oxidation Event. It wasn't in a day, it was over a long time. But this marked the period of time when free oxygen started appearing in the Earth's atmosphere. The Earth's atmosphere now is nowhere near what it was when we first started the Earth. It looked very different.


I'm with the CNCF since the inception. I've been helping students and people to get certifications. My core interest is allowing people to get certificate, helping the people to train them to certifications. So since then, I was very rooted to this program. I want to contribute more, so ambassadorship helped me to go over there. This community is always unique because it will never leave anybody alone. I was struggling at one point of my life within the technology where I have to move. CNCF literally adopted me, I can say like that. So I feel there is a strong bonding within the community and the users really use that benefit, especially me, I have using that. I start up with a simple contributor of nowhere and I grow up to an ambassador, which is my dream and it's really helped for this benefit.


Hello everyone, good morning and welcome to the class once again. You all have been doing Japanese for a long time now, here in the class, we are already through with 20-22 lectures. So, I am sure now you are all very comfortable with the language, whatever we have done you have learnt and you remember it. All these past days, we have been doing nouns, we have done adjectives, how to use adjectives, how to use negative, how to use it in a negative form. We have also done verbs in masu form, masu, masen, masuta, masen deshita. So, well today we are going to do verbs something different in verbs. Why? Because we need verbs to make complicated sentences. We need verbs to say things in an easier way. But in Japanese, there is a system that you have to use masu form in the end of a sentence. Always masu form comes in the end. So, how will you make complicated sentences with masu form in the middle and masu form in the end? So, that does not happen. What happens is what we are going to do now, a little later. But well, before that as I always do, we will just go over our assignments that I had given you last time and we will see, you can check whether you have done them correctly or not. So, well, the first assignment is, look at the picture and tell what they are doing. This practice is for verbs, so let us see what they are doing. Well, we can name each of them or we can just call them Tanaka san or any other name Rao san. So, teribi wo mimasu. You can see, then, sanduichi wo tabemasu, juice wo nomimasu, nemasu, hon wo yomimasu and in the end, we have ongaku wo kikimasu. So, well, this was practice in present tense of verbs. Now the second assignment is match the words in column A with words in column B. So, you have words written in Hiragana and over here in Roman, you have to match these. This is important again as I have been telling you, Hiragana, you have to learn to write and to read to be able to read the language. So, well, let us see what it is. Ikimasu, kaerimasu, hanashimasu, arimasu, sawarimasu, nomimasu, hashirimasu, kaimasu, waraimasu and yomimasu. So, well, this was Hiragana for you. You can practice the syllables here, they are given very clearly, written very clearly. You can look at them again and again and the stroke order is given in each lesson. So, you can go over the stroke order also and practice it. Now, the assignment 3 was look at the pictures and make sentences using adjectives and kara and give reason for doing the activity which you can see in the picture. So, well, the first one is as you can see nomimasu. So, what is the adjective you are going to give over here? Well, it is oishii kara nomimasu. Miluku wa milku, milku wa gyu-niu wa oishii kara nomimasu. It is very cold, samui kara dokoe mo ikimasen, hamburger, hamburger oishii kara tabemasu. Barano hana, barabara is rose, barano hana wa kirei desu kara kaimasu. Now, okimasu, osoi. There is a different form that you have to use over here. I will just give the form and maybe sometime later explain it to you. Osoku narimashita kara hayaku okimasu, kangaemasu, thinking, kangaemasu, mondai, musukashii, musukashii desu kara kangaemasu. Atsui, it is very, very hot. Atsui desu kara shawa wo abimasu, koutou, looks very nice. So what is the adjective? Let us see, yasui kara kaimasu. So well, this was your assignment over here, assignment 3. We have assignment 4 as well, fill in the blanks with proper verb forms. So you have the English clue over here. You have to use that and see what is going to come in the blank space. So well, mainichi nani wo shimasu ka, kino watashi nani mo tabemasen deshita. Minna de gohan wo tsukurimashou. Over here, you can also use ka if it was shall we. Over here, it is just let's make. So the person who is saying this is not asking for opinion. He is not asking, he is just deciding and saying let us make. So that is why it is tsukurimashou. So the next one is, nichi wo bini kazoku to issho ni ega wo mimashita. Rai nen watashi wa kunie kaerimasu. Rai getsu no mikka ni sensei no uchi ni party ga arimasu. Tomodachi wa atarashii kuruma wo gaimashita. Shokuji wo tabemasho ka. Now over here, you can see the ka is given, thus it is shall. So when you are asking, then shall is to be used. When you are deciding yourself, then ka is not going to be used and it will end at show with the verb. Then getsu youobi wa yasumi deshita was a holiday. So I hope you all got it right. Well, the next assignment is look at the pictures below and fill in the blanks with verbs. So you have these pictures over here, I am sure it is very very easy now. So well, I will just do it very quickly. Tanaka san wa hachi ji ni nemashita is fast asleep. Then Tanaka san to Rao san wa ni jikan hanashimashita, they talked for 2 hours. Then they are in a class over here, sensei wa mainichi nihongo wo oshiemasu. Over here they are playing table tennis. So Rao san wa pinpon no renshu wo shimasu. And we have these people waiting here at the bus stop. Well, minasan wa basutei de basu wo machimashita. Then the last one is you see some people crossing over at the zebra crossing, at the crossing here kousaten. This is called a kousaten, a crossing is called a kousaten. So well, we have michi wo watarimasu. To cross over is watarimasu.  So you cross over, go over a certain thing, then it is watarimasu. Well, this assignment is for you to fill in the blanks with proper particles. We have been doing particles, as you know, we cannot complete a sentence without particles. We need particles to join words and make good sentences. So well, this is what we have to do over here. I will just read it out, watashi wa brown desu. ima nihon no kaisha ni hatarakimasu, tokyo ni sumimasu, sumimasu is to live, uchi wa gokai desu, heya no bango wa gozero ni desu, chiisai desu, ga watashi no uchi wa kirei desu, yokohama wa kaisha kara toui desu, kara watashi wa mainichi, densha de kaisha e ikimasu, densha de ichi-jikan gurai kakarimasu, demo watashi wa daijoubu desu, kaisha ni shokudo ga arimasu, shokudo ni iro iro na kuni no ryori ga arimasu, watashi wa mainichi kaisha no shokudo de gohan o tabimasu, toki doki oishi indo ryori mo tabimasu. So I hope you got your particles right. Now we have a sign 1, 7 for you fill in the blanks with appropriate interrogative words, question words we want over here. So well, let us see what we have in the first one, nihonjin wa nande tabemasu ka, nande is by what means, hako no mikan wa ikura desu ka, how much, watashi tachi wa doko de densha ni norimasu ka, where will we get into the train, tomodachi no tanjoubi ni nani o agimasu ka, agimasu is to give, then kai gishitsu wa nan kai desu ka, so well you will see that nani and nan both mean what, nani is to be used with things where places where you do not have numbers most of the time, so well nan is for kai number of stories, so that is how nan is being used. So now you can see the difference between nan and nani, where it is basically going to be used, meaning is exactly the same. Now, dare to piknik e ikimasu ka, tanjoubi wa itsu desu ka, gakko ni, gakusei wa nan nin imasu ka and again you will see nan is used with numbers, nin is for people, how many people. Dono kaban wa anata no desu ka and in the end we have nekutai wa doko desu ka. You have question words over here, you can see the difference in all of them over here and you can practice. Now match column A with column B, you have kanji characters here and meanings over here in roman, so ikimasu, tsuki is moon, nan nin, how many, hidari te, left hand, suitachi first of the month, kaerimasu is to return, mimasu is to see, otona is an adult, kiyobi is friday and hitori is one person. Now we have done these exercises, we have gone over them and I hope you have checked all of it and will not make the same mistakes again.  There is something very important which we need to do over here which you really require, it is very important when you fill forms and what is that, that is senen-gappi which is your date of birth. So that is what we are going to do over here today, senen-gappi wa nan desu ka? So let us see, Tanaka san no senen-gappi wa nan desu ka? Watashi no senen-gappi wa 1985-3-3-4 desu. Watashi no senen-gappi wa 1985-5 desu, nan-gatsu desu ka, kugatsu desu. So well, you can see over here very clearly how to ask for senen-gappi or your date of birth, someone's date of birth. Now the important part over here is tanjoubi wa itsu desu ka?  Please remember nan desu ka and not itsu desu ka. That is very very important. Now what is this actually? Say means when you are born, your birth, then nen is year, gatsu or gup is month and then p over here is day. So this stands for birth, year, month and date. So in Japanese you have the year first, the month and then the date. You have to keep that in mind. Now how will you practice this? Well, for asking you know, you can put ka in the end. So anata no senen-gappi wa nan desu ka? Senen-gappi wa nan desu ka? Over here for anata as we have always been doing, you know a lot of vocabulary now. You can put whatever you want over here, oto-san, sensei, musume or any other vocabulary that you remember, tanaka-san no senen-gappi wa nan desu ka? Now what is the answer? You can give whatever year, sangatsu, shigatsu, jugatsu and mikka, touka, jugo-nichi, hazuka. You can give any of these dates and say this over here. So watashi no senen-gappi wa 1987 nen, sangatsu-mikka desu. We have practiced date, we have practiced month, we have practiced our numbers here, we have practiced nen. So I am sure you can all use this very freely and ask your friends about their senen-gappi. Now you can practice over here watashi no senen-gappi wa 1988 nen, sangatsu-jurokunichi desu. It is also given in the script over here. This means to be born, this is year, month and day wa 1988 nen, sangatsu-jurokunichi desu. Over here as I just told you, you can practice it with whatever vocabulary you remember, you can replace watashi with any of these over here. Then your date of birth can be said like this, you can ask with ka. So you can practice musume no senen-gappi wa 1990 nen, gogatsu-koko no ka desu. Tomodachi no senen-gappi wa 1986 nen, nigatsu-jurokunichi desu. So you can practice like this also look at these pictures and tell me what is the date of birth. You can name these people, could be Tanaka san, Tanaka san no senen-gappi wa nan desu ka? Tanaka san no senen-gappi wa 1989 nen, shigatsu-jurokunichi desu and this is how you can practice. Over here you have Tanaka san again oba-chan is there, so oba-chan no senen-gappi wa nan desu ka? It is given over here. You can say that out loud. Then we have Mariko san probably, you can call her Mariko san. Mariko san no senen-gappi wa 1972 nen, juu-gatsu-jurokunichi desu and this is Tomoko-chan. So Tomoko-chan no senen-gappi wa 2002 nen, san-gatsu-nichi-nichi desu. This is how you can do your senen-gappi date of birth. It is extremely important because anytime you need to fill a form, that is the first thing that you have to do. Fill in your name, write your name and then your senen-gappi. Now as I told you and promised you earlier in the class, we are going to do verbs. We are going to do verbs in a different way today. Now why we need to do them like this is, first and foremost in Japanese unlike English, you can complete a sentence just using the verb. Just by using the verb, a single word, a single verb, the sentence is complete. For example, tabemasu, mimasu, tabemasu is to eat, mimasu is to see. So just by saying this, it is complete, it is understood, tabemasu because most of the time the subject when you are talking, topic of conversation or your subject is not said, it is left out, it is understood.  So that is one very important thing. Now another important thing about Japanese is that you have done masu form. We have been doing in fact masu form for quite some time now and you all understand masu, masen, mashta and masen deshita. Well in Japanese, masu form, verb in masu form always comes here in the end. You have a sentence and you have the verb in the end. Also masu form can be used only once in a sentence which is in the end. You cannot use masu form over here, verb in masu form like this and then try to complete your sentence and put another masu form over here. That cannot be done. So when you make complicated sentences, when you want to say use two verbs or three verbs in a sentence, what do you do? How will you use verbs? For that, we need verbs in their base form. We need verbs in their plane form or dictionary form as they are called and the dictionary form of the verb or the plane form of the verb is used in between or to make complicated sentences to be able to use, make different difficult sentences and then masu form is used in the end. So, that is what we are going to practice over here today. We are going to do plane form of verbs today in class. Now, as you already know the masu form, you know the verb, you just need to learn the plane form. From the plane form of the verb, all different conjugations are done and made. So, plane form is extremely important for us to do the language to be able to speak in Japanese. Now, you have done masu form ikii, masu, ikii, please tell me, masen, ikii, masu, masu, ikimashita in past tense, watashi wa ikimasu, watashi wa ikimasen, kino wa watashi wa ikimashita and ikimasen deshita in past negative. So all these four forms, masu, masen, mashita and masen deshita will have plane forms. Today we will do plane form for ikimasu, which is iku and all the other verbs as well. This will help you to make better sentences. So, let us see what we have over here. Well, I just told you ikimasu, ikimasen, ikimashita, ikimasen deshita. This you already know, we have done it a number of times, we have been making sentences. Well, you have another one here, suwarimasu, suwarimasen, suwarimashita, suwarimasen deshita. Now we have yomimasu, yomimasen, yomimashita, yomimasen deshita, tabemasu, tabemasen, tabemasu, tabemasen deshita. So, now if you remember, I did tell you in the beginning that verbs are divided into three categories in Japanese, basic three divisions of verbs, group 1, group 1, group 2 and group 3. So, we have verbs ending in I-R-U and E-R-U most of the time in group 2. And verbs in their dictionary form as I told you, in their plain form, in their base form end in I-R-U and E-R-U, they are in group 2. If they end in anything else, then they are in group 1 or all verbs ending in U are in group 1. There are a lot of exceptions in group 1, a lot of exceptions are there. Then there are some verbs ending in I-R-U and E-R-U in group 1 also. We have only two verbs, kuru meaning to come and suru, to do in group 3 because they do not follow the normal pattern of group 1 and group 2. So, these are exceptions and thus they are in group 3. So well, for your practice, I have listed a few verbs over here of group 1, though they are all jumbled, all of them some are exceptions and some are not exceptions. But we will have them here in this pattern, so that it is easy for you to understand. It is easy for you to memorize. In the previous slides that we saw, we did just now, you must have noticed that there are some verbs in group 1 and the last verb tabemasu was in group 2. So well, here it is only group 1, let us try to practice that. You know the verb ikimasu iku, kaerimasu kaeru. aimasu au kaimasu kau nomimasu nomu yomimasu yomu hashirimasu hashiru hanashimasu hanasu kakimasu kaku kikimasu kiku arukimasu aruku. So these are some verbs, of course. From now onwards, we will also do the exceptions for group 1. I will explain them to you on and off in class, not all of them in one class because it is a little confusing and you may not be able to remember them. So over a period of time, we will try to do that and I am sure things will be alright. Now practice verbs in plain form, there are a lot of verbs listed over here with pictures. You can practice yomu, well what is he doing? Writing, kaku, then we have cutting which is kiru. This verb kiru is in group 1 also and in group 2 also. Group 1 is to cut, group 2 is to wear, then hanasu. What is he doing? Having medicine, nomu. And in Japanese, we drink our medicines, we do not eat our medicines. So please remember, in Japanese, we always drink which is nomu. Then we have this lady who is pouring sugar cubes in the cup, in her tea. So well, it is nomu. Now there is more group 1 verbs for you. You have done this picture earlier, swimming, furude or yogu. So now the practice is for you to learn the verbs in plain form. In the base form, in the dictionary form, making something, gohan wo tsukuru, pulling the rope, hiku, chiruna playing, asobu, working or studying, hataraku. Studying is benkyosuru, to work is hataraku. Then meeting, greeting with a hae and what is that, awu, awu is to meet. We have verbs in group 2 as well, let us see, miru, taberu. Then we have someone sleeping over here, neru. What is he doing? Kangaeru, to think. Once you have done okiru to get up and of course, what is this, let us see, oshiru to teach or as he is entering over here, well it could also be hairu which is to enter. Now we have some pictures for group 3 as well, let us see, someone is ringing up, calling up on phone. So well, then wa wo suru, having a party, pati suru, then doing work on computer, so benkyosuru. Then cleaning up the whole place with a hoki, well soji suru and then they are enjoying a party or a picnic, picnic suru. So well, these are all in group 3 special category where you have verb, verb and noun combination. This is a noun verb combination of verbs and as I told you earlier in group 3, we only have two verbs which are kuru and suru as I put it over here for you, kuru and suru. Kimasu and shimasu. You have already done these verbs, kimasu and shimasu. Watashi wa mainichi shigoto o shimasu, watashi wa mainichi kurasu e kimasu. So these two verbs are in group 3 and this verb, this set of group 3 verbs that has been mentioned over here is in a special category which sometimes is also called the group. Four verbs, the noun verb category. Now we have more verbs for you. What are they doing? Well, they are getting something, buying stuff, in packets getting it home, so kaimono suru. Now we have this gentleman entering the room, it is 9 o'clock, krasu ni kuru. So the students are already there, they are saying sensei ga klasu ni kimashita or kuru as is given over here in plain form. Also sensei ga klasu ni hairu, enter a class. Then we have these people playing over here again, well, tennis no renshu wo suru. So with this verb suru, we have the noun over here, renshu wo suru over here and what do we have here? With the dog he is going for a walk, so sampo wo suru. Sampo wo is to take a leisure walk, well, sampo wo suru, he is not doing brisk walking. There is a small practice for you. What is this gentleman doing? Can you tell me? Look at the picture carefully and see what is he doing. Nimotsu wo okuru, well, he is going to send this nimotsu as package or luggage or baggage. Nimotsu wo okuru, okuru is to send or also you could say, nimotsu wo hakobu, hakobu is to transfer something from one place to another. Well, we have another picture for you. This gentleman over here, Tanaka san you can call him is relaxing, is happy, so what is he doing? Uchi de yasumu, is relaxing at home. Then you have this gentleman Tanaka san over here asking this lady for some place or some building. So, well, what is he doing? Jyushou o kiku, jyushou is a dress, jyushou o kiku. So, this is how you can practice with all the other pictures also given over there. You can say it out aloud, ask your partner what they are doing and then probably answer in plain form. Now, Tanaka san wa juusu o nomu, Tanaka san is going to have juice or is having juice. So, well, for Tanaka san you can replace it with sensei, okusan, tomodachi, imoto, I told you earlier also and you can replace juice with jyushou, tabemono, presento, fuku and nomu for any of these verbs over here and I can just tell you these ones. Okusan wa tabemono o okuru or okasan wa tabemono o okuru, okasan is mother, tomodachi wa presento o kau, imoto wa fuku o arao, sensei wa juusho o kiku. So, instead of kikimasu, you can use kiku, okurimasu, okuru, kaimasu, kau, araimasu, arao in plain form. Now, this is one expression we did last time, I hope you remember someone leaves the house in the morning, bright sunshine over here in the morning, well Tanaka san is leaving his house. So, what will he say? What is the expression generally used in Japanese? Well, it is itte kimasu, itte kimasu and then the person who stays back at home, what does the person say? Well, please go and come, itte irasshai. I have already discussed this with you earlier, told you. This is just a simple revision for you to remember your expressions. This is an important expression, which generally you will use in Japan. Now, just the opposite of what we did just now in the previous slide, when you come back home, well what do you say? What you say is, when the person who enters the house announces that I am back, I have returned. So well, the word is Tadai ma, Tadai ma and then you have the person at home saying Okaerina sai. Okaerina sai, so well nice you are. So, well as I always do, we will do kanji practice also and let us see what is the kanji that we have here. Hanasu, you have done the verb hanasu, I will just draw it out for you. It is a combination of two characters over here. This character means, well this is imasu and then we have this character over here which joins this and makes it a single character which is hanasu. You would like to know what this character is, well this is tongue. So, you talk with your tongue and that is how you can remember this is a 13 stroke character. Let me see, 1, 2, 3, 4, 5, 6 and 7, 8, 9, 10 and 11, 12 and 13. So it is a 13 stroke character, speak with your tongue which is to talk hanasu. Once again very very quickly 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 and 13 to talk. Now let us see what do we have next. As I told you it is a 13 stroke character, then we have yomu and you can see one character is similar over here. Now I will make it once again for you over here, let us see 1, 2, 3, 4, 5, 6 and 7. Once you have done 1, ichi, juu, doyobi, then we make something like this and yomu. So once again 1, 2, 3, 4, 5, 6 and 7, 8, 9, 10. This is longer, this is smaller, 11, 12, 13 and 14. It is a 14 stroke character meaning yomu to read. So well, let us see how many strokes are there. Yes, it has 14 strokes. You have hanasu over here, so many strokes to it. You can make it, draw it at home, write it down properly as is given and practice. And as I just told you, we did the strokes for yomu well, I do not have it in the slide over there, you can do it like this, like this, you can make it, it is a 14 stroke character. So, you can get the stroke order correctly from there. Now let us see what Kanji we have for you here. You have done this word daigaku, dai and gaku, dai as in big and gaku like this, gaku as in place of learning. So, big place of learning will be a university. Now dai you already know, this is how dai is to be made, gaku is this is kodomo, child and place where there are lot of lights, a room with lights is somewhere you would sit and read or learn, thus it is place of learning, seat of learning. Big place of learning would be dai, gaku which is a university as is given over here. You can see it is an 8th stroke character, 1, 2, 3, 4, 5, 6, 7 and 8, it is an 8th stroke character. Well, let us see what we have next now, we have higashi, you have done tokyo, if you remember all of you, we have done this character in tokyo, tokyo, tokyo, we have done this character over here, this one, this means east over here, I have this character for you, ichi, ni, san, shi, go, roku, nana and hachi, it is an 8th stroke character meaning higashi, meaning east. We have gaku for you, you can see the stroke order over here and practice it. There is hiragana as I always do, well this is the Ra series, you have these syllables over here, the stroke order is given, please get it correctly and see the shape how it is made, draw it on a graph paper and you will be able to get it correctly. Ra, ri, ri has this longer, it is similar to e, but this part is a little longer, ri, ru, re and we have in the end rou. Now you have all of them on your screen for you, you can see how it is done, how proportionate they are and you can draw it on a graph sheet and see how it comes. Well, this vocabulary is for you, the vocabulary that we did here in the lesson, you can repeat after me, juusho, fuku, kaimono, hairu, okuru, morau, ageru, kangaeru, miseru and you have the meanings over here and the hiragana given. Now as I have been doing in some of our, from our previous lessons, I try to give you a proverb which is very commonly used in Japan and today's proverb is something which we also do as children in school here in India also, I am sure other places as well. This is a common proverb, well you can see lot of squirrels here in this picture and they are trying to collect their food for the winter. Well, the proverb is chiri mo sumoreba yamato naru, chiri is lot of small, small things. If you collect it, it becomes into a mountain, even the smallest of things. The meaning is that if you work together, you can accomplish anything. If you work together, you can do wonders. Alone you may be very small, you may not have the strength or the stamina or the capability to do anything. But if you are united, if you stay together, you work as a team, then you can accomplish anything you can do wonders. So, small squirrels all together when they try to collect food, they can collect lots of food for the winter when it is very, very cold and food is not available. Well, you can use it. Now it is your time, your time begins. You can start your assignments. Now you can start doing them right away. The assignments are match group A with group B and make good sentences. Then you have fill in the blanks with question words. You also have to write hiragana for the words given in this column. Then match the kanjis with words, with the meanings in group B. Then as we did in our previous exercise, I want you to do particles over here. Practice particles, write particles and of course, there is an audio exercise here where you just need to listen to the audio and answer the questions given here. On Sunday, I went to PIKINIKU with my company's friends. We were gathered in front of the station's office. PIKINIKU's location is near the mountain, so we went from Tokyo station to Kyoto by train. We changed from Kyoto station to the bus. We arrived at PIKINIKU's location at 11 o'clock. After that, we rested for about an hour at Soba Park. After 12.30, we had a delicious lunch at the restaurant. After lunch, we walked about 3 km along the mountain road. We took a picture of the mountain. We went home at 10 o'clock. Our PIKINIKU was fun. Of course, answer the questions given here. Well, that is all for today for you. I think it has been very long, lot of new verbs, lot of new words. You have to do your senen gapi, revise your numbers once again, your days of the week, date, month, verbs and lot of other things, lot of new vocabulary for you. I will finish over here today and let us meet tomorrow again. Good night. Thank you very much. I will finish here today. I will meet you again tomorrow. Good night. Thank you very much. Good night. Good night. Good night. Thank you very much.


Boys, I would like to formally apologize. My timing has been off. When I recorded that last pack opening, I thought it was so fired. I recorded in early access, thought it was gonna be a banger, and by the time I got the rest of my videos out and posted it, it was so mediocre, especially because Legends Fantasy Bundle and the Superstars Bundle came out. It reminds me a lot of the only fans video I made. Bro, I turned that video around in 20 hours, and I was cranking on that. And even that video was outdated. So, there's two purposes to today's video. Number one is redemption. I'm opening the eight times Legends Fantasy Bundle and the 18 times Superstars Bundle, obviously to get some coins for Wheel of Mud coming up. And then the other thing was I did wanna gauge your guys' interest in something. I always do Wheel of Mud. Everybody knows I do Wheel of Mud. And last year, the other series I kinda did was pack and play, but it wasn't structured enough. It wasn't consistent enough. I really wanna come up with another amazing banger series that can compliment Wheel of Mud. Wheel of Mud is still my flagship, but I wanna come back to something. I don't wanna just open packs and play a game or randomly hop in and play game play with players unless it's super cracked. Like Barry Sanders drops or an insane Bo Jackson. You get the idea. I'm not doing no money spent. I'm not doing anything generic. This is Papa Beaks we're talking about.    I'll definitely use my own noggin, but I'd love your guys' help.  I also have a package to open. So, this puppy right here, listen. They said, Matt, we're dropping a brand new game and we wanna send you an exclusive limited press kit. The mechs are finally here. Now, the game is called Mech Arena. It's a 5v5 mech shooter. I've gotten a chance to play the early access. It's so sick, but that's not why we're here right now. I'll get to some gameplay later right now. I gotta unbox this. They even got Mech Arena wrapping on this thing. Let's go. First, oh my God, this is nasty.  This is gonna be so sick. Dude, and I know if Plarium is gonna do something, they're gonna do it big, so I know this is gonna be sick.  All right, first one right here. Let's see what this is.  That's nuts. She's doing the ad for me. I love you. Okay, that's insane. I can't believe they made that. Hi, Fumi. You wanna play Mech Arena? I got a Mech Arena propaganda poster. It's got some of the weapons in here. Little Mech sticker. That's going on my laptop for sure. How long am I gonna take it to a 500-piece Mech Arena puzzle? That might have to be a live stream one day. Is this Mech Arena merch? Bro, I even got a Mech Arena token! A literal real-life Mech Arena token. Wait, that's so, hey. I gotta wear that. Hold on, I gotta wear that the rest of this. Why does the Mech Arena merch fit so perfectly? Ooh! Honestly, as you guys can tell, Plarium has poured their hearts and souls into this game. I know it's gonna be sick. You guys definitely gotta check it out. So as you probably know by now, there's definitely a link in the top of the description to play Mech Arena. Definitely click that. Since I'm talking this much about it, I might as well show you some gameplay. So let me get my phone out. We're gonna play a little bit of Mech Arena. I'll show you guys around, and then you can play for yourself. Like I said before, it's a 5v5 Mech shooter. No shots are my favorites. He zooms around the map. You can also customize your Mechs with some awesome skins. I'm definitely on the lookout for red floral necks. That's gonna look so good. All right, let's hop into a game. I'm just gonna do a quick match, but there are a lot of different options. The gameplay is awesome. It's so fluid, and each Mech is gonna have a special ability. Kill Shot has this little booster right here. This is why I like Kill Shot. You can really move around the map. The plasma cannon has some range. I'm about to take out a Mech already. Jeez, this thing is crack. Wait, this is so good. All right, Master Blaster's about to go down. Yup, with the help of my teammates. Hey, let's go. I killed the same dude twice. I was the MVP, but we lost the game. I don't mind taking MVP though. The game just launched. So there's tons of launch stuff in the game right now, so you definitely wanna hop in ASAP. Mech Arena has loads of awesome events as well as a great login rewards program, which you definitely don't wanna miss out on. It is completely free to play on Android and iOS, and you can use my link in the description or scan my QR code to get one low-spec skin, 508 coins, and 70,000 credits to help kickstart your game. And if you're quick, add me as a friend. My username is Papa. Link in the top of the description. Mech Arena, thank you, as always for sponsoring. Enjoy the rest of the video. Why don't I get to my content for you? How there's an eight times legends fantasy bundle? Ed Reid is so good. I know there's an Ed Reid and a Lydani and Tomlinson. And bro, there's 89 overall superstars. St. Claude and Barkley. I want them so bad. I'm hoping today to see a brand new sick animation. That's first and foremost. We're gonna get Josh Allen in the opener and an Aaron Donald power-up. I don't mind if I do. Let's get it boys. Casey Hayward, Xavier Rhodes, and that's the legends animation. I like it. This is already shot from recording some super bangers. If we pull anything good today, it's only gonna get worse. DiMario Davis. Ooh, is that superstar? It is. I wonder if there's another superstar animation because that's a gold superstar. I wonder if there's anything for the juiced stuff. Oh, two superstars. Ooh, in the first round we get Johnny United's. Damn, I gotta be slower, bro.  Whoa, that's how we do it, baby. DeMarcus Ware. Bought to build an entire DeMarcus Ware here. Mike Hilton and I got his Jets LT. I don't know how I feel about that. Richard Rogers. Max Williams, okay. CJ Mosley, Brian Bulaga. That's not the new animation still. 84 straight hand though. I need a boss legend, bro. If I can pick and choose, I think I'm out with Danny Tomlinson. Dude, do not let this pack opening be an L. I'm literally here for redemption. Vaughn Bell. Good start, baby. 84 Vaughn Bell. Wyatt Teller, 84 overall. Russell Wilson. Don't you dare give me a big legend now. Dude, it's always the most depressing. You have to pick between two and you get like an 89. Not she yet, not she yet, not she yet, not she yet. Jimmy Ward, okay. Brandon Marshall, 85. I like that. That's a good start. Aren't there two Brandon Marshalls? Is there a linebacker Brandon Marshall and a wide receiver? I think both of them don't play anymore. Casey Hayward. Clowny, dude. Give me that new animation.  Hunter Henry, superstar. Elon, 86, damn. Wait, so there might not be another animation. That's a bummer, but that's a real good pull. Adam Thielen, he's got Slautomatic and the Wind One. I don't know what the Wind One's called. I totally forgot. It's literally already a better pull than I ever got in my other pack opening. So I'm already on the toppers. That is a bummer, 85 plus though. Let me see a new animation, baby. That's not a new animation. It's gonna be an 85. So I need to see the new animation.  I'm gonna go with the superstars model. I don't know if this is any better. This could be significantly worse for all I know. KJ Wright, Richie Incog. Looks like it's gonna have a lot of stuff in it, although I don't know if I can get that saucy pull. Nothing there. So I think you're guaranteed two superstars. Max and Yikes. What do you got? Quit, hey. Yo, wait, he's got un-fakeable and is that no outsiders or inside stuff? I don't know what that one is. I can't believe I don't know from the logo. I'm kind of bummed. But that is good. I'll take 86 Quinn and Williams. Jerry Tillery, no thank you. Joe Burroughs got one. He's got the long range dead eye. Tua just gets run and gun, okay. Max Williams, Tillery, tough pack. Ooh, double time. I got feel it twice, let's go. Work with me here, baby. Just finesse an 89 overall into my pack. I don't know what the topper is. I think it's 83, 84. I don't think you're gonna pull an 89 superstars player though. Domakin, Sue and Yikes. Malcolm Smith. This is a whole lot of the same. I need a new animation. Bro, I feel bad. Terrell Edmonds. And now we're on to the topper. And I already know what this is. It's like a guaranteed one of the 83, 84s. Miles Sanders. Kind of on a fantasy team. I really need that dude to have a good year. Absolutely praying. And then Ben Jones with. Uh-oh, no. I literally have no idea what that ability is. Leonard Floyd with fearmonger. Our final one is Bulaga. Is that a secure protector? I think that's a secure protector. That's huge. So there's a Quinn and Williams, Miles Jack, Theland, Bosa. I pulled Quinn and Williams and Theland. There's Marlon Humphrey, Vaughn Miller, Michael Thomas and Russ Wilson. I could do this. I could actually get one of these right now. He has swim club, speedster. He has edge threat? Is Vaughn Miller's got edge threat on him, bro? What's Slant Man got? Routtec, mid and elite, short and elite. Oh, he's got short and elite. What are the odds? EA just throwing shade.  Roaming dead eye, quick draw, fearless. Marlon Humphrey's got short route KL, acrobat. Deep route KL.  Damn, bro. I almost want to take a peek at this Vaughn Miller. Let's do it, bro. I want to see this Vaughn Miller. Am I going to get a new pack animation for this? Come on. I'm going to show us the animation. Oh, wait. I was about to be pissed. I thought they recycled animation. Let's see. This is the moment of truth right here.  That was sick. Yo, this card is cracked too.  Speedster, edge threat and swim club. So speedster and swim club don't matter to me at all because that's for user D lines, but edge threat. Edge threat is real nice. 82 speed, 88 tackle, 88 play rec, 87 block shed, 88 power move. Now I spoke a little bit too soon. I think you guys are seeing this today, that Saekwon comes out. So if you're going to do this as well, I probably recommend that Saekwon is really good. In just in case there's a little bit of sauce left in me, I'm going to try a Legends Fantasy Bundle one more time. I don't want to, I don't want to milk mirrors another video out of it, but I was going to open this second Legends Fantasy Bundle anyway. So I'll see if I can get the 89 for you guys. This is the one, baby. Marshawn Ladimore to start us out with Henry Ruggs. Michael Brockers, we got a legend in the second side already. It's 84 Brandon Marshall. I need, need that new legend animation. I'm thinking it's going to be the 86 plus Legends players. Is what's going to give you that. Lewis, Josh, Jacob, scary Terry. That's actually such a good pull, dude. He's a demon. Trey Flowers, DeMarcus, Lawrence and Rivas. David Andrews, a superstar. Max Crosby. I just started out with 83 Lydani and Tomlinson. Bro, Jalen Mills. He got the green goblin act on Instagram and he turned his hair like that. You guys doing something. 84 Von Bell, that's a huge pull. Von Bell, Lydani and Tomlinson, Raheem Mostert, which is an awesome pull. Belaga and, oh, if I had gotten eight, I think 84 LT over Mostert though. I might, dude, I'm actually getting to the point where I can almost do Lydani and Tomlinson's set. Hawkinson, let's go. Huge pull.  Don't think about it. Hawkinsocks, baby. It's got matchup, nightmare and he's an 84 overall. Miles Jack, Montez Sweat. Did that look good? They cannot hold me like this. We got two packs and our toppers got to save us, bro. I just got to get the most fire toppers that exist. J.K. Dobbins, Tua, Dante Jackson and, oh, Kuga. 84 Tomlinson again, damn. All right, I was starting to sell with the legend around Rivas. Oh, I wanted the triple legend. Montez Sweat, I still didn't get it, bro. We're on to the toppers. I better see something new, baby. EA, EA, EA, don't do me like this. Wait, so you got to get the boss legend for the new animation? Wait, so if I get the animation right here, it's a boss legend. Come on, baby. What? Oh, Jesus.  I just had a stroke. I learned a stroke. I had no idea what I just saw. 85 plus, like this is the real one I need to go. Spintactic, baby. Spintactic, yeah, you got it coming. Company, let's go. It's kind of cool. The animation's sick. I got the animation. But 87 Johnny United's is not what I expected. Johnny United's had everybody, bro. 1959? I don't even think my dad was alive. I wasn't even a twinkle in my dad's eye because he was still probably a twinkle in his dad's eye. I'm pretty sure we still had legal racism in 1959. Johnny, I don't want Johnny United's, bro. Give me LaDanie and Tom wants it. You're giving me a guy whose wife literally couldn't vote.  Oh, that's a bummer, dude. Or no, it's not a bummer that they could vote. Well, I guess it kind of is. Ah, it's okay. You know what, I got my animation.  You really are so garbage. Unless they drop something super heat, I'm not doing this again. I need to come up with a creative series that does not involve me giving EA money. I did that last year, I'm not doing it again. I will, when there's crazy cool packs, I will open them, but it's gonna be few and far between. It's not gonna be like last year. All right, boys, that is gonna conclude today's video. I hope you enjoyed. Please give me your input on a dope series you guys wanna see. That'd be so much fun. I love you. I'll see you in the next video. Let's get a voice, peace.


All right, so hello, everyone. Hello, internet people. So, last time we discussed, like, the out of the box algorithm used for optimized cost functions, okay, because you remember the point is that in linear regression, at least in the ordinarily square and ridge regression, we have the chance to be able to write down the solution to the optimization problem we want to solve for the learning explicitly. Okay, so linear algebra gives you the answer directly. But in general, you cannot do that. Okay, it's not, and this is not even if the cost function that you are trying to optimize is convex, you cannot always write down the solution by hand. Actually, the cases that we've seen are especially essentially the only cases where you can. So we need a way to find minima of these complicated cost functions, okay. And by finding minima, I mean, really solving this optimization problem where it ties a high dimensional vector in general.  Okay, so you just look, you start from an initial guess for your parameters. And you are your loop. And at each step in this loop, you compute what we call the velocity, which is proportional to the gradient of the cost function. Okay, the gradient at the position at which you are at time t and by position, I mean, at the value of the parameters that is at time t, t at t. You compute the gradient of the cost function and you evaluate it at this position because when I say position, I'm thinking about a particle moving in a complicated landscape. Okay, and this particle is described by this parameter t at t. So it's a moving particle. So you compute the gradient at its position and you multiply by the learning rate. Okay. And I said that this learning rate is a very important quantity to choose carefully. Okay, because if it's too, too low, when you will update here, the new values of the parameter, which is given by the old value minus this velocity, you see that if the learning rate is too small, you will follow the gradient, but very slowly, okay, so it will have a high computational cost. The algorithm will be extremely slow. But you are guaranteed if the learning rate is small enough to really follow the gradient and converge to at least a local minimum, okay, which is not guaranteed to be the global one, of course. But if it is small enough, you will converge to a local minimum. Now, if you start to increase the learning rate, there is a sweet spot where you will converge as fast as you can to the local minimum and actually to in one step, if you choose that correctly, and at least in the simple case where the function is quadratic, okay. But let's say there is an optimal value for the learning rate which allows to converge fast enough without overshooting, okay, because if the learning rate is too high, you will just update your parameters by a value which is with the amplitude too large, and you will go further than the actual minimum, and you will start to oscillate if not diverge, okay. The learning rate is a very important quantity, okay. Then we've said that this gradient descent algorithm has a number of issues. One is that it's a deterministic algorithm, deterministic in the sense that once you have chosen the initial condition, the initial point from which you start the gradient descent, there is no stochasticity, you will always converge to the same minimum, to the same local minimum, okay. So this algorithm is very sensitive to the initial condition. Furthermore, it is actually very costly to compute this kind of gradient, okay, because this cost function again is here, it's a sum over all samples in the training data, okay, and typically you have maybe thousands or millions of training samples, and for each of these samples in this sum here, you have to compute like p partial derivatives, okay. So the complexity scales as p times n times the number of iterations of the algorithm, and p times n is a huge number in the high-dimensional applications that we have in mind, okay. So this is not very practical, and at the same time, again it suffers from this weakness, which is that if you start from closely or let's say if you start and around you, there is a very small local minimum, you will just stick there, okay. So to face all these problems at once, we introduced stochastic gradient descent, okay, and the idea is stochastic gradient descent is exactly, it's not so original, so fancy, it's just to do gradient descent, but each time you want to compute a gradient, okay, you will not compute the gradient on the whole data, but on the mini batch, okay, a mini batch is a subset of samples in the data, which is typically a very small subset compared to the size of the data, okay, and because these mini batches are chosen randomly, okay, this introduce a notion of noise, a notion of stochasticity, okay. So from one run to the other, even from the same initial condition, SGD will give you two different solutions, okay, once it converged, okay. Yeah, so the only difference is here that you compute the gradient on a mini batch, okay, that is a noisy approximation of the true gradient, okay, and by when I say true gradient, we have to be careful also here, because there is so the true gradients would be actually what, when we say true, what does it mean? If we had access to an infinite, let me divide here by n to create an average, okay, an empirical average, you see that this sum here over samples, if we had access to infinitely many samples, you see that this empirical average would become the statistical average, it would become the expectation with respect to the distribution of data, right. So when we say true gradient, it would mean actually finding the minimum of this expected cost, okay, by expected really the expectation with respect to the distribution of data, okay. So the expectation of the cost sometimes is called also the risk, again, when I told you, when we say an expectation with respect to the data of some quantity, we often call that the risk, okay, could be an error, and here the cost is an error, okay, you are trying to minimize an error between the predicted labels and the ones that are in the data. The expectation of the cost is the risk, the full expectation of the cost is the risk. So what we would like to do in reality is to minimize this risk, of course, we don't have access to infinitely many data, so there is a first level of approximation, which is that we are approximating the risk by this empirical average, okay, this is a finite size approximation of the risk, and this is what we call the cost, okay, you understand. All right, so this is the first level of approximation. So even if we could compute really the argument of this sum over all samples in the training data of this cost, this would still be an approximation because we have access to finitely many data, which leads to variance in the bias variance tradeoff and so on and so forth. All right, but then there is a second level of approximation in SGD is that we are not even computing the risk, sorry, the gradient of the, with respect to the whole data, we are computing with respect to a subset of the data, okay? You see the two levels of approximation, and here when I say this is a noisy approximation of the true gradient, true, you can interpret it really as the risk or as the gradient with respect to the full data, okay? This is an approximation of these two quantities, where one is the approximation of the other already, okay? All right, and how to imagine what is going on with SGD is you see that now that the cost itself is random, it's stochastic quantity due to the stochastic nature of the mini batch, the landscape itself in which we are moving is random, okay? At each time you compute a gradient, you are computing a gradient in a different landscape, okay? And hopefully these fluctuations of the landscape itself will allow the dynamic to converge to a minimum, which makes sense, okay? Which generalize well. All right, okay, I would like to continue from there, and I want to discuss, so just so that you see where we are in the notes, I want to discuss this, the momentum, okay? So, okay, what does mean momentum? So, this is yet another version of gradient descent, okay? Which now looks like this, so let me write stochastic gradient descent with momentum. And this algorithm is actually what really people use, even in very advanced applications, okay? And this is actually the most efficient, okay? In the sense that this is the algorithm that empirically seems to lead to the best minima in the sense of the minima that lead to good prediction performance, okay? All right, so, and here I put the stochastic in a parenthesis because this idea of momentum that we will discuss now, you can apply it or to the plain gradient descent, or you can also use it in the stochastic gradient descent, okay? So, the notion of stochasticity here just means if you keep the stochastic, it means that you are considering mini batches. If you don't keep the stochastic, you are considering the full gradient, okay? With respect to the whole data. Momentum, you can apply it in both cases. All right, so the algorithm looks like this. Again, you have a four, okay, over the time. And here I will write the dynamics like if it was just a plain gradient descent, okay? But again, you can do exactly the same for SGD by introducing mini batches, okay? So, the velocity, so you have of course an initial condition. So, you put the velocity and then you have the update, which is as before. Okay, so again, if you want to make this stochastic, which usually you want to, you just replace this by a gradient with respect to a cost evaluated on a mini batch, okay? You remember, okay? That's the only difference. This makes it stochastic. Okay, so what is the difference with before? The only difference is this term here, right? See now that the velocity is not anymore just related to the gradient. But it's also, there is a term here proportional to the velocity at the previous time step, okay? So as before, the theta is the learning rate while this gamma is called the momentum. So why is that called the momentum? This is what we'll try to understand. So the momentum, this vocabulary comes from physics, of course, okay? So the momentum of a moving object, okay? It's the resistance to change its velocity, to change its direction, right? Okay, so related to inertia, okay? And you need to have mass, to have inertia, okay? Sorry, the inertia is the resistance to changing your direction and the momentum is essentially your velocity, okay? Times your mass, okay? All right, so let's try to understand the analogy with physics. Because again, I think a natural way, at least for physicists, to visualize this type of dynamics, this gradient descent dynamics is really as a particle moving in a landscape. And now we will give a mass to this particle and therefore it would have inertia, okay? All right, so let's understand that. So the way to understand the link with physics here is to first realize that this dynamics here, you can write it in an equivalent different way, okay? Which is like this. So this is completely equivalent to this dynamics. So you will see in the notes, by the way, and I hope you do, if you try to read the notes as well, okay? What I do is essentially making them a bit more alive, but I'm really closely following the notes. What do I want to say? Okay, I don't know what I'm going to say. It will come back in one minute. Where t plus one is by definition is delta t plus one minus delta t, okay? Yeah, what I wanted to say is if you look at the notes, you will realize that everywhere the learning rate, okay, they write it as a function of time, okay? They put an index t here. Nothing prevents you to let the learning rate be a function of time, of iterations, okay? This is called the scheduling, okay? You can schedule the learning rate, typically to decrease as time goes on, okay? And if the learning rate decreases, it means that you are reducing the velocity of your particle, okay? So if the learning rate is high at the beginning of the dynamics, it means that you are exploring a larger, larger regions of the landscape, okay? And maybe at some point, your dynamics is kind of trapped in a minima, but at a large scale, okay? And then if you reduce the learning rate, you will start, it's like reducing the temperature in physics. You start to explore more fine grained landscape. You are exploring then the intra landscape of this sub-minimum, okay? There may be many minima, many large minima at large scales, and then as you reduce temperature, you are focusing on, you are zooming on looking at sub-region like this, then a further sub-region until you converge, okay? You see? Maybe at high learning rates, sorry, let's see, I don't know, maybe at high learning rates, you will explore this, you know, you have kind of two minima like this, you will explore both, and then maybe there is one that you will favor, and then you reduce the temperature, the learning rate, so you are now zooming this region, and you further reduce the learning rate, and you get trapped in finer and finer minima like this, okay? It's like reducing the temperature, okay? But I will keep it constant, okay? And anyway, if you take it small enough, it's fine. All right, so let's see the equivalence between these two dynamics, and why do I want to show this equivalence? Because we will see that this second equivalent writing of this dynamic will connect very nicely with physics, okay? So how can you see this mapping? The way I see it is by just checking, okay? So let's see, so from, let's say, the set of equations one, and this is the way we call the set of equations two, so one implies what that delta t plus one, which by definition is here, is below, okay, is theta t plus one minus theta t, gives us what? We look at this update here, this is minus v of t, okay? And we look at the definition of v of t, the first update rule, okay? And this gives us minus gamma v t minus one minus eta gradients of the cost, okay? All right, okay. So now if I look at the second equivalent representation, tells us that delta t plus one is equal to gamma delta theta at t, which is what? Which is by definition t minus t minus one minus eta gradients of the cost at theta t, okay? I did nothing, I just rewrote this second part here, this delta using its definition, okay? All right, so how to see that the two set of equations are equivalent? We see that they have in common the second term here and here. So to see the equivalence, we need to check, is it true that this first term is equivalent to this second term, which means is it true that theta t minus theta t minus one is equal to minus v t minus one, right? What we want to check. And this is exactly what is what we have here, right? With a time index reduced by one unit, okay? So this is true. So the two set of equations are indeed equivalent, okay?  Nothing fancy. Okay, all right. So we look at this second dynamics, which is now written in terms of the increments of the parameter values instead of the actual values of the parameters, okay? And let's see how to connect that to physics. Okay, so parenthesis, some physics. Let me write some equation of motion, okay? Should I write this with theta? Okay, so let's say we have a particle described by its position, okay? And its position will be, as you can imagine, called theta, okay? And here is an energy function, okay? So what does represent this equation of motion? Which term? This? This is a drag. This is a viscous term, right? So this is a term that is opposed to the velocity vector, right? So it's something that tries to reduce your velocity. So it's viscosity, right? This is the velocity vector here, okay? Here, this is an energy term, okay? We have the gradient of energy potential. So this is a force, okay? This is a conservative force. G derives from the gradient of the potential energy, okay? Energy potential, okay? Again, the theta, this is the position of the particle, okay? And this term is what? Is the inertia, right? So it's the term proportional to the acceleration, okay? So this Newton's equation, okay? Or equation of motion represents the particle with a mass, m. This is the mass, okay? Who is in a viscous fluid with viscosity constant mu, okay? And at the same time, experiences a potential energy, okay? So a force derived from an energy potential, which is here, okay? All right? And I claim that this equation of motion is the continuous version, okay? Of this discrete equation of motion, okay? Of this gradient descent with momentum. But you see gradient descent is an algorithm. You have discrete steps, you iterate discrete steps. But I claim that if you take the discrete steps to be epsilon close, okay? Very small discrete steps. This gradient descent has the same dynamics as this particle, okay? In a p-dimensional space, okay? So let's see. Let's discretize. We discretize this equation of motion. What is the discrete version of the time derivative here in the viscous term? It's just theta t plus delta t minus delta t, okay? Over delta t, right? You remember what is the definition of a derivative, okay? The limit when epsilon goes or let me use delta, delta t goes to zero of this object here, right? Okay? This is the mathematical definition of a derivative, right? Now we're not taking delta t really going to zero. We just take it small enough, okay? All right? The last term, the energy term is unchanged. And the second derivative is what it gives. You just apply twice the first derivative and it gives theta t plus delta t minus two delta t, theta t plus theta t minus delta t over delta t squared, okay? And there is the mass that I forgot. Okay? This is the discretized version of power equation of motion, right? So now let's compare that with the equation we had. Let me paste it. How do we identify now? Sorry, the identification is not direct. We need to work a bit more. That's why I was... Now I claim, and this is an exercise. This is like a two minute exercise. If you take this equation here, this discretized equation, okay? And you isolate on the left the delta, so sorry, now you define, you define as before delta t plus one to be delta, okay? By definition. You plug that in the equation here. You write everything in terms of these differences here and you isolate the delta t plus one on the left. What you obtain is this equation to check. It's really one line, but you should do it. Sorry, sometimes I put the time index on the top or it doesn't change anything. You have to get used to this kind of small mistake here. Okay? So this, what I wrote here is this. Just return as a function of these time differences here. Okay? And now let's compare with what we have here. So what is the mapping? If I map, if I define this eta to be delta t squared over m plus mu delta t and the gamma, this momentum to be m over m plus mu delta t, I have the same equations and the cost here is my energy function. Right? And so that's why we call this parameter in the gradient descent with momentum. We call it the momentum because you see that indeed it's linear in the mass. Okay? This is what leads to momentum. The momentum is mass times velocity, right? Okay? And you see indeed also that the learning rate is proportional to the time increment to the square, but it's related to the time increment. Okay? This controls the speed of the dynamics. Okay? All right. So now we have our equivalence between our physical model, our particle moving distance k and this gradient descent dynamics. Okay. Let me just mention that there is a last variant. Okay? Which is called, what's the name again here? Nesterov accelerated gradient. Accelerated gradient or nag, which is the same as gradient descent with momentum, except that the only difference, so as before you have this momentum term, but the difference is that you evaluate the cost, sorry, the gradient of the cost, not at your actual position, but then there is a typo in the notes. There is a plus, it should be a minus. And this is, if you want the expected position, if you were not updating, you know, if you were not updating your position according to the rule below here, if you were just following your momentum at this actual time, you would end up after a unit of time at this position here. Okay? This is your actual position plus a time increment along your previous velocity. Okay? Along your previous direction, right? Again, the minus here is a convention. We call velocity minus the velocity. Okay? So you are not updating the gradient, you are not computing the gradient at the position where you are at time t, but you are looking at the position where you would be if you were continuing your actual dynamics for an additional unit of time and you compute the gradient at this position. Okay? All right. So this is just a small variant. Both are working very well in practice. Honestly, I don't see a difference between one or the other, but now you know it exists. Okay? All right. So let's see how these things work by playing a bit with the notebook number two on gradient descent. Okay? Let me stop sharing. Is there any question here from anyone on these things? There is a question. Why do we call it momentum? I think I answered the question, right? Firas is asking that and you updated your question right now, but I think I answered precisely the question Firas, no? Can you explain again NGG? So N-A-G, Nesterov Accelerated Gradient. The only difference is here with respect to gradient with momentum. Sorry? So the only difference with respect to gradient with momentum is that instead of when you update the gradient, instead of computing the gradient at the actual position, at the actual value of your parameters, you will compute the gradient at the value that would take the parameters if you were following your actual dynamics. So your actual velocity vector for an additional unit of time. Okay? So you look at the point where you would be if you were continuing according to your actual velocity for an additional unit of time and you compute the gradient there and you will update according to the value of the gradient at this position, not at the one where you are right now. Sorry, but how can I see that from the formula of the equation? You see the, okay, let me share that again. So he's asking how can we see that from the equations? So you see, so let's look again at the gradient the gradient descent with momentum, which is there. You see that here the gradient is evaluated at your present position at the actual value at time t of the parameters. Okay? You evaluate the gradient, you think of a particle in a landscape, you look at the gradient at the position of the particle at time t, which is here. Okay? And then you update the position at time t plus one according to this rule, which is the old value minus this velocity vector that has been computing according to the rule above where theta t appears. Okay? But the gradient has been computed at your actual position. While in a state of accelerated gradient descent, the only difference, the only difference is here. You see, what is this position? Again, if I think of a particle, this position is the position where you would end if you were continuing your dynamics by an additional unit of time. You see that your velocity vector at time t minus one would lead you at your actual position at time t minus this thing, right? And you are computing the gradient at this position instead of just theta t. Is it clear? Maybe you have to answer something, so yes or no? Or maybe not completely, but I just answered your question. Is it clear? Yes, it is clear now. Okay, perfect. Maybe this is the vector. You have to imagine that in dimension p. Maybe this is your theta t. And if you were continuing your dynamics by a small increment according to this velocity, you would end up maybe, let's say this is minus gamma vt minus one, and you would compute the gradient at this position instead. And this is in NAG. You compute the gradient here. While in the plane gradient descent with momentum, you compute the gradient here. So you see there is a small difference. Okay. All right. So this is the notebook number two on gradient descent algorithms. Okay. So as usual, I will not go over the details. This is your job. It is these things are self-contained. Okay. You just have to understand and run the examples by yourself. So let's see. So what we want to do is to visualize a bit this dynamics. Okay. Of course, we cannot visualize things in high dimensional spaces. There is no way to project complicated dynamics. I mean, you would lose, you would have to project in low dimensions. So what we'll do is directly set up problems in low dimensions. And by low dimension, we, I mean two dimensional spaces, so surfaces, okay. Which mathematically concretely means functions of two variables x and y. Okay. And the cost, if you want, the height, sorry, the cost function is the height. Okay. It's a variable z. Okay. And this z, this height depends on the position x and y. Okay. So we have a particle moving on a two dimensional landscape. And we are measuring the cost by the height, which is a function of x and y. And we want to visualize essentially how this gradient dynamics really look like. And to play a bit with maybe to see if we add some noise in the dynamics, what changes. If we change the momentum, we change the learning rate, how the trajectories evolve. Okay, to really get a concrete visualization of what is going on. Of course, in again, in very high dimension, where you are not moving in the surface, but in a 10,000 dimensional space. Okay, things are different. Of course, but still we can gain some intuition of what is going on two dimensions. Okay. Okay. So this first cell is just setting up functions for visualization. Okay. So I will not go through. These are just, and I didn't code them actually. And this will define, these are routines that define the surfaces in which we want the particles to move in. Okay. So one is called monkey saddle, et cetera. And these are typical two dimensional surfaces in which this kind of dynamics are tested. Okay. So here is a cubic function in x and quadratic in y. You have another one. So each time we define the surface, which means it outputs the height, the z as a function of x and y. Okay. And we also have a routine which computes the gradient of this function. Okay. The two dimensional gradient. Okay. So you see, for example, that this monkey saddle, which is defined in some way, we have a term which is cubic in x and indeed in the gradient with respect to x, we have three times x squared that appears. And here, the gradient with respect to x is just three times y squared, which is indeed appearing here. Okay. These are just computing ranges. This is the same for other surfaces. Okay. Okay. So here is a visualization of the surfaces. So this is, I think, the so-called monkey saddle. And why is it called saddle, monkey saddle? Because here we see a saddle point. Okay. In what is a saddle point, it's a point which is nor a minima, nor a maximites minimum according to some direction and it's maximum according to some others. Okay. I really don't know. I really don't know. And so just one side remark is that actually in high dimensions, very high dimensional spaces, the majority and the vast majority of extrema, okay, extrema include minima, maxima and saddle are actually saddle points. Okay. So the typical extremum in high dimension is a saddle point. Okay. This is also a saddle point in the middle because along that direction it's minimum and along that one it's maximum, but less rich, you see. Okay. And this one is, here is hard to plot in 2D like this. So we plot the contour curves. Okay. That's the way to visualize. So you know what are contour curves. So one line here means if I follow one line of a certain color, it means the line along which the height is constant. Okay. So we see that here the minimum is here. Here we have a valet. Here we have another valet with a local minima, etc. Actually this function, it's hard to see, but this is actually the case. This function here is also convex. Okay. All right. And we will test the algorithms that we discussed. So gradient descent, the plain gradient descent. Okay. So the simplest update rule. Then we will test gradient descent with momentum. Okay. And this Nasterov accelerated gradient where the only difference you see is that the gradient is computed at the position you would be if you were not updating the dynamics. Okay. After one unit of step. So here there is no stochastic gradient descent or anything. Okay. We are exactly computing the gradient. We have a two dimensional function. Okay. It's slow dimensional. So each time we will truly compute the gradient of the energy of the cost. Okay. No stochasticity here. No need. Okay. All right. So let's see. So here I'm just coding the different dynamics. Okay. This is the gradient descent. GD. These are just initializations. And you see, so here there is a notion of noise that I didn't. Maybe I mentioned it quickly when you add noise. Sometimes we call that long van dynamics. Okay. We add a Gaussian noise to the gradient, which is one notion of stochasticity. Okay. But maybe you remember what I said is that in SGD the noise induced by the mini batches is of a different nature. It's not the same as just adding Gaussian noise. Okay. But still we will emulate a bit what's going on with noise by defining a noise here, which is just a Gaussian noise with a variance even by this noise strength. And that will be added here to the velocity. Okay. And also multiply by the learning rate. So here is a step that computes the gradient. Okay. You see we have a grad. This function will take as input a function, which will be the gradient of the actual function we're trying to minimize. Okay. You know, in Python, you can feed a function with a function. So this function grad will be the gradient of the cost that we try to minimize or the surface that we are trying to minimize. Okay. And here is the update rule. Okay. And here is the same for gradient with momentum. The learning parameter in both cases called eta. We have a number of steps. So here there is no really notion of epochs because there are no mini batches, but we still call the number of iterations, number of epochs. And the gamma is the gamma is the momentum. Okay. And here we have the Nesterov-excited gradient where the only difference is the position at which you compute the gradient. So you see the gradient here will be evaluated not at the actual value of the parameters, which would be params, but instead at the actual value minus gamma times the previous velocity. That's the only difference. Okay. So let's see. Now I will... Okay. So here the first case I will do is actually an even simpler shape than what we looked before. It will be just an ellipsoid. Okay. So parabola, which is not... Which has a different... How do you call this constant? Century city along the two directions. Okay. So for example, if I take a perfect parabola with the same century city one and one, okay. And I took learning rate to be 0.1. I don't have noise and I'm computing the plain gradient descent okay. We see something which is not fancy at all. We follow exactly the gradient. Okay. And we are reaching the minimum. Okay. Now if I put a bit of asymmetry, we see... So you see this... Each time here it's not clear from the plot, but this is actually the case. Each of this line should be... Here it's an interpolation. So it should be at least almost orthogonal to the contour curves, which means that you are following the gradient and indeed it looks like. Okay. Right. If I was taking a very small learning rate, okay. We don't even see anymore, but here now the lines are perfectly, almost perfectly orthogonal to the contour curves. Okay. Let's put a bit of noise in variance one. Okay. So you see that the first steps where the gradient has a strong amplitude. Okay. The noise does not affect much the trajectory, but when you are close to the minimum, the gradient has a small amplitude. So the iteration, the value that will compute there will be dominated by the noise. Right. Because the gradient is almost zero. You are close to the... The clean gradient without noise is almost zero. You are at the minimum. So the only source of change of gradients here is the noise that we are adding artificially. And we see indeed that we are fluctuating around the minimum. Right. But this can be useful if you had a very fine grain structure close to the minimum. Here you would explore what is going on around this minimum. See if you put too much noise. Start to be complete mess. Okay. You are not following the gradient anymore at all. Okay. Okay. Now, let's make things a bit more interesting. So what is the next year? Okay. So now I'm still in this very simple parabola. Okay. But I will run the three different dynamics. Okay. Gradient descent black, gradient descent with momentum in pink or purple and the NHG in blue. Okay. And I'm starting from different initial points. Okay. The function is convex. So anyway, they all should converge to the minimum here. Right. But let's see how things change. So I have 4000 epochs. Let's put a bit of noise. So both for NHG and gradient descent with momentum, the momentum is 0.9. Okay. It's an arbitrary bit value. Let's just check something like a sanity check. If we have no momentum, the free dynamics should be similar. Right. Should be the same actually. Let's check. There is no noise. Okay. Let's increase a bit the number of iterations. Otherwise, or let's increase the learning rate because you see the learning rate was small. It was very slow. After 4000 steps, we are far from the minimum. Right. So here I increase the learning step. All these dynamics are plain gradient descent. Okay. And you see if I increase too much the learning rate, which is already small, but let's say 2 to the power of 10 minus 3. Okay. It's still here. It's converging. We're fine. Let's make it higher. Let's see. Okay. No problem. Let's be a bit more extreme. Let's see. I don't know when it starts to be problematic. It's okay. It still converges. Now it diverges completely. So you see now we have a problem. So let me put 0.1 still converges, but you see suddenly if I increase a bit, let's say 0.5, it starts to oscillate completely. Okay. So here you are overshooting and you would have an infinite oscillation. Okay. You are at a regime where you would go from one point on the surface to the next. Next you are bouncing like this. Okay. You will never converge, but you see if I increase a bit more, it explodes. It diverges. Okay. I have numerical issues. They are just going out from the surface. All right. So now let's play with the momentum and see what happens. So let's put something horizontal, 0.01, let's say. Now I'm adding momentum. So the three dynamics are different now. Okay. We start to see interesting things. We see something that we would, for example, with the gradient with momentum or the NAD, we see something that we would expect from a rolling ball in a sink. Okay. Now you see inertia entering into the game. Right. So you see that here, for example, let's say at this point of the dynamics for gradient with momentum here, the gradient is not pointing in that direction. The gradient is pointing down the minimum. Yet you will continue along that direction due to inertia. Right. So it's really heavy. It's really like you have a ball in a sink like this. Okay. The ball will not follow the gradient. It will follow the gradient at some part. Okay. You have an energy function that you have a potential that leads you to the minimum, the gravitational energy, but at the same time you have inertia. So you follow this. Okay. If you increase too much the momentum, what happens? What do you expect? It should. Right. And it never converges. Okay. It becomes a mess. Okay. So you see this nice trajectories like this. Okay. And here the viscosity is played like for a ball in a sink. The viscosity would be played the equivalent role is the friction with the medium. Okay. The ball has some friction. So you lose some energy and that's why at some point you converge. Okay. And here the viscosity is like the friction. Okay. That's why at some point you converge. But if the viscosity is not enough, it will just oscillate. Okay. All right. So you should play a bit with this. And now let's see the next curves. Okay. This is more interesting. This is now this complicated convex function. This so-called bill function. I think it's called like this. It does not really matter. It's just a rather complicated function. And you see that I start the dynamics from four different initial conditions. From this corner, from this corner here, someone somewhere down here, and somewhere here. Okay. And I'm looking the three dynamics for each of these four initial conditions. Okay. So let's see. So you see now that the learning rate eta is much smaller because the function is much more complex actually. And you see the difference. Why do we need a much smaller learning rate? Because you remember the learning rate is essentially dependent on the curvature of the function. Here on this simple parabola, the curvature is approximately the same everywhere. Right. The function is really homogeneous. Okay. Well, here you see that you have huge differences between what is going on here in the middle. It's very flat. You have very flat directions everywhere. Okay. You are at the bottom. Well, here the curvature is very strong. Right. And you need the curvature, sorry, the learning rate to essentially be able to deal with the steepest directions. Okay. Because if the learning rate is too high, when you are in a steep direction, you will overshoot and start to oscillate. But the problem is that if you have a small learning rate and you start to be close to a bottom where it's flat, it becomes very slow. But that's life. Okay. That's life. But hopefully with momentum, that's exactly the point of momentum, even if the gradient is small and the learning rate is small. And therefore the dynamic should be slow when you are in a bottom where it's flat. Because of momentum, you keep velocity. You see, if the ball accelerates, if you drop a ball in a sink, it will accelerate when it's steep. And even if you arrive in a flat region, due to inertia, okay, due to your momentum, you will continue. Okay. Even if it's flat, well, if you are just following the gradient, you will stop abruptly and then be super slow. You understand the difference? And indeed here we see that, for example, this NAG is accelerating here. And you see that it climbs up the wall due to momentum. Okay. Which a priori is bad. In this case, you would like to reach directly the minimum, but that's life.  Okay. While gradient descent here is directly converging to the minimum. But here we are lucky because we gradient descent is the best, but that's like luck. We initialized the dynamics very close to the minimum. In this case, of course, gradient descent is good. Okay. But this never happens. Okay. And you see this kind of wiggly trajectories here that are following, that are balanced between following the gradient and following your inertia. Okay. So let's see again what happened, but you see the learning rate is very small. And we are doing many steps. Okay. 50,000 in this case. See if the learning rate is 10 times bigger, but still extremely small. What happens? It diverges. Here I have problems. Okay. It means that the dynamics just went out of the curves. Okay. Everything exploded. Okay. So it's super sensitive to learning rate. Let's decrease again the learning rate and put more momentum to see what happens. Okay. So you see all that is very sensitive to the parameters you are using. Okay. So too much momentum means that you see, for example, I don't even know where it started this trajectories, but it started somewhere where there was some stiffness. Okay. It took, let's say this one here. It took some velocity. And because the inertia was so large that even if when you arrived in flat regions like this, you didn't reduce your speed at all and you continued along that trajectory until you reach a point where the gradient is strong enough, which means the gravitational energy physical terms is strong enough so that it competes with the inertia and it brings you back in the direction of the gradient. Okay. But you really need to reach first the top of the hill where the potential energy starts to compete with the inertia because your inertia is very strong. Okay.  Okay. So you have a very subtle competition between the momentum, the learning rate and all that. Okay. So the end point of the discussion is that it's highly non-trivial to select these parameters in such problems even more in high dimensions. Okay. So you have to play with these parameters and how do you select parameters in machine learning? Would you cross-validation? Okay. So you select a set of parameters. You train your algorithm on your data. You see how it works and to see how it works, it means obtaining a prediction error. You test your algorithm on test data. You evaluate some prediction error, gives you a number and then you repeat for another set of parameters in this case of different learning rates, momentum, whatever you want until you reach something which is good enough. Okay. So I really advise you to play a bit with this code. I mean, I think you gain a lot of intuition out of it. Okay. So I wanted to start logistic regression, but I won't have time. So tomorrow we'll do logistic creation, which is a chapter, I don't know, six. I think so it's the last chapter we want to do. Okay. So logistic regression will be, so where is it? Let me stop sharing. Just a small introduction to what we do tomorrow. Yes. So the question is how do we know which gradient descent method is the best for a given problem? There is no magic recipe. So maybe if I would be, I mean, people that play with these things all day long and, you know, that are real numerations, I'm sure they would give you tips. I'm not that type of person. So I don't really know. The only recipe that is systematic to answer that is cause validation. So because there is no theorem, I mean, in very special settings, there are theorems. Okay. But in general, in real applications, of course, they do not apply. So the only way to know which method is the best is to do cross validation, which means essentially testing different methods, scanning over large enough ranges of parameters of hyperparameters values. Hyperparameters means all these parameters you need to fix in the algorithm, learning rate, momentum, whatever, or number of features that you are using. Which means the complexity of the hypothesis class. You have to test different settings and select the one that leads to the best prediction error. That's the only systematic way to do that, to answer that. Okay, there is no magic. Okay. Yes. Yes. It's empirically, me it seems not to be the case. I don't know. And actually, if you read a bit more, they say that in most application, people use just stochastic gradient with momentum. And I cannot really tell you why energy is better in some cases than gradient descent with momentum or the other way around in other cases. I don't know. And I don't think there is definite answers in general. Okay. But honestly, the two are very comparable. If you try to be, if you play a bit with the code, you will see that there are no big differences. Okay. But practically in reality, people in 99% of the cases, people use stochastic gradient with momentum. Okay. Especially in deep learning and all that. This is what works the best. Okay. Other questions. Yeah. So here there is a whole part in the notes. If you are, if you want to read that, I think that's useful. It's methods that try to emulate if you want second order information in the sense that these are methods slightly more complex that I call the RMS prop or Adam, it's other optimizers that are, that have additional terms like this that are mimicking what you would do if you were computing the Sian. Okay. But not really. So yeah, but the idea is basically the same. You follow gradients, but there are additional tricks here that allow you to improve in certain cases. But again, in general, what leads the best performance is what we discussed. Okay. In certain special cases, Adam and RMS prop, this more complicated dynamics here, lead to some improvements, but in general, the out of the box solver is stochastic gradient with momentum. Okay. But if you want to have a lead, to have a read, I think it's useful to know that it exists. Okay. I will not discuss it. Okay. So this is what we discussed today. Yeah, maybe just the final words that are in the notes, which are useful that, but I already mentioned along the course, but let me recap. One thing you have always to do is randomize the data. Okay. When you create mini batches in SGD, each time you start a new epoch, you have to randomize the data to create new random mini batches. Okay. This is how you induce stochastic in the problem. Okay. Transform your input means what it means something that I mentioned at some point, but which you should always keep in mind before processing data in machine learning, you have to normalize the data to standardize and normalize the data. Okay. Because you remember, if you are considering data, let's say you have a number of features. Okay. You are putting all these features into a big design matrix, this matrix X, each column is a different feature. And this feature can represent totally different quantities. Okay. Maybe some feature is the concentration of a very, of a chemical. Okay. So it's very small units. Okay. Maybe 10 minus 6 as an order of magnitude. These are the typical entries of this feature. And maybe the application you are considering another feature is something related to the scale, the age of the universe. Okay. So maybe billions are comparing one feature, which is 10 minus 6 with something which is 10 power nine. So of course, if you process this data without changing anything, naturally, the algorithm will always give more weight to features which have a higher amplitude. Okay. But this is an artifact. What you should need is to standardize your data, which means to rescale each feature so that the variance along one column is one. You are rescaling the values such that you don't have an a priori bias for one feature or another one. Okay. There are the same orders, all the numbers that are in this matrix. Okay. And here monitor the out of sample error is just saying that whatever optimization procedure you want to do and whatever optimization of hyper parameters, whatever parameters you need to fix, learning rate, momentum, number of features. In any case, what matters at the end is the prediction error. So to make choices in machine learning, you should make choices according to what leads the best prediction error. Okay. And also here they mentioned something. When you are optimizing your cost function, okay, you should actually online plot the prediction error, which means that maybe you iterate your learning procedure for 10 steps, you should compute the prediction error on a certain test data of the model that you got after 10 steps of learning. Okay. And maybe you iterate for 10 more epochs, 10 more steps. Okay. And you compute again the prediction error and you do that as time goes on, because what appears is that it's not always good to really find parameters that minimize the training error. Because you remember sometimes when you minimize the training error, what you do actually at some point is that you overfit. Okay. And overfitting means a large difference between the training and prediction error. Okay. So in order to reduce overfitting, sometimes it's good to do what we call early stopping, which means not learning until you really find the best like until you really mean minimize the training error. So when I say find the arg mean of the cost function, in reality, this is not always what you want to do. What you want to do is find a minimum that has good prediction performance. So what you can do is that as the dynamic syllables, as you are learning your parameter, you track also the prediction error. Okay. Which has a cost, of course. Okay. But you should do that. And you should stop when you start at some point, what you will see typically is that as times goes on, okay, the training error will always decrease. Okay. This is the training error or the in sample error. But maybe the prediction error will do something different. It will decrease. And maybe at some point it will increase again. But both these points here, you start to overfit. Okay. So as times goes on, you should track both the in sample error and the out of sample error, the training error and the prediction error. And you should stop when you see that you reached a minimum for the prediction error. This is called early stopping. Okay. And that's it. And here they are just saying what I told you in words. Adaptive optimization methods, which are these methods we did not discuss that try to emulate second order information is more advanced solvers often have lower performances. So I will not discuss that. Okay. Tomorrow we discuss narrowing action did logistic regression. Okay. Or classification, how to deal with data which have discrete labels. Okay. We want to classify images, categories, whatever. Okay. So until now we did, we considered regression where the labels, the outputs were continuous variables. Okay. Now we want to do, we want to process labels which are discrete zero, one dogs, cats, whatever categories. Okay. Okay. And actually this is this will introduce the baby neural network, which is called logistic regression connected to the perceptron. Okay. We will code perceptron. Okay. Any question? Okay. Yes. Yes. What happens if we introduce some notion of noise in linear? Okay. The question is that in stochastic gradient descent, what I mentioned yesterday or not yesterday because we were all barcola or somewhere. I mentioned the other time that the noise induced by the stochasticity in stochastic gradient descent is not Gaussian. It has complicated statistics. It is correlated, et cetera. So your question is what if we would induce artificially this kind of noise in regression or in other problem. We don't know how to induce this noise because even analyzing, let's say there are proofs in simple settings that this noise is non-gaussian, but it's not that we can write down what statistic it has. It's an extremely complicated statistics and we don't know yet how to write it down, essentially. So we cannot emulate it, but we don't need to emulate it. You just do stochastic gradient descent. You just do it, you know? Because it's not only a matter of the type of noise that you introduce, which helps you. It is also a matter that it lowers the computational cost. Okay. So it would be kind of stupid to compute the full gradient and artificially add a noise up to a while you can get the same benefits by just doing SGD and computing gradients, which are much less costly. All right. So see you tomorrow. In the end of the afternoon, I will send a link as usual. Ciao, everyone.