Hey tribe, so not all of you might know that for more than a decade I was a deeply devoted person to spirituality kind of all different practices from Indian Advaita Vedanta, the non-dualistic teachings to Tibetan Buddhism Dzogchen and obviously Zen Buddhism, the Japanese traditions and also the Westerner teachers like Eckhart Tolle, Andrew Cohen Gangaji, like I was listening to and reading about everyone and obviously there's a story about that and also to obviously I did also practice a lot and Throughout those years. I was actually even taught about that I was an instructor of meditation for a while a kind of a spiritual teacher myself believe it or not and It was a wild and interesting journey and through all of it Oh well, since I am naming stuff There were also significant moments where I Was learning about spirituality when I went to India and I lived in you could say a spiritual school for a few years in Switzerland so again, it was a wild journey and full of a lot of discoveries and insights But the thing is I went through various stages You could say the first stage was the stage of recognizing that I don't have a clue what enlightenment is because all those teachings pretty much were all about enlightenment So initially I had I came to initially I thought I kind of have an idea of what enlightenment is Then I moved on to the stage of having no clue what enlightenment is Then I went to the stage of Figuring out what it means and and of course it may sound like a controversial subject or or idea But as I will share with you for this story, I actually Did figure out in my belief what? Enlightenment with what people refer to as enlightenment like in a legit way And after you hear the story maybe can start with her. It's an accurate Kind of thought or not But eventually I went to the next stage of realizing that There is that thing which you could call enlightenment like I see what people are talking about even like spiritual gurus I I am pretty certain. I know exactly what they're talking about and I will even explain it a little bit in scientific terms, believe it or not but in the end it's There is something to it like there is something that It happens like there's something legit about it, but a lot of it is also bullcrap and in this video I am looking forward to share with you a bit of a bit about my wild journey through the pathways of enlightenment and spirituality and I might not go as deep into the whole spiritual topic. Maybe I will but Maybe that deserves another video a different video, but we'll primarily focus on What the heck is enlightenment and why I personally think you shouldn't really Give that much thought about it like it's not you shouldn't be as hyped as spiritual people are About the subject of enlightenment. So all about that in this episode of the journey So as you can see my doggo the Grinch That's his name or an Lithuanian. It's Grinchus. He fell asleep. Hopefully you didn't fall asleep for my intro It was a bit long-winded as usual, I guess, but yeah, let's get to the topic and let's get going. So One of the first stories that I'm interested to share with you is well first of all I was always Inspired to make a difference. I was always inspired to inspire people and and I always wanted to have us to live a Significant life which would have a positive impact on other people and the long story short that let me to become an Aikido instructor or to choose To want to become an Aikido instructor and after high school I moved to Switzerland initially for three months and then for two and a half years To study with an Aikido instructor who was also a yoga instructor and meditation instructor and turns out he was Spiritual guy, so you could say spiritual guru. I think he wouldn't mind if he would call him that And he had a long background. He traveled the whole world but in Japan spent years in India met with some of the most famous Spiritual teachers alive back in the day and some of them are still alive like Papaji And he was a direct student of Andrew Cohen if you know those names you don't don't fucking don't don't don't worry about them so so he he was very knowledgeable at what he did and I became a student and devoted to Live at his school and And study there all the time until I would become an Aikido instructor and obviously meditation I was always interested in meditation. I used to In my teens as I was searching for answers. I would meet Its priests and monks and and and I spent some time with Buddhists and and I would keep asking them questions And and I was always interested in the meditation. I felt like they're something to that that will Actually benefit me and and funny enough the first time I think I caught up to meditation It caught on to meditation was when I was watching a episode of Dragon Ball If you don't know it's it's funny that that's that way anyway But but there's a moment where he's taught meditation the young sangoku and I was like something to it and I think I was like 10 11 next 12 years old and Probably around 10 and and that's when I first tried out the meditation And I liked it and then I start I initially meditated by my own and then I learned some meditations through the people I met and From books and internet, which was still developing back day imagine that but yeah, and eventually I met my Aikido instructor and He as I said was a very spiritual guy Kind of modern to contemporary as well like he had his own way out and approach but in one of our meetings First meetings would have like a weekly meeting where we would sit down and talk about life and he would he would push his students to Pursue their goals, which is actually a good good part of the teaching and one of times we he started talking to me about Enlightenment like if I know what that is and it was interesting because as he was talking about it And I was like what 19 years old Straight out of high school didn't have much life experience, but he was talking to me about enlightenment and I realized he He he didn't say that out loud, but I realized that he considers himself to be enlightened That he not only knows what enlightenment is he has experienced enlightenment And that was kind of a awkward moment because beforehand I thought That enlightenment means that you become perfect and that's a big Misunderstanding I think that's a universal misunderstanding, which hopefully it has evolved and it's not there anymore back in the day was definitely there it's an immature understanding or perception about belief about enlightenment probably From the days of Siddhartha Gurtama the Buddha Initial Buddha got enlightened and people see him as perfect and that's why they think oh so as soon as become enlightened You're perfect, and and that's not the case on any level and like enlightenment is not about that but But then back in the day, I thought that especially as you know as a young adult having little experience of little knowledge and I thought In the back of my mind I didn't say that to him, but I was like You think you're enlightened and also I had that also stereotype like oh only Indians can be enlightened or only Japanese can be enlightened or whatever And then back when I was like, okay, that's kind of weird. I was still open-minded, but that was that kind of Caught me I've got us. I think like does this guy think he's enlightened. Okay, that's kind of interesting But you know as we moved on in our relationship. I developed a huge amount of respect for him and and I started I listened more and more about enlightenment and he kind of directed me to the direction of that If I remember correctly there was kind of that notion that unless you become enlightened you can't really be a great human being capable of Achieve creating great Change in the world and that's that's bullshit. I mean, there's so many people who would never consider this and those items or wouldn't be enlightened by the standards of enlightenment, but you know, they they changed the world and Including some of entrepreneurs and Bill Gates Steve Jobs the first two names that come to my mind and Again argument if you would say what they changed, but you know, you can name anyone and Martha Luther King, you know, was he enlightened? I don't think that was the point. So it was kind of a faulty idea But I think that that was kind of the the idea I got from him that I need to learn if I really want to create major impact I need to learn and understand what enlightenment is and and become enlightened and then I will be like a super human being super useful and And my value to the world will increase and that's when I went wild You know, I went nuts. I am kind of that person. I'm like a hundred percent or nothing and I completely devoted myself to the Pursuing what enlightenment is I read all the books and metadata all the time I kept asking him questions and asked him for challenges and did the challenges and and Also, like all the practices that were there in the school that I learned By the way, I'm checking if the audio is recording last video. It it bailed on me. So So I'll do that some once in a while, but just I won't say anything about it So, yeah, so so the whole three years I studied there was a lot about a lot of that was about Enlightenment a lot of conversations we've had in practices. So I was all into it and Interesting enough again like keep your pants on Let me let listen to the whole story until the end before you make jumpy assumptions and and think whether this is bullcrap or not But one day for the practices with with my instructor He I I got it. I was like something clicked in me And it's a long story again. I don't want to I think I don't think it's as valuable as The main story that I'm guiding you through right now of you know, what that means what that whole thing means, but But that there's a huge question of like who you are who am I the miss main question in spirituality and most of spiritual traditions And when you realize who you are and the idea is Basically theoretically, it's like you realize you are not your body But theoretically if you understand it theoretically, it's like it's there's not much to it Like if you think you understand that that was one of the stages I mentioned to you initially thought Enlightenment is about being perfect and then I wasn't sure when enlightenment is but then I realized, okay, it's about not Thinking you're the body but not thinking you're the body is not the same The thing is you have to you that moment when it clicks with you on that click is like universally Described about an old pretty much spirituality when the person comes quote-unquote enlightened that click happens that that deep Clear realization that you're not the body kind of it's a sense of where your mind opens And I think a lot of people have kind of that glimpsed it And I'll talk about it more in scientific terms like what that really is or what I figured out it is But that thing it clicks with you and you have that kind of mind expansion experience similar to drugs, but you're not on drugs and Then you don't associate yourself as the body and if you have like a spiritual teacher Or you're smart enough yourself he points that He points that finger at that subject, which my instructor did and then you realize deeply like shoot That's true. I'm not the body, you know, I'm like like you get the direct experience of realizing You're part of the whole or you're not even the whole you are the holder You don't exist like all that stuff which spiritual books tell you about in that moment I got it and it was really interesting because kind of my mind did shift and might switch and From then on it took me a while to kind of localize myself again and get into a rhythm of life But from then on when I read spiritual books beforehand, I read them and I thought I understand them But but it was all an idea. I wasn't sure I had to guess but After that click moment, I Just like all the spiritual books became so clear. So I was like, I know what these guys are talking about It's like I know I know it. I've tasted that apple and so it was really interesting experience and it did change me to some degrees but now the thing is that's the other side of the thing which I want to get to and The thing is I lived like that for a while, you know, like believing that I achieved enlightenment and and but I started to also see like, okay enlightenment is not what I thought It's that understanding but the idea is that in enlightenment you lose your ego your ego dissolves Because you realize Kind of the theory is like that If you don't have that click and don't think it's so awesome. It's not as awesome as as I think people make it But let's let's just talk about it in In neutral way as best as I can So until you have that click the individual in the world is all about you It's all about you. It's like you are the center of your world and even if you're altruistic or the center of the world It's the ego's core would be but when you have the click moment you you realize you have a clear realization that you are not the body your your ego is just a construct and then you have a chance to kind of Dumb dim down your Your ego and not be self-centered and and with that realization kind of become a servant of the world Now the thing is I think the degree it has that effect, but I spend enough time With the whole spiritual scene to also start seeing the other side of the picture and that was Your ego doesn't disappear It kind of gets into a different shape and I think you can work hard To keep it low But then also to especially if you're like a guru if you're a spiritual teacher people love you people worship you people are like You know, especially that culture it's so guru focused and it's so easy to develop that sense of I'm the enlightened guy even if you don't say it even if it's subtle and not everyone that I met had it I mentioned to you I went to India I met some spiritual monks spiritual teachers who were officially enlightened and that's some Westerners who were officially enlightened Recognized as my human beings and some of them were humble, you know and everything but they were still human They still had the same crap and some part of me is like I respect them on a huge level especially Some of them like who devoted themselves to extreme levels of servitude like some of the monks I met in India Actually Westerners but also to some of the monks and spiritual teachers I met in India were just they're actually were douchebags You know, they were self-centered in a spiritual way And I'm actually I will make a video and talk to tell you a story about a specific encounter with a well-known spiritual teacher that I had Which is actually very interesting story, but it's for another day, but he was self-centered. He was like Playing the the spiritual guy the spiritual dude. He was playing the role, but then he was still human You know, he I could see especially now when I look back. He still had the same flaws that everyone else It's just an idea that after you become enlightened all of that disappears and When I started realizing that enlightenment is not What? People make you think about that it is I Started my I started to open up and I developed more critical thinking especially for my martial arts journey And I started to look back and look at the details and I started to see a lot of shit In the world of spirituality enlightenment As I said, I'll probably make a different video about The bullshit of spirituality itself, but but in terms of specifically focusing on enlightenment like one of the probably one of the most famous Enlightened Westerners dudes is Andrew Cohen He was the also he created this famous magazine called enlightenment next But yeah, so he was very famous had a huge following until eventually but also he turns out he was very abusive apparently he was disrespectful towards women and He was just a douche. He was like he was like you're either with him or you're with No one and I noticed that pattern is a lot of spiritual schools and teachings worse Even if it's subtle they don't say it but sometimes you can feel it's it's us and it's them and Eventually what happened to Andrew Cohen is what he was all about how the ego dissolves with enlightenment and yadda yadda yadda but Eventually his top students wrote a book about him where they also asked a Psychologist or psychotherapist I think for help and they identified They shared a lot of dark stories about him like what a douche he was and also They the psycho psychotherapist I guess professional pointed out that actually He credited some Like narcissism and and what not like specific diagnosis where that that that guy and to Cohen was as he was a bit of a mental mental guy And that was like a deeply respected huge following spiritual guy officially recognized as enlightened So so he thought he dissolved his ego, but his ego was just on a different level There's also a story actually my kid instructor He told me that he once met one of the well-known spiritual teachers in India and There's a specific moment where she was really giving apparent based on him. She was getting awesome spiritual teachings and and By the end of that somebody asked somebody mentioned that they have a teen Teenage girl who's like really What sort of word She's really like a rebel and she's not following the rules and etc. Etc. And They asked her like what's your advice and she suggested them to put her in like in a psychiatry clinic Because that shows that If you get what enlightenment is if you have that quick moment You're part of your mind shifts and you get a realization that some people haven't yet had you could say But that's it It doesn't make you that much of a better or smarter person Not at least in my experience It's still In the end it's still all about the individual and who you are and how you are Personally for me, I always wanted to serve people are always wanting to Do good to the world before it had that click and after I had that click It did help me somewhat but like not that much like most of the work I did was beyond that with myself the crap I had the crap I keep working with you know It could call it ego stuff and whatnot that stuff That stuff is like enlightenment doesn't take care of that or that click moment It's it's it's all about just working with yourself constantly and never letting yourself off the hook Always realizing you'll never be perfect and you always have to be vigilant because there's always you're always going to slip and there's there's no nasty stuff we Take culturally and we're all human beings and we have our limitations And you always have to be mindful of what you do and and you always have to again work with yourself And I think the dark side of that whole movement of enlightenment is Because to some people the idea that they became enlightened or that click moment Makes them think that that that was it Well now they can Go wild and now they know the truth and and that covers everything and that's kind of similar to martial arts if you practice martial arts It's that phenomena when you have a black belt in a martial art Some sometimes it gets into people's heads and they think now they're black belt at everything Which is nonsense. No, you're only good at you're only a black belt at throwing or breaking people bones and stuff Nothing else doesn't make you a better husband doesn't make you a better Business guy all those kills have to be trained separately and same with the notion of enlightenment That's as much as it does and some people spend decades to become enlightened so last part of the video I think I Mentioned you I will break down the scientific part behind it, which I eventually kind of caught on to just by accident actually so Couple of years ago when I was living in the States. I became fascinated about by a book Called the rise of Superman and I'm not enthusiastic about the name. I keep saying that I think it's a funny name It's funny name, but also I think it just doesn't doesn't do the justice to the book But the book is incredible in short. It's about the state of flow Which is like an actual scientific state where your mind goes into a peak performance It's kind of like being in the zone, but it's all scientifically, you know measured and everything It's it's that capacity of human beings to To turn to switch on To switch on where the brain becomes super focused and your whole body It's like you're you're it's like a super focused state where you're actually like in a peak state Where you you are much more capable than anytime before or after and you can repeat that But anyway, long story short Scientifically this guy went through the journey the book the writer for the book he went through the journey to learn what that is and It's all science-based, which is awesome and turns out and he wasn't even talk about enlightenment But I just realized that's the same shit was he he turns out for the use of science and When you're in that flow state in that peak performance state Your mind the mind part there's a apparently there's a brain part which helps you realize Where you are localize yourself as an individual It's kind of these this the self-centeredness aspect where but that would if it's if that part is dead Let's see and that could be a mental Psycho psychological issue if that part is dead you Then you can go mad because you don't realize that you're here That's your body and and sunny that there's no Tool to localize yourself and to identify yourself as a separate human being that part is specifically responsible for that and in the flow state Which is a feeling of a kind of mental Consciousness kind of expanded consciousness expanded consciousness state which see I talked about that and When you're in the flow state that part Temporarily turns off it doesn't function which means in the flow state Which is almost like a the the guy proves that it's almost like a survival state That that when your your whole system your whole organism realizes you need to Have peak performance and often people have that experience it's not limited to it Otherwise you wouldn't have that in life moment besides that but it's often in the moments of life or death when the flow state happens because your whole system turns on to do this one thing and through meditation You can achieve that too And but that's it It's only that What I'm trying to say to you is that In the state of flow or in the state where people think they became enlightened Temporarily it's the same state Temporarily their brain sheds that the part of the brain which localized themselves sheds down and you feel limitless you feel like I am the universe I Am nothing. I'm not my body because that part is off That's all there is to it in my understanding. I'm pretty sure I have pretty certain Belief that I'm right especially because I was in that world of the crazy belief of spirituality and And I went through that whole journey and I met people who were on it And now I can look back at it through this having this knowledge. I think that's all there is to it and Yes, as I mentioned you can utilize that to kind of dim down your ego for a bit But that doesn't solve all your problems. That doesn't make you a saint To be honest doesn't make you that special either In the end your long-term performance and how you are good as a human being. I think that's What makes you special not that one moment where oh you saw into something that most people haven't Fuck that So, yeah, let's drink a sip of coffee before we end I guess I could continue But I really want to make sure that whatever I say is kind of condensed in the message would be clear enough So I want to expand more I'll probably make a video and episode about spirituality specifically the dark sides of spirituality that I said and talked about it went through and even for a period of my life spread myself But I hope this story gives you a bit of a better glimpse into what potentially enlightenment is and that it's not such a big Deal as people make it. It's not the end-all solution. I think there's better things to to spend time on and To focus on and to yeah work on And if you're on that path, I hope that that makes you think and hopefully you Direct the same energy or you are investing into becoming enlightened Into something more useful in my opinion Obviously, that's just my opinion, you know, but that's why stories are great And that's why I'm inspired to share stories with you that hopefully my lessons will be valuable to you and you'll learn from my mistakes and yeah, hopefully You can come to your own conclusions through my story versus me telling you what's right. What's wrong. This is just my opinion I'm just my experience So yeah, thank you for watching. Let me know in the comments what you think about the whole thing and As always Keep questioning


Thank you. So I'm going to talk on react.js We will cover some basic concepts of react.js how it should be used and we will be building a small application For understanding those concepts. So I'm Prathamesh. My Twitter handle is here. So you can follow me. I Work for big binary. We are a Ruby on Rails consulting shop based out of Miami All of our team members work remotely and after react.js was released We interest we got interested into it and started using it and that's why I'm giving talk on react today. So today's agenda is we will be discussing some of the concepts of react Some of the features that go against the traditional way of web development and we will be building a small app based on open library API For understanding those concepts. So you can check this link bit.ly sip of react.js Here I have mentioned some resources for getting started with react. I have also added a link for a Video series that we have prepared for getting started with react You can check out this video link and then you can find code examples from my talk and link to the presentation So let's start we will be using open library API. So open library is an attempt to attempt to create a page for Every book that has ever been published So it is a open catalog of all the books that are ever being published and it has some API Like I will be using this search API from from them So they allow us to search books based on some query and we will be using a similar kind of We will be building a similar kind of app where user will be able to enter the query and then we will show We will show whatever books which match that query so We will start from scratch and in the end of the session We will have a simple minimal app where user can enter the query and they will get the results So Wi-Fi is slow. So this page is not yet loading. Let's see Yeah, this one just go to bit.ly slash sip dash off slash dash react.js Okay, you can you can go to this link bit.ly slash sip of react.js Thanks Okay, so let's start with react first. We'll discuss what react is react is a simple JavaScript library for building user Interfaces it was developed at Facebook and Instagram and now it is open source for everyone to use Its job is very simple just building the UI So it doesn't care about what rest of the technology that you are already using it also doesn't care about what backend You are using or what other tools in your front end you are using so its scope is very limited We can't compare it with big client-side framework like ember gels or angular gels You can compare it with view layer in traditional MVC architecture So it just takes care of rendering your UI and nothing else It it's aim is to be very simple So it gives developer all the power to be expressive in specifying what their UI should look like and it takes care Of how to manage and handle that UI in the in the background. So in the react We are just specifying what I want not how I want So my UI I will describe it in terms of okay If this is the state then I want my UI to look like this But how that UI is going to change and how the state is going to change from one step to another React will take care of that in the background This is achieved by providing a declarative API So let's say I have a app where I have a video encoding service user uploads the video and I send that Video for encoding to a third-party service and then I'm pulling okay if the encoding has been completed then remove the loader that I was already showing and display the header text that Video has been encoding has been completed. Maybe I will redirect to some other page and otherwise I will keep showing the loader. So here we are talking talking with the DOM We are interacting with the DOM as per the state of our app But in react what we'll do is we'll just specify that if encoding has been completed Then my header will be this and if it is not completed then I will keep showing the loader I will be keep showing the loader as it is. So I'm specifying what I want based on the data I'm not specifying how I want it React also has some unconventional ideas like one way data flow no data binding. It also has immutability So all of these ideas go against the traditional way of doing web development like mixing markup and display logic So we will be discussing all of these features to understand why react relations So let's start our first step is hello world I have already set up the app to work as a npm app I will not be discussing this setup because I want to focus more on react features and how it should be used So let's start our first step is we should be able to see the Hello world message So I'm on the first branch and I will start the app using npm start Let's go to localhost 3000 Okay, I'm able to see the hello world message. So what is happening here is this is my entry point This is my entry point of that. I import the app.js file from Second file and all of my application related code will come into this file app.js So right now I just added a alert hello, why to just make sure that everything is set up properly and working So that is working now. Let's go to our next step So how many of you attended Alex's talk on Kansas on Thursday? Okay, so not many people so he talked about how we should build large application the secret of building large Applications is not actually building large applications But building smaller components smaller reusable composable components and then combining them in the end So that we get the big application that we want and this is the code from author of Kansas Justin Meyer So react also has similar Similar features so react what it has is a concept of components So our UI is broken down into small parts of reusable components And in the end we combine those components to form the complete UI So a component represent a related set of HTML JavaScript and CSS bundle together So for a specific part, let's say I want to show the sidebar So whatever HTML JavaScript CSS required for that particular sidebar will be present in sidebar component So let's start with building our first component now that NPM start will be running and we will be able to Code side code along with it Okay, so my app.js is now changed. What I'm doing here is I'm code is visible, right? Okay, so I'm first requiring react using common.js style Once that is done, then I'm using this create class method So this is one of the top level API method which we need to use for creating the component so create class Method creates a react component and every react component must implement a render method So what render does is it it renders that particular UI on the on our in our browser So this is the minimal method that we need to implement. I'm just Rendering one day with an h1 tag h3 and simple do so if I see My if I refresh the page Yeah, I'm able to see this h1 and h3 and And we are also doing one more thing which is in index.js so here What we are doing is we are first requiring I'm using your ESI a sick style syntax for importing the modules I'm requiring the app model that was that I just showed you and then I'm using this react render method So this is another top level method which is used to Render a particular component on the document. So here the second argument is documented body I'm just mounting that react component onto documented body, but you can also pass a specific element Like I want to render this component on a particular ID particular div with a particular ID. Then that is also possible So let's go to next step Now if you see our component If you see our app.js, it is nothing but a simple function So we pass arguments to this function and it describes our UI it returns our UI So consider it as Consider it same as whatever function that we are writing in our server side back in code Same thing for given a set of parameters. It gives the same UI all the time So it becomes very easy to sort of reason about what my UI will look like for a particular input because it is going to Give same input for that all the time So that's why component is a simple function It is also reusable once I developed the component for displaying a book I'm I'm able to use it on my dashboard. I'm able to do use it on my purchase page on my cart page I'm able to use it everywhere because it's a simple unit It is also composable. So by building small modular components We are able to compose a book list from them. So if I if I Write a component for displaying a book. I'm able to display a book list. I'm able to display a catalog I'm able to display a dashboard using those components And it provides right kind of separation of concerns. So I'm only worried about how my UI will look like and It's sort of similar to building a domain specific library for your particular app So what we are doing is as per our domain we are just building small components and we are able to use all the advantages of Writing classes or writing functions in our back-end code But this time these advantages are present in our front-end one because of the app Let's look closely at our component So here what I'm doing Am I directly writing HTML inside JavaScript? That is bad, right? We should not do it, right? We are we know that we should separate the markup from its display logic all the templating languages that we use for Separating our markup. We're already using it. So my react is forcing us to do this Is it bad? So turns out it is not really HTML. It is JSX and JSX is a syntax XML is syntax for specifying components in our Components in our react components. So what it allows us to do is it allows us to write HTML like tags and It converts them into actual DOM at the time of rendering So this is not actually a div, but it is a function in react React library so it gets converted to react dot create element and it renders that particular div So we are not writing HTML inside JavaScript, but it looks like we are writing HTML inside JavaScript One of the advantage of using this style is our designers can easily understand this code So if if somebody is not somebody doesn't know react, but still they will understand this part of HTML They will see it as HTML. So they will be able to understand this code very easily So it becomes very easy to onboard people who are not into react But still they will be able to contribute to that Another thing is this is completely optional. So if you don't want to use JSX you can skip it But I'm not sure if anybody is not using JSX with react because the syntax for not using JSX is very bad So you have to write a lot of code for it and this is very natural. So we get used to it We get used to it very quickly So if you have still not yeah, so what I have done here is I have used Babel For converting my ES6 code to ES5 and Babel takes care of converting my JSX It has a transpiler for converting my JSX code to normal JavaScript So I don't need to require anything, but if you're using the vanilla react that comes then I think I think they are also switching to Babel now, but Some versions ago you have to require the JSX transformer that comes with react explicitly Yeah, right Correct, so you can do that without any problem because this is a normal because this is getting converted by Babel It is not related to react So if you are using the vanilla JSX transformer that comes with react then you have to include it But here in this case what is happening is it is getting converted using Babel loader. I Will show you after the talk how it is getting read. I just don't want to go into too much details of how the app is set up Yeah, yeah, I will come to that point. Yeah I will come to that point So if you are still not convinced about JSX Think of it as a pill that is test bad given by doctor, but it is actually healthy for you You feel guilty, but it is good and give it five minutes start Write a simple app using react with JSX without JSX. You will you will find out that JSX makes it really easy Okay, let's come back to our app We will start rendering a first book will start with one book at a time and right now We are not worried about data coming from API. So we will just hard-code the book for right now Let's go to our third step Okay So I'm displaying my first book Lord of the Rings and let's see what change in our app component So here I can see that I added a array. Sorry. I added a object for with name and author and then here I Added some code for rendering that particular book. So I'm rendering div with class name So you see that here? We are not specifying classes in our normal HTML style We are specifying class name and n is capital that is sort of react way of Rendering HTML attributes. We have to specify them in camel case Then I'm switching image with a class name height h2 h3 nothing nothing very special here So that's how we are displayed our first book Now I come to the point of templates. So here if you see we are not using any sort of templating language for rendering This we are just using JSX and plain JavaScript. So react philosophy is Not to burden you with any other templating language. So templating language What they do is let's say you want to iterate over a collection and render each item in that collection You have to write a construct for that in your templating language If you want to do conditionals like if the condition satisfies then display this otherwise display this you have to write a construct In your templating language for that, but what we are actually doing in that case is we are trying to mimic the existing Features that are already present in a programming language in a complete programming language. So react takes a reverse turn and says that We should not use any templating language, but we should just use JavaScript So JavaScript has all the features of a complete programming language. Let's use those rather than inventing our own templating language Let's go to next step We want to display a list of books now and I will show you how we are using JavaScript rather than a templating language So here I Have converted my book to a book list and it is a now array of components. It is not very visible So I will decrease the font Is it visible now? Thank you. So here what I've done is I am iterating over that array I'm using map function of JavaScript and then I'm calling a function render book for rendering that particular book and I just moved that code for rendering one book into this function So rather than hard coding those value I'm using here this curly braces So whatever code that is present in those curly braces gets run as JavaScript expression So my book that name gets evaluated in the runtime Whatever object I passed and it gets displayed. So if we see the if we see our page We see that whatever values I was passing passing to that function. I'm seeing those two books here So I just use plain JavaScript. I didn't have to use anything other than that and that comes Now I will answer the question about underscore JS Because it is just plain JavaScript you can use any other JavaScript library that you are already using So just include that in your package.json Require it using common JS and you have access to it. So it is it is plain JavaScript It is not it is nothing special. So you can use any JavaScript library that you want to use Now this is one of the challenge, right? Our apps are becoming more and more interactive We are providing so much functionality in our apps that it is becoming very hard to Managing the data which is changing as per the interactions of the user in early days It was very simple. You just have to refresh the page and everything everything gets loaded again But now we are doing more Ajax. We are replacing only some parts of the page so it becomes very hard to manage data and React is made for building applications with data changing over time. So Facebook is one of the example We know how interactive Facebook it is and Instagram also So piton says data changing over time is root of all evil. He worked on react at Instagram and React sort of tries to solve this problem by giving us props So props are simple properties that are passed to a job to a react component earlier I talked about how a component is a function it gets parameters and it renders UI so those parameters are props so let's see Let's see how we can use props Am I going too fast or is it okay? Just give me feedback so that I will change it Is it okay? Thank you Okay, so let's see how we are going to Yeah, okay, so If you see in my app.js all of the code that was actually rendering the book is now gone And I'm using another component called as book list and I'm passing it sending I'm passing it a prop with name books And I'm using curly braces so that my array gets passed to it rather than string So if I if I want to pass any string then I will have to pass it like this but as I want to pass the array I'm passing it like this and now my book list component what it does is It gets those props and it it calls another component called as book It maps over those props as this dot props dot books So it it can access those props under this dot props and whatever you have passed and then it renders another small component for rendering a particular book and we pass that particular single book object to book component and All of the code related to rendering that book is present now in book component So here I can access the cover and name as this dot props dot book dot cover this dot props dot book dot name So it becomes very sort of Now I can use this book component outside of book list also because this has specified an interface that this component needs Input props in a particular way. It needs it in terms of book object. So this is how we can create props Now if you think of it props are really immutable. We are passing props from a parent component to a child component So child component doesn't own those props. They are owned by parent parent decide what needs to be passed to the child Child only tells okay. I want data in this format But the control of data is present in parent component. So for child the props are immutable The child component doesn't have to care about how my data is coming or what should I do? Do that data it just needs to know that okay I'm getting data in this format and I need to use it for ending the UI and This gives rise to the one way data flow data is always flowing from parent component to child component We are not going in the reverse order props are getting passed from above So this is how you can pass data from parent to child But sometimes you have to pass data from child to parent you have to notify parent that something has happened Some event has happened user has interacted with me in some way. So how that is possible? So turns out that is also possible Using a simple trick So let's see that child to parent data flow So let's say in my app. I Want to I want to do some logging Whenever this image is clicked I want to do some logging that a particular book has been clicked and I want to do Logging in my book list component not in my book component. So how will I achieve that? So here I will go back to my book list again now with passing I was earlier I was only passing book object but now I am passing another another prop which is logger and This is a reference to a function of parent component Okay, so this is a reference to a function. We are not passing the object or any string We are passing a function and in my child component Whenever somebody clicks on the image So I added an on click event handler and this is how we do event handles So in our normal HTML, we we write the event handles as on click on change Similarly, we can write it here. So here what I'm saying is on click call this this dot handle click function And in that function I have access to the function that was passed from parent So I can call that function and pass whatever data that I want to pass and then my parent component will have access to that data In its parameters and then it it will do whatever it wants to do So this is how I can send data from child component to parent component. So let's see that in action Let's see if really it is getting logged or not. I'll just refresh Clear I click on it something got printed Book jQuery in action clicked Book head first jQuery click. So this Logging is happening in my parent component, but from my child component I'm telling my parent that what has been clicked. So this is how you can do the Data flow you can pass data from child to parent. So our child components specify a way of They just say that I want props in this particular way But what if you are interacting with a third-party API and you don't have control over how data is coming to you So in that case our components can also tell our components can also add some Validations for making sure that the props that are coming to them are coming in proper way and that is done using prop types So let's see how prop types can be used Okay, so here I'm saying that Whatever books that I get in my props there should be an array It should be an array object and it is required value. I cannot function without it So whatever whoever it's passing me those books needs to make sure that they're passing an array and they're Passing it all the time if I don't get it. I will not be able to render it So if I change my app.js now to instead of passing an array I just pass the first element of that array now. I will get a warning in my console. So if I refresh now Here I get a warning failed prop type invalid prop books of type object supplied to book list Expected area. So this is also so I will say this is also one of the features of react that it gives you proper warnings It tells you exactly what needs to be done Even if you have typos in your code like instead of capital react use you spell a small small case react It will try to guess that okay You're trying to write capital react and it will show you in the morning. So this is how Prop types can be used for validating your props making sure that your child components are getting data in proper format You can also write props like you want You want functions you want functions to be prop a past as prop you can specify that this particular prop is optional So there is a lot of way of specifying how you want data Next up. We are state. So props Component doesn't have control over props. They come from parent, but in case of state state is internal So every component has its own state and that state can be changed So a component can it's a changes state based on user interactions and then it can do some action if if needed So you can change the UI based on state changes so state is internal and the entry point of Assigning state to every component is get initial state. So let's see how that is done Now if I go to my app.js I just added a function here get initial state Now this is where we are again going back to our application And trying to complete it. So we wanted a search form and we wanted an input box for our for users where they will enter their query So in this step, we have added that input box We have added some default value because right now we are we are not yet reached the step of fetching the data from API and This is how it looks currently so We have added a search box. There is some default value and then there is a search button So we can access this state as this dot state Similar to props props you are accessing as this dot props same to similar to that we can access state as this dot state Okay, so right now. I have just added One event handler for making sure that whatever value that user has been Changing in the text box gets alerted. So here whatever handle change I'm alerting the value. So let's say if I change I will get the updated value Later on what I want to do is pass this value to the search API and fetch results based on that when the search button get button gets clicked So this is about state and get initial state Let's go to our next step because whenever the state has whenever the state is changing. We want to respond to that change Okay so here So on change of input we we are calling this handle change method So whenever user types something we will call handle change and what it what it will do is it will use a set state method So whenever we want to change the state of component We will use this set state method and it expects that you write a JavaScript object inside it and there I'm changing my existing query to even that target dot value. So what is happening here is initially my query is Just a empty string my books array is empty array and then whenever user changes the input Handle change will get called and it will update the state with new query new whatever the user has entered it will update that state so if I And I think I'm also doing console.log to just make sure that that state has been updated No, I'm not so let's add a console.log here This dot state dot query just to make sure that I'm updating state properly Okay Search books now I have removed the default value and You can see jjq jq u jq u e and in the end j query So I added that console.log in my render render method This means that every time that state is getting changed my render method is getting called So react will make sure that whenever you are changing the state it will make sure that your render method gets called Okay, so this is how it it responds to state changes Whenever you change the state using set state method it will call there it will just re-render that particular component by calling the render method So let's see now how we are going to use our submit button Now let's just refresh again Please enter a query because I have not entered any query. I have added a handler for submit button also so on click call this dot handle submit and In that function I Just prevent the whatever event prevent default. I don't want to use that and here I check if this dot state dot query is empty then alert please enter a query. So here instead of So in Traditional way what we could have done was we had a input box And we had a button when button is clicked we check the value of this input box directly Okay, so we need to access this particular input box directly, but here Because we are managing the state of that input box using set state method We can directly access our state because it gets updated every time that something in that input box change So here we can just check that this dot state dot query is empty. Please enter a query Otherwise we will fetch data from open library. So just use Ajax pass that data and Construct an array So here I will just make sure that it is readable Okay, so we pass we use map function iterate over that array and we just parse it because they send data in Particular format we don't want that because our components expect us to send data in another format in terms of author and name So we just make all the required changes that we need to do so if I write jQuery here and Search let's see if internet works and it comes. Yeah So here I got the data from a third party API open library API. I converted it into whatever format I wanted and then Because my state would change because here I'm calling this dot set state and Assigning new books to it render render method gets called again and Earlier I was passing the books array as it is but Now I'm passing this dot state dot books. So whatever the current Books that are that are in my state. They get passed to my book list component. So if I Change the search now, let's say I search for react. Yes Let's see if there is any book. There is nothing because what happened was Search when we click search it called that handle submit handler then it contacted the Open library API fetch data from it updated our state react Got trigger that state has been updated Rerender the component and when that component was re-rendered it displayed the new book list because it was using whatever books were present in our this dot state Okay, so We use set state We use the re-rendering of component after the state was changed and if you think of it It is sort of similar to refreshing the page. So we are where we aren't actually refreshing the page We are refreshing a particular component So we are refreshing only that part without actually refreshing the page and Re-rendering a single component is not enough because if that is the parent component and that is rendering 10 other components Then we are rendering everything that is below under that parent. So we are actually re-rendering the complete component tree and Will it make my app slow? Will it will it be too much like if I'm rendering everything based on just one input change Will it make my app slow? So turns out Incomes virtual DOM so react also has this feature of virtual DOM which makes sure that the re-rendering will not cause any performance issues So what virtual DOM does is is it makes sure that it keeps a copy of your current DOM in memory? It it it converts your it makes sure that it creates a copy of your existing DOM based on the current state Keeps it in memory Then when something had changed using set state it will create another copy of existing DOM based on the new state It will use an algorithm for finding diff difference between those two Representations and it will find minimal set of changes. So how many of you attended yesterday's talk about JavaScript performance? Yeah, so he talked about doing only minimal changes to a DOM DOM operations are expensive So here react will make sure that it will find the correct diff and it will only do the required changes So if your component is not getting changed nothing will get re-rendered it will make sure that you have access to It will make sure that only minimal set of changes get supplied and it will also do batch updates So he also mentioned yesterday that if you do batch updates our DOM operations will be faster So react will take care of doing batch updates for you If you think of it nothing has been new here. We know that DOM operations are expensive We already know that if you do batch updates our app will perform better But what here happening is as a developer? I don't have to worry about it react will take care of it for me in the background This is another feature life cycle hooks I have not added any code for it, but I will just explain it shortly So react component has a life cycle it gets initialized it gets rendered and then it gets unmounted So react also provides us with some hooks where by using those hooks We can tap into life of a component and we can perform some actions at particular events in that in the life of the component So whenever that component gets mounted we can use component will mount so at this stage component is not mounted But that is the next step. So whatever you want to do before mounting a component. You can use this function Component did mount as soon as a component gets mounted if you want to do some cleanup You want to fetch some data from API you can use this hook similarly There are some more will receive props will unmount data and mount you can do clean up Let's say you want to track something then you can use this unmount hooks Let's discuss some patterns of using react so avoid state as much as possible as I Showed you in my app component. I'm fetching the data, but my book list and book doesn't care about how that data is coming So they don't know from where the data is coming. How it is coming. They just know that okay I'm getting today. This makes components reusable now I can use my book list outside of my app also because it is not depending on how the data is coming so avoid states as much as possible and this also Make sure that the state changes are minimal. They are located at a particular place in your app So it becomes very easy to manage that state So if you can render a component using only props and avoid state all together that is better use props This also leads to a concept of dumb and smart components So in our case our book list and book are dumb components. They don't know any logic They are logic sort of logicless because they just only care about rendering the UI and our app component Care about how the data is coming how it should be cleaned up and how it should be passed So this makes components reusable again and composable So use dumb and smart components With every new technology. We have to invest time in it. We have to make sure that And on it it it makes sense. It makes sense for us in long term For react learning curve is very small So you just need to the minimum API is very minimal and you just need to know JavaScript and HTML J6 is there, but J6 again gets converted to JavaScript and it is simple htmlx syntax So you get used to it very easily It also doesn't care about rest of your stack as I mentioned earlier So it is easy to sneak into existing projects try out on some new feature and decide whether it makes sense for you and Don't just use it as Alex said in his keynote Don't just use any tool because somebody's saying about it or Facebook is using it or there is a lot of hype Know about it experiment with it and then decide whether you want to continue using it or not So let me know your experience. You can quit to me How you feel or you can catch me later? That's it. Thank you. Any questions Okay, thank you Yeah, so virtual down batch updates all of them will make sure that your performance is good You're not Like your performance will not degrade But in case of a lot of nesting components a lot of child and parent component And you want to do some communication between this child to that child this child to parent which is not parent of current child Then my the way I showed how to pass data from child to parent will get complicated So in that case you can look out for flux, which is a sort of architecture for using react apps I have not mentioned flux in this this presentation because I wanted to keep scope to only react But you can look for flux which makes this kind of communication between components very easy Correct. So that is a good question actually So what he is saying is a react keeps a memory representation of your current on then whenever state changes It creates another representation dips it and finds the difference But all of this is happening in JavaScript. So we are not touching the DOM yet so this is this is more performant than doing DOM updates and also the algorithm for Finding this particular diff is Return in a good manner by Facebook people so that they make sure that it doesn't take a lot of time And also there are some use cases where this fails So recently there was a tweet from one Google employee where he he created a infinite scroll app on mobile Where react took a lot of time for re-rendering. So there are certain use cases which which where this doesn't make sense also So again, it it goes down to the same thing that you have to use it And then you will figure out like where it is failing so that you can tell react team that improve on this Yeah, yeah, we can we can discuss it in the break Right. No, it is not so consider react components only as rendering layer So I can easily pull out this Ajax code and put it into some utils folder Okay, which will be simple JavaScript plain JavaScript nothing else But for demo purpose, I just kept everything in the same component because I wanted to show you how it how it was I will show you later some code which which separates them properly You can also take a look at flux which user stores actions and Dispatchers where this all of this is separated Something correct react. So that is the main point of the yesterday also. I was discussing with few people Some of them were migrating from angular to react and they were thinking that okay I have to do everything in this react doesn't do all of these things which are done in angular But that's the point right react is not everything. It's it's only concerned about rendering So yeah Yeah, so that's the point right react makes it sort of implicit that or explicit that you will mix The HTML inside Josh what I was mentioning from with respect to separation of concerns was yours sort of I Get the same advantage. So I come from a back in background So whatever advantage that I get from writing classes single responsibility print following some principles like single responsibility Writing modular functions. I get same advantages by building react components But a component as I mentioned earlier also component will have HTML JavaScript and CSS Related to a particular part of here and in long-term that actually makes sense because you have only single place for looking looking for bugs Let's say you have some bug. You know which component I need to look into right Right in this case anyone can work on this Yeah Yeah, because if you're if you're a designer you can still understand JSX because it is similar to it You can consider it as same as HTML. So you can for HTML person He can still understand it for JavaScript back in person He can still understand it because he knows all the logic that is going on So but you can even if you are not doing JavaScript You can still understand the code because it looks like HTML plain HTML. Yeah, and I was trying to so I came from a Back in I'm rails programmer basically So I am talking in that perspective that it becomes very easy to understand what is happening And if I if I want to add a new element to my UI, I know that okay I need to do I need to add a new component or I want to pull out something from existing component So that it is reusable. There is no hack, but again, it's boils down to JavaScript. So correct, right? So I have not used backbone angular ember at all. I can only talk about react and because We are a real shop when react came out We experimented with react and we found out that it is a very easy to replace some parts of our app using react We don't want we don't have to invest a lot of time in making like making so many changes in our app So that we could integrate react. So that is my experience. I am I don't know about Ember or angle. This is just a view layer. So you can I'm not sure about it But maybe you can try react with angular or ember. I'm not sure about it Thanks. Okay, we're done. Thank you


Good. OK, sweet. So, sorry. OK, so today we're going to, so on Friday we talked about this classic file system design, right? The fast file system. And the FFS is sort of, you can think of the Godfather to a lot of classic file system designs, right? There's something particularly fancy or aggressive about what it's doing. It's a nice design. And parts of it have survived fairly well over the years. Today what we're going to talk about is something called log-structured file system. So in contrast to FFS, LFS is an example of a sort of radical paradigm shift in how disks were designed. And it's a nice example of a file system that's responding to technology trends. So a great deal of the reason why we keep redesigning systems over time is because technology changes. And we'll talk a little bit about some of those technology trends later in class. OK, so assignment 3.2 is due on Friday. Weeks are just spinning by. So at this point, hopefully many people are to the point where things are pretty much working and you're sort of hunting down corner cases that might get caused when your core map fills up or other things like that. So we will, Rajesh is going to do another one of his all nighter office hour type things on Thursday night, which sounds like fun. I wish I could be there. I think I'm going to be out of town. So we'll have office hours this week. Please come and get help. And that's where this is. OK, so for Wednesday we're going to start. So the rest of this class is sort of a mixture of some lecture material and then a few chances for us to read some papers in the systems research area. On Wednesday we're going to do the first one of those. And this is a great example of a paper because the raid paper, I mean, academics sometimes worry about having impact. The raid paper spawned like an entire multi-billion dollar industry. So that's the best case scenario when you write a research paper. So instead of having you read that, I posted a link. You can tell you're reading the classic paper where it's been scanned in on some ancient, the scanned copy has clearly been photocopied like 62 times and then scanned. It's barely readable. So rather than reading that, look over the Wikipedia article about raid. And then I haven't read this paper yet, but this looks like a neat retrospective about raid written by one of their original authors of the raid paper, a guy named Randy Katz, super famous at Berkeley. So take a look at that. It's only a couple pages long. It looks like it's kind of a light read. So please read those before you come to class so that we can talk about the paper. And on Wednesday, we'll also talk a little bit about kind of how to approach some of the other papers that I'll assign for the rest of class. All right, any questions about logistics? Questions about FFS before we move on? Oh, and then we're handing out midterms. But you guys know that. OK, cool. All right, so FFS was a file system design circa 1982. Now we're going to fast forward to 1991. How many people were alive in 1991? Yeah, very few. OK, at least it looks like more than one hand up. 1982 was zero. So what's different about 1991? 1982. Anybody? So what's that, nine years? Nine years? Yes, good answer. I like that. That's very literal. So yeah, I mean, music has gotten worse. This is clearly a degradation. I and the Tiger is a great song. That other song is, I don't know, I don't want to talk about it. I grew up with this stuff. So now you can see how I've entered the state that I'm in. I had to listen to those same things. Movies are still pretty good. Silence of the Lambs, classic, classic film. I was actually just looking up Jody Foster the other day because we've been watching the Americans, and there's that scene where Reagan gets shot by Hinckley. I didn't realize that he had an obsession with Jody Foster. That's so weird. Anyway, the things are weird. The world is a weird place. Anyway, Silence of the Lambs, good film. Gandhi, has anyone seen Gandhi? Why? Did you see it in class or something? Did they show it in high school?  Am I allowed to do that? Can I just show a movie for the next month? Social network.  They're really informative, useful stuff. Clearly high school teachers are getting away with this, so maybe I should be able to. Silence of the Lambs? Hopefully everyone's seen Silence of the Lambs. Classic film. These are good movies. All right, but anyway, obviously that's not what we're talking about. What's changed about discs or computers during that time period? Well, things are getting better, right? Now, disc bandwidth is improving. So my ability to get data back and forth to the disc, the buses that I'm building to actually communicate with the disk drive, are getting better, so that's good. So I can get, remember that weird thing that FFS was doing where it was skipping sectors so that it can do continuous, I don't have to do that anymore because I can actually read and actually send stuff fast enough down the bus to keep up with full disc bandwidth. So that's great. Now, computers, one of the big technology trends that LFS responds to is computers are trying to have more memory, like 128 megabytes, right? I feel like you guys should just, you have a, should have a little card in your pocket that has, for each decade, kind of typical computer specifications, just so you can remind yourself about how far the field that you're studying it has come and how lucky you are, right? 128 megabytes, I think a single tab in Chrome consumes that much memory now, right? This is the entire machine. So this is, again, this is sort of like, this should remind you a little bit of your Sys161 virtual machine. That's a lot of memory, though. That was a substantial increase in the amount of memory in previous machines. Unfortunately, seek times for discs. So remember, discs don't, I remember seeing a great talk by the guy who started iRobot, whose name is escaping me right now. And he pointed out that a lot of the trends in computer science in terms of faster and faster processors and more and more speed and more and more memory come down to the fact that you can take a bit and you can make it smaller. You can cut in and deliver little pieces and each one of those things becomes a bit. So that's, for some reason, on some level, that's what we did for a long period of time. That's where we got some of these exponentials that processors wrote and that memory wrote. Is there an active shooter, or is there someone moving a thing out there? Sorry. I like the retroactive trigger warning. I don't know. It's the second time today I've heard random loud noises. So anyway, just trying to keep us all safe. Okay, so, but disks, disk seeks are not getting faster because we're making things into smaller pieces. So disk, what do you think has happened to disk seeks? I mean, a lot more memory in the computer, disk seeks are still what? Really slow. This is still, and the problem is this is becoming a bottleneck. So before the bottleneck might have been the bus or maybe the bottleneck was the amount of memory you had in the machine or something else. But as some of these other parts of the system are evolving, disk seeks are starting to stick out as something that's really a problem. And so people are starting to think about what can we do about this? Okay. So we can use the lot of, if we can solve the seek problem, right? So this is something we'll come back to when we talk about performance. When you try to improve the performance of the system, focus on the thing that's causing the problem. The thing that's causing the problem are disk seeks. So it can become up with the file system design that tries to avoid disk seeks. Oh, and the other hand, and the other thing that's nice about this is let's put all, maybe we can find a way to put all the spare memory that we have, right? All the spare memory of that 128 megabytes that you have in your computer. Maybe we can find a way to put that to use as well. So these are sort of the design parameters of LFS. And it's, again, responding to technology trends. I've got more memory, and I've got enough bandwidth to get things back and forth the disk quickly, but disk seeks are still really slow. So I'm trying to find a way to avoid seeks. So obviously, we've talked all semester about one of the ways that we wanna use this memory is as a cache. So I can use a big cache in front of the disk to soak up some of the disk activity. This is part of my goal, right? You know, and we know what this is. This is called the buffer cache. It's something we talked about last week or the week before. And this is something that starts to become part of file system designs. And LFS really counts on this feature of file systems, which is that I have a reasonable sized memory cache sitting in front of my file system, it's right in front of the disk, excuse me, that's going to absorb certain types of traffic. Okay. Now, so the nice thing is, as the cache gets bigger, its ability to soak up reads improves. So remember, reads are totally safe to cache for as long as I want to. There's no potential harm for caching disk blocks in memory if they're just being read from. So as disk blocks enter the cache, once the cache gets large enough, the number, this is sort of like page fault rate, right? The number of times I actually have to go out and read data off the disk starts to fall for a given workload. All of the files, all of the disk content needs to make forward progress start to fit inside the cache and I'm good. So this is sort of one, again, one design parameter of LFS is counting on a reasonable sized memory cache to soak up a lot of the read activity. But unfortunately, we still have to do these writes. I still have to do writes to the disk. Remember, I mean, changes that I make to the disk still have to make it to disk somehow in order to persist. So I have to do these. So this is sort of what LFS focused on. It said, we're gonna design a system that assumes that a large cache is gonna soak up a lot of the reads. We're gonna focus on reducing write activity. Okay, so now we've talked about, again, your minds are mired in the world of traditional file system designs. But try to reverse that programming and answer this simple question. So what's the best writes as possible and write them all out together? So ideally, I let writes accumulate in the cache for a long period of time and then I write them out at full bandwidth. So the more writes I can pack together, this is always true of any disk. The more operations that I can bundle together at once and send to the disk together, the faster the disk will go, particularly spinning disks. I think, I actually flash drives, it may not matter, but spinning drives, absolutely. The more I can tell, if I can, because the other thing the disk can do and I don't think we, did we talk about disk scheduling? When we talked about disks, Carl, I'm pointing to you. Okay, yeah, no, I don't talk about that because I think it's kind of boring, but the disk is, just so you understand this, spinning disks are also doing their own scheduling internally. So if I give the disk 10 data blocks to read from, the disk looks at that set and says, okay, I know where the heads are and I know which data blocks you wanted. I'm not gonna get them in order. I'm going to try to optimize the movement of the heads so that when I'm done with my operation, I'll have all the data you wanted, but maybe I'm doing one pass across the disk, for example. So if I give the disk larger and larger and larger sets of blocks to read, it can read them all in one pass and it can actually get them pretty quickly, right? Whereas if I send them one at a time, it's like running here, running there, running here, running there. So the more I tell it at once. Same thing with writes. Does that make sense? Okay. So, you know, and I can, so again, I wanna hold writes as long as possible to get the full benefit of the short seeks that I get to do with the law. If I start to write things back, if I start to do, well, sorry, if I start to do more of a write through approach where I'm writing immediately for whatever reason, whether that's for consistency reasons, then I start to lose some of the benefits. I still get some. But the benefits of FFS are greatest when I can do lots of writes at once, because lots of writes are run all over the disk are slow, lots of writes at once to one part of the disk are way faster. This is something else that we just pointed out. LFS inodes move around. And so locating LFS inodes is a problem. So LFS now needs more complicated data structures for finding files than other file systems did. I mean, look, the AXC4 approach of just having fixed locations for inodes is really nice. It simplifies inode lookup, which also simplifies path resolution because that's a big part of taking path names and translating them to inode numbers is doing that thing where re-directories map the directory component to an inode, read the next item. So having this Part B performant is important. LFS does a fun thing. It caches an inode map that stores the locations of the inode, but it also logs that inode map periodically. This is just another piece of metadata that LFS is allowed to log to the disk. And so to find an inode, I read my inode map in, again, that's probably in the cache most of the time, and I use that to map from an inode number to a block on disk that contains that inode. All right. Yeah, pretty much all of the file system data structures can also be logged. So all those things we talked about that we haven't really talked about very much, like the free block bitmaps and the inode bitmaps and stuff like that, the sort of basic file system internal data structures, I can log those two. Anything can be put in the log. Now here's the issue. So this is great. First time through the disk, everything works awesome. There are no issues. Then I get to the end of the log. So you can imagine starting the log at one side of the disk, one end of that flat array, and just I increase it until I get to the end. So what happens at that point? Do I tell the user disk is full, no more space? No? Okay. What happens at that point? Yeah. Yeah, just because the log hits the end of the disk, there's a lot of junk. There's a lot of dead stuff in the log. You know, if I made multiple changes to a file, I've got maybe multiple copies of inodes for that file that are sitting in the log, only the latest one is up to date. Everything else is wasted space. So this is this issue of what's called an LFS, log cleaning. And this is where we get to this controversial part. It's like one of these mirages that you can only maintain for so long. Like I can only, I'm trying to think of an analog, but I mean there's certain things in life where you can get away with kind of faking stuff for a while and then all sort of builds up and adds up and at some point you just have to go back and clean up your mess, right? So the first time through the log, awesome, everything works great.  Okay, so now again, conceptually you can talk about cleaning any portion of the LFS log, like the entire disk. I can clean the entire disk at once. Typically for performance reasons, LFS file systems don't work that way. They clean parts of the disk because what would be the problem of cleaning the whole disk? I mean, we haven't talked about cleaning yet, but cleaning sounds like it might be slow, why? What's slow? What part of the computer is slow? What does cleaning involve a lot of? Reading and writing to the disk. So yes, cleaning is going to be slow. So let's say I wait till I hit the end of the log to clean and I have to clean the whole disk. What happens? Like you're in the middle of trying to submit assignment 3.2 and your computer's like, sorry, I need to clean the log. Come back in six hours, right? Like you don't want that. So we try to do it in little pieces so that I can keep allowing the user to use other parts of the file system where I still have some space while I'm cleaning out other parts. So again, typically LFS will break the file system into chunks. Once one chunk of the log fills, I clean that part and then I, but I allow the log to continue into some other part that's already been cleaned. Okay, yeah, to some degree, yeah. I mean, I don't know what defragmentation in Windows does, right? Have we had this conversation already about defragmentation? I think we did, okay, sorry, I'm not gonna say it again, but yeah, other than move colors around on the screen, I mean, and generate a lot of disk activity. So it's possible, yeah, it's very, yeah. No, because I want to put everything at the end of the log, right? Once the log goes past a spot, I don't want to back up, right?  Stuff in the past, let it be until I clean it and then we'll come back. Yeah, good question. Yeah, I don't back up. I can back up for reads, right? I can pull stuff out of the log for reads, I have to, because that's where all the data is, right? For writes, you know, I'm very fixated on getting all of my writes into one spot so I can minimize this. Okay, so how do we do this? So here's an example segment. This segment contains some active data. Maybe the red stuff is file system metadata, like iNodes and lock bit maps and stuff like that. The green stuff is actual file data. And so this is like compaction. This might be a lot like defragmentation, actually. So I find the stuff that's in use, I write it all to the beginning of a segment and then I find other things and then I'm done. So when I'm finished, now what does this require? Based on the diagram. What's that? Yeah, I need two segments to do this, right? There's probably clever ways to do it in one but that's usually not what they do, right? Usually what I do is I find a completely clean segment or a mostly clean segment. I can clean into the, like this could have some stuff at the beginning of it and I could write it to the end. But essentially what I'm doing is I'm taking one segment that's fragmented. I'm walking through the entire thing and just rewriting all of its data to the end of another segment and when I'm finished, everything should be pretty compact. Yeah, I don't remember actually, I have to say. I think it was supposed to be two different types of data, like file data and inodes and other sort of file system metadata. So when I'm done, I have a clean segment in this case and I have another segment that's mostly clean. So I could keep logging into the end of this one as well. So this is where LFS starts to go from being an incredibly good idea to being a debatably good idea. And this is what people debated about LFS for a long time. So if you look at the debates and the arguments that people had over LFS, which got fairly heated, I'll show you some email exchanges, they're pretty awesome. Then this is the part of the systems operation that was difficult for people to reason about and could account for big performance differences. So for example, let's say I run a test on LFS, a benchmark on LFS, and it never does any cleaning. The performance is gonna look awesome. But the problem is, again, it's all a mirage because as soon as I loop around and actually have to start cleaning, remember the first time through the disk I can get away with not cleaning. At that point, I've gotta clean all the time. I've always got dirty segments that I've gotta be compacting, or frequently, let's put it that way. And so on some level, the first time I go through when all of my segments are clean is like this grace period where everything works really well. And as soon as I have to start doing cleaning continually, performance is going to degrade. How much it degrades is what people debated. So, and there are also lots of questions about integrating the clean operation into the rest of the LFS system. So when's a good time to run the cleaner? There's no right operations, okay, let's extend that answer. It's a good time to run the cleaner. Yeah, okay, can we like merge these things? Well, there's no operations at all. The disk is quiet, right? User might not be using it, but I might be running like a virus scan or something like that. So I want to find a period of time where there's no disk activity.  But on systems, so this is one of these questions that started to cause people to debate how good LFS was. So on user-facing systems, you can make arguments that there are long periods of idleness. So for example, your smartphone says like hours a day. I mean, maybe it does, unless you're one of these people that has a serious smartphone addiction. I sat next to, I was on a plane recently for like seven hours. And I sat next to someone on the plane who the whole flight simultaneously watched a movie and played this like game with like little colored, like I don't know, fruits and stuff like that. Do you guys play these games? What's that? I don't know what it was, but this person played this game for like seven hours straight. I'm not kidding you. I kept looking over me like, are they gonna stop? And they never stopped. And they were also watching TV at the same time. It was amazing. I was like, I don't know what your problem is. I hope that this is the only time in your life that you're playing this game.  But anyway, it was strange. Has anyone ever? Well, anyway, I won't ask. I don't want to know the answer. It just seemed like a weird game to get that into. Anyway, I don't play computer games so I'm not familiar with the stuff. So anyway, so unless you're that person, your smartphone probably has some idle periods where it's not doing anything. And I could sneak this cleaning in there and get some useful file system work done. But there's other types of systems that are not designed to be idle. So big cloud service providers do a lot of work to make sure the machines are not idle. It's idle machines are a waste. You talk to people at Google, they'd be like, we paid good money for that machine and we wanted to run as hard as possible until it falls over and dies. That's what it's for. If it's sitting there idle, we did something wrong. And of course, you can't always make this happen. There are big diurnal shifts and things like that. You can imagine the Google's data centers in North America are a lot quieter at night than they are during the day. But to some degree, once I buy a server, I paid money for that and what I purchased is a certain amount of computation over a certain number of years and I wanna put all of that to work. If it's sitting there doing nothing, it is not useful. So on some instances that don't idle much, this maybe, it's harder to find times to clean. What size segments do we clean? So here's another interesting trade off. Large segments give me the most bang for my buck. I get the most space left when I clean a large segment. But it also takes the longest period of time. So if I'm trying to do something while the user is at home, if I do a large segment that might take an hour and the user comes back after five minutes, I'm in trouble. On the other hand, small segments increase the probability of finding this glorious segment that the log cleaner dreams about at night, which is the segment where everything is dead. If I can find a chunk where everything is dead, cleaning it is a trivial operation. I don't have to write a single block. The amount of work I have to do to clean a segment scales with the amount of live data inside the segment. I shouldn't say it scales. I still have to read the whole thing, but the amount of rights I have to do, the amount of data that needs to be duplicated is a function of how much live data there is in that other segment. The biggest problem with the cleaner in terms of the systems community, reasoning about the performance of LFS was that the cleaner is very workload dependent. So you can imagine thinking about this, right? Really, read heavy workloads don't generate a lot of right traffic, and so the system doesn't need to be cleaned as often, whereas write heavy workloads or other types of operations that generate any kind of right activity. So for example, renaming files. When you guys start to look at file systems benchmarks, there's all these weird benchmarks, like benchmarks that take a file and rename it millions of times. This is a way to put stress on a particular type of operation that file system has to perform. Renames in LFS potentially create right traffic because I have to re-log the inode, the directory that the inode was in, and things like this. So again, this is one of those things where two different groups could take two different benchmarks, run them on similar LFS systems, and one would claim LFS is awesome, and the other group would claim LFS is a piece of junk. And people did this. So there were also issues with reads. I'm almost done with all these caveats, right? If the read, if the cache doesn't, let's say the cache doesn't soak up as many reads as you were hoping. Let's say there's still some substantial read traffic that's going down to the disk. What does EXT4 do? EXT4, I know EXT4 sounds boring now. I've got this static layout. LFS is so much sexier and cooler. But what is EXT4? Remember the cylinder groups in FFS that made their way into EXT4? What is the goal behind doing that? Why divide the disk into these smaller pieces? What does that help me do? I know I'm going to have to do some seeks to get from the inode to the data blocks and things like that. But those systems that do more static file systems and do more static layout, what are they trying to do? Yeah. You try to keep stuff close together. You don't make these promises that all the writes, like these pie in the sky promises, like all the rates are going to go at the end of the log. No, you're like, you know what? I'm just going to do average, right? We're just going to try to put stuff sort of in the same place on disk, which is fine. The problem is, let's say LFS, again, let's say I have to do a lot of reads on an LFS system because the cache isn't working as well as I thought. What problem has LFS created? In order to get this great write locality, what does LFS do? Yeah. Four sakes reads me. Well, four sakes read locality, remember? Like, stuff can be anywhere on disk. There is no high level planning about where things go in the log. If I have to start doing lots of reads, I could be seeking way back to the very beginning of a log if I have to get something that hasn't been modified for a while. So I have totally abandoned any attempt to keep reads close together because I'm pretending that the cache is going to soak up all this for me. If the cache doesn't do as good a job as I wanted to, blocks can be anywhere. And so all the improvements I get by being able to stream reads, sorry, stream writes at full bandwidth to one spot may be lost if I'm suddenly having to go and run all over the place to grab reads to fill the cache. So this is another questionable issue. And again, something becomes very workload specific. If my workload fits inside the cache, I might be good. If the workload spills out of the cache or touches a lot of disk blocks, then I start to see problems with reads. All right, so just to sort of this fun little timeline, so 91 was the original paper about LFS. Now there were also sort of two camps here. There were the FFS guys that were building kind of a more standard design file system.  We can do a way better job with this clever trick. And so these groups went at each other for a little while. So Margot Seltzer was involved in this, the person who also led to the creation of the OS 161 torture chamber you guys are enjoying. So as soon as it happened, it's over a period of a decade, LFS would publish numbers, FFS would make some improvements. They would publish numbers claiming they were superior to LFS. So, and these are like fighting words for scientists, like poor analysis.  What else is there? It's all analysis, right? If you don't do analysis, well, you're not a good thinker, you're done. Yeah, oh, they also implemented it badly. I like that. This is great. This is like, this would just ruin my entire world.  Nice work. And you can't choose benchmarks, but that's just like a, that's just the cherry on top. So now 95, another part of paper by Margot. So, and now we're getting a little bit more specific here. So look at the critique, LFS offers superior performance on small file rights for large files. The performance is comparable. If I tune FFS properly, so one of the arguments I think the FFS community made was when the LFS people were comparing against FFS, they didn't do a good job of tuning the FFS file system so they weren't seeing representative performance. And so now I'm seeing a little bit better than LFS and comparable. So in the best case, I'm a little better at LFS at the thing I was tuned for and the worst case, I'm comparable. So all of these magic improvements that the LFS file systems were promising are not happening. So, and then you know, I'll just describe this paper as misleading in many ways. You lie. Okay. I need questions about LFS, this is kind of a fun. Now there are definitely aspects. You know, I think there are some features of large structure file systems that are made their way into modern file system designs but not really whole hog as far as I know. Yeah. Clusters of logs are adjusted based on workload. Yeah. It's hard though, right? Because to some degree, the two things are sort of intentioned with each other. If I'm gonna write everything into one place, then I lose a lot of control over where things end up. And that's the control that these other file systems are trying to get at. So even more exotic FFS variants, things that use fancier data structures, I still, to get good performance, where those data structures are on disk starts to matter a little bit, right? Now, if you think that now there are some, I think I had an exam question a few years ago. There was some upswing of interest in LFS for flash file systems, flash-based file systems, simply because LFS can amortize writes and for certain types of flash devices, that's important. So there are flash devices where I can read from like a 256 byte chunk, but I've actually wanna modify a byte. I have to erase an entire 32K sector. So that makes writes extremely expensive. And so being able to amortize writes in the same place in an LFS manner might be useful. I don't know. I mean, it would be a great thing to look up. I mean, this is like, and this is LFS vanilla. I suspect that some of the same people worked on this project for a while and I suspect that there were variants and offshoots of LFS that tried to solve some of these issues. But I think on some level, the locality thing is really fundamental, you know? Do I have a top-down design that allows me to do sort of best case or average case performance given certain assumptions about files? Or do I go forward and try to put all the writes in one place and then lose control over where things are on disk? Yeah, it's a good question. Yeah. Yeah, that's a great question. So I'm assuming that when you're doing that process, I need to know things like, where is the current inode for this file? If I read a data structure and it's an inode and it's not the current inode, then I don't understand. I'm probably something similar for data blocks. Yeah, yeah, that's a good question. So you were saying I could jump, I could jump through. I mean, I lose some of the benefits of being able to stream at full speed because I still do have to seek from place to place. So the seeks, you know, potentially I might have to seek across large amounts of active data. So that becomes an issue. I don't think other than that, there's any real reason that's not done that way. They could certainly keep track of where things are in the same way. Yeah. Yeah. I don't know. I suspect the answer is yes. I don't know exactly what techniques were used, right? Yeah, that's a great question. How about that school stuff? Okay, I have a couple notes about reading papers. So we will consume the next minute of class. So it's time for a meme. Now that I'm saying it properly, thank you.  Why would I ever read a research paper? And that's fair. And one of the things I sent you guys to read for Wednesday is a Wikipedia article, which is probably better to read than the original rate paper. It's actually way better because the original rate paper was original, right? I mean, it hasn't aged very well, right? Papers don't age well. On the other hand, big software companies, even if you wanna have some understanding of how things at Google work, how do some of Google's big cloud-scale services actually work, a lot of that stuff is an open source. So you're not gonna be able to find the source and read it. The documentation, in many cases, is a research paper. So things like Bigtable and all of Google's flywheel services and a lot of things that Google works on. Periodically they publish papers at scientific conferences that give you a glimpse into some of the design principles that Google is applying to build things like this. Or Facebook, VMware, does a certain amount of this. Microsoft publishes stuff all the time. So this can be a neat way to learn about these sort of systems that are actually out there in the real world solving problems at a scale that is almost unimaginable. And reading code is, I mean, reading code is something you guys should all get better at, but I understand it's pretty tedious. Like if you had to read the entire code for some of Google's services, first of all, it would take you several months. And you don't necessarily get a nice view of the high level design principles that can inform your own software practice. So the cool thing about computer systems, I mean, maybe you think computer systems is sort of like archaic old stale discipline. But what's neat about computer systems is it's a story about software responding to technology trends over long periods of time. And a lot of the papers we're gonna read have that flavor. So this is why research is ongoing in this area is because technology changes, hardware changes. And so as hardware changes, we need to respond by building different systems. So end of moral loss scaling, does anyone, everyone understand what this means? The end of moral loss scaling, how many people feel like they know that? Okay, I'll post something on discourse. You guys should all understand this. So at a certain point in time, we were writing this exponential and things were getting faster and faster and faster. And at some point, that stopped. Do you guys remember that stopping? You guys were alive when that stopped, maybe you were little. The Pentium 4 computer that maybe you had, did anyone have a Pentium 4 other than me? That was sort of like the end of that era. Do you remember the heat sinks for the Pentium 4? Like that big, right? They're not that way anymore, right? Has anyone installed like an i7 or an i5 recently? Why is the heat sink like this little thing? It's because at some point we ran out of the ability to make transistors smaller and smaller and smaller. And so we started using concurrency to make things faster. So now your computer has 32 cores. It's a very, very different design trend and produces the need for different types of software. Flash, obviously a very new storage technology, cloud computing, battery powered devices, fast networks everywhere. Again, people are irritated now when they can't connect to things and get the airlines are the last ones that we're gonna have to come after and beat into submission so that we can get connectivity. That's pretty much it, right? I mean, that's one of the only places left on earth. So the papers we're gonna read are going to be designed, sorry, left on earth that doesn't have internet access for free. So the papers we're gonna read are responding to these technology trends. So please, on Wednesday, read that short retrospective by Randy Kass. I'm sure it'll be interesting and look over the Radar Wikipedia article and we'll talk about it right on Wednesday.