introduction about myself. I worked as a senior engineer at LinkedIn and before LinkedIn I worked for many different companies but my very first Amber experience comes from Yahoo back in late 2013's. After Yahoo I worked for Apple where I didn't do Amber stuff but that was not for long. The love for Amber pulled me to LinkedIn and here I am working on a lot of cool Amber stuff.  I love playing guitar, not a pro but I can still play a little bit. Cricket fan. For those who don't know what cricket is it's a sport which is very similar to baseball and I myself play cricket and it's very famous back in India. Off late I have this craze of building my own custom mechanical keyboards with new switches and custom keycaps etc. If any of you have any cool ideas feel free to reach out to me now watching movies and who doesn't write. Okay enough said about me now we should focus on the main topic. So today's agenda would be something like this. I plan to cover the journey of Amber JS in terms of its versions like version 1.x, 2.x and 3.x. Going forward I'll be talking about Amber Octane and its native concepts like some core concepts like native classes, glimmer components, templating and octane track properties, modifiers and decorators. After that I'll be showing you a consolidated comparison of the classic syntax versus octane syntax and at the end I will leave you guys with some documentation and references to migration guides and some information about amber octane blogs etc. Okay so let's start with the journey of amber but before that let's take a step back and think about around 2013 times or so. Back in that time we didn't have a lot of options in relation to JavaScript frameworks as we have today. The most evident options during that time were back both JS, AngularJS, KnockoutJS etc. While these were great they still left the decisions of configuration on the developers. It was during that time when one framework emerged which was built on the principle of the developers should not care about the configuration stuff. They should only focus on app specific stuff which means the developer's productivity would be increased because the framework would do the configuration stuff. The name of this framework was AmberJS. Amber has come a very long way ever since 2013 and let's see how it started with 1.x. So since 1.x was the very first version of amber it came with a lot of goodness. To start off with is convention of a configuration. This was a whole new mental model shift for any developer who comes from a different javascript framework world. I will give you my specific example. I came from backbone JS background and when I started working on amber it was very different for me but as and when I started working on amber I started feeling the difference and I started appreciating it more and more because I started feeling more and more productive. So that was when I started falling in love with amber. Also amber provided inbox built-in routing capability. So that means you don't have to write your own routing layer or get it from outside or something like that. It was already ready for you which was a huge thing. Also we had support for amber data that took care of the state of the application etc. And 1.x was built on new driven architecture. What I mean is here you can see the route controller view. That was the path that or that was the architecture that we followed at that time. And for anybody who wants a refresher of how we used to write amber views a sneak peek is in here where computed properties were written somewhere like we annotated it with dot property and observers were written something like very added dot observes to the properties. Also we had the capability of two way bindings which was great because at that time it was really really to have this ability. Attribute bindings this was really good because let's say if you had a backing JS class for template like a component with a template or view with a template and you wanted to associate the attributes value dynamically. So with bind attr syntax you can associate your components or views property directly on the template. So that was pretty good. Okay, so this was a good amber 1.x was really taking off and it was doing great and complex web applications were starting to love it already. But the community always believed that there's always room for improvement. So the lessons that they learned from 1.x frameworks that were evolving during the time when 2.x was being planned out, they embraced all of these things together and added things in the roadmap of 2.x. So here are the few things that landed in 2.x. Instead of view driven now we are component driven. This was because the web community itself was going in that direction and amber was hand in hand with that. So here you can see route controller component is of new way and again a sneak peek of how the syntax changed. The computed properties now amber.computed and observer is now amber.observer. Also we had the glimmer rendering engine in 2.x which was a very very good thing. It gave a dramatic improvement in the re-rendering of the application. It's pretty good and then we had better binding attributes like you remember in the previous slide. I showed you the bind data syntax like you can see here as well in 1.x. But this posed some issues where it became an overhead and it was confusing and so on. With 2.x it became more simplified where you can directly add a property from your JS class onto the template without adding the bind data. So that was nice. Also the template scoping was improved. Let me show you an example. Here we have 1.x and 2.x again a comparison and I want to specifically talk about the second example in the 1.x section where we are iterating over the posts without any in. So here I am doing each posts and whatever is inside of that iteration loses the outside context or it doesn't have access to the outside context.  So as a workaround the above example like each post in posts allow you to do that where the post would act as the inner context and then you can easily access out your context. But in 2.x things are way more simplified and standardized where every time you want to iterate over a property you use the as functionality like each post as and then pipe and whatever the parameters here it is post. So again the post would be the inner context and you can get the outer context as in the new wish. Also while 2 way bindings was great it did impose some issues when your app started becoming more and more complex and more and more bigger. By that what I mean is let's say property a updates b, d updates c, c updates a again. So this kind of infinite cascading started becoming a problem and that's why we started embracing the data down actions of approach. 2.x was not just about all of this it was way more than that. In 2.x itself we had a roadmap towards things like HTML syntax for component invocation routes to drive components like these are angle bracket invocations and routable components if you might have heard already. That brings me to my next topic which is Adverse 8.x. This is very exciting it has a lot of things I'll mention a few of the things here but there are many more things than this in 3.x. To start off with it's cleanup cleanup cleanup. Why am I emphasizing on cleanup so much because a lot of cleanup has been done with this version where a lot of private API's deprecations everything has been cleaned up in that way the code the framework becomes more manageable, maintainable and more cleaner. Also we have removed support for Internet Explorer 9, IE 10, Phantom GNS and also Bower. Points I will be explaining more in details in the coming slides so I'll quickly skim through these. 3.x has the ability for supporting native classes, glimmer components, angle brackets invocation, track properties, modifiers and decorators and lots of documentation which is a huge thing when you are moving forward with a such a huge change so that is a job well done by the folks. So what did we learn? We saw that framework emerged in 2013 and released 1.x and then it improved more in 2.x and then it improved even more in 3.x and then here we are with the first new edition and we are obtained. So you must be thinking why is this a new edition and why is it not a new version. So you can think of an edition as something that represents a shift in programming model due to new features and concepts added in the framework. Like for example here we have a bundle of new features, a lot of documentation, a lot of tool links like code mods etc. Everything bundled together so that's why it's a new edition. So let's take a sneak peek of each or some of the key concepts of AMBER octane itself. Let's start with native classes. So AMBER relies very heavily now on native features of JavaScript. When I say it relies on native features I mean there is very less framework-specific code and because there is very less framework-specific code the overhead on building the framework itself is very less. That means increased performance and AMBER is known for its steep learning curve. I mean I've seen many folks who have spoken to an editor, I did some blogs as well, where people choose not to use AMBER because of its steep learning curve and I can imagine why but I think with octane that won't be an issue anymore because since now we have again I'm repeating less framework-specific code so you don't have to learn a lot of things out of the box. So it gives you a smooth learning curve experience. Also because again the same thing it's more native stuff, it is more aligned with the JavaScript community and you can share code more easily and there is no more dot gets. Can you imagine like me, myself, personally I used to feel like why do we have this like this dot get foo, this dot get bar. I'm sure a lot of you must be sharing the same feeling like I am right now. So there's no more gets. You just directly do this dot foo, this dot bar like you usually do to property in your JavaScript currently and of course it is clean and easier to read. In other words, let's see an example. So here in this example you see on the top we have this classic AMBER object syntax and at the bottom we have native syntax. Now if you see the main difference is how we are creating or defining the object on the top we are doing AMBER object dot extend but at the bottom with native class syntax no more AMBER objects. You can just say class person, that's it, you're done. Also in the classic syntax you can see the computer property full name is created by passing in the property's first name and last name and we are invoking the first name and last name is this dot get first name, this dot get last name. Whereas in native class we are just writing the native getters of JavaScript and then just adding a little splash of computer decorator on top of the native getter, that's it, you're done, you don't have to do this dot gets anyone's, this is your first name, this is your last name. So that's pretty cool. This is a little bit about native classes. Now let's move on to Glimmer components. In my personal opinion this is one of the biggest wins in AMBER's journey so far because AMBER's component library was pretty old and with this new component library it's a great refresher. Not only does Glimmer components take all the goodness of native classes but it also makes it more simplified component library in AMBER. Let's talk about this following example again. So here Glimmer component is defined by importing Glimmer component instead of AMBER component as you can see and instead of exporting default component dot extend like we do usually in our classics syntax we just do a native class and then extends the Glimmer component. Also there are now fewer hooks and properties that is a great thing like previously we had 13 lifecycle hooks and 29 properties. Now can you imagine a new person coming and joining working on AMBER and you say hey if you want to be productive if you want to start working and make an effect you need to know 13 lifecycle hooks and 29 properties that would be immediately daunting and overwhelming for anyone who comes across this. I think with this with Glimmer components no more fear because now we only have two lifecycle hooks and three properties. Also you must be already knowing that every time you create a component in the classic component world a wrapper is always wrapped around your template like for example here the number of guests is just a value but when it renders on a DOM there'll be a div or in this case there'll be a label that will be wrapped around it. This might become confusing and it has actually I've had many people come to me and ask me hey I'm just rendering this template what is this new div that is being wrapped around it I'm not even adding it in my template and then I had to explain them the reason why but with glimmer components it's what you write is what you get or what you see here you can see very well the bottom when I'm wrapping it around with the label tag itself so I'm not doing anything special in my class rather I'm just adding everything in my template itself so that is amazing also it's important for us to identify what things are local in our component and what things come from outside with the classic components there was no way to do it because like you can see here this dot number of guests and this dot max guest they both are this dot and this dot there's no way for you to know which is a current one and which is and which is a local one and which is coming from the parent context but if you see in the glimmer component case this dot r's dot max is so I know now for a fact that max guest is coming from somewhere outside so it's from a parent context so that is a great way to identify what is the point of origin for any particular property so this is a little bit about glimmer components for you guys now let's talk about templating in octane now there are a lot of good things that have come in in the world of templating for octane let's start with angle bracket syntax now this angle bracket syntax is a dramatic simplification of api like let's say for example in here in this example we are seeing in classic templating we are invoking the component using the curlies but in octane we are invoking the component using the normal html syntax which is angle bracket so now it's very easy for me to understand which thing is coming or which thing is a component which thing is a helper which thing is a property whereas before it was very hard for me to understand right when I see it from the first go also with angle brackets we have something called as named arguments named arguments are a way where in the previous slides we saw this dot r's it is similar to that but in the templating side where it helps you understand where is the origin of your property coming from for example in here on the top everything has no context whatsoever you don't know where the employee name is coming from you don't know where the employee id is coming from but in the bottom it's very clear like for example the employee id when i'm prefixing it with the at the rate symbol i know that it is coming from an outside context so now i can understand that okay this is a pas de nard and not something local to my javascript class also a good thing to keep in mind is on the left side like on the classic templating side on the left side we have name mid and add employee now it's hard to understand whether this thing is part of the html attribute or is it a part of your component but with octane it's very easy because now everything that is pertinent to your component will be always prefixed with an at the rate like we have right now in here also we have required this in the templates like we have at the rate symbol for something coming from outside we should know what things reside locally in your backing class and that's where this dot employee name becomes very handy before there was no way for me to know but now i do so it's a great way to make things very specific and very concerned and very clear so this was a little bit about templating in the octane world now let's move on to track properties my personal favorite the reason being the simplification that it brings on the table but beneath it is very strong let me start by going through the syntax real quick here you can see instead of computer property when it importing on the top i am importing a track property from glimmer tracking library then any local property that i want to listen to or that i want to depend on i will mark it as track like i'll prefix it as as track in the track syntax the magic happens on the top in classic syntax you can see there's a computed property that is depending on force name and last name whereas the bottom i'm not doing anything it is a simple plain javascript getter but the magic happens now where any property that is marked as tracked will make sure that any getter that uses that track property and if there is any chance where the track property changes it will make sure that it recalculates the getter and sends it back to the browser so in a way the functionality or the behavior would be similar to computer but this is way better just imagine when you are writing a huge component class and you have several properties in your class and if you have to depend on few of the properties here and there for multiple computer properties there'll be so many computer properties in your class and it will be not very clean but with this you can just mark those or prefix those with track and just write your regular getters that's all so very clean and concise as well also no more sets like in native classes we have no more gets now we have no more sets so no more this dot set full set bar or for example in here this dot set count equals this dot get count no longer all this confusion just do what you regularly do in your JavaScript this dot count equals this dot count plus one and you're good to go so that's amazing so these are a few things of track properties how it works and how it behaves now let's move on to modifiers and decorators so what is a modifier you can think of modifier as functions or classes that are directly applied to the template like it's applied directly to the element itself like you can see in this example on the button tag i'm adding the on modifier itself which is just doing some usual event handling with this jordan increment now you must be thinking why do i need to do this i can currently with my dick insert element hook i can associate my event listener and i'm all good to go but just think about it you can do that easily with your top level element or the root level element for your component but what about the child elements you could still do it but it will be a bit messy instead of that modifiers allow you to target specific elements so you can go in whichever elements you want to associate things with you can just add the modifiers in there directly and moreover it also cleans up the code for you and it also i mean cleans up the state and registers if any parameters ticked so that is a great thing and they are easy to reuse so there are a lot of benefits of using modifiers or not using them now let's move on to decorators what is a decorator now you can think of decorators as something that enhances the functionality of what it is prefixed on for example here you can see the increment function and decrement function has action decorator prefixed now just the increment and decrement functions are doing the job of incrementing or decrementing the counter but with action decorator it's doing more than that it understands that oh now this function is becoming an action handler rather than just being one function so that's how it enhances the functionality so it's kind of an abstraction for the developer but it adds a lot of value so this was a little bit about modifiers and decorators now let's move on to the meat part now we will do the comparison between the classic and the obtained syntax now here on the left very very left side you see there is a custom off label that's what we are talking about right now it is invoked using the curlies like you can see in the bottom with toggle label and I'm passing in few arguments and in the right you can see the actual implementation of this component all this is good this is classic component you can see a lot of things happening in the component as itself when I move it to obtain this is how it looks like now the template specific stuff is in template the component specific stuff is in component and you can see it is invoked using the under bracket syntax let's go through it one by one so right away you can see that the import is different on the left on the classic side we have component under component and on the right side we have glimmer component next we are no longer doing the classic component syntax like on the left we see component.extent but on the right we see native class syntax and then we are extending from glimmer component then no longer implicit wrappers on the left I am overriding the tag name with label but on the right if you see at the right bottom the label element is being enclosed in the template itself also any template specific stuff goes to template itself on the right in the obtained part but that's not happening in the classic we are still doing attribute bindings and class name bindings in the JavaScript class also a few things to note on the left side in the classic syntax we have a property called is awesome and a label value of computed property that depends on that and then we are invoking and using this .get but on the right instead we are prefixing it with tract the is awesome and then we are just using a normal regular getter and no longer this .get it is just this .get awesome next thing to check is on the left we see that we are depending on a property called value but I don't know really where this value is coming from but on the right it's very evident that it's coming from this .r this .value that means coming from parent context so easy to understand also on the left we can see that we are handling the click handler on the left by writing the click handler in the classic syntax but on the right we are using the on modifier to make sure we are hitting on the element itself like registering it and then handling it using the actions decorator so this was a little sneak peek for you to give you guys the feel of how it was before how it is now how simplified and how um good the separation of logic has happened now and I hope it gives you a good understanding of what is coming forward as promised here are a few reference and guides where it would help you to migrate your current existing amore to octane like with these code mods we have a lot of code mods in this link that you can follow the folks have done a lot of hard work and it is easily seen in there hopefully this will be a very smooth transition for you guys also if you have any questions about migrations or migration order or something like that amber atlas would be the right place for you to go because it will give you the right amount of information for you to migrate very smoothly and the rest are a few random information for you to dig down more deep into octane and its key concepts and deep understanding about each one step so this was it guys thank you very much for patiently listening to my talk and I hope all of you are as excited as I am for octane and I'll see you writing octane right away thank you


In 2021, actor-screenwriter Dionne Cole, who's well known for his role on the comedy series Black-ish and it's been off-grown-ish, purchased his current home in Los Angeles for $5.7 million. This mansion spans over 7,500 square feet and is built in an Italian villa style with luxury features like an elevator, resort-style swimming pool and much more. Considering his multi-faceted career, Cole has earned himself some stunning places to live over the years. Prior to this place in 2019, he sold a mid-century property in the Woodland Hills neighborhood as well. Before we get into this video, I wanted to point out the gorgeous bracelets that I'm wearing from Otter Spirit. They've been the perfect accessory for the New Year, bringing all the positive vibes. These bracelets are not only authentic grade A gems but they also contribute to a good cause. I'll share more about my bracelets at the end of this video as well as the discount code so stick around. Known for his sharp wit, infectious humor and undeniable charisma, Dion's lifestyle is full of laughter and success. Actor-screenwriter Dion Cole has most recently appeared on screen starring as Alfonso in the 2023 film adaptation of The Color Purple. But he's well known for playing Charlie in the ABC comedy show Black-ish and it's been off-grown-ish. Formerly a member of the writing staff for the Tonight Show with Conan O'Brien, Cole grew successful as a writer, actor and comedian. At the heart of Dion's world is his kingdom of laughter. As a seasoned stand-up comedian, Dion has concrete stages around the globe, leaving audiences in stitches with his razor-sharp observations and anecdotes. From sold-out shows to late-night TV appearances, Dion's comedic timing knows no bounds. When he's not busy entertaining, even Dion needs a place to unwind, so it makes sense that he's rewarded himself with some stunning California real estate over the years. In 2021, Dion Cole upgraded to his current living situation, paying $5.7 million for a coveted estate in the Encino area of Los Angeles. Approximately six months before selling a Woodland Hills residence for just over $1.5 million, Cole next invested $2.5 million into a slightly larger 5,800 square foot home and the Coldwater Canyon area of Studio City. After flipping and listing that home, he decided to really embrace opulence with his current mansion, a grand 7,500 square foot of boat that's a serious upgrade from his former places. Veteran stand-up comedian, writer and actor, twice nominated for an Emmy for his writing on The Tonight Show and Conan, hidden joy just over an acre of hillside land and true privacy and peace on his current property. Tired as an Italian villa, Dion's three-story home is a hidden gem, shielded from view by a lengthy gated driveway. Boasting seven beds and seven full and two half bathrooms inside, his home features a double height entrance with an ornate, wrought-iron staircase, a majestic formal living room with a carved stone fireplace and an expansive dining room tied together with an epic chandelier overhead. A covered patio off of the dining area provides amazing views extending all the way to the San Fernando Valley Mountains. Dion's property boasts an array of impressive features, including a spacious eating kitchen with modern appliances, a family room with a fireplace, a home office and a media lounge with a walk-in web bar. The top floor homeowner suite occupies its very own wing and offers up an area with a morning bar, a beautifully carved fireplace, a private terrace, two fitted walk-in closets and a spa-style bathroom set with polished marble and a jetted garden tub. According to Paul Salazar of Hilton and Highland who handled the mansion, the property's outdoor features, like the illuminated tennis court, had attracted a ton of interest throughout LA. Salazar highlighted how rare these offerings in Encino were, emphasizing the property's unique combo of an expansive acre, a tennis court and its breathtaking views. He said about Dion's home, this is one of the best-viewed tennis courts in LA. Certain buyers specifically seek a tennis court property and we attracted interest from buyers across various locations who coveted a premier tennis court property like this one. To appeal to today's modern home buyer, Salazar took strategic steps like including a fresh interior repaint, moving outside the estate's grounds off a mature landscaping, a secluded swimming pool and spa surrounded by high hedging for ultimate privacy as well as that tennis court, which is situated on a high plateau offering panoramic views. Dion Kohl's lavish life is on full display in this luxury home where entertainment and sophistication seamlessly intertwine. Now let's take a step back and look at the former homes of Dion Kohl starting with the one he lived in just prior to that Italian villa. Located in Studio City, $2.52 million a few years back. He listed it more recently in 2021 for the increased price of $4.1 million dollars. Constructed in 1994, Dion's former home has undergone extensive updates over the years, but its most remarkable feature, the breathtaking views spanning the San Fernando Valley remains timeless and these are accessible from nearly every room. Boasting over 5,800 square feet of space, the two-story home is made up of six bedrooms and five bathrooms shielded by towering hedges and a gate. The back of the half-acre hillside lot also ensures privacy on the deck where you'll find a pool and transparent railing. Stepping into the home, a double height entry welcomes visitors with a dramatic staircase and elegant chandelier. The interiors seem to seamlessly connect the living room, the dining room, the family room and kitchen, all of which are full of natural light thanks to statement windows that frame the views of the valley. The kitchen, which has a bright and white aesthetic, both stainless steel appliances and a sizable island with seats. Additional dining space is found in the breakfast nook, complete with French doors leading to the yard. Dion's one-time master suite offers a private balcony, peninsula-style fireplace and a spacious bathroom decorated with marble counters and floors. There's even a games loft elsewhere in the home while in the office with a separate entry presents the possibility of turning it into a recording studio. Finally back in 2019, Dion parted ways with another charming home, this time located in the Woodland Hills neighborhood of Los Angeles. This mid-century style abode sold quickly after two months on the market for $1.56 million, a comedian was asking just above at $1.6 million. Acquired by the comedy writer and old-spice spokesman over three years ago for $1.23 million, this single-story 1960s property sprawls almost 3,800 square feet. There are also four bedrooms and 3.5 baths. Protected by locked wooden gates, the courtyard entry while minimally landscaped offers privacy. Once you enter the home, a tile-floored foyer leads to a wood floor living room featuring a built-in surround sound system. A double-sided white marble fireplace is shared with the dining room, seamlessly connecting indoor and outdoor living spaces through picture windows and glass sliders while the space's white walls and large windows create an open and airy ambience. A nearby den of comedy to pool table, the kitchen shows updated stainless steel appliances and a separate breakfast room open to the backyard. The house is smartly designed with three guest bedrooms set on one side including an en suite and a family room. Then the master suite occupies a private wing on the left side of the home complete with sitting room, office, fireplace, and glass sliders to a private patio. The walled backyard boasts various sunny and shaded patios, a swimming pool, and behind a tall chain-link fence, a well at basketball court. While Dionne ultimately sold this home, the enclave provided the perfect backdrop for celeb-style privacy as well as working from the comfort of your own residence. After checking out Dionne Colt's homes, that will bring today's videos to a close, but before we wrap things up, answer this question for me. Would you prefer a one-level, more open concept home or a home with multi-levels throughout? Let me know in the comments below, don't forget to hit subscribe and turn on your notifications so you never miss a video. Follow me on Instagram to chat and I'll see you all in another one. Bye!


They really do. They do indeed. Put on a great show these two fellas. But Woffendern, this is a massive plus for him here. It is indeed. Smarslik is the man in the red helmet colour. He's come to that first turn ahead of Woffendern. Woffendern's trying the outside run. Now he's going to pass it. He's going to switch up the inside for Woffendern. Lovely. Ah, they are going for it. Woffendern in yellow. Now Smarslik tries the inside run to return the compliment. But Woffendern holds on right now. Let's up for the night so far. Now Smarslik coming back for more. Wheel to wheel down the back straight. And Smarslik has got the speed. He has indeed. Didn't last too long. But don't rule out Ty here. He's going to come back for a little bit more. He's got plenty of speed. I think he can even hang on to the handlebar. I think potentially we can still good week. Because he's going to go through to the final. This is a great race. This is super stuff. Smarslik out in front. Woffendern riding a stormer in second place. Side by side. It doesn't come out so far into the last lap. Don't rule out Ty going for a win here. This is a proper speedway race. Brilliant. Best race of the night. Here we go. He's going to try. Is he going to try? Sam or inside? He's just holding the track. Coming back up the inside. Looks like Smarslik's doing it off here. Going into the final stages. Good speedway race. Bartle Smarslik versus Ty Woffendern. Just like the good old golden helmet match races. Back in the day. Smarslik and Woffendern two world champions. Through to the final. And look how much they respect each other.


All right, let's get started. So the second to last lecture in the class is going to be about remote procedure call. So we're going to review some of the history of remote procedure calls and their applications, especially in remote file systems and ultimately in web services. So we'll introduce RPC and then dive into some examples. NFS, which is the network file system that's based on an RPC, and then we'll talk also about the Andrew file system, which is a second generation file system that sort of influenced a lot of things that came after NFS. And then we'll move in the second half of the class to talk about the RPCs, basically soap that's become a critical part of the distributed functions that are on the worldwide web now. OK, so we've talked quite a bit about the coordination between different machines, between clients and servers as part of operations that we've already talked about, like the network itself with TCP. TCP is a distributed protocol that involves maybe a client and a server exchanging messages to establish the connection. Check, do flow control, check for lost packets. And it's a complicated protocol, if you recall. We spent about a lecture and a half discussing all of the aspects of getting that protocol to work reliably. Both client and server have to be doing exactly the right thing for this to work. We talked about two phase commits in databases. So that's another message exchange format that guarantees that the state of the client and the server, as far as some transaction goes, remains consistent and is robust across machine failures. And get and post, we haven't talked as much about, but there are two important message passing protocols as part of the web, the old web. And last time we talked a little bit about, or at least we brought up, the messaging that's part of SQL Server, really just because it was a massive vulnerability that led to the slammer worm. So those are the one traditional way of doing distributed computing. It's just exchange messages, having agreed on protocol, and make things work somehow. And making things work entails that both client and server have to maintain some view of their state. So at both ends of the connection, you have state machines that can be quite complex. With TCP, the compliance to the protocol has to be very rigid. Luckily, there's plenty of TCP machines that people have been able to test against. And there's an elaborately agreed on protocol because it was a web standard. People know exactly what TCP is supposed to do. But it's more challenging with proprietary systems like the protocol in SQL Server, or even in two-phase commit where the implementation's proprietary, generally speaking. And that leads you into difficulties of version control, different versions of those protocols might be implemented slightly differently. And features tend to accumulate. People want to increase the feature of some service. And that leads to potentially inconsistencies between different versions of the same, ostensibly the same protocol, executing on different hosts. And of course, you have to worry about error recovery, which means really any potential pair of states might happen if there's a failure in the middle of one of these operations. And you have to figure out how to recover from that state, get things back to a consistent state. You have to worry about protecting the data that's going across the connection, data integrity, as well as privacy. And you also, at some point, need to check for message integrity. So last time we talked about buffer overruns, which are caused by parts of a message being too long, not really complying to an assumed standard of length. And also last time we talked about denial of service attacks, which are deliberate attempts to lock up resources on a server. And whenever you have a complex state protocol, you can cause denial of service by executing part of the protocol that puts the server in some known state with some typically additional data in its memory, and you just leave it there. So the more complex the protocol, the harder it is often for the server to perhaps uncommit from where it is. And the more ways often there are to get the server into that state. So stateful protocols have more risks as far as denial of service goes. And finally, you have to make this work. When you have a distributed system, in general, the client and the server are not running the same operating system or executing the same target language to execute the code. So that makes life pretty difficult for these systems. And so there's an imperative to design something that's a bit higher level that does some of this in a generic way, but still allows you to do application-specific things. And so that's the basis of a remote procedural. So the idea is that, conceptually, it's very simple. It just looks like executing a normal function call on the client. And it translates automatically into an equivalent call on the server. So in this case, this would be a handle into some resource on a remote server. And this would be a request to read from a specific location. And by wrapping a very complicated protocol inside of this RPC, we're able to hide a lot of the functionalities from the previous page under the hood. So very likely the RPC will have a lot of the complexity of message passing that we saw on the previous page. There isn't one RPC standard. There's several standards. They may or may not, for instance, actually execute some sort of transaction process with multiple messages to make sure that if the procedure call fails, then both clients end up in a state of basically an error state that the procedure call didn't execute. That's not that common, but it does exist. And it's actually typically part of RPCs that support database systems. More general RPCs don't implement this for performance reasons, because there's a very big overhead in delaying their commitments, and it limits the amount of concurrency. So usually they're not transactional, but still they often implement some of the other features here. They manage the state at the two ends. You would use some particular RPC version of some particular RPC standard, so the two ends of the connection should be able to communicate consistently. The RPC would do some error recovery for you, at least it'll sort of signal if the RPC itself failed, and they can sometimes provide data protection. That's sort of a sub-layer to do with marshalling the arguments. And there's typically type checking on messages. All right, so RPC is a clean way of implementing remote protocols through a simple abstraction that takes care of a lot of the details of particular messages going back and forth between the client and the server, and allows the programmer to just focus on coding up the impact of the message. But they do that in exactly the same way as if it was a local execution of a function. All right, all right, so the execution, at a very high level, is based on two pieces of code that exist at either end of the connection. Those are the stubs of the connection, and the primary function of the client stub is to marshal, collect together the arguments to the function call, and then un-marshall return values. And the server-side stub does the opposite. It un-marshalls the arguments to the function call, and then marshalls the return values. That sounds like a wild Western. What does marshalling mean? It means, among other things, mostly it means serializing the objects. There may be some canonicalization of objects. If there are data structures, potentially with cycles, it'll have to somehow put them in a particular form, it's a tree form, probably, with references. If there are also, so with references to other objects in the address space, there's a complicated set of choices that you have to make. The simplest choice is to copy everything, but in practice, that's not always practical. Depending on what this protocol is for, or rather what the RPC is for, the reference actually may get translated into a reference to the object on the client machine that the server can understand and potentially access. So the object sometimes approximates the objects that exist on both sides of the connection. And actually, that's one of the key decisions that's made as part of designing a remote procedural. All right, and then, more or less, the RPC abstracts away the details of the operating system, but there's various things that you do have to take into account, such as the threading model, whether you wanna have a blocking operation at both ends, or whether it should be a synchronous and so on. Those often have implications for the coding up for different platforms. All right, so the compiler typically generates stubs from an intermediate, excuse me, interface definition language. And the interface definition language mostly specifies the types of the parameters and the types of the return argument. And it outputs the stub code for client and server. And the intermediate, excuse me, interface description language is an abstract language. The stub code will be in C or Java or some appropriate source language. Okay, so RPCs usually initiated by the client. So the stub code is what the client actually executes. So that stub code is going to take the real arguments to the call, bundle them up somehow. So in Java, it's easier to think about this because you've probably all seen Java serialization, which is turning objects into strings. So serialization happens as part of the bundling. And then that text that's the serialized version of the arguments is packaged up with some headers, put into a packet maybe in TCP and then sent through, sorry, the packet encapsulation in TCP happens here. The packets go out over the network. And then there's a receiving packet handler which will extract the marshaled arguments and pass those to the server stub. So then we'll take the serialized data and turn it back into objects in memory.  So this again is like a normal function call. It says though you sort of short circuited this call from the client to the server. So whatever, you know, of course this could be a different programming language, which is part of the beauty. But if it were the same programming language basically it's equivalent to the client directly calling the server with those same arguments. So then everything goes unrolls backwards. The server returns arguments in its native language. Those get serialized somehow into a string sent to the server packet handler which packs them up in network packets, sends them back. Those get forwarded on to the client stub code which is going to now deserialize and produce objects in the address space of the client and then basically execute a return. And this really just really looks like a regular function call to the server. Excuse me to the client.  All right. Everything clear so far? All right, so some important details. In remote procedure call is which machine do you send the call to? And that normally involves a naming service that resolves usually a high level name of the service. Maybe I don't know, look up address. Maybe it's sort of address service colon look up a home address as a symbolic address that gets resolved down into a particular machine and port number because the service is globally known under that name, like look up address. Then that endpoint is either resolved by the compiler if it's, if you're using a sort of a static look up process you'd have to know at the time of compilation what the address server colon address look up port and machine was. And then you'd code those non symbolic addresses into the code or you perform a look up at runtime which is for obvious reasons, pretty obvious reasons, much better solution. It's called dynamic binding. So you have to have a name service online to do this. The cases where you do static compilation might be in some kind of embedded device that may not have access all the time to the network. So it's normally gonna correspond to services that are known at compile time. All right, so dynamic binding also provides, allows for access control to check if the user is really permitted to use the service and it also allows you to do some failover for reliability. Let's see, so that's a simple story. Very often though it's in larger scale RPC systems, more complex RPC systems especially. People go a little bit further in as well as including a name service to look up the service names. There's an object registry which provides both, which provides generally a bit more. It may provide host port but it may also provide the name of an object in a distributed object setting. There's actually an object in the sense of a C++ object or Java object on the remote machine that you would like to access. And it will also often provide client stub code. And the reason for having non-standard client stub code is to allow the client to be smart about how it manages for instance, references to objects that are more complex. Sometimes the right thing is to just pass a reference to the original object. Sometimes this particular call might not need the whole object, it may only need a tiny piece of a field of the object. And the client stub code can just look up that piece of information when the stub code is called, pass it on to the server. So it basically allows you to do the most sensible thing for that particular call. And it also allows you to deal with potential version problems by making sure that the client's stub code is always current and it always matches what's on the server. Yeah, so registries are pretty common in complex, internet of things type environments where you have a lot of different, a lot of different potential clients, some of them very thin, some of them some of them are very old and using potentially legacy systems and which you can assume they have, they're running some particular current version of client code. Okay, so let's see. Okay, so RPCs as we've described so far are oriented towards communication across the network boundary, but actually they have also evolved into a convenient technique for communicating transparently within a system to other processes either in the same system or across the network. So we've seen already, you can communicate in shared memory using things like semaphores and monitors and so on. You can communicate through the file system as Hadoop does. You can communicate through pipes which are an inter-process abstraction that look a lot like a file. And you can do a true remote procedure called even to access a process that's on the same machine. And so the RPC abstraction works perfectly well. You have to address a lot of the same issues because the different process on the same machine is still running in a different address space, different set of virtual addresses. So in order to do an RPC from one address space to the other, you've got to do essentially the same thing. And indeed the other address space on the same machine may also very likely be running a different compiler. So this allows you, for instance, to call a remote method in C running on the same machine from Java, for instance. So yeah, so you can do that. And in fact, once you do that, you have the ability, typically, pretty easily to extend that to remote machines as well. And it looks exactly the same from the client code point of view. So for instance, yeah, so sorry, before we do that. So an example of this is the X-Windows system which is the Windows system under Linux, for instance, if you're running GNOME or KDE or whatever, those are actually built on top of X-Windows. And as you've noticed, I'm sure you can just run X-Windows pretty much transparently just by setting the display variable either on the same machine or somewhere else. And X is gonna execute a lot of RPCs to move data around and you won't see the indirection that happens when it has to go through the network. So it's an elegant solution and so RPCs have also sort of generalized from true remote access to be local access, but through the same mechanism. Okay, so some examples of RPC systems. The probably most common one is Open Network Computing RPC, which used to be Sun RPC before they opened it up. And that's the workhorse of remote file access in Linux, of NFS in Linux. Windows implements it as well, so it can access Linux resources. And actually many other Unixes as well use the same thing. So this is sort of the first generation really widespread RPC. And then there was a big consortium effort to make it a more open standard because originally this was proprietary and it was first generation and they wanted to fix a bunch of things. So there's a big consortium under the label of distributed computing environment and they went through as big consortia tend to do many, many generations of cramming features into a standard called DCE, which RPC was a critical part. So this was pretty highly evolved and complex and it didn't really manage to take off. There were some implementations of it, digital equipment had one, HP had one. There was starting to be some convergence on this thing, but then if I recall Sun around that time brought out a brand new version of NFS, which was completely incompatible and but had some of the same fixes that were in DCE. And that propagated a lot more quickly and more or less this became an orphan. Also the companies that were supporting this were starting to move out of the Unix business as well. So but anyway, DCE was picked up by Microsoft and it did become, it got renamed and reworked and became MSRPC, which is the foundation of a lot of their distributed file system and distributed protocols. All right, and Java has its own remote method standard as well. Okay, so RPCs are used in an interesting way in micro kernels, because as we've said, they're a natural way to implement remote access to a different process. And micro kernel operating systems sort of break up the, if you recall the idea was to break up the key pieces of an operating system into separate processes and have them use inter process, sort of universal inter process communication system to communicate with each other. And that's usually implemented with an RPC. So RPC also became kind of backbone of micro kernel systems. And we talked about that some earlier in the course, so I won't go over that stuff again. And there's a black's window showed up here. Okay, so let's examine those some weaknesses of RPC in some ways of working a little bit more high level. So first of all, the implementation of RPC is, as I said, is does vary from one standard to another. Not all of them implement fault handling the same way. And in fact, generally they're fairly conservative about trying to, they are on the side of performance. So they generally guarantee not very much as far as fault tolerance goes. That's a deliberate choice to make them more usable. So they defer fault tolerance and fault handling to the author at high level. That's not true of all of them. As I mentioned, there are database oriented RPCs which invest a lot more effort in doing this. Yeah, so, yeah, right. Without RPC, a failure normally causes things to crash. But, all right, so if you're executing a normal procedure call on one machine and there's a problem, then in a sense, the caller and the call E, first of all, the call E finishes execution. The call Er will accept an error. So there's really only one process running. So there isn't a consistency problem. But with RPC, you know, you actually have two different processes running. And so you have many options if there's a fault. If the remote machine crashed, potentially you could return something that allows the client. You could return a fail message and that would allow the caller to continue. So that's the decision. The trouble is, though, that then the computer, the client that called the RPC may not know or generally won't know what happened on the server. So did their call get executed? Did it somehow get partially executed or did it not get executed at all? So we don't know. And again, that's not part of the standard. Some of them implement transactions. Some of them just try to keep it as simple as possible for performance reasons. Okay, so the performance of RPC is pretty intuitive. A normal within process procedure call is much faster generally. It doesn't normally require a marshalling of arguments. A same machine RPC is considerably more expensive because it does involve taking binary arguments and somehow packaging them up so that they can work across different languages potentially and different address spaces. And then RPC is inevitably more expensive because you've got to, in addition, take your serialized data and send it across the network. Well, so there's a really nice abstraction away of where the function call is being executed, but on the other hand, from a performance perspective, it's pretty obvious where things are being executed. In X Windows, it's excruciatingly obvious because it was, as it turns out, for whatever reason doesn't seem to work very well over current networks. It was meant to operate, was originally designed at MIT for people to work on machines in different parts of the campus and it really doesn't work well across the wide area network. So the locus of these procedure calls does become pretty obvious in a lot of cases. So you can try to do caching in certain contexts and then we'll look at that. But caching introduces then potential inconsistency between caches and their source data so that makes handling problems difficult again. All right, so in the distributed file system, we have some client over here. He wants to read a file across the network and there's a server there which holds the data. It's a very common model. Actually, in the old days, this was sort of a universal model. Now people tend to operate more with Windows PCs which don't necessarily, without relying as much on a network system. But still, the distributed file system provides these kind of affordances. The idea is, at high level, you want to make things transparent. So you want to make the user feel as though they're using some files that are on their local machine. What does that mean? Transparency means transparent concurrency, which is you like the clients to appear to have access to exactly the same files. Failure transparency, well, something crashed. If you're not actually using the application at a time, you'd like to ideally come back and see that your files are still there. Replication transparency, for reliability, you might actually have multiple copies on the same or different servers. And it would be great if you don't have to be aware of that. And let's see, yeah, it would be nice. Also, if you're gonna migrate files to maybe do some load balancing or bring them closer if it's a really big data center, maybe migrate them to get them closer on the network to the client, then that should also be transparent to the client. Oh, yeah. All right, so those are the desiderata. You also have to figure out consistent and global naming conventions. So, all right, so, but of course complicating that is that the naming choices are often need, well, okay, so a complication of having global namespaces is that they get very complicated very quickly. And in fact, remotely mounted file systems are usually aliased because there's a good chance of collisions. They have fairly conventional names like home slash user, which probably exists on many different machines. So, the mounting process in whatever operating system you're running is going to manage the, basically a connection to each file server, some sort of registration that you have handled to the files on that, or files or directory structure on that system. The mount point is the root of that directory tree. And it'll manage the translation between the local mount file system name and the remote file system name, which the server understands. So, something like this is a translated file name. That helps you avoid aliasing problems that would otherwise be pretty inevitable in a distributed context. All right, so this is what's most commonly done. You can also try to have a global namespace. So some of the distributed servers like Andrew file system will try to do this. And there's some cleanness to it, which is that you don't need to worry then about what machine you're operating from. As you move around, that address will always be the same. Okay, so here's a simple distributed file system. Basically, every read and write is executed and atomically, typically writes are executed and changes are made to disk before the return happens. And that will give you the most, the simplest way to get a consistent view of the file system from multiple clients. But the trouble is performance. So, you know, being very careful about making everything consistent normally means that there'll be delays for changes to happen before other clients can access the content. And so you'll have the network also adding delay to the movement of the data and to the signaling to the other clients that data is ready. All right, so caching often helps. It allows you to basically get data onto the server, hold it there more quickly, excuse me, hold it there, make it available more quickly to other clients before it's actually committed to disk. But then of course, you lose potentially on robustness because if there's a crash, those updates are lost. So anyway, so there's the usual trade-offs that you have to do. Let's see, so a server crashes client. Yeah, so there's a few options about dealing with crashes. It actually is sometimes implemented with a kind of a lock where the client does have to wait until the server comes up. Trouble is that data in the server's memory is then lost. Yeah, and the other approach is to try to share state across the connection so that both sides may have a buffer containing part of the update. And that way the client can retrieve what it's already got in its local buffer, even if the server content is lost. But then you have a consistency problem. So let's see, so this is another, one of those very complicated things. It's probably a good compromise, but it entails a lot more effort to make it robust. Let's see, so message retry. So if the server crashes, yes, after it's done a remove, but before it receives an acknowledgement, you have to retry. Yeah, and we're then, all right, so with NFS, excuse me, NFS is more of a best effort system, so it'll attempt to, it'll start removing files, but not in any particular guaranteed order. And when it removes blocks of a file, it'll also unlink those in an order that's not guaranteed. So potentially you can end up with stuff that's partially removed in a failure of an RM process. Okay, so for some of those reasons, it's desirable to make the system as stateless as possible. So a stateless protocol is one in which the information required to process a request is passed with the request, so both ends of the request know exactly what's being asked for. And it saves the server from keeping a lot of information about the client. Except potentially it's hints to improve performance. So the request is consistent. The, it normally means that the request can be repeated. If there is a failure, maybe client doesn't know what happened, they can repeat the request. And it should execute the second time and it normally would be item potent, meaning it, the outcome is the same. Okay, so yeah. Well, all right, let's leave the client crashing off for a minute. So examples of stateless protocols are HTTP by itself. So get and post. You know, modulo the side effects of some other site modifying the contents. Those are item potent operations. You can keep doing a get operation and it should retrieve the same content. That's the basis of web caching. If you do a post, you should be able to repeat doing the post. And if one of them failed and you do it again, it should eventually cause the update that you want. And REST is a design pattern that we'll talk about in detail at the end of the lecture, that goes even a little bit further as far as trying to make something like an RPC that's stateless. All right, so we'll get into that later. Interestingly, well, recently REST has perhaps become the dominant remote procedural for the web, but SOAP is still widely used as well. And whereas the REST model is a stateless protocol, SOAP is more deliberately and directly at a remote procedural. So SOAP is stateful, but it's implemented over HTTP which is stateless, okay? So it seems like a conflict, but it's really not. In the same way that low level IP protocols, like UDP is also a stateless protocol, it doesn't keep track of connections, you just send packets to a host. But SQL server, which implements transactional protocols, uses UDP, at least in one mode of operation. And there's client and server code that's maintaining a lot of state to make that work. So SOAP clients and servers maintain their own state. They execute, though, messages atomically more or less over HTTP. So you can, you know, stateless protocols, you have to be careful, you have to specify exactly which layer of the protocol you're talking about. Because statefulness is just relative to one particular layer. Okay, so NFS. It's a layered file system. Designed to make access to remote files transparent. And peers, though, they're the same as local accesses. So you have a file interface that's familiar. It's a Unix file interface. You can open, read, write, and close things with file descriptors. The VFS layer, when you look down from the top, it just looks like a file system that implements those operations. What it does is then, under the hood, though it spreads out the operations to a physical file system or to an NFS stack. And then the NFS server sits down below that and implements the actual details of the NFS protocol, network file system protocol. And VFS works at both ends of the connection. So it looks overall like this. The client code sits up here. Makes them read, write, open calls to the VFS interface. So VFS then farms out the actual calls to either directly to a Unix file system, maybe a Windows file system proxy, or to the NFS client code. And so NFS executes the ONC, RPC, and XDR is kind of a data bundling format that goes along with that. So it allows you to package up your data and transport it then over the network as network packets. So the NFS client basically does this remote procedure call, like the other RPCs we saw that turns into sort of marshalling and eventually transmission. And this thing un-marshalls the data, passes it up to the NFS server code, and then there's another VFS server layer which is implementing the bridge between the physical Unix file system and the NFS server which then allows you to access the disks. Okay, does that make sense? And you can see there's a symmetry here. You know, in a real environment, especially with NFS, either one of these machines could really be treated as a client or a server, either one of these machines could be exporting its disks through NFS. And so you can see how potentially somebody over here on this machine could also come down here, go through this NFS stack and access to machine on the other side. All right, so that's the idea. That's the rationale for having this interface at both ends. All right, so NFS is a protocol that you run at both ends that allows you to, that uses RPC to allow data, that is basically file data to go over the network. So it provides the usual capabilities of a file system, reading and writing data, and then manipulating links and directories. And let's see, file access, yeah, the usual things that you would do on a physical file system. So, you know, but it doesn't just pass those through to the physical file system because, well, you can figure some of this out, right? The links themselves may be to other physical file systems or to other NFS systems. And accessing file attributes, attributes may actually be different on this client and host machines. Access control has to be managed very carefully. So NFS is not just a, what's the word, a middleman in these processes. The NFS software actually has to manage those explicitly. Let's see, so NFS normally does write through caching, meaning that the data that's committed to the server, sorry, the data gets committed on the server's disk before you reply to a client that you finish the write. All right, so it's not very performant. The client has to wait and might have to wait a long time. You know, caching isn't really, well, you're not benefiting from the cache, right? It's sitting in memory, but you're gonna wait for it to be written before you can continue. And it turns out NFS is actually, and it doesn't do anything smart for readers to notice that you've made the change. The readers actually will have to ping the server to figure out that there was a change. Okay, so NFS servers are stateless, meaning that the request provides all of the arguments and information to complete. So let's see, yeah, so for instance, unlike TCP, there's not a notion of a connection or there's not really a notion of opening a file and using a file handle. The read operation will actually have, you know, the client, when it opens, the client can still open files, but it will receive information about the I-node on the server machine and the file position, I guess. And so it'll read at that specific address that it's got from the server in an earlier transaction and at the given byte position. So yeah, so there's no notion of a persistent open file. This has to actually be a physical address that's universal for that particular file, yeah. They don't, it can happen. So NFS doesn't manage locks or prevent people from doing that. There would be a right time that you would get as an attribute when they access it, so they'd be able to tell that it was modified recently. What else? I mean, the client, yeah, the client, if you're reading a file and somebody modifies it, yeah, you would normally get a message about that during the operation, but the trouble is it won't stop the right from happening. You don't have a lock.  I'm almost certain we'll tell you if you read a block, that's, well, actually, maybe not. I guess there's no reason for it to tell you the modified date, but the modified date may just be part of the metadata that you get of every transaction. That would be logical. I don't actually know it, but. All right, so no NFS is really not very careful about managing consistency, and it's very, it definitely admits a variety of sort of inconsistent and corrupted files to happen. Sure, I mean, that's the ideal situation for any distributed system. But it can work okay in those distributed contexts, but it's rather like when you have, even if you're using GitHub or something, it doesn't stop you from doing inconsistent changes. It just makes you aware that there's an inconsistency and it tries to resolve them, but NFS, it really leaves that responsibility to client code and to people using the code. But it's by no means considered to be a clever solution for this problem, so we'll get to some better solutions soon. So an important property that's shared by almost every stateless system is the idea of item potency, which means that when you repeat the request, the outcome is the same. Modulo, state changes from outside. So reading the same file, calling the same read operation returns the same argument. So intuitively that's different from just saying read on a local file because read on a local file returns the next block and maintains an implicit file pointer. At the low level, NFS reads are always just reading a specified block, so you'll always get the same one with the same file arguments. Similarly, when you write, you're always writing an explicit block to an explicit file, so it'll always modify just that block. Yeah, and we'll see this again and again, actually. All right, so the advantages of this are that if the operation somehow fails or you don't get an acknowledgement that it's succeeded, you can just do it again, and the outcome should be the same, and it shouldn't matter whether it happened the first time or not. Yeah, yeah, it's sometimes said to be side effect free. Yes, well, remove is a tricky one, because normally if you try to remove something that's not there, it should give you an error. So, but in NFS it's done as an advisory error or sort of a warning rather than causing your code to crash because the system doesn't know that you're doing the right the first time or maybe doing it as a backup. Okay, so the, yeah, the failure model, well, let's see, it tries to hide it from the client system to the extent that it can in the hope that maybe the problem will fix itself. Or, you know, so two options that NFS does provide for dealing with Fox. One of them is to lock up, you know, which is the, which it definitely uses for short periods.  It won't return anything to the client. If it succeeds eventually it'll come back. So the client will be blocked until that happens. It'll sometimes return an error, but that's neither of these is required. So the behavior is, I guess, dependent on a particular file for some implementation. I don't know if it's configurable in the OS, but both of the options exist in NFS. Okay, so let's see how we're going. All right, so let's actually, all right, cache consistency. NFS has a very weak model of consistency. We talked about weak consistency before. It uses polling for clients to check whether changes have happened. So client can poll, get the last written date, and then determine that a file's been updated. Let's see. Yeah, and clients also have caches. The server gets notified that the client's done a right to one of its caches, which should eventually cause an update on the server, but the server, first of all, is told that the updates happened, but doesn't try to invalidate the other client's local copies. So in other words, it's deliberately allowing inconsistencies for a time. It's something like eventual consistency until the client successfully writes its data to the server and the other client's check, you have inconsistent data. Okay. Yes, and so, and a very bad kind of consistency in terms of multiple writes. Because clients can write blocks basically independent of each other, one client may actually succeed completely. Very likely though, if they're writing different parts of the file, you'll get scrambled data. Okay, so it's, on the good side, NFS is simple and highly portable. On the bad side, it's inconsistent. It has scalability problems, too. Very quickly, a better system that followed on from NFS, excuse me, was the Andrew file system, which was a research system at CMU. And basically it was meant to address a lot of the problems that people had noticed with the NFS. So instead of the polling mechanism, it used callbacks. It's kind of like more of a published subscribe system, where, excuse me, clients that have a handle on a particular file, or a file system, notify the server that they have that handle. The server then basically maintains a list of all of the clients that have a handle on that file system. When there are changes to that file system, they'll do a remote procedure call to the client, say, oh, this has changed. What do you want to do? It's left to the client to actually figure that out. And instead of writing through, on updates, it's right through on closed, on closed. So you can do a lot of changes. They'll just operate on the cache until you close the file, and then they'll get committed. So this is a lot more performant for the client that's making the rights, but really it's hiding the changes from the other clients until you're done. So this, I guess, is more, it's basically a locking model, because if you are modifying the file, no one else can see it until you're done. All right, so that's the first set of things. Yeah, it has a two-level caching system. Okay, I think we better, we'll just move on, because let's just keep going. Sorry, I don't wanna, sorry, we have to skip some of that stuff, but we're gonna run out of time. So the midterm's coming up, which I've shown you to no one. It's coming up 5.30 next week, Wednesday. And we have these two rooms, similar format to last time. You can have one double-sided set of notes. And it's gonna cover material, all the material since last time, last midterm. Okay, so we do have the review session scheduled from seven to nine in 306 Soda, Sunday, December 1. Don't forget we still have office hours, et cetera, for Monday, Tuesday, Wednesday, next week. And any questions about the midterm? Okay, and Project 4 Initial Designs do Monday, so make some plans for that, and that's it. Okay, so let's take a break and we'll finish up and look at web-based RPC in the last half hour. Okay, let's wrap up. So let's review some of the things we talked about with RPC and NFS. So first of all, does RPC require special networking and functionality? Yes or no? No, all right, good, yeah. It operates at a high level. All it has to do is sort of serialize data that can go over the network. All right, client and server for RPC must use the same hardware architecture. Right, that's, almost the whole point is to allow things to work across architectures. All right, local procedure call is much faster than same machine RPC, it's much faster than remote machine RPC. True. And let's see, NFS provides weak clients over data consistency. Yes. Okay, so as we said earlier, RPC is still considered a pretty low level interface for distributed computing. It does some things very well, it does abstract a lot of the details of sort of message passing, communication, reliability, some of those. But for instance, it doesn't support a lot of shared state of persistent shared state between a client and a server. So you can argue that you don't want that, but there are some cases where that's inevitable. For instance, if you have two different, if you have a database that's distributed across multiple servers, you need a lot of state on each one and you need it to be consistent. So just as in regular programming, distributed systems like any piece of complex software can benefit a lot from careful software architecture and especially object oriented programming. So object oriented programming allows you to sort of break up the state of some complex system into objects and sort of simplify their interaction by calling methods. So same idea works at scale with distributed systems. And in fact, probably the most effort in making distributed systems work has gone into object systems. Most people have heard of Cobra, I'm assuming, have you? Yeah, it's not obviously the answer to the question now and it probably will be less obvious in future because the system was, these two systems were incredibly important for a while but they kind of faded from visibility. Still important for those cases though that I mentioned like distributed databases. All right, so Cobra was one of those other monstrous committee standards. Common object request broker architecture and almost unpronounceable as well. But the idea was to allow distributed objects to exist across machines and to allow them to exchange method to call each other with methods and to maintain consistent state. Decom is a Microsoft standard just as we saw with their RPC. They borrowed Microsoft RPC from the earlier consortium F at DCE and they similarly built a component object model on top of this distributed computing environment and they completely appropriated the name DFS which used to mean an open standard that was part of DCE and now it means a Microsoft Distributed File System. So a lot of effort went into that and it's a lot more complicated than an RPC. The idea is to really do the opposite of the things that make your life easy because you're making a commitment to actually have a lot of data shared across servers. It could be clients and servers but it's especially important with servers because they really do have a lot of shared state. With client service systems maybe this is less important because you can arguably maintain everything on the server and just allow the client to access that. So in addition to just remote function calls you have to have our object proxying which means the object's appearing on both sides of the connections, executing methods from either side and maintaining consistency and you even have to implement garbage collection. Most of these systems have garbage collection which includes the references from the other side. So you've got to figure out for an object that's maybe hosted in one machine, maybe proxied in another machine, how many other objects have references to that object across the network. You've got to count those and only when they've all gone to zero or when some of the machines that had references are definitely offline and won't be coming back with those references then you can garbage collect those objects. Anyway, it's very complicated to do all of that. So anyway, they were successful standards. They were implemented and very widely used and they're still used. And people figured that they would also become the protocols that people used across the internet for distributed servers, I don't know, large scale data centers and so on. And to first order that didn't happen. And anyone have an idea why? So most of the answer comes from last time actually, indirectly from last time, which is that what happened while those standards were evolving is that computer security became a much more acute problem and the response to the very dangerous worms and viruses that were going around on the net was the introduction and very widespread use of firewalls, corporate firewalls and packet filtering within routers over the internet. So that made it very hard for arbitrary port number of packets to move around, mostly impossible. So the one thing that always works, one of two things that always works is HTTP and email typically works as well. So anyway, but HTTP, to first order is the only reliable protocol that goes everywhere on the net. So there are two ways of leveraging that. One way is to try to tunnel other types of payload, in other words sending packets that are not HTTP packets through port 80. That can work but it's very tricky to work because at the end point you can demultiplex, you can have a router that examines carefully the packet headers and routes them to an appropriate address locally. The trouble is though that a variety of things can go wrong because for instance, packet filters in the middle of the network may be looking at the type of payload and filtering non-HTT packets anyway. So yeah, the system that really got deployed widely though instead was actually just taking HTTP as the standard protocol for remote computing. So you can treat HTTP as just a transport and send well whatever you want across it. And that led to the technologies that have become the web service technologies, which are SOAP and it's discovery, it's service description protocol, WSDL, and more recently REST. So these are the dominant systems for what amounts to RPC over the web. And they're heavily built on top of XML and JSON. So whereas these are not necessarily part of the web standard, they are sort of the core technologies that enable it and web services have driven a lot of development and a lot of what's the word, interchangeability with these standards. So a lot of application software will speak XML and JSON in order to be able to work across the web. All right, so the idea of SOAP and REST is to allow you to actually send more or less an arbitrary remote procedure call across an HTTP connection, which among other things implies it has to be text. So SOAP is the standard for remote procedure call. So SOAP directly implements an RPC model where you send some arguments and receive a result. So it has to specify a message format because it has to tell it what kind of message is gonna be sent across the HTTP connection. In some cases over, actually I think it's always HTTP. And then there's a description, oh no, I'm sorry, it can do email as well, sorry. So two media can be used for the SOAP message and a set of rules and a set of conventions. So that's a bit abstract. So let's just look at some examples. So a high level of the SOAP message just has a header and a body and some XML schema boilerplate at the top to specify the schema, which means the structure of the XML. And here's an actual SOAP message, which is the calling message. So there's the envelope in the body. Here's the method name that's being called. It's order goods here. And here's a couple of parameters being passed. Everything's gonna be converted to text, in fact, to an XML string. The string's getting sent over HTTP to a server. And the server's gonna build a string that looks like this. Again, with an envelope, a body, a method name, which in this case is just return, it's a return method. And then the value, which in this case is gonna be some ID, some integer ID. Okay, so where are we? This is the, here's a more detailed message. Let's see. Here's the body. Here's the method name. In this case, get flight info. All of that is schema naming. Sorry, namespace aliasing, excuse me. And then down here, we have an airline name, which is, I guess, UL. And then, no, wait a minute. Yeah, LA name is UL, sorry, I should be doing a better job of reading this. And then a flight number, 506. So it's UL506 is the flight information. And then the response gives you back more information about the flight, namely the gate number and the status is on time. So is that, is it clear what's going on here? We basically encapsulated with a serialization into an XML string the arguments to those function calls and the return value. So it's very similar to the other RPCs we described. But when we described them the first time, we didn't specify what kind of serialization we would use. With web services, it's always an XML or JSON serialization so that it can run over a text port. So associated with SOAP is a description language, web services description language, which allows you to specify exactly what SOAP messages look like for a particular protocol. So it describes what the service does, what operations in other words, what verbs it implements, where it resides, just a URL and how to invoke it which will include argument types typically. And in some cases I guess protocols. All right, so, well this is not too bad actually. So here's the interface section which specifies a set of methods. Let's see, get flight info here, are the methods down here? Oh, here's another method, sorry, here's, all right, each of these is a method name. And they have parts, each message has these sub parts that are shown in the sub parts of the XML. All right, so this is very abstract, just telling you what the structure of the messages is. The implementation section contains a list of the same message names, but in addition, the types of arguments should be visible here. And also encodings and namespaces. So let's see, can I use the equals literal? Well, probably just because they're strings so they're all ending up with string types. Anyway, so there's these two different levels of the specification and that's enough for a compiler typically to generate the stub code to actually produce the RPCs at both ends. Okay, so the last topic is REST which is a reaction or a second generation system that is diverging from SOAP. SOAP was the dominant standard for a while when web services first became popular. But SOAP had the limitations that we described before as far as RPCs have. There's an issue of what happens if there's a failure. How does the system work with web caching? Because caching is normally an intermediary. There are web caches acting as intermediaries between clients and servers. Clients sometimes move around in IP address so there's a number of things that make it more complicated even with a fairly simple round trip RPC like SOAP. So REST was the reaction to that to try to make things even simpler. And the idea is that every message in a REST protocol contains all the information needed to execute the request. Also, there's just one type of object which is called a resource and those can either be websites or objects within parts of an object, an object that's hosted by a website and it could be an object or a part of an object as well. So you can have these very long hierarchical URIs. Let's see. And REST sits on top of HTTP so you have the usual operations but they're just one-way operations not a round trip RPC like SOAP. And in particular, they're idempotent. So REST tries to be idempotent and it is idempotent in the same way HTTP is. You can keep doing the same REST operation many times and the outcomes are the same modulo, the effects of an update. But if you're the only one interacting with a service, they're idempotent, okay. Yeah, and REST supports the resources using a description which is basically the URL description of XHTML or XML. Okay, so quickly here are the principles of REST. It's a cleaner client server separation than in SOAP or normal web protocol. So for instance, in particular, it avoids the use of cookies on the client. It avoids distributing state onto the client. In REST, the server should house all of the state if your client has to move or if you transfer from your laptop to a smartphone and you use the same URL. You might need to get credentialed, that's the one stateful piece of this, but once you're credentialed, it will do exactly the same thing if you access it from a server, from your home computer, from your smartphone. So everything's maintained on the server for that reason. It's stateless, meaning there's no client context stored on the server between requests, and that allows it to be idempotent. It allows repeated requests to work. And let's see, yeah, it's base, what's the third reason for that? Yeah, it allows you to basically recover from disconnects or server breaks by just doing the operation again. Let's see, yeah, it's consistent with web caching, so clients can cache responses or somebody in the middle of the client in the server can cache a response because the response is going to be the same as long as the URI is the same. So some intermediary can see that somebody else did the same URL and received a certain response. They can cache that and give that to you to reduce some loading for common URLs. Yeah, and those two sort of go together. So it more generally supports layering where you can't really tell if you're connected to an intermediary or the real server. The idea is that there's this one-to-one mapping between URLs and responses, or URIs, excuse me, that an intermediary can copy and reproduce to provide the same service or approximation of the service. And finally, as we saw earlier on, it's often useful to be able to give the client some custom code for efficiency. So basically give the client stub code so they can execute a more complex call. And for instance, provide just the data that the server wants for that transaction. That's actually an optional part of the REST standard or the REST principles. But it should be clear why that's useful. We already explained that example in the context of RPC earlier. All right, so quickly, here's an example of a REST transaction. Here's the URL. URL, URL. Oh yeah, all right, so yeah, here's the location, the URI for this resource. Here's the contents of the resource. So in order to get information about the resource, the client just needs to do an HTTP get of that address and they'll get this information. If the user is suitably credentialed, they can also put data back there. They can modify some of the attributes here and it'll appear on the server quickly, immediately, actually, in most cases. There's no need to cache in this setting. Yeah, and then that information normally becomes immediately available to other clients. So all right, so this is very clean when you're doing simple update operations, as we've seen here. On the other hand, though, the client has to be fairly smart. It has to be able to produce these URIs, which in many cases involve hierarchical access to data structures. So for instance, for access to some complicated data set or database, the client's probably first gonna download a directory or the schema of the data set in order to know how to invoke the subfields in order to traverse to something that they wanna do. So that's not too difficult, but the client has to have the ability to parse XML, understand schema, and so on at a minimum. Okay, so a simple way to think about the contrast between REST and RPC is that RPC systems are based on verbs and based on actions and arguments to actions. So in other words, in SOAP, we saw that there were operations typically involved in adding, removing users, or updating user content. In the REST system, the emphasis is on nouns. So for each of these transactions, there's an associated user or resource that who's state you wanna change is a response to the action. So the first four here are user. So in the REST system, though, it would be the URI for user and you would probably try to read and then update the contents of that URI if you're trying to modify it. So in the sense that the client has to be a lot smarter about what this all means and they have to be, the server also has to be careful to check that things that are being posted by clients are valid, compliant to the schema and valid contents for hosting in the URI if it's a put. But with modulo, those constraints, REST makes life, it makes the implementation a lot simpler and it makes a lot of things work in terms of working across caches and through firewalls and so on. And it has become the dominant system for computing, distributed messaging and RPC across the web. Okay, so to wrap up, RPC conceptually is a way to execute function calls across machines as though you're executing a simple local function call. And the RPC code, the RPC system implements some amount of error checking and some of the difficult to write state maintenance between client and server and provides the programmer with a simple interface where they only have to worry about implementing the functionality of the function call. So it's basically two pieces of code which would be similar to writing the code on one machine. So RPC is being the backbone of distributed computing. It's still widely used as a interface for, especially for local communication on one machine. We talked about distributed file systems, NFS and briefly about Andrew that are, implement file access remotely using RPC. And then we actually talked about object systems which are a layer above the RPC system that allow you to view the world as though a distributed environment as though it's objects that you can access from your machine but are actually distributed across many machines. And we talked about SOAP and WSDL, which is a remote procedure of protocols that runs over HTTP. So it can work across the heavily firewalled web and includes a description format, a meta format, WSDL which typically allows compilers to generate stub code for the calls for a specific set of messages. And finally we talked about REST which is a kind of second generation system that avoids the round trips of RPC in order to make things even more simple and more robust and more compatible with the web in terms of caching and recovery from errors and people having a kind of a mobile life where they're accessing their resources from many different endpoints. So that's it. So have a great holiday and we'll see you on Monday.


Good day, I'm Don Boland, B-O-L-A-N-D, Executive Director, California Utility Emergency Association. We are the embedded partner for the last 60 years with the California Emergency Management Agency here in the State Operations Center in Mather, California. Actually what started was we were notified by E-E-I, the Electrical Institute in Washington, D.C., which handles the major carriers for private-fessor-owned utilities throughout the United States that they had a mutual assistance request and has asked our association to step up and provide assistance for the power outages on the East Coast caused by Hurricane Sandy. What we did on Friday afternoon at about 2.30 was mobilize our mutual aid organization within the state of California, which encompasses almost 60 of the power companies, PG&E, Sempra, Southern Cal Edison, SMUD, and multiple others. We engaged them to see what kind of resources they could pull together, how fast they could pull them together, and how quickly we could get these resources dispatched to the East Coast to start mitigating the national disaster that we're now facing. In the process of this we also took on the assignment of being the co-facilitator for the Western States for the Western Energy Institute and facilitating Washington, Oregon, New Mexico, Texas, Arizona, and all of those other states so that logistically we had put together strike teams and task force, mobilized these strike teams and task force, and got them on the road. Understanding that we are the third dimension away from the disaster, the impacted area was the primary, they're in the zone. Secondary became Midwest, their proximity was within hours. We're looking at 2700 miles of travel. Logistically it became a monumental task for the West Coast. We've sent several hundred out of PG&E. We're sending several hundred out of Southern Cal Edison, Sempra. We have amassed 17 major utilities already in California, put them on the road and got them underway. Now we are also as of today, which is Thursday, loading them on Air Force aircraft at March Air Base. We have 15 aircraft down there, loading 70 more emergency equipment trucks and crews from San Diego Gas and Electric, Southern Cal Edison to expedite getting them into Rye, New York in the areas hardest hit. This morning we started to put together our second wave of emergency response. We are now pulling more from PG&E and all the others, assembling them and sending them. Now this cannot happen without the public partnership that we are embedded in in California. California's Cal EMA organization has stood up supporting our initiative in the association and put together a task force to assist us in the travel plans. Part of the task force is CHP. They're working with the scales, Caltrans working with road access. We have DOT working with permits and the warning center from the state of California has stood up and is tracking all of the vehicles as they cross state lines. We have put together a co-operational task force, which is calling every state DOT to alert them these convoys are coming. And as such, they remove any roadblock impeding our ability to get to the disaster zone safely and expeditiously as possible. Convoys are made up of both air assets going out and we're running massive amounts of low voice semi trailers that carry multiple utility trucks on them so that we can get that equipment to the coast. At the same time, we have chartered big aircraft, 737-800, 737s and a lot of commercial seats to put the crews on the aircraft, yet the crews positioned out ahead of the Ramparanus so as it arrives in New York and New Jersey and New England, we have the crews ready to de-embark, get in their vehicles, they're fresh, they're ready to go, they're outfitted, they can be immediately deployed to start restoring the power infrastructure. This exceeds, I was involved in Hurricane Andrew and I've been involved in Katrina. This exceeds from a mutual aid request any size and scope that I've ever been involved with from a utility aspect. We presently have in excess of 50,000 electric linemen on the ground and we're deploying another 9 to 10,000 as we speak right now. This is taking every resource the Domestic United States has to address this issue. One of the things that we always take a look at is making sure that we are capable of responding to our constituents and our residents here. We will never drain our reserves to the point that we cannot cover the essentials that we need to be and that's why this mutual assistance and this public-private partnership is so important because now we're down to minimum staffing, which we still need to maintain what we're doing for our residents, but we're still able to move crews from San Diego to San Francisco to Los Angeles to San Bernardino with each other if we need them. Our headline is today that we have put every resource at our fingertip and we have unilateral support and cooperation from the state and federal government as it has never been seen before.