So I'm going to thanks thanks for inviting me back and I'm going to talk about Cloud native what that means. That's really the why you should be doing this part then Netflix I was asked briefly the open source tools that we've put out there and Something about the scale and complexity of what we're doing and some benchmarks that we've published over the last few years So cloud native what what is it and why does it matter? Well, what happens is we're always striving for Perfection we want to have perfect code We want it to run on perfect hardware and we don't want to make any mistakes when we're operating it and that's sort of the engineering ideal That's the utopia we're trying to get to but that utopia takes too long to get to so you always ship code with bugs in it And you always push the wrong button and break it when you're trying to operate it all those kinds of problems So there's always this compromise where you're trading off time-to-market versus quality and these utopias So permanently out of reach just one of those frustrating things about engineering just give me another six months. I can make it better But there's a bunch of markets where time-to-market is the most important thing If you're making a land grab and Netflix is making several land grabs. We're doing a global land grab We're launching in Holland this fall. We're doing a content land grab We just announced the deal where we're taking the Weinstein Movie output deal in a few years time. So we're accumulating all of these large-scale content deals and we're disrupting our competitors by Getting inside their OODA loop and the OODA loop is observe-oriented side-act I'll talk a bit more about that later But if you can figure out how to react to something faster than your competitors you disrupt the market And basically anything delivered as web services you can just speed it up and go faster and faster So if you're competing with a service that's delivered on a disk or is delivered in some slow mechanism and take six months Or a year between updates you can disrupt them very easily So this is the OODA loop observe-oriented side-act came out of the dogfighting in the Korean War and Colonel Boyd said basically he was teaching his pilots to figure out what the The people that were dogfighting him were doing and just confused them by being able to react more quickly So when you apply this to business What you really want to do is observe a land grab opportunity Right something that somebody else hasn't seen yet You want or a competitor makes a move and you want to react to that competitor Or you find a customer complaining about something and you want to address that pain point this There are other things but these are sort of several of the different kinds of things We are just trying to look for an opportunity Next thing you're going to want to do is do some analysis on it research alternatives figure out what to do about that Then you want to plan a response get buying from the rest of your company and commit resources to go do something about it And finally you're going to implement whatever it is deliver it engage customers Market sales whatever it takes to get it to the customers And then there's one more step which is you back into observer you're measuring the customers And did they actually use this thing and does it make sense then you carry on around the loop Looking at how the customers reacted to this new thing you did so the faster you can go around this loop The better you can get your products and the faster you can address a market and you can beat your competitors So this is there are big Industry buzzwords sort of that really tie into this and one of them is innovation a lot of people have innovation strategies or innovation groups Or there are big company memos saying we're not being innovative enough and what they're really talking about is their inability to See and react to these of these opportunities The next thing is really big data and big data is about getting the analysis and getting Pulling in the data the unstructured data and being able to ask questions that you couldn't previously ask and Get answers quickly so that you can carry on to the next step The decision parts a lot of this is around corporate culture if every decision goes all the way to the CEO through layers and layers and layers of management by the time it gets there it's unrecognizable and It takes just too long so you want to have decision making that is pushed low in the organization And so people can see an opportunity Do their analysis? React to it and get stuff done really quickly and cloud is really in this this fourth box It's the ability to implement stuff quickly deliver it quickly engage customers quickly cloud-based services like sales force or You know real-time bidding all those kind of marketing and sales operations things where you're very agile So if we're trying to get around this loop, how fast can we get around this loop? How soon can we do it what I'm talking about here is the ability to do code features in days instead of months We have an idea on Monday We've implemented it by Friday the customers use it over the weekend And we're looking at the data to see what happened on that the following Monday And we had a whole series of projects that we did where we cycled 20 or 30 Personalization ideas through the system through Netflix's production systems on that one week cycle So it can be done and we typically take a little bit longer for bigger projects like this morning We launched an update to the what you probably know was the instant queue That was a long project lots of steps in it, but We can get small you know small changes and incremental changes made in a few days When we want hardware it get it arrives in minutes I'll talk a bit later about some benchmarking we did where we were just talking about minutes to create really huge No sequel environments that we could benchmark Instead of taking weeks having meetings with it meetings with finance meetings with management trying to get approval By the time you've done that you've probably spent as much if you look at the total salary cost of the people in the meetings It's probably more than the actual benchmark would have taken if you just ran the machine for a day Or ran the set of machines for a day an Incident response if you're pushing out code very quickly. It's going to be broken sometimes And you want to be able to tell that something's broken in seconds and you want to be able to respond in seconds either automatically or by building Visualizations that show you what's going wrong so that you can react in seconds as well So if it takes you if you're looking at one hour updates or yesterday's data, it's already way too late So this is this new engineering challenge we're trying to construct a highly agile and highly available service from ephemeral and Assumed broken components so our default assumption is that everything we're talking to and all our dependencies are broken Right, we don't assume that everything's perfect and that takes a little bit of getting used to but that switching that assumption is the key thing here So how to get to clown native the first thing you do is you have to give freedom and responsibility to the developers That means they can actually observe the the things that need doing you know There's probably product managers and whatever in the in the innovation part But quite often the developers are there coming up with ideas, too You want to decentralize and automate the ops activities if you have to go have a meeting with it to discuss the code You'd like to push to production every time you do that then you can only turn that code over so fast we're turning code over in a couple of hours from Pushing code to all the way through to production and it's done We do that because the developers are managing that code into production They manage it when it's broken and they understand the exact state It's in at any point in time. There isn't time to explain to the ops guys You know exactly what the state of the system is at any and so that they could react to it And so what we've basically done is take this DevOps organization and integrate it into the business organization So we don't have a separate business separate engineering in a separate operations group It's really one three one sort of group I called biz dev ops or whatever you want to call it So how do you get there? Unfortunately for most companies it means a reorg And that's that's kind of the bad news if you are there already organized this way Or you can figure out how to organize groups and give them the autonomy to operate in this way To maybe build in your product line or something like that But it's very difficult for organizations to actually respond to this which is why you know new organizations come along and disrupt it and Engineers get frustrated in the odd organizations and move to organizations where you can actually Where you are do have this freedom of responsibility So a lot of it a lot of the speed comes from corporate culture as much as any as much of any of the technologies So these are the four transitions the first thing integrating all of your Management all your roles into a single organization so that you can iterate around this OODA loop in days rather than having to all these Handoffs and and meetings and taking weeks or months to go around it Developers this is the one that's actually the hardest transition that we made the move to cloud was relatively simple the move from You know a traditional Oracle my sequel schema based environment to a Relational schema environment to a no sequel environment was the hardest thing to actually get the developers heads around and One of the things here is you have to completely denormalize your data model so that every team what used to be a Materialized view or a table that you know some query talk to becomes a completely distinct database All right, so we have totally separate clusters. We blew up our schema to the point where the back ends are not even in the same cluster We don't have one huge or Cassandra cluster with all of the different tables in it We have 50 different Cassandra clusters Yeah, each of those if you looked back at the way that would have been implemented in Oracle would have been a Materialized view or a table or something but all part of one big schema And if you wanted to touch any part of that the effects would ripple through everything else And you have to go and bounce Oracle or bounce the database to do the altar tables and all those kinds of things So we are continuously altering all of our tables because it's somewhat unstructured and also because it's partitioned and denormalized and People are uncomfortable with denormalized and you have to get over it and because of the speed that it gives you It's one of those things where yes It's a little bit broken, but you can build data checkers to clean things up and you can go so much faster It also makes it possible to be polyglot. We have largely Cassandra in our back end, but we do have some MySQL in there We have occasionally had bits of Mongo in there There are other databases that you can put in there and you can experiment and you can move things back and forth It gives you some migration abilities Because once you've broken it up and denormalized it it doesn't have to all be one big thing because you're never going to try And do a join across it anyway, and then as I said we move responsibility from ups to death for continuous delivery So that means the developer pushes code when they are ready for it And then they understand their dependencies and who depends on them and most changes are hidden behind layers of Abstractions so that you can actually iterate faster. You can make changes that don't change an interface quite quickly So we're talking about decentralized small daily production updates by each team Each you might actually some teams might update once a week others may update multiple times a day But it's very decentralized And then this agile infrastructure, so the ops to dev You have to get developers used to pushing their own code and being on call when it breaks So this is the run what you wrote idea that some companies call it and If you can make a developer be on call They get very good at not getting called by writing code that doesn't break and building automation to make sure that when Something downstream breaks their code points the finger at it to say that broke rather than just sort of throwing errors itself So then we get very good habits by taking away the crutch that developers typically lean on sort of the operations crutch Oh, they'll deal with it. Now if you have to deal with it, then you make it better So inspiration I got a bunch of books here There was actually a blog post today on the black duck software site based off of a previous talk I gave that goes through some of these books Who here knows this book release it Michael Nygaard?  Okay, so this is one of the classic books in How to get code out there in production we learned patterns from this that like a bulkhead pattern and the Circuit breaker pattern that we baked into our code. So bulkheads prevent failures from spreading circuit breakers Basically say this is back currently bad stop trying to call it let it recover and we'll check it occasionally in the circuit breakers Flip back in so typically are the same way electrical circuit breakers work Thinking in systems isn't really about computing at all It's mostly about economics and and about how to build large-scale complex adaptive systems that have the right emergent behaviors So it's a really fundamental book in this space If you're trying to build what looks like a very chaotic system, but you want it to behave in a very stable way It's important to understand how to build feedback loops and things like that anti-fragile is a book that came out last year that talks about how when you hurt things a little bit they get stronger and this is the Principle that we've been using for a while The best analogy for this is if you go and work out really, you know do a really good workout You probably really hurt the next day, right? So that's kind of bad Why would you do something that makes you hurt? Well, it turns out that you get a little bit stronger each time you do it and the reason you're working out is to get stronger And what we do is we work out our systems We work out the Netflix infrastructure by by inflicting a little bit of pain on it That actually finds the weaknesses and makes it stronger over time Drift into failure is the the best way I like to describe this book is don't read this if you're about to get on an Aircraft or about to go into hospital. It's got lots of examples about Everybody making the right decision based on all the information they had available to them And you end up with this tragedy of the commons kind of thing where the connection when you connect together all of these Perfectly good decisions the end is a massive failure Where if we keep pushing the maintenance intervals because the plane's never broken So we don't need to maintain it very often and eventually the plane falls out of the sky Because they finally went a little bit too far and it's never done that before so everyone's all surprised But there's no actual fault. Nobody is at fault in any part of this everyone was optimizing for the right thing so this the problem with this This highlights is that if you build a really really highly available system and Netflix has Run into these kinds of problems when it does fail. Nobody's used to dealing with it failing So all the things creep up on you or the sort of technical debt creeps up on you And then when it fails the people the team that's trying to fix it Has it hasn't been on a call because it thing hasn't failed for a long time And you end up with people thrashing around and not knowing what to do So part of this is to have sort of fire drill type game day type of activities where we practice Breaking the system in a way which involves the users that the the developers actually getting called and having to deal with it As we as we get more and more automation in the system you have to understand what happens when the automation breaks down basically and Part of that as you get into the sort of usability thing is well whenever you have the incident review well It's obvious why it broke right? It's always obvious after the event and the point about this book is how to make it obvious Before the event so that when things break, it's obvious what to do about that Our systems built out of hundreds of distinct services each of them has a rest API We let our engineers figure out what API they want It turns out that making up how to do APIs isn't really is on your own in from first principles isn't really a good good idea There's lots of bad patterns in there So the rest API handbook by George Reese is a very good background book lists all the bad kinds of APIs that are out there And if you're doing cloud he's been Interfacing to every cloud API from every vendor that's ever done cloud for for quite a while and that's where a lot of this background comes from Continuous delivery. I've mentioned this a few times Just humble in particular is a very well known in this space. There's a conference coming up I think it's November the 1st called flow con. It's the first time it's being held and it's in Santa Clara I think or it might be here if it's somewhere in the Bay Area It's a one-day conference first time I'm doing the opening keynote on it So I'm just trying to figure out how to write that up, but Very interesting about how to go faster and faster at getting value delivered to customers basically and finally if you're trying to do So TCO analysis and things like that Joe Weinman's got exhaustive coverage of all the algorithms and all of the formulas you need to figure out You know, what is the real way to do cost analysis and trade-offs between different kinds of environments? So there's this nice quote that says genius is 1% inspiration. That was the inspiration and 99% perspiration So we had quite a bit of perspiration as well as reading a few books And we've open sourced our perspiration way of thinking about it in this open source platform If you go to netflix.github.com you'll find I think currently 34 projects that are out there Why are we doing this?  That's really useful information. You can go and look at the alternatives So we're testing our ideas in public to make sure we've got the best practices in there We also use it to hire retain engage really top engineers. Some of you may have heard of the J clouds library It's a patchy standards coming out Last late last year were able to hire the guy who has bent the last three years building J clouds as an engineer He came to us particularly because of our strong support for open source That's the other the other Adrian Adrian Cole It also builds up the Netflix technology brand so I get invited to do keynotes at conferences, which is fun And then we're benefiting from a shared ecosystem as other people are working on our code We've been engaged recently with IBM who have done a demo port of the The Netflix example application and they've been working on actually figuring out how to port it to open stack and things like that and soft layer and all their environments, so, you know, we've got engagement from large vendors like that and also from end users the PayPal took some code We that we built and repurposed it to work on at their internal cloud things like that And we're getting benefit from that. We're learning more about how you know What the weak spots in our code and the missing things that we want to add to it? So that's our perspiration We've also got a cloud price. There's about one month left to run on this. So Deadline September the 15th This is to boost the ecosystem again. There's information about this on github We have 10 prizes of $10,000 for the best contributions to the Netflix open source platform That's $5,000 of Amazon credit included with all of those and you get a very geeky trophy that beeps and flashes lights and things like that And that the prizes will be announced at the AWS reinvent conference in Las Vegas So the prize winners get to we'll announce the nominations and then the prize winners will be secretly told and given tickets The flight to Vegas so they get announced there Okay So one of the things that the having our code out there Gave rise to was some demand for people that wanted to use it in ways that we hadn't been intending to use it ourselves In particular, we've got some vendor driven portability here There was interest in using some pieces of our code in these enterprise private clouds. So eucalyptus Basically the last build they did the last product release. They did they said it's it's done when it runs the Netflix code And they used us as an extended test suite and added some features So they started shipping that in June Cloud stack they have a they put up a 10k prize for the best integration You know inspired by the fact that we were giving away prizes. So there's some work going on there and With open stack There's some vendor and end user interest and PayPal built a console based on our console so that's kind of Interesting you put code out there people start leveraging it and figuring out other ways of using it And we're trying to drive that which but it's largely an experiment at this point We weren't quite sure what would happen when we open sourced everything and and had the prize and it's Been quite interesting to see what what's the reactions been So now I'm going to talk a bit about Netflix streaming and the challenges that we have there. So this is a cloud native application it's But it's based on this open source platform So we don't really have any enterprise software in up in our stack We can scale it as much as we want we never have to go talk to vendors We get all our code from github or we write it ourselves And that's a very different approach to the way that most people of all large-scale applications The website home page looks something like this The way this works is you have your own device which is in blue your web browser or your TV connected device That's talking to the all these yellow boxes which are all running on AWS and the red one which is our content delivery network Which is our own box? So the first thing that happens is you visit the website. There's a discovery API There's all kinds of personalization all sorts of information needed to figure out what movies to show you The next thing is you choose a movie you click play on it then the streaming API fires up and gives a the Basically the DRM points you at the right CDN to get the content and starts logging information about whether you're having a good time Watching it what bit rate you're using Whether you're rebuffering and it remembers where you are so that if you turn stop or you switch to another device We can continue from the place you were at so there's quite a lot of logging and a lot of up traffic going back into the system and then the CDN We we get all our content from studios studio partners We have to encode it into the right format so that happens on AWS And then there's a bunch of CDN management and steering back ends for controlling and tracking Where all this stuff is and we have a very large number of these boxes scattered around the world providing content They're like big static web servers There are box most of them are hundred terabytes in a for-you box basically and we ship them out to ISPs and peering peering points So that we do Several well many terabits of traffic through these and we basically outgrew the ability of the Commercial CDNs to provide us bulk traffic capacity that we need and the way we needed it So back in November there was a report that said how much bandwidth is being used and this is the fixed Traffic to North America fixed traffic For media streaming at peak time. So this is no not media. This is the total internet bandwidth delivered to people's houses over DSL lines and cable modems and things like that and Netflix is a third of that traffic So we kind of like that And most of our competitors a long way down the list. So that was good, too Then six months later. They came out with another report. The top line number is up 39% So the total amount of bandwidth being delivered that they measured averaged across all the houses was 39% more So this is a very fast-growing area. Everyone's got more machines connected. They're watching more. They're getting more bandwidth All those kinds of things and Netflix is still about a third and our competitors are still mostly small YouTube's getting big pretty quickly as well. So that's the main other platform Netflix and YouTube together is now more than 50% of all delivered bandwidth. That's quite an interesting way of thinking about it So if you look at that in more detail, this is what our web server really looks like It doesn't have those if you take all those big orange boxes that I had and I look look inside It's there's about 20 services behind the web server and it goes several services deep So in order to generate the home page, this is roughly what the the fan out of requests looks like Some of these nodes are Cassandra. Some of them are memcache s3 buckets random things like that Now if we lose one of these services because they would totally broke for some reason Maybe one of the rows on the website stops appearing Maybe it's the similar's row or the Facebook row or something like that stops appearing But the rest of the site still works because we have all of these other services So let's look at how that works from a no sequel storage point of view What we've built is this highly scalable available and durable deployment pattern using Cassandra and the way it works is that we have this single function microservice pattern and Cassandra isn't the thing in the middle of this. It's the thing in the top right That's a single function Cassandra cluster. It's probably got one key space on it It's got maybe a couple of column families Maybe one sort of think of it as one indexed table That's logically the way to think about it Maybe a little more complex than that depending on exactly how we're trying to structure the data in it Our smallest cluster we deploy six nodes the biggest ones right now are 144 We have quite a few in the sort of 24 48 kind of kind of scale In front of that you put a single function rest data access layer service And everyone goes through that so all of these things on the left are all the clients that want to get access to this information They all make HTTP calls to our to our auto-scaled layer in the middle And that totally hides the fact that it's even Cassandra. You don't know it's not not available to the clients They're making a rest call. Okay, so we have over 50 clusters following this pattern over a thousand nodes in Cassandra the backups compressed backups are over 30 terabytes and The biggest clusters are doing several million writes a second right now You also can see this optional data center update flow The point about this is this gives you a migration mechanism So if you've got your existing system, it's a big database big sequel database or whatever You put a data access layer in front of it and you could convert your applications to get everything out of that layer In fact, you put several in front of it You take each of these tables and materialized views or whatever you've got you put a date to access layer for each of them now behind that It's still your big relational database But now you can put caching and now you're building against this new world of a fine-grain Distributed system and now you can start making copies of this data in the cloud and that's what we did That's how we got from we still have the DVD business runs on a very large You know IBM machine with a very large Oracle license and a big sand and all that That was the code base. We were on four or five years ago We split it off one piece of functionality at a time into distinct Cassandra or in fact originally simple DB But we switched to Cassandra clusters in the cloud So we made a copy of the data in the cloud and we kept everything up to date we wrote checking code that would copy things back and forth and Eventually we turned off the data center access and made the master copy the copy that's in the data set in the cloud and We now have new functionality that's only in the cloud and the data centers just has the remains of the DVD business in it which is I think we have I know 37 million or something Approximately streaming customers now 36 37 million and something like eight or nine million DVD customers I remember I forget they vary a little bit, but that's basically where we are Now when you're building these client systems So these are the systems these single function rest clients they're all trying to use Cassandra to do something or your data access layers trying to do something and You end up reinventing the same recipes over and over again So we started collecting these recipes and we published them. This is part of our open source project So for Cassandra, we have a client library called Aston access for Java client library And these are the some of the recipes that we have for it You can see there's a fairly high-level operations. Some of them are quite powerful like Large file storage if you want to store a gigabyte size chunk or a really really big chunk in a no-sequel database And you store it as a single blob of data. It tends to blow up the database There isn't enough memory and caching and it will hit it will land on one node Which will probably time out and you'll retry and eventually just fail what this does is it splits the data into lots of chunks spreads those chunks over the entire cluster and Make sure the chunks are small enough that they if you fail you just retry a small trunk S3 has a similar Multi-part write mode and this is so modeled the same way Very powerful for and it works for reading and writing the same way And we've actually from the cloud prize somebody contributed a high cardinality reverse in index pat recipe for us So this is what it really looks like we have all these microservices. We're testing them I was talking about anti fragility. We're testing them with chaos monkey that kills an individual service We use latency monkey, which doesn't kill a service It makes it slow and it injects errors into it so you can get a machine and say I want you to just return 500s for a while and see what happens So I want you to respond in 10 seconds instead of you know 100 milliseconds Whatever the normal time would be just to see what how how the errors ripple out and Do we have bulkheading properly set up so that you know that the consumers of that service should sort of contain the fact That it's now a bit misbehaving We have conformity monkey, which goes around looking at all these services making sure that they're set up correctly We have a bunch of rules that we write that that basically are the architectural patterns for how things should be set up And we have a whole load of other monkeys, too This one we have chaos gorilla so we if we take that pattern we replicate this pattern three times So we use availability zones on AWS. So we have three zones We have three copies of all our data We use a load balancer to randomly pick a zone and make a request to a machine in that zone and then we use Cassandra and at the bottom to move all the data back and forth and keep everything in sync and we have a Evie cache is a memcache debased Replication library that we built that makes sure we have three copies of your memcache one in each zone so now we also have a chaos gorilla and We actually took run this in production once a quarter I was at the know which conference at cloud connect conference in March, and we actually did it during the conference It was actually the Tuesday of the conference We ran this in production took out a third of our infrastructure Almost 3,000 machines were shut down as fast as we possibly could and there was no Netflix outage There were a few errors We found a few things that weren't quite right, but it didn't cause a customer visible outage It caused it was basically this is the anti fragile stuff in action. We actually are stronger because we run these tests We run them once a quarter just in case we do get an outage We don't want to hit the quarterly availability number too hard but if you do get this if you lose an entire building because of you know a hurricane goes over a Virginia or something or There's a power outage what you really just want to do is turn off traffic to it So that you just tell the load balancer stop sending traffic We can do continue to run on two out of three It's a quorum based voting system as long as I've got two out of three or my queries will continue to run So that gives us a very powerful mechanism to aggregate any kind of failure up to a zone failure It's like if it's anything at all goes wrong if I can just hide it You know, I don't really care what went wrong as long as the only goes wrong in is that one zone at a time And AWS has got better and better at zone isolation that some of their outages from two or three years ago were caused by things leaking across zones and You know an outage in one zone trickling across to the others and they've got better and better isolating those So what we're doing also now is isolating regions We use Europe EU West and US we decided we wanted to isolate those two regions So that they're running very independently, but keep the customer data You know the fact that you're a Netflix customer is global. So if you travel to Europe you your machine You know your account still works You don't see the same set of movies because you see whatever movies we have in the UK, but your account works Now if we lose the connection between the two because of the way Cassandra works, it's a an AP system It's a available under partition. This is a partition both sides are available for rights So all reads and writes continue and if you happen to figure out how to you know update both sides at once Then whichever upstated latest wins when when it gets fixed it comes back and it re-synchronizes automatically So this is the powerful This is the thing Cassandra is actually really good at that we've leveraged a lot If you lose an entire region the other region keeps running. So that's the key thing So these are the failure modes that that we're looking at Application failure we expect this to be happen all the time because we're shipping code And if it's not happening occasionally, then we're probably not shipping code fast enough, right? So we're expecting that to happen. We want automatic degraded response Region failure is relatively infrequent But it's happened often enough that we decided that we wanted to be able to switch traffic between regions and we're working on that now I'll talk a bit more about that later Zone failure it's part of sort of the normal operation of cloud is that you should expect zone failure occasionally So we want to continue to run under two or three zones with no impact data center failure We've now moved all our dependencies into the cloud So we can continue run to run with our data center completely down and we've had that happen a few times in fact Many of the Netflix outages over the last two or three years were actually data center outages that caused the cloud side to fail One of the last things we moved was hardware security modules and AWS has HSM options There's sort of safe net lunar boxes that hold all your crypto keys We were calling back into the data center to get at those keys for a long time and AWS has that in the cloud now so that we don't have to If you get a data store failure, so you know Cassandra corrupts something or application code scribbles over data And then you need to go back to back up So we have S3 based backups were continuously storing stuff in S3 and if S3 fails totally We want to copy of our data that wasn't on S3 So we have one extra copy that we call we store in a remote archive That is not on S3 We do that sort of a daily update just sort of you know, we still have all of that Netflix data somewhere else So now talk a bit about Cassandra at scale and what we've really been doing here is benchmarking to try and retire risks As we went out and use we pushed Cassandra harder and harder some of these benchmarks were us going well How hard can you push it and and is this going to work? The first one we did In 2011 two years ago was we were currently running on 24 We maybe just started using 48 node clusters and the question was well How fast how far can you scale and we scaled up to 288 nodes just linearly went, okay? I give up it's probably going to continue to be linear. It's that's far enough And what we really came down to as I said can we get a million rights a second? I don't care how much with lots of machines and The data sacks guys started using it in their marketing. So there's actually this million rights in a second pop-up Advert was happening if you clicked on it eventually took you to the blog post. I wrote on how we did this This was using 288 really quite small machines only For CPUs eight ECU's which is the sort of compute thing the biggest Amazon integers are 88 CPUs So this is a much smaller system than you could use A year later We've got solid-state disks So now we ran a benchmark which compared a non-SSD versus SSD and we were able to do away with memcache D and get Basically the same throughput and lower latency But a half the cost because we could run on such a small cluster of machines compared to how many we needed to keep Cassandra running with regular disks So that's we now run. I think Well over half of our Cassandra nodes are SSD based several hundred of them This year we've got some cross-region use cases We've been using it for geographic isolation US to Europe as I mentioned, but we've got a new one We want to do redundancy for failover between East and West Coast and there was some people Well, I'm not sure how well that's going to work Can we trust it how reliable is it if I write it in California in Oregon? How soon is it going to get to Virginia all those kinds of questions? So we decided to run a benchmark on this This is what the benchmark looked like It's we took our most right intensive cluster The one that we hit hardest because previously we'd only be doing multi-region and read intensive because that seemed like an easier thing to Go do so what's the most right intensive one we could and it turns out that AWS had enough spare SSD instances at that point in time a few months ago that we actually grabbed 96 of them and we stood up a 96 node cluster with 48 in Oregon and 48 in Virginia Which took about 20 minutes And this is off a hallway conversation. There was no like going and arguing with anybody about how much this was going to cost Because we only needed it for a few days anyway, so this is a hundred ninety two terabytes of SSD And find it all up and running 20 minutes. So who here can do that? I mean who here thinks that they can fire up a hundred ninety two terabyte SSD Cassandra cluster in 20 minutes Okay, all of you can all this code's open source It's public cloud you just download the code push the buttons and 20 minutes later It's running really it's this is the level of automation. We've got but the all of this is open source code There's nothing special here. We might take a bit longer to get it set up the first time But but that's that's the power of having this stuff up So challenge you to go away and build a cluster now So we've got these two clusters all hooked together The first thing we did was pull 18 terabytes of backup data into one of the clusters that took an hour or two to suck out Of s3 it's 48 nodes in parallel just pulling data in Each of these nodes is two terabytes of SSD and a 10 gigabit network port So my bisectional bandwidth across the country is 480 gigabytes gigabits per second, which is quite a lot really For the internet and then we said well, let's push that 18 terabytes to Aragon and see how quickly it gets there We're a little worried about how fast it was going to get there it might get there a bit too quickly and break something But actually it only ran at 9 gigabit. So it wasn't too bad That's actually not very much That's 48 single threads of data being copied over and we in doing that the we use the boundary comms Network monitoring tool to look at all the traffic flows here And it was measuring a TCP round-trip latency of 83 milliseconds very stably for this. So that was quite nice The boundary I've got a bunch of graphs I use for black boundary, which I didn't have time to include today so We then put test load on both sides and then we also Put a validation load where we wrote a million writes to one side and we read them all back from the other side Half a second later. So you write and then you read back just to make sure that the data got there and they all got there So this internally engineering sort of people that it within Netflix that are a bit dubious about it mostly went Yeah, I guess that's going to work and And we're we've now have this running in test and we're bringing it up in production So over the next few months, we'll actually be running live before the end of this year in this mode a Couple more things we've now got customers that we need to split between these two coasts So we need to use DNS to do that But every DNS vendor has a different API or had different features So we built a library called denominator, which is the highest common denominator of DNS features a very powerful library And it's also got a command line tool if you're doing any DNS management with any vendor just go and get this tool It's a really powerful way of doing it This is what Adrian Cole has been working on for the last six months if you know him. He's been having a lot of fun with it So now if we lose a region with a zone We don't care if we lose a region We stop talking to it and send all the traffic to the other region if we lose a DNS vendor We switch to a different DNS vendor. We have all of the DNS data abstracted outside it and a globally available Cassandra backed data store So what we're trying to do here Is get our get the problems to go away? So the biggest problem you have is a PR level incident and a few times a year There's an outage that's big enough that it makes it into the press and people write articles about it And that has a much bigger impact on our customers than the actual outage that outage itself So you want to avoid anything that hits the public relations impact Below that there's probably ten times as many that actually cause customers people to call CS We don't want those because customers that call CS are more likely to quit Netflix So we want to keep people happy you never want to push them to the threshold where stuff doesn't work And then below that there are ones that cause the just affect the quality of what you got in the test results So by active active and game day practicing we're trying to push down incidents that would otherwise have been PR incidents to be just CS incidents and then by Better tools and practices get the things that would have been CS incident and move them down to be just feature features that got disabled And then by better data tagging we can take all the feature disable and Have clean data on what who actually saw which feature when we're doing all of our AB testing analytics So that this is kind of the way to think about that Okay, I've just got a couple more slides Cloud security is always a big question I don't there's a whole presentation on this that I've linked to here from Jason Chan If you go to slide share slash Netflix slide share net slash Netflix you'll find lots of these But one of the things we do is automate our attack service monitoring So we whenever you create a new s3 bucket There's a thing that will go and find that you've created it and check the permissions and make sure that they're correct Whenever you push new code We automatically do a penetration test against it to make sure that it's good code and we'll send you a little note saying by the way We're testing it your logs might have filled up I've talked about cloud HSM key managements really critical and then the other thing is that AWS is at such a large scale that a lot of the Concerns you have about DOS attacks and things are really mitigated by the Amazon layer So managing scale and complexity these impossible deployments things like hey I need 200 terabytes of SSD in 20 minutes, and I just just thought that up That's not really possible any other way We're jointly building code with partners in public our Cassandra work was done jointly with people from lots of other companies The denominator DNS management. We've been working very closely with the ultra and dine and rack space and HP and the support for a bunch of other DNS backends that we don't use ourselves This gives us this highly available and secure system despite the fact that we're we're running at big scale and we're running really fast So here's some links to various things. We've got We've got some blog posts of their meetups that we've done Where we had contributors and different lightning talks talking about the individual projects You can't read the URLs, but the slides will be out and about if you're particularly interested in cost aware Monitoring how to optimize costs on AWS. I did a joint talk with Janesh Varia, which is actually on the AWS cloud Slide share site rather than the Netflix one Okay, so final slide Which basically we're using cloud native to manage scale and complexity at speed and Open-sourcing it to make it easier for everyone else to become cloud native. So that's the that's your takeaway Happy to connect to anyone linked in I'll be around for the next few hours I'm afraid I have to wander off before the evening, but thanks very much


What's up everyone it's your girl Kenty here from Leighton Sports Cards, and I'm ripping a personal year for Where we go? Patrick M. We've got a order of the 2022 Panini Father's Day packs again these come in bundles of five got one two three four and five Best of luck here for you Patrick. Thanks for hopping in Got Henry Davis and rookie of Garrett Wilson Nebron James and a Joey B Pack it got Julio Rodriguez rookie and a Cade Cunningham rookie another nice pack We've got a Cade Cunningham rookie and Red rookie of Garrett Wilson out of 99 color match there on the Ohio State Nice and last pack here for your Patrick Very solid pack so far Price Harper and a David Beckham all righty that will do it for your five pack sinks again Patrick and we will get these right out to you


A bill that seeks to establish the North-West Development Commission has killed second reading at the Senate. The bill, sponsored by Deputy Senate President Barack Obama, is one of the three bills read for the second time at today's plenary of the Senate, presided over by the President Gutswill at Mavio. The bill was introduced to the Ninth Senate in 2019, but was not assented to. Now, with the reintroduction of the NWDC Establishment Bill 2024, it appears every region wants a commission. Leading the debate, Senator Gebrin noted that the commission seeks to act as a catalyst to develop the arrays of potentials of the Northwest, as well as address the gap in infrastructural development of the region. The bill seeks to provide for the establishment of the North-West Development Commission to act as a catalyst to develop the arrays of potentials of the Northwest, as well as address the gap in infrastructural development of the region. The Northwest, Jupru Zone as a region, has contributed immensely to the overall development of Nigeria in terms of agriculture and other areas of endeavor, yet the region remains usually underdeveloped. What the region requires now from the federal government is the support to develop its infrastructure and educate its team in use, so as to drive the process of developing the zone in its entirety, in order to grow at par with more developed zones in the country. Lawmakers who supported the bill include Gatsby, Lafabio, Adamo, Aliero and Arvomoro. The North-West is very large, the youthful population much, the need for development very clear and insecurity will be minimized if more development enters there and educational facilities are made more available, so for us development anywhere in this country is development everywhere, so I'm not lobbying you to accept, I will see put the question but my opinion is that I'm a lover of development. A lot of devastation, a lot of destruction due to the activities of Boko Haram and also the bandits. The North-West had the largest number of out-of-school children, in fact only yesterday I saw the Minister of Education addressing the members of the House of Representatives on the need to give attention to out-of-school children in Nigeria that numbered out of 20 million. If the President is the only country in the world where we have 20 million people out of school. As a result, Mr. President, it has become imperative that if we have to develop through the aggregation of developmental efforts from the states and the zones, then Nigeria has a need to accommodate these zones as developmental centers and that is what this bill seeks to achieve. And so Mr. President, as the world continues to modernize, there is an imperative need for all of us to agree that we must establish centers of development.


Zdaj. As I was telling you, there is now a change in the schedule. Professor Suganidi will teach now his third lecture in place of Felix Otto. And Felix Otto will teach tomorrow in place of Professor Suganidis. So thanks also to Takis for giving the possibility to make this change. Thank you for waiting. Can I start? Okay. I think I did. Where's my friend? He didn't come today. Where's my chalk? You move from the seat. There you drink for that. So I derived the formulation with a distance function. So in the case where we had motion by mean curvature, we said that a tripled was by mean curvature if and on if the sine distance function x, t satisfies inequalities. So it is a subsolution to the heat equation in the set where it's negative and a super solution to the heat equation in the set where it's positive. And in addition, when in this set also satisfies it's a distance function so the gradient will be 1 and in this set it's also a distance, I'm sorry. In this set it's a distance function so it will have a gradient 1 and here it's a distance function and it will have again a gradient 1 but since we are describing this in the viscous distance remember the actual condition is it solves this problem and in the viscous distance these are not equivalent. And I didn't mean that I meant minus, minus plus, yes. And I saw you waving my hands that that's equivalent to the level set formulation. That's equivalent level sets. And then I said that now we are ready to start studying the asymptotic behavior of reaction diffusion equations and I wrote down this the u prime of U epsilon equal to zero where w is a double well potential like that with wells of the same depth. Let's say to normalize them at minus 1,1 and that implies that w prime is something like that in the two areas that's the other way around. Anyway, it's this or the other, it's not this. All right. And I said that the result, the conjection actual result here is that U epsilon goes to 1 minus 1 inside outside in mean curvature flow. Make it precise at zero let's say is positive in omega zero and negative in omega zero complement and to simplify things let's assume that we have no interior at time t equal zero. So I promise to show you why this happens formally but right now let's accept it because it fits very well with what I did here and try to prove this in this very particular case using this result here is a nice cute proof that it's like magic so I'm going to make the assumption I have to make another assumption U zero is well prepared know that this is a very illuminating assumption which means that's a special about such a reaction diffusion equation. Special thing is that meets traveling wave solutions. So when you have a by stable nonlinearity because that's as I said it's either this or the other when you have an inequality like that there is a traveling wave there is a standing solution there is a solution to this problem of the form minus c minus q double prime plus w one prime q equal zero for q being plus minus one so any time you have a nonlinearity that comes from a potential like that there is a theorem that says there is a solution of that form so this is in one dimensional solution of the type that solution exists the speed c is unique and the solution is unique after translation by constants and the only condition is that it decays exponentially to the equilibria at plus minus infinity so that's a fact for any w like this there is a solution like that so I rewrite it and unique it's increasing it's unique after translations so let's make it unique if we assume that q of zero is zero and c is unique that's a fact of life and as a matter of fact you can even have an expression for c because if you multiply the equation by q prime and integrate so if I multiply by q prime integrate from minus infinity to infinity from here I get minus c q prime square that goes away and here I get plus w at one minus w at minus one equals zero so the speed is exactly like that and somehow it's unique although it's multiplied by this factor but it's unique so you see from here that if I'm in special situation where the two points are at the same depth these are the same so if w at minus one is the same as w plus one then the c is zero and it is in this case we are working here because I'm making that assumption and as I will indicate later on this is why I can put an epsilon square there ok, so we do that so my normalization here is that this will be where this is a traveling wave ok, so with this assumption let's see what we do this assumption suggests that perhaps we should make a change of variables you may think this is crazy but once I show you the formal proof you're going to see that that's exactly what the formal proof does but let's start with this and now let's compute what does the epsilon do so the epsilon solves this equation a little bit faster, just through the computation up to here without dividing by q prime you get what comes from the Laplacian in the time derivative then I use that f of q I use the fact that q double prime is equal to w w prime q I use this fact q double prime that q remember when c is zero the w solves that so that's the q I'm using and if you do here the calculation ok, let me do it slower so I compute the equation for z epsilon so I write down derivatives and I get 1 over epsilon q prime z epsilon t minus 1 over epsilon q prime Laplacian z epsilon minus 1 over epsilon q double prime epsilon square dz epsilon square plus 1 over epsilon w prime q and that should be zero on the other hand by the choice of q this term here is q double prime that's what q does so that eliminates w prime brings in the q double prime 1 over epsilon q dot z epsilon t minus 1 over epsilon q dot Laplacian z epsilon minus 1 over epsilon square so this is q double prime I bring it in and I get I simplify by the 1 over epsilon divide by q prime remember the travel wave is increasing so I can do that and I'm left with the equation z epsilon t minus Laplacian z epsilon minus q double prime over q prime evaluated at z epsilon over epsilon dz epsilon square minus 1 and I claim that this problem is much easier to understand that what was the original before so for the students this is a situation and it happens often in PDE where if you start with a linear problem or semi linear in this case in the form of w prime and you want to get some estimates and it turns out that occasionally it's better to make the problem nonlinear because once you make the problem nonlinear you break the symmetry of the linearity so you may be able to to get something more useful in another context when this thing comes up all the time in a very simple form but the problem is more general than that for example if someone gives you the Laplacian if you assume that u is positive often what you do is you make this change this score is known as the hope of cold transformation and that leads to that nonlinear PDE but because of the presence of this nonlinearity you can do more things not of course in terms of value but if you want to get estimates and so on you can do it this way and in some sense this is a transformation that we are doing here the only difference is the q double prime being a function that is like that you have different behavior for this factor at either end ok so this is perhaps one of the few proofs I'll give in full detail so clearly I want to send epsilon to zero and if I send there is an epsilon here because this was the only term with a 1 over epsilon squared so I want to let epsilon go to zero and the only thing I am going to get if I let epsilon go to zero is that if the z epsilon has a limit I will get that d z squared equals 1 multiply by epsilon let epsilon to zero this factor this coefficient goes to limits so let's say this goes to which are non zero you can compute it easily from here so if I let epsilon go to zero the best you hope to get is d z squared equals 1 let's assume that the z epsilon have a limit assume for now that the z epsilon has some limit and what do you expect at the limit to have you would expect that d z squared to be 1 that looks good because it looks like a distance and if you could solve that that goes to zero really fast you will get z t minus Laplacian z equals zero and somehow if you were on the front you had a front movie by me curvature of course this is a dream and we are not going to get it but let's see what we can get is the limit satisfies that so I will prove for you assuming there is a limit I don't want to get into these technicalities that the limit is a sub solution to the heat equation and it is a distance with a negative sign when it's negative satisfies that when it's positive and therefore it's motioned by main curvature so let's see how we do that don't want the quality at the limit I want inequality so somehow the only hope I have to get this inequality because I know equality will not hold is if I can create a sign here if this thing can we can find a sign for that quantity sign s i g n so the first claim I am making is that since d zero is less equal to one gradient of z is less equal to one if you say I know how to do it it's a parabolic PD there is a code for the kinds of things you can do for the equations to get Lipschitz bound and it's called Bernstein argument Bernstein type argument which suggests, which basically says write down this quantity you want to compute to estimate subtract something that depends on you write the equation satisfied by that and apply the maximum principle after you make a lot of changes here, there if you try to do this you are doomed and the reason is that once you hit this with derivatives you bring in more epsilon and you bring in cubic in some sense you create a cubic term in the gradient and that is a problem but there is a shortcut to that and this is I always do that when I teach course solutions it's a beautiful shortcut which says the following let's define the function and w is not good anymore psi of xt to be the soup of zepsilon xt minus x minus y I'm saying something stupid sorry for that zepsilon xt minus zepsilon yt minus x minus y look at that expression you double the variables we want to claim that we want to prove out of that this estimate it holds initially so the only thing I care is that look at the maximum of that for t positive because if the maximum happens zero then I have the answer so you look at that and then what happens at this maximum and then of course you have to assume there is a maximum so look at the maximum of that so two things will happen so let's call the maximum x bar, y bar either x bar is equal to y bar at the maximum that will imply that zepsilon xt is less equal zepsilon yt so that's one direction of that bound that's one direction of this bound or x bar is not equal to y bar there are two alternatives but if x bar is not equal to y bar remember the zepsilon has smooth functions gradient zepsilon would be equal to x bar y bar over x bar minus y bar because that's when x bar at the maximum minus y at the maximum that's a differentiable function if you're not at zero and since the zepsilon is a smooth function the gradient will be like that which of course is one and if you use the definition of the maximum that implies the answer because that implies that zepsilon xt is less equal zepsilon yt plus x minus y so that's the idea of the proof so you get a lepsich bound for free but it was important to have it initially that's why I said I have to prepare well my initial data once I have this bound once I observe that I have this bound now I'm home free why? because let's see what happens I let now epsilon to zero look at the set first z negative z remember is the limit of the zepsilons so look at the set where is negative if the limit is negative that means the zepsilon for small epsilon is negative somewhere which means that this thing is negative so when the limit is negative for small epsilon this quantity is negative because this quantity is positive I'm sorry what is this quantity? it's positive at minus infinity and it's multiplying by minus so negative negative positive again we know it's negative this is negative it's positive for small epsilon that means that actually I have this inequality because if this is positive this has to be negative and therefore letting epsilon to zero I get this inequality in the set where z is positive I still have this negative but now this is positive and so I get zepsilon t minus Laplacian zepsilon greater equal zero and after I let epsilon go to zero z t minus Laplacian z greater equal zero I have half of what I want but I did not use the big thing I had from the beginning that goes to zero so if you let epsilon also go to zero you get this goes to a positive so now we look what happens here you multiply by epsilon let epsilon go to zero this goes to negative constant back there and you get at the limit minus dz squared minus one less equal zero because dz squared is less equal one you get equality but with the minus which together with the fact that dz is less equal one will give us minus dz squared plus one equal zero and we get dz squared minus one greater equal zero and now we switch together with the dz less equal one gives us dz squared minus one equal zero this is in the z negative this is in the z positive and now I am done because you go back to the original chains and what we prove is that whenever the limit is positive this goes to one and whenever the limit is negative but when it's positive is inside the set that moves by mean curvature and it's negative outside the set that moves by mean curvature and that's in some sense the slickest proof you can have for that but in general when you have a very slick proof you don't hope this to work for everything in particular this proof will not work if I make the following simple chains so I allow next dependence on the potential or I put something anisotropic in front and the reason is that for this particular proof it's going to fail very quickly with when you write down the equation where it's going to fail it's going to fail because x by x the traveling wave now will depend on x the assumption is that for every x this is a cubic nonlinearity I mean it's a by step whatever is a double well potential with wells of the same depth so the q will depend on x this is a parameter so when I write when I make the change of variables q x z over epsilon I can do that nothing prevents me from making that change of variables once I start getting derivatives I'm going to start producing derivatives in q with x which will create additional terms here and if I create additional terms here so my trick is not going to work anymore are there any questions about this this proof I know it looks like magic and it's short but as I said it's life ok some history about this problem with proofs first proof for something like that was given by Kohn and Bronzer in the radial symmetry case then there were a result was a very nice result by Timotoni and Satsman up to the first time you had singularities and I mentioned Timotoni also because Timotoni had a house a little bit out of here and the last time I was interested he invited us and had a beautiful view and so on he died in an accident so years ago so Timotoni and Satsman and then Chen who is at pitch proof there are a lot of initials there also provide the same proof again up to the first time there was singularity the first proof past singularities by Evans and Sone and myself some time ago and then more work happened took place ok now let me go back I show you this proof, it looks like magic let me describe to you if you were an applied mathematician which in my department will be a sin but if you were an applied mathematician you would have figured out that motion by mean curvature plays a role in this problem so we forget this now nice rigorous proof and we go to the argument of this classification of mathematics into pure and applied is really awful because it suggests that people who do applied math are not clean or pure or whatever so together we decided that the better definition is classical versus modern mathematics you pick which one you prefer to be that and if you wanted to go even further you could say static versus dynamical mathematics anyway so so here is a formal which in the end turns out after a lot of a big cycle of work to be able to justify rigorously and as matter of fact there was a whole cycle which I will try to describe to you so the first proofs were very elegant and so on and then at the end with Bar we figured out a way to say that we don't need all these elegant proofs that we can do the problem in a different way faster ok so formal derivation we start with this problem we expect so W is let's say a potential double wells just to simplify things one minus one a general potential like that and you expect to have you want to see what happens as epsilon goes to zero so this is the same as saying I start with this reaction diffusion equation in study what happens in the solution as t goes to infinity of course this problem had been there even before Allen and Kahn Allen and Kahn made the connection with mean curvature but people like researchers like Aaronson and Weinberger had looked at the behavior as t goes to infinity of this problem and what you expect to see as t goes to infinity is the solution becoming either plus one or minus one and then of interest is to see what happens to the interface in between the plus one and the minus one which for t large whatever is a smooth transition so if you want to figure out if we want to figure out long time behavior it is often very convenient to introduce a scaling because the behavior of u as t goes to infinity is similar to the behavior of u epsilon as epsilon goes to zero and if you do that you have to figure out what do you space in principle if you do that x may go far away so what you are trying to do is you try to keep things in a compact set so you introduce the scale so what this does is it tells you what happens for long time and long time large space so this is long time large space behavior and basically what you expect is this to go to one and minus one long time it will go to the equilibria and for large space to the right and left you go there so it makes sense therefore to start with this problem and so what Keller Rubinstein I really recommend this paper it's a beautiful applied math paper with nothing rigorous but that's how you learn to do formal asymptotics if you don't know how to do and it takes a little bit of getting used to it so I don't mess up the notation what they make they make the conjecture that they can write a solution which will depend on epsilon they can have an asymptotic expansion for the solution and the asymptotic expansion will have a first term that will call it u0, x, tau bear with me I'll put all the variables in it and then there will be a next term in the expansion and of course other terms if you are brave so let me tell you what the things are the z here will be like some phi of x t over epsilon so this z there will be like the capital z epsilon I had before tau will be the time t over epsilon and eta will be the time epsilon t so we are looking something for a long scale we are putting in a scaling and this type of expansion allows for the possibility that we over scale the problem that actually what happens happens at slower t and that's the eta t there or it happens for longer t which is the t over epsilon writing it like that you capture both behaviors and the assumption is that in this phi in principle will depend on eta 2 and you make these ansatz the ansatz is that the solution should look like that and now we try to figure out what phi does so to do that we write down equations namely you do the calculation I did before with the z epsilon but now you do it for all this mess so if you do that you are going to get terms that have one over epsilon in front terms are going to have nothing in front in terms that are going to have epsilon in front and so on we care only about the terms that have one over epsilon in front and nothing so you get the following two equations u0 tau plus phi t u0z minus d phi square u0zz plus u0 equal 0 this is the term you are going to get the expansion for everything that has one over epsilon in front notice if I take a time derivative here I get a one over epsilon derivative with respect to tau I have the epsilon there and when I do this term I will get this same way I got a q double prime there so this is the thing that multiplies the one over epsilon and the thing that multiplies epsilon to the zero is u1 tau plus phi t u1z OK, where did this thing pop out? I do a formal expansion so if I plug that in I have w prime of u0 plus epsilon u1 and I will write this thing as w1 u0 plus epsilon w prime u0 u1 plus higher other terms there is a one over epsilon in front so when I find the terms that do not depend on epsilon I will get rid of that epsilon if you ask me why don't I go higher I know I don't need to go in more derivatives and everything we are assuming we can do these things so we got these two equations I didn't finish that this is going to be equal minus u0 t plus left plus and phi u0z plus 2 grad phi dot du u0z minus phi eta u0z OK, that's as general as it can go and here I didn't say at t equal zero is phi zero x over epsilon so that was our initialization does not depend on eta t and tau of course say it again, how do I get a second derivative this is dw alright so we do a formal analysis so basically what we need to do is we need to find u0 and u1 and we need to find phi and what is going to happen is phi will come up as what phi does is going to turn out to be a compatibility condition in order to be able to solve u0 and u1 so when you do something like that a formal expansion you are assuming that at some point you can take the variables to be independent of each other I know that's weird but that's in this is the applied aspect of what I'm doing is although everything here is mixed with each other I'm going to look at these things and I say once I have one variable the other variables are constant it's a completely formal argument and then you have an equation here this is a straight equation for u0 there is nothing else there so either you are able to solve it or not and then you have an equation for u1 which looks more or less the same up to here and there is a right hand side and anytime we learn in PDE to solve a problem with the right hand side we know that we can do it provided the right hand side is orthogonal to the kernel of the adjoint so we know therefore that in order to be able to find u prime this thing to be perpendicular to the kernel of the adjoint problem in that condition that compatibility condition will give us the equation for phi so you should always think of that because that's what's happening in these problems the equation satisfied by phi is a condition you need in order to be able to close this expansion that more or less says that I don't have to worry I can close the expansion, I'm done ok, now let's figure out how we do that and so now we make more assumptions we're going to make the assumption that, I mean somehow you know what you want to prove so we're going to assume u0 of zx tau t eta looks like a traveling wave t for tau large tau, which is this variable t over epsilon and of course we can think of tau large because it's fixed t, but it's like epsilon going to zero, so you make these answers now with these answers we try to simplify the equations now if you ask me how do you make these answers when you start something like that you have a little bit of an idea of what you're trying to get you know for these problems where you get the old results of Arons and Weinberger and others where you knew that there were traveling waves and the solutions are attracted the traveling waves attract the solutions and all this ok, so you make these answers and once you do that you can simplify the problem so the first equation if the u0 is like that the first equation becomes so now I replace the z derivative by derivatives q double prime becomes a prime so this is like that so now you see I start coming back to the thing I had, like the traveling wave so the q has to satisfy like that also the assumptions here will be that as z goes to plus minus infinity this should go to 1 or minus 1, the two equilibria or the w so I have this that's what the u0 has to be like the q which is like that now I do the same thing, I multiply by q prime integrate the xi let's say and I get that as before that phi t minus c should be equal to minus w plus 1 minus w minus 1 divide it this is the computation I did before so I multiply by q prime integrate and so on so I get this and now if you want to simplify things a little bit more if you make a change of variables because remember what is the q? what is the argument of the q? the argument of the q is phi minus ct over epsilon that's really the argument of q and so if you want to eliminate that, eliminate the epsilon from there so in principle the way it's written the q depends on epsilon of course it's not but there is an epsilon inside the q so to eliminate that you make an additional change of variables you write a z which is now z minus c tau over d phi that's just addressing now in this problem in that case you find that this problem becomes forget that so you make this change and if you plug it in and use the calculation you find that phi t minus over c over d phi is minus w plus minus w minus so if we write this as like that we keep it away ahead and so you see that if there is a non-trivial right hand side the phi solves this equation and the non-trivial right hand side has to do with whether this is zero or not if this is zero best have something here I think you get zero which is the case of the double well potential of the same depth then basically you get that phi is independent of t and the next dependence on phi is now on eta so that implies that phi is really a function of x and eta so to figure out what happens now we have to go into the next term so we go into the next term finally how the formal thing takes so much work but I decided to show you all the details so if this thing gives us zero information because if c is zero basically there is no c here this is gone so this is phi t minus c this is gone and what you find is the traveling wave I had before the one so now you make the same assumption in u1 bare with me the same assumption I made u0 now I'm making it for u1 and if I do that to simplify let me write directly what we get here we get minus d phi square p double prime plus w double prime q p is equal to a plus in phi plus 2 grad phi grad ok so now how does the Fragment alternative come in Fragment alternative is going to come down for this operator so the operator in which you are going to apply Fragment alternative is this one you have this problem let's linearize it it's easy to see by taking a derivative that q prime solves this problem and q prime goes here and the form of the w implies that the kernel of this linearized operator linearized operator is minus d phi square p double prime plus w prime of q prime the assumptions of w you imply that there is only one dimension in the kernel so in particular any solution and because the q prime does it the kernel of that operator the kernel the joint of L star is basically q prime which means which implies this mess here integrating from minus infinity to infinity Laplacian phi I'm messing up something here sorry for that just the correct right hand side it means that Laplacian phi minus phi eta dot square plus 2 q prime integrate from minus infinity to infinity grad phi grad q prime square d psi should be zero that's the orthogonality condition I mentioned that's Fragment alternative the q prime comes up to the factor here the big assumption since everything is formal is that the variables we can freeze any variable we want so we get that now this looks a little bit more weird but this because of the two this becomes minus q becomes d phi dot gradient with respect to x of q prime square now you say oh there is gradient we integrate in part remember the integration is in psi is not in x there is nothing to do with mean curvature the reason why this thing has to do with mean curvature is the following remember this change of variables we made that change of variables had the effect to tell us that the only x dependence on q was through that so q is really q of psi in x is really r psi that's what it is so now I can compute the derivative of this because I know exactly what is the x dependence and if I do it carefully using that in mind I'm going to get that q double prime square phi prime, phi eta minus delta phi minus grad phi grad phi over grad phi x i equals 0 so the only thing I'm doing is I'm using this form of q so when I compute q psi I bring in a gradient a derivative of 1 over d phi when I compute a derivative of 1 over d phi I'm going to create this term I mean this is d phi here but this is nothing else but the mean curvature so this is a constant this has to be 0 so that is 0 and that's exactly if you compute these terms this is exactly the motion by mean curvature equation so that's the connection in a very semi rigorous semi so just 5 minutes this approach that I presented I'm going to do next time next time I'm going to show you how to look at problems just to show you what else you need to do if there is additional dependence and the answer will be again now that we know how things work it will be that the u epsilon will look like some q let's make it simple x plus epsilon p distance over epsilon x the meaning of these things is the same and we're going to have to choose the p so that we can control the errors we make when we bring in x derivatives of q so it's the same argument there are however bunch of problems real interest where you also scale the x environment how do you get this equation let's go back for a minute I started with the idea that we obtain these problems by scaling a reaction diffusion equation so how do you get that you get this by scaling of course this problem but to get the next there you have to put the next there to get that you need to start with this problem which basically says that you are not far away from being homogeneous because as epsilon goes to zero you don't have it there what happens however if at that level I want to scale this problem now if I scale this problem this is going to bring me to that regime and now it's going to create a homogenization problem combined with front propagation but now I have to worry about two things I have to worry and let's assume that w is periodic in x so there are two mechanisms that are going to take place here one is again this front propagation an expansion in a direction orthogonal to the front and the other is the averaging that is going on from the x of ereptions so these two mechanisms will act together and combine to give you the answer the method I presented is going to collapse here because I cannot put anymore the chains q over distance epsilon plus there will be an x over epsilon here blah blah but there is an interaction with these two scales now so you cannot do it with an actual traveling wave so to solve this problem you have to use what is known as pulsating waves which I will explain next time but these pulsating waves will lead to expansions of the form and now there will be an additional gradient dependence because these pulsating waves are anisotropic so they depend on the direction you go and so if you try to make an expansion that will bring in a gradient of d which is the direction of the normal now forget any kind of previous argument if I try to write down an equation for this I get in trouble because I will have to compute the Laplacian of the gradient that will bring me a third derivative and that's a sin in this course the theory because there is nothing you can do with a third derivative if the distance is smooth of course that's not a problem it's multiplied by an epsilon so if I were in the smooth regime it's not a problem and on the other hand the whole argument, the whole theory is to go beyond the smooth regime and to do that there is an issue and so because of that I will introduce next time yet another way to describe the front where now instead of writing a PDE I'm going to think about the function of sets and they are instead of using smooth test functions to define viscosity solutions I'm going to use smooth surfaces evolving by the right velocity as test surfaces and once I do that everything will be reduced to dealing with smooth things and therefore I will not have a problem ok so I'll do that whatever tomorrow thank you and sorry for being late you can blame the people who keep the door alright thank you very much