Talk about the building of the open paths and in the booklet it says about VM force and we will tie those two threads together My name is Derek Colson. I work for VMware and the cloud services division We work for a public company and these are forward-looking statements So we get to look at an eye chart So everyone read that as quickly as you possibly can I'll nod like this say I understand what you're talking about then We'll get on to the fun stuff So what is the open paths? What is even a pass? it's a pretty exciting time in terms of a Combination of Ruby as a language and what it's done in cloud computing and Amazon and people like heroku and engine yard and things like that I've done some amazing stuff There's a lot of different opinions on what even a pass is platform as a service or an open pass What we're talking about here is more about an application platform as a service And what I mean by that is is that it's an application delivery platform That favors choice and openness and we're really going to beat on that theme today I think everyone at VMware and the team that I work with Feels really strongly about how we want to position ourselves going forward And I'm interested to hear what you guys have to say about some of the choices. We've made what we want to do though Very similar to what was said by Dave Thomas in the keynote is You have the option to choose DHH just said it just recently a little bit differently But it's the same thing you have the right to choose what you want to do And we're going to talk about that like I said a lot So with all that a what what is it? What is an open pass? The application is a unit of currency So instead of you getting into a position where you say I've got my app running. It's running. It's a rails 3 app It's a Sinatra app I'm ready to go Keep that thought process through the deployment cycle. Don't all sudden stop and go. Oh, I got a follow help desk Tisk help desk ticket I've got to go and make sure I get the right server and the right version of Linux It has right security patches and all I have to follow ticket to get the URL and we've got to get the F5 Going and oh, that's right. We need a database and we need to do this It's the application. That's what you've been working on. That's what you've been sweating over Application is what we're talking about everything else has decided for you everything and You shouldn't care You will and we're going to talk about what that means But you shouldn't care. So what makes it open? And again, this is Very very opinionated on what you're about to see We think multi-framework We don't see in our line of work a lot of enterprise houses or even Generic customers of VMware that are very very in tune with our virtualization products say oh, don't worry about it We're only up x shop We have a lot that favor a lot of different frameworks, but might concentrate on one We just don't see that and again, I think a theme of this whole conference is favorite choice and openness So you want to run a Sinatra app? I love Sinatra. I'm a big fan of Blake's Great. You want to run rails? Of course spring spring is obviously something that VMware Owns and has a lot of interest in especially in the Java Enterprise development community Of course, why not node? Sure scala. I don't care Python. Yeah, I agree with DHH. She's kind of ugly, but sure knock yourself out favor choice multi-framework from day one I Feel very very strongly that you can't bolt this on This has to be something that was written down on the original back of the napkin in the restaurant going Yeah, this would be really cool to build one of these things Here's another one and this one again is also going to be controversial even within VMware multi-cloud There's a past meetup that was sponsored Last night at six o'clock where we had people from engine yard and heroku and VMware was represented there And we heard a lot from the audience talking about when the infrastructure as a service platform bleeds in to the effectiveness of the past and A lot of times for example like with Amazon a lot of times all you can do is throw your hands up and say Yeah, that kind of sucked They had a big hiccup and well, yeah, we kind of look bad for it.  There is no such thing But if you had the ability to say, oh, I didn't like what happened yesterday with my application I'm gonna go over here. I'm gonna target my system over here and I'll deploy my app in about four seconds And I have a flat line in terms of Iops performance We think that's important We think it's not only important. It's something that you guys should want and demand Working for VMware. It says Amazon. Yeah, Amazon choice You should have choice The other choice that we feel very strongly about is the services and the services ecosystem And I think a lot of past providers have done an amazing job And I think we're just kind of following the lead there a little bit There's some things that we're potentially doing different, but you know, I like my sequel great knock yourself out Well, I like postgres. Okay, great knock yourself out It's not our decision in terms of providing a past service to decide what you should do We should help and we should guide But I don't think we should choose for you This is another quick eye chart, but when the slides actually come out you can actually click through on these links and See our CTO Steve here It actually did a pretty good right up on what VMware's position on the open paths and talk specifically about our agreements with sales Force and VM force, which I'll talk a little bit about and also Google Wait there. You're wrong. I want control. I Used to be a server hugger. I'm old. I get it You know, I need to have things like I Need to have access to the web server. You can't write The engine X config as good as I can or Ezra Ken, which everyone probably uses his stuff. I Need access to the app server absolutely. I have to control everything Relax This is a tweak that went by from another Colleague, I always call him dr. Chen Jerry Chen who's here. He says relax They got rid of the horizontal vertical settings years ago They don't exist on TVs anymore and no one really cares anymore, but seriously We understand we get it. We understand that we're living in a complex world We understand that even though we want to do an open path where the application is a unit of currency And that's what you're dealing with. That's not the reality We don't feel that that means that you have to compromise the purity of what you're trying to offer and convention over configuration So we get it we understand we have utopia VMs. We have layer for connectivity the interesting thing is is that apps will bind to them and Then they the utopia VMs will bind to provision services. It's kind of a neat way to say I'm stuck I can't do x for example. I want to run a solar instance to do searching over my database If you look at those last two or the Second to the last bullet points in terms of apps will bind to them Your app understands where it needs to go to get to the solar service that you're running on your own utility VM They the utility VMs will bind to provision services The solar instance understands how to get to the database Which is being totally controlled by your open paths installation to say when someone sticks something in here I'm going to pull it out index it and make sure it's all available So we get it Why do we care? Cloud is about OPEX not capex I was very fortunate enough with some other people that came over to VMware with me to work at Google and It was abundantly clear at Google nobody cares about capex anymore It's all about OPEX. What is your operating cost in terms of moving forward? Paz reduces that I think everyone in this room kind of agrees that at least that is the stated goal And I think there's offerings in the marketplace today that actually do that very very well a Major portion of dollar spend is on the deployment and management of applications So when we say we're switching from capex to OPEX you really want to say well, what dominates the OPEX? There's lots of different answers there, but what we're seeing is a trend towards that is going to dominate you simplify that process Dramatically and what I mean by dramatically is is? Yes, it feels great that you can get your app launched in two weeks when it used to take a month What if it's like three seconds? Storage and computer trending towards zero. I think we all know that and what that means again is it highlights that third bullet deployment and management of your applications will dominate your OPEX spend Paz is supposed to fix this problem. So Paz is trying to fix the deployment and management life cycle Why else do we care? Well speed and agility are really important these days especially when you pull an all-nighter and you've got that app running and you go in the next day and you're like Okay, we're ready to go and again you get hit by that fun little wall sometimes of saying well great Follow your ticket and we're gonna have this thing just ready to go in about two months.  What do you mean? I just spent eight hours last night and we're ready to go All right. Well, yeah, you're right We'll get it down to a couple weeks and we'll be out the door and it'll be great It's not only frustrating sometimes it actually hurts business it hurts you even as an individual software developer of saying I need to get going fast Again, what if it was seconds for any type of app and I've seen this before I just think it's slowly going to become what we all really envision is what if the cloud is really your other computer? And I think a lot of people represent in this room who actually Work in the past industry already do that for example I know a lot of the guys at Heroku that you know They use the cloud as as they're testing and developing and going back and forth And I see that a lot more in terms of even the people that I work closely with on a day-to-day basis But for that to happen a lot of different things have to fall into place Because if it's not as easy and as fast as local development, it's really gonna not not fly so in terms of What do we think is important? Reflecting back on the slide that said about favors openness and choice When we look at what's important is things like SLAs Multiple service providers and what I mean by that is is that I don't want to undergo the hiccup That happens every couple days when I get a noisy neighbor problem Might not be the past provider's fault Might not even be the infrastructure providers for potentially could be a hardware thing. I just want choice I want to be able to pick and choose what I want to do without having to bend over and do unnatural acts to make that choice I want everything I do to be exactly the same no matter where I'm actually trying to push my application For me. This is one of my my pet peeves a lot of my co-workers laugh and say the you know I don't get it yet. I say well One either I'm wrong which is happens a lot or two. It'll just take a little bit longer, but I I call it the light bulb and The light bulb. Yeah, the light bulb and what I mean by that is that I Remember about five or six months ago. I actually went into a Walmart and decided to radically up my capex bend on CFL versus incandescent why Looked at my electric bill. I said what the heck, you know, why am I spending all this money on these lights? Op-ex versus capex Every time you know about six seven years ago. I was worried about what things would cost I was always in computers talking about capex. How much is the server gonna cost? How much is this gonna cost? How much is this gonna cost? Google changed that a lot for me in terms of even internally Google was okay You're gonna have to you know pay this much for the computers up front Then next week it was like well pay us funny money and then but you got to pay us a monthly rate Then it was how many resources do you want? There's this kind of weird bartering system and then pay us every month And as we moved over to VMware Everybody that came into Google said what is capex? I don't even know what you're talking about All I know is is I can get a hundred computers and I just have to pay a monthly bill So framework X versus framework Y and it's like what do you mean? Again, it's the light bulb. How much does it cost to actually do what you're trying to do? How much does it cost you to light the room? What if you could light the room for half the cost? What if lighting the room at half the cost was actually easy if we get into an Op-ex dominated world within cloud computing All right where everything is kind of trending towards zero Eventually, you're gonna have a bill that says my app is generating this and I pay this Economics 101 says either you make the top line go up or you make the bottom line go down In cloud computing if all of a sudden someone came you and said hey, guess what? I can have you light this room for a quarter of the cost and It's not 30 bucks a month. Let's say instead of $5,000 a month. You can do it for $600 a month Very very large companies. Yeah, maybe I don't know tough decision Individuals. Hey, that might be a really good thing The point we're trying to make is is that if the system that you're working on is not favoring open in this in choice You kind of lose out on that So what is VM force? VM force is actually VMware and sales force. It's an open-pads instantiation. That's at salesforce.com Spring and roof frameworks it uses salesforce.com data and services So it's in their data centers, but it's essentially providing everything we're talking about So how does one build one of these things? Again last night at the the Paz media if we talked a lot about These things aren't trivial. They're not, you know, two chef scripts that you run in your MacBook now can do this type of thing The way we actually did it is we have this concept of inner and outer shells and they all ride on top of the infrastructure as a service So IAS essentially gives you CPU network memory disk all those fun things and they're all kind of looking like big blobs The outer shell is the one that for us is actually going to do the creation and orchestration of The IAS we have the concept of a stem cell VM So we can clone these things almost instantaneously and we shoot them a little message saying hey We want you to become one of these and they spin up very very quickly It's in charge of change management So when the Paz is running, it's going to be in our mind. It's a service. It's always in flux It's always changing out from underneath of you But that change can never affect an end user You shouldn't ever have a thing where I didn't touch my app and all of a sudden my app stopped why? So it's a critical critical piece to our architecture of how that's going to work the inner shell is the the piece that runs the core components and In our verbiage and and others you would see it too We have this concept of routers and cloud controllers and execution agents and health managers and services The basic premises of when again some of the first things you write on the back of the napkin That you really just can't bolt this on At least in my opinion The whole system should be self-healing you should be able to shoot anything in the foot hand leg arm Anything except a kill shot, which is you just got you know a nuclear bomb dropped on you The system should heal itself completely It should be horizontally scalable at all levels so those components that don't really mean anything yet until the next slide like routers And cloud controllers and things like that They all should be NY scalable. You should be able to run one to end and you pick in to meet your needs Should utilize distributive state No single point of failure. You can't have a gunshot to the foot take the whole system out This point brings in a lot of pieces favoring the service or a favoring choice on a service provider goes to this bullet point right here, so if we actually want to Make any type of statements about no single point of failure We can't do it running on something that we don't know that they can also guarantee that So the choice of saying I want to layer My software on top of a pass that's layered on top of an is that's layered on a piece of hardware that has 16-way peering four times, you know power redundancy X Y Z needs to be a choice Above all it should be simple and dumb. This is one of my big tenants Distributed systems and cloud-based systems are complex inherently don't try to make them more complex Dumber as you get closer to the core For us messaging is a foundation. We use messaging for addressing component discovery and command and control Jason is our preferred data format of exchange We don't have XML. I don't foresee us ever having XML HTTP or file systems for data transport so as data components are being moved around the system They're either a urn point or there might be an optimization where a file system or another technology can actually make this an easier transition So app basics For us execution is start and stop and that's it And what I mean by that is that when we talk about multi-framework and supporting multi-framework from day one The way we actually Accomplish that is is that by the time the execution agent gets one of these things it sees a start button and a stop button and that's it It doesn't know if it's rails if it's Sinatra if it's node if it's spring if it's rude if it's grails Doesn't know it doesn't care. It knows it has a start button and when it's told to stop it It has a stop button Be cloud ready now This one's tough There's lots of different opinions on what cloud ready means and I'll be frank We don't even think we know exactly what we mean when we say it But if you feel like you have to do unnatural acts and you're not doing stupid things Maybe that pass is not what you want. I Don't think you should be told that if I know how to write my app. I know my QPS will be 10 I can do this all on a single my SQL database. I should be told no no no You actually don't know what you're doing. You need something that will scale because you're gonna be the next Twitter When I come back and say no no no, this is an internal enterprise app Peak 10 QPS if we're lucky my SQL is fine. I have active record. I that's what I want to use But you can't do dumb things I Can't do file read on slash home slash Derek slash stuff slash whatever that runs on my my laptop That doesn't exist in the cloud. I can't do you know event machine start server on port 80 You don't own port 80. Don't worry about it. You know everyone will be able to find you on port 80, but you don't own port 80 So simple things, but they are they do exist I just don't think if you're doing unnatural acts that that's necessarily the concept of what we mean by my cloud ready Here's another one Send us everything and I mean everything I don't understand what your app is supposed to do or how I'm supposed to create a start button unless you do Bundle is good so to speak and what I mean by that is is that bundler is trying to solve many problems with version control And version skew and all this other stuff But it's also trying to Define a way to say for you to run this app.   What the system which you guys are about to see in a second does is You can send us everything And we'll actually do fine with that after about 15 minutes of downloading your 600 megs of whatever But more interestingly is is you can actually send in a fingerprint of everything you're about to send us and We can come back and tell you great have it have it need it need it have it need it have it need it And then you go, okay, great, then I'll just send you whatever I really need What's interesting about that pool is is that that pool is is the fingerprinting is not only just a shawl one But it's a combination of a shawl one size And so it's actually shared globally within the whole pass So if anyone has ever uploaded a rails 3 gem When you bundle everything together and you're like, oh, this is gonna suck. All right, let's push it in you Don't have to send that to us So we get the best of both worlds you actually tell us everything you need to run your app But we want to actually download very many things at all This has been a big issue with us and we've carefully watched bundler both Excel and trip and stumble a little bit in terms of trying to solve this problem But it's a problem that has to be solved. I actually get a kick out of following a Ryan and node and and I've been watching He is TJ right as one of the guys that does the MPM stuff MPM has this magic of it somewhere on your computer and we'll figure it out for you But you don't need to really know about it But when you actually go to take that app and deploy it somewhere else. Yeah, you do And so, you know, that's what one of the things bundler was trying to solve And we think it's a valid problem because we want you to send us absolutely everything It's kind of like the old days where you compile your your C binary against a shared lib Send it to your buddy and he doesn't have it and it just doesn't run Bigger problem newer problem. It's got the cloud world around it So it's actually cooler than it was back in the day, but it's the same problem. So the tools Ruby, you know, the majority of this system is actually built in Ruby We use a lot of event machine messaging as we talked about in terms of a foundation Http, you know simple basic stuff json rack Sinatra thin em Http request So a lot of people when we talk about this say ruby Really ruby you actually are doing systems all programming in ruby Ruby's a great language. It's very very easy to understand. It's well designed A lot of the team feels comfortable in it. They can move very very quickly But more importantly distributed systems architecture has little to do with language choice I'm not saying it doesn't come into play But it has very very little to do with it and if someone says so it's hard to read that but it says walk the other way They don't know what they're talking about but We all want faster ruby. I do I love actually going into 192 and We're following rabbinius and j ruby and all kinds of fun stuff So I want everyone who's involved in that for keep pushing because it's it's great for all of us But again with distributed systems If that's the first recourse in terms of oh, it's because ruby is too slow No So some of the patterns hopefully this rings true to a lot of every you know people in the audience in terms of How you would build a system that's supposed to do all of these kind of neat things It's event-driven. It's non-blocking asynchronous ny scalable Um make no assumptions and what I mean by this simple things like Well, you need to start a first then you can start b Then you can start c Systems should be able to come up in any order at once and she'd be able to self heal itself Automically without intervention. She'd do all those type of things So more on okay, how we built this Components are dynamically discoverable they subscribe to are you there? Anything that comes up Publishes a yep. Hello. I'm here So it doesn't matter which order these things come up in anything can happen Never store central state That becomes your single point of failure And for us health status and state is always available via HDP and so you'll see in a second with monitoring I can Walk into a system. I've never seen before in my life Run one command which will discover every single component running in the system Comes back to me. It says what I am what my version is where my hdp port is and here's what you can get information wise for me So more on the components the routers the routers are these uh dynamic pieces that essentially bind a request that's coming to Food vcloud labs dot com to your app wherever that is These have to run in real time. They have to be able to sustain 100 k per second Changes per second Remember they can come up in any order So you could have 100 000 apps in your cloud paths and you say well, we're running out of bandwidth I want to do it start a new router It's going to get 100 000 messages saying i've got app foo. I've got app bar. I've got app as I've got all this other stuff Cloud controllers are what you use to interface with the system and this is going to be hdp rest We'll talk a little bit more about what that looks like they define the state of the world They define the concept of users apps and services Cloud controllers are special apps so to speak within the paths open paths But they actually talk to the end user through the routers just like all the other apps So again simple and dumb we're not making exceptions for Our system apps so to speak Droplets is a funny little thing we came up with in terms of what we were calling the apps early on Uh And the droplets and apps essentially is this massive execution field of of virtual machines that Have the ability to run your app the cloud controllers actually will figure out where to put your app And then of course your app will actually when it starts itself up The agent that's actually watching that the the app is running correctly will bind it to the router Remember when we talked about being dumb Um The cloud controllers actually figure out who to use by being really dumb They have no assumptions about the state of all the execution agents whatsoever Whenever you ask it to run an app it actually says i need help It sends out a message The message has a small signature on it says this is the shawl one of the app This is kind of the memory that we need this is the other kind of tints or flavors or something like that And the execution agents come back and say okay, I can help you no problem And the cloud controller says great. Thank you very much Here's here's a fingerprint of what I want you to run and go for it What's interesting about being that dumb is that actually does very very interesting things that look smart So if those execution agents are actually spread across Local resources and remote resources So hybrid types scenarios where you have stump stuff that might be on-prem some stuff might be off-prem If you actually watch and visualize what happens the system fills up the local resources first And then it kind of bursts into the overflow capacity on the other machines If you have two overflow capacity Reserves so to speak and one's latency is 70 milliseconds and the other one is 15 milliseconds Again, the visualization fills up everything locally starts spilling over here and starts spilling over here Without us saying oh you have to be cognizant aware that these guys are over here And they might be slower because they're further away Just because we're being dumb we always ask for for help We assume the first person to get back to us is the best fit for being able to do that When we do certain things like hey, we want to make sure that the app's not running next to itself Right because that's a bad thing. You don't want your app to have five instances all running the same vm Again, the the dumb part worked out really well Which is when you get the shell one and it says hey, I want you to run this app here's a signature It knows if it's running it and if it is it delays by some Small amount the message that it sends back So if there's anyone who's better fit to run it they will if no one is available You will eventually respond and the cloud controller will pick you Dumb is good Concept of services apps these days don't exist in a vacuum So they have the concept of services and you know databases and key value stores and all kinds of fun stuff The cloud controllers know about them and can provision them and then they connect up to the application so that they can use them And then there's a concept of health not health of the system itself, but health of your apps Um What I talk about a lot is the cloud controllers are the king of the court. They say what is supposed to be The health manager is the jester that runs around and makes sure that that's it And if it's not he jumps up and down and yells at the king the cloud controller says They're not doing what you told them to do And the cloud controller says oh, well, I'll tell them again. You're supposed to be running four instances. That's not three But all those interactions happen in real time So is any of this real or is this just a really fun talk? Yeah, it is it's real This is a fun eye chart of showing the last diagram in more detail about what is real monitoring Two things about distributed systems that I believe Very very strongly one we've heard a lot about the dumber the better. The other one is monitor everything Figure out what to throw away later We talked a little bit about this so I'll go pretty quick has to be simple Publish who's there everyone responds with a I'm here. This is what I am and this is my end point You pull some results Then you ask again I understand the push versus pull I understand it very very well and push models are obviously applicable in very very large scale system So this could be a push some of those data to me The point being is is that a monitoring system just says other parts of the system shouldn't be able to make any assumptions I shouldn't have to write a 45 line configuration file to deploy a monitoring system in one cloud versus another cloud And that's currently what our monitoring system kind of looks like And one of the pages So what are the interfaces to the system currently we have two we have a command line interface Which is a ruby script which we're about to get friendly with here in a second The other one is we have an sts Which is the spring version of eclipse plug-in that actually does a lot of these things for spring and grails developers Who are near and dear to be mware's heart because of the the relationship with the enterprises that we have This is another eye chart, but this is essentially some of the things you can do with the system you can Push applications you can start stop you can Create and bind services to your app you can scale them up you can scale them down You can map and unmap different URLs so vcloud labs.com is one instantiation of this that Is up it's real we're going to see it here in a second then some of the The people from vmware are here and we're going to start Taking invites talking to people and getting people on the system to actually test it out. So if What you're looking at is interesting and you like it Either come find myself or Vadim or Ezra for more of a demo and if you want to get on the system see Jerry or deco or cillian or carmine and they're all up here in the front Currently vcloud labs is on vcloud express with a partner of ours called terra mark So all the boring stuff is kind of out of the way Do a demo So this is the client That wasn't good That's not good either Bundle. Oh, I know what I need to do. I'm sorry Now we're in business. Sorry about that Rvm is your friend, but sometimes they can bite you So i'm logging to vcloud labs and I can say info and it actually sends you tells me This is how you're logged in. This is your memory usage number of service and apps you're allowed to use I wrote down my little cheat cheat sheet in terms of what I wanted to talk about This is real so vcloud labs.com With our lovely network. It's about 99 milliseconds away. So we'll have to see how this goes So this is a Real simple synod rap. It's kind of hard to see But essentially it says Hello, and it'll tell you where it's running. We can make it a little bit smaller so we can actually see what it's doing So it said, oh, okay. I understand your your synodra application. Is this what you want me to do? I say yes It uploads it It's trying to start it And it's running so if I go now to Food vcloud labs.com If I spell it right So we got hello from the cloud via blah blah blah Hello from ruby conf 2010 lots of exclamation points update foo Hello from ruby conf instances scale up. We talked last night about it's really hard to understand when you want to scale up But once you know you want to how hard is it? So now all of a sudden we've got two of these guys running. That's kind of interesting The network won't let me reload it fast enough So as they're spinning themselves up, we've got more and more that are starting versus running versus all the way up So to start up 100 instances of this. I timed it the other night takes about 10 11 seconds So they're all running now I go all the way back to one network stinks So back to one so in terms of Again talking about you know when you you've decided on your own you want to scale it up Does the system help you do that? How fast is it? How fast can you scale up and down? We talked last night about auto scalability is really really good until you get the bill Because your friend played a trick on you and ran a patchy bench all night long Um But once you've made the decision, you know, how hard is it to do that? Um, so for example, here's a redis application Push, I'll call it r and these are all pretty small apps. So we'll we'll see one that's a bit bigger here in just a second So if I want to test it says nope can't do that That's because it doesn't have a service So what services are available in the system? Currently we have rabbit mysql database and a redis redis is the one I want Create a service of redis and I'm going to bind it to my our app Now suddenly we have redis bound into the application Now if I want to say let's say redis sample Um, I want to go down here and say require H max show one Remember I talked about send us everything We really do fundamentally believe that's what you need to do But if you notice this app has nothing except itself We do have system gems and things like that that that form a default pattern on the system so that If you want to get started you want to do a little world and you want to kind of push things around and I think we have 30 to 40 of the more common system gems including rails 3 rails 2 3 10 All that kind of stuff so you can actually you don't stub your toe on day one But I just added something that I'm pretty sure the system doesn't have and so when I say update art It's actually going to go ahead and do everything that I'm telling it to do it'll send it in It tries to start it and what immediately happens is the system prints out It's harder to see Um That there's a startup log error that says Can't find h max yaw one So if we put in a gem file that says oh my app needs sinatra redis and ruby h mac require h max yaw one When we do this If I go and I say okay Now update my app It's going to pick up that gem file I copied the gem file dot lock so we couldn't all just sit here watch a ruby gems and bundler go along It's going to go ahead and says oh there's a pending thing and it's trying to actually pull in ruby h mac 0.040 And this time it said he couldn't actually find it. It's trying to work on it and things like that That experience is it works it kind of it bubbles along but Bundle package remember in the presentation I said that's that's your friend Put everything in there send us absolutely everything Because if we've ever seen it anytime ever before you actually won't physically upload it Oh, that's not good Sorry, I'll try to make this smaller so we can actually read it. I'll try it again We'll see if it actually works. Nope. I must have typed something wrong In terms of the uh The required stuff. All right, we will move on since it's a live demo In terms of The point was trying to be made Which I failed at but the point was trying to be made is is that we give you a default set of system gems Those will kind of be in flux, but we want to keep them small Sending us everything is the answer so bundle package for rails and snotter and things like that is you know is the answer Um, so what about other stuff? Um Pet clinic is a downloadable jar file that runs grails, which I would imagine not a lot of people here actually run Um But that war file actually comes directly from the internet and uses a database does some cool stuff in terms of putting together a web page Um Remember the unnatural acts. I should be able to just say go ahead and push this app and I'll call it pc for pet clinic Again, the further away you are from the court. It tries to be smarter. So it says, oh, I detected that you were a a grails app Sorry, you can't see that there we go Do you want me to go ahead and do this? I'll go ahead and provision a database on the fly. We will auto bind that into the app So for example with the red is that example I was doing things on my own trying to pull the environment variables that described The service, but if we know what you want to do, we can actually do every all the hard lifting for you If you notice it says uploaded application pet clinic 8k Pet clinic world file is actually about 22 meg big And it says it has started and so now we have A trivial app but getting more complex that was simply downloaded from The spring grails site that now is running on vcloud labs alongside of a sonata app What about rails 3? This is a rails new Put in a little hello from the cloud type stuff and a bundle package Is the steps that we did I'll call it r3 So again, you can see it says rails application is detected But once it actually gets to the execution point within the system running on whichever VM that's going to it just sees a start and a stop button So now this is uploading again. And so the bundle package is All that data is there. You're trying to quote unquote send everything into the cloud But because of the finger fingerprinting stuff, you actually send very very little. So we sent 5k of that app Same thing, but now it's all rails 3 actually vying to a database Going back to the pet clinic, which I know everyone laughs at but One of the interesting things I wanted to show you one is if I do target Remember we talked about VM force just a little bit Essentially, I just said take my Working environment what I'm familiar with my tool set Simply take it from pointing here and point it over here vCloud labs is actually running inside of terramarx facility in miami Sales force is obviously on the west coast. So now some I've just pointed my utility at totally a totally different thing and said, okay Do the exact same thing I just did with this cloud with this cloud So I say, yep, sure that sounds great. URL is obviously going to be different Yes, you're no problem And if the network cooperates with us So it's going to do the exact same thing it does with every single instantiation of the system It's going to unpack everything that you have It's going to fingerprint everything and send the manifest in whoever the cloud is whatever the global resources are It comes back and says great have it needed have it needed We go ahead and send it and now it says now it started on VM force So if now if I do PC Now it's running on a totally different cloud. My experience was exactly the same You have choice We favor openness And the last one I'll do is uh, I'll switch back the target to vCloud labs lack of sleep This is a note app Who here ever has worked with note or deployed a note at? It's kind of cool. Non-blocking sometimes is frustrating, but it's neat in terms of how well it scales um This is the app, which is again kind of hard to see but this one line right here of code Is what we did to get it to run on vCloud labs This is very similar to what hiroku's done In other words, just make the app so it doesn't assume what port is going to run. We'll pass you the port Don't worry about it What's interesting though is is that note is one of the newer frameworks on the scene and already the ability to take this and deploy it into a system that is Totally different than your laptop is one line of code And so now if I actually do a vmc push I'll call it chat So it's actually running and we can do Chat v cloud There's ryan's a chat app Again, we're going to be taking sign up and interest. Please find ezra. I know there's talking a lot of stuff in it and I think we lost a lot of people early on Ping one of us if you want to learn more please talk to some of us if you want to have Access to the system and get signed up for it And I'd be happy to take any questions Well, there's two pieces of that one is a technical question and one is a business Question the business question. I'm going to kind of scoot and direct you over there to the people that know that better than I do The technical question is is though that remember we talked about those inner and outer shells The outer shell essentially takes a pool of resources Now what we currently don't do is we don't Taint the resources based on what we want to put where We're trying to be very, you know, we just say here's all the resources and we're just going to play stuff as we we need to in terms of You know, we need More capacity So for example, the inner shell has a lot of slack capacity on its own So when you deploy an app it feels like it's instant even if you scale to a hundred instances That took like 11 seconds or something like that That's because the inner shell actually has the last tissy built into it where it has a capacity that it's actually running on Underneath of it the the bosh, which is what we call our outer shell It understands the impedance mismatch between what the inner shell thinks its capacity is and as it gets closer to that high watermark and what it actually has and so then it actually says Oh, you're getting close to your capacity. I'm actually going to start spinning up my own capacity and that's in the The way that happens is spinning up VMs cloning VMs and spinning them up And remember we talked about them being stem cells And then if it's you need more app capacity, we shoot a message saying this is what you're going to be And so the system is designed from the ground up to at least achieve this concept of as a user You feel like you have endless capacity. It just keeps coming and it's free and it's it's easy I mean to kick the tires a little world app of the cloud is Push it. Wow, that looks great spin it up to a hundred instances And it's hard to do that on a lot of different systems So we try to do that. Now, of course, there's also the failout of bosh says He's full. I'm full. Somebody's got to order more hardware, right? And so that's the that's the end of the game so to speak Sure Yeah The big thing there is essentially is is that the rules that are being defined for the system itself Might not necessarily apply to your app So the services that we're showing up there were for app consumption So the apps can store that and what happens at that point is with an ecosystem of service providers Is that you can provide the mysql instance that has disaster recovery or high availability Or hourly snapshots or things like that So, you know, it's it's this balance of Not making your app do unnatural things and having the system do the right things for you But when I said no central state, I was actually talking about the the past system itself in terms of anymore Yeah, and it's actually Put into the system now. There's two different ways you can do that One is you can actually do an update and an update will actually start a canary And it'll make sure that the canary is okay, and it'll actually sweep, you know, swap things The other one that we talked a lot about last night in the meetup is If I make a change For the demo, it's fine for me to just say update my app. I think I know what I'm doing Obviously, I don't all the time, but I you know, I think I do and production You wouldn't do that, but it'd be nice if you just said push this app and put it under New app dot internal Dot test dot, you know, me cloud labs dot com, which is vpnable And only accessible inside the company and you look at it and you play with and you say, okay, it's great And what I didn't demo was is that you can then just say map the url Swip them, you know, and you will be guaranteed to never drop a request So you can say, okay, it looks great and you flip it The other thing too is is you can obviously run lots of instances And so the router knows it has lots of choices to actually deliver your request We were only running one instance in that that demo when I was bouncing it up and down Yeah That's a good question. The question was is what are you thinking about around ci development and uh or ci systems in terms of If this looks like a black box The short answer is is that we've actually thought a lot about it We've actually manifested something called code the cloud which we're still working through Which has the concepts of test frameworks and staging frameworks and continuous integration and bug reporting All kind of in conjunction with a system like this Um, and so we're we're working pretty hard on what that feels like and what that looks like The short answer is is that even though app cloud is a black box, uh, I mean, uh, the system is a black box Um, you know, we do have the concept of these utility vm So we understand that there's going to be needs that aren't met by uh, oh, you just worry about your application and its associative services Any others? Thank you


Yes, I am Ron Evans, Dead Program. That's my more important identity on Twitter. And I am one of the members of the hybrid group here in Los Angeles. We're web consulting and development. Let's talk about LA for a quick second now. Los Angeles does not get the same attention that other areas of the country located in Northern California might, as far as technology and entrepreneurship, but we've got an amazing community here. We've seen some of our local members here today. Evan and Michael, we've got some other really remarkable people coming up this afternoon. We have a tremendous community here in Los Angeles. I would definitely want to recommend that if you're around on Tuesdays, reliably, we have the Los Angeles Ruby Parade, which is put together by Evan Phoenix and by Shane. We, every week, Thursday, or excuse me, Tuesday for seven to 10 p.m. Just roll out. There is no agenda. It's someone of an anarchist society. We just hang out, talk code. We did some work on KidsRuby there. So definitely come on out. And also, I do want to take a brief moment to mention KidsRuby, which is an actual Ruby intended for kids. Go check it out through website kidsruby.com. It's on GitHub. We look forward to your patches. All right. But I'm not going to talk about KidsRuby yet. I'm going to talk about music. Music is a world within itself with the language that we all understand. At least that's what Stevie Wonder said. And that lyric has always really stayed with me because it seemed to describe programming at the same time. Let's take a look here at the visual representation of music. We have the either absence or presence of sound, which are known as either rests, in the case of the absence of sound or notes if the sound is present. We can combine those together using specific timing doing it. We combine that together into either scales if we played them one after another or into chords if we combine them. We can take those patterns and combine those in the higher order patterns to create measures. We take those and we can create an entire song representing in the visual pattern language that is entirely different than the actual day that itself, which is auditory. So as an example, we can take even one of the most complex pieces of music ever written and we can represent it entirely in a visual format. It's amazing. So that's all we need to know to play music. That's it, we're done. Cool, right? Wait a second. Small problem, right? We need to practice to play music the same way that we need to write code in order to be programmers. You can't be a coder if you don't actually do coding just by reading books on programming, although obviously very good ones. And if you want to code in a group of people, just like a band, you actually need to practice together. You can't do it all on your own. So let's talk about the structure of playing a song and how that might be able to be compared to the process of developing software. First of all, here we have a very high order specification known as a set list and that is the songs that the band is going to play together. One of the rules about jamming is that at least two members of the group need to know a particular song in order to play it, otherwise we can't get started. Then you need to decide on which key of music. Music is a relative relationship between the frequencies of sound and so you could have the exact same song playing two different keys sounds entirely wrong. So you need to synchronize the key signature against signature of vocabulary we use frequently in the technology world. So then we're ready to start the song. We know which song we're gonna play and the drummer comes off the tempo. Keep in mind that the drummer is responsible for how fast or slow the song is and it indicates to the rest of the band that the change is about to come but it's not actually responsible for the rhythm or the groove. That is actually the bass player. The bass player is who establishes the foundation for the song and helps provide the basis that all the rest of it coming on the singer. Many songs have the exact same chord changes. The melody or the words are the thing that distinguish one song from another and also this is what allows the user to connect directly with the interface of the information being transmitted. I mean that person to tell what song is being sung. So then once we've got that foundation established then improvisatory sections can be added onto this differently known as solos. This might be new features that have grown in that add some particular value that wasn't originally planned to the song but ended up making it a lot better, really nice. Now it's time to end the song. We've got to make contact and one thing about music is it's not just auditory. It's the entire spectrum of the sensory experience making eye contact is critical when you're communicating with other people and that is definitely the case when playing music and it's time to end the song. Sometimes it's not enough. You're jamming away and no one's paying any attention. You need to get extreme like, hey, the song is about to end right here, right now. We need to really get people's attention. The trick here is to realize that there's always a center. There's always a focus within any band and it's not necessarily the lead singer. It could be the bass player. The important thing is that there's a foundation for the creation of this particular information set that everybody is looking to this one person to establish this is the locusts, this is the center of control. So let's talk about different styles. This section is actually inspired by John Dahl talk that I heard originally in future Ruby and Canada last year. Excellent conference anyone who made it to that. So let's talk about the evolution of music styles and how that might be able to be compared to the evolution of technology. When we first see music being actually documented in some type of form, it's a highly specialized art form which is restricted to basically a very small segment of the population of our privileged class. Here's an example of some IBM 370 assembler code that's circa early 60s. And here we see this is Hello World in IBM 370 assembler. Somewhere in there is Hello World. You can find it if you look, yeah, somewhere in there. So it takes a lot of specialized knowledge to even do the simplest activities. And this is very similar to the professional priesthood class who basically ran data centers in the early days of vacuum tubes. So music continued to evolve. Now it becomes much more accessible to common people but at the same time requires very sophisticated regimentation. The music itself is very, very strict and still is very structured. Here's an example of Hello World. And as somewhere in there, yes. I bet how many people have ever seen cobalt in this room, more than I thought. So somewhere in there, yes. You see that the structure is getting a little bit simpler. We're seeing a little bit less ornamentation but there's still considerably more form than function. And this is very typical of the kind of hardware that is being used at the time. So then music continues to evolve. The jazzier, the smaller combos it becomes more about improvisatory. Still there is a tremendous high bar of required ability in order to do anything at all. This is very similar to out all that we see in the mini computer revolution. We see a lot more people starting to have access to information technology. But, and we see here a lot less ornamentation now. You can actually recognize the Hello World in the code. The big revolution hits. Who here knows who this band is? That's Sam, that's Sam. This is the Grateful Dead. And the Grateful Dead is probably one of the most significant organizations in the history of information technology. I'm not kidding, I'll tell you why. This guy here, Jerry Garcia, just sort of hanging out in the back there looking sort of casual. He actually was a marketing genius. He basically invented the freemium business model. They would allow anyone to come into their concerts and to record, sell, distribute, do anything they wanted with the music. While the bands of the era that were maybe a little better known like the Rolling Stones were constantly in hoc to their record labels. The Grateful Dead was cashflow positive from day one. Also they created an entire economic ecosystem around their touring. There were people whose livelihoods were literally going around with the band buying and selling merchandise and contributing to the lifestyle of other fans. So what is the personal computer revolution hits and the Hindi language? Listen. But look how interesting this is. We see that for the first time the function actually outweighs the form. You can recognize immediately that this is Hello World. And now we're in the modern re-combined era. The mashup era. Anything goes. Perhaps we are doomed to continue to reinvent the past but we're doing it in new and interesting ways. And of course, the language of the present and the future. This is the only Ruby code in today's presentation. So let's talk about this rotation. Most software development projects have more than one person. There are a few very successful ones like desktop tower defense where one guy in a spare time is able to create something of significance. Michael Harville, obviously being a tremendous example of this. It's a lot more common to find duos. The two sieves. Larry and Sergey. There's new mobile examples of highly successful companies and open source projects that are the result of two people working together. A little less common is the trio. But the power trio has a lot of capacity. Jason Free wrote a blog post I can never seem to find the link to. But he was talking about, in his opinion, three was the ideal number to start, particularly startup. You've got a business guy, a tech guy, and a designer. I can get a real. I'm sorry, I'm getting this getting very odd. I wonder if I can find the blog post. Thank you. So then we have combos, which are a little bit closer to consultancies like maybe the hybrid group is. You have different specialized individuals who collaborate together. Then we have large development organizations. And most of us, when we think about significant development projects, we think about something that's very structured, very regimented, very ordered. But that really is not always the case, right? A marching band, like the extra action marching band here, is a great example of sort of a controlled chaos. And what is a very successful software project that functions in a similar way? How about Linux, right? So it's an idea that individuals can self-organize themselves, that they can figure out what they need to do and they can combine this together. So let's talk about a couple of anti-patterns. You're probably wondering how do musicians know so many songs? The answer is they don't. They use something called a Facebook, which is very much like a pattern directory, which describes in sometimes a very synopsized form, here are the ways to play Louis Louis. Maybe it's just blues and E. And so unfortunately though, being able to read the book of the patterns does not mean we know how to apply them. You still need to actually know the song to play. The idea here is that it's just reminding us. Also, don't try to do everything yourself. You need to know when you need to ask for help.  And very importantly, no one to stop. If it's not working, just stop and walk away. Some combinations just don't really work out well together. And some combinations are just entirely unique and I couldn't compatible with anything else. I mean though is when you collaborate with others, just let them have some space. It's not all just about you. Because when the ego goes to the ultimate extreme, you find suddenly you're all alone on stage and your project is dead in the water replaced by a much cooler project where the people wanna talk and collaborate. So let's say we've talked the theory section. Let's get into some actual practice. This is an amazing thing that I'm gonna ask you to do. Put away your laptops.  You've got a couple of FTCs. We need to sort of be closer together physically for this. This is a kind of experiment. So bear with me. First time I saw this was actually in Canada done by Misha Gluberman who is a really remarkable guy. So this is what we're going to do as I look for my props. We're going to learn a symbolic musical language together right now. Okay, so this is how we're gonna do it. We're all going to start out with the first musical symbol which is going to be a vocal sound, like an ah or a do, the o, anything that's a vocal type sound. And we're all gonna sing this together on the count of four. So if we one, two, three and then we'll all sing and it can be any kind of vocal sound but the trick is it has to be a vocal. Okay, like a vowel sound. So, in English it's a vowel. All right, so let's do this on the count of four. Are you ready? One, two, three, ah. Beautiful, all right. That's our first symbol in our musical vocabulary. Now let's do our second symbol in our musical vocabulary. It's going to be a clapping sound. The idea here is that it should be like a light clapping, like maybe a golf clap. But you can vary the way it is. Like it could be like this. You could change the sound. The idea is don't be too loud and don't hit anybody right next to you. So it's a little clapping sound as our second musical symbol. Are we ready? We're gonna do this on four. One, two, three and. I'm not there anymore, I'm just gonna stop. Beautiful, all right. That's our second musical symbol. Now our third musical symbol, the third and the last of our basic vocabulary is going to be a cibulant sound. Which if you don't know what that is, it's like a hissing, like a shushing. Like a shh. Or it could be a ss.  You know, whatever you feel like doing. All right, we're gonna do that on four. You ready? One, two, three and. Shhh. And stop. Beautiful. All right, that is our basic musical vocabulary. So now let's use those basic symbols and we're going to use these props. The, let's say that the American flag is gonna be the vocal, the ah, the ho. Let's start that out right here. Thank you. Oh, oh, by the way, everyone stand up please. All right, so now the cibulant will be the pirate flag. And we'll start that right here. You're the pirate. You look kind of pirate to me. And then the last one is the clapping. We're gonna do, clapping is California. So we'll start the ah, ah, not you. You sir.  All right, so now we'll get back up to the front. Now we're, this is how it's going to work, all right. For now, ignore all the flags except for the vocal, okay. You guys hear the other flags, just hang on. Hold up your flag as high as you can. This is going to be the center for the vocal sound. He is going to start the sound and he's gonna pass the flag around. As the flag goes, whoever's right around it and the immediate vicinity make that sound. If you're not in the immediate vicinity, do not make any sound, okay. So move a little closer to your comrades. Thank you. Hold up the flag as high as you can. You're gonna pass it directly back to you to out as soon as you start to sing. And all of the people in the direct vicinity. I wanted people away. Yeah, only one, only directly around. Okay, we're ready to start? Let's do this. One, two, three, and, the person who I gave the pirate flag to, right, you're gonna be doing the, exactly the hissing sound. So only the people right around the vicinity, it's the exact same thing we just did with the vowel sound, but we're gonna do it with a similar sound. Again, only direct vicinity around the flag. I'm gonna start on four, you ready? One, two, exact same thing, except with the last flag, the California flag, and we're gonna do it with the clapping, all right. We know the game, let's do this. One, all right, so now let's go one level deeper. Now what we're gonna do is we're going to play flag game with all three going at once, right. So you're clapping, where's the vowel? Which is the US, US over there, vowel sound. And where is the simulant? All right, so if you're directly around that flag, make the shushing sound. If you're directly around this flag, make the clapping sound. If you're directly around that one, make the vowel sound. All right, and you guys hold those flags up really high so people can see them as you pass them. All right, you guys ready? Can we do this? One, two, three, exact same as the flag, and, except, no flags. All right, so use the number you are. I'll take that. Remember where you are. So now you're supposed to hand out.  Where's the other one? Back there, okay, very good. All right, so the advanced flag, exactly the same as the flag game, but no flags. Now, I used to do this. It's gonna make it a lot easier, believe me. All right, so it's the exact same game, but with no flags with your eyes closed. You guys ready? Let's do this. One, two, vocabulary is where you're going to simulate a con-based game of life. Using each of you as cellular. The rules are simply this. If you are next to two people who are making the exact same sound, let's start making that sound. If everyone around you is making the same sound as you are making now, stop making sound. That's it. You may be wondering, what happens if they collide? I don't know, let's find out. We got to run the simulation a couple times, okay? So, okay, let's, we're gonna have to see the data set a little bit.  The shh, all right. So, remember, if, actually, this is gonna be a bad idea. You too, are we gonna do the similar one? We need to make sure we see it with an updated, or else the simulation will die. So, once again, let's close our eyes. We can't trust our senses. We have to concentrate on other parts of the experience. Think about what's happening, which sound are you gonna be making, and let's all begin. Are we ready? One, well, one of the things that happens when you sing, society's, the spiritual or religious practices are associated with singing is because the experience of singing collectively in a group actually does something to the brain causing a peptide called oxytocin to be produced. This particular chemical is known as the love chemical because when we start feeling that sense of belonging collectively in a group, that is the chemical that's actually being produced. And one of the really interesting things about this is that it's also associated with something that's called flow consciousness. Flow was first discovered by the university of Chicago psychologist, Mihe Chiksemiyagi, if I pronounce that right, poor guy. Anyway, a state of flow is that state of consciousness when every problem you encounter seem to have an easy solution, where hours go by like minutes, you're not even aware of the passage of time, and you are having a wonderful experience, you're having a great time doing your work. You ever heard that, I love my work? Well, this is the reason why. What we just did together was we practiced a state of collective flow consciousness, which is the same state of consciousness where we can optimally produce any type of information together as a group, like software development, for example. Because really what we were doing here today is what Dr. John Lilly would call programming and bioprogramming of the human biocomputer. So this type of metaprogramming is something that we want to take with us, and we want to try to experience on a daily basis. So how do we improve our lives? Study theory of the work that we do, understand the reasons why, experience that work together collectively, like reading code on open source projects together, not just alone, but by yourself, and just like a good musician will record their practicing. So you need to document the code that you write, even if you're not sharing with others, you need to not throw away so that you can go back and learn something from it later. And do the same thing collectively in a group. It's not about some impressive performance. It's about having the experience of reaching this shared psychology together, this shared state of consciousness. You can come together in a group with this type of shared flow. You can take a very simple pattern and the merchant phenomenon can result. Some of the greatest music ever made was improvised music. So all I can tell you is please go out and do it. Thank you.


Welcome everybody to my talk today about developing your own Swift middleware My name is Christian später. I'm a software developer working at in advance and I'm mostly working on Swift and testing and automation Personally, I started using Swift at the end of 2012 and at that time point I was evaluating different object storage solutions for my former company and And one of these solutions was open-stack Swift and I was very happy with open-stack Swift because of the flexibility of Swift itself But unfortunately one key feature was missing that we needed and that was quota support at the end of 2012 so I started talking to the community and different people from them and Finally, I created my own Swift middleware to add account quota support for Swift and finally in March 2013 this account quota patch got merged within Swift and Was released with a grizzly release? so I want to talk to you about how to use middlewares to extend the swift functionality and because Swift is using the web standard gateway interface and Several other packages like paste or deploy. I also want to introduce you to these packages We will continue with developing our own Swift middleware for the Swift proxy server and Of course have a look into testing and packaging your middleware and Finally, I want to conclude my talk with some references for you So why do we want to write some Smith middle whereas? Actually, if you look into the Swift proxy server configuration file You will see and notice that most of the features are already implemented as a middleware For example, we have stuff like logging temporary oils form toast dynamic large objects Authentication middlewares and quota middlewares and these middlewares give you a great flexibility to extend your currently existing functionality and another great benefit of Using middlewares is if you want to add a functionality You don't need to fork the Swift code and to modify the existing Swift code You can just use the existing Swift code your upstream release for example create your own develop your own middleware and use both at the same time and Several people already have done this. So there are existing third-party middlewares outside of Swift for example for authentication and adding additional API layers and There are also some companies that created their own middlewares for their own private use So they're not released to the public but to extend functionality is these companies needed So let's first have a look in general How is the web standard gateway interface working and what's it's all about middlewares? so web standard gateway interface is a simple interface between web servers and web applications and it was defined as a pysen enhancement proposal triple three and Also updated for pysen 3.3 lately And this web standard gateway interface defines and defines an application and a server The application simply has a callable object that has a call method and the server invokes these callable for each client request So the client sends a request to the server and the server simply calls your application These application objects must accept two positional arguments and the first one is a pysen dictionary called Environment and the second one is a callable called start response The start response callable itself takes two arguments one a status for example a 200 ok or 400 found and the second one is a list of tuples And these are the heaters that are sent back to the client for example content type text plane or whatever you have The environment dictionary has some useful informations that are required to write some useful middlewares The first one is a request method for example. You have a put or get post delete copy and options Second one is the actual pass that is sent to the whiskey server For example in the case of swift this contains your account name your container name and your object name We have also something like a query string So for example, you can send a question mark and then a parameter name and the value to this web standard gateway interface And this is also included in your dictionary If you send some heaters to the server these heaters are also included in the environment And they are prefixed with an HTTP underscore and finally you have a file like object called whiskey dot input and This file like object you can read from it and it contains the data that was sent to your server So let's have a look at a simple sample server Everything you need for this to run this is included in Python itself so you don't need to install any additional packages or modules or whatever and What we're doing here is simply starting a server on the local host port 8000 and the lines on the bottom of the page of the slide and this server calls the application defined above and the application itself Takes the environment. I already told you and the start response as arguments and what it does here is that it returns the body and Inside this body we return an element from our dictionary in that case It's a pass info. It's just an example, but it works really nice and Gives you an idea how this everything works So but it's not very useful if we want to extend it a little bit We might either need to modify that code or we can add a middleware to this So what is the middleware a middleware can act both as in server and as an application? So the server is calling the middleware so from the middleware perspective It's just calling it's just getting the request from the server, but the middleware itself is also calling an application So it acts as in server for this application And by doing so you can run multiple applications side by side You're not limited to one middle bear or two or five you can well I don't know but you can run a lot of middleware side by side and This is very useful because you can add stuff like Authentication or you can do reroute requests or do some content processing and if you look at The bottom line we have a client that sends still the request to the server and now the server called the middleware and the application Inside the Swift proxy server configuration file You will finally see a line called pipeline and this is something like that imagine It's a pipeline one middleware called the next middleware and finally we call an application that application In case of Swift as a proxy server So let's have a look at a small example On the bottom part of the screen you see just I just started a pysons session and called my application And on the upper part you see I have a career request prepared I want to set to my local server listening here and I added some parameters also for example. We have a pass in for ABC We have a query string called hello and we send some custom header. So what now happens if I sense this I Modified the application a little bit it will return our data So this is some part of the dictionary of the environment dictionary that the server is responding to the client So if you want to extend our application, we need a middleware. So how does the middleware look like? The middleware itself is a class and this class has some callable and Within this callable we have our actually codes that is executed And if we start our server it no longer takes the application as an argument But the middleware and the middleware calls our application and in case of Swift this would be the proxy server itself So what's happening here inside our call method? We first start with the response. We prepare the response here and then we try to find or to identify the path info and if the path info equals to echo Something will now happen and in this case We simply read the data that was sent from the client to the middleware and to the server and responded So we have something like an echo So coming back to the example from before if I now send Sorry Different response You see That's a server is now returning a different response I send the data hello world to the server and the response also with a hello world because of the past and this was Intercepted by the middleware and no longer the application itself so One drawback here is that if you are adding more and more middlewares You have a really long line finally and you make server Call because each middleware needs to call the next middleware and we have stuff like the server name inside it We have the port name as a they have a port number inside it So you have a lot of configuration stuff in your Python code and you don't want this you want to have a separate configuration file We also want to have some testing of course for a middleware hopefully and We need a well better way to deploy the middleware and the server. So let's have a look into this Actually testing middleware is quite easy if you think about it for a moment the middleware takes an application and If I send a request to the middleware, I expect something from the middleware. So what do we need to do? we are using a Python unit test for this and If you look at the lines below We prepare our middleware and Instead of using a real application as an argument We just use a fake application and you see this a lot for example inside the Swift tests also So in this fake fake application just returns with a for example 200 Okay, we don't test that one here, but we want to simply test that the middleware is working so we prepare a request and in this case a put request so we need an environment dictionary for this We need of course some bodies that we want to send to the middleware and this is also defined there hello world and We want to send it to slack slash echo as a pass so in the second Last line there below we try to get the response back from our middleware and finally We compare this to the data that we have sent to the middleware before Now if you want to deploy your middleware and especially in case of Swift You don't want to ship Well single files and you don't want to have the configuration codes inside your files So there's a very nice package called paste or deploy and paste or deploy loads whiskey applications and servers from a uniform resource identifier that might be for example a pysen egg and paste or deploy uses an any-style configuration file and It's makes it very nice because now you can you separate your code from your configuration There's also a script included paste script and paste script can serve your application directly from your configuration file So you don't need to write any Init files if you don't want to and simply can use a configuration file for this and Paste or deploy is also widely used in OpenStack itself not only in Swift But also I think a nova keystone glance and I would assume a lot many more projects so an example configuration file looks like this we define our application itself first two lines we define our middleware in this case it's only a single middleware and What we now do is we define also pipeline and this pipeline has every middleware that we want to use in it And finally our application And last we define the server part and in this case it should listen on port 8,000 What I also did here is if you look at the filter middleware part. I have something like suffix equals echo response so I Can add some configuration options to these configuration file that are used inside the middleware so I need to modify my little bit middleware a little bit, but not that much and Can use any configuration option that I add here inside my middleware now How does Python know that? We have an egg for example like sample and that there's a middleware included we need a setup script for this and The most important part in this case The two entry points here We have two entry points one for the Application itself and the other one for the middleware and if you look into swift We have also a lot of entry points there for each of the middleware for example And if you want to write your own middleware you need to have something similar You don't need an app factory for swift if you want to write a swift middleware Middleware, but if you want to write a middleware outside of swift you need something like that So and what pace deploy does It makes it very The behavior now it's changes a little bit Okay, so now I can simply start a Server with a configuration file like this and it's now listening on port 8,000 and the behavior is Equal to the previous behavior from the previous example, but if we look at the configuration file serve There's some parts the first part exit command equals serve and demon like true We also have the ability to start this process as a demon in the background and As I told you also you can use the paste strip itself to serve directly from the configuration file So I can just call the configuration file Executor start and now we have a listening demon in the background Which is really nice and useful because you can write very fast some small applications And of course you have your user stuff like status and of course also stop Now we are here to write something like a swift middleware and not basic whiskey application So let's have a look and what we do want to do here and For this talk, I was thinking about a small preview middleware that is useful for example if you want to write I don't know maybe your next image sharing service or something like that So what we want to do here now is for every put and the put is in that case an image that I sent to the swift proxy I want to create a small preview image and start it separate from the original object If I execute a get request and append a query string preview I don't want to get back the original object, but only the small preview object and And finally because I'm not storing two objects I also want to also want to ensure that the preview image is also deleted if I delete the original object There are a few helpers and useful tools that we are going to use inside this middleware one is called split pass and split pass takes the part of your URL and Returns your version your API version your account name container name and object name and you can Modify the or you can add some arguments to split pass to ensure that split pass returns or raises an exception in case For example, you have no object name Well, it doesn't make a lot of sense for our middleware if we want to create a preview image If there's only a request on the container level or account level So we want to exit in that case early and continue within the proxy server And because we need to store our preview image We need to execute a request within the proxy server itself. We need to do a may a sub request and To make this working you need to add this middleware after your authentication middleware because otherwise it won't work and Finally, there's a very nice decorator with Gify and what with Gify does it simplifies a lot of stuff for you and If you remember before we had our core method with an environment and in start response And if we add this ticket decorator to it We only take a request as an argument and yet this request has some attributes for example the query path information or your method environment and your body where you can read from and Finally in your middleware, you just need to return yourself app That is the next application or the next middleware from your pipeline. So let's have a look What happens in real in our middleware? so we first start and try to get the account name the object name and the container name and In case that we don't have an object name. We immediately want to return so What I already said we need to modify the split pass arguments And if you do like I've done here, it will raise an exception if there's no object name found in your pass So and in that case I exit my middleware here and the next middleware in the pipeline Will take over so but let's assume I have an object name and I want to store my preview image in a container that has in different suffix So for example, if my container more original container number was test I now want to store the preview image and test underscore previews so for a get request and The get request that has a parameter preview appended I want to modify the path and Yeah, they will help us that will be served by these with proxy server So I no longer use the original path and now use my preview pass Of course to do so. I need to my preview and I simplified the code a little bit this method called create preview as External one in that case. I will give you the code later on so you can have a look into this It's simply a Python imaging Call that just creates a smaller image But let's assume we were able to create this image and this preview so what now happens is We call a sub request and the sub request itself takes the current environment from our request the new preview pass where I want to store my preview and a new body the preview itself and Then to execute the risk risk request. I just need to call get response So in a case, I want to delete my object. I also want to delete my preview object So in that case, I only need to modify and to execute another sub request in this case Also with a modified preview on the modified path So in that case, we are we delete both objects at the same time This request will silently fail. There's no Preview object. So it will just continue to work. You don't get an exception or something like that in your proxy server Now, let's have a look How this looks like in real I have Small swift all-in-one Little machine running here Currently, there are only two Containers one container is called test and the other one is called test preview and both are empty If I now upload a small image into the test container This is like your normal behavior that you were expecting from your swift proxy from your swift cluster But now let's have a look into the test container Just preview container There's a small object and the first number there is a number of bytes And if I compare this one with the original object that is stored in the other container It's much smaller. It's my preview So these containers are both public so I can just request From anywhere never do that from a browser. I get the small premium edge, which is nice And in this case here, I appended the query path To the URL. So if I remove that part Like this I Will get my original object back So I have I don't need to remember Where are my preview objects start? Of course, you don't are you are not fixed to this you can well You can think a lot of different ways how to handle this. It's just an example So I Already said that there are existing millwares and third-party millwares outside of swift that adding that are adding functionalities to swift itself This is a small list of well, I would say the most popular millwares that are existing We have swift ours with ours is a millware that Stores all the data that you need for an authentication system inside swift itself So you don't need a database or a directory service We have swift 3 that is adding a s3 compatible API to swift We have the CDMI middleware the CDMI middleware CDMI is a cloud data management interface and some applications are using that it's different API and protocol and By using this millware you can use a CDMI like interface also in swift We have swift informant swift informant sends Statistic data to a to a stats D server So you can collect your requests and do some processing on on this Statistic data and if you want to use swift together with the content delivery network We have the swift origin server that you can use for that And for example, there's another project see the meter I think everybody heard about it already and see the meter itself is also using a swift middleware to collect the data that it needs to process process billing and whatever you need and It's included in the cylinder meter project itself now if you want to have a deeper look into what I've shown you to before my examples, I prepared a small Git repository and It's available on our github account and advanced github account and There's a step-by-step guide. I will start with a simple sample whiskey application adding more and more functionality and Finally, you have this preview middleware that I've shown you Complete with all of the tests so you can just have a look at the code and what I've done there and Yeah, it's a step-by-step guide basically There are some other existing documentation available Within Swift self so we have a great overview about existing middlewares and all the options these middleware take There is a guide how to develop your own middleware Which is which also has an example if you want to develop your own Authentication middleware you need to be a little bit carefully and you should strictly follow that guide also and Finally, if you wrote a middleware that is useful to other developers and operators It would be nice to just submit a patch to Garrett and at your project and your middleware to our associated projects And if you want to look a little bit deeper into whiskey and the web standard gateway interface and middlewares in general Have a look at the Python documentation On the link below So I'm done with my talk. Thank you very much for attending and if you have any questions Just let me know now or just write me an email Come to our booth and ask me there. Thank you very much No questions. Okay Thanks


Struggling to take care of your garden and wish that someone else did it for you? Well, this project is here to save your day. Meet AgriBot, a smart agricultural robot that monitors the moisture level of the soil and waters it wherever the level is low. We are going to modify the 4WD robot that we previously made in one of our videos to make the AgriBot. Click on the card above or the link in the description box for finding out how to make the 4WD robot. Let's begin. For this project, you will need the 4WD robot, evive, a chasey, motor clamp, a servo motor, a servo horn, a soil moisture sensor, a DC water pump, a pipe, an ice cream stick, cable tie, a lipo battery, relay, spacers, jumper cables, bottle and some nuts and bolts. Let's begin by setting the servo angle. We will set the angle to 90 degrees using evive's menu. Connect the servo to servo channel 1. Next, from the menu select controls, then select servos and finally navigate to servo 1. Set the servo angle to 90 degrees using potential meter 1. Now glue the servo onto the clamp using some hot glue. Attach the clamp to the front of the chasey. Now let's test the soil moisture sensor. Connect VCC to evive's 5 volt pin. Connect ground to evive's ground. Connect signal pin to evive's a0 pin. Next, from the menu select pin state monitor and navigate to analog pins. Observe the reading next to pin a0 when the sensor strip is in the open air. The number 1023 will be displayed. Now dip the sensor into the water. You will observe that the value will decrease. This means that the sensor is working perfectly. Now that we know that the sensor is okay let's mount it on the robot. Take the sensor and attach it to the front of the chasey. Now it's time for making the connections. Connect VCC to evive's 5 volt pin. Connect ground to evive's ground. Connect signal pin to evive's a0 pin. Extend the servo wires and connect VCC to plus 5 volt of evive. Ground to ground of evive and signal pin to evive's pin 3. Now let's make the soil moisture testing assembly. Take an ice cream stick and attach it to the servo horn. Next, glue the probes of the soil moisture sensor to the ice cream stick. Now let's assemble the automatic watering machine. Take a soft PU pipe and glue one of its end onto the ice cream stick. Now to switch the motor on and off automatically we'll use a relay. Now we need to add the storage stand. Past in the 45mm spacer to the chaseys. Next attach the servo horn to the servo head. Now bring the pipe and the jumper cables of the relay to the top. And mount the second chaseys onto the spacers. Then fix the relay on the top chasey. Time for connecting it to evive. Connect VCC to evive's 5 volt. Connect ground to evive's ground. Connect signal to evive's digital pin 10. We will be using a plastic bottle as our storage stand. Take the bottle that is already cut in half and glue it on the top chasey. Now it's time to power up our robot. Attach the lipo battery to the chasey using cable ties. Next, take the water pump and fit the pipe to the outlet of the pump. Then place the pump inside the bottle. Now connect the negative end of the battery to the pump connector. Then take a small piece of wire and connect it to the positive terminal of the pump. Now connect the positive end of the battery to the common on the relay. Connect the wire from the pump to NC on the relay. With this, the assembly of the agri-bot is done and has stayed. Time to write its core in pictoblocks. Since we need to control the robot wirelessly, we are going to write the code in the upload mode. Since the agri-bot is a modified 4 wheel drive robot, let's first write the code for the 4 wheel drive robot. The when evive starts up block will make sure that the program starts running as soon as evive starts up. The relay connected to pin 10 should initially be off and the screen should be blank. On pressing up on the gamepad in Dabel, the robot should go straight and straight should be displayed on the screen. Similarly on pressing down, the robot should move backward and backward should be displayed on the screen. The robot should turn left on pressing the left arrow and left should be displayed on the screen. Finally, the robot should turn right on pressing the right button and right will be displayed on the screen. Till this point, we have written the code for controlling the robot's movement. Now let's write the code for sensing the soil monster level and watering it. On pressing the triangle button, the servo angle should change to 0 degrees, thus inserting the terminals of the soil monster sensor into the soil. If the reading of the soil monster sensor at analog pin A0 is more than 800, it means that the soil is dry. Therefore, the motor should turn on and water the soil. Once done, it should turn off. Finally, on pressing the cross, the servo should return to its original angle. But what if nothing is pressed? In that case, the robot should stop moving and brake should be displayed on the screen. Upload the code to evive and watch your angry bot. Take care of the soil for you.


Oh God, today we give you thanks. We thank you for another one of our soldiers that we recognize today, how much we appreciate this hero and all the heroes that precede us. May we always live in the freedom and liberty bought and preserved by such great men.