Good afternoon Good afternoon Thanks everyone for coming. Hope the conference is going well Hope you're having some good food having a good time So I'm Dan Wendorf I'm a talker wait up and Where we work for Pivotal for the cappy team on cloud foundry cappy team is responsible for building the cloud controller Which is the API front-end to all the great back-end components of cloud foundry This is the API that gets used by tools like the CF CLI to provide the developer experience Dan and I have both worked on as consumers of the API as well as on the API for over a year So we're especially excited to talk to you today about the cloud controller v3 API and the things that you can do with it So the cloud foundry API is used by a lot of CF tools Including the CLI, but lots of people maybe you have developed tools directly on top of it or have interacted directly with it And as a result we get a lot of feature requests Unfortunately, some of the more frequent feature requests are Hard or impossible to do by incrementally improving the API So we needed to make a more drastic change to allow us to respond to the changing needs of app developers So what were our goals? Backwards compatible We want to maintain v2 behavior eventually we will be deprecating v2 and new features will be added Exclusively to v3, but for now we know that there are a lot of tools out there, and we don't want to break that experience Modular the cappy team As members of the cappy team, we have a lot of great ideas about how to improve your app deployment lifecycle But we are just 10 people We want the API to be modular so that your ideas are possible and so you can customize your app development lifecycle to your needs flexible Flexible code base for us means we can deliver more features for you Faster We want users to spend less time repeating the same process over and over again and more time delivering Easier to manage We want to get rid of hidden complexity and they need to have a deep understanding and a deep knowledge of the internals of cloud boundary Develop developers shouldn't have to know that an app needs to be restaged if a new build pack doesn't work out and Your app development process should be easy to understand and customize Consistent Lastly we want the API to be discoverable and unsurprising So how do we accomplish these goals? We initially wanted to extend the existing API But we were blocked by overall domain modeling and design So the v2 app is a monolith It's defined by app configuration like the start command memory limit CPU number of instances and the droplet in the package And it doesn't really allow for things we value like the flexibility of implementation or a client-side manipulation There is a meta programming layer that's closely coupled With and shared with the database models are shared between the database models And it can make adding simple features for one API resource a very complex task For instance if we make a post request to v2 apps We hit the same code as it as we'd hit if you were making a post request to v2 spaces So if you were to try to change the code for how apps behave we could make an undesirable change in how spaces behave This previously existing implementation made new feature work very slow So we needed to redefine some of the constructs to provide flexibility and future-proofing So our solution was to extract things like the droplet and package from a v2 app into top-level domain objects That have their own API endpoints that all live under an umbrella v3 app By splitting the app monolith into more manageable and independent concepts We're able to provide an API that supports more flexible and nuanced management of your cloud boundary apps So for the technical details we encourage you to check out Jim Myers the law in Santos and Zach Robbins's talk from last year's summit The details are provided in this site So Dan what does using the v3 look like? fantastic question So I think the best way to understand what using v3 looks like and the benefits of v3 is to go through and build a little bit of a sample app That uses v3. All right, so we're going to start by talking about building an app in v2 and how we can make it better We'll start with some really basic setup What we're going to build is an app about my favorite monkey of all time boots So we've created a boots org already and we'll use the CF CLI to create a space for development That we're going to work in right away a space for production that we're not going to work in just yet we're going to target the development space and Let's say we've got a little bit of a database somewhere. So we'll create a user provided service that Just has username and password. By the way, that is my password. Please don't use it All right, so the first thing we're going to do is we will push the app This is going to be a normal web app So it's a little bit complex which means it's got a few different processes the most important one I would say is the web process. So we push boots web We're going to need to need a worker process to do things in the background. So we push boots worker and Finally, we're going to have to have a scheduling process. So we'll push boots scheduler In order to do this we have to push the same code multiple times, but we've got these different start commands with the dash c flag After we do that, we'll bind our database service to each one of those applications so that it can connect to our database. It's great All right, so if we look at what's going on here The push is a little bit more complicated under the covers than just a CF push What the CLI is doing is orchestrating a bunch of different API commands to get our app up and running All right, the first thing that we do is we create an empty app with all the metadata for our app As it taco said things like CPU instances memory We create a route for our app and we bind that route to the app Then we upload the files which might take a little bit of time depending on how big your app is This real interesting one staging of the droplet. This is where once we've uploaded the source code We compile the source code. We bring in dependencies. We build the droplet, which is the runnable file the runnable zip file That's going to be sent to the DEA's or to Diego If you've pushed an app cloud foundry, which I hope you have You'll notice you'll probably have noticed that staging an app takes a very long time compared to the rest of the steps So once that's done, we can run the droplet and we have our app up and running But it does take a lot of time because we're doing the same thing over and over again We're repeating ourselves a lot. So what does this look like in v3? Well, it's a lot simpler. You can just push boots your one app We're going to be doing a lot of the same steps But we only have to do these steps once instead of once per app Once we've done a single stage and we've made our single droplet We're able to just run that same droplet three separate times What we're showing here the CF push boots. This is implying that it's using the v3 of our API all the new features This CLI command doesn't exist as we're showing it now What we're showing is a potential for the CLI as they do the work to incorporate the v3 CLI or v3 API But we're going to continue to show this as what the CLI could look like or if you were to write a plug-in What your plug-in could look like Now when we see just the CF push boots in my estimation this feels better It's this early a better experience because I just have one app when I'm talking about my app of people I say I have boots calm. I don't say I have boots web and boot scheduler So it always felt weird to have to do the same thing three times The way we get this to work is by adding a new file to the root of your application called a proc file It's a very very simple file. It's been used by the other places like heroku Where we just define a one-to-one correspondence of we want a web process. It's going to run our bin slash web Executable same with worker same with scheduler when this proc file is present at the root of your app Cloud Foundry knows what this means and knows to spin up three processes for the one app I think this is really really powerful It means that we can talk about our app as a cohesive cohesive unit and Cloud Foundry is is doing Single things to the app even even though we've got multiple parts and now that we know that we can manage Each of these processes in one single app means we can do all sorts of other cool things We can control services all together for our app and affect all of our processes So the same that same way that we were able to compress push into one command. We can also simplify service bindings So there's no chance of forgetting to bind a service to one of your apps and not your other app They're all orchestrated at the same time a lot more consistent All right, so that's that's a simple development flow for building your app and development and Presumably at some point we're going to want to go to production So what does it look like to move your app from development to production? In the v2 world, it's very very similar. You're doing the exact same processes. You're pushing all three apps you're binding services all three all three apps and You're going to map the route to a route that you think is a little more production ready But because we're in v3 and we can orchestrate these things together we can do a lot better We can do something like copy Copy an app from development to production This wouldn't wouldn't need any sort of staging. It'd be a lot faster You don't even need the code to be on your development machine the code the droplet This is all in cloud found real found real ready. So why would we expect someone to have to provide it to us again? It's a lot of duplicate information All this is doing is it's copying the app configuration It's copying the same droplet that was already staged. It's binding the services and it's going to run the droplet Very simple a lot faster in v2. We would have had to stage six separate times three times in development three times in production and In v3 we only have to stage once we've cut Maybe 15 20 minutes out of this process depending on how complicated your app is Updating So in v2, this is what we do We stop the app we push our updated code We create a package We upload the files to that package and then we stage those files and create a droplet Once that's done. We start the instances of the new droplet in V3 we do something similar But a little different We push the updated code We create a package we upload those files and we stage that droplet Next we stop the instances with the old droplet and then we assign the new droplet to the app Once we've done that we start the instances with the new droplet Now what does downtime look like in the v2 world? As you can see by the grayed out area in v2 your app is down for almost all of the updating process In v3 It's only down when we're swapping the old droplet with the new droplet So what if the update wasn't so great and boots comm is down What do we do in v2? Well in a typical deployment process, maybe we would Freak out look through the logs to find the previously deployed version Check out the older shop And once we have done that on our local machine We'll deploy that local We'll deploy from that local machine and this will be outside of CI which could be a potentially very destructive or dangerous process Once you've done that you sit around hoping that you deployed the right thing and wait for your app to come back up In v3 This could be as simple as one command CF rollback Because v3 keeps track of your apps last five droplets you can roll back to any one of them So we've talked about how existing workflows could be improved with v3 Now we'd like to talk about something completely new We're really excited about this feature and users have been requesting this for a really long time and Diego's had this feature around for a while now, but we just didn't have the structure in place to expose this Implementing v3 has made it possible You can now run arbitrary commands on your app Your task and app share the same code the same environment variables and the same service bindings So what can you do with them? You can run a migration You can send an email blast out to your users You can make a query against your database Anything your heart desires You'll also have easy access to the output of every task as well as a history of tasks you run So Dan, what's in store for the future? Thank You Otako When we've been practicing this Otako use the phrase anything your heart desires and she really wanted me to sprinkle confetti or glitter I really wanted to but I thought you guys probably don't want to get covered in glitter. So I restrained myself All right future v3 now that we've done most of the work to get v3 off the ground We're getting ready to release it in an alpha form means we're going to start moving a lot faster and getting new features Reimplementing the push and the update was really great made a lot of wins But we were excited about tasks because this was something that we couldn't do before and we're excited to bring out some new features But first what we've got to do is work with the CLI team to get this baked into the CLI itself So you're using v3 without even knowing it We think that backwards compatibility is really really important with this We know people have all sorts of scripts. I know I do that use v2 already. We want that to continue to work While you can start incrementally taking advantage of new tools that are using new features in v3 Our goal is that you're you'll be able to use v3 tools and v3 v2 tools at the same time to manage the same app We're also really Thinking about zero downtime We've we've reduced the amount of downtime you're going to have when you push an app when you update an app That's still not great What what we want to have is zero downtime people have found ways to work around this with custom blue green deployments But we think that we can do this especially working with Diego to do some of this orchestration People have also been wanting to take a droplet from one cloud foundry and move it to a separate cloud foundry maybe you have a development cloud foundry with less restrictions for developers and a production cloud foundry behind some network infrastructure developers don't have access to now You'll be able to or you will be able to take Droplet that you've built a one cloud foundry and port it out. That'll be nice and simple and We also want people to be able to create processes outside of the standard push process You saw creating the three processes with the proc file. That was nice but in in The vein of v3 wanting to provide these building blocks for developers to do new things have new workflows We'd like people to be able to add processes modify processes and remove them from a running app without having to do a push So you can tell you're all at the edge of your seats. You're ready to run home and start using v3 So, how can we do it? So now that we've gone over some possible ways to use v3. We'd love to have you to try it for yourselves today It's free You could start out by checking out our new docs We'll be updating them as we add new features and if something could be more discoverable or more consistent or easier to understand Now is the time to let us know You can try v3 by using cf curl or better yet write your own custom plug-in to build your own workflows and You can also try out our v3 CLI plug-in, but I think it's better to build your own V3 has been experimentally available on recent versions of Cloud Foundry Experimental means that we make no guarantees about the API endpoints themselves and they may disappear And your data might be white Really realize this may be challenging to work with so we're working hard to produce an alpha release Our goal for the v3 alpha is a guarantee of no data loss, but there will still be API changes And it's in the track to be released in the next quarter. So your feedback is especially useful now You can submit an issue on github or reach us on the cappy channel on the Cloud Foundry Slack Community involvement is especially important to our team and to I think Cloud Foundry in general So we try to have a pair spend time addressing these concerns every day Thank you very much. And are there any questions? You mentioned about the app bid promotion from between spaces without restaging So what if I am actually in a staging environment and that's staging space and then I move this to prod then without Restaging I would be actually aiming at it staging database rather than the production database So why what's the reason behind not staging the bits? I understand that you want to avoid the staging for the web the process or the scheduler Whatever, but you need to stage across spaces because the service endpoints the service credentials those things would have changed, right? the The credentials for services are available as environment variables Yeah, the build pack actually a lot of the build packs look at those environment variables and put the configurations into the App bits into the droplet. So if you don't do staging you don't really get those Yeah, it depends on the build pack some of the build packs will try to build that in Right custom configuration files. So to take advantage of this particular feature You would have to not not do that in staging and only look at the service environment variables at runtime So it might not work for every app Depending on the build pack Do you have plans to make the droplet available to our admin? So there are instances where the droplet crashes the app basically crashes and there is no way to actually get into the water cleaner because it's a short lived one like it's gone in five seconds and Either you have to put in like a sleep statement or sleep forever So the water container or the garden container does not get removed right away Evacuated it right away. So is that an option to export the droplet? So you can test it on your Linux environment just to see what what's failing Fix the code and then repush it Would that would you take that as a request enhancement request? I'd have to check with our PM, but I believe there are plans to make downloading a droplet available. Thank you. I Think is downloading available already. No, take it back. That's how much I thought it was in the plan Any other questions Right now we can copy the droplet between apps But not spaces That'll be maybe I mean, that's like a implementation thing If you have those custom needs if I want to copy this but not that the API does support Through multiple multiple endpoints. I want to copy stuff from this endpoint to that one from this endpoint to that one So if you have different needs, that's where a custom plug-in or maybe a plug-in with lots of extensibility would come into play That's a great idea And I believe our PM Nick Kaluga will be available to talk about that That's also something that Nick would be open to talk about Yeah, I think in general these sorts of questions are really great ones to hear Let's us know the sorts of use cases that are important to you So that's we're talking to us in Slack or submitting GitHub issues We'll let your voices be heard and we'll definitely take that into consideration RPM is also very vigilant on the CF dive web mailing list So really any of those things are great ways to talk to him take a bow neck. Are there any other questions? far too long for at least if I don't Don't you know? Some amount of time Probably at least a year or your v2 stuff will continue to work And we'd like to make it a point like v2 should just people who are currently using v2 now should not be able to notice That the v2 functionality has changed at all. They'll just notice that there are more features in addition to that That's all Thanks everyone. Thank you