What in the shit? Julian has already taken the liberty of choking this person in the beauty store. It's a nice little jiu-jitsu. As promised, I said that if you want me to make a video just like tweet it at me and I'll make it at this point. So that's what I'm doing. Someone has been requesting on every platform, Facebook, Twitter, YouTube. Like they're on every- My vlogs. Julian's vlogs. Tell Jenna to get a bald cap and cover it in fake eyelashes. Look, the world's gonna end. We might as well do what we can while we're here. You know what I'm saying? I could put it on and glue them on but like I won't be able to get the back at all.  He's had it with me. But I'm trying to figure out a way to get them on here so that when I stretch it onto my head, they don't all like pop off. My theory is that it's just gonna look like short hair and this is just gonna be a whole- I'm essentially making a wig out of tiny short hairs.  I'm going to cover this bald cap in fake eyelashes. Boy, I swear to God. And then I'm gonna put it on my head and I think it's gonna look like hair but we don't know. So let's find out, shall we? Don't forget I need her. She's not tapping.  She doesn't have any arms. Should I start in the front or the back? Shulk in the front. Joke in the bunch. Draw my ears like a little bit. I'm gonna draw the inner part of your ears like she said. No, like the inner part. So we're working with that. Should I give her a part or not? Should they be facing down like that or up like this?  It's looking so good already. We only got one pair on. Oh, no. That's a blood bath. I'm just gonna try to do the back because I don't want them to be too close to each other because I don't want to run out of lashes. This is much better because then I can lay them on top of each other instead of like trying to get them under. I think that that looks pretty nice and natural.  This is gonna look like if a newborn baby was given a hair transplant two seconds after being born. Like, this is already like two packs of lashes and I've only covered that much.  Hell yeah. Watch how I'm done with this. I'm going to be like, girl, where did you get that wig? And I'm like, yo, I made it myself. And I'm like, yo, let me get one.  Oh, it looks so good. Hell yeah.   Y'all gonna be jealous of my exquisite wig. So I've made significant progress. I think it's looking nice. Why do you look disappointed? Because I am. What? Why are you disappointed? You look dusted. Your hair's uneven. Oh, no!  Business in the front, butthole in the back.   No, it looks great. You need to finish though because right now it looks a little bit caramel. A little bit garbo? A little bit, like warm garbo. Not quite hot garbo. Like imagine being our trash person finding this in the trash. Trash haul. What's in my trash? That's a good video. Let me know in the comments if you want to see what's in my trash video. Subscribe to my trash bin.  Do you want me to start unpacking them? You can, but see how nice and neat everything is? Yeah, I don't operate like that. I'm a burgo. Things have to be organized. We're gonna have a lot of hair clips when you're done with this. What do you mean hair clips? What's hair clips? These are the hair clips. They're tweezer. What? To like apply the lashes. They could also work as hair clips. Comment below that this is a hair clip because it is. That's not a hair clip. It fell out. It's a bad hair clip. Stop. Julian. I was trying to intimidate her. What?  Oh, I love spending time with you. Stop. It's quality time. His head's sore. I think we should release him. He was decapitated hours ago. I think he's a little sore. I don't know where my ear is gonna go yet, but I think that should be okay. Maybe we trim around it.  Walks into barber. How can I help you?  You walk out like this? The hardest part when you're done with this is gonna be dealing with my boner. You look like you took someone's head and dipped the tip of it in a bowl of unfortunate. It's shitty on the side, but also on the top and also on the back.  Are you not impressed? I'm impressed with people who've climbed around Everest and not impressed here. Oh, my God, it just kind of glimpsed up like the lower part of its neck, and it looks like a dude. It looks like a person, dude. Look at... It looks like a person.   Oh, my God. I can't even tell you how creepy that is. It feels like a spider. You know, like, if you have kids and like, grandkids and you're like, man, I wish I could meet like, my great, great, great, great, great, great, great, great grandkids to see, you know, what their life is like.  It looks and feels like a rat. My arms are exhausted. My hands are covered in glue. But... It's so nasty. It looks good.  That is so nasty.   Thank you for being such a wonderful model.  I want to keep her. Truly, yeah! Here, you've watched Face Off. You know how this goes. Be really careful. Where does the vacuum form go? Ow! Ow! Ow! Your head's still... You have to stretch it. I am. You have to keep your head strong. It's fine. It doesn't hurt that bad. Ow! How does it look? Oh! It looks beautiful. Natural looking. There's a crank right here. Let me just... It's like you're allergic to gentile. Can I start cutting? Eh! You just cut my hair. I felt it. I just nailed both of them. Oh! No, I'm gonna cut off all of them. You need to cut my rubber mullet.  Well, what do you think? On subscribe for this night. On subscribe? Why?  Why? Why would you want to subscribe? Man, look it. It's a tiny masterpiece. I think that it came out looking really natural and graceful. Like graceful is the word that I would describe this as. Look at the top. Look at this eloquent circle. It's so beautiful. It feels almost like sharp. Like it... It doesn't feel right. And like... It sounds terrible to touch it. Listen to that. When you look at me straight up, it looks so weird. Don't do that. No. Why?  It is my hair. Oh, it's wrong. This is my new like Saturday night getup. Like I'm just...  It's a party hat. What if this was my hair? You wouldn't date me? I would still date you. Ew. I would give up boners for wind.  I would still give them up out of respect. So you're saying I don't look super cute with short eyelashy hair? I already said I'd date you, okay? Smash or pass? Little down, little down. Eyes up at me? Smash. Touch it. Smash. Smell it. Pass. This is what the back looks like. Is that nice? To the person that kept requesting this, I hope this is everything you wanted because I don't know why you wanted this or honestly what's wrong with you. It came out pretty great. So I'm gonna keep this as my party hat. What? Something about you with this hair holding your hamster is just, it's just a lot. I look like that person that rides the bus to nowhere in particular. What about this is creepy, Julie? End it. End it all. Make sure you subscribe to my channel for whatever reason after this. Yeah, I don't think you actually have the right to tell people to subscribe to your channel after this one. Subscribe to my channel. This is what happens when you put like 400 eyelashes on a bald cap. It turns into this. I think it's a good look and it's gonna catch on. Looks like when you look up on natural in the dictionary, they link you to this YouTube video. Whatever, talk to me or I'll stun again. Run your fingers through my hair. Okay. Oh, I'm gone. I'm gone. I'm gone.


And we are back with the breakfast, let's delve into the next conversation. Now, over time, it's been a lot of argument as to the involvement or the impact of ethnicity and religion on our elections. But really, do these factors play out during the elections? And do they influence, you know, the political terrain for Nigeria? It's a conversation we'll be delving into as in no time. We have Tunde Kolaulihu be joining us this morning. I mean, Tunde, it's good to have you join us. Good morning. All right then. Now, just a bit of a background to this discussion we'll be having. A recent poll conducted by SBM Intelligence in partnership with enough is enough, Nigeria, has shed light on the impact of ethnicity and religion on Nigerian voted turnout. Now, the poll gathered data from 11,534 respondents across the country, probing into the voting intentions and preference in the upcoming presidential and parliamentary elections on Saturday 25th of February 2023. The survey highlighted the impact of ethnic and religious division or divide on voted turnout with respondents in some region indicating that these factors would play a significant role in their decision to participate in the election. Now, according to poll result, many respondents cited ethnic and religious factors as key determinants of their voting decision to make sense of all of this and analyze these findings ahead of the forthcoming election is Tunde Kolaulihu, who's a legal practitioner right here in Lagos. Tunde, it's good to have you join us. Good morning. Good morning. Tunde, do you agree with the SBM? The Nigerian constitution expands the mixing of religion with politics and the constitution says that nobody should be discriminated against on the basis of religion or ethnicity or disability. For the reality of life is that it's been difficult for women to take religion, to take ethnicity out of their political beliefs, out of their political practices, out of their voting patterns. And unfortunately, it is not peculiar to Nigeria, you know. If we assign democracy to a society that is well educated and has impacted democracy for years, you still find out that religion and ethnicity, in one way or another, influences their political beliefs and the way and manner in which they vote. I give you an example. In the country like Germany, we even have political parties that are formed on the basis of religion. Say, for example, the Christian Democratic Union in Germany.  And they will use that as a basis for institutions to create another state, to create another country in the place like Spain. Also, when you look at the Arab world, you'll find out the Shia, the Shia, and some of these other religious, Islamic religious denominations tend to vote for political parties that spring up from their religious beliefs as homogeneous as a country like Kirani in terms of religion. But they fear the challenges of some people in their side, some in their Shia, and no manners of religious activities and all that. So, the findings of that research is not far-fetched. Here in Nigeria, there is a number, especially the Yoruba people. They are most conservative. They actually want to vote for somebody who comes outside the ethnic base or culture. Now, the reason you find out all the time, all who have played this politics, the Yoruba are always totally behind them. Also, in the northern part of the country, you'll find out that there are a lot of those who are full of, are always voted. And as there are people who vote for a political party that has a fuller, more announced man as his presence here and then as his governorship candidate. Religion is also a very potent weapon in the northern part of the country. In fact, sometimes when a lecture is made about 20 hours, on the Friday, for example, the Islamic clerics, the Muslims, the Imams, in some of those churches, who give a directive as regards the religious, I mean the political parties, that the Muslims in the north are for the aggressions in the north to vote for. So, it's a very, very potent weapon. And it's all prequels in Nigeria. In a place like England, an Anglican church has a very significant influence as regards the political processes that take place in those places. We cannot wish it away. And the funny aspect of it is that as educated as some of these people, you will think that when people become well educated, when they become cosmopolitan, when they are taboo, param, white, they should be shedding their tinnish toga. They should be shedding their religious toga. But education, cosmopolitanism, cosmopolitanism, and what happened to the Kala Walei, and the tinnitus out of it is political living. So, I mean this report or findings saying that the 2023 elections would have or witness a lot of voted turnout as compared to previous years. And you have agreed that these issues would always shape the elections. But let's cast our minds back to the 1993 elections where it wasn't a problem because it was a Muslim-Muslim ticket. Why is that different?  How come religion and ethnicity did not play or have a huge role to play in that election? And why would the 2023 election be decided by religion and ethnicity? Well, it is not totally true that religion had nothing to do with the 1993 elections, in spite of the fact that the Muslim-Muslim ticket was able to win the presidency. The circumstances of 1993 is totally different from what we experienced in 2023. Me and I would rule. At the end, I was a cosmopolitan person, highly charismatic. He was a philanthropist and then half white from the different parts of the country. Everywhere he goes, in Nigeria, it was like a hope for him. He was such a very charismatic and colorful political leader. And then in vice-president of Akin Gibi, was also somebody of that nature and status. The white was only the Akin Gibi and then also, because he's been in politics for a long time, he's made friends from across the country. So because of their own charisma, it was not too difficult to really sell the Muslim ticket, the Muslim-Muslim ticket to the average Nigerian person in 1993. But me and I would rule that a lot of water has passed under the bridge since 1993. For example, there was no Bokoara in 1993. For example, issues, mundane issues, like wearing of hijab and no wearing of hijab. What is our mission in before 1993? Furthermore, the Christian military that we begin to see all over the country too is not as competent as it is today. So, so many things are happening in Nigeria. Touch that, a wildlife. There is a scale. They are religious beliefs or they are religious dispositions and others. They've been spoken and manipulated by a religion to achieve whatever they want to achieve, either in the area of politics, or in the area of Christian, or in the attempt to dominate people from the other areas and sectors of the Nigerian society. The religion has become a political thing in the hands of a wildlife. Which of them is very much a development? To De Kola Wally, I mean, if we look at this, if you look at the art democracy from the time that we say we have been practicing, the 1993 elections might just be the only elections where these issues were downplayed. And I really don't know if we will get the answers. The answers have been given. Well, you want to ask yourself, how come these issues were not even an issue? Because over time, elections have been shaped by this factor. And the 2023 elections, it's not going to be different, however it is. But do you also think that the dichotomy between the North and the South is also a major issue? What we classify as rotational presidency? Do you think that that also will contribute to whether the South or the North or the ethnicity and religion becoming a factor for the 2023 election is predominant? There is no doubt about it. I have no doubt in my mind that this regional dichotomy also would have the role to play in the 2023 elections. And like I said earlier, let's be very careful. In the South Republic, religion was an issue. We had another element, progressive union, and then you have NTSC and some of these other different political parties that even though they may not see to say it openly or loudly, they still come back to vote based on religion. Now, the journalism has become even more escalating. You know, if you recall, it was under Jerusalem's attack that Nigeria was quietly and tacitly divided into six regions. North, East, North, West, and West. And that itself arose out of a desire to pacify those who are grieved with the notification of the election that was conducted in 1993. And I think it's a our political history. Each time I went over, for example, as to the election, the people in the South had never voted for him. They go with NTSC or with whatever political party that Dr. Namda Zikwe might have established. I suspect the reason for that was partly because of the role that he provided in the whole of place, not only in the Nigerian civil war. And also, the way and manner Dr. Zikwe was shoved aside from the politics of Southwest of Nigeria, because if you look at our political history, Dr. Namda Zikwe's political party, the NTSC, was the one that was dominated in the West, in the Southwest. They were winning elections. They were having people in the legislators in the pattern, whatever. But when all started, the action group, and he mobilized these people based on ethnicity and what have they, they were able to, so to say, with the apology, remove Dr. Zikwe away from the politics of Southwest of Nigeria and never to care. This fraternalism has become worse over time. Yes, please. We don't have so much time, but I'd like to ask you this and let you share your thoughts in just a few seconds. Now, it's very obvious that this has always been an issue up until 1993. And after 1993, ethnicity and religion, according to history, still seem to play a major role. Now, for 2023 elections, few days before the elections, there are also statistics and research that have confirmed that people would turn out to cast their vote because of these factors. My question is, should we, I mean, shouldn't we have a country outgrown all these diversities that has held us bound and back since independence? No, no, we shouldn't. We shouldn't have an outgrown it? You know, the way to look at that is this. The impression that we have in the country today doesn't spare the Muslims, doesn't spare the cultural professionals, the traditionalists, and all that. So the best of my knowledge, we have been under the helmet of inflation. Second, when you look at this security, the banter, the kidnapping, and the lack of safety on the highways, on the railways, even in the air, and this security and religion, you guys know, all of us are caught in the web of all these criminalities that are taking place all over the country. So if that did the case, what is the emphasize thing? Are those things that really unite us as a people? The things that will put food on the table of the average Nigerian person, the thing that creates security for the average Nigerian person, the thing that will make it possible for the average Nigerian person to live a comfortable life without having to steal whether in the private sector or in the public sector. But it is our allies in order to call the advantage, in order to have an image over the allies of the other religions. Who keeps talking the fire of religion? Who keeps talking the fire of ethnicity? Who keeps talking the fire of religionality? If we are the nation to develop and get to the situation we also feel by now, we must be seen to put aside as the position of the empathize that we should never discriminate anybody on the basis of the religion, on the basis of the time they can stop. I'm just saying that Kola Wale, we need to go now, but you say that we shouldn't have, you're asking yourself, why are we still grappling with the issue of religion and ethnicity in 2023? Why is that even the case? What was different in 1993 when these factors wasn't even a yardstick? But is that really the case for 2023? You want to ask another question, but in all of this, how far have we been? 1960 up until this moment, we're grappling with that. Should people not cast their vote based on, it's not important wherever it is that you're from or whatever religion that you hold, but what it is that you are, who you are, should that not be the basis that we're one entity, what is important that you have, who is qualified to be part, I mean to be elected to that position. Tunde, I know you have your question, I mean your response to this question, but we don't have time to get that right here. We have to go and I hope that we can have this conversation just before the elections or after the elections. Thank you so much for being part of the conversation. Good morning. Okay, well why that hasn't been the case for us is what we're trying to understand would be right back here to have further conversations. Please stay with us.


This means there's angels with you You get shot in the leg fam. You don't deserve a leg shot nigga. You ain't earned that but that's That's the mercy of a lot of Yeah, we on boss talk TV shot at the heat of reason you see very much So what when I say this name when I say she'll night will come to mind. I Mean, you know she'll night was not a good person and me and him had a very Antagonistic relationship so much that it put me in jail.  Oh, here's this Six, yeah, yeah Kobe Kobe Brian was there Amari stoddermeyer So I'm I'm late I get in I'm like I ain't got time to go to my room and get in the show to make sure Kanye straight You know, I would be backstage is Encouraging member will say the prayers and I thank God for Kanye West and I like bro It's time, you know other places we's in London. He like I can't do this I'm like, bro. We got to like we don't have a choice.  So I'm not gonna do the show Can't do that my nigga. So so I'm like, no, let me leave my bags at the at the at the guest desk And just go into the concert. I'm VIP with this nigga. Yeah, he getting more famous at this point It's just hot this point. Yeah, he's getting fire, right? He came out the gate. Yeah, yeah I had the famous crack music song with him me him and game at that point.  I run directly in the sugar and this is This is six months. No five four four or five months after the shooting Hmm, and you know, and I'm surprised that I got I went to jail for that because I didn't shoot him Right that part. I mean, look at the law. Look look what the court say the court say I didn't shoot him.   This means there's angels with you You get shot in the leg fam. You don't deserve a leg shot nigga. You ain't earned that but that's That's the mercy of a lot. Oh So you get a leg shot and how I go to jail for that man. Wow, how I go to jail How do I go to jail for that and everybody all my leak is you don't even know he's the illuminati He would puff in your name. He name me me me Puffy my big brother.  We had a good thing going. I was paying you said man The money got way higher than that my league. Wow too much was at stake. This is an LA We've been Santa Monica. He says money was too much man.  That's my big bro. We talked, you know quite often actually But I had to understand because I felt like abandoned by him now. He got money. He don't need me. No Not it's up to me to get up to it now Hmm. So, you know people saying oh, he would put me here He judge is such and that's why he he a bad person like sugar the bad person not me, right? Why y'all saying I'm the bad person?     You know, I'm saying how you know How you know it was me and Why don't you tell So wait a minute.  No, I definitely it's possible.  You seem just you deem yourself as powerful and that you made it by yourself and other people are not worthy That's what that's what it looks like to you That's not the truth change that lens and look at it through a clearer lens not the lens that with all the mud and Scratches on it and that lens has to shrink and observe you only Yeah, we on boss talk TV shot at the e-heater reason you see


Hi everybody. I'm Shankar. Let's start with a quick demo before we start with the session. Let me get my laptop set first. What is that? All right. Let's do something interesting to... All right. And let's see if icing is up. All right. And let's get the username and password. It's according to the docker. Okay. Yes. Here it is. All right. So it took me what? A minute, Max, to get icing going for you on Rancher. All right. So, yeah. So let's imagine the world as being run on docker and everything is a container. That's an ideal world, but we don't normally have the luxury of running everything in docker on day one. We all have to start somewhere. And the idea with this session is that you can see the flexibility that docker gives you. And Rancher is a management platform for running your docker apps. What do I do? I am a freelance consultant. I work with enterprises and startups on their operational issues. I also do a lot of work for Apache Cloud Stack. I build private and public clouds using Apache Cloud Stack. Previously, I was at InMobi and Yahoo. And traveling with open source for 15 years, even before I started working. So all my efforts are completely on open source and I support open source wherever possible. A show of hands first. How many have thought about using docker? Excellent. How many are using docker in production? Not too great. Okay. So it's like five people using docker in production. So much for developers love the user experience. That must be something missing here. So the point I would like to... I mean, everybody take away at the end of the session is that, yes, we can get things done very efficiently with docker. It doesn't have to be complicated. And why should you be doing it? So, see, from a developer perspective, right, they make too many changes all the time and make it most often the responsibility of the operational person to manage the change management. And there's a lot of conflict of interest when that happens. Developers want to push code as fast as possible and the operational team wants stability. And since I come from an operational background and I think most of us attending here also from the operational background, what is it that we can do to make things more robust? And we had this boss session yesterday about automation and everybody had the same concerns that some machines got a sink, one change breaks things, and so many, so many, so many different things. So here's a perspective. So what we want to do is we want to have a consistent image right from the time that the developer does his development on a laptop and that goes into production. And we want it to be very, very quick deployment. You saw in the demo earlier that I actually launched Icinga with less than a minute of overhead. I mean, it took more time for me to find the username and password than to actually launch Icinga. So that's a key takeaway from an operational perspective, right? For you to go and do a deployment, that's the time that it takes for you. Which means now you can do an upgrade or a rollback in the same instant. I mean, 50 millisecond or one second is too short if you ask me even for Icinga or Nagios to catch because your pollers run every 60 seconds. So if you can roll forward and roll back within a minute, excellent. Nobody's going to see a blip. And if you can do such things, we can run every app as a microservice and scale up, scale down and do all those interesting things that we often hear large companies talk about, right? But given the fact that only five people raise their hand when they said they are using Docker in production and what are the most common reasons why nobody wants to go with Docker at this point in time. The biggest, biggest challenge right now, at least for the teams that I work with in Docker adoption is there is no decent orchestration and management tool. If you are on AWS or if you are on Google Compute Engine, they provide Docker as a service. But what about for everybody else who's either a multi-cloud or are having physical bare metal servers or if they are running on their own laptop, for example in VMs, there are lots and lots of people who use VirtualBox or VMware to do development, right?  Now, if you do development on Docker, or rather put Docker into production, then we have other issues to look at like monitoring tools, log management, and my personal favorite is networking. Docker assigns itself its own IP. You need to manage port forwarding, which ports to expose. How do you know how app A running in container A talks to container B? There's so many plugs that you have to plug in before you actually have a complete working environment. Storage management. Docker apps are very ephemeral, right? You stop them and start them.  You have to maintain configuration and the data. Imagine MySQL, right? You have your varlib MySQL. If you keep that inside a Docker, there's not going to be any consistency. You need to still manage the data for your apps and the configuration for your apps separately. User management. Though from an operational perspective, I would rather solve the bigger issues of storage management, the networking, and the actual UI to manage the whole thing. I can forego multi-user management, but yes, Rancher does support all of that. Essentially, it's an excellent framework. It's brand new technology. It's not been there for a long time. It's like six, seven months, and it can do a lot of things for people who want to run Docker introduction. So how does it work? It supports multiple clouds. It supports bare metal, like I mentioned. It takes care of your lower building blocks, which is your networking and storage services, does your user management, resource management, service discovery. Service discovery is going to be based on HCD, and it's on the road map. It's going to come up soon. Orchestration we do right now, and monitoring, you can get, not icing that kind of monitoring, but stats about what Docker itself provides.  So that's what gets exposed to the management infrastructure. So how do we get started, right? We installed Rancher. Rancher itself is a Docker app. So you can launch an EC2 instance or on your VM, and you do a Docker pull, and you start. And it's already set up to connect to the default Docker registry. Many people who run Docker in production, I know of, use their own private repositories. So we can talk to private repositories, and if you have your own customized Tomcat with a var, and so forth, you can point it to your local registry and continue to use Rancher. And where are you actually going to deploy your containers in? The containers could be anywhere. It could be in your physical data center, say in Netmagic or any of the local data centers, or you might have tied up with any public cloud provider. You might have your own private cloud. All Rancher requires is a machine that supports Docker, which, if you're using anything like in the last two years, Docker is probably built into all of them, Ubuntu, CentOS. If not, app to get installed Docker or YAM installed Docker from the Docker repo, and you're set. And once you have added physical machines or virtual machines, you can now start deploying apps using Rancher. My personal problem with Docker has been to manage its networking, like I mentioned earlier. For people who are running Docker in production, the IP addresses that is assigned to a Docker container are private. Your host has... Let's take an example of Amazon EC2. You have a public IP or an EIP assigned to it. That's one. You have a private RFC 1918 network address assigned to the physical interface. Then you have a container which has got Docker 0, Docker 1, Docker 2, and so forth. They have got their own private IP addresses. And somebody has to manage exposing the ports out into public. Normally the way you do it is that when you write your Docker compose file, you would say expose port 8080 or 80 into the private IPs of the Docker container itself. That's one. But what happens when you're running multiple containers on multiple machines? Or when you migrate services from, say, machine A to machine B, who's going to keep track of all of it? To solve this problem, the way Rancher manages it is that it creates an overlay network. So the network actually spans across all the physical and virtual infrastructure that you have added to the management system. If you use Google Compute Engine, you will see that they provide you a subnet that spans all the regions. So you don't have to worry about VPCs between multiple regions, running VPN tunnels, so on and so forth. So when you actually add host into a Rancher management framework, it does an IPsec tunnel between all the physical and virtual machines and gives you a single namespace from an IP perspective. So each machine is spingable. You don't have to worry about any kind of NAT, VPC tunnels or physical lease lines, whatever. Everything goes over IPsec. So it's like your private SDN, software-defined networking. And once you have a single overlaid network, you can start putting in load balancers anywhere. So you can actually have your load balancer terminating in, say, Amazon, Singapore region, but it's directly able to talk to your physical data center where you're running a container. I mean, that's brilliant. Today, most of us can't imagine using, say, an AWS ELB and add another instance in, say, another region. We can't do that at that level. You have to use GSLBs or something like that to get it done. But thanks to an overlay network, now it doesn't matter which physical data center or which public cloud hosting provider you are on, you can connect between machines anywhere in your overlay network. And the other major building block is the storage itself. You can manage the volumes. You can snapshot directly to S3. And the idea is that snapshots are consistent, right? I mean, if you throw away your app, you can still restore it from your snapshot because the data is kept in the snapshot. The app is, by itself, is stateless. It's the data that gives more meaning to an app than just like Tomcat. It's the data that you write or process and talk to MySQL or a DB to make things more meaningful. You can throw away the MySQL service itself, but as long as you plug it back into the right data store, you're back up and running. So does that kind of make sense? Docker compose? So here we are saying, okay, we define a load balancer using an image and we map both 8080 and that links to web and the web uses a PHP image. It links to a database and ZK. ZK, in turn, uses the zookeeper image and the DB uses MySQL. And developers should be able to do this, right? Define the environment right at their laptops. The way hiring happens in companies is like there are freshers, there are experienced people and there are people who actually do the job of day-to-day management on the production environment, right? And all of them need to understand what is the environment that they're working with. And if I were to share this with anybody in the organization, they have an idea as to how the app deployment actually works. Today, if you go into an organization, somebody will start drawing on the whiteboard, okay? Here is where the application is, here is the out. Traffic comes in, traffic goes out and suddenly somebody comes in and says, oh, we just added a new component yesterday or, you know, ten years back and everybody forgot about it. Now imagine if we can describe our environment in a YAML file. Brilliant. Everybody's on the same page and if I say to somebody, look, use Docker and you can build this environment on your laptop and you can have an end-to-end functional environment. It hopefully changes the way you do development and do end-to-end testing.  Though we wouldn't want somebody to deploy directly into production go through a change management process but the important thing is that, yes, we can now build environments in a very, very lightweight way without too many dependencies. I would say no dependencies as long as your machine can run Docker via set. It may not be the same class of machines as you would run in production but from a functionality perspective you can build the whole thing out. Now the same thing when you move it to say in production you might have additional things like you wanted to scale up you wanted to scale down you have health checks so on and so forth. I mean these are all very pertinent from very large scale deployments who have high peaks and they want to optimize on cost and stuff but don't get hassled with this most of us on day one don't care about auto scaling it's about getting the product out making it consistent and being able to do updates but yes for the most advanced people probably 1% scaling up and scaling down is a lot more important than having a sane environment yes we have the flexibility to do that so rather than do all of this on right very important thing in linking up applications is service discovery spinning services up and down is a lot of overhead I think somebody was making a point a little earlier that can I start listening to events and then spin up Isinga checks based on event because people are spinning things everywhere from an operational perspective you don't know what new machines have spun up and what old machines have been destroyed and that's a fundamental problem in the cloud world because you can click and start and click and destroy everybody has got say Amazon control panel access and they just spin up things and next thing somebody comes and says that service stopped working and we in operations realize that hey nobody told us that somebody has spun up a service like that so service discovery is something that I'm very very keen on and it's one of the fundamental aspects of rancher is that everything has to be plug and play kind of a model so are we there yet it's a different thing all together but yes it's on the roadmap to make it happen alright so I'll be happy to take questions before I go on to the second demo the one that I actually wanted to show before rancher always coming in there is a service from I mean I think not before it's after rancher always coming in Amazon ECS came in where Amazon ECS have pretty familiar things what rancher always does I mean like at a level where it does but what difference you see at a level where rancher always will be capable enough like doing it on my own instead going with the service like ECS the question was what's the big deal between rancher and say Amazon's container service rancher is multi cloud simple so you're not stuck to so if you wanted to move your workload to Google what would you do let me ask you a simple question what would you use can you use ECS not exactly so what we're proposing is a rather what the project wants to do is have a single platform for orchestrating docker it doesn't matter where you physically your resources are located it could be your laptop it could be your physical bare metal servers it could be VMware instances it could be Zen server it could be cloud stack it could be open stack it could be Google compute engines it could be to digital ocean droplets anywhere so coming to the networking part I mean is it compatible like a plugin format where you can bind it up with open which or you know the pipe work or wave or funnel or any of the networking tools which are right now there is absolutely no integration it is a dedicated overlay network it's an IPsec tunnel that goes between all the participating instances so if I'm not wrong if the agent which is running on the instance for the ransom OS to take in so is it at a privileged mode it runs and takes the namespace and the cgroups of that particular instance or the VM and then get the information from there for the other content. Okay so let's kind of do something else after this session how many of you actually plan to try to work on production right I want to first make sure that because we have a buff later I want to make sure that people understand the difference between containers and virtual machines that's a key mindset change that has to happen if you want to move to containers we'll get to what you're asking for I think it's an excellent technical discussion but from a practical perspective at least my personal challenge is to get everybody interested in containers the implementation details are something that we can talk as a buff session probably much later because we are having one yeah in fact the reason I'm asking is we use Docker in production wonderful so in fact we consolidated the answer was but somewhere it didn't work out okay sure so if now's a good time to ask questions about Docker before we get into yeah so what's the difference between Docker and Kubernetes of Google okay that's a excellent question so Kubernetes is more of cluster management framework not necessarily infrastructure management framework okay sorry Rancher with support linking Docker containers across hosts which Docker does not natively support Docker swarm even does not support linking containers across hosts it has to run on the same host does Rancher solve the problem yes we create an overlay network right because we are on an overlay network everything is one large contiguous resource of CPU memory and disk or whatever you want excuse me here yeah we have a question does Rancher support still have any support for the Docker swarm let's say like feature like these are the IP address of the servers that I need to set up a cluster with Docker swarm absolutely so if there are no questions on Docker itself or why we have Docker maybe I'll just explain a bit to you on how we got here right just to give you an idea as to where we were previously and where we are right now see once upon a time we used to deploy from tar balls we used to download the latest Tomcat W get it make make install it would go into user local and you would have a start script that is probably a stock start script copied into hc init.d and things start up but the time would come when somebody would need to upgrade and then you wouldn't know how to go about it you would again recompile it or you would have to figure out reverse engineer what somebody previously had done that's how we all started off with and then somebody came and told us hey there's something called a package manager you should be doing rpm IVH or dpkg-i but where the heck would you get the actual package from now you would go to fresh rpms or rpm find and you would download it put it in on your disk and then say dpkg-i or rpm-i then you would complain about 100 dependencies and then you would start getting libraries one at a time and satisfy the dependency or just say hey I'm not worried I'll do a force install then somebody came and said you know what we have something called a repository you can do an apt-get update or you can do a yum update and suddenly everything was packaged in a repository on a network and you could install packages it would solve your dependency issues and we are having a much more saner stable environment where there's consistency between different machines then people said hey look I need to do rollbacks I need to do roll forwards and if I use the built in the vendor provided Tomcat or Apache it doesn't have all the fixes that I need so then people started building their own packages and when they started building their own packages that would conflict with the stock once provided by the vendor so now you would say okay why don't I build it with a different path I'll call it as slash opt my company slash whatever so all the artifacts started going into slash opt and so on so that's evolution for you we're still using the same package management system we're doing things differently but the end of the day it's the same result then you said okay now I have things in a different place same package manager what will happen if I do a yum upgrade or a apt-get upgrade would it destroy whatever I have done so far now you tell the ops people don't ever upgrade any system just stick to what you have right if you want to do an upgrade call me then we'll do an upgrade that's that's where we are today and so smart ops guys decided okay let's do things differently why don't we start using jails or charooted environment so that is a completely different environment running in the same box and is network accessible so you would start charooting into a different folder and you could do your yum dpkg whatever and it wouldn't spill over onto the system area where your ops guys wanted to make sure it's patched the next step of evolution was containers containers have existed in solaris for ages in linux it came very recently freebsd had jails for a very very very long time and I personally in yahoo we were deploying on gfreebsd jails for a very very long time but linux didn't have an equivalent of that and finally linux learned that there are better ways of doing things and containers came about and that's where we are today so the container strategy is it's an app strategy you no longer worry about the OS itself you put everything together say what developer really wants to run is Tomcat or Apache with PHP he is really not interested whether it's open to 14.04 or CentOS or whatever it just so happens because he was stuck to the packaging system provided by the operating system he started insisting that he wants x, y and z but if he was given the flexibility to run whatever it is that he wanted and a way to manage it you can do development and deployments just as easy as deploying packages you would simply say here is a manifest file build a container container has the app I don't care where it is because whatever the developer wants in terms of a user land environment is already put into the container so that's where we are right now we now have containers for applications we no longer care about virtual machines actually virtual machines came before we would say here is a VM take a snapshot of it it has a running copy of our application deploy to as many places as you want then you worry about things like host names every machine has the same host name every machine has the same SSH key so in monitoring perspective you are suddenly seeing multiple events from machines all call the same thing because your cloning machines left right and sent out so yeah that's that's where we are today and hopefully Docker can deploy the application environment in a much much much same way sure the mic is not on I think yeah so question is more regarding the performance of the dockers let's suppose we are deploying a docker in a production so how people should do the density management on the host right let's suppose as of now we try to calculate the some density for the VMs like this is a 32 GB 32 core box these many VMs we should apply based on the application benchmarks so how you calculate the density for the dockers and how we have seen that okay if you have a 32 GB or 32 core boxes how many containers you will run on those boxes that's a one question other queries more about that okay do you recommend updating the same image again and again or do you feel okay you should actually always use the fresh images for the deployments okay I'll take the first one the first one was about resource management right see in a VM you are actually doing strict partitioning of CPU virtual CPU and virtual memory so it's a very different ballgame out there and in terms of performance it depends on the hypervisor itself whether using HVM or paravirtualized VMs technology has come a long way now we support something called VT which is the CPU so CPU vendors claim that there is absolutely no overhead provided you use paravirtualization drivers so my take is that the performance difference is going to be so negligible I wouldn't bother trying to do benchmarks on VMs actually I am not talking about VMs at all I am talking more about a docker so from a VM perspective itself I don't care too much about performance overheads so now when it comes to docker in general it's the same system cache that you use so whatever is buffered in the main memory gets buffered I mean that's one of the security issues around docker you are saying that you know what you are actually sharing resources it's not perfect isolation so my take is that the resource utilization ultimately depends upon the application that you run something like say solar which I actually have a client running is a beast as far as professional perspective running solar in containers is great from a developer perspective but from a resource consumption perspective it continues to be the same whether you run it on a bare metal or on container it's going to eat the amount of memory that it wants to eat so from a performance perspective resource utilization perspective I would say it's the same if you are running say solar on bare metal and it takes x amount of c2 and x amount of time it's going to take the same that's fine but still there should be some mechanism to make sure you do the capacity planning for the containers as well right because if we are deploying 100 containers on a single box that also doesn't work so there should be some can you deploy 100 apps on a single box so still I think if you are using a so does it mean that we are seeing one container one box is that what we are seeing it's just an app right I mean if your app is using x amount of CPU you need to figure out how much your other app is going to use that's how I think in your Docker world you can do the CPU bindings right yeah you could do that but it still uses cgroups at the end of the day to do limitations but my point is that's not what what we are supposed to be doing it's just supposed to be using it as a regular app but in isolation of user land that's my take away you want to run it as say a resource a cluster resource management system then that's where you run things like resource or Kubernetes to manage your resource that's how I would look at it I wouldn't use Docker itself to do things like that I would use a much more intelligent system that is capable of managing resources the final question that I have is like the Docker itself like people are building Rocket app now the core is people are building Rocket app containers so what is your take on that like why they feel like Docker is becoming a more monolithic system rather than microservices so what is your thought process about that Docker is becoming monolithic is that the question they feel like this is the one system that does the imaging part then upload the images then manage the containers as well so why do you feel like if you have any take on that why Rocket app came into the picture as compared to the see Rocket vs Docker is a political thing I would say okay I don't think it's a technology requirement at all to have I mean it's good to have competing frameworks but I think Koro has just had a different way of doing it probably like Icinga and Nagios but Icinga does much better than Nagios so that's why Icinga exists I use both I use the right tool for the right job and that's how we as operations should always be if you're stuck to one particular technology I think it's going to be you know bit rot at the end of the day you have seven more minutes so do you want to finish the demo yep let's get to the demo how it's being marketed today is it's the silver bullet so it'll be nice if you could highlight the cases where you should not use Docker okay I'll start with where I am using Docker I am using Docker to containerize applications which absolutely have no developer interest from an ops perspective right these are all legacy applications which nobody cares about and is unmaintained and for you to put things back in case the machine that's hosting it goes down or whatever nobody knows so these are the things that I have found companies interested in saying nobody loves this application but we want to make sure it's running because it still makes money nobody knows how to set it up if it goes down what can we do about it so all these legacy applications are great targets for moving into a container because you can always store it in your Docker repository or your private registry and then say okay you want copy of some php 4.x app that somebody wrote some time back here you go it'll run out of the box now where is it difficult to actually where I am finding it tough to get developers to put it in is when each developer is developing a different module in the same app okay it's they've written very big monolithic apps and everything gets compiled into one big jar or something and it's you have to go all round so there are lots of developers who have to get together to build something for you so for them they don't want to use docker because they feel they're doing very dynamic things I don't know what the perception is he's doing something I'm doing something I think it's more of a distributed development issue than a docker issue but then trying to tell people it's alright start using docker even for things like that they don't want to get into it they think it's like an overhead for them to start dockerizing at that point once we're done with the development we will dockerize it so that's one challenge that I'm seeing from developers I don't want to I don't think there are any technical challenges at this point in time there is one see one technical challenge that I've seen that developers say is that because they're used to SSH into a machine to check logs they find that convenience missing that's again a logic it's not a technical thing I've seen anything so one of the major things in systems like these that are handled well otherwise when you start accessing disk one of the things that you do in docker is do a docker mount or you expose file systems or Unix domain ports into docker and the problem is how do you ensure consistency of disk across the boundaries consistency of disk somebody else changes from the outside disk you've mounted a file system into your docker and now you're writing to that disk reading and writing for whatever purpose you cannot say your app probably should not write there there are a lot of legacy apps that actually use the file system and once you do that the whole docker paradigm falls apart and then you are left with the challenge of also handling availability of that data which is not a part of the docker now absolutely yeah so data you don't keep data in containers yes but how do you how is it what is there to handle in it I mean you don't want to containerize data at the end of the day you want data to be outside and that's where you would like to keep it the order is right to slash temp or var temp or slash data or something those are the things that we have to make sure that is outside of the container itself so we have to make those mounts available and consistently across well that's the process of containerizing an app is what I would say it's alright so it's the screen back alright so I'm going through the UI now if you go to a docker run on any of your VMs or EC2 instances and you will get the UI up and running once you have a UI you add host to it in the interest of time I already added two EC2 instances before coming in it took a bit of time probably because git pull takes a lot of time I guess I don't know it took a bit of time but it's up and we deployed I sing a web a little earlier so let's go through the process of adding a new host and so right now there's ready integration with digital ocean and EC2 you can give your AWS IAM credentials and it can provision an EC2 instance for you same thing with digital ocean you can give your credentials for digital ocean and it will provision an instance for you and by default it uses the <IP_ADDRESS> for image and you choose the size and the region and private networking and you have your host up or if you're using custom you build your say a physical instance you build a physical instance make sure there is the latest docker running on it and you install the agent using the above command as long as your management server is reachable on the network you would be able to control the resources that you want to run on your physical machine or on your virtual machine takes roughly a minute and a half for it to come if the network is good it does a docker pull and docker pull the first time takes a bit of time once the host is added it picks up whatever is the host name that you have assigned to it so I assigned a host name of R1 to 1 and the other one is left empty the first thing that comes in is the network agent which is responsible for running your overlay network 1042.69 1042.16 is a subnet that I am using and all the machines that are part of Rancher ends up being on the same subnet and because they are on the same subnet they can all talk to each other no matter physically where they are located it goes over IPsec so that is UDP 500 and UDP 4500 now once you are done with this you can always add a container so let's start with say something like Tomcat alright so my Tomcat is up the first time you do a container install it takes a long time because it has to do a git pull but every subsequent runs that you do it comes up in like a few milliseconds I am going to do the same thing in another machine I am going to call this Tomcat 2 same Docker image alright Tomcat 2 is up so you can see the IP addresses that has been assigned to them 250.29 and .79 next I go set up a load balancer make it a bit small because it is too big for me where I would like to run the HAProxy load balancer and I choose the two containers that I wanted to run and I map for example port 80 to the Tomcat port so once it is added it should not take so much time I should be able to access it on so I map the elastic IP to the hostname r2 and now I have got Tomcat a public Tomcat load balancer with two Tomcats running behind it so it is as easy as four clicks five clicks Icinga was two clicks first one to get launch the Icinga instance and the second one was to get its username and password that is it not yet we are in version 0.16 I think open source completely open source go to github check out well the license is actually the Apache ASL now to go back to the presentation I just want to quickly cover one other thing so in order to make deploying docker itself the rancher guys have come up with an OS a rancher OS which does nothing but run the latest and greatest docker and runs everything in a container so the idea is that for you to upgrade docker becomes super easy and you can manage all the services because it is containerized anyway you can use the docker apis and make changes and do whatever you want yes but no it is standardized there is nothing special there is no rocket or anything like that right I mean the whole point with core OS is sorry well we actually run hcd for our own service discovery but core OS seems to be doing its own thing I mean that is what is happening they do not want to do anything with docker so that is what is happening so we could the other thing is core OS does not run everything in a container it runs docker itself but for example dhcp inside core OS is not a container you can upgrade everything that you want as a docker you can say docker upgrade my OS so your OS upgrade itself is a docker update sorry I have to cut you we are running late so you will have to wrap it up so to put the whole thing into perspective so from an enterprise perspective these are all the things that we want to take control of today it is possible for us to do all these things say using VMware's fancy enterprise systems or cloud stack or open stack to manage the physical resources and make data center much more efficiently managed so what rancher wants to do along with its set of projects is to make application management as the next way of looking at environments so how do you get started check out from git and join the community thank you


All right, so good afternoon. I'm Greg Otto and I lead the cloud platform engineering team at Concast and I have responsibilities for our pivotal cloud foundry environment It's really an honor to be here Any of the folks roaming around outside if you want to come on in? so I'm really excited to be able to share with you an update on our journey since last year and again, you know, we have the whole team here all four of them So we're really excited to talk to folks and you know, we're going to be here For the rest of the conference. So I'm going to leave plenty of time at the end for Q&A So please make sure that you if you're not going to ask questions during the session Feel free to stop us when we're walking around the conference So for those that don't know love it when the clicker doesn't work so Comcast is a Global and NBC Universal is a global media and technology company and A lesser known fact is that we actually have the world's largest IP network. It will be really awesome If my slides were working Sure Yes, so we you know, we were here last year So this is you know part two I am definitely not in marketing because the title of my session kind of stinks But yeah last year when we were here, we were talking about our journey So I spent you know quite a bit of time talking about how do we get involved with Cloud Foundry? And how do we start what were some of the challenges and then today what I want to share is I'm going to briefly cover that and Then I'm going to focus more on kind of where we've been since last year and then What were you know some of the things that we're working on now some of the changes in scope and scale for those that may have Seen us in the keynote with the panel. We gave some facts and figures But this allows us an opportunity to provide more detail more granular detail and again an opportunity to have an interactive Discussion which you know, we really can't have on the main stage. So again Comcast We have a lot of different products and services across, you know all the different entities that we have so We have a lot of great stories to share and with respect to our adoption of Cloud Foundry So for us it started again briefly in 2013 We had some real challenges around our legacy applications We had some brittle applications that were causing impacts to both the business and Then also for our end customers. So the challenge from the business was hey, you know, we need better stability We need more resiliency around the applications That are being delivered to our customers and supporting our care organization. So that was a huge focus But we also knew that this was a transformation and I say this all the time not just of the technology But people as well. So we were very committed to investing in our development organization creating a safe environment to change Mindset and work in a new way with these newer methodologies around agile development All the modern best practices for 12-factor apps test-driven development and so forth So I'm going to share some of the things that worked well And then we're also going to share some of the things that maybe didn't go so well some lessons learned and Hopefully that will help you in your journey. So as we go through From the part one up to up till now We'll highlight some of those some of those things so in 2014 we got the Foundation laid and that wasn't just with the technology in the infrastructure It was with getting agreement from our business partners at the executive level On what were the things? What were the metrics that they wanted to measure our success by right? So we heard about the resiliency Time to market was certainly a key element there and then of course with Comcast we have challenges around scale so We one of the things that went really really well was that we started this journey together So this is hard stuff. There's no magic pixie dust so you definitely want to bring lots of friends and You know going at this in a better together type of approach is you know really the best way to go that's my best advice and Then again what we learned was you know starting small getting some of the DevOps teams You know early adopters was really key build success with them first In 2015 we started to move some of the first services over and again what worked well was We tackled not just the easy things we didn't want to just you know get some easy wins We wanted to focus on some real high-value targets, and I think Chris shared during our keynote that we picked three services to move and Those were the services that were very very active it represented about 40 percent or about a hundred million transactions On one of our critical platforms that process around 250 million transactions a day so those three services are part of Over a hundred services in a monolithic application that we decided to move so it was pretty ambitious Definitely want to pick high value targets your ambition level might not be as high Be definitely want to pick things that are meaningful that you can demonstrate that business value One of the things that we learned really quickly was don't try to boil the ocean don't take on things that Are even more complicated so for us what that meant was deferring the data persistence issues until after the initial migrations So that allowed us to realize value more quickly and be able to start to see what success and the the expected outcomes would be as We started to move some of the first services over 2016 for us was a huge breakout year You're going to see that in our operational metrics that we really stepped on the gas We moved a ton of services a ton of traffic over our most business critical and customer critical services that I'm also going to show are All running on this platform Activation systems order entry systems any of that customer kind of transactions that flow through our back office our Services that we've moved on to this platform some of the things that we learned and went well really well was around embracing and Accelerating the DevOps culture so realizing the benefits of shifting things left with test-driven development paired programming And then of course the CI CD pipelines getting those established Realizing the benefits of having an effective CI CD process One of the things that we learned Was to include the BA you teams up front so again bringing things in you know bringing teams in earlier It's really a big help definitely want to make sure that again the more folks that you include in the earlier that you do that The better that your journey will be as well So here are some of our operational metrics. I will get to the business outcomes. That's kind of the big reveal We shared some of these metrics last year And they're significantly different so we had about 490% growth last year and then you can see from the top That we started our journey on our internal private cloud There's some other folks that have actually started their journey on public cloud and are moving things internal So our journey is kind of the reverse most of our Cloud Foundry environment and the transactions that are flowing through it are still running predominantly on our private cloud But over the last couple of years Again based on feedback from the development teams. We've enabled multi cloud delivery to include cloud and multiple cloud providers So that's a trend that we expect to continue and again. It's really all about choice So cloud Foundry for us enables that choice so the developers can still focus on their code and Their applications and building awesome products that deliver new experiences for our customers And they don't have to worry about what cloud provider that they're running on So that was really powerful for us and we're starting to see some of the application teams actually Move it to a multi cloud delivery model So again, you know the application growth total hockey stick, right? insane level of growth there The number of developers the number of developers has grown significantly I think when I gave this talk last year, I probably should have remembered what the what the number one I think it was somewhere around 400 developers now We have over 1500 developers that are innovating on our platform every day and One of the really proud Pieces for me is calling out our slackers. They're actually not slackers We use slack for real-time communication and collaboration and We created a slack channel for cloud foundry We have over we have almost 800 people I don't know it may have gone over 800 since the beginning of the conference, but having first of all having over half of your customer base actively Communicating collaborating in real-time on a daily basis is pretty incredible But what's really rewarding for me and this was not a planned outcome. It just was something that happened It was a residual benefit, but it's really a Another transformational moment for us because it's changing the way that we're communicating and collaborating So we have development teams that are otherwise never talking to each other because they're working on separate products And not only are they helping each other with respect to cloud foundry, but they're helping each other with what are some of the best Practices around application design What are some of the techniques that they're using? They're sharing across these different product teams. So that's really pretty awesome And then some of the application teams like Uma and Chris that were on stage with me on Tuesday They've taken it a step further and they've created their own product channels for their Individual products that literally hundreds of developers are working aligned to this product So instead of just a scrum of scrum type of format These development teams are communicating collaborating in real-time. They're seeing issues They're able to know what's going on and be able to jump in and help and you know really streamline the delivery And again, you're going to see that in some of our business outcome metrics in a moment But it's totally transformed how the development teams are working with each other So, you know that again the transformation is you know, not just around the technology but the people and also the way in which we work and the way in which we collaborate and It's even more open than what I would have envisioned it to be I'd be remiss if I didn't point out that I don't need more than one hand to count the number of folks from my team that actually support all this insane growth All the developers that we have any applications that were onboarding So those folks are up here in the front Hopefully took separate flights But it really is an honor and a privilege Because I do push them hard it really is an honor and a privilege for me to have such talented people Working on the team and you know, I know that the trust that the business partners have with us is just off the charts So and that's a testament to the folks that are actually doing the work I just get to stand up here and take the credit for it So again to make this real These Applications are right in the business and customer critical path. They're running Real applications both supporting our frontline employees the people that are taking phone calls or the people that are in your homes installing product and Then we also have some direct customer facing applications some of the URLs that you could go to right now today They are running on our platform so things like the splash page for the Xfinity stream app and you know a bunch of the other things that are up there They're you know, absolutely in the customer critical path And again, I think that I pointed out at Abby. I had told Abby that we were around 180 million transactions a day That was maybe a couple of months ago And now we've gained I don't know where but 30 million additional transactions a day. So You know the growth as you saw is just phenomenal and it keeps growing and growing so Last year I was talking about tens of millions of transactions a day And now I'm talking about hundreds of millions of transactions today I get a little bit queasy when I think about what next year might be but it's pretty exciting and We'll probably still have four people supporting the platform All right to the big reveal So some of the business outcomes that we've been able to capture and to share first thing was a resiliency that was the biggest that was the biggest issue that the business had and I know that there's a few folks like Chris and Ooma and my team that were a little nervous because we actually committed to 40% reduction in the amount of time that our applications were being impacted by outages and those outages were impacting our customers as well So I'm not going to read them all off, but just to put it in the context An 81% reduction in the amount of time that our applications were impacted is huge But then when you add the fact That when we did have issues the teams were able to solve them twice as fast and The occurrences in terms of frequency was happening half as much So we definitely got like a huge, you know check mark on resiliency totally, you know Even my wild expectations far surpassed So then time to market Business always wants to be able to deliver new experiences to our customers faster. This is an average across the platform So the average across our platform is about a 50% time to market improvement pretty significant There are some of the applications that are off the charts transformational So taking an idea and putting it into a product as a feature or a new service that we offer that process Sometimes I've seen it take three four months and now that takes less than two weeks There's instances that we have that much of a transformational difference between idea to feature so big big transformation and Another thing I want to call out is that developer happiness, right? so everybody talks about happy employees are generally more productive and we've been able to put real metrics behind it to show that The developers that have anywhere from 50 to 75 percent Improvement and productivity because they're not working on assembling all the infrastructure pieces They're not doing transactional tickets to get work done Those happy developers are able to get things out the market twice as fast on average and sometimes a lot faster than that So that's definitely pretty pretty transformational and then again just because Chris numerous sitting here one of the examples is Our services platform that's in the middle of Basically every customer interaction. That's the 250 million transactions a day. That's running on that platform That specific example is on the board there. So there used to be 18 releases a year Which is was an enormous amount of work across many teams in a fairly well-refined process But using older methodologies That same application product and all the development teams that we've enabled to relearn Adopt newer methodologies are now doing 120 in the same year So features are getting back to the business that much faster to our end customers that much faster The experience is improving for our customers that much faster. So huge transformational change there and then Scale is pretty important It's important to note that these applications that are scaling an incredible pace That we're improving our resiliency and Then accelerating time to market. They're not sitting still Right. So those same applications that are on the platform while we're moving them to the platform have seen an average Increase almost three-fold. So scale is really really important for us and to be able to do all of these things At the same time in a fairly short window. I think it's took us about a year. I think right so You know, it's pretty it's a pretty proud moment for us And again a big update from last year that we're able to you know We're happy to share these These metrics and these business outcomes that the business challenged with and the team totally knocked it out of the park So it's pretty exciting Now I'm going to shift focus on where are we at now and what's next? I'm going to share more granular insight into how we manage the platform and some of the things that we're doing to try and Deliver even more services and some of the things that we've recently done So what this picture really depicts is that even with that insane ridiculous growth that we had The four people that we have supporting the platform. We're actually spending less time managing and running the platform and More time with the developers and understanding what their needs are we're working with them in a real-time collaboration environment every single day and It's really just it's unbelievable to even say this, but we probably spend about 80% of the time Over the last year with the development teams Understanding what their needs are and delivering new services Even with that that growth which again, it's a testament to the talent of the folks that we have on the team So what are some of the ways that we're expanding the platform? I'm happy to announce and this just timing is everything and Nithya is here and certainly was a big part of helping us give back to the Cloud Foundry community So Sergei who's sitting right here? Definitely want to talk to him We've open sourced Telegraph for both the build packs as well as for Bosch deployment And what that means in English is that when application teams spin-off applications on our Cloud Foundry environment? One of the things that they requested from us is that hey those same metrics and telemetry dashboards that you share with us It would be awesome if we had that for our applications and Because of the integration with the build pack and Bosch deployments Now within moments of birth anyone that deploys an application onto the platform Automatically gets these metrics and it all happens automatically and it's no effort on the developers part Huge transformation and again, I'm happy to say that that's all open sourced and available for you all to use and Hopefully, you know add value to your development teams There's a few other pieces that I wanted to call out. I'm not going to name every single one of them one on the slide here But we started to use Cloud Foundry kind of even beyond just the platform itself So using components of Cloud Foundry to deliver services Even outside of the platform and actually a really good example. We've heard about Kubo in the conference Which has taken the pieces of Bosch and you know UAA from Cloud Foundry authentication and authorization and Delivering value for spinning up Kubernetes clusters with a single command So that same pattern we've used to deliver some other services that we've provided first within our Cloud Foundry environment And then shortly after to the rest of our environment things like global load balancing is a service So we talk about 12-factor apps multi-site application deployments Now we enable a programmatic way for people to be able to globally, you know get that global load balance services So that they can spread their applications wide across multiple availability zones And the global load balancing services is not a ticket-based transaction And that's being expanded to not just the Cloud Foundry environment. So that's really key Then there's a few others there one. I want to talk about real quick is Jenner lemur. So Neville who's sitting up here as well. He wrote that and Folks may have heard of chaos lemur that runs around breaks the environment test the resiliency to make sure things are functional We created Jenner lemur which runs around and does clean up because Sorry, but the developers can be sometimes a little messy and we're lazy We don't want to constantly sweep up after them So this takes the application instances that haven't really been doing anything and goes and cleans them up for us Because it's so easy to deploy new ones anyway, and it allows us to spend even more time Calling up to our developer friends and helping them deliver these awesome experiences to our customers in the business And that's something that is going to be in process To be contributed back to the open source community So with you know the help from our friends like Nithya. We can get that pushed through and make that available so we're really happy about being contributing back to the community and We hope to get the gender to lemur open sourced Quickly so that you all can benefit from a little bit of clean up So more insight more granularity in terms of how do we manage the platform? so this slide depicts our overall architecture and strategy for metrics and telemetry and The strategy is really best depicted with the elements on top right the status and notification eventing and Then the trend analysis that we use most of this is all open source One of the things that we learned really quickly is that there's no one tool to rule them all So we took very much a microservices architecture approach to monitoring and telemetry and We picked the pieces that made the most sense for us that we can manage and scale in a loosely coupled way and Not going to go through everything But you know our infrastructure and platform type metrics influx DB is is the heart of the of the system of the environment we use capacitor to provide some business logic as we Publish the information that we consume and again that all the development teams can also consume as well and You know one of the things that we really want to point out that builds that truss that partnership as you go through the journey And I'm going to show you what our dashboards actually look like all of the information that we use a hundred percent of the Metrics and telemetry that we use to manage the platform is completely available to all the development teams So there's no question around how are things running my apps not performing right? What's the platform doing and getting that visibility that you might not otherwise get we completely make that open and Transparent the feedback we've gotten from the development teams is really really positive. They really appreciate the transparency And when we do have issues on the very rare occasion We hold forum talk about what went wrong We can show the metrics and then we can also talk about what we're going to do to prevent future occurrences So they really appreciate that openness and transparency So we have three examples. We're going to show I'm gonna go through it fairly quickly so we can get to some of the Q&A So these is an example of our platform metrics So these are on TVs and on people screens and so forth and of course any of the development teams that like to keep an eye on us This is an example of our metrics and telemetry that we use to show the health and status of what's going on in our platform environment Monitoring telemetry for applications, right? So we talked about the open-source tools that are available for you to download today and This is what we provide automatically for application teams And it's just a sample and then of course application teams can make changes and Add elements that are important to them But again this happens within moments of birth of the application without any user intervention at all And then this provides the infrastructure kind of underlying. Do we have enough scale? To support the environments to support the demand We don't want the development teams to worry about whether or not we have enough capacity to support What it is that you're trying to innovate so we keep an eye on these type of metrics to make sure that Our platform has enough horsepower to perform and again, this is for internal and external as well But yeah, we always want to make sure that we have enough scale to support the environment A few of the big lessons learned we've stepped on a lot of landmines for sure But this is representative of kind of the best advice that we can give and we talked to a lot of companies And this is usually encapsulates what we talk about If you're an infrastructure team that's working to deliver this I can't overstate the shift and focus from Focusing on the infrastructure to delivering platforms and services and getting closer to the development teams hugely important You can manage the environment using things like puppet and ansible and so forth but the faster that you learn things like Bosch and Concourse and have CI CD pipelines for how you upgrade and manage your own Underlying cloud Foundry environment the sooner that you're going to late make life a lot easier for yourself. So definitely that's a big Piece of advice because you know some things like Bosch are not the easiest thing to pick up There's a little bit of a learning curve but Sooner you do it the easier your life will become The emphasis on partnership and community Again just huge You saw the development teams up on stage, you know, we're really one team We don't really look at it in terms of Organizational boundaries So we're all in it together. You definitely want to establish that sense of community and partnership with the development teams and then simply stated If you make it easy you do not have to worry about adoption So that insane level of growth zero outbound marketing from us It's really just all the development teams talking telling a friend and so forth and being able to deliver value with much less effort and That's what enabled our environment to scale. There was no mandate the Alps shelves must not sit It all just happened organically and again because we provided a safe environment for the development teams To learn and grow and adopt these new methodologies. So that's a big piece of advice as well So just to wrap things up because I do Think that it's invaluable to talk to the folks that are doing all the work Get some questions going Start together bring friends No one wants to go in alone. So you really want to start this together. There's comfort in numbers We focused a lot on people the technology change can be complex but Once you get everybody committed and on board and Comfortable that you're in it together and channel people's passions Then it's a lot easier to make all the technology changes that you need to make Start small. Don't try to solve world hunger. Just feed the family first Be strategic about the applications that you that you start to migrate first so that you can capture the winds and communicate the business value as you Establish an open forum and a feedback loop make sure that you're actually listening to customers So all of the things that we deliver is Based directly on the customer feedback. So they feel that right so you want to make sure that you definitely listen to your customers Having a solid foundation is fairly intuitive Automate everything so we're able to support this environment because of just completely ruthless levels of automation Automation is in our DNA There's not anything that we do that we don't look for ways to automate that so we don't have to do it again manually and That enables us to really spend all of the amount of time that we do Collaborating with the product teams and the development organizations and then lastly but most importantly Join the community. You're all here as part of the cloud Foundry Foundation community and Give back give back and sharing the lessons that you've learned We learn a ton from you all as well So if you've created capabilities in your environment, please contribute those back to the community We're seeing a lot of growth in Contributions for cloud Foundry, which is awesome to see love to see that you know happen even faster and Even just sharing the stories so we're we're here to share with you But we're also here to listen and learn from you all as well So, you know, it really do mean it when I say that, you know, we're anxious to get feedback. We're anxious to have conversations We're going to leave time for Q&A now and But you know, it's not a one and done So we're you know, we're here for the rest of the conference and really excited to talk to you all Thank you very much That there's a question Yeah, that's a that's a great question. Certainly appreciate the the challenge So the the question was was around how Awesome has our platform and the work that we've done to improve the customer experience and you know I think that I shared some of the metrics with you, right? So we know that when our products are working people love our products, okay? We don't want to be Netflix. We've we've communicated that you know We have a premium product and we you know, our customers expect a premium experience and Folks interact with our product more so than most any other product I can think of right? So just the always-on internet and Wi-Fi and all the other products and services that we have so we know that When we can create a an experience That is a self healing when problems can correct themselves because of our ability to scale up and scale down applications Responding to business demands in real time. That's a way that we've enabled our applications to support our customers So that 81% reduction the amount of time that we're impacting folks ability to call and get service or To activate new products that definitely helps improve the customer experience We've shown pretty significant gains and again. We have a lot of work to do for sure But in our net promoter scores in terms of customer satisfaction We've seen really big strides in improvement and I can't share necessarily a direct correlation between you know net promoter score the customer satisfaction and some of these metrics but we do know that when our products and services are more available then Customers don't have to call in and the products just work So creating that experience where things are more self-healing and more self-service I don't know Chris if you want to talk about some of the digital first Type initiatives you want to come on up? Yeah, so for that one in particular actually there's a metric that we've been tracking for a while that we referred to as customer dissatisfaction and a couple years ago. We started correlating that to a line graph also showing the Availability or what was at the time the lack there of of our systems and there was I forget what the correlation number was But it was so incredibly closely correlated that the business was really pressing on us around That's really the impetus to improve So not surprisingly after implementation of the platform and all the journey that Greg just described Our customer dissatisfaction line has gone gone down in correlation to the uptime of our our environments The next leg of the journey as Greg just alluded to is certainly around digital first That's where we want to provide Mechanism for our customers to interact with us first through their smartphone or xfinity.com or whatever the case may be Without having to pick up the phone and call a person or more likely pick up the phone and go through the IVR And wait on hold for a while and then get the person So not unlike some of the web-scale companies like Netflix and Amazon We really want that to be the first way that customers can engage with us That's a way to start the Q&A that's cool Hey, we are open right so No, it's a great question. Thank you Okay Based on traffic Yeah So so let me so let me just repeat the question and tell me if I got it right So the question was we talked about three we chose three services were our strategic selection process and was that based on traffic and The answer is is that you know, certainly there was a lot of traffic these were the services that were called most frequently and There was also another dimension which was in terms of the resiliency. They were also the critical pieces of the services platform that Were most likely to cause other, you know, kind of near neighbor type of issues as well So the the three services that we we chose was not just based on traffic But also the potential impact for improving our customer experience So and again, that was a testament to Chris and the development team in terms of Choosing and being really ambitious with respect to the services we chose That's a good question anything else great. That is an awesome question and I'm just going to be kind of shameless and I'm going to give an opportunity to some of the folks to come on up so Yes, alright, so the question was and I can flip back to it so the question was around our monitoring and telemetry strategy, there was a lot of pieces up there and We do get a lot of questions about this So the question was, you know, can we provide more granular detail into how we selected the pieces and what the benefits are and I Wanted to you know bring up the folks that actually were responsible for sure. Thanks so So my name is Tim and this is Sergei so sir Yeah, they should work like thing one two three four thing one two three four Yeah, so I don't know he's number one so the so the strategy kind of built organically and One of the things that we really wanted to depict with this picture is that what Greg alluded to was that We chose the best tool for the bet for the job and decomposed our monitoring strategy into things in flux DB is a really great Time series metrics database. It's very fast and reliable capacitor does a really good job at Massaging those numbers and being able to do some business logic on it Grafana, you know that is it is a great obviously is it is a great Trending tool to be able to visualize some of these metrics and You know, we we just we just over time came up with this strategy and I don't know if there's really any Overall thing that we can say to say this is the best it's just that you know Look at what works for you and just employ that for that specific job. Don't look for something that's gonna Do everything for you at once actually, you know, I forgot right because I'm getting old and my memory is failing But so Jim you want to talk about the Bosch config box? It's kind of like a little box down there and it's not open source yet But again in the spirit of giving back. Yeah, I mean that was one of the challenges We had was that you know the platform is always changing, right? So so when you're deploying new VMs when you're scaling you're always gonna have new things you need to monitor so we have a method by which we can take a Bosch config and Translate that into a Nagios config and we use Nagios because it's simple It's extensible and and everything else and it does a pretty good job So we were able to to build a script that will generate a Nagios config based off of Bosch Bosch output or the Bosch API? Yeah, so that means that we don't have to worry about forgetting oops You know this component chains or the IP change or whatever the case might be It allows us to again to just spend more time with our development friends than you know worrying about managing the platform But again, that's something that we're going to work on contributing back and again, you know, if he is glad you're here I Think that you know folks might benefit from from that Yeah It is it is both and so this this topic actually my Need a separate session. It's a pretty big but I'll address this questions so from Selection and strategy perspective what strategy do we have to select this these components? well, one of the strategy is We don't want to commit for 20 years from now We want to build system not based on a you know some one single enterprise solution that is not replaceable But instead pick up pieces that can be replaced when there will be a new product that will fit better so work on work on Standards industry standards or or you know the standards Don't try to do too much proprietary things Select components that are play well with other vendors other products and Make sure that you can you can easily pick up Remove one component and replace it with something new technologies that will be available next year So that is that is on a on a strategy question about monitoring platform or and Applications we do both and Greg mentioned Telegraph build pack. So this is one of the things that we started to use very actively to monitor the platform, but also a Lot once once we start to advertise and release it to show to our consumers in Comcast Many teams started to use it right away. So what is this build package is doing you just create a telegraph dot-on configuration and You do CF push this configuration. That's it. You don't need anything else and it will Automatically spin out a telegraph instance based on this configuration. You can bind in flux database To this this application and it will start to stream your your metrics to that application. So our consumers application teams devops teams are Very active and love this feature. So they're using this for application monitoring as well And you know, we're here all week as well. I want to make sure that we answer any other questions I think we might be at time kind of looking at Bridget to see but Yeah, it surges obviously, you know very knowledgeable and proud of the contributions we made certainly you can look at the source code Do we have any other we have any other questions that we can we can answer quickly or we? Okay, so I'm sorry we are at time But what we'll do is we'll come down and we'll come and talk to you and talk outside so that we can get The room give the room back All right, thanks a lot